Usability Testing: Are You Asking The Right Questions?

December 3, 2015In Usability4 Minutes

Usability testing is one of the most effective tools to aid in uncovering problem areas or stuck spots in a website’s design and structure. While any measure of testing is helpful, different types of testing and how questions are asked can greatly affect user response and, ultimately, the outcome of solutions that will be put in place.

The first thing to determine is whether to take a scripted or un-scripted approach to the test. Scripting is recommended for usability testing specific use cases and offers many benefits, including delivering the most controlled environment. Reporting from scripted testing delivers quantitative and qualitative data, depending on how the script is crafted.

Setting the Scene

Foremost, its important to indicate what areas of focus will be tested – this could be anything from the name of a website and how it will register with users to how the product/website functions. Here is where specifics are key: how much will be covered in the test, ensuring that there are no surprises moving forward.

Determine whether the test will be moderated or un-moderated. Moderated tests are ideal for complex sites such as eCommerce that may be content or pathway-heavy – however, moderators will be required and should be chosen carefully. An unmoderated test can lend a more ‘realistic’ approach to where and how the user will truly behave within site.

Developing A Structure

Define your usability testing goals by determining concerns, questions, needs, and problems that will be driving the test. The questions asked in preparation for the test are just as important as the actual questions asked when usability testing. These can range from the widespread, general topic or drilled-down, detailed and specific concerns. Consider internal questions such as:

  • Can users access all of the relevant elements and tabs on our home page?
  • Will our users find a specific tab or button on that same page or will this button have a better effect on the other side of the page?
  • Which call-to-action buttons are users clicking on?
  • How can we optimize conversion when products are already in the shopping cart (i.e. abandoned carts)?

As queries for the test are developed, separate complicated or multi-step questions to give the user ‘room to breathe. If multiple complex problems are asked one after another, users may become fatigued and skip questions or leave answers that can negatively affect the test outcome.

General questions are not recommended, as vague answers will be returned. Ask a series of very specific questions about one subject which will garner a more accurate depiction of the user’s feelings. Using either a scaled answer (Would use again, Would never use again), (Very Satisfied, Not Satisfied).


Testing should be done at least once a month, so questions can be a work in progress. Define the main goals or problems to achieve or fix and start there. By conducting ongoing testing, a site can gather fresh information and continue to optimize regularly. This will create a healthy and incremental approach resulting in the probability of increased user experience and heightened conversion.

A person logging into their laptop.

The Art and Science of UI Design: A Guide to Effective Interface Creation

User Interface (UI) design is a multifaceted discipline that focuses on creating visually appealing, easy-to-use, and efficient interfaces for digital products. With technology's rapid evolution and user expectations constantly shifting, UI…

A UI designer creates a responsive wireframe with pen and paper.

The Future of User Experience

A Comprehensive Guide to Enhancing User Engagement through Design Language and Interactive Design The landscape of digital design is rapidly evolving, with new technologies and methodologies emerging every day with significant considerations…

The Use of Color in User Interface Design

Color is a crucial element of user interface design, not only shaping the mood and emotional resonance but also significantly affecting usability. An interface that employs well-thought-out color schemes can enhance user experience by improving…