Usability Testing: Are You Asking The Right Questions?

December 3, 2015In Web Usability4 Minutes

Usability testing is one of the most effective tools to aid in uncovering problem areas or stuck spots in a website’s design and structure. While any measure of testing is helpful, different types of testing and how questions are asked can greatly affect user response and, ultimately, the outcome of solutions that will be put in place.

The first thing to determine is whether to take a scripted or un-scripted approach to the test. Scripting is recommended for usability testing specific use cases and offers many benefits, including delivering the most controlled environment. Reporting from scripted testing delivers quantitative and qualitative data, depending on how the script is crafted.

Setting the Scene

Foremost, its important to indicate what areas of focus will be tested – this could be anything from the name of a website and how it will register with users to how the product/website functions. Here is where specifics are key: how much will be covered in the test, ensuring that there are no surprises moving forward.

Determine whether the test will be moderated or un-moderated. Moderated tests are ideal for complex sites such as eCommerce that may be content or pathway-heavy – however, moderators will be required and should be chosen carefully. An unmoderated test can lend a more ‘realistic’ approach to where and how the user will truly behave within site.

Developing A Structure

Define your usability testing goals by determining concerns, questions, needs, and problems that will be driving the test. The questions asked in preparation for the test are just as important as the actual questions asked when usability testing. These can range from the widespread, general topic or drilled-down, detailed and specific concerns. Consider internal questions such as:

  • Can users access all of the relevant elements and tabs on our home page?
  • Will our users find a specific tab or button on that same page or will this button have a better effect on the other side of the page?
  • Which call-to-action buttons are users clicking on?
  • How can we optimize conversion when products are already in the shopping cart (i.e. abandoned carts)?

As queries for the test are developed, separate complicated or multi-step questions to give the user ‘room to breathe. If multiple complex problems are asked one after another, users may become fatigued and skip questions or leave answers that can negatively affect the test outcome.

General questions are not recommended, as vague answers will be returned. Ask a series of very specific questions about one subject which will garner a more accurate depiction of the user’s feelings. Using either a scaled answer (Would use again, Would never use again), (Very Satisfied, Not Satisfied).


Testing should be done at least once a month, so questions can be a work in progress. Define the main goals or problems to achieve or fix and start there. By conducting ongoing testing, a site can gather fresh information and continue to optimize regularly. This will create a healthy and incremental approach resulting in the probability of increased user experience and heightened conversion.

Person on computer browsing website with usability

Transforming Usability Throughout the Health Care Experience

There's perhaps no industry where usability and overall user experience is more critical than that of health care. Personalized engagement is of the utmost importance to an environment where the relationship with between patient and provider can at…

Desktop on table near window displaying home screen content

Reading Between the Lines: The Content and Usability Connection

Very often, when developing a website or app, the first step is presenting the client with wireframes and comps that feature placeholder text which can, and usually delivers a design that is well received, but that may not be ideal in its user…

Think Creative, Work Effective. User-centered design mug.

Serving the Brand with User-Centered Design

Delivering a web design that both effectively resonates the brand it represents and puts the user first can be a complex task - especially when multiple markets or user segments are at play. User-centered design, sometimes referred to as…