The best usability testing questions

What kind of questions do usability tests answer?

Usability tests are run for many different reasons:

• To validate a prototype

• To find issues with complex flows

• To gather unbiased user opinions

• To get the insights that help create a better overall user experience

Each situation requires slightly different feedback from your user testers—in fact, the larger questions you want to answer and the specific questions you end up asking can be very different.

What brings people to your website?

Website usability testing can help you pinpoint the drivers that bring people to your site. In particular, moderated interviews can help you determine a current or potential visitor's motivations and their goals for visiting your site.

Key questions to ask via testing:

• What’s the main reason people visit your website?

• What are they trying to achieve?

• What other websites might they visit before or instead of yours?

What stops people from completing a task?

Usability testing can identify barriers that keep users from completing a task and, subsequently, drive them away from your website:

• What, if anything, is stopping users from [action]?

• What can they not find on the page?

• Do they find the wording or navigation confusing?

What convinces your users to take action?

User testing can also pinpoint the hooks that persuade users to take certain actions, such as signing up for a newsletter or completing a purchase:

• What’s convincing them to [take action]?

• What did they like most about their experiences on your site?

How valuable is your product?

Another common reason for usability testing is to measure user reactions and attitudes toward products, websites, and concepts.

• Does the user understand the point of your site?

• Is your site easy to navigate?

• Would they use your website again?

20+ questions to ask in usability tests—and when to ask them

Usability testing has at least four different stages—screening, pre-test, test, and post-test (we cover them all below)—all of which provide the opportunity to collect important data from your participants. In each stage, your different goals will dictate the questions you ask.

Before diving in, a quick reminder. Most questions can be asked using two different formats, depending on the type of data you want to collect:

• To collect quantitative data, you can use scales (e.g., from 0 to 10) and multiple-choice questions (e.g., ‘pick between a, b, or c’)

• To collect qualitative data, you should ask open-ended questions (e.g., ‘what do you think of this page?’).

Pro tip: when collecting qualitative data, it's good practice to follow a yes-or-no answer with additional, open-ended questions, sometimes called probes. For example:

Question: Have you ever shopped online?

Follow-up probes: if no → Why not? If yes → How often do you shop online? What sorts of products do you usually buy online?

🔥 Practical example: let's imagine you need to test your e-commerce website, and run through the different testing stages and questions you would use during each. 

Phase 1: screening

Before testing begins, you need to establish a set of criteria and decide what types of participants you want to include in your test—for example, you can select testers based on your user personas or target audience, or choose to focus on specific niches such as people who do their online shopping on a mobile device.  

During the screening period, depending on which usability testing method you choose, you should have the opportunity to ask basic demographic and experience questions. In addition to using this data to evaluate potential participants, hang on to it and use it later in your analysis of the final results.

Examples of screening questions:

• How old are you?

• What is the highest level of education you've completed?

• What is your total household income?

• What is your profession?

• When was the last time you purchased an item online?

• Have you ever used [your website]?

Phase 2: pre-test

Once you've selected your test subjects, but before the actual usability testing begins, you have another opportunity to interview them, either via a questionnaire or through a moderated interview.

During this stage, your goal is to learn about the knowledge and experience that the subjects are entering testing with: their backgrounds will inform their actions and opinions as they interact with your product.

The questions during this phase go beyond just collecting demographic data; they can be used to gather psychographic data about the participants' experiences, behaviors, and attitudes toward the product or subject being tested.

Expertise

These questions will evaluate the user's experience level with the actions they will be asked to perform:

• How often do you shop online?

   Their answer will reveal their buying habits and their familiarity with online shopping.

• How confident are you with browsing, shopping, or other online shopping-related tasks?

   The answer sheds light on their confidence level with online shopping.

• Which device(s) do you usually use for online shopping?

   The answer will help you determine which devices (e.g., mobile or desktop) users are more comfortable using. If possible, use their preferred device for testing to recreate a ‘natural’ environment.  

Brand knowledge

These questions will check the user's background and existing knowledge of your brand and products (or similar ones). Existing knowledge may sway their opinions or ability to use your product:

• Have you used this site before?

   This tells you about their familiarity with the brand and their knowledge of the site.

• Have you used a similar site before?

   This tells you about their familiarity with competitors.

• What would make you decide to buy X from brand Y?

   This gives you some insight into their motivations and drivers.

Phase 3: test

During the actual testing stage, your goal is to collect data that explains why users make certain choices while navigating the assigned tasks.

The test may be highly interactive, with the user talking through their thought process as they complete each step. Conversely, it could be silent, with users working independently and then answering questions after each task or section. Either way, questions like the ones below should help illuminate what the user is doing, and why:

• I noticed you did ____. Can you tell me why?

   Follow up on any interesting behavior you observe during the test to get a better idea of the thought process behind the user's actions.

• Did you notice whether there was any other way to ___?

   You are trying to determine why the user did one thing instead of another.

• Which of these two approaches/options do you find best? Why?

   This is useful if you're trying to determine the more appealing of multiple options.

• Can you tell me what you think of ___?

   By asking about specific aspects of the page (icons, menus, text), you will gather opinions on the design and functionality of the site and rework the confusing components.

• How did you find the experience of using the website to complete this task?

   Ask this after every assigned task to learn how the user's experience and opinions evolve as they interact with your content:

• What did you think of the layout of the content?

• What did you think of the checkout experience?

• What did you think of the on-page explanations?

Pro tip: communication techniques to employ during moderated usability testing

Usability testing moderators need to be careful with how they interact with participants during the testing phase so that their words don't influence the testing in any meaningful way. If you are leading the session yourself, make sure you aren't asking questions in a leading way or answering questions in a way that could give the user hints or extra information about the tasks they need to complete.

This can be challenging when a subject has questions during the test. There are some common communication techniques that moderators often use to respond to user questions during testing. These include:

• the ‘echo’, where the moderator repeats the last phrase the user said, as a question. For example, if the participant says “this cart doesn’t work,” the moderator simply repeats it as “this cart doesn’t work..?”

• the ‘boomerang’, where the moderator uses generic, neutral questions like, “What do you think?” in response to any of the participant’s queries

• the ‘Columbo’, where the moderator channels their inner Detective Columbo and trails off a sentence or question, so the participant can step in and fill the silence by elaborating further.

Phase 4: post-test

After a usability test, you have one final chance to ask the user questions that haven't been covered elsewhere. This is the time to gather feedback about their impressions and opinions of your website and get a feel for the overall user experience.

Common questions during this phase include:

• What was your overall impression of [x]?

   Solicit feedback on the user's general opinion of your product. This is a broad question, so be prepared to dig deeper with the following question(s).

• What was the best/worst thing about [x]?

   Get more specific feedback about the features that make your product stand out.

• How would you change [x]?

   This open-ended question is a good way to gather constructive feedback and ideas for future iterations of your project.

• How would you compare [x] to [competitor]?

   This is an opportunity to see how different details of your website stack up against your competitors in the eyes of users.