The different types of usability testing methods for your projects

3 types of usability testing

Before you pick a user research method, you must make several decisions about the type of testing you need based on your resources, target audience, and research objectives (aka: the questions you want to get an answer to).

The three overall usability testing types include:

• Moderated vs. unmoderated

• Remote vs. in person

• Explorative vs. comparative

1. Moderated vs. unmoderated usability testing

A moderated testing session is administered in person or remotely by a trained researcher who introduces the test to participants, answers their queries, and asks follow-up questions. Conversely, an unmoderated test is done without direct supervision; participants might be in a lab, but it's more likely they are in their own homes and/or using their own devices to browse the website that is being tested.

Moderated testing usually produces in-depth results thanks to the direct interaction between researchers and test participants, but can be expensive to organize and run (e.g., securing a lab, hiring a trained researcher, and/or providing compensation for the participants). The cost of unmoderated testing is lower, though participant answers can remain superficial and follow-up questions are impossible.

As a general rule of thumb, use moderated testing to investigate the reasoning behind user behavior, and unmoderated testing to test a very specific question or observe and measure behavior patterns.

2. Remote vs. in-person usability testing

Remote usability tests are done over the internet or by phone; in-person testing, as the name suggests, requires the test to be completed in the physical presence of a UX researcher/moderator.

Compared to remote tests, in-person tests provide extra data points, since researchers can observe and analyze body language and facial expressions. However, in-person testing is usually expensive and time-consuming: you have to find a suitable space, block out a specific date, and recruit (and pay) participants.

Remote testing doesn’t go as deep into a participant’s reasoning, but it allows you to test large numbers of people in different geographical areas using fewer resources.

3. Explorative vs. assessment vs. comparative testing

These three testing methods generate different types of information:

• Explorative tests are open-ended. Participants are asked to brainstorm, give opinions, and express emotional impressions about ideas and concepts. The information is typically collected in the early stages of product development and helps researchers pinpoint gaps in the market, identify potential new features, and workshop new ideas.

• Assessment research is used to test a user's satisfaction with a product and how well they are able to use it. It's used to evaluate the product's general functionality.

• Comparative research methods involve asking users to choose which of two solutions they prefer, and they are used to compare a website with its primary competitors.

User testing methods

Each usability testing method gives answers to your research questions. The method you choose will depend on both your resources and your objectives.

1. Moderated + in-person usability testing

Tests that are moderated and conducted in-person offer the most control. They are resource-heavy but excellent for collecting in-depth information.

# Lab usability testing

This type of usability research takes place inside a specially built usability testing lab. Test subjects complete tasks on computers/mobile devices while a trained moderator observes and asks questions. Typically, stakeholders also watch the proceedings and take notes behind a one-way mirror in the testing area.

A major benefit of lab usability testing is the control it provides: all sessions are run under the same standardized conditions, which makes it especially useful for comparison tests. However, these tests are expensive and usually based on a small population size (8-10 participants per research round) in a controlled environment, which is not necessarily reflective of your actual customer base and/or real-life use conditions.

# Guerrilla testing

In guerrilla testing, test subjects are chosen at random from a public place, usually a coffee shop, mall, or airport. They are asked to perform a quick usability test, often in exchange for a gift card or other incentive.

Guerrilla testing is used to test a wide cross-section of people who may have no history with a product. It's a quick way to collect large amounts of qualitative data that validate certain design elements or functionality—but it's not a good method for extensive testing or follow-ups, as people are usually reluctant or unable to give up more than 5-10 minutes of their time.

2. Moderated + remote

Moderated and remote usability tests are performed via a computer or phone and require a trained moderator. They’re good for picking from a wide range of testers while still taking advantage of a moderator's skills and ability to dive deep.

# Phone interviews

In a phone usability test, a moderator verbally instructs participants to complete tasks on their computer and collects feedback while the user's electronic behavior is recorded remotely.

Phone interviews are an economical way to test users in a wide geographical area. Because they are less expensive than in-person interviews, they help collect more data in a shorter period.

# Card sorting

Card sorting involves placing concepts on virtual note cards and allowing participants to manipulate the cards into groups and categories. After they sort the cards, they explain their logic in a moderator-run debriefing session.

Card sorting is a great method for both new and existing websites to get feedback about layout and navigational structure. Its results show designers and product managers how people and potential customers naturally organize information, which can help make a site more intuitive to navigate.

3. Unmoderated + remote

Relying mostly on computer programs, these passive testing methods provide insight into how users interact with a website in their ‘natural environment.’

# Session recordings

Session recordings use software to record the actions that real (but anonymized) people take on a website such as mouse clicks, movement, and scrolling. Session recordings are a fantastic way to spot major problems with a site's intended functionality, watch how people interact with its page elements such as menus and Calls-to-Action (CTAs), and see places where they stumble, u-turn (go back to a previous page quickly after landing on a new one), or completely leave.

# Online testing tools and platforms

There are a variety of online testing tools that allow you to remotely observe user behavior on your website; some let you pay participants to take short tests, while others monitor the behavior of real users as they interact with your site. You might receive audio recordings of users talking while they navigate your site or videos of users completing a set of tasks. Here are some of the most common tools:

5-second test

In this test, website owners upload a screenshot of their webpage with a single question like “What is the main element of the page that stuck with you?” or “Who do you think the intended audience is?” Test subjects have five seconds to look at the page before they answer the question.

This is an easy way to collect a large amount of qualitative data about people’s first impressions and reactions to your site.

Unmoderated card sorting

Card sorting, described above, can also be conducted in an unmoderated and remote manner if you skip the debrief session afterward. This is faster and less expensive than moderated sorting (though you miss out on the opportunity for follow-up questions), and users can complete the test at their convenience.

First-click

The goal of first-click testing is to evaluate whether users can easily identify where they need to navigate to complete a given task. The participant is asked a question like “Where would you click to buy this product?” and the software records where they direct their mouse.

First-click testing is useful for collecting data on user expectations and determining the prime location for menus and buttons. By measuring how long it takes users to make a decision, you learn how intuitive your site design and linking structure are.

4. Unmoderated + in-person

Unmoderated in-person tests are conducted in a controlled, physical setting but don't require a person to administer the test. This gives you many of the benefits of testing in a controlled atmosphere and reduces the possibility that a moderator could lead or influence participants with their questions.

# Observation

In this sort of test, the researchers watch but don't participate, acting as a sort of ‘fly on the wall’ as participants run through a set of instructions in a lab. They may interject if a participant gets stuck, but otherwise, they remain quiet and concentrate on taking notes.

Observation testing allows researchers to see the body language and facial expressions of participants without interference from a moderator.

# Eye-tracking

During eye-tracking tests, researchers observe and study users' eye movements using a special pupil-tracking device mounted on a computer. By analyzing where users direct their attention when asked to complete a task, the machine can create heatmaps or movement pathway diagrams.

Eye-tracking studies can be used to glean information about how users interact visually with a page; they also help test layout and design elements and see what may be distracting or taking someone's focus away from the main page elements. The downside? Cost: an eye-tracking study requires you to rent a lab with special equipment and dedicated software (plus the trained technician who can help you calibrate the device).

Methods that are not usability testing

Usability testing is all about having individuals test and experience a website’s functionality. The techniques listed below are occasionally labeled as usability testing—and although they technically are not, they can (and should) be used in conjunction with usability testing to generate more comprehensive results:

• A/B testing: unlike usability testing, which investigates user behavior, A/B testing is about experimenting with multiple versions of a webpage to see which is most effective. It's an important tool for increasing conversions.

• Acceptance testing: this is often the last phase of the software-testing process, where users follow a specific set of steps to ensure the software works correctly. This is a technical test of quality assurance, not a way to evaluate if the product is user-friendly and efficient; still, acceptance testing is an important step in creating a well-vetted product.

• Focus groups: when conducting a focus group, researchers gather a small number of people together to discuss a specific topic. It's a great method for discovering participants' opinions about a product or service (but it can also introduce bias when some participants are more vocal or persuasive than others).

• Surveys: a gauge of user experience, surveys can be used in conjunction with usability testing as a follow-up or a method of gathering user feedback.

   

• Heatmaps: heatmaps and scroll maps produce a visual representation of how users move around a page by showing its hottest (most popular) and coolest (least popular) parts. They are technically not usability testing because they report on user actions in aggregate, but they are a good way to observe and objectively measure behavior on your website.

Start usability testing with Hotjar

With so many different usability testing methods, we won’t blame you if you don’t immediately know what's right for you. So here’s a recommendation: start small and take your first step.

One of the easiest ways to get started with user testing is through session recordings: this remote and unmoderated technique helps you identify usability issues with your site by just watching real people interact with your site pages and elements. It requires little setup and is a simple way to start improving the functionality and effectiveness of your website.