Usability studies: How many participants are enough?
Usability Testing

Published on March 23, 2022

Usability studies: How many participants are enough?

Selecting the optimal number of testers for your usability tests can help you save costs while getting maximum insights in a short time. So how do you decide on the number of usability testing participants? All the answers can be found in this article.

Usability tests are necessary if we are to create valuable and successful products. But conducting such tests can become a headache if they are not done efficiently by recruiting an adequate number of usability testing participants. 

The 5-user rule

According to the reputable Nielsen Norman Group, ‘testing with 5 people lets you find almost as many usability problems as you’d find using many more test participants.’

The logic behind their ‘5-user’ suggestion is that as you test more and more people, you uncover fewer new insights at a higher cost. After testing up to 5 people, the same usability issues would continue to be mentioned by additional participants with very little significant change. 

So it’s economical and optimal to test just enough participants who can give you sufficient insights at a low cost. Thus, the 5-user rule.

When to use more or less than 5 participants

There are instances, of course, where the 5-user rule is not applicable: In some situations, the type of product tested requires a larger number of participants to obtain significant test results. For example, e-commerce websites can be much more complex than software products, so testing as few as five participants may only uncover 35% of their usability problems

You may also need more or less than 5 participants for your usability study if you:

Have different target audience segments 

When a product has different user groups who behave differently, you’ll need to test participants that represent each group. 

For instance, a marketplace app would need participants for both the buyer and seller profiles. In this case, you don’t need to have five participants for each of the different user groups since there would be overlapping observations, and you could use as few as 3 or 4 participants for each group.

Need quantitative data 

In conducting quantitative usability studies, your results must be statistically significant to obtain accurate insights. For this, you will need at least 20 to 40 test participants

Since the focus is on measurable metrics and not qualitative findings, you will need many participants to get enough data which can accurately predict the behaviors of your overall target users. 

Use an Agile UX process

In an Agile UX approach, you run multiple usability tests as the product develops since it is an iterative process. In this development style, you conduct tests, use the insights obtained to make changes, then test the new version – and on and on it goes.

So rather than using five or more participants for each test, you can run multiple tests with as little as 3 participants each since there would be overlapping insights and discoveries. Using fewer participants also helps to save costs on your study.

Pros of using 5±2 participants in your usability study 

There have been different optimal numbers of participants suggested by researchers over time. Having a minimum of 3 participants ensures that you capture diversity in your user group, while having as many as 7 participants ensures that you uncover almost all the usability problems in the product. 

So the ideal number of participants lies in a baseline of 3 to 7 participants for the following reasons:

1. It maximizes the law of diminishing marginal returns

In the usability testing context, the law of diminishing marginal returns states that as you test participants, your study will reach a point where increasing the number of participants will lead to lesser insights obtained. This graph put forward by the Nielsen Norman Group explains it better. 

usability testing participants

(NN/g)

Here,

  • Testing the first participant gives fresh new insights, 
  • Testing a third participant uncovers many problems the first two participants may have missed, 
  • Testing the sixth participant still uncovers new problems the first five participants may have missed, 
  • But by the 12th participant, there would be little to no new insights as most usability problems have already been discovered.

So for practical reasons, testing between 3 to 7 participants gives you that sweet spot where you obtain sufficient insights without falling into the diminishing marginal returns trap.

2. Testing 3 to 7 users is cost-efficient

When developing a product, one main reason usability testing faces a lot of pushback from stakeholders is the cost of conducting usability tests. Depending on the type of usability test, the number of participants recruited, as well as the product type among other factors, study costs can range from as little as $250 U.S dollars to $10K U.S dollars and even much more. 

From recruiting agency fees to software costs and procuring incentives for your participants like gift cards, there are a lot of expenses and hidden costs incurred in running a single usability test. And since costs rise as the number of participants increases, testing a few participants helps you minimize costs and maximize your budget. 

3. Testing 3 to 7 users is time-efficient

Usability tests often take a lot of time, from setup to test sessions to analyzing the results. Depending on the number of participants and the type of study conducted – moderated or unmoderated, it can take a day or many days for completion.

It is estimated that conducting usability tests could take as long as 11 to 48 hours for only 5 participants. So rather than spending a lot of time testing many participants for your study, test only a few participants that would give you more or less the same results at a lesser time. 

Does the 5-user rule hold for different usability tests?

Usability tests include focus groups, surveys, A/B tests, and many more, so the type of usability study carried out plays a huge role in determining the number of participants selected. For instance, usability tests such as card sorting and eye-tracking have different optimal numbers of participants.

usability testing participants

(UXtweak)

Card Sorting

Since card sorting is a generative method whose goal is to find out how people organize and find their way around content, you would ideally need more than 5 participants for the usability test if you’re to get representative results, since people’s mental models widely vary.

The Nielsen Norman Group proposes testing with 15 participants – three times more than the suggested standard of 5 participants, while other researchers insist on testing 20 – 30 participants to get sufficient insights for the test. The difference is that 15 participants would give you a correlation of 0.90 between study results and the actual real-life results, while 20 participants would give you a correlation of 0.93. 

However, if you’re dealing with a large project and have a lot of risks involved, you could test as many as 30 participants.

usability testing participants

(UXtweak)

Eye-tracking

The eye-tracking study can be used to get both qualitative and quantitative data, and as a result, the type of data you want affects the number of participants you need. For qualitative data, Pernice and Nielsen suggest using 5 participants, and these participants can then be tested for a long duration. 

However, if you’re looking to generate heat maps, they suggest recruiting and testing as many as 39 participants. Since heat maps are classified as quantitative tests, several factors can affect their credibility, such as eye problems or other things that could affect people’s gazing behavior.

The perfect number doesn’t exist 

For decades, we have had many debates in the research world, and still, there is yet to be a consensus on the number of participants needed for usability studies. Several factors such as the product type or size, the stage in the usability life cycle, the tasks conducted, the skill of the researcher, and even the personality of the participants all impact the appropriate number of participants necessary for your study.

But one thing all UX researchers agree on is that there is no one-size-fits-all number of participants for your usability study. So always take your own context into consideration.

Uzoma Ibekwe
March 23, 2022
All author's articles

Uzoma is a freelance UX content writer. She also runs UX Writers Learn, a publication that offers a step-by-step guide to user experience writing.

Try our Free plan

It's free for small projects.

Bee
User
testing &
surveys
Card
sorting
Session
recording
Tree
testing

You might be interested in