Unmoderated testing 101 – Basics of Unmoderated Usability Testing
Usability Testing

Published on January 26, 2021

Unmoderated testing 101 – Basics of Unmoderated Usability Testing

This article will introduce you to the basics of Unmoderated Usability Testing. We will explain why you should conduct such testing, how to prepare tasks, or how to recruit the right participants.

Unmoderated usability testing is definitely on the rise. The fact that it’s cheaper, faster, and more accessible than in moderated or in-person research makes it popular. Nowadays, it is sometimes even the only option for UX teams to conduct user research. Let us take a look at what unmoderated usability testing is.

What is Unmoderated usability testing?

Usability testing is the process of testing designs with users, used for collecting information about user experience – how difficult or easy the product is to use. Because the experience of the user is subjective and intangible, it relies on observing how users interact with the product as a way to learn about their experience. 

According to the to  NNgroup “User Experience encompasses all aspects of the end-users interaction with the company, its services, and its products.”

Unmoderated usability testing is where respondents of the test are not guided or supervised by a moderator (researcher). The test is prepared beforehand, and respondents complete it on their own. 

It is a great way to test and collect valuable feedback on websites or other products. It is frequently used as it is useful throughout a product’s whole lifecycle and provides a lot of insights into a user’s behavior.

With unmoderated usability testing, all you have to do is to prepare the study and recruit people to complete it. The advantages of unmoderated usability testing are that the number of participants is pretty much unlimited. As participants complete the study on their own, no supervisor is needed. As a result, with unmoderated testing, you can collect much more information in a much shorter time at a lower cost.

What is Website Testing?

Website Testing is a tool by UX research platform UXtweak for conducting unmoderated usability tests that also integrates techniques of Session Recording. This gives you the ability to conduct user tests online and then replay user interactions like a video. Allowing you to pinpoint the exact place and time when users had issues completing their objectives.

What can you find out with Unmoderated Testing?

Unmoderated Testing can help you find answers to questions such as:

  • Can people use the website/app for important use cases?
  • How do users interact with the website/app? What are they thinking when using the website/app?
  • How do users tackle their objectives?
  • Is there something that keeps users from fulfilling their objectives? 
  • Do they take a completely different way than they should have or you expected? Or do they take the correct path but get lost and confused along the way?

Thanks to Unmoderated Testing, you will find out answers to these questions and discover any usability and user experience issues. Knowing how users interact and use your website, will allow you to optimize and improve it.

When should you use Unmoderated Testing?

The great advantage of Unmoderated Testing is that it is very flexible – meaning that can be used pretty much anytime during the development or design process. You can use Website Testing to test simple HTML prototypes as well as full of live websites.

All that you need to make it work is a working piece of HTML that’s hosted somewhere public. That way, Website Testing can link to your website, web app, or prototype via URL.

Usability testing using Unmoderated Testing is versatile and scalable. You can set up a simple Unmoderated Testing study with minimum effort in a few minutes, or you can run studies with tens of tasks that cover main use cases, or you can achieve that with just a single task. Tests can be conducted with only a few people, or you can collect results from hundreds of respondents. With Unmoderated Testing, you can create studies that suit your needs.

Preparing Unmoderated Testing

During the preparations of your Unmoderated Testing you should consider asking yourself the following:

What are you going to test?

Obviously, you should start by defining the scope of your usability testing. There will be situations that call for testing all the main use cases of your website or app because you just launched a full redesign to find out what works and what doesn’t. At the same time, all you need might be more focused testing of a few scenarios that cover the new functionality.

Who should be the respondents?

There should not be any user testing without considering your users’ identity. If you already have a website or an app, you’re probably well aware of whom your target group is. You can use any of your existing personas, real customer data, or even data from surveys and other sources of feedback you have.

Information and tho more of it you have is about your user is useful in many ways. Apart from the obvious (to be able to recruit respondents who represent your audience), it will help you prepare the right use cases and formulate relatable tasks that represent the user’s natural way of thinking. So, for these reasons – keep your users in mind when you start planning usability testing.

How to write tasks for Unmoderated testing

At its core, the Unmoderated Testing study consists of several tasks that the respondents are asked to complete to successfully finish the study. By preparing your tasks carefully, you will increase the chance that your user study will provide good results. Well-defined tasks lead the respondents to try to complete them as naturally as they can. 

When preparing keep in mind that you need to:

  • Write tasks that cover your objectives 
  • Write realistic tasks 
  • Write actionable tasks 
  • Write tasks without giving away the answer

Recruiting respondents for your test

Once your study setup is completed and you want to launch the test, now you need to recruit respondents. Recruiting respondents for an Unmoderated Testing study is easy. Simply share a link to the study with your respondents. You can use social media like Facebook, Twitter, and LinkedIn to get it to your target or send the link directly to your testers. Don’t forget your own channels like newsletter subscribers list – they can provide a great source of respondents.

Of course, there is more to recruiting respondents than sending out a link. It is important to prepare a recruitment strategy as a good one will make the difference between learning a lot of valuable information to improve your website to learning nothing at all – wasting a bunch of time and money.

 Questions that should help you with recruiting respondents:

  • Whom do I want to recruit?  

Your respondent sample needs to be representative of the end-user so you can learn about actual user behavior. When using the Website Testing Tool by UXtweak this is very easy as you can make use of – UXtweak Recruiter Widget which will allow you to recruit respondents directly from your website.

  • How many people do I want to recruit? 

According to NNGroup 5 respondents are the optimal number of respondents. With five respondents, you find out most of the main problems in the tested scenarios. Although you might want to gauge the statistical significance of metrics in your study so to get narrower confidence intervals, we recommend at least 20 respondents

  • What information do I want to communicate to the respondents and how?

This will depend on who your respondents are and how you recruited them, but the golden rule is that the respondents should be always well informed. A well-informed respondent has more value for you than someone who has no clue of what’s going on.

UXtweak Recruitment Widget used on e-shop.

Information we recommend sharing with your respondents before testing:

  • How long does it take – e.g. 20 minutes
  • What is expected of them – e.g. completing tasks on the e-commerce website of a fashion company
  • What is the point of the test – e.g. to make our website easier to use 
  • Why would users want to participate? – you should always keep in mind that respondents are not always willing to give you their time for free, especially if the test takes more than a few minutes.

Therefore it is an excellent idea to offer respondents some incentive for participating in the study (could be monetary or something, like a chance to win in competition or different benefit). Always make sure that they’re well informed about the reward and how they receive it.

Analyzing results

After a successful study, you obtained a lot of data. But you still need to analyze it to make something useful out of it. 

Overview

The overview gives you all the information that you need for checking on the state of study. The first thing in the summary is the dates when your study was first created when it was launched, along with the number of respondents who participated in it. Check the donut chart to see how many respondents completed the study and whether there were any quitters.

Time Taken is a number that tells you how much time the respondents spent doing the study. Checking whether Time Taken matches the expected completion time is a good way to keep tabs on any unexpected issues so they can be addressed quickly.

The Success score is the percentage of tasks that the respondents completed by submitting the correct page as the answer. Overview Tab in Unmoderated Testing Tool.

The overview contains a breakdown of various attributes of your users such as:

  • Top Locations  
  • Device Breakdown 
  • OS Breakdown 
  • Browser Breakdown
  • Screen Resolution Breakdown 

Respondents tab

In the list of respondents, you’ll find the information about all respondents who participated in your study. It’s here where you can replay the completion of tasks, filter your respondents, and select which ones should be included in the analysis. If you want to learn more, check out Interpreting WebsiteTesting results in our Definitive Guide to Website Testing.Respondets Tab in Unmoderated Testing Tool by UXtweak

Task RePlay

Pressing Play on the task will open up the session player, showing you exactly how the interaction between the tester and the website/app went down. You will be seeing what the respondent was while they were trying to complete the task. You can control the session player like any video player. If you read our Session Recording guide, you know that even though it looks and feels like a video, it is a re-interpretation of how the respondent interacted with your website/app.

Conclusion

Unmoderated usability testing is a simple way to get a lot of information about how your users behave. The Website Testing tool gives you the option for recruiting the right respondents that fit your needs easily – by sending them a link or using the recruiter widget.

All results are analyzed after the end of the study, giving you detailed information about tasks, answers to questionnaires, respondents, etc. This helps you to understand the issues on your website and to fix them quickly, easily, and in a budget-friendly way. Try it yourself – register to Utweak!

Free 5-day Pro trial

Try Profesional plan with all features for free.

Bee
User
testing &
surveys
Card
sorting
Session
recording
Tree
testing

You might be interested in

User Research UX Research-current trends and the future

Research is the inevitable part of UX as such and we are glad to see that UX research is becoming a trend itself. Let’s find out more about current trends and the not so far future of UX research. Read more ...

UXtweak Team
UXtweak Team
April 16, 2021
Usability Testing Loop11 Alternatives

Are you using Loop11 as your usability testing tool, but you're not quite satisfied with how it's working? Don't worry! To help you explore your options, we prepared a pricing and feature comparison of some of the alternatives. Read more ...

UXtweak Team
UXtweak Team
April 9, 2021
Usability Testing Usability Testing Questions: How to Ask Correctly

This article will give you tips and advice about writing the most effective usability testing questions for your research. We will explain why it is important to ask good and effective questions and how to do it. Let’s dive in.  Read more ...

UXtweak Team
UXtweak Team
April 2, 2021
News and Updates 10 Best tools for startups in 2021

Choosing the right tools can help your startup with managing communication, project management, marketing, UX and SEO optimization of your webpage. We made you a list of the 10 Best tools for startups in 2021 to help you grow your new business. Read more ...

UXtweak Team
UXtweak Team
March 26, 2021

Stay informed

Subscribe for UXtweak news & useful UX tips and tricks

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.