Artikel - When to do an Unmoderated UX Test, and when not to?

When to do an Unmoderated UX Test, and when not to?

An Unmoderated UX Test – a test where no researcher is present – is different from a usual UX Test in many ways. It is less labor-intensive, faster, more quantitative, and smaller in scope. In this article, we briefly take you through everything you need to know about Unmoderated UX Tests. When do you choose it? And when not?

Unmoderated vs. in the lab

First of all, it is important to clarify the difference between a classic UX Test and an Unmoderated UX Test.

A classic UX Test is done live in the lab or online. As a researcher, you are with your respondent in the same (online) space. The user goes through your site or app. You ask questions, you provide guidance, you see what they do and then you also discover why they do it. There is room to go in-depth. It is a qualitative form of research. You often do a UX Test with six people.

An Unmoderated UX Test is in many ways the opposite. It is a lot more quantitative. If you do an unmoderated test, you let users get to work with your site or app without you as a researcher being present. They do it online, from home. You give them a number of tasks and assignments. They perform these. And afterwards, they answer a few questions about them. So you don't present an entire site or app, but a small part of it; one screen, one small flow. It is very much framed. You cannot intervene or adjust, so you must clearly describe the tasks and assignments. Because it all happens online, you can easily have the test performed by many people. We assume a minimum of thirty people.

'The more you want to learn about 'the why' behind behavior (qualitative), the more suitable a classic UX Test. And the more you only want to learn 'what' people do (quantitative), the more suitable an unmoderated test.'

Reasons to choose an Unmoderated UX Test

You might ask yourself: why would you do an unmoderated test if you can have a qualitative conversation in the lab with much more depth?

There are several reasons for this.

Your test material is simple and compact

Sometimes what you want to validate is simply too small for a UX Test. If you interview someone and that person only has to click through a few screens, then a UX Test costs too much time and money for what it yields. An unmoderated test is a perfect alternative for short questions, for small, clearly defined parts of a flow, app or site. In short: the more you want to learn about 'the why' behind behavior (qualitative), the more suitable a classic UX Test. And the more you only want to learn 'what' people do (quantitative), the more suitable an unmoderated test.

If you interview someone and that person only has to click through a few screens, then a UX Test costs too much time and money for what it yields.

Your target group is difficult to reach

Sometimes it is difficult to get your target group on location for an interview. With an unmoderated test, you don't suffer from that. Your respondents can do the test when it suits them. So you can easily increase your reach by doing an unmoderated test.

You want to weigh variants against each other

An Unmoderated UX Test works well if you want to compare variants of the same page or flow, like an A/B test. Suppose: you have chosen a concept direction. Within that direction, you still see two ways to elaborate on it. Then you present the two variants in an unmoderated test – this way you discover which one works best.

You have little time

You don't have to be present yourself when you perform an unmoderated test. Your time is mainly spent on creating the prototype and preparing the tasks. Once that is done, you send it out. A UX Test involves much more: you have to write the test script, set up the entire test day. That is a process of a few weeks. An unmoderated test can theoretically be set up and performed within a few days. The analysis can also be very fast, provided they are closed questions.

A UX test is a process of a few weeks. An unmoderated test can theoretically be set up and performed within a few days.

What do you lose with an unmoderated test?

The most important thing that is missing in an unmoderated test is depth. You see very well what happens, you can make strong statements about behavior, but you have no (or little) idea why it happens. What the motivations of your target group are. With a classic UX Testyou do learn that. So an unmoderated test is never a substitute for a UX Test – you mainly need to understand very well when to use which method.

An unmoderated test is never a substitute for a classic UX Test – you mainly need to understand very well when to use which method.

Want to do an Unmoderated UX Test yourself? We'd love to help you get started.