Article: Testing in the Lab or Online – Which Is Better?
Testing in the Lab or Online – Which Is Better?
Remember Covid?
Up until 2020, in-person testing was the standard. Almost all of our research took place in our labs in the heart of Amsterdam. Occasionally, we’d run a test online, but that was rare. Clients came to the lab. Participants came to the lab. And then Covid hit.
Suddenly, everything had to move online. It was a sharp shift: from nearly 100% offline to 100% online. This demanded a lot from our researchers. No more face-to-face conversations with participants. Technology became more fragile, since we now relied on people’s devices and internet connections at home. More detailed instructions were needed. In the lab, everything was easier to manage and more controlled. But since we had no choice, we quickly got much better at running UX tests online. In fact, we built our own livestream environment that technically works just as well as our lab. Eventually, having conversations online became almost as natural as sitting across from someone in the lab.
After about two years, Covid ended. You’d expect everyone to return to the lab — after all, wasn’t that what we were all longing for? The opposite turned out to be true: most research is still conducted online. But does that mean online testing is better?
'It was a hard shift: from almost one hundred percent offline to one hundred percent online.'
Online Testing Has Its Advantages
Running UX tests online offers many practical advantages — for the participant, the client, and the researcher. Clients can follow the research from home or the office and continue doing other work. It takes less time and energy. The same goes for participants; they no longer need to travel. They can join from home, or even from a vacation spot — as long as their connection is stable.
There are also substantial content-related benefits to online testing:
A natural environment. People are in their own space, where they feel comfortable. They’re less aware they’re part of a research setting, which leads to more natural behavior. They use their own phone or laptop — the one they use every day — not a test device from our lab. They feel less watched, and the distance helps them feel more at ease — even more honest. Somehow, it’s easier to give critical feedback when there’s a screen between you.
Easier recruitment. Since participants don’t need to travel to the lab, your pool of candidates becomes much wider. You can include people who live far away or those who can’t easily travel due to a disability.
So… Do We Still Need the Lab?
Absolutely. There are plenty of reasons to test in the lab. We recently conducted research for DELA, and the value of in-person testing was crystal clear.
We spoke with people about the period after a funeral — everything that still needs to be arranged. On day one, the interviews were online; on day two, we were in the lab with participants. The difference was striking. On the second day, people cried in front of us — that didn’t happen on the first. This showed us that face-to-face interactions allow for deeper emotional connection and more nonverbal communication. Online, that largely disappears. It’s more distant. There’s always a screen between the researcher and the participant. In some ways, that distance helps — but it also creates… well, distance.
'With face-to-face interviews, there’s more room for connection, emotion, and nonverbal cues. Online, much of that is lost.'
Online testing has made research more accessible. But some of the unique strengths of lab testing simply can’t be replicated online. True human connection is one of them. But there are more:
Not all prototypes can be tested online. Some contain confidential information and can only be shared in a secure environment. In such cases, lab testing is required.
Smoother interaction. In the lab, it’s easier to share and discuss ideas — whether it’s a printed sketch or a visual aid. You can quickly draw something and lay it on the table. That kind of spontaneous exchange is much harder online.
The Sweet Spot: Hybrid Testing
In many cases, the sweet spot lies in combining both approaches — partly online, partly in-person. We call this hybrid testing.
'Participants Remote, Stakeholders Together.'
Interviews can often be done online without issue. But the real difference comes from sitting together with your stakeholders. Sharing insights, brainstorming, exchanging thoughts — we’ve seen that this happens far less effectively online. People get distracted. They multitask. When you’re in the same room, you stay aligned as a team. You keep each other sharp. You feed off each other’s energy — by talking, by being together, even just by having lunch. These “energy moments” vanish in an online setting.
When you sit together, you also share ownership of the insights and quotes. This creates more impact within the organization. When everyone watches from a distance, they feel more distant from the research. No matter how well you organize it, online sessions tend to lose group dynamic.
'When everyone observes remotely, they also feel more removed from the research.'
The Bottom Line: Never 100% Online
If you want to get the most out of your UX test, make sure your stakeholders are physically together on research day. Whether you also bring participants into the lab depends on practical factors: Can you share the prototype online? Can your participants easily travel?Do you want to show physical sketches?
In short: 100% online is almost never ideal.
Need help planning your next UX test? Feel free to call or email us.