Do’s and don’ts for a good user test

What’s in this article?

User tests are the best way to discover how people interact with your website, app or SaaS. What do they appreciate? Where do they hesitate? Where do they get stuck? And why? That’s what you’ll discover during a user test. You can do a user test not only on a live website or app, but also while your product is still under development. Or to verify whether that new feature you’re looking to develop really is useful and convenient.

What is a user test?

A user test is a research method used to investigate how people use your website, app or product. Based on the observations made during a user test, your website, app or product is made more customer centric and userfriendly.

Naturally, a usertest is centered around people from your target audience: customers or potential customers.

A focus group is not a user test. 

A user test is an individual test, where a 1-on-1 relationship between the user and the moderator is key.

This is in contrast to a focus group, where a group of people has a cozy chat and makes up answers to a bunch of largely hypothetical questions.

… Okay, that last point may be a little bit biased. But that’s because we consider focus groups to be a complete waste of time when it comes to usability. Every self respecting usability professional agrees with us here, by the way.

Focusgroepen zijn waardeloos
Focus groups are worthless for UX research.

Why should you do a user test? What will you learn? 

There’s five important reasons to do user testing.

1. What’s going wrong and most importantly: why? 

By observing your customers and potential customers while they use your website or app, you’ll discover what’s wrong with it and what is going well.

That’s nice. But in and of itself that informational isn’t super useful.

The real value of a well performed user test is that you’ll discover the why of it all. Why is it going wrong? How come people aren’t finding this or aren’t seeing that? Why do they not understand it? But also: why do they think a certain feature is super useful or fun.

These kinds of insights cannot be gained using any other user research method. 

2. More in-depth and complementary insights to quantitative data

This builds on my previous point. But it’s so important that we’ll address it in more depth.

Google Analytics and similar tools tell you so much. You can see where people are quitting, which products sell the best and the worst, and a thousand other things.

Interesting.

Together with scroll heatmaps, click heatmaps and form analyses, these kinds of data are a blessing to gain insight into the behavior of visitors and customers.

But all those tools won’t tell you anything about the why. It’ll tell you what’s happened and how often, but not why. 

And when it comes to discovering the why, there is 1 method which outperforms any other: usertesting.

Kwantitatieve data zijn interessant maar hebben hun beperkingen
Quantitative data is interesting, but it won’t tell you about the why. Drawing by Indeed UX Research Manager Dave Yeats.

You need quantitative and qualitative methods. 

When it comes to user research, this isn’t a question of either-or. You need both.

Quantitative methods will show you where the largest problems are and how many people are affected by them. Qualitative methods can help you understand why something is a problem.

Because the quantitative analysis tools will almost never help you figure out whether people drop off from a page because of missing information, unclear texts, photos which do not appeal to them, missing social proof or whatever. And that’s exactly what you’ll find out during a user test.

3. There’s no U in UX without users

A lot of usability rules and UX guidelines are actually pretty vague. Like:

  • Your navigation should be clear to all of your users.
  • Your product copy should be clear and answer users’ questions.
  • On your category pages, display the most important decision criteria.
  • Your design and interface should be intuitive, so that users don’t have to think.

Cool.

Makes sense.

But how? How do you find out all that stuff?

Your Google Analytics and your heatmaps won’t help you with that 

And your own opinion?

Forget it. You are not your customer.
And you never will be.

Impossible.

Oh, but you’re a UX specialist with a lot of experience? So you know everything.

Don’t make me laugh.

Without users, there is no U in UX. 
And the more experience you have, the more you realize how true that is. After 20 years, not a day of usertesting goes by that I don’t learn something new.

For an answer to all those questions, and insight into what and how your customer thinks, you need to do user tests. Because your own perception of your users and customers is often a far cry from reality.

Wat jij denkt over je gebruikers en klanten staat vaak heel ver af van de werkelijkheid. Daarom moet je een gebruikerstest doen.
How you feel about your users and customers is often a very far cry from reality.

4. Empathy

Numbers are fun. But often, I see them having very little impact.

Because they’re anonymous. Because there’s no feeling there.

43% of people drops off at this page!
“Well”, you see people think during a meeting.

During that same meeting, show people 4 recordings of a user test where you can see customers struggling, digging around and grimacing. And you’ll feel the pain of those users spread through that meeting room. 

Het e-commerce team van de Bijenkorf voelt de pijn van hun klanten tijdens een gebruikerstest.
The Bijenkorf e-commerce team fobserving a user test in our office. You can tell by looking at their faces how much it pains them to see someone struggling with their mobile website.

Watching your customer struggle, helps. It’ll make you feel that customer’s pain. And that will drive you to change things.

Thanks to a user test, you’ll see the people behind the numbers. And that’s much more convincing than a high drop-off number in your analytics tool.

Not just for the web team, but for the whole organisation. Show those recordings at a board meeting and suddenly you’ll receive the budget you need to really improve customer experience.

5. Companies that do user research grow up to 5 times faster

When should you do a user test? What can you test?

  • A live website or app
    Just because you’ve had your app or website for a while, doesn’t mean it’s perfect. From every user test you’ll learn something new. That’s why companies like Booking, Bol.com and Coolblue do user tests continually.

  • Designs and prototypes 
    One piece of advice: do a user test on your website or app before you launch it. That way you’ll uncover all the important mistakes. Before they have a negative impact on your customers and your revenue.

  • The sooner, the better 
    The sooner you test, the cheaper it is to change and adjust things. A clickable wireframe or prototype is the perfect basis for a user test. And it can be changed and adjusted much quicker than a website or app that’s been fully programmed.

  • Iterate. Incorporate usertesting in your agile process.
    Perform user tests in every fase of your creative process. You’ll learn much more by doing user tests with 5 participants in every phase than to do one big round at the end with 25 participants.
    Read how iterative user testing helped make itsme one of the most popular apps in Belgium.

 How does a usertest work? What do you need? 

  • Select the right participants
    A usertest is only valuable when the participants belong to your actual target audience. People who give a damn about what you’re trying to tell them or sell them.

  • How many participants do you need?
    That depends on the nature of the project and the diversity of your target group. The minimum is five people.

  • In person or remote
    You can invite users to a certain location, like your office. But a usertest can also take place completely online, through remote tools like Skype, Zoom or Lookback.

  • A scenario with typical tasks
    The basis of a usertest is observing the user while they perform typical tasks. That’s why it’s important that you create a scenario with a few of these tasks. What kind of tasks they are, depends on your website or app.

  • A good, educated moderator
    The moderator manages the usertest so that it goes smoothly. They ensure that the participants are comfortable, assign tasks, ask questions at the right time and refocus the participant when they stray too far from the task. And they do all of this in a natural way, without influencing or offending the participant. That seems simple, but it’s not. There’s a reason that our in-company training to turn your team into good moderators takes two days.

  • Make sure the team can follow along with the test live
    Reading a usertest report never has the same impact as following along with the same test live. That’s why we encourage our customers and their web teams to watch the tests live. An added advantage is that the team can ask the participant extra questions through the moderator. Following a usertest can be done in a tracking room, which is quite cozy. But it can also be done online, which can be more practical.

  • Record everything
    Record the screen, the expressions and everything that’s said during a usertest. This is necessary so that you can watch and analyse everything after the test is over. And it will allow you to create a video compilation. We often see that videos have a huge impact, more so than a written report. Because it is much easier to emphasize with a video of a user.

Which questions do you ask during a usertest? 

1. You don’t ask questions. You set tasks. 

The foundation of a user test is to observe people. “Don’t listen to users“, Jakob Nielsen says.

Usertesting is not asking people how they feel about something. Or what they would do if xyz?

That may be the biggest misunderstanding about usertests. That you should ask people what they want or what they would do. Argh. That has nothing to do with user testing.

Your user test scenario should mostly consist of tasks. Not questions.

Els Aerts over taken in plaats van vragen tijdens een gebruikerstest.
Our very own Els Aerts emphasizes what user research is really about at a conference.

Drawing up a good task sounds simpler than it is. So I’ll provide some extra tips.

2. User test tasks should align with the top tasks

When a site’s main goal is to sell computers, we ask a user during a usertest to buy a computer. Or a graphics card. Or to contact support.

Makes sense.

Although we often see that during user testing, people set tasks for exceptions. With tasks like: “Imagine if you wanted to know what this company does with regards to GDPR compliance, what would you do?” Don’t do that.

Focus on realistic top tasks.

3. Tasks should align with your participant’s life 

This is where a user test often goes wrong. Often, every participant is asked the exact same questions. But every person is different. 

An often-made mistake is to set a task that is too specific. 

For example: “Find out how you can purchase traveler’s insurance for a skiing holiday”. That’s a good question. If your participant likes to ski. But it’s a ridiculous question when that person’s never skied in their entire life, and isn’t ever planning to.

The biggest problem with questions like that, is that participants will perform the tasks really superficially. Because it doesn’t really interest them.

For the right results, it’s really important that the tasks in a usertest align with the participant. 

You can do that by firstly asking the right questions during the pre-interview. The pre-interview is the first part of a user test, and it’s where we get to know our participants: what they do, what they like, what interests them and what doesn’t.

And then we adjust our tasks to align with that information. For example: “You just told me that you’ll be going on a holiday to Canada. Can you figure out what kind of traveler’s insurance you might need for that?”

4. The more realistic the task, the better the result 

Even better is when you connect real consequences to the completion of a task. If your website sells shoes, for example, then a great task could be: “Is there a certain pair of shoes that you might like?” (…) “Okay, can you try to find those? If you find what you like, you can buy them. We’ve got a 50 dollar voucher you can use today.”

By setting realistic tasks like this, you get the most natural behavior. And you can see what users really pay attention to when choosing and buying your product.

5 tips for professional user tests

Okay, okay. You got it by now, user tests are important.

And now you’re probably thinking, I could do that myself.

That’s true.

If the alternative is that you don’t do any user testing, go ahead and do your own user testing. Go sit next to a user, let them use your website and learn.

But if it’s an option for you, then you should let a professional do your user research. Why? 

1. We won’t allow your mom to participate 

Why not? Who would rather help you than your very own mother? And she’d do it for free! Major bonus!

But your mom won’t really be very objective, will she?

As we’ve said earlier: choosing the right participants is absolutely crucial. 

We will select the right participants. People from your target audience. Of the right age, education level, background or whatever else is important to get a good cross-section of your actual target audience.

2. We’ll react in the right way, at the right time

That sounds simple, but it’s not.

It can be really difficult not to react to a user’s behaviorAnd not to intervene.

Asking the right questions at the right time is a skill.

Asking hypothetical questions, for example, simply isn’t useful. Asking people how much they would spend on a product – just don’t. Because non-existent money is all too easy to spend.

What we do ask, depends entirely on the participant. Their behavior, expressions and remarks form the basis for our questions. And those questions serve first and foremost to find out the why. Why is someone hesitating? Why are they frowning, why are they sighing?

3. We won’t ask your users to think aloud

Maybe you’re familiar with this technique. Asking your user: “Please use this website as if you were at home. Oh, and could you please think out loud for me the whole time?”

Of course! Because that is exactly what you’d do at home. Right? Talk out loud to yourself while you’re using a website?

And yet this think-aloud methode is used very often. Because how else could you possibly figure out why your users are doing something, and what they’re thinking?

We’re not really fans of this talk aloud method. Because if you ask participants to think out loud, they’ll use your website or app much more consciously than they would normally do. And a lot of natural behavior is lost. Which influences your results.

And that’s why having a professional moderator is handy. They’re trained in user research. And that means a hesitation, a small sound, or a fleeting expression, is enough input for them to ask the right question. At the right time.

So that your user will use your website in the most natural way. And you will get the most reliable research data to base your decisions on.

4. We don’t have a flashy usability lab

I can hear you think. “That sounds like a disadvantage. I do want a well-equipped user lab, right?”

No. No, you don’t.

Because how often, at home, do you go and get online in your comfortable user lab. Cozy relaxation time in that room with the two-sided mirror, tracking room, four screens and fifteen cameras?

You don’t.

It’s really hard to generate natural behavior in an unnatural environment. Even if your lab has been fitted to look like a cozy lounge or a living room.

5. Even Steven Krug thinks you should hire a professional, and he wrote the book on do-it-yourself usability testing

Steve Krug wrote the genius and insightful book Rocket Surgery Made Easy. Perfect for those who want to usability test, but who don’t have any budget.

But even Steve Krug wrote in his book:

If you can afford to hire a usability professional to do your user testing for you, do it. There’s no question they’ll do a better job of testing than you will.”

Check out how we do user testing.

Related posts

Read more articles about:

Don’t miss a single tip

  • Be the first to know about new trainings
  • Receive tips and advice about your sector and professional interests
  • No spam

Or follow us on LinkedIn