What’s in this article?

• What is a usertest?
• Why should you do usertests? What will you learn?
• When should you usertest? And what can you test?
• How does a usertest work? What do you need?
• Which questions should you ask during a usertest?
• 5 tips for a professional usertest

What is a usertest?

A usertest is a research method used to investigate how people use your website, app or product. Based on the observations made during a usertest, your website, app or product is made more customer centric and userfriendly.

Naturally, a usertest is centered around people from your target audience: customers or potential customers.

A focus group is not a usertest. 

A usertest is an individual test, where a 1-on-1 relationship between the user and the moderator is key.

This is in contrast to a focus group, where a group of people has a cozy chat and makes up answers to a bunch of largely hypothetical questions.

… Okay, that last point may be a little bit biased. But that’s because we consider focus groups to be a complete waste of time when it comes to usability. Every self respecting usability professional agrees with us here, by the way.

Why should you do a usertest? What will you learn? 

There’s five important reasons to do usertesting.

1. What’s going wrong and most importantly: why? 

By observing your customers and potential customers while they use your website or app, you’ll discover what’s wrong with it and what is going well.

That’s nice. But in and of itself that informational isn’t super useful.

The real value of a well performed usertest is that you’ll discover the why of it all. Why is it going wrong? How come people aren’t finding this or aren’t seeing that? Why do they not understand it? But also: why do they think a certain feature is super useful or fun.

These kinds of insights cannot be gained using any other user research method. 

2. More in-depth and complementary insights to quantitative data

This builds on my previous point. But it’s so important that we’ll address it in more depth.

Google Analytics and similar tools tell you so much. You can see where people are quitting, which products sell the best and the worst, and a thousand other things.

Interesting.

Together with scroll heatmaps, click heatmaps and form analyses, these kinds of data are a blessing to gain insight into the behavior of visitors and customers.

But all those tools won’t tell you anything about the why. It’ll tell you what’s happened and how often, but not why. 

And when it comes to discovering the why, there is 1 method which outperforms any other: usertesting.

You need quantitative and qualitative methods. 

When it comes to research, this isn’t a question of either-or. You shouldn’t choose between quantitative and qualitative research. You need both.

Quantitative methods will show you where the largest problems are and how many people are affected by them. Qualitative methods will provide you answer to the question: why.

Because the quantitative analysis tools will almost never help you figure out whether people drop off from a page because of missing information, unclear texts, photos which do not appeal to them, missing social proof or whatever. And that’s exactly what you’ll find out during a usertest.

3. There’s no U in UX without users

A lot of usability rules and UX guidelines are actually pretty vague. Like:

• Your navigation should be clear to all of your users.
• Your product texts should speak to your users and answer their questions.
• On your category pages, display the most important decision criteria.
• Your design and interface should be intuitive, so that a user doesn’t need to think.

Cool.

Makes sense.

But how? How do you find out all that stuff?

Your Google Analytics and your heatmaps won’t help you now. 

And your own opinion?

Forget it. You are not your customer.
And you never will be.

Impossible.

Oh, but you’re a UX specialist with a lot of experience? So you know everything.

Don’t make me laugh.

Without users, there is no U in UX. 
And the more experience you have, the more you realize how true that is. After 20 years, not a day of usertesting goes by that I don’t learn something new.

For an answer to all those questions, and insight into what and how your customer thinks, you need to do usertests. Because your own perception of your users and customers is often a far cry from reality.

4. Empathy

Numbers are fun. But often, I see them having very little impact.

Because they’re anonymous. Because there’s no feeling there.

43% of people drops off at this page!
“Well”, you see people think during a meeting.

During that same meeting, show people 4 recordings of a usertest where you can see customers struggling, digging around and grimacing. And you’ll feel the pain of those users spread through that meeting room. 

Watching your customer struggle, helps. It’ll make you feel that customer’s pain. And that will drive you to change things.

Thanks to a usertest, you’ll see the people behind the numbers. And that’s much more convincing than a high drop-off number in your analytics tool.

Not just for the web team, but for the whole organisation. Show those recordings at a board meeting and suddenly you’ll receive the budget you need to really improve customer experience.

5. Companies that do user research grow up to 5 times faster

We’ve written about that before in this article on why you should be doing user research.

When should you be doing a usertest? What can you test?

• A live website or app
  Just because you’ve had your app or website for a while, that doesn’t mean it’s perfect. Everything can always improve. From every usertest you’ll learn something new. That’s why companies like Booking, Bol.com and Coolblue do usertests continually.
  
  
• Designs and prototypes 
  One piece of advice: do a usertest on your website or app before you launch it. That way you’ll uncover all the important mistakes. Before they have a negative impact on your customers and your revenue.
  
  
• The sooner, the better 
  The sooner you test, the cheaper it is to change and adjust things. A wireframe or prototype is the perfect basis for a usertest. And it can be changed and adjusted much quicker than a website or app that’s been fully programmed.
  
  
• Iterate. Incorporate usertesting in your agile process.
  Perform usertests in every fase of your creative process. You’ll learn much more by doing usertests with 5 participants every fase than to do one big round at the end with 25 participants.
  Read how iterative user testing helped make itsme one of the most popular apps in Belgium.

 

How does a usertest work? What do you need? 

• Select the right participants 
  A usertest is only valuable when the participants are a part of your target group. Preferably we ensure a mix of customers, competitors’ customers, and potential customers. That way you’ll gather accurate insights into the behavior of your target group. How many people we invite? That depends on the nature of the project and the diversity of your target group. The minimum is five people. 
  
  
• Individual sessions with a moderator
  We cannot stress this enough. Individual sessions.
• In person or remote
  You can invite users at a certain location, like your office or ours. But a usertest can also take place completely online, through remote tools like Skype or Zoom.
  
  
• A scenario with typical tasks
  The basis of a usertest is observing the user while they perform typical tasks. That’s why it’s important that you create a scenario with a few of these tasks. What kind of tasks they are, depends on your website or app.
  
  
• A good, educated moderator 
  The moderator manages the usertest so that it goes smoothly. They ensure that the participants are comfortable, assign tasks, ask questions at the right time and refocus the participant when they stray too far from the task. And they do all of this in a natural way, without influencing or offending the participant. That seems simple, but it’s not. There’s a reason that our in-company training to turn your team into good moderators takes two days.
  
  
• Make sure the team can follow along with the test live
  Reading a usertest report never has the same impact as following along with the same test live. That’s why we encourage our customers and their web teams to watch the tests live. An added advantage is that the team can ask the participant extra questions through the moderator.
  Following a usertest can be done in a tracking room, which is quite cozy. But it can also be done online, which can be more practical and of course it’s corona proof.
  
  
• Record everything
  Record the screen, the expressions and everything that’s said during a usertest. This is necessary so that you can watch and analyse everything after the test is over. And it will allow you to create a video compilation. We often see that videos have a huge impact, more so than a written report. Because it is much easier to emphasize with a video of a user.

Which questions do you ask during a usertest? 

1. You don’t ask questions. You set tasks. 

The foundation of a usertest is to observe people. “Don’t listen to users“, Jakob Nielsen says.

Usertesting is not asking people how they feel about something. Or what they would do if xyz?

That may be the biggest misunderstanding about usertests. That you should ask people what they want or what they would do. Argh. That has nothing to do with research.

What this means, is that your scenario should mostly consist of tasks. Not questions.

Drawing up a good task sounds simpler than it is. So I’ll provide some extra tips.

2. Tasks should align with the top tasks

This one is very often done well. When a site’s main goal is to sell computers, we ask a user during a usertest to buy a computer. Or a graphics card. Or to contact supports.

Makes sense.

Although we often see that during user tests, people are searching for exceptions. With tasks like: “Image that you wanted to know if this company was conforming to GDPR laws, what would you do?” Don’t do that.

Focus on realistic top tasks.

3. Tasks should align with your participant 

This is where a usertest is usually derailed. During many usertests, every participant is asked the same exact questions. Even though every person is different. 

An often-made mistake is to provide a very specific task. 

For example: “On this website, find out how you can purchase traveler’s insurance for a skiing holiday”. That’s a good question when your participant is an enthusiastic skier. But it’s a ridiculous question when that person’s never skied in their entire lives, and isn’t ever planning to.

The biggest problem with questions like that, is that participants will perform the tasks really superficially. Because it doesn’t really interest them.

For the right results, it’s really important that the tasks in a usertest align with the participant. 

You can do that by firstly asking the right questions during the pre interview. The pre interview is the first part of a usertest, and it’s where we get to know our participants: what they do, what they like, what interests them and what doesn’t.

And then we adjust our tasks to align with that information. For example: “You just told me that you’ll be going on a holiday to the United States. Can you figure out what kind of traveler’s insurance you might need for that?”

4. The more realistic the task, the better the result 

Even better is when you connect real consequences to the completion of a task. If your website sells shoes, for example, then a great task could be: “Is there a certain pair of shoes that you might like?” (…) “Okay, then you can try to find those. And you can take home whatever pair you select. It’s on us.”

By setting realistic tasks like this, you trigger the most natural behavior. And you can see what users really pay mind to when they choose and buy a product.

5 tips for professional usertests

Okay, okay. You got it, usertests are important.

And now you’re probably thinking, I can do that myself.

That’s true.

If the alternative is that you don’t do any usertests, please do your own usertests. Go sit next to a user, let them use your website and learn.

But if it’s an option for you, then you should let a professional do your user research. Why? 

1. We won’t allow your mom to participate 

Why not? Who would rather help you than your very own mother? And for free!

But your mom won’t really be very objective.

As we’ve said earlier: choosing the right participants is crucial. 

We will select the right participants. People from your target audience. Of all ages, education levels, background or whatever else is important to get a good cross-section of your users.

2. We’ll react in the right way, at the right time

That sounds simple, but it’s not.

It can be really difficult not to react to a user’s behavior, for example by laughing or expressing surprise when they cannot find something which is obviously right there in the middle of the screen. 

But asking the right questions at the right time is also an art.

We will never ask a participant, for example, how do you feel about this? Or: if this feature existed, would you use it?

Because that information gets you nowhere.

Asking after hypothetical situations isn’t useful. Asking people how much they would spend on a product – just don’t. Because non-existent money is all too easy to spend.

What we do ask, depends entirely on the participant. Their behavior, expressions and remarks form the basis for our questions. And those questions serve first and foremost to find out the why. Why is someone hesitating? Why are they grimacing, why are they sighing?

3. We won’t ask your users to think aloud

May’be you’re familiar with this technique. Asking your user: “Please use this website as if you were at home. Oh, and could you please think out loud for me?”

Of course! Because that is exactly what you’d do at home. Right?

And yet this think-aloud methode is used very often. Because how else will you figure out why your users are doing something, and what they’re thinking?

If you ask us, it’s pure horror. Because by asking participants to think out loud, they’ll use your website or app much more consciously than they would normally do. And a lot of natural behavior is lost. Which influences your results.

And that’s why having a professional is handy. They’re trained in user research. And that means a hesitation, a small sound, or a fleeting expression, is enough input for them to ask the right question.

So that your user will use your website in the most natural way.

4. We don’t have a flashy usability lab

I can hear you think. “That sounds like a disadvantage. I do want a well-equipped laboratorium, right?”

No. No, you don’t.

Because how often, at home, do you go and get online in your comfortable lab. Cozy relaxation time in that room with two-sided mirror, tracking room, four screens and fifteen cameras?

You don’t.

It’s really hard to generate natural behavior in an unnatural environment. Even if your lab has been fitted to look like a cozy lounge or a living room.

Cute toys. But their value? 0.0.

5. Even Steven Krug thinks you should hire a professional, and he wrote the book on do-it-yourself usability testing

Steve Krug wrote the genius and insightful book Rocket Surgery Made Easy. Perfect for those who want to usability test, but who don’t have any budget.

But even Steve Krug wrote in his book:

“If you can afford to hire a usability professional to do your testing for you, do it. There’s no question [they’ll] be able to do a better job of testing than you will.”

What’s next? 

Wherever you are in the development of your website or app, it’s time for a usertest. Check out how we do user testing.