Never miss another tip to improve your website

  • 8,113 subscribers
  • Once a month
  • Exclusive tips and special offers

User testing: what, why and how?

User testing is a great way to discover problems with your website or app. And to increase empathy with your users. 

But how does it work? Why is it such a valuable method? And which type of user testing can you use in which situation? 

How does user testing work?

  • You ask a number of people to perform a number of typical tasks on your website or intranet.
    Or on a mock-up if you're in the process of building a new one.  
     
  • The test is performed on an individual basis.
    So it’s not like a focus group where there's a bunch of people giving you feedback all at once. 
    Please, don't ever call a focus group a user test
     
  • The test participant should belong to your target audience. Recruit for engagement, not demographic representation.
    Ideally, the test participants are existing or potential clients.  
     
  • Everything the test participant does and says is recorded.
    That way you and your team can review and analyze the data afterwards.  
     
  • Encourage the test participant to think aloud.
    Simply let them vocalize whatever's going through their head while they're surfing on your site. If this doesn't come naturally to them, don't push it. Observing participants' behaviour is more important than listening to what they're saying.

Why is user testing more valuable than an expert review?

4 reasons why user testing is more valuable than an expert review or heuristic analysis.

4 reasons why a good usability expert will often recommend to do a user test.

1. Experts don’t know everything

No matter how much expert knowledge and experience you have, you can’t know everything.

You can’t always judge situations perfectly. That’s a lesson I've learnt in more than 17 years as a usability expert. You can read more on that topic in our article 'Experts don't know everything, not even usability experts'.

2. Facts trump opinions

Facts are perfect to do away with opinions.
In a discussion, bring facts as your weapon of choice.

Clients can always dismiss expert advice with the argument "Yeah sure, but that’s just your personal opinion, right?".

User tests reveal facts.

And facts trump opinions any day of the week.

3. Ignoring your own clients is difficult

Or at least, ignoring your own clients is more difficult to do than ignoring expert advice. 

If you have seen with your own eyes that 5 out of 6 people have trouble using the filter navigation in your webshop, there's no way you can't realize this is a problem you have to fix.

User testing creates empathy with your users. If they're struggling on your site, I can tell you that's very painful to watch. You get to share in that struggle. 

4. Convincing and durable proof

A video that shows your customers and potential customers having trouble with your website, cursing your interface?
That has immense power. That kind of video will shut up even the most die-hard, ego-tripping know-it-all experts and directors during board meetings. 

Also, you can re-use the recordings at any stage of the project. Whenever that same old discussion rears its ugly head again, you just whip out that recording: “Remember, this is what we saw during user testing...”.

Keep hammering on that nail.

Most common types of user testing

1. Moderated user testing

Launching a website without any user testing is like playing russian roulette.

The classic user test set-up is in-person moderated user testing. The moderator sits next to the test participant and asks questions. 

This old-school method still has a lot of benefits. Observing the user in person and being able to ask them additional questions based on what they're doing or saying at a certain point, can give you very deep insights into what's really wrong with a website. 

To me, that is priceless information. And it can't be replicated by any other method. There are some things you simply can't get out of mouse clicks and scroll maps.  

Having said this, good moderating skills are obviously key in moderated user testing. The way the moderator asks the questions and even just the way she sits or looks at the user, can heavily influence your test results. 

2. Remote moderated user testing

The biggest disadvantage of the 'classic' in person user test is that your test participants either have to come to you or you have to go to them. And time is simply money.

Remote moderated user testing is an excellent alternative for in-person user testing.

With remote user tests, the moderator and test participant can be wherever they want to be, as look as they're looking at the same website on some screen or other. With screen sharing software that’s a piece of cake nowadays.

If your user has a webcam, you can also see their face. Which makes it a good alternative for in-person moderated user testing.

User testing with GoToMeeting
We use GoToMeeting for remote moderated user testing. The moderator can see the user’s screen, as well as their face. This random test participant is not very happy with KBC Bank’s homepage.

We often do remote moderated testing if we need test participants from different countries. It's also a great solution if you only want to test something small: one particular page or a step in a process. 

3. Remote unmoderated user testing

As you probably already guessed, there is no moderator in this type of testing. 

There are 2 main variations of remote unmoderated user testing where you have footage of the test participant's actions on your website as well as feedback:

  • Recorded image and sound
    Users are asked to think out loud.

    To be fair, I'm not a big fan of this method. It relates to classic user testing like sardines to caviar.
    Many test participants you get with these tools sound as if they're semi-professional test users. 
    You can also test with your own user panel. Which is definitely an improvement on using the tool's testers.
     
  • Recorded mouse movements and time
    In this method, you don't see the test participant and you don't hear them. They see the task on the screen, perform the task and fill out a questionnaire to give additional feedback after each task.

My biggest issue with this type of testing is that it leaves out any real communication or interaction with the user. If they make a comment you don't really understand, you can't ask them that all-important question: why?

If a test participant says something during a user test you want to get to the bottom of, keep asking why until there are no why's left.

And what about eye tracking?

We’ll discuss eye tracking in a separate article. Eye tracking can be very interesting, but it’s all too often misused and abused.

Other user research tools

There are of course lots of other user research tools out there.

I really like HotJar's user session recordings if you want to get a quick look of how people use your pages.
Sometimes a problem is so obvious all you need to see is those erratic mouse movements and rage clicks.

For our information architecture projects, where we often have mock-ups we like to test on, I'm a big fan of Optimal Workshop's first click testing tool Chalkmark
 

Do you have a preference for one user testing method over another?
Share your thoughts and experiences in the comments!

Don’t miss our next update!

  1. Follow us on Twitter for interesting links and extra tips.
  2. Get our monthly newsletter