Online surveys: useful or useless?

What will you read in this article?

Why most online surveys suck

On websites, surveys are commonly used to ask people what they think or how they feel. “How do you feel about our search feature?” or “Wat do you think about our menu options?”

That might seem like a good idea, but in practice we see that 9 out of 10 online surveys like that are useless.

This is not the way to find out how to improve your website.

The problem with the ‘before your visit’ survey

The visitor arrives at a website, and before the page has well and truly loaded, they’re asked to take part in a survey asking them how they ‘feel’ about this and what they ‘think’ about that.

But how can anyone say anything sensible at that point? They haven’t even been on your website for more than 5 seconds? It’s impossible.

The problem with the ‘after your visit’ survey

After looking at several pages, or when leaving the website, the visitor is asked to participate in a survey. Sometimes, they’re asked for their e-mail address when landing on the website, and they receive the survey in their mailbox afterwards.

So is this the way to go?

No. Absolutely not.

The human brain is an odd thing. Mere seconds after an experience, our brain begins to alter the memory. Very often, people tend to remember things being more positive than they really were.

If you don’t believe this, try observing a user test.

A while ago, we did moderated user testng on a website with a fair few shortcomings. The percentage of correctly completed tasks per test user was between 20% and 40%. That’s not too great. (But also not that exceptional, if we’re totally honest. So don’t automatically assume your website would fare much better…)

After the test, participants were given a post-test survey where we asked their opinion about the site. Wat did they think and feel? The result: 70% of participants thought the website was okay. “Most of it went alright, didn’t it?” “It wasn’t always smooth sailing, but I found it in the end.” Remember: these people had been struggling to complete tasks on this very site just minutes before.

And trust us. These weren’t compulsive liars. They were people like you and me.

How to make your surveys worthless?

Tip #1: Ask for a rating

You’re familiar with them: those questions with a rating from 1 (yuk, rarely seen anything so horrible) to 5 (wow, amazing).

Like we just mentioned, asking for a rating is usually not a good idea. Some facts to refresh your memory:

Memory is more rose-colored than reality
Ratings often do not match reality
Every person has a different scale of value by which they rate things

So, why do so many companies do this?

Why do so many companies ask these kinds of questions?

Absolutely no work to set up
Standard questions and ditto answers. Just a graph. Congrats to those who’ve turned this into a living.
Better than reality
Even with a bad website, you’ll get high scores.
Results with one push of the button
When the surveys are completed your results chart is jut a click away. Impressive.

Tip #2: Fill out our survey and win stuff

Another brilliant find. “Fill out our survey and have a chance to win a weekend to Paris.”

And then the first question: “How do you feel about our website?”

Even I’ll click, “wow amazing”. Who cares about your website? I want to go to Paris!

How do you ask for feedback from your visitors in a sensible manner?

Tip #1: Ask about facts

A few sample questions:

What’s the main reason for your visit to this website today?
What problem or question brought you to this website today?
Did you find what you were looking for? If no: what didn’t you find?

The answers you’ll get from these questions, will help you to get to work on concretely improving your website. And actually your whole communication and marketing approach.

And we only ask 1 or 2 questions at a time. That’s why the response rate to our surveys sometimes tops 50%.

Some examples of questions our surveys can answer for you:

What are your users’ most important decision criteria when it comes to buying your product?
What do and don’t they know about your product/market/…?
What’s stopping them from becoming a customer? From requesting a quote?
What do people consider your website or intranet’s greatest strengths and weaknesses?
Why don’t people use your intranet or extranet?

Tip #2: Ask open questions

What’s so great about open questions?

Spontaneous answers
Nothing has been spelled out for them like it is when you use check-boxes. Indeed: a good open question is the best assurance for honest and spontaneous answers.
You get to know your visitors’ vocabulary
How do they describe the reason for their visit? This way you might discover what words you’re not using yet.
Once you’ve discovered those words, you can use them to write copy that really resonates with your audience.
You’ll discover what brings people to your website. (And believe me: that can be soberingly boring.) You discover, in other words, your top tasks.
People will tell you bluntly whether they found what they’re looking for or not. It’s not always nice to hear, but you can get to work with that information right away. You can’t say that about a pie chart.

Tip #3: Limit the number of questions

We advise always starting with just 3 questions. That all fit into 1 screen.

The fewer questions, the higher the response rate.

The added advantage of the web is that you can target where and when a survey appears very precisely. So you can ask the right questions at the right time.

Why are there so few useful surveys with open questions?

I don’t know. This isn’t just my opinion by the way. In professional literature (concerning usability, not marketing) this is also stated.

The most important reasons, according to me

Unfamiliar = unloved
Many marketing folks haven’t a clue that they’re really doing badly, and that they could be doing this differently. They never learned.
“Pfff. That’s a lot of work.”
Indeed. No pretty graphs at the push of a button, but endless lists that you’ll have to read and process. But nothing good is ever easy. Frank de Winne (Belgian astronaut) isn’t floating around the earth thanks to a pie chart.
It’s not sexy to present
It’s true, you can’t just put this into a little graph. You can only do that by reading all the answers, then assigning those answers to different categories, connecting a lot of dots, etc.
And by the way: don’t you prefer real answers, that you can get to work with right away, to a stupid graph?
“3 questions? You’re not serious, are you?”
That’s a response we often get. Even from clients. People who are familiar with our down-to-earth approach.
But I guarantee: there will be more people who respond to these three questions than your “step 1 of 7-survey”. And you’ll actually have something useful afterwards.
Honest answers by your visitors
Few will openly admit this, but many marketing people and web responsibles are scared of the blunt judgememnt of the visitor. Be startled by the facts.

Surveys: no one fills those out anymore, do they?

10 to 25% of visitors fills out our surveys. That’s a very high number.
How?
- We ask only 2-3 questions.
- Those questions are immediately presented on the screen.
- So people will be able to see what they’re getting themselves into.
- We have tested dozens of variations in terms of content and terminology to get to these statistics.
- Contrastingly, on average less than 1% of visitors fills out a survey.