Introduction

Creating surveys that respondents enjoy is more than just a “nice-to-have,” there is a genuine business case for committing to quality survey design and respondent experience. Poorly designed surveys have shown to increase respondent dropoff rate as well as produce weaker insights due to rushed unattentive answering from participants. Respondents want to be heard, it has been proven that they enjoy giving their feedback, but in order to get the most out of your respondents you need to ensure that you’re avoiding these 5 pitfalls of survey design and user experience.

Point 1: Email Invitation

• Problem: Using email to reach your respondents is the equivalent of bringing a knife to a gunfight in terms of preparation. As a researcher, you are putting all of your eggs in one basket by using email in isolation to reach a given population. This becomes even more severe when you are speaking to 13-30 year olds who have surpassed the capabilities of traditional email in favour of the smartphone.

• Solution: The medium in which we reach, engage and invite people to participate in online research has to change. That is not to say email is still a powerful tool for older generations, it just means as researchers we need to have multiple tools at our disposal to maximise engagement. It is not just our role to design the survey, it is our role to find and excite new people to participate in research. For example, at Bounce we leverage push notifications to increase engagement of our panellists. As a result we boast an incredible 86% engagement rate throughout our entire user base.

Point 2: Profiling Questions & Screening Out:

• Problem: Perhaps the most frustrating and damaging experience for respondents is the repetitive and monotonous answering of the same set of questions every time you are invited to complete a survey. As quality starts to dip after spending anytime longer than 4-7 minutes in a given survey, spending this time answering screening questions not relevant to the data being collected is a cardinal sin of research that simply must change.

• Solution: For a first-time participant, it is fundamental that you gather the traditional profiling data so that the respondent is receiving surveys where their opinion is valued. However, this profiling data should be built on with each survey they fill out, not repeated upon every survey to leave them frustrated and bored. This can be achieved by pre-targeting what panelists fit the screening criteria for each survey, and only inviting those who fit the criteria to participate. This means their valuable time is spent answering the questions that matter most, rather than reaffirming their gender, age and profession.

Point 3: Survey Quality

• Problem: Online surveys are in desperate need of a re-birth. The penny has dropped with industry leaders. No person is willing to spend longer than 10 Minutes in a survey, which means that quality will dip into anything longer than 7 Minutes in our experience. However, it is not just length, it is the over-complication of questions, the poor design and the disconnection from where these results are actually going in a mundane experience for all.

• Solution: Without impacting the value of the research, there are a few simple adjustments you can make to improve the respondent experience. There needs to be a drive for clarity on questions, with simplicity at the forefront of the researcher’s mind. After that, add a progress bar, be honest about the length of the survey, and think about how you can make the questions more conversational. Finally, in order to show respondent’s that you care about their opinion, try and close the ‘feedback loop’ by thanking people for their response, warning those who give a bad response, deliver public insight reports from time to time or present the results on social media to show they are being heard.

Point 4: The Incentives & Rewards

• The Problem: There is a fundamental flaw in how researchers incentivise their respondent pool. It is the idea that if we make it as difficult as possible to receive a reward, or redeem an incentive earned by doing surveys, that they will save money and therefore ‘win’. In reality, by offering unachievable gift cards of €20, small discount codes or an entry into competitions, what you are actually doing is disincentivizing consumers from ever participating in your survey again. Suddenly, the ‘goal’ of the incentive to engage has delivered the complete alternative.

• The Solution: The simple ideology that if you invest in your panel, you will earn it back in multiples we have proven with Bounce Insights. We focused on instant gratification, rewards tailored to what respondents told us they wanted, and an ability to purchase and redeem rewards on their phone in a frictionless way. This resulted in a panel with a survey completion rate of 67%, meaning we have saved fortunes on panel management thanks to this philosophy of prioritising the people answering the surveys.

Point 5: Asking Poor Quality Questions

• The Problem: “One way to solve a mystery is by asking the right questions until answers start to emerge.” – Brian Stelter. Asking the right questions is probably the most important aspect of consumer research. Too often, researchers commit to a set of unanswered questions and rigorously seek out answers by any means necessary. The trouble here, is that as humans, we often inherently bias our own interpretations of questions especially those in which we have pre-existing knowledge. 

• The Solution: The key to asking good quality questions is by putting yourself in the shoes of a 7 year old. How simplified, jargon less and comprehensible would your question be if asked to a 7 year old child? Bring this level of empathy to your question design and you will instantly benefit from clearer respondent understanding as well as more thoughtful insights from participants involved.

The ‘research first, respondent second’ attitude has delivered a catastrophic collapse in online survey engagement and reach. It’s time for a fresh approach to research, combining trusted research methodologies with a new understanding and appreciation of the 21st century consumer. By merging the old with the new, we can begin to empower an industry currently settling for mediocrity.