The Truth Behind Consumer Research Panels

Introduction

When it comes to panel quality in the consumer research industry, there is a much deeper issue at play. Why should panel suppliers change when their clients keep accepting the status quo on quality while simultaneously demanding lower costs? According to Ron Sellers, there are two defining truths in consumer research today:

1. Panel quality is objectively awful
2. It is not the panel suppliers fault, it is the clients

Let me explain this further. In order for panel quality to improve, suppliers need to invest in new technologies and implement a panel management system that removes those respondents who don’t meet quality standards. However, there is no incentive within the industry to do this, so why would they? By increasing their overheads through investment and quality assurance practices, they further decrease their margins in a ‘race-to-the-bottom’.

Simply put, consumers have become commodities in the eyes of the researcher. In order to change this, it must become the goal of the client to demand transparency, reward investments in new technologies and ultimately pay more for increased quality, just like any other industry.

Expectation vs. Reality in Market Research

• Expectation: Any respondents who are cheating, straightlining, speeding, guessing or lying will be removed from these panels and not invited into further online surveys.

• Reality: Panel providers cannot afford to remove respondents who are engaging in surveys from their panel, particularly if no one is holding them accountable. Thus, instead of quality improving over time, it has continued to plummet.

• Expectation: Clients will strive for the highest quality research possible in order to make the best informed business decisions.

• Reality: They must settle for mediocrity. With engagement rates below 3%, and the average screening out for poor quality at 10%, the final results for any research carried out today is so far removed from real consumer opinion that it is simply incomparable to any other industry. Where else would we accept a 10% chance of defect in a product or service?

• Expectation: With increased access to technology and consumers all around the world, it must be better than ever to carry out fast, effective and high quality research.
• Reality: The constant pressure for lower costs and faster results, along with the repeated acceptance of poor quality, leads to what we have today: a whole lot of bad data being used, and a constant, time-wasting, soul-draining fight to cull valid responses out of a morass of awful data.

The Path to Improvement

Unless clients start demanding better, and being willing to pay for demonstrably improved quality, no incentive to improve exists and we’ll continue to be stuck where we are today:  lamenting the poor quality of panel samples while simultaneously pressuring companies to get their costs down, yet scratching our heads as to why nothing is improving.

Researchers must demand transparency and incentivising higher quality from panel suppliers. It is the objective of the researcher to communicate to clients the truth behind consumer panels today, and why you must invest in higher quality rather than settle for mediocrity. By lifting the benchmark, the expectation will become the reality in years to come.