According to the Pew Research Centre, the number of bad respondents for a given online panel ranged between 4-7%. In most industries, this would be considered abysmal, but considering falling engagement, weak quality checks and the rapid technology adoption by respondents, this makes a lot of sense. However, this research only omitted the most obvious poor responses i.e. duplicate accounts, those outside the country of the research, gibberish or contradictory answers.
The reality is, when Grey Matter carried out a more intensive quality assessment, disregarding the obvious conflict of interest in panel companies investigating their own quality control, they found that a shocking 46% of respondents had to be omitted for low quality. This was if a respondent committed four or more issues, which identifies just how much panel suppliers are under-analysing their output for clients.
The purpose of this article is not to defame the online panel suppliers, nor is to take a superior higher ground as a new entrant. Quality control is incredibly difficult, it requires a huge investment in technology and a meticulous strategy for pursuing data that you can trust. This is not helped by the fact that panels have become commodised, with no incentive to deliver higher quality.
That being said, researchers are simply not doing enough to address these challenges, ignorance is certainly not bliss. Using over 40,000 interviews per month through a series of online panels, Harmon Research found that:
Every insights methodology has challenges and quality issues. Good researchers recognize this and do everything they can to reduce the problems and mitigate the challenges without throwing out the methodology. Yet, online panels have dropped the ball, and it’s time to accept the fact that there is a deep rooted problem in the industry.
Unfortunately, these ‘uncaught’ respondents have damaging knock-on effects for the end-client. They fundamentally distort key business decisions, misinform strategic choices, and can lead to the demise of many companies across the world. The only thing worse than waste is being misled, and there has never been a greater need for transparency in the online panel industry.
Do you want to rely on a brand awareness study showing familiarity that’s three times as high as it should be? Or a channel study showing a social media platform is 188% more popular than it really is? How about a consumer spending study that reveals spending to be 16X higher than in the real world. Unfortunately, this is the truth behind online panels around the world.
The solution is to make the best of a bad situation, and start implementing the following:
Ultimately, what you don’t know can most certainly hurt you. As an industry, it is essential that we tackle these problems head on. We must work together to demand more of both panel suppliers and research buyers to incentivise the time, money and effort necessary to solve these widespread issues.
Cookie | Duration | Description |
---|---|---|
cookielawinfo-checkbox-advertisement | 1 year | Records user consent for cookies in the advertisement category |
cookielawinfo-checkbox-analytics | 11 months | Records user consent for cookies in the analytics category |
cookielawinfo-checkbox-functional | 11 months | Records user consent for cookies in the functional category |
cookielawinfo-checkbox-necessary | 11 months | Records user consent for cookies in the necessary category |
cookielawinfo-checkbox-others | 11 months | Records user consent for cookies in the other/unclassified category |
cookielawinfo-checkbox-performance | 11 months | Records user consent for cookies in the performance category |
CookieLawInfoConsent | 1 year | Records the default button state of the corresponding category & the status of CCPA. It works only in coordination with the primary cookie. |
elementor | never | This cookie is used by the website's WordPress theme. It allows the website owner to implement or change the website's content in real-time. |
viewed_cookie_policy | 11 months | The cookie is set by the CookieYes plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data. |
_GRECAPTCHA | 5 months 27 days | Provides spam protection and distinguishes between humans and bots. |
__hssrc | session | This cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session. |
Cookie | Duration | Description |
---|---|---|
bcookie | 2 years | LinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser ID. |
lang | session | This cookie is used to store the language preferences of a user to serve up content in that stored language the next time user visit the website. |
lidc | 1 day | LinkedIn sets the lidc cookie to facilitate data center selection. |
__cf_bm | 30 minutes | This cookie, set by Cloudflare, is used to support Cloudflare Bot Management. |
__hssc | 30 minutes | HubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie. |
Cookie | Duration | Description |
---|---|---|
AnalyticsSyncHistory | 1 month | Used to store information about the time a sync with the lms_analytics cookie took place for users |
hubspotutk | 1 year 24 days | This cookie is used by HubSpot to keep track of the visitors to the website. This cookie is passed to Hubspot on form submission and used when deduplicating contacts. |
_ga | 2 years | Calculates visitor, session and campaign data and also keeps track of site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognize unique visitors. |
_gat_gtag_UA_138017392_1 | 1 minute | Google uses this cookie to distinguish users. |
_gat_UA-138017392-1 | 1 minute | A variation of the _gat cookie set by Google Analytics and Google Tag Manager to allow website owners to track visitor behaviour and measure site performance. The pattern element in the name contains the unique identity number of the account or website it relates to. |
_ga_RFDCRNP6J3 | 2 years | This cookie is installed by Google Analytics. |
_gcl_au | 3 months | Provided by Google Tag Manager to experiment advertisement efficiency of websites using their services. |
_gid | 1 day | Installed by Google Analytics, _gid cookie stores information on how visitors use a website, while also creating an analytics report of the website's performance. Some of the data that are collected include the number of visitors, their source, and the pages they visit anonymously. |
__gads | 1 year 24 days | The __gads cookie, set by Google, is stored under DoubleClick domain and tracks the number of times users see an advert, measures the success of the campaign and calculates its revenue. This cookie can only be read from the domain they are set on and will not track any data while browsing through other sites. |
__hstc | 1 year 24 days | This is the main cookie set by Hubspot, for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session). |
Cookie | Duration | Description |
---|---|---|
bscookie | 2 years | This cookie is a browser ID cookie set by Linked share Buttons and ad tags. |
fr | 3 months | Facebook sets this cookie to show relevant advertisements to users by tracking user behaviour across the web, on sites that have Facebook pixel or Facebook social plugin. |
GoogleAdServingTest | session | This cookie is used to determine what ads have been shown to the website visitor. |
IDE | 1 year 24 days | Google DoubleClick IDE cookies are used to store information about how the user uses the website to present them with relevant ads and according to the user profile. |
test_cookie | 15 minutes | Used to check if the user's browser supports cookies. |
UserMatchHistory | 1 month | To provide ad delivery or retargeting |
_fbp | 3 months | This cookie is set by Facebook to display advertisements when either on Facebook or on a digital platform powered by Facebook advertising, after visiting the website. |
Cookie | Duration | Description |
---|---|---|
intercom-id-gz33jl8i | 8 months 26 days 1 hour | No description |
intercom-session-gz33jl8i | 7 days | No description |
li_gc | 2 years | Used to store consent of guests regarding the use of cookies for non-essential purposes |
__gpi | 1 year 24 days | No description |