Introduction
Since starting a research company, Bounce Insights, 5+ years ago, I’ve been amazed at how disappointed most insights buyers are by self-service market research tools. If technology is meant to make researcher’s lives easier, the evidence I have seen is that we have failed them so far. After hundreds of conversations with brands and researchers, this is my attempt at explaining why companies are struggling with self-service market research tools, and how we can recapture the hearts of researchers.
How did self-service tools come about?
The holy trinity of disruption in any industry is delivering something that is faster, cheaper and better. Typically, when choosing a supplier, you can only pick two of these three decision criteria. In the research industry, the trade-off for buyers is clear: agencies are great, but slow and expensive. Self-service tools are fast and cheap, but lack the quality of execution and necessary guidance in many cases. In consumer-centric, data-driven organisations, all key decisions should be backed by research. For all important decisions, quality cannot be compromised. The saying “no one ever got fired for using IBM” rings true here. Like many industries, technology promised to innovate and disrupt. Self-service tools entered the space to support decisions where speed was important, where methodologies could be productised or when high-level consultancy wasn’t required. This drove more research being done, which on paper, is a good thing. However, the challenge arises when teams optimise too much for speed and low cost. This causes an expectancy within the business to run more research and get high-quality data to back up decisions. In recent years, this has led to bad research because expectations of what self-service tools could achieve have not been the reality. This disappointment in self-service tools can be explained in two parts. For research and insights teams, they promised speed for users, but added more tasks into their already-busy jobs. For senior decision makers, they promised low-cost and democratisation of insights, but delivered reduced quality of output and decisions. To be clear, self-service tools have achieved disruption where the research being implemented is homogenous. The problem is this only covers a component of the research process, and the industry still relies on a large degree of human judgement and expertise that takes years to develop (which in turn is why agencies remain slow and expensive).
So, where have self-service tools gone wrong?
1. They view research teams as musicians, not conductors:
There is a common misunderstanding of how the research function operates, particularly in large companies. The individuals in this team are like a conductor of an orchestra. They act as partners within their organisation, working to understand the business needs so they can marry the best research solutions. The mistake is that self-service tools viewed researchers as musicians performing the score on individual instruments. Almost every researcher within a brand, small or large, that I have spoken to has connected with this pain point. When self-service tools came into these organisations, there was an expectation that you could standardise and streamline the various research methodologies, and focus on execution via an easy-to-use tool. This would drive cost efficiencies and democratisation of insights across the organisation. The reality is that every piece of research starts with a business question or information gap, and ends with a decision. Even knowing what question to ask or information gap you’re trying to fill is difficult, often varied and not conducive to templated methodologies or do-it-yourself scripting tools. Research is inherently non-homogeneous. Researchers are used to collaborating with agencies, working with people they trust that are flexible on the methodology required or problem being faced. This is a very different proposition to performing each stage of the process yourself. To use the same analogy, It’s hard to be a master of the violin and the piano, while continuing to conduct other parts of the orchestra. Self-service tools failed to recognise the behavioural and skillset shift required for researchers to move away from this conductor-model and into a lead-musician role for each tool.
2. They don’t understand the square-peg, round-hole pricing problem:
One of the most consistent problems I have seen is the misalignment between the business models of self-service tools, and the way in which research budgets work. Most of these self-service tools demand a recurring revenue model. For those that are venture-backed, their valuations are built on them. For larger agencies launching self-service tools, they wanted a piece of this action, whether that’s driven from private equity or the pursuit of higher margins, the end result is the same. This means upfront contracts, auto-renewals and strict commercial terms that their sales team must abide by. The problem is, research budgets don’t work that way. Typically, these teams have global, long-term commitments with agencies that lock in the majority of their budgets. All other money is allocated with great care to reactive/short term needs, briefs and challenges. This has gotten even tougher to manage for research teams as economic conditions have worsened, causing budgets to shrink and margins to get even tighter in a world that has been in flux in recent years. The word ‘subscription’ causes an allergic reaction for many research buyers. The rigid, forward-looking and ruthless business model of self-service tools, built for software-as-a-service companies, does not align with research teams that require flexibility, reactivity and support to solve their problems on constrained budget and time. Here’s an example as to why I believe technology companies need to adapt. The author, Seth Godin, also likes to teach people how to juggle (literally, not figuratively). The way he teaches is quite counterintuitive. He asks the student to simply focus on throwing and letting the balls drop for their first session. Why? Because for any adoption of a new thing to happen, you must master it one step at a time. If you nail the throw, the catch looks after itself. For self-service tools trying to force a behavioural and skillset change for researchers, trying to achieve this alongside enforcing a budget adaptation is a step too far in my mind.
3. They focus on questions, not answers:
Finally, I believe that self-service tools focus too much on the questions, and not on the answers to the business problems they are actually trying to solve for their clients. Einstein’s approach to problem-solving paints this idea well, “If I had an hour to solve a problem I'd spend 55 minutes thinking about the problem and five minutes thinking about solutions.” The point he makes is important: being clear on what problem you’re trying to solve makes the solving of that problem relatively straightforward. In research, the critical step is trying to understand what business challenge you’re trying to solve. Once you know that, and how you will do it, good execution is all you need from there. Self-service tools help answer the questions, once you overcome the most important part of the process yourself, which isn’t what researchers need a lot of the time. There is also the stumbling block of when the data is gathered, and there is little guidance on analysis or providing the solution to the business challenge posed. Researchers are problem solvers. For a company to successfully solve their problem, it requires collaboration, context and clarity on what they are trying to achieve and why. To date, self-service tools haven’t supported this part of the journey well enough, which I believe is a grave error.
What does the future look like for self-service?
Humans are terrible at predicting the future, so I won’t try. As per Morgan Housel’s new book ‘Same as Ever’, I feel we’re better off focusing on the things that won’t change rather than the things that will if we are to be ready for the future. I’m going to dive into this in more detail in future posts, but here are some final thoughts on the things that I believe won’t change that are relevant to this discussion:
- Good outcomes are driven by good decisions. Good decisions are driven by data and insights. Good data and insights are driven by good research.
- Technology and innovation isn’t going to go away, true disruption of an industry = speed, efficiency and quality.
- Humans are remarkable, but they don’t like change. Technologists must have deep empathy and understanding for the people involved if they are to enact any significant behavioural change, in any industry.
