Respondents Are Not Inventory

The purpose of the market research industry is to deliver insights that are used to make better, more informed, business decisions. These insights all start somewhere though, and that somewhere is “sample,” and what is often considered to be “inventory” in our industry.

In the primary online quantitative realm of market research, inventory has become pretty commoditized. Coming from someone with a degree and experience in both Accounting and Finance, this is pretty much expected. As technology improves and businesses learn through experience, they become more efficient and naturally cut costs. Who wouldn’t want that, right?

The Dilemma

The trouble we run into when viewing sample the same as other inventory is that we end up cutting costs in ways we probably wouldn’t in other industries. This is because, unlike other industries, sample isn’t just a data point and can’t be treated as such. You don’t have anything until, ideally, a fully engaged respondent participates in your survey

So, essentially the inventory is an engaged respondent-a human being. When you cut costs on sample, respondents are incentivized less than what they feel their time is worth. Once their pay starts getting cut, the desire to cheat, speed, and otherwise game the system increases. Most respondents aren’t paid well enough for their time to begin with in my opinion, but cutting their incentive further almost always increases the number of removals due to poor engagement.

Another aspect that is often overlooked is non-response bias. A respondent could log into their dashboard or click on a survey invite, view the incentive, and decide it’s not worth their time. Assuming a portion of people feel they deserve a higher incentive and don’t take your survey, you’ll be missing out on a portion of your data that attitudinally and behaviorally are different. And if the respondents are different because of incentive structures, it’s likely they are different in other ways, like how they make purchasing decisions. This is where blending panels with different incentive structures can minimize bias and be great for census representation. Regardless, squeezing incentive from respondents will squeeze the quality of insights, and we should make sure to consider the respondent’s experience as they’re at the core of our insights.

Graphs!

It can be a mess to grasp at times, but the easiest way to look at incentives for me is to view it as a supply and demand curve; optimal incentive is where supply (engaged time of a respondent) meets demand (need for quality interviews).  As shown below, if we demand a higher engagement level, the incentive will need to increase.

The length of the survey, the incidence, and the activities required of the respondent would also require an incentive increase; however, engagement can also be improved through surveys lead to positive experiences for the respondent and panels that properly manage their respondents. A panel that understands the value of its respondent’s time can and will provide more engaged respondents, better data, and overall higher-quality insights. Paired with a researcher who has an appreciation for and designs surveys to be positive experiences for the respondent, will certainly deliver more engaged respondents.

Solutions

All in all, looking at respondents like they’re a part of a normal inventory chain, capable of cutting costs for better profit or to fit budgets, is one of the challenges our industry faces. The solution isn’t to just pay a respondent a higher incentive. I’ve had a handful of clients in just the past couple of months ask about tripling or even quadrupling incentive, and unfortunately, additional incentive can only help with feasibility and engagement so much. At a certain level of incentive, respondents are as engaged as they will ever be. Plus, there are always folks out there willing to do anything to receive an incentive for less work (by cheating your survey).

Thinking a bit outside of the box, what if the MR industry had a regulating body focused entirely on respondent experience? Outside of just PII collection, respondents could be limited to seeing a certain number of studies and termination pages in a day. A minimum incentive per minute could be put in place where (in the same way minimum wage exists) respondents would be guaranteed a certain incentive for a minute of their time. Plenty of businesses would face challenges from such a thing of course, but for a good cause. After all, the insights we reach through research influence decisions that brands, firms, and companies around the world make and do end up influencing what marketing, products, etc. make it in front of consumers across the globe. I believe we should have more firm policies in place when it comes to something that influential. What are your thoughts?