Sample Industry Research: Why We Do It

I worked in market research for 15 years before I truly realized sample is not a commodity. This is especially surprising when you hear I worked at multiple companies in the past that had panels. We tended to utilize our own internal panel until it was exhausted and then we used outside sample. I always assumed that the only reason outside sample was frowned upon was because of costs. Looking back on those cases where I had to use outside sample (and there were many as those of you that work in sample will understand) the data was different. How different? Were bad decisions subsequently known?  Unfortunately, I won’t ever know.

Fortunately, I now work at a company with a culture of understanding the sample landscape and it has allowed me to design research to understand all the differences in sample providers. We talk about it as “not good or bad, just different.” It makes logical sense that panel companies are different, at times very different, and the data you collect can lead to very contrasting results. Each panel company recruits very differently – some use telephone research, many use social media to recruit, such as Facebook and LinkedIn, others piggyback rewards programs; and many still use in-person recruiting and leverage referrals. They incentivize differently, they send their survey invitations different from each other, and they have varying rules in place for their panelists/members.

In the past, many researchers like me focused on feasibility and pricing rather than focusing on the best, most consistent solution to allow our clients to make the best business decisions they can. 

Throughout 20+ waves of research-on-research that I’ve been able to design, we’ve learned about the differences that exist and have been able to educate our clients on what customized sample solution is ideal for them. We’ve learned that even with strict balancing criteria with demographics (age, gender, income, household size, education, etc.), you’ll still see significant differences in the data depending on which sample partner you work with. We’ve learned that for consistent long-term results, your sample design should be a strategic blend of multiple partners. We know that sample providers change over time and we know how to replace/replicate specific partners if they aren’t meeting your needs or if there are mergers and acquisitions. 

Here at EMI, we work with sample providers from around the globe to find the absolute best sample for our clients and their projects. To best serve our clients, we need to understand the industry, specifically the panel specialties, panel quality, targeting, service, pricing, accuracy, and what differences there are. 

It’s taken us 20 years of focusing on the sample landscape and our 150+ global partner network to identify these differences.  We are constantly doing additional waves of research-on-research, learning more about mobile research, survey invitations, scaling, testing length of interview, data quality, levels of engagement, and much more.  For the first time, we are going to make our findings available to download. Click the button below to download your copy of The Sample Landscape, and make yourself a sample industry expert!