How much do you know about your sample provider?

How much do you know about your sample provider?

If online research were a biome, it would be the Great Barrier Reef, with more inhabitants than can be counted—some strong and beautiful, some bottom feeders, and frankly some that are in danger of becoming extinct.

We spend a lot of time vetting sample sources to decide with whom we’d like to work and when they would be a good fit in terms of quality, feasibility, and price. With new sample sourcing surfacing frequently, it is astounding to see the diversity of sample sources that go through our internal panel partner assessment process, and that come and go through ongoing pruning of our network.

The bottom line is that not all sample sources are created equal. But we already know this, of course. None of us would work with a partner in whom we weren’t confident. How is the industry screening sample partners? What questions are we asking to ensure that we’re working with a lean, mean sampling shark and not a clownfish?

At EMI, here are a few key questions we ask when we evaluate potential sample partners:

Are they prescreening? 

Prescreening not only allows your sample partner to be more efficient with their respondents, but it can also reduce costs and time-in-field due to the increased conversion rates.   On the other hand, prescreening means that the sample you are pulling from is not representative of the full population. This can affect incidence rates and survey respondent composition, so it is very important to understand not only if your partners are prescreening, but on what questions.

What type of targeting is available?

We want our sample partners to be able to target exactly the respondents we are looking for. How much have you discussed targeting with your partner and at what stage of the project? If you do B2B research, your partner should have expertise in profiling to reach your target group.  If you sell infant car seats, can the panel you’re working with target only moms in general, or can they pinpoint moms with children 0-6 months old? Having this discussion early on helps you design the screener and proper quotas to develop a more accurate incidence rate assumption and cost estimate with your provider.

What level of service do they provide?

It’s important to know who will be “on call” for your study. Will there be someone there to contact when your quota stops are exceeded or when IR is coming in much lower than you expected? If so, who is it going to be picking up the phone? What is the normal bid turnaround time and is there a deadline for launching the same day? Are they available after-hours when you’re reviewing completion rates?  While you can’t know exactly the level of service provided until you are already engaged, it’s crucial to discuss service capabilities with your provider so that proper expectations are met.

How are respondents being recruited / invited to the survey?

Where respondents come from can impact data quality at multiple stages. What types of sources were used to recruit the respondent to initially joint the panel? Was it through email invitations, banner ads, co-registration, web publishers or even offline methods? How does the respondent make it to your specific survey? Is it an individual email invite, a notification when logged in to the panel or community, or via router / prescreening technology? Ultimately, the most vital aspect in the survey world is the actual respondent, so it is important that they are treated well throughout the entire process from registration to ongoing survey completion.

How are respondents being incentivized? 

In what ways are respondents being enticed to participate in surveys? How much of a typical CPI is incentive? What incentive amount is offered for different target groups like general consumers or small business owners or heart surgeons?   Whether your sample supplier is incentivizing respondents with entries into a sweepstakes, gift cards to various retailers, or in cold, hard cash can make a difference in response rates and general respondent engagement.

There are plenty of fish in this sea of sample and we encourage you to ask these and many more questions of your sample providers to ensure you are getting the best catch possible. If you are not satisfied with the answers you get or don’t have time to go deep sea fishing, then please give EMI a call.

Have questions about sample sourcing and how we can help? Get in touch with EMI Research Solutions at [email protected] or call us at 1 (866) 661-7075.