Respondent Quality – We are in this together!
July 23, 2015EMI – Over 6 million surveyed
January 14, 2016Like most researchers, you’ve spent time investigating the differences in survey data from one wave to the next. You’ve reviewed demographic, attitudinal, and behavioral data to see why these differences exist. Keeping your sample as consistent as possible from wave to wave is ideal, but can be challenging in our world with low incidence studies, a need for higher base sizes, hard to find targets, etc.
At EMI, we know that another source of differences in data can be attributed to the actual sample source (a.k.a. sample bias). We’ve been conducting our research-on-research for over a decade to better understand how sample sources differ. Although targeting and balancing is replicated across all sample partners, there are significant differences in the data for all sources. The chart below illustrates some of the differences we saw testing various concepts. You’ll notice that the top box concept rating is very different based upon the sample partner you select.
There are a host of reasons sample can look different from source to source and even wave-to-wave with a single source such as:
- Recruitment approach
- Opt-in process and verification
- Incentive structure (points, cash, etc.)
- Sample source composition
- Panel management (tenure, profiling, targeting, a community or just visitors, etc.)
- Sample delivery mechanism (direct email, router, prescreen)
- And many more
One way of reducing sample bias is to strategically blend multiple sample sources. At EMI, we have been strategically blending sample for more than 20 years and our IntelliBlend® blending product provides a much more controlled approach to sampling. We help identify which studies benefit and will put together a comprehensive sample plan utilizing multiple sample sources that ensure consistency. In this way, you can feel more confident about comparing results from one test to another. We are also able to help you transition your existing sample plan to a better, more realistic long-term option.
Learn more about IntelliBlend® on our website!