Sample Quality Optimization Rating

Data quality is a vital component of the insights industry. Low-quality data can lead to inaccurate insights, thus creating risky business decisions – potentially costing an organization millions of dollars. At EMI, we are dedicated to leading the industry in data confidence and quality. Our commitment to quality comes from our extensive supplier ecosystem knowledge and our drive to deliver unbiased, actionable, quantitative data tailored to the needs of our clients. Our unique position in the industry has enabled us to be the experts in monitoring and tracking the entire sample landscape, while avoiding the bias that comes with owning a panel. With that in mind, we developed our Sample Quality Optimization Rating. This new metric allows us to continually monitor and evaluate our network and perfectly complements our innovative Data Quality Suite.

What Is EMI's Sample Quality Optimization Rating?

The Sample Quality Optimization Rating is EMI’s proprietary sample quality metric used to evaluate the quality of the sample that we provide to our clients. It was developed to ensure that EMI’s team of expert consultants had access to data-driven insights to craft solid sampling strategies for the most optimal quality experience for both our clients and panels. EMI’s Sample Quality Optimization Rating considers pre-study traffic health, in-study participant behaviors, and post-study data validity consistency. This metric aids in monitoring and tracking over time how well a panel partner performs within Consumer, B2B, and Healthcare work, both domestically and globally.

Ready to get started?

The Science Behind Our Sample Quality Optimization Rating

Our Sample Quality Optimization Rating was developed by our Quality Council comprised of team members representing our executive, consulting, insights, and operations experts encompassing 70+ years of market research experience. As a starting point, we dove into our unparalleled research-on-research that we have been conducting for over a decade, providing us thousands of data points based on millions of responses. Next, we compared data quality trends from over 12,000 completed projects, 100+ sample suppliers, and nearly 40 million survey attempts across 200+ countries. With vast, rich insights from this data, we crafted a formula that considers variables contributing to pre-study removals, in-study removals, and post-study removals. Each of these vital quality metrics are then weighted to generate a quantifiable quality scoring system.

Pre-Study Removals
The first metric that our Sample Quality Optimization Rating considers is pre-study removals. Over 40 variables monitored by SWIFT, our proprietary sample management platform, are considered in the pre-survey flags. Some of these variables include geo-location checks, captcha, ghost completes, and fraud threat potential tracked by our licensed software. While this measure isn't apparent to most sample buyers, it's incredibly important to monitor and evaluate sample providers based upon how many respondents are blocked prior to reaching a client’s survey.
In-Study Removals
The second metric measured in our Sample Quality Optimization Rating is in-study removals. For this measure, we work closely with our customers to advise on screener and questionnaire design to ensure adequate real-time engagement checks are embedded within the research study. In-study removals include variables like red herring and attention check fails.
Post-Study Removals
The final metric that impacts our Sample Quality Optimization Rating is poor data quality reconciliations. Post-study data reviews are conducted by our Research Managers, the client, and our suppliers. Via human analysis and artificial intelligence, a variety of data quality checks are considered in the data cleaning process.
Previous
Next

SWIFT And Our Sample Quality Optimization Rating - A 1-2 Combo For Data Quality

SWIFT, EMI’s proprietary sample management platform, is an integral component of gathering the required data to calculate our Sample Quality Optimization Rating. SWIFT’s built-in quality module allows us to customize our utilization of several industry-leading quality assurance tools to detect and eliminate more fraud, bots, and duplications than other sample providers. The data SWIFT collects around an individual study feeds directly into our Sample Quality Optimization Rating calculation, allowing for near real-time updating.

The SWIFT sample management platform features the best-in-class techniques built to combat quality challenges and keep up with the ever-evolving data quality landscape. We are constantly exploring and integrating the leading methods and technology into SWIFT to catch and prevent potential fraud while increasing data reliability and validity, allowing EMI to set the standard for data quality in the market research industry.

EMI’s unique position in the sample industry makes the combination of our Sample Quality Optimization Rating and SWIFT so powerful in measuring sample quality. Our strategic partnerships with a large network of online sample suppliers promote our top-down view when it comes to quality at the panel level.  

SWIFT's Quality Module Capabilities

Industry-leading digital fingerprinting and de-duplication

Fully integrated with Research Defender’s advanced bot and fraud technology

Next-gen CAPTCHA

Geo-IP block

Features Of EMI's Sample Quality Optimization Rating

  • Factors 40+ different fraud and duplication markers across pre-study, in-study, and post-study
  • Views at an overall level, by panel, device, geography, client, sample type, and any combination with the ability to drill down for additional detail
  • View at a holistic level, right down to the project level
  • Ability to look at specific time periods, down to a 24-hour period
  • Ability to calibrate metrics by country and vertical to determine a panel’s strengths and areas of improvement

Ready to get started?

How We Use Our Sample Quality Optimization Rating

We use our Sample Quality Optimization Rating in many ways, most prevalently in the maintenance of our panel partner network and strategic sample planning.

It is an integral part of how we manage our extensive Partner Network. In addition, our Network Manager and Quality Council utilize our Sample Quality Optimization Rating to track and evaluate the quality of the data each panel contributes to our research studies over time.  

Once a sample partner has passed our rigorous Panel Partner Assessment process and has been admitted into our network, we leverage the Sample Quality Optimization Rating to provide insights into the quality of the sample they are providing and we leverage this data along with our Strategic Sample Blending and IntelliBlend offerings to ensure stable, reliable, quality sample plans.  

Ongoing, we closely monitor each measure of our Sample Quality Optimization Rating, and we take different actions with our clients and partners depending on which metric bucket is having the most impact on reducing the metrics below our optimal thresholds. For instance, for partners whose pre-study metrics are reducing their rating, we work with the panel on improving the accuracy of targeting, share fraudulent trends and provide consultation on ways to improve the overall health of their database and recruitment practices. Our Sample Quality Optimization Rating also helps us provide data-driven consultation to our clients to ensure adequate screener practices are in place, in-study checks are implemented and consistent quality review methods are employed. It allows us to improve every step of the research experience from the respondent level all the way to the end client.

We use our Sample Quality Optimization Rating internally when building our custom strategic sample blends. In addition to behaviors, attitudes, specialties, feasibility, etc., our sample consultants use it as one of the many components they look at when crafting a custom strategic sample blend for a client and their study.

Our Sample Quality Optimization Rating aids in our unparalleled knowledge of quality trends across sample panels in EMI’s partner network. as well as among our customers. In addition, our internal dashboard allows for deep dives into the quality by panel, customer, device type, sample type, country, and more.

EMI is committed to being the most trusted, passionate research partner bringing our values to life every day as we serve our clients, community, and each other. Our team of online sample experts believes in providing our clients with the highest levels of data reliability and quality so that they can better trust their business insights. Our Sample Quality Optimization Rating, combined with our Data Quality Suite, and SWIFT, provides our clients with a powerhouse of tools to ensure their data quality and trust in their insights.

"I have had a data review on the projects we did together this year. And there has been a tremendous improvement in the data quality we have been providing on EMI projects with fewer rejections in comparison to the projects we have run in the past.

As per our discussion earlier, we have not only strengthened our internal security measures, but we have also embedded our tool with Research Defender as well. The combination of both is helping us gain the highest quality of responses. Other than that, we can customize the threshold score in the RD tool that stops the concerning respondents from accessing the survey.
Inquisitor Media Analysis and Data Research