Big Screens, Shrinking Hype? The 2025 Movie Matchup
May 23, 2025
Intellicast S8E7 – Bots, Buyouts & the Sample Landscape
June 9, 2025
Brand tracking studies are the backbone of strategic marketing decisions for organizations worldwide. Longitudinal studies provide crucial insights into brand health, consumer sentiment, and marketing effectiveness over time. But the value of tracking studies hinges entirely on one critical factor: consistency of sample quality.
After more than two decades of research on research and thousands of tracking studies, EMI has identified six fundamental problems that can destroy the reliability of brand tracking data:
Problem 1: Massive Data Variance Between Sample Providers
The differences between sample providers aren’t minor statistical fluctuations. There are substantial variations that can completely alter strategic conclusions. Our research reveals shocking disparities where awareness levels can vary by as much as 25 percentage points based on your sample provider selection. In our tests, we saw this translate into brand ratings varying by as much as 20 percentage points, and concept ratings can vary by as much as 30 percentage points. A 25-point swing in brand awareness alone could mean the difference between celebrating campaign success and questioning your entire marketing strategy.
Problem 2: Single-Source Sampling Vulnerabilities
When you rely on one panel for tracking studies, you’re putting all your eggs in one basket and becoming vulnerable to that provider’s operational changes, quality fluctuations, merger activities, or business decisions. What happens when your single provider changes recruitment strategy, gets acquired, or experiences technical difficulties? Your tracking continuity is completely at their mercy, leaving you with no control over the consistency of your most important strategic data.
Problem 3: Top-Up Situations and Uncontrolled Bias
When studies don’t reach their target sample size from primary sources, providers often “top up” with whatever sample is available, introducing significant bias that can skew your tracking data. These unplanned additions create artificial spikes or drops that have nothing to do with brand performance, making it impossible to distinguish between real market changes and data collection artifacts. Without proper controls, your tracking study becomes unreliable at the most critical moments when you need additional samples.
Problem 4: Panel Conditioning and Response Bias
Panel conditioning occurs when the same respondents repeatedly take similar market research surveys, potentially influencing their responses over time and creating artificial trends in your tracking data that reflect survey fatigue rather than genuine market shifts. These respondents become “trained” to answer in certain ways, developing response patterns that don’t reflect authentic consumer sentiment. This problem compounds over time, making long-term tracking studies increasingly unreliable as the same respondents become more experienced with similar questionnaires.
Problem 5: Unpredictable Panel Evolution
Panels are not static entities that maintain consistent characteristics over time. While every panel has differences, from how they recruit, how they incentivize their panelist, and how they invite panelists to take surveys, these differences are exacerbated by mergers or acquisitions of panels, back-end operations, and changes in partnerships. All of these differences lead to differences in the overall participant pool a panel has, along with their attitudes and behaviors. Even panels delivering high quality can change without warning due to market conditions, competitive pressures, and internal business decisions, meaning a panel performing consistently today may deliver different results six months from now and vice versa.
Problem 6: Lack of Wave-to-Wave Consistency Standards
Most sample providers don’t have systematic approaches to ensure consistency across tracking waves, treating each wave as an independent data collection rather than part of a longitudinal measurement system. Without proper controls and standardized procedures, each wave of your tracking study becomes a unique experiment rather than a consistent measurement tool. This lack of systematic consistency management means that tracking studies often show variation that reflects methodology changes rather than actual market dynamics.
EMI’s Comprehensive Solution Framework
Understanding these problems is only half the battle – EMI has developed a comprehensive solution framework that systematically addresses each challenge through two integrated approaches: Strategic Sample Blending and IntelliBlend® methodology, supported by technology-enabled consistency management. Together, they create a comprehensive strategy to ensure brand tracker consistency over time and deliver the highest data quality for reliable insights and better brand impact. Here’s how the solution comes together:
Strategic Sample Blending
Strategic sample blending directly tackles the core problems of provider variance, single-source vulnerability, and uncontrolled bias through an approach that goes far beyond traditional sample aggregation. This approach takes traditional blending to the next level and is the best sample design to ensure confident business decisions. It is blending three or more sample providers, but the selection and blending of the selected providers is done in an intentional and controlled manner. Providers are selected to complement one another while reducing the overall sample bias and any potential behavioral or attitudinal impacts a panel can have. This methodology distributes samples across multiple sources, ensuring that changes in any single provider don’t dramatically impact tracking results while eliminating the need for problematic top-up situations. By strategically selecting providers and managing their allocation, you increase overall feasibility while avoiding ‘top-up’ situations and panel bias, which can skew your data.
IntelliBlend®
EMI’s flagship methodology, IntelliBlend®, specifically addresses panel conditioning, evolution challenges, and consistency standards through a sophisticated approach that combines traditional double opt-in research panels with carefully selected non-traditional sources in controlled proportions. “IntelliBlend® is EMI’s premier strategic sample blending methodology, designed to excel in longitudinal research scenarios. This proprietary approach delivers the most representative and accurate demographic, behavioral, and attitudinal data by combining carefully selected sample sources in an intentional and controlled manner.” For tracking studies specifically, IntelliBlend® provides crucial wave-to-wave consistency by implementing systematic procedures that remain constant across all waves. “For ongoing longitudinal work, we replicate the custom sample plan from wave to wave using our advanced sample management platform, SWIFT. This platform enforces sample source quotas and maintains strict quality controls, ensuring data consistency throughout your study’s duration.”
The Business Impact of Solving Consistency Problems
The importance of addressing these six problems extends far beyond statistical significance to real business impact, as inconsistent tracking data leads to misguided strategic decisions with substantial financial consequences. When brand awareness fluctuates dramatically between waves due to sample variability rather than actual market changes, marketing teams may make costly adjustments to campaigns that were actually performing well. In contrast, artificial improvements in brand ratings could lead to premature celebration and reduced marketing investment when continued effort is actually needed. Consider the real-world implications: a false 15-point drop in brand awareness might trigger an expensive campaign pivot. At the same time, inconsistent concept testing results might kill promising product innovations that could have driven significant revenue growth.
Conversely, solving these consistency problems enables confident strategic planning that drives business success. Marketing teams can identify genuine trends, measure campaign effectiveness accurately, and make data-driven decisions about resource allocation and strategic direction without second-guessing whether their data reflects reality or methodology artifacts. This confidence translates directly into better business outcomes, more efficient marketing spend, and strategic decisions based on reliable insights rather than statistical noise.
The Winning Approach
Our comprehensive solution framework works because it addresses the root causes of tracking inconsistency rather than just the symptoms, combining deep industry knowledge with systematic problem-solving methodologies. By understanding why panels differ, how they change over time, and which combinations work best for specific research objectives, we deliver the consistency that makes tracking studies valuable strategic tools. We’ve made significant investments to obtain this unparalleled industry knowledge, which is the driver to providing transparent sample consulting and advice to our clients that puts the emphasis on what is right for their research, and not what is right for any specific panel.
The sample industry continues to evolve with new technologies, regulatory changes, and market dynamics constantly shifting the landscape. Our commitment to research on research ensures that our understanding of panel differences and optimal blending strategies evolves alongside the industry. For organizations investing in brand tracking studies, often representing significant research budgets and critical strategic insights, choosing a sample provider and methodology isn’t just about cost or convenience but about ensuring that the foundation of strategic decision-making remains solid and reliable over time.
Through strategic sample blending, continuous quality monitoring, and deep industry expertise, EMI delivers the consistency that makes tracking studies truly valuable because when your brand’s future depends on understanding change over time, the last thing you want is for your data source to be the variable that’s changing the most. Your tracking studies deserve better than sample roulette – they deserve the consistency that comes from systematic problem-solving, industry expertise, and unwavering commitment to quality that represents the hallmarks of EMI’s approach to longitudinal research excellence.
