Sample Jail – What You Need to Know

If you are reading this, then you have no doubt had a study fall short of the completes you needed.  It’s happened to everyone.  It probably extended your field time, or you had to bring on an additional supplier to close the gap.  This is a regular occurrence in market research, both quantitative and qualitative, and oftentimes the reason for it is a good one. 

Most studies are fielded with a past participation requirement as part of the criteria – meaning that if a respondent has participated in a study on the same (or similar) topic within a certain time frame, they will not be eligible to participate in your study.  I want it to be clear – this is a good thing.  If a respondent just took a survey on the same topic as your study’s objective, it could influence the results of your study.  Clients and sample companies both have different criteria that make up their past participation rules. 

This is where the term “Sample Jail” comes into play.  A respondent is in sample jail when they meet the targeting criteria of your study and would normally get an invite, but they don’t because a past participation rule excludes them.

Again, this is not a bad thing.  Yes, your feasibility could be impacted by this; but for those of you making the business decisions, do you really want respondents who answer surveys on the same topic over and over again to provide you data you are using to make critical decisions?  Probably not.

Sure, you could relax past participation rules to boost feasibility and increase the number of completes, but it will have an impact on your data – particularly with questions regarding awareness and purchase intent (which are obviously very important metrics for brands to track!).

Having said that, it’s actually hard to predict who might be in sample jail at any particular time.  Sample companies are getting bids every day for their panelists to participate in surveys on a wide range of topics (even ones like yours), and any feasibility estimate you get is based on that moment in time.  It can be days, or even sometimes weeks, before a study is officially commissioned and fielded, and a lot can happen in that time.  There could be similar studies fielded, other respondents could be released from sample jail, and other factors that can impact the number of available panelists that were originally available in the bidding stage.

With this in mind, here are a few tips to avoid sample jail:

Understand the past participation criteria of your sample provider

Knowing the criteria they use prior to fielding your study can help you avoid any potential problems during fielding, and reduce any potential impact to your results.

Have a back-up sample plan

If you have strict fielding times or quotas, it is a good idea to have a back-up sample plan just in case you run into any problems while your study is in field.  By lining up a back-up sample provider before a study launches, you save time getting them up and running during your fielding (if they are needed).

Use a sample blend

By using a single sample supplier on your study, you have a much higher chance of having your data impacted by respondents who are in sample jail.  By using a blend of sample providers, you can reduce the potential impact you would have with a single supplier and spread it among multiple suppliers.  Also, you will probably get better feasibility by blending multiple partners!

Want to learn how EMI can help keep you out of sample jail with a custom sample blend?  Click the button below to request a free consultation.