Response Rates in the Time of COVID-19

Here at EMI, we sit in a unique seat in the industry – we don’t have our own panel but have built a network of 150+ sample providers from around the globe. We reached out to our top 25 partners to obtain their feedback on response rates they have been experiencing during the Coronavirus pandemic. 

Please note that these results are from a specific point in time (April 3-7) and respondent behaviors may change. 

Overall, most panels are seeing an increase in response rates. This is likely due to most states and even most countries having some form of a “stay at home” rule in place. Respondents likely have more free time in order to take surveys. Here are some additional details and some of the responses our partners provided:

1. Most panels indicate an increase in response rates across the board, regardless of demographics.

“We have seen an uptick in response rates across the board regardless of demographics and tenure.“

“The response rate from the first days has improved after the start of the lockdown. It has been equally steady for demographics and age groups.”     

“We have seen an increase in response rates across the board with both new and tenured members.“

2. Some panels note specific demographic groups have experienced an increase in response rates.

“We’ve seen a 5% increase in the usual response rate for Korea and Japan.  In terms of demographics, we’ve seen the increase more in Females age 20-49.“

“All studies have response rates that are slightly better, except some studies with healthcare professionals have dropped. However, we do not have a significant number of healthcare professionals to provide a significant gauge.“

“This is depending on the geo-target; for example, currently in Italy, response rate is too low for both consumer and B2B studies.However, in other European countries, B2B response rate is very low where consumer response rate is high. In the U.S., B2B response rate is still handsome but depending on geo (i.e., California has very low RR).“

“In the UK and Germany, a slight downward trend is emerging. Entries and respondents are showing a slight drop, completes were down slightly, but this could be leveling off for UK and Germany.“

“Response rate is high, especially among the young generation“

3. A few panels note that there has not been a change in response rates.

“We have seen them hold steady.“

“We have not seen significant changes to response rates so far.“

“We haven’t seen a noticeable difference in any particular studies receiving increased response rate or an increase in members taking more surveys.“

Taking our partner’s responses overall, we found that the increase in response rates is primarily coming from current panelists who are simply taking more surveys than usual rather than a large increase in new panelists, although some panels do see a large increase in new registrations.

A couple of panels identified an interesting trend: there has been a significant increase in surveys taken on laptops or desktops rather than tablets or smartphones, likely due to more people working from home and having access to personal computers. With this in mind, here are a few takeaways and best practices:

• Why do panels experience different response rates? We feel that it is due to the differences in panels at their core.  They all recruit differently (social media, ad networks, phone, etc.), and manage their panels differently.  Therefore, you have different types of members.  We know that panelists on different panels have different attitudes and behaviors (Learn More) and this will come across during times of disruption such as the current time period.
• Monitor the device respondents utilize to take the survey on. We know that those who take a survey on a smartphone differ from those who take surveys on a different device, even when isolating demographic variables. Monitoring this may help researchers understand major changes when comparing data to previous research. 
• As always, demand transparency from your sample partners. Ensure they’re using proprietary panelists, ask lots of questions, etc. to ensure you do not introduce any additional bias into your research.
• Response rates will continue to change over the next few weeks.

Recommendations

It is important that researchers do their best to take advantage of these increased response rates by ensuring a good respondent experience and creating surveys that are:

• Mobile-first and device friendly
• Short – this is the time to decrease LOI
• Adequately quality controlled; respondents should not receive surveys with errors and an extra quality check should be in place
• Designed with an adequate screener to ensure respondents are in the correct survey
• Sent to respondents with ideal profiles, in order to increase incidence rates if possible

While these measures should always be in place, it’s even more important during this time.  We will continue to monitor via our sample providers and will update as changes occur or new information arises.