Designing Surveys: 13 Tips To Avoid Common Mistakes – Part 1

Designing the perfect survey is like baking the perfect birthday cake, it takes both science and creativity.  You may think that you’ve phrased questions that were easy to understand, only to have others look at them and be totally lost.  You don’t want to waste the time and resources of your team or your respondents on a flawed survey that will give you bad data.  Luckily, we have 13 tips on mistakes to avoid when you’re developing your next questionnaire.

Here are the first six tips:

1. Spend extra time designing the screener

While most researchers focus on the main questionnaire, design errors in the screener can raise dropout rates and decrease data quality.

Our recommendation is to always write the screener as if you’re sampling to the general population.  Sample companies have strong targeting, but it isn’t perfect.  Including a question for all your screening criteria will ensure that the right people enter your survey, bettering the respondent experience as well as give you peace of mind about your data quality.

2. Ensure you have a data quality check plan prior to launching.

While we always wish we could have asked questions after the fact, spend some time ensuring you’ll have enough information to determine if the respondent is truly engaged and if they should be removed.  Some ways to do this include:

• Ensure open-ended responses are included, even in the screener if possible. This is the best method to identify a quality respondent
• Consider adding in a trap question/red herring or reverse scaling questions. This can help screen out bots or other fraudulent respondents.

3. Optimize Your Survey For Mobile

An ever-increasing percentage of respondents are taking surveys from their mobile devices, whether it’s their phone or tablet, so you need to take this into account in your design.  Here are a couple tips when designing for mobile:

• Check your survey on multiple devices – it can render differently on each device-type.
• Avoid lengthy questions or large grids – they require too much scrolling on smaller screens.

4. Avoid Leading Questions

Leading questions uses question phrasing to create perceptions in the respondents’ mind to “lead” them to respond a certain way. This leads to responses that are biased towards a specific response.  To avoid making your questions leading, avoid adjectives or adverbs that can lead to a respondent to, either directly or indirectly, believe any answer should be chosen over another.

5. Don’t Overuse of Yes/No Questions

Yes/No questions have their place in a survey, but if you are looking to collect deeper information such as opinion-based or if your survey is high stakes—you’ll be making large business decisions based on the results – you need to utilize other question types.

6. Use the Right Question Type

Just like Yes/No questions have a place in a survey, so do other types of questions.  You want to make sure you use the right type of question for the type of information you are trying to gather.  These can include multiple choice, check boxes, matrix, open-ended questions, or even other advanced question types.  An example is if you want to respondents to choose from a list of options, use multiple choice, but if you have a question where there could be multiple answers, use check boxes.  Be sure to understand questions types and when it is best to use them.

Be sure to check out the second part of this blog series for the last 7 tips.  Want the full list now?  Download our eBook, Getting The Best Results: Best Practices in Survey Design.