What are the common survey mistakes to avoid?

There are many common survey mistakes when designing surveys. Here are a few to consider when designing your next survey.

Forget to include demographic questions

Demographic questions can be very important for analysis of the survey results. If you don't include them you may not be able to filter and analyse your survey results as effectively as you wish. For example, if you wish to analyze the perceptions of different age groups then you must include a question with age group options. Or, maybe in an employee survey you need to differentiate the perceptions of different employee levels or roles. If you don't ask for those details then you won't be able to filter results by the different levels or role groups.

Fail to properly introduce the survey

In order to engage respondents and make them want to complete he survey, you should ensure that the survey is properly introduced. Creating a clear context for the survey is very important.

Fail to pilot the survey

Always pilot the survey. A pilot is an opportunity to get feedback on the communications, inbstructions, survey questions and the scales used along with the survey structure and usability on the devices likely to be used. Based on the pilot feedback, you can then refine the survey and the communications and address any issues identified.

Send the survey to a large group without starting with a small group

Wherever possible, don’t send the survey to a large number of respondents without first sending it to a smaller group. Confirm that everything is fine with the smaller group before sending to the large group.

Ask irrelevant or unnecessary questions

Don’t add questions that you don’t need to ask. Don’t throw in a bit extra to the survey, just because you can. Stick with questions that support your survey goal. This is one of the most common survey mistakes.

Introduce bias

If there is an answer that you’re hoping to get, you can bias your questions so that you get it. Leading questions is the easiest way to do this. An example would be, “How happy are you with our fantastic service?” In that question, you tell the respondent that the service is fantastic. A better approach would be to ask about the service and then use a rating scale that allows the respondent to make an assessment of the service level.

Create survey fatigue and tire out your respondents

Have you come across the term ‘survey fatigue’? Ways to do this include:

• Ask too many questions
• Put a lot of questions on every page
• Ask long questions
• Ask confusing questions
• Ask overly complex questions

Piloting the survey will give you good information as to whether or not the survey is tiring.

Poor Communication

When respondents don’t understand survey questions, it will make the survey data worthless. Don’t use complicated language that is ambiguous or full of jargon. Poor communication is another one of the very common survey mistakes.

Assume respondents know more than they do

Don’t leave room for ambiguity and don’t rely on prior knowledge. Also, don’t ask respondents to remember things from some time ago.

Ask two questions in one

Make sure your questions contain mutually exclusive ideas.

Ask too many open-ended questions

Choose open ended questions wisely and don’t over-use them. Open-ended questions are time consuming. They also involve much work analyzing the responses. So, be prudent in asking open-ended questions

Lack of attention to detal

Pay attention to the detail in the survey. Poor grammar and spelling errors are unprofessional. Details like this send a message about the importance of the survey. Small details in grammar can also affect the vailidity of the data.

Inadequate response options

Make sure that the response options cover the needs of the target group of respondents. It can be very frustrating to be forced to make a selection and not have a relevant choice.

Use too many different rating scales

Using many different scales has two draw backs. First, it becomes difficult and time consuming for respondents to answer the survey. Second, interpreting the results becomes difficult. A standard rating scale allows easier comparison of responses across questions.

Use too many response options

In multiple response questions, don’t have too many response options. Choosing answers from long lists is diificult and can lead to respondents not taking the time to respond accurately. So, keep the response options reasonable.

Use the wrong page structure for the type of survey and the audience

Carefully consider the length of the survey, the type of audience and the device that the respondents are likely to be using when they complete the survey. Then design the structure to suit. For example, long surveys can be very frustrating if only one question is placed on each page. On the other hand, placing multiple open ended quetions on one page creates the risk of respondents losing what they have entered if they are interrupted and they have not saved their responses.

Force responses to every question

Do not overuse the mandatory questions. Use them sparingly. You may be forcing people to make a selection that they don’t want to make and thereby invalidating the data. Forcing responses can be very frustrating. And, if someone does not really want to answer a question then they may just choose any response so they can move on.

Not labelling rating scale steps

We prefer not to use long rating scales like a 10 point scale. Where possible, each point in a rating scale should be labeled to give meaning (albeit subjective) to the scale points. A 10 point scale with Highly Unlikely at one end and Highly Likely at the other end can be frustrating. And, when there are no labels for the options 2 to 9 in between then we question the point. Five point scales and seven point scales are preferable. Not alabelling rating scales is one of the common mistakes aspparent in many surveys.

Not providing a Not Applicable or Don’t Know option

Consider the audience and the questions being asked. In many cases, you should include an option for the respondent if they do not have experience with the question being asked. It is often unreasonable to expect that respondents have relevant experience to respond to every question in the survey. So, consider including a Don’t Know or Not Applicable option in the rating scale.

Use this resource as a checklist to avoid the common survey mistakes.

Contact Spark Chart if you need help to make sure your survey project is a success. Or sign up for the Spark Chart survey software to make the process easy. Here is a summary of Spark Chart survey software features.