Stéphane Ganassali’s team at the University of Savoie were interested in looking at how survey design affects the people taking the survey. They created eight different surveys to compare specific design elements (long surveys versus short surveys, surveys with illustrations versus surveys without illustrations, and so on). Their findings are interesting because they suggest that wording might have as much of an effect on dropout rates as survey length.
Measuring Survey Length & Language Complexity
It’s important to note that for this study, the short survey had 20 questions and the long survey had 40 questions. In other words, the “long” survey is still much shorter than many established scales. The complexity of the wording was measured by Microsoft Word’s Flesch analysis. On a scale of 0 to 100, with 0 being least readable and 100 being most readable, the complex survey had a readability rating of 40/100 and the simply worded survey had a readability rating of 60/100.
The team found that reducing survey length and using simple, direct language showed the lowest drop-out rates.
Drop-Out Rates Affected by Survey Length and Wording
|Direct Wording||Complex Wording||Total|
However, the results suggest that when using a long survey (in this case, any survey of more than 40 questions), keeping the survey’s directions and questions simple can reduce survey drop-off rates. The next logical question is how to keep survey writing simple. Three techniques can help researchers here:
- Use fewer words. Try to cut as many words as possible from directions, descriptions, and questions.
- Keep words and sentences short and easy to understand. Word and WordPerfect both have built-in readability tools to act as a guide for how complicated your writing is.
- Test the language out. Try giving the survey to one or two people who match the demographics of your group. Once you’ve solicited general responses, try asking about specific directions and questions. Request suggestions for better wording, and leave yourself open to any other advice those testers might want to share.
Simple writing is difficult to craft, and few research programs offer training or guides in this area. As with many skills, though, fieldwork offers a great venue for improvement.
Ganassali, S. (2008). The influence of the design of web survey questionnaires on the quality of responses. Survey Research Methods, 2(1): 21-32.