The Survey Academy Series: 1st Chapter
With the rapid move of surveys from offline to online, we need to reevaluate how questionnaires should be designed. Therefore, we've listed 7 golden rules that are a good starting point for creating any online survey.
Migrating questionnaires from “offline” to the Internet has been a massive phenomenon that has not received the methodological attention that it deserves. How should online questionnaires be designed? Should we use the same question formats on the web as we used in face to face or telephone surveys? Is it recommended to use matrix format questions?
This is the first post of a series which aims to share valuable knowledge that can be applied to creating online surveys.
To begin, we will share 7 general tips that will help you create online surveys. The next posts in this series will be more focused on very specific topics.
You may already know these 7 tips. If so, this post will refresh your memory. Otherwise, you can use this as a starting point for everything we are going to discuss in the next 14 posts.
Let’s start!
1. Remember: an online questionnaire is… a questionnaire
This might sound obvious: the Internet has changed everything, from the way we interact with people to the way we do our shopping. However, there are some things that will never change, such as all the good practices from offline questionnaires that can be applied online too. For example:
- Keep a clear and unique purpose throughout the questionnaire.
- If there are many topics to be addressed, it is preferable to make more than just one survey.
- Write the questions in a clear and concise way.
- Ask one thing (and one thing only!) on each question.
- Make sure the response alternatives cover all possible choices (e.g. by adding an “other” option).
2. Respect the online “language”
Internet has developed its own language that users learn while browsing through different websites. Your questionnaire should observe these unwritten rules. This will make it easier for respondents to understand your survey. For example:
- An underlined word is usually a link to another site. Avoid using underlining to emphasize a concept. Use bold letters, different colors, etc.
- Webs are designed upright, and this is more obvious with mobiles. Avoid using horizontal scrolling. The content should be arranged from top to bottom.
- The header and the footer of an online questionnaire page are the right places to place additional items as logos, support links or navigation buttons.
- In response alternatives, use radio buttons to indicate those options are mutually exclusive, and use check-boxes to indicate that it is possible to check more than one option. These are standard rules on the Internet.
3. Be compliant with the GDPR
With the implementation of the GDPR, law developing Privacy Policies and communicating them to the respondent is a must. It is very important that people who are answering your questionnaire are not concerned about the confidentiality of their responses and know that they won’t be used to get advertising. You must also be transparent about how their responses will be used and ensure that the brands will manage the information in a proper way.
By reducing the effect of social desirability (i.e. the will of the respondent to be liked by the interviewer), Internet is especially appropriate for doing research on sensitive topics (health, personal hygiene, sexuality…) but we should offer some guarantees to respondents. (Mavlatova & Couper, 2013).
4. Take advantage of the technological possibilities
You should respect the usual conventions of any questionnaire, either online or offline. But do not just shift your questions from paper to web. The online world has skyrocketed user interactivity. Now, users are willing to give their opinion in new ways, not just answering the classic questions. Add dynamic response questions to your survey by using the most of the opportunities that new technologies offer (pictures, videos, elements that can be dragged on screen…). For example we recently conducted research on how to use images and voice inputs to answer surveys. These tools are likely to improve engagement by reducing the respondent’s burden, particularly among younger people.
Remember that an online survey can be answered from different devices, so you have to make sure it can be correctly displayed on all of them: laptop, tablet, and smartphone. If a question can’t be displayed on a device, let the respondent know about it and provide an alternative.
5. Be brief!
If a questionnaire is short, respondents will pay more attention (Galesic & Bosnjak, 2009). Long lasting surveys have a higher dropout rate (which means more self-selection of participants). The length should not exceed 10-15 minutes, but there are some factors that can influence the time that a respondent is willing to invest in a questionnaire (Mavletova & Couper, 2015):
- The respondents’ interest in the subject of the study.
- The level of reward for participating.
- The quality of the Internet connection of the country that the survey is sent to. In countries with poor connectivity, the respondent will need more time to respond.
- The relationship between who sends the questionnaire and who receives it. The respondent may feel obligated to respond (eg. internal working environment survey to employees).
6. Reward your respondents (properly)
In offline research, rewards in exchange for participation was not very common. The respondent participated based on “empathy” to help the interviewer which was asking for this support (or for being too ashamed to deny their participation).
Internet has worn off the “empathetic” survey value and we must replace it with some other kind of motivation for the respondent. The most common reasons are:
- Fun; the respondent has a good time taking a survey.
- Personal interest; answering the survey can have an indirect positive effect (employee satisfaction surveys).
- Social responsibility; the survey will help improve society (public health surveys).
- Economic; when the participant receive some goods (or money) in exchange for participating.
Most non-commercial research does not cover topics that personally interest the respondent, and they are usually not fun, so the economic reward is mostly used.
Please also note:
- There is a wide variety of economic rewards. Choose the most convenient for the respondent within your capabilities: money, raffles, discounts, donations, etc.
- If your survey is sent to an online panel, they will take care of this problem.
- The rewarding should be proportional to the effort, particularly in regards to the length of the questionnaire.
7. Give thanks!
We often forget that behind every “complete” (complete response valid for the study) there is a real person. It is important that participants feel themselves valued from beginning to end, and understand the usefulness of their work. Some recommendations:
- Give thanks at the beginning and at the end of the questionnaire.
- Check the survey thoroughly before sending it: spelling, images, links, messages, answers, etc.
- Be empathetic. Try to answer your own questionnaire. Make sure you can answer it correctly.
- If you can give feedback to the respondents on their results once completed –although it is not always possible- do it.
In summary, making a good online questionnaire requires a healthy dose of common sense, adapting best practices of traditional questionnaire design to new technologies.
References
Mavletova, A., & Couper, M. P. (2015). A meta-analysis of breakoff rates in mobile web surveys. Mobile research methods: Opportunities and challenges of mobile research methodologies, 81-98.
Mavletova, A., & Couper, M. P. (2013, November). Sensitive topics in PC web and mobile web surveys: is there a difference?. In Survey Research Methods(Vol. 7, No. 3, pp. 191-205).
Galesic, M., & Bosnjak, M. (2009). Effects of questionnaire length on participation and indicators of response quality in a web survey. Public opinion quarterly, 73(2), 349-360.
