The survey Academy Series: 3rd Chapter
To get quality data in a survey we must first clearly define what do we want to get: What is the topic we want to study? Once we have defined the topic, what do we actually want to measure? Which are the concepts of interest? Particularly in online surveys, a detailed definition of the aim of the study is crucial, since there is no interviewer to clarify the meaning of a question, or to explain what anything which is not understood.
For example, a seemingly clear question such as “Do you like football?” can have different interpretations to different people. Whereas the majority of young boys will probably understand the question as “Do you like to play football?”, most older women may understand it as “Do you like to watch football?”. If this question was included in a face-to-face survey, the interviewer could clarify that the question refers to “playing football” (assuming the interviewer has more information about what we want to measure). But in an online survey there is no external help, thus the importance of asking questions accurately and in the appropriate way to measure what we really want to study is even greater.
In this example, the research topic is football, but we still have to define more precisely what we want to measure: playing football, watching matches on TV, listening to games on the radio, etc. It is also necessary to understand the difference between the topic (football) and the concept that we want to measure: here “do you like it?” indicates that the concept of interest is a feeling (i.e. enjoyment). On the same topic (football) we would pose a different question in case we wanted to study a concept like evaluation. In such case we could ask: “Do you think playing football is right or wrong?”.
Once we defined a topic and the related concepts to measure, we have to operationalize these concepts into questions. Operationalize means transforming a concept into a question or a set of questions. Following the tips of Saris and Gallhofer (2007), this can – or should – be done in 3 steps (“3-step procedure”):
Distinction between concepts by postulation and concepts by intuition
A concept by postulation is a complex concept which cannot be measured directly with a question. The opposite of a concept by postulation is a concept by intuition, which can be asked straight away through a unique question (Northrop, 1947). For example, if we are interested in people’s attitude towards politics, this is a concept by postulation: it is a complex concept, which can be defined using different concepts by intuition, which in turn can be a feeling, a cognition and a tendency of action.
Therefore, the first step is to determine if our concepts of interest are concepts by postulation or by intuition. If they are concepts by postulation, we have to define them explicitly in terms of various concepts by intuition. For each of these concepts by intuition we can apply, afterwards, the step number 2. In the example of a feeling towards playing football, we already have a concept by intuition.
1. From concepts by intuition to assertions
For each concept by intuition, we make an assertion. This assertion can be made in different ways (see tables, chapter 2 in Saris and Gallhofer, 2007) following the basic linguistic rules and depending on the language. In any case, we always have to favor the most simple and direct ways to ask.
In the example of a feeling towards football, many assertions can be made:
- I like playing football
- It’s nice to play football
- It’s annoying to play football
- Playing football is a thing that I love
It is also possible to make more complex assertions: “it is nice to play football with your friends in the beach when it is sunny”. But in doing this, we are adding information that is going to alter the concept we are actually measuring. We are not measuring a feeling towards football any more, but a feeling towards football under certain conditions. Any unnecessary information has to be excluded.
2. From assertions to requests for an answer
Assertions do not ask for an answer. Therefore, in a survey, we have to transform them a bit so it is clear what we are asking and that we want an answer. This is the step that takes us from the assertion to the request for an answer. This request is not always done as a question with a question mark. An imperative, for example, can also be used (“please, check the box that corresponds…”). Hence, instead of “question” we use the more general “request for an answer”. There are also different ways of making requests for answers from an assertion. For example if the assertion is “I like playing football”, we can transform it into these requests for answers:
- Do you like playing football?
- Please, could you tell us whether you like playing football or not?
- Now we would like to ask you if you like playing football.
- To what extent do you like playing football?
- Indicate whether you agree or disagree with the following statement: “I like playing football.”
Again, the important thing is to use the simplest possible form and stay as close as possible to the initial assertion, avoiding adding any information not needed and/or that may detract us from measuring the concept of interest.
In conclusion, it seems easy to ask questions but we should follow some basic rules:
- Define in advance what exactly it is we want to measure (research topic and concepts)
- Start defining the concepts and then ask the questions step by step.
Otherwise (if the questions are asked directly), there is a high risk that the questions at the end do not match what the researcher really wants to measure, which of course, can affect the obtained results.
Bibliographic references:
Northrop F.S.C. (1947). The Logic of the Sciences and the Humanities. New York: World Publishing Company.Saris, W.E. and I. Gallhofer (2007). Design, Evaluation, and Analysis of Questionnaires for Survey Research. New York: Wiley
