How to Deal with "Don't know" and Other Non-Response Codes in Online and Mixed-Mode Surveys 1 |
|
| Coordinator 1 | Mr Tim Hanson (Kantar Public) |
Researchers have long debated the treatment of “Don’t know” codes in surveys. Some have argued that a “Don’t know” response is often an expression of satisficing and that the explicit inclusion of this code may not improve data quality (e.g. Krosnick, 2000, 2002). Others have argued that an explicit “Don’t know” option is necessary as there are times when information asked of respondents is unknown; by not offering a clear “Don’t know” option there is a risk of collecting “non-attitudes” (Converse, 1976).
The treatment of “Don’t know” codes has increased in importance with the movement of surveys online, often as part of mixed-mode designs. In an interviewer-administered context the interviewer can code a “Don’t know” response where it is offered spontaneously. This approach cannot be replicated in a self-completion setting (including online surveys), meaning alternative approaches are required, which can impact on the way people respond. Therefore, developing best practice across modes for how to present “Don’t know” and other response options that have traditionally been coded only where spontaneously offered by respondents is becoming an increasing need within survey research.
A range of approaches have been used to deal with “Don’t know” and other non-response codes in online surveys. These include: (1) displaying these codes as part of the main response list so they are always available to respondents; (2) hiding these codes from the initial response list with instructions for respondents on how to select them (e.g. on a hidden ‘second screen’ that can be generated should none of the initial responses fit); and (3) removing these codes altogether from some or all survey questions. All three approaches have potential flaws in terms of comparability with other modes and risks of satisficing behaviours, reporting of non-attitudes and lower data quality. Currently there is no clear consensus among the survey research community over the best approach to take.
We welcome papers that have used different approaches for dealing with “Don’t know” and other non-response codes for online and mixed-mode surveys. Papers that include quantitative experiments or user testing to compare different treatment of these codes are particularly encouraged.