Tuesday 14th July      Wednesday 15th July      Thursday 16th July      Friday 17th July     

Thursday 16th July, 14:00 - 15:30 Room: O-206

Cognition in surveys 1
Convenor	Dr Bregje Holleman (Utrecht University )
Coordinator 1	Dr Naomi Kamoen (Utrecht University/ Tilburg University)

Session Details

Cognitive research in surveys covers a wide range of approaches. In recent years, various models describing the cognitive processes underlying question answering in standardized surveys have been proposed. A lot of research is guided by the model of question answering by Tourangeau, Rips and Rasinski (2000). This model distinguishes four stages in question answering: (1) comprehension of the question, (2) retrieval of information, (3) deriving a judgment, and (4) formulating a response. In addition, there are dual-process models, such as the satisficing model proposed by Krosnick (1991). In this model, two groups of respondents are distinguished: those who satisfice, and try to do just enough to give a plausible answer versus those who optimize, and do their best to give a good answer.

Cognitive models such as the two described above, have many applications. For example, they help in understanding what is measured when administering surveys, and they provide a point of departure in explaining the wide range of method effects survey researchers observe. Also, cognitive theory in surveys is used by psychologists, linguists and other scholars to obtain a deeper understanding of, for example, language processing, the nature of attitudes, and memory.

Recently, similar cognitive approaches are also used to describe the ways attitudes are formed using standardized surveys. In this type of research, so-called 'decision aids', such as Voting Advice Applications (VAAs), are studied. In VAAs, users answer attitude questions about political issues in order to obtain a voting advice. How do design choices in these decision aids affect users' answers, attitudes and behavioral intentions?

We cordially invite researchers addressing one or more of these topics to submit their papers to this session.

Paper Details

1. Cognitive processes underlying the answers to Voting Advice Applications. Think aloud studies
Dr Bregje Holleman (Utrecht University)
Dr Naomi Kamoen (Utrecht University/ Tilburg University)

When answering attitude questions, respondents roughly go through 4 cognitive stages: they understand the question, retrieve relevant information from LTM, form a judgment, and translate this into a response option. Voting Advice Applications contain about 30 attitude questions, answered voluntarily to obtain a personalized voting advice. The questions are very much like surveys questions, but the motivation to answer them and the context in which they are asked is different. In a series of think-aloud studies we investigated the cognitive processes underlying VAA answers. We focus on the processes of understanding the questions in the tool.

2. What do respondents mean by selecting non-substantive or middle alternatives?
Ms Franziska Gebhard (German Internet Panel, SFB 884, University of Mannheim, Germany)
Professor Annelies G. Blom (German Internet Panel, SFB 884, University of Mannheim, Germany)

Satisficing theory suggests that non-substantive response options like “don’t know” or refusal and middle alternatives attract respondents likely to shortcut the answering process. But are these answers really shortcuts or rather actual attitudes mapped onto these categories?

We present the results of three-by-three experiments, where we tested, if middle alternatives, don’t know options and refusals are used interchangeably by respondents. Our key research questions are: Do respondents use the middle alternative to express a non-attitude, if there is no don’t know option or refusal present? Do respondents distinguish between non-substantive options?

3. Analysis of interviewer-respondent interaction on subjective probability questions
Dr Sunghee Lee (Michigan Program in Survey Methodology)
Ms Colleen Mcclain (Michigan Program in Survey Methodology)

A wide range of political, health, and economic surveys ask respondents to estimate probabilities that various events—from the chance of living to voting in an upcoming election— will occur. Given these items’ cognitively complex nature and predictive power, it is surprising that no studies have focused on the interviewer-respondent interaction surrounding them. To fill this gap, we present results from behavior coding of 2010 Health and Retirement Study recordings. We quantify interviewer behavior, respondent behavior, and response latency at the question level, examining how each relates to response outcomes and exploring differential patterns by various respondent characteristics.

4. Attitude towards surveys as a general predictor of data quality in dual-process models of response behavior
Mr Christoph Giehl (Technische Universität Kaiserslautern)
Professor Jochen Mayerl (Technische Universität Kaiserslautern)

Empirical studies of dual-process models of response behavior show that the groups of responders with slow and quick response-latencies are susceptible for different types of response effects. Additionally, studies show that the generalized attitude towards surveys is also associated with response-latencies. Therefore, we propose that there is a general association between a responder’s attitude towards surveys and the general data quality within the survey. To examine this assumption, we investigate the link between experiences with past surveys and the attitude towards a current survey to explain method effects according to dual-process models.