Direction of Response Scales
|Convenor||Dr Florian Keusch (University of Michigan )|
|Coordinator 1||Professor Ting Yan (University of Michigan)|
The diversity in types of response scales in survey items on political attitudes raises questions about the inter-survey comparability. In this paper I present results of a survey experiment with response scales applied in question-items on trust in political institutions: the national parliament, political parties, and justice system, focusing on two features: scale length (number of scale points) and direction (negative to positive and vice versa). Exploration of these effects and errors contributes to the development of survey theory, as well as facilitates analysis based on combined (harmonized) survey data.
Previous research found that different scale directions do affect response distributions. This effect is usually attributed to scale direction as another case of response order effect caused by satisficing. Drawing on data from two individual experiments with members of a nonprobability online panel and students, we found that different scale direction has a significant and extreme impact on the mean ratings. However, respondents who were more prone to satisficing did not necessarily show stronger scale direction effects. Scale direction effects are observed across the board among those who are prone to satisficing and those who are not.
Our study examined response order effects across a number of online surveys with over 500,000 total respondents, examining the impact how the number of responses, dimension assessed, and response orientation. We generally found that vertically displayed scales showed higher levels of primacy effects, and those scales measuring evaluative aspects were more likely to show the primacy effect than intensity aspects.
Scale direction effect is understudied. An important gap in the literature is the lack of attention to the moderating effects of scale- and question-level characteristics on scale direction effects. To fill this gap, we conducted an experiment varying question type, scale direction, scale length, and scale labeling. This study is the first to experimentally manipulate scale features. It will allow us to tease apart the moderating effects of scale and question feature on scale direction effects and to provide guidance on direction of scales.
This study aims at examining whether the effect of the direction of the response scale (from negative to positive or from positive to negative) varies between modes (face-to-face and internet). Using data from the American National Election Study, we answer this research question focusing on attitudinal items with a 5-point scale. We find that depending on the mode the direction of the scale impacts on response. Moreover, we report findings that take into account respondent's background and we conclude with a discussion on the implications of the findings on survey design.