Tuesday 16th July
Wednesday 17th July
Thursday 18th July
Friday 19th July
Download the conference book
Problems and perspectives of piloting and measuring attitudes in survey research |
|
| Convenor | Dr Tilo Beckers (Heinrich-Heine-Universitaet Duesseldorf) |
| Coordinator 1 | Dr Pascal Siegers (Heinrich-Heine-Universitaet Duesseldorf) |
| Coordinator 2 | Dr Eva Jaspers (Universiteit Utrecht) |
85 years after Thurstone's seminal article social scientists continue to measure attitudes and those who do so are convinced that "Attitudes can be measured" (Thurstone 1928). But many practical
problems remain and new perspectives have been developed. This session addresses researchers working on (new) attitude measures. We are particularly interested in papers using specific research
designs (e.g. vignettes, randomized response, implicit association tests, cognitive interviewing, recall and projective methods), scaling properties and techniques (e.g. Guttman, Rasch) as well as techniques of analysis (e.g. CFA, LCA) to further develop and improve attitude measures or to test the reliability of survey attitudes (Roberts/Jowell 2008; Jowell et al. 2007; Alwin/Krosnick 2001; Sirken/Herrmann/Schaechter 1999; Krebs/Schmidt 1993). Empirical reports about projects on question testing and piloting of attitude measures are particularly welcome. This session is thus strongly interested in practical aspects of measurement in concrete research projects. Papers may address relevant substantive aspects but should have a strong focus on methodological innovations. The scope of the papers may be national, subnational as well as cross-national/cross-cultural.
Occasionally, researchers need to adminster, to large samples of people, attitude questionnaires which are very long. However, when the instruments are very long, the respondents are less willing to participate and they often respond carelessly or skip questions. Response rates fall and survey costs rise, and non-response is one of the factors behind rising costs. One idea to reduce the length of the instrument, is to administer only a part of the questionnaire to each individual. That way, each individual will only experience a part of the questionnaire (thus a shorter completion time) but we will collect information on all questions. Assuming that there is an appropriate link (usually called "anchor" in educational and psychological test development literature) between the different sub-questionnaires, we should be able to reconstruct almost all the information as if we had administered the full instrument to everybody.
This study uses the concept of structurally incomplete designs (planned missingness) in order to investigate the necessary conditions under which we may achieve maximum information and best possible linkage between the sub-questionnaires. For this purpose, we use two empirical datasets: one very short attitudes scale and one very long. We use Item Response Theory as well as other imputation techniques to investigate the dgree to which the statistical method used may affect the results. We also investigate the various psychometric properties of the "anchors" in orer to achieve the most optimal link between the sub-questionnaires.
Question design presents challenges for survey researchers, especially when it involves topics that respondents seldom consider, or of which they have little knowledge. Since its inception, the European Social Survey (ESS) has aimed to include a broad range of topics and has included some that many respondents think about rarely, if at all. One of the modules selected for Round 6 of the ESS focuses on attitudes towards democracy and is an example of such a topic. The module comprises a set of questions measuring 'understanding of democracy' and a corresponding set of questions focusing on 'evaluation of democracy'. The questions were developed in collaboration with a team of academics led by Professor Hanspeter Kriesi from the University of Zurich.
In addition to the challenges encountered in asking respondents about complex, unfamiliar concepts, the module also presented 'structural' design challenges. To try and overcome these difficulties, a variety of qualitative and quantitative pre-testing techniques were employed during the development of the module, including expert review, cognitive interviewing, SQP coding, omnibus surveys (involving a split-ballot ordering experiment) and a large scale quantitative two nation pilot study involving MTMM experiments.
This paper outlines the conceptual and practical difficulties experienced in designing suitable questions to measure respondents' understanding and evaluation of democracy. In particular, it explores how the different pre-testing methods were used and how the results informed the subsequent development of the content and structure of the module.
In current literature on prejudice there is a growing trend to analyze and treat the prejudice as a one-dimensional syndrome of generalized devaluation of out-groups. In current study we analyze the detailed structure of this construct. Specific aim of the study is to disentangle the effects of attitudes towards different ethnic group and specific forms of prejudice on responses in prejudice surveys. The main question of the analysis is what shapes specific responses in surveys on inter-group relations? Do they result rather from generalized attitudes towards specific ethnic groups (e.g. Jews, Germans) or from specific prejudice (e.g. fear, social distance) aimed at different social groups?
The analysis is based on Polish Prejudice Survey 2009 database. The survey contained questions measuring different dimensions of prejudice: stereotype content model measures, social distance, realistic and symbolic threat, inter-group emotions, inter-group behavioral intensions, measures of emotional infra-humanization, perceived similarity and group entitavity. They were used to measure attitudes towards 7 different ethnic groups.
The fact that the center category of the LeftRightScale is by far the most popular response to this stimulus, is a finding which is valid in nearly all western societies irrespective of the survey mode. On the other hand it is known, that many of these responses are not driven by a "center" ideological orientation but express ideological indifference - concerning other ideological indicators or voting behavior as well. So how to differ between those who choose ideologically and the factual "don't knows"? On the basis of a representative CATISurvey, we show that there is in fact a very simple way to clear the center category from opinions that are not ideology-related: (1) Giving a Left-Right-Scale as a stimulus that has a real center category. (2) Asking those who have chosen the center category, what they would choose in a situation in which they are forced to decide. (3) Using the information from this second question to transform the original response into a left-right-measure that accounts for the responded tendency (which is expressed by one third of the former "center" respondents). The new created left-right-measure as a whole shows a significant reduction of the center category choice and a slight enhancement concerning explained variance of several variables. A possible improvement of the LeftRightScale is an important issue facing the significance of this measure in Public Opinion Research: in Western Europe it is the most common single item survey question to examine ideological orientation.