Tuesday 14th July      Wednesday 15th July      Thursday 16th July      Friday 17th July     

Wednesday 15th July, 09:00 - 10:30 Room: L-102

Experimental designs in online survey research 4
Convenor	Mr Henning Silber (Göttingen University )
Coordinator 1	Mr Jan Karem Hoehne (Göttingen University)
Coordinator 2	Professor Dagmar Krebs (Giessen University)

Session Details

Experimental studies have become increasingly popular in survey research and are carried out in various disciplines such as sociology, political science, linguistics, economics and psychology. In survey research experimental designs are useful tools to get a better understanding of cognitive processes in order to give better practice advice for improving study and questionnaire design. In particular, the technological advances have made it significantly easier to use experimental designs in online field experiments as well as in computerized laboratory experiments.

This session invites presentations on empirical studies and theoretical discussions of experimental designs in online survey research.

- Empirical online research can include studies on response behavior and social desirability bias, as well as experiments on response rates and question design effects. Furthermore, we especially encourage presentations with replicated experimental results and welcome replications in different social contexts such as different cultural, educational and ethnic groups.
- Additionally, we invite presentations that discuss the value of experiments from a theoretical perspective. Theoretical presentations could contrast the merits and the limits of different forms of experimental study designs or provide a future outlook on the prospects of online experiments in survey research.

Presentations could cover the following research areas:

- Theory of experimental study designs
- Replication of experimental results
- Comparisons between different experimental designs (e. g., laboratory and field experiment)
- Split-ballot experiments (e. g., context effects, question order, response order, acquiescence, visual design effects, verbal effects)
- Choice experiments
- Laboratory experiments on response behavior (e. g., using eye tracking)
- Experiments with incentives
- Vignette studies
- Future prospects of experimental designs

Paper Details

1. Recruiting processes in German Companies: How are foreign diplomas currently valued? – A Factorial Survey Design –
Mrs Alexandra Mergener (Researcher; Federal Institute for Vocational Education and Training (BIBB))
Mr Tobias Maier (Researcher; Federal Institute for Vocational Education and Training (BIBB))
Dr Robert Helmrich (Head of Division; Federal Institute for Vocational Education and Training (BIBB))

This paper intents to analyse recruiting strategies and skill requirements of German firms in different industries using a Factorial Survey Design. The focus in on the labour market chances of immigrants who obtained their qualifying degree abroad. We introduce a vignette study in an online-survey to present decision makers in companies different recruiting situations. The relevance of specific characteristics of applicants is judged by varying those characteristics in the vignettes. Therefore we can control in what way sex, nationality, language skills, vocational degree, and work experiences influence the hiring chances and inform about possible discriminations in the recruitment processes.

2. Method effects in factorial surveys: Investigating effects of vignette construction and presentation
Dr Carsten Sauer (Bielefeld University)
Professor Katrin Auspurg (Goethe University Frankfurt)

Factorial survey methods combine the advantages of experiments and surveys. We investigate method effects that can invalidate results, including range-effects (span of values of dimensions), number-of-level effects (low/high number of levels per dimension), presentation style effects (table vs. running text), ceiling effects (extreme vignettes first vs. random allocation), and answering-scale effects (rating vs. metric scale). Based on an online survey with 2,500 university students and experimental variation of design features (between subjects) we are able to provide recommendations on how to achieve more valid and precise estimates of the causal impact of experimental factors.

3. Mixed-mode experiment - evaluation of effects on data quality, response rates and cost reduction
Mrs Mikaela Järnbert (Statistician)
Mr Johan Eklund (Statisticial)

Over recent decades, response rates in general surveys have decreased rapidly in Sweden as in many other countries. This has led to higher uncertainty in estimates as well as higher costs of data collection. One way to deal with this problem is to allow other response modes. In September 2014, Statistics Sweden carried out an experiment with mixed-mode (telephone interviews and web questionnaires) within the Swedish Party Preference Survey. In the evaluation we use data set of Party Preference Survey together register data and para data from the data collection (from telephone system as well as web system).

4. How do implicit attitudes affect the perception of candidates in TV debates during the campaign: Evidence from Austria
Dr Kathrin Thomas (University of Vienna)
Dr David Johann (Univeristy of Vienna)

What is the role of implicit attitudes in citizens’ perceptions of candidates in TV debates? Assuming that implicit and explicit attitudes co-exist, this paper tests the impact of both on the evaluation of the FPÖ frontrunner’s performance in the TV debates. The analysis relies on an online panel survey collected by the Austrian National Election Study in 2013. It includes semantic differentials to measure respondents’ perceptions of candidate performance and a Single Category Implicit Association Test to capture their implicit attitudes towards the FPÖ. The findings confirm that implicit attitudes matter for the perception of candidates.