Estimating effects of modes and mixed modes designs 1
|Convenor||Mr Alexandru Cernat (Institute for Social and Economic Research, University of Essex )|
A number of high-quality repeated cross-sectional and longitudinal surveys using ‘traditional’ single mode designs are considering switching to a mixed mode alternative. This has stimulated a large number of studies designed to assess the impact of mixing modes on the estimates produced, and in particular, their comparability with estimates obtained in the original survey design. In this paper we present the results of a mode experiment conducted in the context of the Swiss European Social Survey. We raise questions about the relative importance of mode effects alongside the multitudinous other variables that influence survey data quality.
This presentation compares a mixed mode telephone and online interview to a single mode telephone interview with regard to (1) response rate and (2) measurement error due to social desirability bias. Within a health survey, an experiment was conducted. Individuals were randomly assigned to a single mode telephone and a mixed mode telephone and online survey (respondents choose to participate via telephone or online survey by themselves). In the mixed mode group the response rate was significantly higher, and unit nonresponse error was smaller. Results of regression analysis indicate that social desirability bias was reduced as well.
Using multiple modes to collect data is becoming a standard practice in data collection agencies. While this helps to save costs and may decrease non-response it may have detrimental effects on measurement quality. In this paper we use a quasi-experimental design from the Health and Retirement Study to compare the measurement quality of a number of scales between face to face, telephone and web. Some scales, depression and physical activities, show systematic differences between interviewer administered modes and the self-administered one while others, religiosity and cognitive ability, show inconsistent patterns.
Like the European Social Survey, many survey research programmes are considering the possibility of introducing mixed-mode data collection, generally driven by a need to reduce costs while maintaining a similar standard of quality. Survey programmes also wish to use new technologies because of their flexibility and timeliness. The ESS has been conducting experiments to test the feasibility of implementing mixed-mode data collection designs for over 10 years. This paper summarises findings from 6 studies conducted in 10 countries, involving 4 modes and numerous experiments, highlighting consistent findings, contradictory findings and research gaps.