Mixed Mode: Past, Present, and Future
Edith de Leeuw
One of the earliest mentions of mixed-mode in a methodological monograph was by Dillman and Tarnai in 1988. Mixing survey modes appears almost inevitable today, especially for academic research and official statistics. Only now online surveys have almost replaced telephone surveys in the mix. Furthermore, in international comparative surveys, mixed-mode is almost inevitable, as countries differ in survey technology and customs.
Just as in 1988, today there are three important reasons to use a mixed-mode survey design: improving coverage, increasing response rates and reducing costs. However, there are also potential drawbacks, such as, increased administrative and logistic burden, and potential for mode specific measurement error. From a Total Survey Error perspective one wants the best of all worlds and reduce overall error. The steps involved in a high quality mixed mode study are: design, diagnosis, and adjustment. Early methodological publications mainly focused on design (De Leeuw, 2005; Dillman, 2000). During diagnosis it is important to discern between desired differential selection effects, which help reduce coverage and nonresponse error, and unwanted differential measurement effects. Only then one can estimate the unwanted mode measurement effect while controlling for the wanted selection effects, and if necessary adjust for unwanted differential measurement error (e.g., Tourangeau, 2017; Hox et al., 2017).
A new technological challenge facing survey researchers are mobile devices, such as smart phones and tablets, which are increasingly being used to access the Internet. Web surveys are now morphing from a computer-oriented into a multi-device oriented concept, and although a mixed-device survey is not a mixed-mode survey in the traditional sense of the word (all devices are self-administered), the devices used vary widely in screen sizes, data entry interface, and social customs in use. Therefore, the question arises whether or not answers obtained via smartphone and tablets are comparable to answers obtained from pc or laptop, and how to design high quality mixed-device surveys.
In this presentation, I will summarize the state of the art in traditional mixed-mode surveys and discuss implications for mixed device surveys.
Edith Desiree de Leeuw is MOA-professor in survey quality at the Department of Methodology and Statistics at Utrecht University. She was a Fulbright scholar with Don Dillman at Washington State University and visiting scholar with Jan de Leeuw (no relative) at the Program on Social Statistics, UCLA. She is a fellow of the Interuniversities Joint Institute for Psychometrics and Sociometrics (IOPS) in the Netherlands, and was awarded the Visiting International Fellowship at the Institute of Social Research, University of Surrey. Edith has over 140 scholarly publications and is co-editor of 4 internationally renowned books on survey methodology: The International Handbook of Survey Methodology, Total Survey Error in Practice, Advances in Telephone Methodology, and Survey Measurement and Process Quality. Her recent publications focus on design and analysis of mixed-mode studies, online surveys, nonresponse, and total survey error. Edith is currently associate editor for the Journal of Official Statistics (JOS) and editor of MDA (Methods, Data, Analyses); she is also on the editorial and scientific boards of international journals in the field of survey methodology, such as IJPOR, Field Methods, SMR, & BMS.
De Leeuw, E. (2005). To mix or not to mix data collection modes in surveys. Journal of Official Statistics, 21(2): 233-255.
Dillman, D. A. & Tarnai, J. (1988). Administrative issues in mixed mode surveys. In R. M. Groves, P. P. Biemer, L. E. Lyberg, J. T. Massey, W. L. Nicholls II, & J. Waksberg (Eds.), Telephone Survey Methodology (pp. 509-528). New York: John Wiley & Sons.
Dillman, D. A. (2000). Mail and Internet surveys: The Tailored Design Method. New York: John Wiley & Sons.
Hox, J., de Leeuw, E., & Klausch, T. (2017). Mixed-Mode Research: Issues in Design and Analysis. In P. P. Biemer, E. de Leeuw, S. Eckman, B. Edwards, F. Kreuter, L. E. Lyberg, N. C. Tucker, & B. T. West (Eds), Total Survey Error in Practice (pp. 511-530). New York: John Wiley & Sons.
Tourangeau, R. (2017). Mixing Modes: Tradeoffs among Coverage, Nonresponse, and Measurement Error. In P. P. Biemer, E. de Leeuw, S. Eckman, B. Edwards, F. Kreuter, L. E. Lyberg, N. C. Tucker, & B. T. West (Eds), Total Survey Error in Practice (pp.115-132). New York: John Wiley & Sons.