Investigating Survey Fieldwork Processes: Interviewers and Their Strategies
|Convenor||Dr Wojciech Jablonski (University of Lodz )|
In this paper we set out a new method for the analysis of interviewer variance; the mixed-effects location scale model extends the random part of a standard 2-level mixed effects model by introducing random effects on the level 1 variance (Leckie, 2014). This enables response variance to be re-parameterized in a way that affords a more flexible and causally focused assessment of the factors associated with interviewer-induced response variability. We apply this approach to data from wave 3 of the UK Household Longitudinal Survey (UKHLS), which we link to a diverse range of interviewer characteristics.
In some survey research settings it may be not attainable to interview individual respondents without involving bystanders or ‘third-parties’ in the interview. Due to complex living circumstances or group culture, respondents may be ‘helped’ by others in answering questions. Involvement of ‘third parties’ raises questions on data quality and poses challenges to the data collection process. Recognizing this, a natural field experiment was performed in the Philippines that allowed for spontaneous “third party help” during parts of the interview. Results showed that “third party help” did not negatively affect data quality, but rather improved it for most issues.
Ensuring that interviewers work in a standardised way is key to achieving high quality survey data. Survey researchers use various techniques both during interviewer training and fieldwork to keep the research process standardised, but these techniques are constantly evolving and will depend on the particular requirements of the study. This paper reports on innovations in the standardised interviewer training that was designed for the sixth wave of the UK Millennium Cohort Study (MCS), the interviewer accreditation process, and the fieldwork quality and monitoring procedures that have been put into place.
In an attempt to reduce the cost of data collection while maintaining data quality, the National Ambulatory Medical Care Survey (NAMCS) staff reviewed the fieldwork paradata and designed an experiment to test new data collection procedures in 2015. The testing protocol requires interviewers to first attempt to contact the physician’s office over the telephone, thereby reducing the number of personal visits required to complete the CAPI interview. This paper discusses the analytics supporting the protocol design, and the procedures used to monitor and evaluate interviewers’ compliance. Monitoring interviewers’ strategies enables the detection of deviation from the testing protocol.
Past studies generally applied three main strategies to identify interviewer effects: analyzing the influence of observable interviewer characteristics on responses to related questions, estimating intra-interviewer correlation, and using data quality indicators. We propose a novel strategy based upon longitudinal data on respondents and interviewers from the Socio-Economic Panel Study (SOEP). We argue that changes in responses accompanying a change in interviewer serve as an indicator of interviewer effects. To evaluate the new approach, we address differences between identification strategies and discuss their respective (dis-)advantages and peculiarities.