ESRA 2019 Draft Programme at a Glance

The future of face-to-face fieldwork in a multi-source and multi-mode data environment.

Session Organiser Ms Gerry Nicolaas (NatCen Social Research)
TimeTuesday 16th July, 16:00 - 17:00
Room D22

Over the last couple of decades, the survey research industry has witnessed a shift from face-to-face interviewing to mixed mode data collection, including the use of web alongside face-to-face interviewing and other offline modes. More recently, there is also increasing pressure to use other data sources such as administrative records and transactional data. Furthermore, the industry continues to be challenged by the rising cost of face-to-face interviewing and declining response rates. The panellists will be invited to discuss the impact that these developments are having on the practice of face-to-face fieldwork, how interviewers are being trained and rewarded in light of these changes, and other issues related to the sustainability of a pool of face-to-face interviewers in this multi-source and multi-mode data environment.

Five panellists will be recruited among field directors and survey researchers/methodologists from a selection of national statistical institutes and survey agencies. Expressions of interest have been received from NatCen Social Research, the UK Office for National Statistics and a market research company. These and other leads will be pursued if this proposal for a panel discussion is approved.

Keywords: face-to-face interviewers,

Understanding mode switching and non-response patterns

Dr Alexandru Cernat (University of Manchester) - Presenting Author

The shift from single mode designs to mixed mode, where a combination of interview modes are used to collect data, is one of the key challenges of contemporary longitudinal studies. The decision about what modes to use and how to implement them can have a big impact, influencing costs, field-work procedures, non-response and measurement error.
The research proposed here aims to better understand one of the key characteristics of a mixed mode design: how people transition from one mode to another in time. This is essential for a number of reasons. Firstly, it can inform targeting strategies. For example, it can be used to target those people that are more likely to shift from a cheap mode to a more expensive one. It can also be used in models for dealing with non-response after data collection, such as weighting models. Thirdly, it can be used to explain measurement error that appears due to the mode design.
This paper will investigate the process of changing modes by looking at waves 5-10 of the UKHLS Innovation Panel where a sequential Web-Face to face design was used. Latent class analysis will be used to find the underlying patterns of change in time of modes. The clusters found will be used both as dependent variables, to understand who are the types of respondents in each, and as independent variables, to predict future wave non-response and mode selection. Findings will inform the design and use of the main UKHLS study.

Do low-response rate online surveys provide better quality data than high response rate face-to-face designs? Separating sample selection from measurement effects

Mr Eliud Kibuchi (University of Southampton) - Presenting Author
Professor Patrick Sturgis (University of Southampton)
Professor Gabriele Durrant (University of Southampton)
Dr Olga Maslovskaya (University of Southampton)
Dr Joel Williams (Kantar Public)

Face-to-face interviews are held as “gold standard” method in terms of data quality which all other modes of data collection are compared to. However, declining response rates, increased survey requests and costs have promoted many surveys to switch to online administration. The available evidence on data quality between face-to-face and online surveys is mixed. Effective evaluation of data quality between the two modes is further complicated by selection effects that are confounded with measurement effects. This study applies propensity score matching to online probability and face-to-face surveys as a means of separating the measurement and sample selection components of the mode difference. In addition, we evaluate different methods of handling survey weights in propensity score matching analyses under a complex survey design. The results show measurement effects contribute the bulk of mode differences between the online and face-to-face samples before and after matching. However, propensity score matching had only a minimal effect on the magnitude of mode effects for the surveys considered. This was true even when both surveys were conducted online, which suggests that propensity score matching cannot be assumed to remove selection differences between surveys

Mixed-Mode Design (MIMOD) in Official Statistics: Where are we going?

Mrs Karen Blanke (Federal Statistical Office, Germany) - Presenting Author

There is an ongoing discussion on the future of data collection in social surveys in Official Statistics. The debate is rather complex, as there are several perspectives to be taken into account: Firstly, the general question, if we still need any data collection, as data are possibly available by other sources and technics (e.g. administrative data, big data, sensors). Secondly, if surveys are still considered to be as relevant, by which mode(s). Thirdly, If Online-First is to be taken seriously, which devices are of main impact. Finally, If Smartphones and possibly Apps are of main relevance, can we still apply our survey designs as adopted in the past.
Partly, these different topics are investigated in the ESSnet project Mixed Mode Designs in Social Surveys (MIMOD), conducted in 2018/2019, coordinated by ISTAT in partnership with CBS (Netherlands), SSB (Norway), STAT (Austria) and Destatis (Germany). MIMOD aims at supporting NSIs in facing challenges when implementing multi-mode and multi-devices data collection within social surveys at the European Statistical System (ESS). While five work packages cover the entire topics, one work package is addressing the use of smartphones for social surveys. The latest will be captured by this presentation. There have been several tasks undertaken to address the issue: a set of fitness criteria have been developed to assess the social surveys to be fit for smartphones, stylesheets are provided to show best practices on the design on smartphones. Conclusions and further reflections on the future of social surveys within the scope of Social Surveys in the ESS shall open the floor for discussion.