All time references are in CEST
Challenges and Consequences of Switching Face-to-Face Population Surveys to Self-Completion (Mixed-Mode) Designs 1
|Session Organisers|| Dr Joss Roßmann (GESIS - Leibniz Institute for the Social Sciences, Germany)
Mr Michael Blohm (GESIS - Leibniz Institute for the Social Sciences, Germany)
Ms Caroline Hahn (GESIS - Leibniz Institute for the Social Sciences, Germany)
Dr Oshrat Hochman (GESIS - Leibniz Institute for the Social Sciences, Germany)
Dr Jan-Lucas Schanze (GESIS - Leibniz Institute for the Social Sciences, Germany)
|Time||Thursday 20 July, 09:00 - 10:30|
The public health regulations that were introduced across Europe (and beyond) since Spring 2020 due to the Corona pandemic impeded the realization of personal face-to-face survey data collection or made it even temporarily impossible – potentially with long ranging consequences for this data collection method. For this reason, surveys of the general population, or important subgroups of the population such as voters, chose to switch from face-to-face interviewing to a mix of self-completion survey modes to secure survey responses from all groups of the target population (e.g., respondents without Internet access or those unwilling to respond in a certain mode).
The Corona pandemic, thus, strongly accelerated a continuing trend of moving surveys from interviewer-administration to self-completion modes. With respect to the general trend and the sustained consequences of the Coronavirus in particular, we welcome submissions that present insights on the methodological and practical challenges of switching population surveys from face-to-face to (mixes of) self-completion modes, and on the effects of doing so on the quality of the data.
Contributions may cover but are not limited to the following research topics:
- Approaches to overcome methodological and practical challenges of switching modes (e.g., regarding sampling and coverage)
- Conception and implementation of self-completion mixed-mode designs for general population surveys (e.g., adaptations in questionnaire design)
- Experimental testing of survey designs to maintain or enhance response and quality of measurement (e.g., use of conditional and unconditional incentives, split questionnaire designs)
- Consequences of switches to self-completion on data quality and total survey error
- Implications of mixing self-completion data collection modes on measurement invariance and possible remedies
- Impact of switching data collection modes on time series in repeated cross-section and panel population surveys
Keywords: Self-completion mixed-mode surveys, mode switch, total survey error, data quality
Ms Katie O'Doherty (NORC at the University of Chicago) - Presenting Author
Ms Anna Wiencrot (NORC at the University of Chicago)
Mr Stephen Smith (NORC at the University of Chicago)
Dr Colm O'Muircheartaigh (University of Chicago)
The National Social Life, Health, and Aging Project (NSHAP) is a longitudinal, population-based study of older adults. It has been conducted face-to-face since its first round in 2005. For the most recent round, NSHAP was developing ways for self-collection of a complex questionnaire and biomeasures to reduce costs, allow for participation from respondents who prefer remote modes, and to evaluate the potential of a hybrid remote/face-to-face approach for future work. Due to the pandemic, the need to adapt data collection from face-to-face to remote was strongly accelerated.
In the first three rounds of NSHAP, an interviewer visited the respondent’s home to administer a questionnaire collecting data on their social network, cognition, physical health, and other aspects of wellbeing. Interviewers also collected biological specimens, anthropometrics, and physical and sensory measures.
To collect questionnaire data remotely, NSHAP adapted the instrument to be self-administered by web or paper, or interviewer-administered by phone. For biomeasure collection, NSHAP developed a kit of supplies and instructions for respondents to self-collect their own biomeasures and send back for analysis. Approximately half of the NSHAP respondents were eligible for remote data collection in 2021. The other half, as well as non-interview respondents were approached face-to-face in 2022-2023.
We will discuss the challenges of balancing methodological rigor and minimizing respondent burden when adapting a lengthy, complicated longitudinal survey for remote administration. We examine overall data quality and item-level nonresponse comparing data from remote and face-to-face data collection. Our results show the promise of self-administration for collecting complex questionnaire data and biomeasures. These findings provide valuable lessons learned and context for future work given the need for remote data collection is likely to increase in importance after the pandemic.
Dr Alberto Vitalini (Istat) - Presenting Author
Dr Simona Ballabio (Istat)
Dr Flavio Verrecchia (Istat)
Over the past decades, technological innovations and the development of Computer Assisted (Ca) techniques and the growing digital readiness of households -of which the Covid pandemic has been an accelerator- have led to the spread of new modes of interviewing and self-compilation via the Web. Starting in 2010, the Italian National Institute of Statistics (Istat) started a process of progressive extension of Ca techniques. A strong impetus has been given to the use of mixed mode techniques, partly in response to the need for cost containment, improved coverage of statistical surveys and, at the same time, reduced burden on respondents. Among the others, the Cawi technique (self-compilation of web-based questionnaires by respondents) assumes specific relevance although the propensity for self- compilation of web-based questionnaires by households does not seem to be homogeneously distributed in different territorial contexts and, at the macro level, seems to be related to certain contextual and socio-demographic characteristics. Using data of Istat's Permanent Population Census in the Lombardy context, the aim is to investigate which variables at the macro level have an impact on the rates of self-compilation of web-based questionnaires. The municipality is used as the unit of analysis. The study of macro-variables related to the propensity for self-compilation of the questionnaire is particularly useful, starting from the design stages, both in defining the right mix of channels to use and in deploying strategies to incentivize compilation in territorial realities characterized by a low propensity to use the Web channel. A fortiori, in implementing surveys with an exclusively Cawi data collection technique, it would be advisable to take this aspect into account even at the estimation and sampling stages, by planning, for example, for sample overrepresentation of territorial
realities with a lower probability of participation.
Dr Joss Roßmann (GESIS - Leibniz Institute for the Social Sciences) - Presenting Author
In 2021, the German Longitudinal Election Study (GLES) decided to switch the data collection mode of its pre-election survey from face-to-face interviewing to a self-completion mixed-mode design with web and mail questionnaires due to the Corona pandemic. As the fieldwork period of the survey was ultimately limited by the day of the federal election on September 26, a particular challenge was implementing a fieldwork protocol that promised to meet two objectives: First, achieving a desirable distribution of interviews across the fieldwork period, which included collecting substantive numbers of responses within the two weeks before the election; and second, facilitating overall survey response. Two survey design features were implemented to tackle these challenges: First, the sample for the survey was randomly split into three equally sized tranches, which were fielded at intervals of about a week within the four-week fieldwork period. Second, while all invited sample members received an unconditional 5 Euro prepaid incentive, 80 percent of the sample were randomly assigned to be offered a delayed conditional 10 Euro incentive for their timely participation with the second reminder letter.
The present contribution investigates the effects of the two design features on survey response and data quality in the pre-election survey of the GLES 2021. Preliminary results show that fielding the sample tranches at intervals of about a week facilitated the desired spreading of interviews across the fieldwork period. Thereby, a considerable number of interviews were collected in the two weeks before the election. Offering a delayed conditional 10 Euro incentive to reluctant respondents with the second reminder letter increased the response rate by more than two percentage points in each sample tranche. The final analyses will include more detailed assessments of the effects of the design features on sample composition and data quality.