Longitudinal studies in times of COVID-19 |
|
| Session Organiser | Dr Alessandra Gaia (University of Milano-Bicocca) |
| Time | Friday 16 July, 16:45 - 18:00 |
In response to the Covid-19 pandemic, governments in many countries in the world have adopted lockdown measures which caused important disruptions to existing longitudinal surveys. This section is aimed at exploring two main topics. Firstly, the session includes contributions on the adaptation of ongoing longitudinal surveys to the lockdown policies. For example, the session discusses: the rapid adaptation of existing face-to-face surveys to other modes of data collection; the transition from in-person interviewer training to training delivered through virtual platforms; the development of Covid-19 secure approaches for survey interviewing; and the identification of strategies to collect health measurements/conduct cognitive assessments in absence of face-to-face interviewing. Secondly, the session promotes discussion on the development of ad-hoc surveys on the societal consequences of the current pandemic and strategies to maximise data quality in such studies.
Keywords: Cohort Studies; Covid-19; Longitudinal studies; Data Collection; Survey mode
Mr Matt Brown (Centre for Longitudinal Studies, UCL Social Research Institute) - Presenting Author
Ms Carole Sanchez (Centre for Longitudinal Studies, UCL Social Research Institute)
Ms Darina Peycheva (Centre for Longitudinal Studies, UCL Social Research Institute)
Ms Lucy Haselden (Centre for Longitudinal Studies, UCL Social Research Institute)
Ms Claire Bhaumik (Ipsos Mori)
Ms Kate Taylor (NatCen)
Ms Kirsty Cole (NatCen)
The COVID-19 pandemic and associated lockdown and other restrictions has caused major disruption to surveys in the UK and across the world, particularly those conducted face-to-face.
The Centre for Longitudinal Studies is home to four of the UK’s national cohort studies. Participants are typically surveyed every four to five years with data collection most commonly taking place face-to-face. Interviews are typically long, complex and often involve the collection of non-standard survey data including cognitive assessments and health measurements. Each follow-up is meticulously planned, preceded by extensive piloting and development and has a long data collection period.
At the time of the outbreak in March 2020, the National Child Development Study (1958 cohort) was in-field with our Age 62 Survey; the 1970 British Cohort Study Age 50 Survey was in development with fieldwork due to launch and development of the Next Steps (1990 cohort) Age 31 Survey was upcoming. All surveys were planned to be conducted face-to-face. Ongoing uncertainty with regard to future infection rates and associated restrictions made it difficult to gauge when resumption of face-to-face visits would be possible. It soon became clear that more flexibility with regard to methods of data collection and development of COVID-secure approaches would be required.
Working with the UK’s major data collection agencies NatCen, Kantar and Ipsos-Mori who conduct these surveys on our behalf, we’ve conducted a series of pilots testing new ‘COVID-secure’ methods which aim to replace or supplement traditional face-to-face interviewing whilst retaining some of the advantages of the approach – including video-interviewing, providing participants with devices so that surveys can be self-administered, in addition to making more use of web and telephone. We will describe some of the key challenges arising including adapting cognitive assessments for administration across different modes, how to collect highly sensitive information via telephone and how best to share visual material in remote methods. We’ll conclude by considering the advantages and disadvantages of the various approaches trialled, will confirm key decisions taken with regard to moving forward with the cohort studies and will draw out potential lessons for other studies.
Dr Eoin McNamara (Economic and Social Research Institute) - Presenting Author
Dr Aisling Murray (Economic and Social Research Institute)
Dr Amanda Quail (Economic and Social Research Institute)
Dr Desmond O'Mahony (Economic and Social Research Institute)
Ms Rebecca McClintock (Economic and Social Research Institute)
Dr Lisa Kelly (Economic and Social Research Institute)
Growing Up in Ireland (GUI) is the national longitudinal study of children in the Republic of Ireland, beginning in 2007 and currently ongoing. The study aims to describe the lives of Irish children and young people in terms of their development, with a view to positively affecting policies and services available for them. The study simultaneously traces the development of two distinct cohorts; Cohort ‘98 (age 23 at the time of writing) and Cohort ‘08 (age 12/13 at the time of writing), with data collection taking place every 2-4 years.
Traditionally, data-collection fieldwork has taken place in the study child’s family home; an interviewer visited the home and administered paper- and laptop-based questionnaires to both the study child and their parents. The interviewer administered cognitive tests to the study child, also taking height and weight measurements from all those involved in the study. The face-to-face nature of data collection had many advantages for such a survey: the potential for a positive effect on response rates, especially where the same interviewer returns to households; a greater capacity for a rapport to develop between interviewer and respondent in the context of personal questions; and a more engaging experience for respondents in a survey that typically took 2 hours to complete. It also meant, especially in earlier years, that a family’s capacity to participate was not impeded by poor or absent internet connectivity.
However, this previously successful study design was incompatible with the onset of COVID-19. As of mid-March 2020, and in response to the COVID-19 pandemic, the Irish government has introduced successive restrictions to limit the spread of the disease; these restrictions focussed on limiting social engagements and avoiding non-essential interactions with others.
In light of the COVID-19 restrictions, immediate adaptations to the study design were deemed necessary for fieldwork that was planned for the pilot phase of the 13-year wave of data collection in June-July 2020. Interviews with participants became a mixed-mode of telephone and online. Essential ancillary tasks also had to be adapted: all interviewer-training was adapted to a virtual platform, and information sheets were made available on the study website with password-access, accompanied by videos to preview questionnaire content. However, some sacrifices were necessary in the move to remote data collection, for example it was not possible to conduct cognitive tests or physical measurements.
The need to move to remote interviewing (and preparation) did provide valuable learning insights for a special ‘COVID’ survey conducted online with both GUI cohorts in December 2020, and the experience will inform preparations for future phases of fieldwork to be conducted remotely, if necessary. It also meant that this longitudinal study could continue largely to its expected timeline and collect important contemporary data on the well-being and development of young people during an international pandemic.
Dr Jamie Moore (Institute of Social and Economic Research, University of Essex) - Presenting Author
Professor Michael Benzeval Benzeval (Institute of Social and Economic Research, University of Essex)
Dr Jonathon Burton (Institute of Social and Economic Research, University of Essex)
Professor Thomas Crossley (Institute of Social and Economic Research, University of Essex)
Dr Paul Fisher (Institute of Social and Economic Research, University of Essex)
Dr Colin Gardner (Ipsos MORI Social Research Institute, UK)
Professor Annette Jackle (Institute of Social and Economic Research, University of Essex)
Understanding Society – the UK Household Longitudinal Study (UKHLS) is the UK’s largest household panel survey. It samples the UK population concerning a range of health, economic and social topics, and has been running since 2009. From April 2020, in response to the Covid-19 pandemic, survey participants were also invited to partake in a series of web surveys designed to capture higher frequency information than the yearly UKHLS (the Understanding Society Covid-19 Study).
To prevent Covid-19 survey estimate non-response biases (deviations in estimates from population values caused by unit non-response that is non-random with respect to subject attributes), the UKHLS team has pursued a strategy of both bias prevention (the use of measures to reduce biases during data collection) and bias adjustment (the use of measures to reduce biases remaining after data collection). To prevent biases during data collection, the Covid-19 survey invites the full range of Understanding Society subjects – both those who regularly use the internet and those who do not – to take part. Moreover, periodically non-respondents are invited to a telephone follow-up. To adjust for biases remaining after collection, a non-response weighting strategy is implemented that takes advantage of the rich background information available from past annual interviews.
We also investigate the efficacy of these bias reduction and bias adjustment measures. We quantify the socio-demographic characteristics of respondents in the three subject groups (regular internet users, non-regular internet users, and telephone survey respondents). In addition, we evaluate non-response weighting adjustments, using a novel statistical test developed to quantify Covid-19 survey estimate biases compared to Understanding Society main survey estimates and Kish’s DEFF to quantify inflation in survey estimate variances. We find that both our data collection strategy and weighting adjustments act to reduce biases. Telephone respondent sociodemographic characteristics differ from those from in the other subject groups (in particular, they tend to be older), and their inclusion reduces weighted survey estimate biases and Kish’s DEFF compared to the web survey only sample. However, we also find that inviting non-regular internet users to the web survey appears to be of little benefit: including them as well as regular internet user web respondents has minimal impact on weighted survey estimate biases, and results in a larger Kish’s DEFF. We then discuss the implications of the findings for the design of web (and other) surveys.
Dr Kerstin Hoenig (DIE e.V.) - Presenting Author
Adult Education Centres (Volkshochschulen; AECs) -- publicly funded community learning centres that offer a wide range of formal and nonformal courses -- are among the most important providers of adult education and lifelong learning in Germany. The German Institute for Adult Education has been collecting data on all German Adult Education Centres since 1962 through a yearly survey. This longitudinal survey contains detailed information on course programmes, funding and finances, and institutional characteristics. Although participation is voluntary, the survey typically has a response rate of over 95% of the almost 900 German AECs.
In autumn 2020, the DIE administered an additional survey that focused exclusively on the impact of the Covid-19 pandemic on AECs. The survey contained detailed questions on changes to the course programme, the expansion of virtual classes and the problems encountered with online teaching, and financial impacts. In a departure from the panel survey, the questionnaire consisted of newly developed items that required detailed specific information. Furthermore, some of the questions were subjective or speculative, whereas the regular panel survey only asks factual information. Consequently, the survey posed a high burden on administrative AEC staff and only 38% of AECs took part, with smaller AECs and those with less experience with virtual classes less likely to participate in the survey. Fortunately, it is possible to construct weights to deal with these biases from the longitudinal survey.
In addition, we constructed an add-on to the regular longitudinal survey to assess the most important effects of the pandemic that will be administed to the panel waves for the calendar years of 2020 and 2021. The add-on was designed to be as brief as possible and is limited to information that contextualizes the regular indicators contained in the panel survey. For instance, as the annual survey questionnaire asks about amounts of funding, the add-on asks how much of that funding came from Covid relief programmes.
With this two-pronged approach, we hope to maintain the panel survey's high response rate while also obtaining more detailed information on the impact of the pandemic from a smaller subset of AECs. If this endeavour is successful, it might serve as a model for future assessments of events that affect AECs and thus further enhance the quality of our panel data. Both surveys will be released as scientific use files.
The presentation will offer an overview of the design and most important results of both surveys, lessons learned from the data collection process, and an outlook for similar projects in the future.