ESRA logo

Keynotes

From Mail Surveys to Chatbots: Changes in Survey Modes, Methods, and Data Sources Over Time

Kristen Olson (University of Nebraska-Lincoln)

Abstract:

Survey research is in flux. As response rates decline, survey costs rise, new data sources become available for sample frames, survey data are increasingly linking to administrative data sources, technological changes shift how data are collected, and trust and legitimacy in the scientific enterprise are under attack, survey researchers are examining what and how inferences are made about populations of interest. At the same time, time-tested approaches of collecting data persist and remain important tools for understanding populations of interest.

In this talk, I will examine shifts in the landscape of survey methods and survey statistics research from multiple perspectives, including new and emerging data sources and changes among more traditional methods. First, I will provide an overview of the changing nature of survey methods and statistics research over time, including changes in modes, types of data sources, analytic approaches, methods of inference, and other important features of articles published in major survey methods-focused outlets. Second, I will report on changes in uptake of traditional modes of data collection, including results from a large-scale meta-analysis of mixed-mode web-and-mail surveys, and other sources. Finally, I will explore what these new technologies and data sources mean for future conduct of, research about, and training in survey methods, including implications for survey costs.

Biography:

Kristen Olson, Ph.D. is the Leland J. and Dorothy H. Olson Professor in Sociology and Director of the Bureau of Sociological Research at the University of Nebraska-Lincoln. Dr. Olson’s research focuses on mixed-mode surveys, questionnaire design, interviewer effects, the intersection of nonresponse and measurement errors, and within-household selection in self-administered surveys. She recently finished a four-year term as the Editor-in-Chief of the Journal of Survey Statistics and Methodology. She is a fellow of the American Statistical Association, the American Association for the Advancement of Science, and the Midwest Association for Public Opinion Research. She has an M.S. in survey methodology from the University of Maryland and a Ph.D. in survey methodology from the University of Michigan.

We can do better: An ESRA session in memory of Don Dillman

The ESRA Committee

Don Dillman passed away in June 2024. This special session is dedicated to his legacy with regard to survey methodological research. Don was a world-leading expert in survey methodology, who won the ESRA Outstanding Service Award in 2023. His many great contributions include his milestone book “Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method” (with Jolene D. Smyth and Leah Melani Christian; most recent edition published in 2014), which most of us have on our bookshelves. He is also known for his important collaborations with ESRA members across different career stages. This includes the influential “International Handbook of Survey Methodology” (with Edith D. de Leeuw and Joop J. Hox; most recent edition published in 2018) as well as the article “An experimental comparison of three strategies for converting mail respondents in a probability-based mixed-mode panel to Internet respondents” (with David Bretschi and Ines Schaurer; published in the Journal of Survey Statistics and Methodology in 2023). Moreover, Don contributed to research published in the ESRA journal Survey Research Methods, for example the articles “Does Visual Appeal Matter? Effects of Web Survey Aesthetics on Survey Quality“ (with Taj Alexander Mahon-Haft in 2010) and “Visual Design, Order Effects, and Respondent Characteristics in a Self-Administered Survey” (with Michael Stern and Jolene D. Smyth in 2007).

The last ESRA conference which Don attended in person was in Zagreb in 2019, where the title of his presentation was “We Can Do Better: The Application of Theory for Improving Response Rates to Mixed-Mode Surveys”.  We will remember Don as someone who was always dedicated to “doing better” by designing the best possible surveys. By his great example, he encouraged us to do better, too. In this session, we honor Don’s legacy and highlight examples of research in ESRA influenced by his work with a focus on mixed-mode surveys.

Introduction: Vera Lomazzi 

Moderator: Daniel Seddig

Speakers:

  • Edith de Leeuw 
  • Pablo Cabrera-Álvarez 
  • Olga Maslovskaya 
  • Carina Cornesse  
  • Rory Fitzgerald  
  • Barry Schouten 

Edith D. de Leeuw

Edith is emeritus professor of survey methodology at the department of methodology and statistics at Utrecht University.  She received the Monroe G. Sirken award in interdisciplinary survey research method (AAPOR/AMSTAT), the WAPOR’s Helen Dinerman prize for lifetime contributions to the field of public opinion and the ESRA award for outstanding services to survey research. She was a Fulbright scholar with Don Dillman at Washington State University and visiting scholar of the Program on Social Statistics at UCLA.  She is a fellow of the Interuniversities Joint Institute for Psychometrics and Sociometrics (IOPS) in the Netherlands, and was awarded the Visiting International Fellowship at the Institute of Social Research, University of Surrey. Edith has over 150 scholarly publications and was co-editor of 5 internationally renowned books on survey methodology.

Pablo Cabrera-Álvarez 

Pablo a survey methodologist working on response maximisation strategies at Understanding Society. Previously, Pablo worked as a Survey Statistician at NatCen Social Research and holds a PhD in Social Science. The PhD research addressed the use of administrative data to minimise selection bias in nonprobability samples. Pablo’s research interests cover survey non-response and response maximisation strategies, inference from nonprobability samples and public opinion.

Olga Maslovskaya

Olga Maslovskaya is an expert in survey methodology interested in all aspects of surveys.  Her special interests are in the areas of survey data collection, data quality, representativeness, and online surveys.  Olga is Deputy Director for ESRC-funded Survey Futures project, which is Survey Data Collection Methods Collaboration in the UK.  Olga leads Research Strand (RS) on Methods for Surveys without Field Interviewers. This RS focuses on assessment of different design features and their effectiveness in recruitment process. It also investigates effectiveness of knock-to-nudge approach as a household contact method among other issues associated with recruitment. She is also a Co-Investigator (Co-I) on Generations and Gender Survey (GGS) in the UK project, where she leads the work package, which is responsible for the methodological assessments of the UK GGS.  Olga is a Co-I on “Understanding Coverage in the UK Population Longitudinal Studies” project and a Principal Investigator (PI) on the “Under-represented population subgroups in social surveys: Methods for respondent-driven sampling with probability-based seeds”.   Additional to her survey methodology expertise, OM has an extensive experience of analysing a wide range of large-scale datasets and of employing various advanced statistical methods in different research contexts. 

Carina Cornesse

Carina is head of the department Survey Design and Methodology at GESIS Leibniz Institute for the Social Sciences. Previously, Carina  served as a guest professor in Social Stratification and Survey Methodology at Freie Universität Berlin, led the SOEP Innovation Sample and the German Social Cohesion Panel within the Socio-Economic Panel (SOEP) at the German Institute for Economic Research (DIW), and held research position at the University of Mannheim and University of Bremen. Her research focuses on the development and quality assurance of innovative data collection methods, evaluating the strengths and limitations of probability-based and nonprobability samples, and panel study recruitment and maintenance of panels.

Rory Fitzgerald 

Rory is Professor at City University of London, and became Director of the European Social Survey (ESS) in 2013 having been a Senior Research Fellow at City, University of London since 2004. In November 2013, the UK and 14 other European governments established the European Social Survey as a European Research Infrastructure Consortium (ESS ERIC) – an independent international research organisation hosted at City, University of London. ESS ERIC now has 27 Members (November 2021). Rory is the first Director of ESS ERIC and oversees the ESS Core Scientific Team (CST) and the ESS National Coordinators Forum. In addition to these committees he works closely with the General Assembly as well as the Scientific and Methods advisory boards. 

Barry Schouten

Barry is senior methodologist at Statistics Netherlands. After a PhD in mathematical statistics, he joined Statistics Netherlands in 2002. He has been a research professor at Utrecht University between 2017 and 2024. His research interests are in survey nonresponse reduction and adjustment, mixed-mode survey designs and analysis, adaptive survey designs and smart surveys. He has been active in several Eurostat-projects in these areas. Currently, he is also involved in ESRA short movies.