ESRA logo

ESRA 2023 Preliminary Glance Program

All time references are in CEST

Survey Nonresponse Trends and Trust in Surveys: A global perspective on the current survey climate 1

Session Organisers Professor Bettina Langfeldt (University of Kassel)
Mrs Ulrike Schwabe (DZHW)
Dr Henning Silber (GESIS)
TimeTuesday 18 July, 11:00 - 12:30
Room U6-07

In our complex and interconnected world, there is a strong need for databased scientific approaches to solving diverse local as well as global problems. However, to transform scientific recommendations successfully into policy measures, societal trust in scientific methods and results is required. Yet, mistrust toward scientific results seems to be on the rise in recent years.
In a democracy, surveys can be an important tool for measuring public opinion and informing political decision-makers about the views of their constituents. Yet, decreasing survey participation, attempts to manipulate polls, and misleading accusations of “fake polls” as well as polls carried out not in accordance with established scientific standards, put the validity of the gathered data in jeopardy. If the survey climate continues to be on the decline, this will have drastic consequences for survey-based research since both policymakers as well as the recipients of political interventions have to believe in the accuracy of the data.
Against this background, this session aims to bring together current research on the following topics:
Survey Climate: What is the current state of the survey climate? What caused changes? What can be done to foster a positive survey climate, both in terms of increasing survey participation and increasing the quality of survey data?
Trust in Surveys: What can be done to increase trust in surveys and their results? Which are determinants of participation and giving truthful answers that can be used to strengthen the quality of survey data? How is declining trust in surveys related to the declining trust in science?
Surveys and Society: If and how is political participation linked to trust in science and the generalized attitudes towards surveys? What is the role of surveys in the democratic process? Can process-based data (e.g., digital trace data) help to mitigate the survey

Public Opinion Surveys and the Democratic Process Among Young Adults: Attitudes, Beliefs, Knowledge, and Behavior

Professor Allyson Holbrook (University of Illinois at Chicago)
Ms Evgenia Kapousouz (University of Illinois at Chicago)
Professor Timothy Johnson (University of Illinois at Chicago) - Presenting Author
Dr Henning Silber (GESIS)

The problem addressed by our research is to better understand the lack of knowledge about surveys and survey methodology and low levels of trust and participation in public opinion surveys as a form of civic engagement in the democratic process. We examine this problem using primary data collected from three sources. The first study will involve qualitative examination of perceptions of public opinion surveys using four moderated focus groups of young adults. These were completed in November 2022. Findings from the focus groups are currently being used to develop materials for the second study, which will involve a web survey to be administered to 600 respondents from a volunteer panel. This survey will be conducted in early January 2023. The goal is to develop reliable multi-item measures of several key constructs, including attitudes toward and beliefs about public opinion surveys, knowledge about survey methodology, and survey-related behaviors). Using these new measures, Study 3 – to be conducted in early Spring 2023 – will involve assessing how each of the constructs developed during Study 2 predict responses to vignettes about survey results and participation in surveys using an online factorial experiment involving 1,800 respondents. This project will help us better understand how people think about surveys and their importance as a form of civic engagement. This will help researchers, policymakers and others who conduct or use surveys to better discern how the public understands and values surveys as a form of public expression and identify potential strategies for encouraging and increasing survey participation, particularly among young adults.

Trust in surveys, income non-response and linkage consent - the SHARE perspective

Ms Imke Herold (MEA-SHARE and SHARE Berlin Institute) - Presenting Author
Dr Arne Bethmann (MEA-SHARE and SHARE Berlin Institute)
Dr Michael Bergmann (MEA-SHARE and SHARE Berlin Institute)

Like most surveys, the Survey of Health, Ageing and Retirement in Europe (SHARE) suffers from a considerable amount of item non-response, particularly regarding income questions as well as respondents’ reluctance to consent to record linkage. Both aspects have a potentially severe impact on data quality leading to less precise as well as biased estimates and thus might hamper substantive analyses.
While SHARE tries to alleviate problems due to item non-response, e.g. through income imputations, it seems especially important to understand the underlying reasons that keep respondents from reporting their income or consenting to record linkage. In order to add to the available body of research from the SHARE perspective with its specific sample of respondents 50+ and its study focus, we used the opportunity to include an additional paper-and-pencil drop-off questionnaire with the German sub-study, focusing on issues like trust in surveys and organizations, data privacy concerns as well as attitudes towards income questions.
Using data from the German SHARE Wave 8 and Wave 9 drop-off questionnaire, we want to contribute to the question whether several aspects of trust are interrelated with income non-response on the one hand and denial of linkage consent on the other hand. By choosing income questions – as one of the classics of item non-response – and consent to record linkage – as a more innovative survey tool – we try to cover a broad range of aspects that can be relevant for respondents’ trust. Analyses based on these specific SHARE waves allow us to compare the survey climate before the onset of the pandemic with the time after. Furthermore, we are able to investigate consequences of income item non-response on potential panel dropout in subsequent waves.

Trust Us: The Role of Polls in Society in 2022

Mr Jon Cohen (Primera Capital)
Ms Laura Wronski (SurveyMonkey) - Presenting Author

Early results indicate that the polls conducted prior to the 2022 midterm elections might be the most accurate in the past decade, and yet the American public continues to be skeptical of their accuracy and the accuracy of the polling field in general. This paper will use the results of three sets of polls conducted in the US, the UK, and Canada. In the US, the first poll was fielded the month prior to the 2022 midterm election, and the other in early 2023—both fielded using SurveyMonkey’s proprietary river sample. In the UK and Canada, both surveys were fielded in early 2023. These surveys ask about the role of polling in society today, probing respondents on how much they read/hear about polls in the news, what characteristics of a poll make it more trustworthy, and how confident they are in their ability to distinguish a high quality from a low quality poll. We also compare polling with other means of collecting data from individuals (e.g. focus groups, town halls). We will analyze the results by various subgroups to understand how pollsters can build trust in their work among those groups who are the most skeptical, and will create an international profile of what characteristics those poll-hesitant individuals share. We will also compare the trend over time, from October to December, to see how engagement with and trust of polls is affected by a high-stakes election.

Public Opinion Research in Russia After February 2022: Crisis of Confidence or Business as Usual?

Dr Eli Feiman (United States Department of State)
Ms Dina Smeltz (Chicago Council on Global Affairs) - Presenting Author

A crackdown on protests and free speech in Russia since February 2022 have undermined trust in the reliability of public opinion data about the Ukraine war and other key attitudes. Pollsters and journalists can only describe the current conflict in Ukraine as Russia’s “special military operations,” prompting skeptics to question the practical value of survey results when a climate of fear could be suppressing participation in public opinion surveys among critics of the war. At the same time, public opinion research by both Russian and international organizations continues apace, showing trends consistent across sources and resonant with changes following Russia's 2014 annexation of Crimea. This paper addresses key challenges to collecting and understanding public opinion data from Russia at this critical moment. It outlines the changes as well of continuities in the field of public opinion research in Russia before and after February 2022, assesses Russian polling organizations’ current tests for reliability, and offers some new approaches for future consideration. Recognizing both the challenge and importance of measuring attitudes in the current climate, this paper situates today’s Russia in broader literatures related to public opinion in authoritarian contexts and wartime. Leveraging multiple sources of data as well as the authors' practical experience in the field, this paper lays out a framework for researchers and policymakers for interpreting and building trust in research on Russian public opinion.

Survey attitudes and confidence in leaders in polling institutions

Professor jibum kim (Sungkyunkwan University) - Presenting Author
Professor Jae-Mahn Shim (Korea University)
Dr Sori Kim (Gallup Korea)
Dr Tom W. Smith (NORC at the University of Chicago)

Confidence in leaders in polling institutions is vital for the proper function of polling. Using the 2021 Korean General Social Survey (KGSS), we examined the relationship between survey attitudes, such as survey enjoyment, survey value, and survey burden, and confidence in leaders in polling organizations. Of the 19 leaders in major institutions, confidence (a great deal + only some) in leaders in polling institutions is ranked 7th (67%), preceded by medicine (85%), financial institution (83%), education (81%), major companies (81%), national statistical agency (78%), and scientific community (73%). Using the logistic regression, we found that survey enjoyment is positively related to confidence in leaders in polling organizations, and survey value and survey burden are not associated with confidence in leaders in polling institutions. Of control variables, trust in society and news is positively associated with confidence in leaders in polling organizations. Compared with liberals, independents and conservatives are less likely to have confidence in leaders in polling institutions. The survey organizations need to improve the ways to make the survey enjoyable to respondents.