ESRA 2025 Preliminary Program
All time references are in CEST
Merits and limits of respondent recruitment through social media |
| Session Organisers |
Professor Simon Kühne (Bielefeld University)
Professor Jan Karem Höhne (DZHW, Leibniz University Hannover)
Mrs Jessica Donzowa (Max Planck Institute for Demographic Research)
|
| Time | Tuesday 15 July, 09:00 - 10:30 |
| Room |
Ruppert 002 |
The demand for high-quality data from surveys – especially from web surveys – is at an all-time peak and continues to grow. At the same time, (web) surveys struggle with low response rates and researchers constantly look for innovative and cost-efficient ways to sample, contact, and interview respondents. One promising approach makes use of social media platforms, such as Facebook, Instagram, and TikTok, for respondent recruitment. Utilizing sophisticated advertisement and target systems offered by these platforms, research has shown that social media recruitment provides an easy, quick, and inexpensive access to an unprecedented and diverse respondent pool (including rare and hard-to-reach populations). Nonetheless, many open questions remain with respect to sample representation, recruitment and advertisement strategies, and data quality and integrity.
In this session, we welcome contributions that are based on empirical studies as well as methodological and theoretical considerations dealing with respondent recruitment through social media platforms. This includes the following research areas (but not limited to):
- Comparing social media platforms for respondent recruitment in terms of costs, quality, and field phase management
- Ad design, invitation wording, and participation metrics
- Strategies for incentivizing respondents and payout
- Targeting designs and sample compositions
- Strategies to cope with user comments and undesired user interactions with ads
- Approaches to deal with missing data in the form of nonresponse and dropouts
- Measurement quality in terms of reliability and validity
- Threat of bots and fake interviews for data integrity
- Comparing social media, nonprobability, and probability-based samples
- Weighting procedures to increase representation and generalizability
- Replications of empirical studies and findings
Keywords: Social media recruitment, Data quality, Data integrity, Representation, Sample comparisons
Papers
Exploring Problematic Response Behaviors in Social Media-Recruited Surveys
Mrs Zaza Zindel (German Centre for Integration and Migration Research) - Presenting Author
Social media recruitment has transformed survey research, offering efficient and cost-effective access to diverse and hard-to-reach populations. However, this method also presents significant challenges, such as satisficing, low-effort participation, and identity misrepresentation-issues that remain understudied in social media-recruited surveys. These behaviors pose substantial threats to data qualityand raise important questions: To what extent do such behaviors occur in social media-recruited surveys, and how do they affect analytical results?
This study examines data from a web survey on labor market discrimination against women wearing headscarves in Germany, conducted in 2021 and repeated in 2024. Recruitment was conducted through targeted Facebook ads designed to reach diverse respondents. Problematic responses were identified using indicators such as item non-response, straight-lining, speeding, and identity misrepresentation. Differences between problematic and non-problematic respondents were analyzed using statistical tests (e.g., chi-square, Mann-Whitney U), and multivariate regression models were used to assess the impact of these behaviors on key outcomes, such as perceived anti-Muslim discrimination.
The results reveal systematic differences between problematic and non-problematic respondents. For example, younger and less educated participants were more likely to engage in behaviors such as speeding and straight-lining, leading to distorted results in multivariate analyses. Data cleaning significantly improved the robustness of the findings, highlighting the critical importance of quality control measures in surveys recruited via Facebook.
This study provides a detailed examination of the prevalence, predictors, and consequences of problematic response behavior in social media-recruited surveys. It offers practical recommendations for survey practitioners, emphasizing the need to incorporate quality assurance strategies to enhance data validity and reliability.
Strategies for social media advertising: insights from a non-probability online survey series
Ms Eszter Sandor (Eurofound) - Presenting Author
Mr Michele Consolini (Eurofound)
Eurofound launched a non-probability online survey in March 2020 aiming at measuring the quality of life and work during the early pandemic (Living, working and COVID-19) with a quick turnaround time. The survey was promoted on Meta platforms via a conversion campaign, using COVID-19-related pictures and messages. The survey was repeated four times during the pandemic, and a panel of regular respondents was also established. In spring 2023, a sixth wave was launched, with the pandemic no longer in focus, titled Living and working in the EU. Pixel technology was abandoned due to potential data protection issues, and a traffic campaign was implemented. Instead of pandemic-related imagery, advertisement pictures and messages were tailored to the main contents of the survey. Response to the survey was much lower among social media respondents.
To address this issue, in spring 2024 (wave 7), Eurofound implemented a new advertising strategy, based on advice from previous research on social media advertising. Advertisement sets (assets) were developed for four distinct topics, each with a salient and non-salient image and message combination, as well as a final neutral and general advertisement, so altogether nine assets were launched. Advertisements were placed on the Meta Newsfeed only instead of automatic placement, only static images were used, and ads were placed on an additional platform (LinkedIn).
This presentation compares the sample size and composition achieved in the 7th wave with previous waves, and the number and types of respondents reached by each of the nine advertisements and the two social media platforms.
