All time references are in CEST
Social Media Ads as a tool for targeted survey sampling 1 |
|
| Session Organisers | Dr Michael Zoorob (Meta Platforms) Dr Steffen Pötschke (GESIS - Leibniz Institute for the Social Sciences) Dr Bernd Weiß (GESIS - Leibniz Institute for the Social Sciences) |
| Time | Thursday 20 July, 09:00 - 10:30 |
| Room | U6-05 |
The sampling of hard-to-reach populations remains a challenge in survey research. For example, conventional survey methods are often incapable of reaching very mobile groups, such as international migrants, or employees of specific industries, who are defined by an attribute that is usually not included in readily available sampling frames in most countries. Additional challenges occur in the field of comparative social research when a researcher must grapple with multiple national sampling systems that prevent consistency in sampling design.
Though not without its problems and challenges, the wide reach and detailed targeting options made available by social media advertising provide a potential solution to some of these core-problems in survey research. A nascent but growing suite of research tools, including automated Chatbots that provide customized messaging and automated survey response collection, has developed to enhance this approach. Recent research has underlined that social media constitute a useful recruitment tool allowing timely and cross-national data collections on relevant issues such as (forced) migration and the COVID-19 pandemic. This panel will discuss recent empirical research that samples via targeted social media advertising sampling in the absence of feasible alternatives. Preference will be given to contributions that, besides demonstrating the added value of this sampling approach, address additional methodological issues in terms of survey quality, e.g., sample composition in comparison to appropriate benchmarks, or nonresponse bias. We also welcome contributions that discuss issues of research ethics when applying this sampling approach.
Keywords: Sampling, sampling frames, social media, advertisements, hard-to-reach populations
Ms Jessica Donzowa (Max Planck Institute for Demographic Research) - Presenting Author
Dr Daniela Perrotta (Max Planck Institute for Demographic Research)
Professor Emilio Zagheni (Max Planck Institute for Demographic Research)
During the COVID-19 pandemic, we have witnessed many primary survey collection efforts, especially relying on advertisements using social media recruitment. However, such surveys may be biased due to self-selection depending on interest in the study subject. In this study, we assess how the display of the survey topic in advertisements during recruitment affects responses. For this, we use the survey data collected via the “COVID- 19 Health Behavior Survey” that ran between March and August 2020 in eight countries in Europe and North America (N = 133,069). Respondents’ recruitment took place via Facebook advertisements that differed in the image shown in the ad. Here we investigate whether survey responses resulting from images with stronger (or weaker) relation to COVID-19 are associated with higher (or lower) compliance with recommended preventive behaviors, such as wearing a face mask, increasing hand hygiene, or perceived threat of the pandemic. We observed no image effect on the outcomes in the majority of the countries. Where an effect was found, there is no clear association between higher topic salience and the adoption of protective behaviors or higher threat perception. Our findings are reassuring regarding the low impact of self-selection bias on survey responses based on topic interest, thus strengthening the reliability of surveys using social media recruitment during health crises.
Dr Ken Brackstone (University of Southampton) - Presenting Author
Dr Douglas Leasure (University of Oxford)
Dr Michael Head (University of Southampton)
Professor Brienna Pirelli-Harris (University of Southampton)
Since Russia’s invasion of Ukraine in February 2022, more than 7 million people have been internally displaced within Ukraine, and 7 million people have fled to Europe and beyond. In this study we aimed to: (1) understand more about population movements and health circumstances, and (2) circulate results with humanitarian groups and policymakers to direct support and assistance. We administered a cross-sectional survey between April and July 2022. The survey was advertised as a “health needs” survey, inviting IDPs and refugees to report on current health circumstances. An advert was professionally designed for the survey and shared on social media using Facebook Ads Manager. We surveyed Ukrainians (18+) based in every European country apart from Russia. The advert reached 1.1m people at a cost of 2100 GBP (2400 Euro) and entailed over 4.5m impressions. Respondents saw the advert 3.92 times on average. The survey was also shared by 1.4k people on Facebook, resulting in an additional reach of 64k people. In total, 10,159 participants – 40.9% IDPs and 59.1% refugees – completed the survey. We recognised that our survey was not representative of the Ukrainian population or those who fled. To improve representativeness, we created weights based on Facebook user statistics and pre-war population estimates, which attempted to account for biases in gender, age, and origin (i.e., Oblast). Approximately 89% of our respondents were female, but this over-representation may have been due to the greater number of female refugees, or because the Facebook algorithm targets people more likely to click on the advert (i.e., women). To garner more responses from men, we administered an additional advert via Facebook targeting men only. Our survey was a trade-off between a rapid understanding of health-needs during war versus population representativeness.
Professor Anja Neundorf (University of Glasgow)
Dr Aykut Ozturk (University of Glasgow) - Presenting Author
Social media advertisements on Meta platforms offer a significant potential of participant recruitment for online surveys as more than two billion people visit these platforms on a regular basis. By buying advertisement space on Facebook or Instagram, scholars can recruit enough people to form a survey sample in a short period of time. However, despite the widespread use of Facebook by people from various demographic groups, participants recruited through Facebook tend to be more liberal and more college-educated than the average population. This is a problem especially if researchers need to recruit supporters of radical right-wing parties, who are less educated and more sceptical towards international institutions.
In this study, we test whether researchers can use advertisement content and tools to increase the recruitment of radical right-wing party supporters. Our study relies on advertisement campaigns conducted in Turkey and Spain. In Turkey, our goal was to recruit supporters of the ruling Justice and Development Party. In Spain, our goal was to recruit Vox supporters. In our order to recruit supporters of these parties, we used advertisement images that included symbols associated with these parties’ ideological positions. Furthermore, we used demographic targeting options offered by Facebook Advertisement Manager, such as interests in right-wing newspapers in these countries. Our results show that these strategies improve the recruitment of radical right-wing voters to a certain extent but they do not solve the problem entirely.
Mr Björn Rohr (GESIS - Leibniz Institute for the Social Sciences) - Presenting Author
Dr Barbara Felderer (GESIS - Leibniz Institute for the Social Sciences)
Dr Steffen Pötzschke (GESIS - Leibniz Institute for the Social Sciences)
Dr Jan Priebe (Bernhard Nocht Institute for Tropical Medicine)
Dr Henning Silber (GESIS - Leibniz Institute for the Social Sciences)
Dr Bernd Weiß (GESIS - Leibniz Institute for the Social Sciences)
Sampling respondents via ads on social media is a promising new approach to reach respondents both in hard-to-sample regions and at a cross-national level. As one option, Meta optimizes their advertisement to reach the most clicks on a provided link (e.g., the survey link). The optimizing algorithm includes the layout of the ad and user characteristics. Knowing this, multiple targeting strategies are possible, for example, targeting all Meta users of a selected group or within separate strata of that group. Yet, the influence of the targeting strategy on the sample composition is less studied.
In this study, we evaluate the sample composition of surveys we conduct using Meta ads in Sub-Saharan Africa with varying Internet coverage. Specifically, we ask which targeting strategy leads to a survey respondents' composition that comes closest to the respective benchmarks of the general population. As benchmarks, we use socio-demographics and health-related variables such as vaccination prevalence from the Demographics and Health Survey (DHS), the Afrobarometer, and administrative data. Our target sample consists of Facebook and Instagram users from three African countries (Ghana, Kenya, and South Africa). We vary the two sampling strategies over the course of several weeks. Our stratified targeting strategy groups the countries by region, age, and gender, whereas the second sampling strategy is without demographic targeting. The study results inform researchers on how a Meta survey of African countries compares to surveys realized through a probability-based approach and provides insights into the advantages and disadvantages of employing stratified targeting as a sampling strategy.
Mrs Zaza Zindel (Bielefeld University) - Presenting Author
Mr Simon Kühne (Bielefeld University)
Mr Emilio Zagheni (Max Planck Institute for Demographic Research)
Mrs Daniela Perrotta (Max Planck Institute for Demographic Research)
A growing number of research projects are using social networks such as Facebook to recruit (infrequent) populations to participate in online surveys. These social media platforms are based on an advertising revenue model that allows researchers to recruit survey participants by buying ads. However, researchers need more information about how users view the ads, perceive them, and why they ultimately participate in a survey. As a result, different ads can lead to polarized response patterns that favor, for example, users with very strong opinions on certain survey topics.
To date, there is no evidence in survey research on how different ad designs affect the composition of survey respondents.
Our presentation includes a Facebook ad experiment conducted as part of a web survey project on climate change and migration in 2022-2023. We contrast different ad images and examine whether the polarizing features of the ads are also reflected in the survey data. The talk thus provides valuable insights into data quality hurdles, such as sampling errors in survey data generated in this way. Moreover, our examples can serve as a starting point for further research.