All time references are in CEST
Surveying or gaming: How to best measure socio-economic behaviors and attitudes? 2 |
|
| Session Organisers | Mr Jakob Jonathan Kemper (University of Duisburg-Essen) Dr Jan Karem Höhne (University of Duisburg-Essen) Professor Achim Goerres (University of Duisburg-Essen) |
| Time | Wednesday 19 July, 16:00 - 17:30 |
| Room | U6-05 |
People’s socio-economic behaviors and attitudes are of key interest in a variety of scientific disciplines, including, but not limited to, social, political, psychological, and economic research. These disciplines partially differ with respect to their measurement methods. On the one hand, there are many researchers that rely on indirect behavioral and attitudinal measures from large-scale sample surveys. On the other hand, there are also many researchers that rely on more direct measures from behavioral games, such as the dictator, solidarity, and trust games, that are commonly conducted in labs with small convenience samples. Considering the eminent literature, there is some conventional wisdom about the merits and limits of both measurement methods. For example, survey measures are frequently criticized for their hypothetical touch and that they do not decently mirror people’s real socio-economic behaviors and attitudes. Behavioral games, in contrast, are frequently criticized for their artificial settings and samples, which impedes drawing conclusions beyond the studies. Despite this conventional wisdom, there is only a small body of research investigating and evaluating the soundness of both measurement methods. In this session, we therefore invite scientific contributions that present experimental and/or non-experimental research on measuring socio-economic behaviors and attitudes in a variety of research settings (e.g., lab or field) and modes (e.g., in-person or online). We also welcome mixed-method contributions that, for example, combine surveys and games to improve the measurement of socio-economic behaviors and attitudes.
Keywords: socio-economic behaviors and attitudes, experiments, observational studies, behavioral games, interdisciplinary research
Professor Achim Goerres (University of Duisburg-Essen)
Dr Jan Karem Höhne (University of Duisburg-Essen) - Presenting Author
Mr Jakob Kemper (University of Duisburg-Essen)
What if we could experimentally manipulate all characteristics of states, economies and public policies and estimate their effects on citizens? This paper puts forward the evidence from Novaland, a virtual liberal democracy that only exists online and that has characteristics realistically drawn from German and Romanian welfare states. The study consists of an experimental online platform based on text and images in which volunteers are surveyed after they have exited the experience, are randomly assigned to different experiences, such as defined by income, corruption or unemployment, interact with each other simultaneously and thereby co-create collective decisions, such as elections or donation pools, that then determine the course of Novaland. The study was conducted in November 2022 with participants that were recruited via social media platforms, such as Facebook and Twitter. Our main results are: (1) The newly programmed Novaland Experience worked technically well in that over 300 participants could simultaneously interact online at the same time. (2) Participants behaved in an internally valid manner, even though there are some signs of inattentiveness. (3) The political ideology of volunteers was reflected in how they behaved in Novaland, whereas socio-demographic variables showed no systematic effects.
Dr Soomin Oh (University of Birmingham) - Presenting Author
The pandemic and the cost-of-living crisis that has followed suit have changed the global giving behaviour. Donations in Germany, for example, has fallen from 23% of the public having donated to a global charity in the past 12 months in January 2020 at the start of the pandemic to 16% in September 2022; similar trends are seen in France and the UK (Development Engagement Lab 2022). What explains the change in the public’s charitable behaviors? The literature has examined the role of temporal relative deprivation (RD)and egoistic (interpersonal) RD in individuals’ giving behaviours. However, these two types of RD have been looked at separately behaviours (Pak and Babiarz 2022; Pettigrew 2015; Webber 2007). The rapidly changing economic situations warrant a joint consideration of both the public’s perception of their RD over time and against others. Using nationally representative samples from Germany, France, the UK, and the US (n = 6000 respectively), I design a survey experiment (split sample) where groups of respondents are primed to think about (1) their RD from 2019 to 2022, (2) their RD and their community’s RD, and (3) their RD and people in poor country’s RD. They are then asked to allocate $/£/€1000 to themselves, their community, and people in a poor country. The results show heterogenous effect of temporal and interpersonal relative deprivation; the donor public tend to allocate more to themselves when they have gotten worse, regardless of others’ situation. When they have gotten better from 2019 to 2022, they show more pro-giving behaviours to their community and poor countries. The paper contributes to the literature conceptually and empirically with its novel experiment to understand the joint effect of temporal and interpersonal RD on the public’s giving behaviour in four donor publics.
Professor Harald Pfeifer (Federal Institute for Vocational Education and Training (BIBB))
Dr Caroline Wehner (Federal Institute for Vocational Education and Training (BIBB)) - Presenting Author
Professor Andries de Grip (ROA/ Maastricht University)
Professor Julia Kensbock (University Bremen)
Firms with work works councils tend to provide more secure jobs and higher wages (Addison et al. 2010). Consequently, works councils provide an incentive to apply for a job in the respective firms, especially for workers at the lower end of the skill distribution (Kugler/Saint Paul, 2004). This suggests that these firms must screen applicants more carefully to prevent future costs of dealing with “difficult” or less co-operative employees afterwards.
Recent literature shows that personality and attitudes are important predictors for an employee’s productivity and behaviour at the work place (see e.g. Deming/Kahn 2018). Agreeableness has been shown to be an important determinant for conflict-, compromising- and obliging-styles of employees (Tehrania/Yaminib, 2020). If works council firms aim to avoid “difficult” employees, recruiting employees with high agreeableness is particularly attractive.
Thus, we ask whether works council firms value agreeable applicants relatively more than recruiters from other firms to prevent counterproductive work behaviours. To answer this question, we use a discrete choice experiment among firms to assess whether recruiters in firms with a works council have a higher preference for more agreeable workers. Both the discrete choice experiment and the presence of employee representation are assessed as part of the BIBB Cost-Benefit Survey 2017/2018. Our analysis relies on 10,466 worker profiles from 5,233 recruitment decisions from 786 respondents. The job applicants differ in the following attributes: professional competence, personality, and gross wage claimed. We analyse the choices made using a mixed logit regression model (e.g., Revelt/Train (1998)).
Our results confirm that recruiters from firms with a works council have a higher preference for agreeable workers than do recruiters from firms without a works council. Our results provide empirical evidence for employer selection mechanisms induced by works councils.
Mr Ruslan Suleymanov (HSE University)
Mr Daniil Lebedev (HSE University) - Presenting Author
In an era of increasing number and complexity of web surveys, one of the methods to reduce cognitive load and increase motivation of participants is gamification. Existing studies of gamification effect are limited to marketing research, while academic studies were mostly conducted on schoolchildren. This study is aimed to check the effects of gamification on cognitive effort and motivation of respondents measured by pupil diameter dynamics and paradata among respondents aged 18-25 years.
We conducted a laboratory experiment in which 124 students were randomly assigned to two groups with traditional and gamified web survey design. We used emojis for ordinal scales and motivating memes between sets of questions as gamification in line with previous research.
Multimodal approach to measure cognitive load and motivation included a combination of three types of measurement: (1) objective indicators and paradata (item nonresponse rate, the length of the mouse cursor path, alerts); (2) subjective evaluation of survey burden (NASA-TLX); (3) neurophysiological indicators – pupil diameter dynamics (eye tracker).
According to subjective perception (via the NASA-TLX scale), a gamified survey is perceived as longer to complete. Two-way ANOVA comparing average cursor path showed the importance of the interaction factor of the group and the question – different questions are more cognitively demanding in two groups. The average cursor path is lower in the group with gamification. Traditional questionnaire design produced 4 times more warning alerts than gamified. Deviation of the pupil diameter from the baseline was higher in the gamified group which means gamification can increase cognitive load. Different modes of cognitive load assessment were not consistent. Although the results are somewhat inconclusive - subjective cognitive load did not increase while neurophysiological did - we state that gamification has advantages over traditional web survey design among respondents aged 18-25 in terms of increasing motivation.
Mrs Marije Oudejans (Centerdata - Tilburg University)
Mr Stein Jongerius (Centerdata - Tilburg University) - Presenting Author
Dr Seyit Höcük (Centerdata - Tilburg University)
Mr Joris Mulder (Centerdata - Tilburg University)
Social and economic experimental research is commonly performed with non-representative groups, often in a university laboratory setting using students as subjects. Complementing laboratory experiments with field experiments adds external validity and a more diverse subject pool. oTree, an open source Python-based tool for interactive behavioral research and experiments facilitates this in an online laboratory environment (Chen, Schonger and Wickens, 2016).
oTree can be utilized in small-scale controlled laboratory settings or with large-scale online research platforms, such as MTurk or Prolific. By implementing oTree in the Dutch LISS panel, we offer best of both worlds, a large-scale controlled setting. The LISS panel is a probability-based online panel representative for the Dutch household population. The open-access LISS Data Archive comprises an extensive database of background characteristics, which can be merged for more in-depth analyses or used to select specific population groups. In addition, respondents can be specifically instructed before starting the experiment, while they can receive technical support if needed.
We fielded an oTree game in the LISS panel for a large-scale experiment on human behavior regarding coalition formation. The project, Compose 2.0, aims to understand how human behavior affects scaling up horizontal collaboration in supply chains. A game was designed in which 600 LISS panel respondents played the part of a transport company. The oTree game automatically distributed players into 200 groups of three, who collaborated on the transportation of goods and negotiated on profit distribution (actually paid out).
5,000 panel members were invited to participate in the game. A quarter of the respondents committed to participate in the experiment and 82% of those actually joined the game. The study was successfully completed and taught valuable lessons to further optimize controlled real-time online multiplayer games in the LISS panel.