ESRA 2017 Programme

Tuesday 18th July      Wednesday 19th July      Thursday 20th July      Friday 21th July     

     ESRA Conference App

Thursday 20th July, 14:00 - 15:30 Room: F2 108

Different methods, same results? Comparing the consequences of alternative methods of data collection and analysis 2

Chair Professor Elmar Schlueter (Justus-Liebig-University Giessen )
Coordinator 1Professor Jochen Mayerl (University of Kaiserslautern)

Session Details

No doubt about it – recent years have seen an ever increasing proliferation of methods for survey data collection and analysis. Think about the growing administration of surveys via the internet and mobile devices, the combination of large-scale surveys with experimental designs, the multiple approaches available to examine data from respondents nested in different levels of analysis or the wider application of Bayesian statistics. Such methodological innovations certainly help to open up important novel avenues for research. However, a central yet somewhat understudied question coupled with the plurality of methods is: To what extent do different strategies of survey data collection and analysis applied to the same research question lead to converging conclusions? Specifically, this session starts from the observation that for most research problems a single appropriate strategy of data collection or analysis does not exist. Instead, researchers typically face alternative defensible methods which may or may not converge in their results. Thus, the aim of this session is to stimulate the debate on the methodological as well as substantive issues that might arise when applying multiple methods of survey data collection or analysis. Does the application of alternative research designs or statistical methods lead to converging results? Are social science results with different methods replicable? We invite researchers to submit papers discussing the consequences of applying alternative methods of survey data collection or analysis in the following two scenarios:

A. Same research question, comparing at least two different methods of data collection
B. Same research question, comparing at least two methods of data analysis

Please send your paper proposals (no more than 500 words in length) to:

JProf. Dr. Jochen Mayerl,
Prof. Dr. Elmar Schlüter,

Paper Details

1. Hello, neighbour! Comparing different modes and instruments as a measurement for individual housing preferences in a survey-based implementation of Schellings’ segregation model
Mr Andreas Schneck (Ludwig Maximilian University of Munich)
Dr Christiane Bozoyan (Ludwig Maximilian University of Munich)

Following the segregation model of Schelling (1971) residential segregation is solely explained by individual preferences for a specific ethnical composition of a neighborhood which determines moving behavior and finally may lead to large segregation patterns on the macro level. We examine individual housing preferences in a 2016 population survey in Bavaria (Germany) with different measurement-instruments as well as their implications on segregation.
In empirical studies the common way of measuring preferences is to ask subjects which neighborhood regarding the ethnic composition they prefer to live in (Clark 1991, Farley & Frey 1994). Two instruments of direct questioning can be distinguished: a verbal instrument in which respondents choose their desired (numeric) proportion of minority (Clark 1991) and a (graphical) visual instrument in which respondents see pictures of the equivalent composition (Farley 2011). Furthermore in recent years, more and more questionnaires are conducted online while it is not known if housing preferences collected online or via paper and pencil yield the same results.
To obtain a more realistic upper and lower preference window the willingness to move out is asked in addition to the most attractive neighborhood (Farley 2011). In an alternative variant respondents name the neighborhood composition they would like to live in and in which not at all (GESIS 2006). More sophisticated versions integrate also a correction equation resulting from the true composition of the neighborhood the respondents actually live in (Pforr & Horr 2014). But even if different variations of measuring individual housing preferences are used, no systematic comparison of the different forms exist.
In order to compare different instruments (verbal vs. visual which were experimentally varied) and modes (online vs. paper and pencil) we asked in every condition for the most attractive as well as barely attractive ethnic composition of a possible neighborhood. Beyond the comparison of the different conditions and in order to derive a best-practice-continuum measuring segregation preferences, we will validate the different conditions (visual vs. verbal, online vs. paper and pencil, attractive vs. barely attractive) with preferences asked within a factorial survey module as well as the real share of migrants in the neighborhood based on the ZIP-Code. We assume that the composition varied in the vignettes are a more accurate measurement of the true value of housing preferences compared to direct questioning, since effects of social desirability are reduced. Also the factual composition should reflect the underlying true preferences better, because people should already have been moved if they do not tolerate the composition of their neighborhood.

Clark, William A. V. (1991): Residential preferences and neighborhood racial segregation: A test of the Schelling segregation model. Demography 28(1): 1-19.
Farley, Reynolds (2011): The waning of American apartheid? Contexts 10(3): 36-43.
Pforr, Klaus, and Andreas Horr (2014): Measurement of stated preferences about two-dimensional bundles of good: application to ethnic preferences at choice of residence. Symposium of the Graduate School of Decision Sciences (GSDS), Konstanz, Germany.
GESIS - Leibniz Institute for the Social Sciences (2007): German General Social Survey 2006.

2. Neighbor’s influence: Different strategies to model spatial spillover effects
Mr Tobias Rüttenauer (TU Kaiserslautern)

Regional measures of economic and social indicators have become increasingly available during the last years. These data offer the opportunity to include space as a dimension of interest and connect different indicators by their spatial proximity. However, spatial data suffer from the problem that units are not independent and identically distributed. Thus, standard regression approaches are inappropriate to correctly estimate the connections in spatial data. A variety of models from spatial econometrics address this specific problem and allow to control for spatial autocorrelation and spatial interdependence.
However, all of these models use different strategies to address the spatial nature of the data. While spatial error models (SER) incorporate a spatially lagged error term, spatial autoregressive models (SAR) incorporate a spatially lagged dependent variable. A third type of models, in contrast, includes the spatially lagged explanatory variables (SLX). In addition, these different modeling strategies can be implemented separately or in combination (e.g. SARAR or spatial Durbin). The different spatial modeling techniques, thus, use very different approaches to incorporate the spatial interdependence.
This study makes use of the full range of spatial modelling strategies to compare the consequences of the available model specifications. I use socio-demographic data from the 2011 German census and pollution data from industrial facilities to investigate the patterns of environmental inequality in Germany. These data allow to investigate the connection between minority share and environmental pollution on the level of nearly 100,000 census grid cells (of one squared km size). In addition to the influence of a unit’s own minority share on pollution, I also investigate the influence of spatial spillover effects from neighboring units. Though all models use different strategies to account for the spatial nature of the data, the results are not model dependent and indicate that neighbors do matter. The minority share of neighboring units is even more important in predicting the amount of pollution than the unit’s characteristics itself.

3. A potential pitfall in modelling the link between education and social network quality
Professor Peter Winker (Unviersity of Giessen)
Dr Marina Trebbels (University of Hamburg)

The analysis of social networks is of high relevance in different fields of social sciences. The collection of survey data on the size and quality of social networks requires substantial input, both from interviewers and respondents. A substantial number of studies have reported that interviewer effects might particularly affect the quality of data on network size, while the quality of data on measures of network quality might more strongly depend on the respondents’ cognitive abilities. One way to avoid interviewer effects is using self-administered questionnaires. This, however, comes at the cost of a stronger bias due to differences in the respondents’ cognitive abilities.

We report empirical findings for a complex instrument used in a self-administered questionnaire applied in the National Educational Panel Study (NEPS) to 9th-graders in the classroom, which was designed to measure the social resources young people have at their disposal at the transition from general into vocational education (including the network’s educational background and different social background data). The data allows identifying participants and population subgroups who face particularly strong difficulties in using the proposed instrument in a consistent way. This selection is highly correlated with the educational track the participants are attending as well as – for low levels of education – with students’ migration background. Given that also measures of social network quality are found to correlate with educational and migration background, ignoring the selection caused by the complex instrument may heavily bias estimates of the link between education or cognitive skills, migration background and social network quality.

We compare results for the nexus between education and social network quality obtained using two approaches. For the first approach, all available observations are used employing a naive procedure to correct for inconsistent answers. This might be considered the state of the art when working with the data at hand. For the second approach, the inconsistent cases are treated separately, either by simply excluding them from the analysis or by taking also the selection into this group into account.