ESRA 2013 Sessions

Data Quality Management in Cross-National Surveys 1Dr Jessica Fortin-Rittberger
Achieving data of the highest quality is the aspired goal of all survey programmes. While a growing literature offers guidelines on standards for cross-national research, most of the recommendations are directed at the design phase of surveys. Meanwhile, much less attention has been allocated to the post-data collection stage so far.

Information about data processing from data cleaning to quality checks is rarely available in details and thus not always clear to researchers and analysts. Since little is known about the steps taken in data processing, we have reasons to believe that practices differ between survey programmes. This represents a unique opportunity for collaboration. Contrasting different approaches to data cleaning and quality control will help shed light on each approach's particularities, and by the same token help harness the strengths of each method through comparison of individual positive features and drawbacks. For some survey programmes, this might provide the first steps towards a more systematic approach to data cleaning and quality control.

The proposed session aims to cover this important gap by exposing in details a series of cross-national models of data cleaning and quality control processes, clarifying which procedures are used, and highlighting commonalities and dissimilarities. By confronting different methodologies and experiences of various approaches to data management, it will be possible to engage in a discussion, and most important, a critical evaluation of the processes employed by each approach: highlighting strengths, finding ways to minimize weaknesses, and suggesting strategies for possible improvements. This discussion could contribute to establishing guidelines of best practice in this field.

While the session is primarily geared towards the process of post-collection data management in cross-national programmes we also welcome papers that focus on national or subnational surveys.

Data Quality Management in Cross-National Surveys 2Dr Jessica Fortin-Rittberger
Achieving data of the highest quality is the aspired goal of all survey programmes. While a growing literature offers guidelines on standards for cross-national research, most of the recommendations are directed at the design phase of surveys. Meanwhile, much less attention has been allocated to the post-data collection stage so far.

Information about data processing from data cleaning to quality checks is rarely available in details and thus not always clear to researchers and analysts. Since little is known about the steps taken in data processing, we have reasons to believe that practices differ between survey programmes. This represents a unique opportunity for collaboration. Contrasting different approaches to data cleaning and quality control will help shed light on each approach's particularities, and by the same token help harness the strengths of each method through comparison of individual positive features and drawbacks. For some survey programmes, this might provide the first steps towards a more systematic approach to data cleaning and quality control.

The proposed session aims to cover this important gap by exposing in details a series of cross-national models of data cleaning and quality control processes, clarifying which procedures are used, and highlighting commonalities and dissimilarities. By confronting different methodologies and experiences of various approaches to data management, it will be possible to engage in a discussion, and most important, a critical evaluation of the processes employed by each approach: highlighting strengths, finding ways to minimize weaknesses, and suggesting strategies for possible improvements. This discussion could contribute to establishing guidelines of best practice in this field.

While the session is primarily geared towards the process of post-collection data management in cross-national programmes we also welcome papers that focus on national or subnational surveys.

Educational attainment in cross-national surveys: instrument design, data collection, harmonisation and analysisDr Silke Schneider
Educational attainment is of key interest for both academic research and education and social policy at the national and international levels. It is usually measured by the highest formal education certificate achieved and is one of the most used, but also most difficult to harmonise, socio-economic variables in survey research. Compared to labour market and occupational measures, the comparative measurement of education has received much less attention in sociological research and official statistics. Consequently, there is no consensus yet on how to best conceptualise and measure educational attainment: Different surveys, even within countries, implement different concepts, measurement and coding procedures.

Many surveys however try to improve their methodologies: For example, in round 5 of the European Social Survey, a new measurement procedure was implemented. The Survey of Health, Ageing and Retirement in Europe has also improved its measurement instruments in 2012. Finally, official bodies such as Eurostat and OECD are currently working hard on the implementation of the new International Standard Classification of Education (ISCED) adopted by the Unesco General Conference in late 2011 in official surveys such as the European Labour Force Survey.

There are signs of both convergence and continuing disagreement between academic and official surveys concerning the issue of education measurement. The ESRA conference 2013 is thus a good opportunity to
- reflect on the ongoing changes across surveys and stakeholders,
- review and evaluate recent changes in measurement procedures in specific surveys,
- support coordination across stakeholders, and
- work towards a cross-national standard across surveys.

This session intends to bring together researchers and practitioners working for and with different cross-national surveys or official data concerning instrument design, data collection, harmonisation and analysis of education attainment and related concepts.

EU-SILC: Some key methodological outcomes of the Net-SILC2 EU projectMr Eric Marlier
The "EU Statistics on Income and Living Conditions" (EU-SILC) covers the 27 EU countries and a number of other European countries. It is the main data source for comparative analysis and indicators on income and living conditions in the EU. Since the launch in June 2010 of the "Europe 2020" Strategy for smart, sustainable and inclusive growth, the importance of EU-SILC has grown further: one of the five Europe 2020 headline targets is based on EU-SILC data (the social inclusion EU target, which consists of lifting at least 20 million people in the EU from the risk of poverty and exclusion by 2020).

A lot of EU-SILC methodological work is being undertaken in the framework of the "Second Network for the Analysis of EU-SILC" (Net-SILC2). Funded by Eurostat for the period June 2011/ May 2015, Net-SILC2 brings together expertise from 16 European partners: the Luxembourg-based CEPS/INSTEAD Research Institute (Net-SILC2 coordinator), six National Statistical Institutes (from Austria, Finland, France, Luxembourg, Norway and the UK), the Bank of Italy and academics from 8 research bodies (in Belgium, Germany, Sweden and the UK). Two main aims of Net-SILC2 are: a) to carry out in-depth methodological work and socio-economic analysis based on EU-SILC data (covering both the cross-sectional and longitudinal dimensions of the instrument); and b) to develop common tools and approaches regarding various aspects of data production. Net-SILC2 is the successor of Net-SILC1 (Dec 2008/ Dec 2010) whose final output was a book on "Income and living conditions in Europe". This special "Net-SILC2 session" will consist of four methodological contributions prepared by some Net-SILC2 members.

European Values Study - methodological and substantive applications 1Dr Ruud Luijkx
The European Values Study (EVS) is a unique research project into Europe's basic values. First, it spans a period of almost 30 years with surveys in 1981, 1990, 1999, and 2008. Second, EVS has an extensive geographical coverage. In Europe, the survey has gradually been expanded from mostly Western European countries in 1981 to the whole of Europe in 2008. Third, even though several items have been changed in the consecutive waves, EVS still includes an impressive number of unchanged questions. Fourth, researchers have combined survey data with macro-level data so that multi-level models can be estimated. Finally, and perhaps most importantly, the questionnaires pertain to a very broad spectrum of life domains: family and marriage, economics, work, leisure, politics, religion, morality. This allows to introduce overarching concepts and to examine their effects on attitude and reported behavior in different domains. Such a rich data source also offers a unique chance for substantive and methodological investigations. We are particularly interested in papers which make use of the comparative potential of EVS from a methodological and a substantive perspective. To give a few examples: Do the measurement instruments that have been used in EVS guarantee comparability across time and space? In which domains do we observe a generational change during the last decades and in which domains are life-cycle effects more plausible. Can we reliably estimate the long-term change? Which models are suited best for that purpose? To what extent can differences between countries traced back to cultural influences? How can the latter be measured? What are the main problems of these multi-level models? However, other empirical and methodological topics are possible too.

European Values Study - methodological and substantive applications 2Dr Ruud Luijkx
The European Values Study (EVS) is a unique research project into Europe's basic values. First, it spans a period of almost 30 years with surveys in 1981, 1990, 1999, and 2008. Second, EVS has an extensive geographical coverage. In Europe, the survey has gradually been expanded from mostly Western European countries in 1981 to the whole of Europe in 2008. Third, even though several items have been changed in the consecutive waves, EVS still includes an impressive number of unchanged questions. Fourth, researchers have combined survey data with macro-level data so that multi-level models can be estimated. Finally, and perhaps most importantly, the questionnaires pertain to a very broad spectrum of life domains: family and marriage, economics, work, leisure, politics, religion, morality. This allows to introduce overarching concepts and to examine their effects on attitude and reported behavior in different domains. Such a rich data source also offers a unique chance for substantive and methodological investigations. We are particularly interested in papers which make use of the comparative potential of EVS from a methodological and a substantive perspective. To give a few examples: Do the measurement instruments that have been used in EVS guarantee comparability across time and space? In which domains do we observe a generational change during the last decades and in which domains are life-cycle effects more plausible. Can we reliably estimate the long-term change? Which models are suited best for that purpose? To what extent can differences between countries traced back to cultural influences? How can the latter be measured? What are the main problems of these multi-level models? However, other empirical and methodological topics are possible too.

Generalized Latent Variable ModelingMr Dominik Becker
Various frameworks have been proposed to overcome the shortcomings of so-called classical test theory and to address the issue of measurement error. Given certain similarities, these approaches still vary with respect to a couple of assumptions such as the metric of the latent variable (e.g. continuous in case of both Item Response Theory, IRT, and Confirmatory Factor Analysis, CFA; and categorical in case of Latent Class Analysis, LCA) or whether correlation of measurement error is allowed.

However, recently, several propositions have been published to address latent variable models in a common framework which is known as generalized latent variable modeling (Bartholomew/Knott 1999; Bollen 2002; Muthen 2002; Skrondal/Rabe-Hesketh 2007). An issue that has been addressed within that framework is the one of multilevel latent variable modeling considering measurement models at both the within and the between-level (Vermunt 2003; Raudenbush 2003, 2009; Asparouhov & Muthen 2008; Marsh et al. 2009). Moreover, also regarding the question of measurement equivalence - comparable parameter estimates of a given latent variable model in distinct groups -, respective tests of different latent variable frameworks such as multiple-group CFA and Differential Item Functioning were evaluated within a common model (e.g. Kankara et al. 2011).

Given the still limited amount of studies comparing results from different latent variable frameworks, this session cordially invites for both simulation-based and survey data-driven applications and evaluations of generalized latent variable modeling in general and their multilevel and/or measurement equivalence extensions in particular.


Issues in Cross-national Data Colection and AnalysisDr Catalina Lomos


Measurements of immigration status in cross-national surveysDr Bogdan Voicu
Most of the contemporary large-scale surveys include some measurement of respondent's immigration status. In the past, being or not being born in the country of interviewing was the only information collected. Most recent waves of EVS and ESS added questions on the actual country of birth, both for respondent and for his/her parents and spouse. Can such information be efficiently employed to study immigrants, or to control for the immigration status when analyzing the overall population? Is it accurate enough? How can one overcome the existing limitations (e.g. under-representation of illegal immigrants)? Are the measurements similar in different (European) countries? Do they lead to comparable and reliable indicators? This section plans to bring together both methodological reflections and substantial research which make use of the immigration variables existing in comparative surveys. Papers dealing using large-scale datasets are at the core of the section. Analyses based on surveys of smaller clusters of countries, including the ones targeted on immigrants, are also welcome.

Measuring Occupation Cross-nationallyDr Eric Harrison
Analysis of peer reviewed publications based on ESS and the GSS shows that socio-demographic or 'background' variables are the most widely used. Of these, occupation of respondents and where applicable, their partners and parents, is a rich source of information but one that is problematic to measure precisely, accurately and consistently.

Much effort has been invested in producing standardised coding frames for use worldwide, the relevant one here being the new International Standard Classification of Occupations (ISCO-08), but its consistent implementation across countries will present many challenges to cross-national measurement equivalence, as countries attempt to map from a) their own classifications to ISCO and/or b) ISCO88 to the new revision.

We invite methodological and substantive papers in this session. Methodology can include evaluations of the quality of existing occupational data and/or new approaches to the collection and coding of occupational data in the field. Contributions which use data that has been coded to the new revision of ISCO will be particularly welcome.




Measuring trust and quality of society in cross national surveys 1Dr Tadas Leoncikas
During recent economic downturn, trust in people as well as in institutions seems to be affected in many countries. The changes experienced by societies as an outcome of the economic crisis provide a broad background to reconsider the measurements as well as the links between the trust indicators and a broader set of quality of society measures. While trust and quality of society are contested measurements, but also because they are considered to influence a wide range of economic and political phenomena it is important to improve their methodological understanding especially when cross-national comparisons are concerned.
To improve understanding of the phenomenon of trust and quality of society, as well as to reflect on their measurements, the session invites proposals along the following lines:
- How to assess trends with regard to changing levels of trust in cross-sectional surveys?
- How to interpret the effects of different measurements of trust and quality of society in different countries?
- How measurement of general trust levels can be complemented with understanding of radius of trust?
- What implications for quality of society does trust have as evidenced by links between trust indicators and other quality of life indicators?
- Whether crisis environment is depleting trust or whether and in what circumstances it generates societal solidarity?
Paper givers are invited to send in an abstract of no longer than 800 words.



Multi-level Relationships and Social MechanismsProfessor Juergen Friedrichs
Explaining cross-national differences between macro level variables often requires to include the micro level (and possibly the meso level) in the analysis, resulting in a macro-micro-macro model comprising contextual hypotheses and aggregation based on to the "Coleman boat". These are not sufficient to describe the much more complex relationship between a country and its residents. In the following, we list major methodological problems associated with "contextual" studies:
* Number of levels: In many contextual analyses, one or more meso levels, either spatial (e. g. regions, cities, neighborhoods) or social (e.g. social networks, school classes, departments), may have to be considered, as well as mediating effects for example of the mass media.
* Which characteristic(s) of the country level to consider? If macro-micro-correlations are not caused by composition effects, often the question still remains whether the country level characteristics have to be seen as a proxy for other country features, which are probably hard to measure. For example, the positive correlation between a country's affluence and its residents' life satisfaction is well established in the literature, but the causal relationship is not: Is it really collective wealth that leads to individual well-being?
* Social mechanisms or effects: How does a given characteristic or condition transpire from the aggregate to the individual level? What exactly is a "social mechanism"- if not a set of propositions?
* Similar Effects? Do we find similar mechanisms, such as "role models" for context effects on different forms of individual behaviour?

The topics listed show that cross-national analysis still raises many methodological problems, if one goes beyond stating correlations between country characteristics and tries to explain such relationships. We invite scholars to present papers which address the above questions from a theoretical, methodological or empirical point of view.

Multilevel analysis in comparative researchProfessor Elmar Schlueter
We invite researchers to submit paper proposals for the session "Multilevel analysis in comparative research" at the 5th European Survey Research Association Conference, to be held from 15th to 19th of July 2013 in Ljubljana, Slovenia.

Point of departure for this session is the rapid growth in the analysis of survey-based multilevel data structures over the last decade. In large part, this growth has been driven by the availability of novel methodological tools, such as multilevel structural equation models and/or multilevel models for cross-classified data, to name just two examples. These innovations allow comparative researchers not only to explore previously inaccessible research questions, but also to revisit classis research topics with stronger tools. Against this background, the aim of this session is threefold:

- First, this session wants to stimulate the debate on conceptual as well as statistical issues that might arise when applying multilevel techniques in comparative research (e.g. the small-N problem, the black-box problem or Galton's problem).

- Second, this session aims to bring together studies that provide empirical comparisons of multilevel techniques with alternative approaches, e.g. hierarchical geostatistical models, fixed effect models or two-step hierarchical estimation.

- Third, we also invite papers that demonstrate how innovative applications of multilevel modeling techniques further social science understanding of substantial research problems.


The list below presents exemplary topics for contributions:


- Papers dealing with the consequences of and solutions to the small-N problem; techniques for combining mediating and moderating relationships; studies which add a spatial perspective to multilevel research

- Multilevel regression models that address substantive research problems considering hierarchical or non-hierarchical data structures

- Multilevel structural equation models

Paper proposals for this session should be sent to:

Elmar Schlueter (elmar.schlueter@wiso.uni-koeln.de)
Bart Meuleman (bart.meuleman@soc.kuleuven.be)

Quality of Life, Quality of SocietyDr Eric Harrison
The last decade has seen an increasing international focus on a number of dimensions of wellbeing. Many cross-national surveys carry measures of life satisfaction or happiness, and some invite respondents to evaluate their lives in terms of a series of domains. As a result we have data about differences in levels of wellbeing in a large number of countries. The next questions that arise are: 'How do differences among societies affect the well-being of those who live in them? Are some types of societies more successful than others at promoting individual lives and the collective development of the community How might the character of a society have such effects, and how are such societies built?' (Hall and Lamont 2009).

We invite submissions that: a) address the measurement of individual and/or societal wellbeing, b) tackle the problem of measuring the effects of 'national context', or c) present substantive analysis of data on individual or societal wellbeing in one or more countries.


Substantive studies using cross-national dataMr Dominik Becker


The Use of Probing Questions to Evaluate Items in Intercultural ResearchProfessor Michael Braun
Equivalence of measurement across countries is a necessary prerequisite for intercultural research. The traditional method to establish invariance of measurement is applying one or more data-analytic approaches. However, most of these approaches are only helpful in deciding whether measurement invariance is obtained or not (e.g., multigroup structural equation modeling) but not at getting at the causes of problems with functional equivalence.

When it comes to identifying the causes of non-equivalence, probing techniques are an ideal device. After all, they allow discovering what respondents - across countries - have in mind when answering survey questions. Knowing about potential causes of non-equivalence is an important step towards improving measurement instruments for future use in intercultural research.

For this session, we invite papers on probing in intercultural research. Such probing make take place as part of source questionnaire development or as part of translation testing. Alternatively, probing may come in as a follow-up to an already fielded survey to shed light on statistically suspicious data. Furthermore, the probing may take place during cognitive interviewing or as part of imbedded follow-up questions at the main data collection stage itself (regardless of mode). Papers are welcome on both substantive findings and on methodological challenges and considerations.