All time references are in CEST
Contemporary issues in measurement invariance research 1 |
|
| Session Organisers | Dr Daniel Seddig (University of Cologne) Professor Eldad Davidov (University of Cologne and URPP Social Networks, University of Zurich) Professor Peter Schmidt (University of Giessen) |
| Time | Tuesday 18 July, 14:00 - 15:30 |
| Room | U6-01a |
The assessment of measurement invariance of survey data is a prerequisite for meaningful and valid comparisons of substantive constructs across countries, cultures, and time. A powerful tool to test measurement invariance is multiple-group confirmatory factor analysis. In addition to testing “exact” (full or partial) measurement invariance with the traditional tools, recent methods have aimed at testing “approximate” measurement invariance using Bayesian structural equation modeling or alignment optimization, assessing clustered measurement invariance, using visualization techniques, and separating response shift bias and true change. In addition, many researchers are concerned with paying more attention to survey methodological aspects in order to advance the development of invariant measurements, such as rating scale and survey mode decisions, cognitive pretesting and web probing approaches, and cross-cultural scale adoption and translation methods. Finally, multilevel analysis and qualitative methods have been used to try to explain noninvariance.
This session aims to present studies that address questions such as “How much can we trust the above methods and related methods to test for measurement invariance?" or "What is the need to test for measurement invariance in different situations?”. We welcome (1) presentations that are applied and make use of empirical survey data, and/or that (2) take a methodological approach to address and examine measurement invariance testing.
Keywords: Measurement invariance, comparability, cross-cultural research, longitudinal research, confirmatory factor analysis, SEM
Mrs Petra Raudenská (Institute of Sociology of the Czech Academy of Sciences) - Presenting Author
Ms Radka Hanzlová (Institute of Sociology of the Czech Academy of Sciences)
Survey-based measures of subjective well-being are more and more often analysed cross-culturally. However, international comparison of these measures requires measurement invariance. The traditional tool for measuring "exact" measurement invariance is multiple-group confirmatory factor analysis (MGCFA), but this has very strict assumptions and it is difficult to achieve full comparability (scalar invariance), especially for more complex concepts and comparisons across multiple countries. In recent years, new approaches based on modern Bayesian statistics have emerged that favour testing for "approximate" invariance, which does not require the highest level of "exact" invariance to be achieved, but still meets the basic conditions for meaningful comparisons of phenomena. In this paper, we would like to present our study in which we applied both approaches in order to compare their results using the example of measuring subjective well-being. Specifically, the five-item scales from the International Social Survey Programme (ISSP) from 2011 and 2017 were used. However, these are not typical standardized scales, but the items were selected by us to capture the multidimensionality of the subjective well-being. The Bayesian approach detected in both scales two non-invariant items that were problematic for cross-national comparison and could be dropped from the scales, resulting in the testing of two three-items scales. Consequently, measurement invariance was established in all countries for the reduced scales, allowing researchers to meaningfully compare their latent mean scores and the relationships with other theoretical constructs of interest. A sub-objective of the study was also to highlight the importance of measuring subjective well-being using multiple indicators rather than just one question on happiness or life satisfaction and in general the necessity of measurement invariance testing in subjective well-being research.
Ms Radka Hanzlová (Institute of Sociology of the Czech Academy of Sciences) - Presenting Author
Measuring wellbeing is an important topic for many disciplines. As there is no clear definition of wellbeing, it is very difficult to measure wellbeing correctly and appropriately. It is most often measured using a single question on happiness or life satisfaction, which is neither methodologically nor theoretically ideal. Therefore, multidimensional measurement have to be used. In my paper, I will focus on a new theoretical model for measuring subjective wellbeing that was introduced in Round 6 of the European Social Survey (ESS). This model consists of 35 items divided into six dimensions (evaluative wellbeing, emotional wellbeing, functioning, vitality, community wellbeing, and supportive relationships). To be able to reliably compare the level of wellbeing between countries, the model must achieve a certain level of invariance. The analyses carried out showed that this model is not of sufficient quality, and cannot be used for cross-cultural comparisons. Therefore, the model must be modified. The modification process based on exploratory factor analysis resulted in the "Overall model of wellbeing", which is composed of 22 items divided into 5 dimensions (relationships, engagement, vitality, meaning and purpose, and emotions). For measurement invariance of the “Overall model of wellbeing” I will apply two methods: the traditional multi-group confirmatory factor analysis (MGCFA) testing “exact” measurement invariance and the newer alignment method testing “approximate” measurement invariance. The results of the MGCFA showed that partial scalar invariance was achieved in 20 countries out of the 28 participating in ESS 6. Testing using the alignment method revealed a relatively high proportion of total non-invariant parameters (28.9%). However, the results are reliable for N > 1000, which is a typical feature of ESS surveys, and thus it can be concluded that the data are approximately invariant and allow comparisons of the level of wellbeing across all 28 countries.
Mrs Amelie Nickel (University Bielefeld )
Dr Wiebke Weber (LMU Munich) - Presenting Author
While the European Social Survey (ESS) is one of the most widely used surveys for cross-national research on attitudes towards migration, only few studies have evaluated whether the used measurements are comparable across countries and over time. Those that did used different methods and analytic strategies which complicate the comparison of their results. To ensure comparability of results, we employ Multigroup Confirmatory Factor Analysis (MGCFA) and test for measurement invariance of attitudes towards immigration with the same approach in each round of the ESS.
Our results reveal that metric invariance holds for all countries but one in all rounds, indicating that covariances and regression coefficients can be compared meaningfully. While scalar invariance only holds for different subgroups of countries within each round, partial invariance is fulfilled in all countries, meaning that at least one indicator is equal for all countries allowing for latent mean comparisons. Moreover, we estimate the measurement quality and find that the attitudes towards immigration index is similarly good across the different countries and rounds.