ESRA logo
Tuesday 14th July      Wednesday 15th July      Thursday 16th July      Friday 17th July     




Tuesday 14th July, 11:00 - 12:30 Room: O-106


Assuring Measurement Quality in the Social Sciences – new standards for quality documentation 1

Convenor Professor Beatrice Rammstedt (GESIS - Leibniz Institute for the Social Sciences )
Coordinator 1Dr Natalja Menold (GESIS - Leibniz Institute for the Social Sciences)
Coordinator 2Dr Constanze Beierlein (GESIS - Leibniz Institute for the Social Sciences)

Session Details

Conclusions drawn from survey data can only be reliable if the survey instruments (e.g. survey questions and questionnaires) are of sufficient quality. Based on the Total Survey Error approach accessing and documentation of measurement quality in surveys will be addressed. The aim of the session is to evaluate existing standards and current survey practices with respect to the measurement error in surveys and to discuss advantages and limitations as well future developments. Researchers are invited to submit papers dealing with standards and practices in assessment and documentation of measurement quality in surveys, such as assuring the validity of a measure as well diminishing non-systematic, systematic, and processing errors.

Paper Details

1. Assuring Measurement Quality in the Social Sciences - new standards for quality documentation
Professor Beatrice Rammstedt (GESIS - Leibniz Institute for the Social Sciences)
Dr Natalja Menold (GESIS - Leibniz Institute for the Social Sciences)
Dr Constanze Beierlein (GESIS - Leibniz Institute for the Social Sciences)

Conclusions drawn from survey data can only be reliable if measures used are of sufficient quality. In 2012, the German Data Forum established an expert group to work on common quality standards for measurement instruments. Based on the Total Survey Error approach (Groves et al., 2004), six quality standards were derived. These quality standards address issues such as assuring the validity of a measure as well diminishing non-systematic, systematic, and processing errors. We introduce these quality standards for survey instruments and contrast it with existing alternative standards, present examples of instrument documentations and discuss advantages, limitations and future developments.


2. Reliability Estimation Methods
Mr Daniel Danner (GESIS – Leibniz-Institute for the Social Sciences)

The reliability coefficient describes the systematic variance proportion of a measured variable. The reliability is important because descriptive statistics or association between variables can only be estimated accurately if observed variables are reliable. There are several methods for estimating the reliability such as the test-retest correlation, Cronbach’s alpha, the maximal split-half coefficient, or structural equation models. Usually, different methods reveal different reliability estimates because they make different assumptions about the underlying measurement models. I will discuss different estimation methods, their underlying assumptions, and which method is most appropriate for which data.


3. Survey Quality Prediction (SQP 2.0) a tool to quantify survey quality
Professor Willem Saris (UPF)
Dr Daniel Oberski (Tilburg University)
Dr Melanie Revilla (UPF)
Mrs Diana Zavala Rojas (UPF)

Quality standards in survey research are in theory established: a good survey shall minimize the Total Survey Error. However, it is difficult to quantify this because it is linked to many decisions taken when a survey is developed. In this presentation we describe the approach behind the software “Survey Quality Predictor “(SQP 2.0) which aims to fill the gap in methodological research by being a tool that quantifies the quality of a question in a survey. SQP 2.0 is a tool to improve survey quality at the stage of questionnaire design and after data collection.



4. Implementation of quality management models and strategies for documentation in CBS
Mr Mario Gavrić (IASS)

Croatian Burreau of Statistics accepted Total Quality Management approach as the general model for quality management, quality assessment and quality improvement.
Further, CBS established database of quality information which is planned to become a key tool for quality assessment, quality documentation and quality reporting for CBS surveys. The database contains the exhaustive list of quality information, which is based on two two widely accepted ESS structures, ESMS and ESQRS. There is approximately 100 items included in the database which can be divided in two parts: numerical information (quality indicators) and descriptive (textual) information.