Tuesday 16th July
Wednesday 17th July
Thursday 18th July
Friday 19th July
Download the conference book
Weighting: approach and sources 1 |
|
| Convenor | Mrs Kim De Cuyper (GfK EU3C) |
| Coordinator 1 | Mrs Sara Gysen (GfK EU3C) |
There are numerous reasons why a given sample may not be representative. The main reasons for this are uncontrollable deviations from randomness which may arise from numerous sources: systematic non response, deficient address material, interviewer bias, heterogeneous contact probabilities, etc. Moreover, it will rarely occur that the raw data 100% match the population, especially since data validation and data cleaning also impact this. To eliminate bias as much as possible and to correspond to the population, weighting is applied. Weighting is a complex process and goes through various phases in which the approach towards weighting as well as the sources are key and determine the validity of the weighting.
In terms of approach a distinction can be made between:
- types of weight: probability or design, post-stratification or non-response, national, population, etc.
- single stage or multi stage weighting
- variables to weight on
- method: iterative proportionate fitting (IPF, a.k.a. Rim Weighting or Raking), linear weighting, etc.
- software package: SPSS, Quantum, etc.
- trimming
- imputation of missing data
The sources for weighting are utmost important and can divided in two main groups
- public vs private data
- type of data
o individuals vs households in consumer surveys
o workers vs establishments (in -or excluding group structures) in business surveys
A well designed weighting procedure results in a high weighting efficiency and a high effective sample percentage.
Therefore this session invites papers looking into one or more of the different weighting angles as per above. Paper givers are invited to send in an abstract of no longer than 1000 words.
CoRolAR (Continuous Rolling survey on Addictive behaviors and Risks) is a Swiss national dual-frame study where about 11'000 phone interviews are conducted yearly. In this talk we will discuss several weighting issues related to nonresponse and the complex study design.
A first emphasis will be on a careful choice of the design weights that appropriately map the inclusion probabilities influenced by the dual-frame approach (mobile/landline) and a boost among young adults (Häder et al., 2012).
Furthermore, we will review various ways to facilitate post-stratification with respect to given auxiliary variables (i.e. known population totals) including, e.g., raking, calibration, calibration with bounded weights and generalized raking (Särndal, 2007). A comparison of the different probability weighting approaches with respect to the size of weights and appropriate weighting efficiency measures will be conducted.
The main focus will be on the analysis options offered by the R package survey and exemplary R code will be shown (Lumley, 2004). Additionally, ready-to-use functions with similar capabilities offered by other software solutions will be discussed.
All considerations will be illustrated using data from the CoRolAR study 2012.
Literature:
Häder S., Häder M. & Kühne M. (2012), Telephone Surveys in Europe, Springer.
Lumley T. (2004), Analysis of complex survey samples, Journal of Statistical Software, 9 (8).
Särndal C.-E. (2007), The calibration approach in survey theory and practice, Survey Methodology, 33 (2), 99-119.
The European Foundation for the Improvement of Living and Working Conditions (Eurofound) carries out three recurring Europe-wide surveys: the European Working Conditions Survey (EWCS), the European Quality of Life Survey (EQLS), and the European Company Survey (ECS). An approach to weighting has evolved involving the consecutive construction of three types of weights, correcting for (1) deviations from equal selection probability due to survey design, (2) observed deviations between sample and universe, and (3) differences between countries in the sampled proportion of the population. In the EQLS, the design weight corrects for unequal selection probabilities due to sampling individuals using a household frame. The post-stratification weight at the national level incorporate this design weight, and was calculated by Iterative Proportional Fitting (IPF) using Quantum software, which allowed the adjustment of deviations between sample and universe on multiple dimensions, including age, gender, urbanisation level, region and household size. Finally, the cross-national weight incorporates the post-stratification weight, and corrects for the different ratios between sample size and population size in each country. This paper focuses on the implications of this approach for some key distributions in the 3rd EQLS, while addressing questions such as the number and type of weighting variables, the availability and timeliness of data sources, and the methods and implications of weight trimming.
Register information as income, education and work opens up, together with demographic information, a unique opportunity to adjust for non-response. The reduction of bias with weighting for non-response has an effect on the statistical uncertainty, which is increased corresponding to the variation in the weights. However, there is also a reduction of the statistical uncertainty corresponding to the degree of explained variation in relation to the variable measurement. This benefit are difficult to measure without having access to the entire register, but it is possible to make estimates of this with repeated weighting which can be handled by standard programmes.
The aim of this presentation is to compare the two weighting methods normally applied in order to reduce the consequences of non-response bias: 1) post-stratification weighting (PSW), 2) propensity score adjustment (PSA).
A quick review of both weighting procedures will be followed by an illustration of their methodological properties on the basis of selected empirical analysis. An examination will follow focused on the impact of refuses to the question about the income on the accuracy of point estimation of mean income. Firstly, using the ESS data set, it will be demonstrated that the likelihood to refuse is not random, but rather proportional to the level of income. Secondly, based on the data from the "Polish General Social Survey", an assessment will be provided of whether PSW or PSA leads to lower total survey error (TSE). By removing the known values of income I will consider three patterns of missingness: (a) the random one, (b) the systematic one without 10% of the lowest income values and (c) the systematic one without 10% of the highest values.
Findings are four-fold: (1) PSA is much more effective when missingness mechanism is systematic, however PSW is slightly more effective when non-response is random; (2) PSA increases variance a little bit more than PSW, but (3) PSA decreases bias much more efficiently than PSW. Taking (2) and (3) together, it turns out that (4) PSA estimator seems to be better on the ground that it implies much smaller TSE.