ESRA 2013 Sessions

Attrition in Panel Surveys - Prevention and Correction 1Mr Ulrich Krieger
Focus:
In recent years, longitudinal surveys have become increasingly popular for studying change and stability in a wide variety of phenomenon within society. The availability of accurate and valid data, therefore, on change and stability is paramount. This session focuses on one of the most important sources of error and bias: drop-out or attrition.

Correction:
Differential non-response in panel surveys can lead to large errors over the course of the study as the survey estimates may become biased. Research into the nature and causes of attrition in various panel surveys typically finds that attrition often coincides with major events in the lives of respondents. In addition, fieldwork procedures, survey innovations, social and psychological characteristics of the respondents as well their survey experiences are also thought to be factors generating longitudinal non-response error. Knowledge of these correlates can serve as a basis for correction methods to correct for attrition.

Prevention:
Rather than correct for attrition, it is better to prevent attrition from occurring. Evidence for what procedures are successful in limiting attrition is however scarce. We invite contributions that assess different ways to prevent attrition, and evaluate their consequences on attrition rates, and attrition bias.

Session Details:
Examples of contributions sought for this session include but are not limited to:
• Experimental studies contrasting different survey procedures and their effect on attrition
• Examples of best practices in preventing attrition from occurring.
• Examples of studies where attrition has been corrected by post-survey adjustments
• Papers that propose new statistical methods for correcting or ameliorating the effects of attrition.
We particularly invite methodological papers that incorporate experimental data.

4th co-organizer: Dr. Jon Miller, jondmiller@umich.edu, University of Michigan

Attrition in Panel Surveys - Prevention and Correction 2Mr Ulrich Krieger
Focus:
In recent years, longitudinal surveys have become increasingly popular for studying change and stability in a wide variety of phenomenon within society. The availability of accurate and valid data, therefore, on change and stability is paramount. This session focuses on one of the most important sources of error and bias: drop-out or attrition.

Correction:
Differential non-response in panel surveys can lead to large errors over the course of the study as the survey estimates may become biased. Research into the nature and causes of attrition in various panel surveys typically finds that attrition often coincides with major events in the lives of respondents. In addition, fieldwork procedures, survey innovations, social and psychological characteristics of the respondents as well their survey experiences are also thought to be factors generating longitudinal non-response error. Knowledge of these correlates can serve as a basis for correction methods to correct for attrition.

Prevention:
Rather than correct for attrition, it is better to prevent attrition from occurring. Evidence for what procedures are successful in limiting attrition is however scarce. We invite contributions that assess different ways to prevent attrition, and evaluate their consequences on attrition rates, and attrition bias.

Session Details:
Examples of contributions sought for this session include but are not limited to:
• Experimental studies contrasting different survey procedures and their effect on attrition
• Examples of best practices in preventing attrition from occurring.
• Examples of studies where attrition has been corrected by post-survey adjustments
• Papers that propose new statistical methods for correcting or ameliorating the effects of attrition.
We particularly invite methodological papers that incorporate experimental data.

4th co-organizer: Dr. Jon Miller, jondmiller@umich.edu, University of Michigan

Continuous Time Modeling in Panel Research (N large) and Time-Series Analysis (N = 1 or small) 1Dr Johan Oud
Although virtually all processes in social reality develop in continuous time, continuous time modeling of those processes by means of differential equations is extremely rare. Scattered early attempts have been taken by the well-known scientists Herbert Simon in 1952 and James Coleman in 1968 but did not lead to much follow-up. Time-ordered causal modeling is almost always done in discrete time with the cross-lagged panel design being its most popular representative. The preference for discrete-time modeling is likely motivated by the inherently discrete-time nature of our measurements. It can be shown, however, that failing to properly account for the continuous time intervals between measurement occasions may lead to quite paradoxical and even contradictory conclusions. The session is open for all continuous-time modeling approaches in social science. Data sets analyzed may range from N = 1 and T large to N large and T small, from observation time points and intervals that are equal for all N subjects to individually varying observation intervals within and between subjects. Also welcome are contributions that discuss the different approximate and exact estimation procedures in continuous-time modeling. The trajectories analyzed may take arbitrary forms: oscillating and nonoscillating, with and without random subject effects. We especially welcome papers on substantive topics that apply continuous time modeling in their analyses.

Continuous Time Modeling in Panel Research (N large) and Time-Series Analysis (N = 1 or small) 2Dr Johan Oud
Although virtually all processes in social reality develop in continuous time, continuous time modeling of those processes by means of differential equations is extremely rare. Scattered early attempts have been taken by the well-known scientists Herbert Simon in 1952 and James Coleman in 1968 but did not lead to much follow-up. Time-ordered causal modeling is almost always done in discrete time with the cross-lagged panel design being its most popular representative. The preference for discrete-time modeling is likely motivated by the inherently discrete-time nature of our measurements. It can be shown, however, that failing to properly account for the continuous time intervals between measurement occasions may lead to quite paradoxical and even contradictory conclusions. The session is open for all continuous-time modeling approaches in social science. Data sets analyzed may range from N = 1 and T large to N large and T small, from observation time points and intervals that are equal for all N subjects to individually varying observation intervals within and between subjects. Also welcome are contributions that discuss the different approximate and exact estimation procedures in continuous-time modeling. The trajectories analyzed may take arbitrary forms: oscillating and nonoscillating, with and without random subject effects. We especially welcome papers on substantive topics that apply continuous time modeling in their analyses.

Household panel surveys: recent developments and new challengesDr Emanuela Sala
Traditionally, panel surveys have been considered powerful research resources to study how people's socio-economic circumstances change over time. Although this remains their main goal, panel surveys are facing additional challenges. These include the increasing demand to supplement survey data with administrative data; the requests from the funding agencies to collect direct measurement of health conditions; the need to implement methodological experiments to improve the quality of the data collected; the trade offs between granting certain quality standards (in terms, for example of response rates) and limited financial resources available; the increasing constraints posed by the ethical committees .
With regard to these challenges, panel surveys have to deal with methodological issues that are specific to this kind of surveys. The aim of this session is therefore to provide an overview of the methodological issues that survey practitioners have to deal with while facing these new challenges. The focus of the papers will be on household panel surveys. Examples of the topics of this session include (but are not restricted to):
- methodological issues in linking survey data to administrative data,
-experiments in questionnaire content or fieldwork procedures,
- methodological issues in the collection of biomarkers and cognitive measures,
-innovative ways of collecting data.


Longitudinal surveys โ€“ Field logistics in panel studiesDr Jutta von Maurice
The session will cover the organisation of panel studies, from tracking of respondents and sample review prior to field enumeration, the recruitment and training of Interviewers, through to field logistics, field monitoring, and reporting. The focus is on the particular challenges faced by those running panel studies such as:

- Finding effective and reliable tracking methods to find non-contacts and participants with changed life situations from previous waves prior to field enumeration;
- The value of continuing with participants that have been long-term non-contacts or refusals. For sample management and data collection, there is a lot of effort put into these groups, through tracking and interviewer f2f visits, but also a lot of effort goes into this group through instrument design (roll-forward and catch up questions);
- Developing effective engagement strategies aimed at ensuring the long term commitment of respondents of different age groups;
- Conducting standardised Interviewer training across a large interview panel and also conducting of Interviewer training each wave when a significant amount of the content remains stable, yet the Interviewer panel contains a mix of new and experienced Interviewers;
- Regular field logistics, such as starting the field and taking into account that environment and context could have changed since the last field period;
- Monitoring field progress, taking into account the length of the enumeration period, keeping track of refusal and non-contacts, and managing transitions in life paths such as from kindergarten to school or from primary school to secondary school;
- Reporting during enumeration - how often, what to report on, presentation and usefulness of reports.

Longitudinal surveys โ€“ Special challenges and innovative solutions in panel studiesDr Jutta von Maurice
The session will cover the organisation of panel studies, from tracking of respondents and sample review prior to field enumeration, the recruitment and training of Interviewers, through to field logistics, field monitoring, and reporting. The focus is on the particular challenges faced by those running panel studies such as:

- Finding effective and reliable tracking methods to find non-contacts and participants with changed life situations from previous waves prior to field enumeration;
- The value of continuing with participants that have been long-term non-contacts or refusals. For sample management and data collection, there is a lot of effort put into these groups, through tracking and interviewer f2f visits, but also a lot of effort goes into this group through instrument design (roll-forward and catch up questions);
- Developing effective engagement strategies aimed at ensuring the long term commitment of respondents of different age groups;
- Conducting standardised Interviewer training across a large interview panel and also conducting of Interviewer training each wave when a significant amount of the content remains stable, yet the Interviewer panel contains a mix of new and experienced Interviewers;
- Regular field logistics, such as starting the field and taking into account that environment and context could have changed since the last field period;
- Monitoring field progress, taking into account the length of the enumeration period, keeping track of refusal and non-contacts, and managing transitions in life paths such as from kindergarten to school or from primary school to secondary school;
- Reporting during enumeration - how often, what to report on, presentation and usefulness of reports.

Longitudinal surveys โ€“ Tracking in panel studiesDr Jutta von Maurice
The session will cover the organisation of panel studies, from tracking of respondents and sample review prior to field enumeration, the recruitment and training of Interviewers, through to field logistics, field monitoring, and reporting. The focus is on the particular challenges faced by those running panel studies such as:

- Finding effective and reliable tracking methods to find non-contacts and participants with changed life situations from previous waves prior to field enumeration;
- The value of continuing with participants that have been long-term non-contacts or refusals. For sample management and data collection, there is a lot of effort put into these groups, through tracking and interviewer f2f visits, but also a lot of effort goes into this group through instrument design (roll-forward and catch up questions);
- Developing effective engagement strategies aimed at ensuring the long term commitment of respondents of different age groups;
- Conducting standardised Interviewer training across a large interview panel and also conducting of Interviewer training each wave when a significant amount of the content remains stable, yet the Interviewer panel contains a mix of new and experienced Interviewers;
- Regular field logistics, such as starting the field and taking into account that environment and context could have changed since the last field period;
- Monitoring field progress, taking into account the length of the enumeration period, keeping track of refusal and non-contacts, and managing transitions in life paths such as from kindergarten to school or from primary school to secondary school;
- Reporting during enumeration - how often, what to report on, presentation and usefulness of reports.

Measurement in panel surveys: methodological issues 1Ms Nicole Watson
All surveys are affected by measurement error to some degree. These errors may occur due to the interviewer, the respondent, the questions asked, the interview situation, data processing and other survey processes. Understanding measurement error is particularly important for panel surveys where the focus is on measuring change over time. Measurement error in this context can lead to a serious overstatement of change. Further, recall effects of events between two interviews may lead to serious understatements of change. Nevertheless, assessing the extent of measurement error is not straightforward and may involve unit record level comparison to external data sources, multiple measures within the same survey, multiple measures of the same individuals over time, or comparisons across similar cohorts who have had different survey experiences.

Session Details

This session seeks papers on the nature, causes and consequences of measurement error in panel data and methods to address it in either data collection or data analysis. This might include (but is not limited to):
- Assessments of the nature and causes of measurement error
- Evaluations of survey design features to minimise measurement error (such as dependent interviewing)
- Examinations of the consequences of measurement error
- Methods to address measurement error in analysis.


Measurement in panel surveys: methodological issues 2Ms Nicole Watson
All surveys are affected by measurement error to some degree. These errors may occur due to the interviewer, the respondent, the questions asked, the interview situation, data processing and other survey processes. Understanding measurement error is particularly important for panel surveys where the focus is on measuring change over time. Measurement error in this context can lead to a serious overstatement of change. Further, recall effects of events between two interviews may lead to serious understatements of change. Nevertheless, assessing the extent of measurement error is not straightforward and may involve unit record level comparison to external data sources, multiple measures within the same survey, multiple measures of the same individuals over time, or comparisons across similar cohorts who have had different survey experiences.

Session Details

This session seeks papers on the nature, causes and consequences of measurement error in panel data and methods to address it in either data collection or data analysis. This might include (but is not limited to):
- Assessments of the nature and causes of measurement error
- Evaluations of survey design features to minimise measurement error (such as dependent interviewing)
- Examinations of the consequences of measurement error
- Methods to address measurement error in analysis.


Measurement in panel surveys: methodological issues 3Ms Nicole Watson
All surveys are affected by measurement error to some degree. These errors may occur due to the interviewer, the respondent, the questions asked, the interview situation, data processing and other survey processes. Understanding measurement error is particularly important for panel surveys where the focus is on measuring change over time. Measurement error in this context can lead to a serious overstatement of change. Further, recall effects of events between two interviews may lead to serious understatements of change. Nevertheless, assessing the extent of measurement error is not straightforward and may involve unit record level comparison to external data sources, multiple measures within the same survey, multiple measures of the same individuals over time, or comparisons across similar cohorts who have had different survey experiences.

Session Details

This session seeks papers on the nature, causes and consequences of measurement error in panel data and methods to address it in either data collection or data analysis. This might include (but is not limited to):
- Assessments of the nature and causes of measurement error
- Evaluations of survey design features to minimise measurement error (such as dependent interviewing)
- Examinations of the consequences of measurement error
- Methods to address measurement error in analysis.


Problems and Applications with Longitudinal Data Collection and AnalysesDr Aviad Tur-Sinai


Weighting issues in panel surveys 1Ms Nicole Watson
A range of issues arise when constructing weights for longitudinal surveys. In addition to the issues typically faced in cross-sectional surveys, we also need to consider how populations are defined over time, how to treat deaths and other out-of-scopes and how to best adjust for attrition. Household panel surveys have additional layers of complexity due to changes in the household structure, such as births, deaths, household splits, household mergers, and people moving into or out of the household. Further, when refreshment or top-up samples are added to household panel surveys, how should the additional sample be incorporated into the weights for researchers who want to use the combined sample?

This session seeks to bring together survey methodologists who are involved in constructing weights for panel surveys to explore the approaches taken for longitudinal weights and (where relevant) cross-sectional weights. Some discussion of this topic has occurred in the Panel Survey Methods Workshops in recent years, but we are looking to broaden the scope of these discussions.

Papers submitted to this session might include comparisons of alternative methods, analysis of the impact of particular components of the weights, or suggestions for new methods.