ESRA 2017 Programme
|ESRA Conference App|
Wednesday 19th July, 16:00 - 17:30 Room: N AUD5
Third-Party Presence in Face-to-Face Surveys
|Chair||Ms Magali Rheault (Gallup )|
Session DetailsThe need to conduct face-to-face interviews in a private setting cannot be overemphasized. It is critical to protect the respondent’s privacy, reduce social desirability bias and minimize measurement error. Interviewers’ training usually include a discussion about the importance of confidentiality and the need to ask for a private setting in which to conduct the interview. Field observations show that interviewers vary greatly in how they request such a private setting and often times, one or more relatives (or even neighbors) want to listen to the interview. Even after extensive training, interviewers may not be able to conduct a one-on-one interview with a respondent, due to local customs and norms. For example, in many countries in sub-Saharan Africa, it may be inappropriate for a young female interviewer to be by herself with an older, married male respondent. Further, parents or custodians may insist they be present when their adolescent child (between the ages of 15 and 18) is the respondent. Surprisingly, scant empirical data are available to measure the effect of third-party individuals during face-to-face interviews.
This proposed research session seeks to achieve two main objectives. First, the session would present innovative methods researchers have used in recent years to strengthen interviewers’ skills and techniques during training. What metrics and tools are being used to monitor interviewers’ implementation of the privacy standard? In addition, what guidelines have researchers developed in settings where the privacy standard cannot be implemented due to local customs? The second objective of this research session is to present the latest empirical results about the effects of third-party presence in survey data. More specifically, it would be useful to have quantitative examples from a variety of settings (developed and developing countries) and types of questions (general and sensitive). This proposed session seeks to raise awareness about the importance of respondents’ privacy and increasing knowledge about the successful implementation of such a standard in a face-to-face setting.
Paper Details1. Effects of partner presence on responses to questions on the division of household labor
Dr Jette Schröder (GESIS - Leibniz Institute for the Social Sciences)
Dr Claudia Schmiedeberg ( University of Munich (LMU))
It has long been acknowledged that face-to-face interviews should ideally involve only interviewer and respondent, without the presence of others. Third party presence may compromise data quality as respondents may refuse answers to sensitive questions or questions relating to the present person or tailor their answer to the present person. However, research on third party effects is scarce and results are mixed. A weakness of most of these analyses is that they are based on cross-sectional data and thus results may be biased by unobserved heterogeneity.
In our presentation, we focus on the effect of partner presence on responses to questions on the division of household labor. Existing literature shows such effects. Results might, however, be biased due to unobserved heterogeneity. If third party presence depends on unobserved factors also affecting survey responses, over- or underestimation of third party effects may be the result. The spouse might for example be more likely present during the interview in more conservative couples so that responses on items measuring the division of labor may not (only) be affected by spousal presence but also depend on their (unobserved) conservative values associated with a more conservative division of labor. A positive estimation bias of the effect of partner presence on own share of household labor reported by women would result as well as a negative bias of the effect of partner presence on own share of household labor reported by men.
We use the first seven waves of the German Family Panel (pairfam), a large, nationwide randomly sampled panel study with annual interviews, to investigate whether respondents’ answers to five items regarding the division of household chores are affected by the presence of the partner. To rule out that estimates of the effects of partner presence are biased by time time-constant unobserved heterogeneity we apply fixed-effects models which take advantage of intra-individual changes of partner-presence over the waves. The analysis includes more than 4,500 respondents who had the same partner in at least two panel waves. In approximately 30 % of the partnerships the partner was present in at least one of the seven panel waves (and not present in another wave).
2. An analysis of third party presence/intervention in Spanish social and political attitudes surveys
Dr Mónica Méndez Lago (Centro de Investigaciones Sociológicas)
The Centro de Investigaciones Sociológicas is a public institution that carries out around 40-50 political and social attitudes face to face surveys every year, including a monthly barometer. Most of its surveys are face-to face. Since 2008, all of its surveys include a question addressed to interviewers to check whether there are third-persons persons present and whether they actively participate in any way during the survey.
In the paper/presentation the effect of third parties being present during survey interviews will be analyzed at three different levels: survey, question and individual. The first exercise will consist on an exploratory analysis on the incidence of third party presence/interference, to see its evolution in time and to check on whether some pattern can be discerned. The main topic of each of the surveys will be coded in order to measure the extent to which third party presence is associated with the type of survey topic, and the potential influence of other survey level characteristics, such as interview duration or place where the interview takes place (house door, indoors, etc.) will also be analyzed.
The second part of the paper/presentation will look at the impact of third party presence/intervention depending on type of question and to the individual characteristics that might make third party presence more likely (socioeconomic status, household composition, among others). The data from the face-to-face Spanish General Social Survey (Encuesta Social General de España, ESGE 2013 edition), with a full probability sample of 5,094 respondents will be used for this purpose. It is a good survey to test the effect of third parties presence since its questionnaire contains both a large set of “factual” questions about the interviewee, his/her partner and family composition and also some sensitive questions about ideology and vote recall, whose results are more prone to be influenced by third party presence. This combination of questions is thus a good opportunity to check whether findings on previous analysis of third party presence/interference are applicable in Spain both regarding “factual” and sensitive questions (Aquilino 1993, Aquilino&Loscuito 1990, Smith 1995, Evans et al 2011). This survey´s questionnaire also contains a couple of more detailed questions about third parties that find out about presence, actual interference, and an additional question that inquires about who that person was in particular.
Aquilino, W. S. (1993): “Effects of spouse presence during the interview on survey responses concerning marriage”, Public Opinion Quarterly, 57, 358 –376.
Aquilino, W. S., & LoScuito, L. A. (1990): “Effect of interview mode on self-reported drug use”, Public Opinion Quarterly, 54, 362–395.
Evans, Ann, Gray, E. & Reimondos, A. (2011), “Relationship reports: The effect of presence of others during the interview”, paper presented at the HILDA conference 14th July 2011 Ann Evans, Edith Gray and Anna Reimondos
Smith, Tom W. (1995): The impact of the presence of others on a respondent´s answers to questions, NORC GSS Methodological Report 86.
3. Third Party Presence Measurement of Respondent and Interviewer Predictors of Third Party Presence in Jordan
Professor Zeina Mneimneh (University of Michigan)
Ms Julie de Jong (University of Michigan)
Professor Mansoor Moaddel (University of Maryland)
The presence of a third party during the survey interview generates an important contextual factor that could affect the reporting of sensitive information (Aquilino, 1997; Casterline and Chidambaram, 1984; Pollner and Adams, 1994). In spite of being instructed to conduct interviews in a private setting, interviewers can find it difficult to establish complete privacy. Previous research has found significant variation in the privacy achieved across interviewers (Mneimneh et al., 2013, 2015). While some interviewers have greater success in limiting the presence of others during the interview, others do not. Less understood are predictors of such interviewer variation and how they affect the association between third person presence and the reporting of sensitive information. The effects of third person presence on reporting sensitive information is also dependent upon the accuracy of recording of such presence by the interviewer. Generally, the presence of a third party is noted by the interviewer at the end of the survey interview. However, these observations may be subject to measurement error. Previous literature is silent about the quality of such observation measures.
In this presentation, we address these gaps in the literature and extend the research on third party presence during face-to-face interviews by using data from a recent face-to-face survey of a nationally representative sample in Jordan. The questionnaire collects information on a number of sensitive items related to political and religious attitudes. Information about third person presence came from observations recorded multiple times during the interview, at the end of a number of questionnaire sections, as well as at the end of the questionnaire as an overall measure. The end of the interview measure is the standard method used across surveys for collecting observation on third party presence during the interview. In addition to the commonly used measures of interview privacy (who was present and duration of presence), detailed data were collected on the specific situation that lead to the presence of others during the interview. Moreover, interviewers’ socio-demographic characteristics, including gender, age, education, religious attire (the presence of a veil for females), and previous interviewing experience were recorded. Additional measures related to interviewer’s attitudes toward privacy in general and towards obtaining privacy in the survey interview were collected using a self-administered questionnaire completed by all interviewers prior to data collection.
Using these data, this investigation extends the literature by exploring the quality of third party presence observations. Section-specific measures will be compared to the standard end of the interview measures. Predictors of each of the two types of measures (including interviewer-level variables) will be explored. Finally, the predictive power of each of the types of measures will be tested by investigating their effect on reporting sensitive information.