Unanticipated Effects of Drive to Increase Response Rates |
|
| Chair | Dr Stephanie Eckman (RTI International ) |
| Coordinator 1 | Ms Kristen Himelein (World Bank) |
The presentation is focused on how the additional efforts in reaching the reluctant respondents translate into response rates and coverage ratios. Respondents from the 7th round of the European Social Survey conducted in Poland were divided into two categories: ‘reluctant’ (using criteria presented in the short abstract), and ‘non-reluctant’. Among the ‘non-reluctant’ group nearly 75% of interviews were completed, which corresponded to the response rate of 60% for the entire sample. In the ‘reluctant’ group, only 33% of interviews were completed, what gave the 6% increase in response rate.
Next, we created two data files: first, which included the interviews from ‘non-reluctant’ respondents only, and second, merging data from both groups. The first corresponds to surveys where data come from the respondents who are relatively easy to contact before closing the fieldwork. The second, in turn, corresponds to surveys in which much more is done to conduct as many interviews as possible. We compared gender and age distributions in both data files, but the differences were not statistically significant. However, it was not the case for size of community. From this point of view, the population of Poland can be divided into three categories, approximately similar in their sizes: rural areas; small and medium towns below 100,000 inhabitants; and big cities over 100,000 inhabitants. The ‘extra’ interviews with reluctant respondents significantly increased the share of respondents living in big cities.
The reluctant respondents are particularly prone to refusals and generally require more interviewers’ visits, what may extend the necessary time of contacting respondents over a term of closing the fieldwork. In Poland, the scheduled duration of the ESS fieldwork is rather long (2-3 months) and can be extended in individual cases, for example where the respondent works abroad and is rarely available at home. Compared to other countries, Poland has a very high response rate in ESS, but it requires a substantial extension of time in the field. For ‘non-reluctant’ group the average time needed for successful interview is 26 days from the start of the fieldwork, and in ‘reluctant’ group 51 days.
In this paper, we analyze how the number of e-mail reminders influences the response rate on the second stage of a two stage household travel survey and whether this might negatively impact key measures of the survey.
The German household travel survey will include interviews with approximately 270,000 persons living in about 135,000 households.
The survey consists of two consecutive parts: An introducing household interview is designed to collect basic information about the household, its members, and the respective car fleet. It is complemented by individual person/trip interviews to collect more detailed information about each household member as well as detailed trip information for the respective travel day.
Data collection on household and person level is conducted via telephone, paper or online questionnaires. Respondents can switch the survey mode between the household and the personal level.
Due to the peculiarities of a household travel survey with different modes, a temporal gap of seven to 14 days is needed between household- and person-level-interview. During this period, respondents are informed about their travel day and receive survey materials respectively the paper questionnaire.
A key goal of our design is to achieve a low non-response rate respectively a high transition rate between household and person level of the survey. E-mail re-minders are useful tools to increase the response on the second stage, if the e-mail adress is known. Especially as e-mail reminders are a cost-effective method.
For households choosing online as their mode for the second stage, we started the survey with two e-mail reminders. One reminder was sent the day before the travel day and the second reminder was sent the day afterwards. Households were then able to complete the questionnaire within 14 days of the travel day. To increase the transition rate between the household and person level, two additional reminder e-mails were introduced over time for households. A third e-mail reminder was sent after four days, if the household was incomplete on the person level. A fourth reminder was introduced on a later stage and sent after nine days.
As the travel day and thus the field time of each household was randomly dis-tributed over the gross sample, it is also randomly assigned whether a household will receive two, three or four e-mail reminders. Thus we can interpret the effect of each additional e-mail as causal.
In this paper we will answer the following research questions:
a.) How much does each additional e-mail increase the transition rate from household to personal level?
b.) Does each additional e-mail increase the risk of break-offs?
c.) Does an additional e-mail reminder has a negative effect on key variables like number of reported trips and trip details?
We expect that the transition rate increases with each additional e-mail remind-er. However, as the reporting of trips can be cognitively demanding, we expect higher break-off-rates and. lower data quality on