ESRA 2017 Programme

Tuesday 18th July      Wednesday 19th July      Thursday 20th July      Friday 21th July     

     ESRA Conference App

Tuesday 18th July, 14:00 - 15:30 Room: Q2 AUD2

Push2web surveys: How to encourage people to go online when using a different contact mode? 1

Chair Ms Gerry Nicolaas (Ipsos MORI )
Coordinator 1Dr Patten Smith (Ipsos MORI)

Session Details

The term ‘push-to-web’ was first used by Don Dillman and colleagues to describe surveys that use traditional modes of contact to encourage people to go online and complete a web questionnaire. This design may or may not offer alternative modes of data collection but, if so, only in subsequent contact attempts among those who failed to complete the web questionnaire. In recent years we have observed increasing use of push2web survey designs for random probability surveys of the general population, including surveys that produce official statistics. Furthermore, this design has recently been used for the 2015 Japanese Census and it is envisaged that a similar design will be used for the UK 2021 Census.

The main challenge for push2web surveys is obtaining an acceptable web response rate. Even among populations with high internet access, web response rates tend to be much lower than surveys using traditional data collection modes (Lozar Manfreda et al., 2008). It would seem that the extra effort to go online and to follow instructions for accessing and completing a web questionnaire acts as a strong disincentive to participation. Offering an alternative mode to web non-respondents is an effective method for boosting the final response rate but some studies have shown that equivalent (Millar and Dillman, 2011) or even higher response rates (Messer and Dillman, 2011; Lynn, 2013) can be achieved when the alternative mode is offered on its own without the option of completing the questionnaire online. Furthermore, alternative modes are more expensive and their use will reduce potential cost savings significantly. This is particularly true when the web response rate is much lower than the final response rate.

The aim of this session is to explore a variety of methods for boosting web response rates when contact has to be made using traditional modes such as postal, telephone and face-to-face. The session welcomes papers on contact strategies, incentives, and mode sequencing. Papers exploring how smartphones can be utilised to increase web response rates in push2web designs are also welcome. We also invite papers on methods for encouraging participation in such a way that aims to enhance sample representativeness and reduce the risk of nonresponse bias.

Paper Details

1. The worldwide increase in use of web-push methods that start with a mail contact; what have we learned and where might we be going?
Dr Don Dillman (Washington State University)

In the last ten years there has been a large increase in the use of web-push methods, whereby mail contact is used to request a web response, with the possibility of responding by other modes such as mail, telephone and in-person being offered later. Use of this methodology makes it possible to obtain response rates much higher than those that can be obtain by telephone or email contact alone, with most of the response coming over the Internet. In this paper I will review the development of this methodology and describe different ways that web-push surveys are being implemented, drawing from surveys done in Europe, North America and Asia. I will also discuss the advantages of this methodology as well as the considerable challenges that must be overcome for it to effective. The nature of these challenges range from questionnaire construction and question wording to overcoming the legitimate fears many people have over clicking on and responding to web surveys.

2. Push2web or less is more? Experimental evidence from a mixed-mode population survey at the community level in Germany
Dr Robert Neumann (Technische Universität Dresden)
Professor Michael Häder (Technische Universität Dresden)
Mr Oliver Brust (Technische Universität Dresden)
Mrs Elisabeth Dittrich (Technische Universität Dresden)
Mr Hagen von Hermanni (Technische Universität Dresden)

The ‘Tailored design’ method (Dillmann et al. 2014) does not just seek to improve response rates, but also to alleviate the resulting non-response bias. Offering a tailored response option for respondents also provides the additional benefit of a substantial reduction of survey costs, considering the cost differential between an online and a paper responses. Yet, offering the web option implicitly relies on the assumption of equal and complete access to the internet and corresponding skill sets among respondents. Millar & Dillman (2014), who “conducted two experiments within a population that has complete access to the Internet and is believed to be highly Web literate” (Millar & Dillman 2014: 2) therefore covered only a (still) special case and found that offering both response option may not be a first best choice to achieve higher response rates. Do these results hold for samples of the general population as well?

To gain insights on this question, we systematically varied the postal recruitment process of a large mixed-mode survey we conducted in the fall of 2016 in the federal state of Saxony in Germany. We applied a sequential design similar to Millar & Dillman (2014) for the invitation of 8000 citizens to a community survey on waste disposal and environmental behavior. In the first invitation wave, we randomly allocated 40% of the respondents to paper-only survey, another 20% to a web-only and the final 40% to mixed-mode design with both response options. The follow-up consisted of a ‘Thank You’-postcard after about two weeks and a second invitation wave (completely) in mixed-mode design, offering both response options. We will present results of the competing designs on the overall response as well as a non-response analysis, based on register characteristics.

3. Transformed respondent engagement strategy; an overview of the research at ONS to develop respondent materials
Miss Laura Wilson (Office for National Statistics)
Miss Sophie Nickson (Office for National Statistics)

With the rise of available web services more generally, the public have come to expect to be able to engage with government via digital channels. The UK government is very aware of this demand and has in turn established the Government Digital Service (GDS) which is leading the digital transformation of government. The Office for National Statistics (ONS) is investing in the introduction of an online mode to its existing Social Surveys, defaulting to an online first approach. For social surveys this vision of a future data collection model presents substantial research and survey response challenges.

All social surveys in the UK are voluntary and in the absence of an address register achieving response is heavily reliant upon the rapport built between an interviewer and a potential respondent in a face-to-face interaction. The online first approach removes the interviewer step from the respondent journey in the first instance. Therefore the role of the letter inviting the respondent to take part becomes paramount. ONS is investing heavily in researching how best we can create this strong connection with a respondent in an online self complete approach by reviewing and radically redesigning our respondent materials starting first with the envelop (to break down our initial barrier to response). We are also considering the value of taking a behavioural economic approach, promoted with considerable success of the UK's Behavioural Insight team. In doing this we are applying four main principles to any interaction - make it Easy, Attractive, Social and Timely.

This is a long-term research strand and this talk will share with you our approach to this research, including an overview and update of the most recent findings from the traditional research methods such as in-depth interviewing and focus group with novel methods such as guerrilla pop-up testing to learn what engages and motivates respondents. This presentation will also set out plans for ongoing research and the context within which this work is being taken forward. It will discuss some of the challenges faced to date and those that we foresee to be on the horizon. We invite discussion and encourage others to share their experiences and recommendations with us.

4. Quantitative testing of the most effective advance communication strategies for a mixed mode (including online) UK Labour Force Survey
Mr Andrew Phelps (Office for National Statistics)

With the rise of available web services more generally, the UK public have come to expect to be able to engage with government via digital channels. The UK government is very aware of this demand and has in turn established the Government Digital Service (GDS) which is leading the digital transformation of government.

Over the last decade ONS has conducted numerous qualitative and quantitative investigations into online questionnaire development for Social Surveys and successfully implemented an online completion mode for the 2011 Census. Previous research and past experience has highlighted the need for an extensive research programme to support successful introduction of online data collection, looking at the short and long term opportunities and goals.

The Data Collection Transformation Programme at ONS will transform ONS data collection into a more dynamic and efficient model, maximising the use of non-survey data sources and digital by default collection of data in the production of National and Official Statistics. For Social Surveys this means that ONS needs to deliver a redesigned social statistics model, as well as transformed end-to-end systems.

ONS conducts a number of UK household surveys that feed into a number of important National Statistics indicators. The largest survey that ONS conducts is The Labour Force Survey (LFS) (and sample boost), which is the largest UK household survey, resulting in around 400,000 productive household interviews per annum. The survey is used to produce a range of high profile cross-sectional and longitudinal labour market and Annual Population Survey datasets that are widely used for analysis and publications in the UK and Europe, including for the monthly estimates of labour market supply (including estimates of change in the employment and unemployment rates). The survey is currently governed by European Regulations.

Currently the survey is being redesigned to enable multi-mode data collection and specifically to enable on-line data collection. Qualitative development methodologies have been used by survey methodologists at ONS to elicit the best advance survey documentation and LFS question wording to suit online collection.

ONS is now quantitatively testing the most effective advance documentation strategies via a series of large scale online surveys using revised LFS question wording. The testing work will establish the optimal way of introducing an online LFS survey to the general public.

This presentation set out plans for ongoing quantitative research. It will discuss some of the research findings and challenges faced to date and those that we foresee to be on the horizon. We invite discussion and encourage others to share their experiences and recommendations with us.

5. The 2016 Canadian Census: An Innovative Wave Collection Methodology to Maximize Self-Response and Internet Response
Mr Patrice Mathieu (Statistics Canada)

Every five years, Statistics Canada conducts a census to provide population and dwelling counts for Canada as well as a wide range of information about Canada’s demographic, social and economic characteristics. The collection methodology has considerably evolved over the years in order to reduce respondent burden, improve the efficiency of the Census Program and data quality. Building upon insight drawn from previous research and censuses, the 2016 Canadian Census achieved one of its best collection response rates, and more importantly, the best Internet response rate since the Canadian Census’ inception.

The Wave Methodology used for the 2016 Census collection consisted in a series of mailings to the dwellings in order to stimulate self-response, in particular Internet response. This paper will present an overview of the methodology as well as collection results. Statistics Canada is at the forefront of the use of internet-based data collection tools and results of ongoing investigations on paradata analysis to better understand the respondents’ behavior will also be presented.