Push2web surveys: How to encourage people to go online when using a different contact mode? 1 |
|
| Chair | Ms Gerry Nicolaas (Ipsos MORI ) |
| Coordinator 1 | Dr Patten Smith (Ipsos MORI) |
In the last ten years there has been a large increase in the use of web-push methods, whereby mail contact is used to request a web response, with the possibility of responding by other modes such as mail, telephone and in-person being offered later. Use of this methodology makes it possible to obtain response rates much higher than those that can be obtain by telephone or email contact alone, with most of the response coming over the Internet. In this paper I will review the development of this methodology and describe different ways that web-push surveys are being implemented, drawing from surveys done in Europe, North America and Asia. I will also discuss the advantages of this methodology as well as the considerable challenges that must be overcome for it to effective. The nature of these challenges range from questionnaire construction and question wording to overcoming the legitimate fears many people have over clicking on and responding to web surveys.
The ‘Tailored design’ method (Dillmann et al. 2014) does not just seek to improve response rates, but also to alleviate the resulting non-response bias. Offering a tailored response option for respondents also provides the additional benefit of a substantial reduction of survey costs, considering the cost differential between an online and a paper responses. Yet, offering the web option implicitly relies on the assumption of equal and complete access to the internet and corresponding skill sets among respondents. Millar & Dillman (2014), who “conducted two experiments within a population that has complete access to the Internet and is believed to be highly Web literate” (Millar & Dillman 2014: 2) therefore covered only a (still) special case and found that offering both response option may not be a first best choice to achieve higher response rates. Do these results hold for samples of the general population as well?
To gain insights on this question, we systematically varied the postal recruitment process of a large mixed-mode survey we conducted in the fall of 2016 in the federal state of Saxony in Germany. We applied a sequential design similar to Millar & Dillman (2014) for the invitation of 8000 citizens to a community survey on waste disposal and environmental behavior. In the first invitation wave, we randomly allocated 40% of the respondents to paper-only survey, another 20% to a web-only and the final 40% to mixed-mode design with both response options. The follow-up consisted of a ‘Thank You’-postcard after about two weeks and a second invitation wave (completely) in mixed-mode design, offering both response options. We will present results of the competing designs on the overall response as well as a non-response analysis, based on register characteristics.
With the rise of available web services more generally, the public have come to expect to be able to engage with government via digital channels. The UK government is very aware of this demand and has in turn established the Government Digital Service (GDS) which is leading the digital transformation of government. The Office for National Statistics (ONS) is investing in the introduction of an online mode to its existing Social Surveys, defaulting to an online first approach. For social surveys this vision of a future data collection model presents substantial research and survey response challenges.
All social surveys in the UK are voluntary and in the absence of an address register achieving response is heavily reliant upon the rapport built between an interviewer and a potential respondent in a face-to-face interaction. The online first approach removes the interviewer step from the respondent journey in the first instance. Therefore the role of the letter inviting the respondent to take part becomes paramount. ONS is investing heavily in researching how best we can create this strong connection with a respondent in an online self complete approach by reviewing and radically redesigning our respondent materials starting first with the envelop (to break down our initial barrier to response). We are also considering the value of taking a behavioural economic approach, promoted with considerable success of the UK's Behavioural Insight team. In doing this we are applying four main principles to any interaction - make it Easy, Attractive, Social and Timely.
This is a long-term research strand and this talk will share with you our approach to this research, including an overview and update of the most recent findings from the traditional research methods such as in-depth interviewing and focus group with novel methods such as guerrilla pop-up testing to learn what engages and motivates respondents. This presentation will also set out plans for ongoing research and the context within which this work is being taken forward. It will discuss some of the challenges faced to date and those that we foresee to be on the horizon. We invite discussion and encourage others to share their experiences and recommendations with us.
With the rise of available web services more generally, the UK public have come to expect to be able to engage with government via digital channels. The UK government is very aware of this demand and has in turn established the Government Digital Service (GDS) which is leading the digital transformation of government.
Over the last decade ONS has conducted numerous qualitative and quantitative investigations into online questionnaire development for Social Surveys and successfully implemented an online completion mode for the 2011 Census. Previous research and past experience has highlighted the need for an extensive research programme to support successful introduction of online data collection, looking at the short and long term opportunities and goals.
The Data Collection Transformation Programme at ONS will transform ONS data collection into a more dynamic and efficient model, maximising the use of non-survey data sources and digital by default collection of data in the production of National and Official Statistics. For Social Surveys this means that ONS needs to deliver a redesigned social statistics model, as well as transformed end-to-end systems.
ONS conducts a number of UK household surveys that feed into a number of important National Statistics indicators. The largest survey that ONS conducts is The Labour Force Survey (LFS) (and sample boost), which is the largest UK household survey, resulting in around 400,000 productive household interviews per annum. The survey is used to produce a range of high profile cross-sectional and longitudinal labour market and Annual Population Survey datasets that are widely used for analysis and publications in the UK and Europe, including for the monthly estimates of labour market supply (including estimates of change in the employment and unemployment rates). The survey is currently governed by European Regulations.
Currently the survey is being redesigned to enable multi-mode data collection and specifically to enable on-line data collection. Qualitative development methodologies have been used by survey methodologists at ONS to elicit the best advance survey documentation and LFS question wording to suit online collection.
ONS is now quantitatively testing the most effective advance documentation strategies via a series of large scale online surveys using revised LFS question wording. The testing work will establish the optimal way of introducing an online LFS survey to the general public.
This presentation set out plans for ongoing quantitative research. It will discuss some of the research findings and challenges faced to date and those that we foresee to be on the horizon. We invite discussion and encourage others to share their experiences and recommendations with us.
Every five years, Statistics Canada conducts a census to provide population and dwelling counts for Canada as well as a wide range of information about Canada’s demographic, social and economic characteristics. The collection methodology has considerably evolved over the years in order to reduce respondent burden, improve the efficiency of the Census Program and data quality. Building upon insight drawn from previous research and censuses, the 2016 Canadian Census achieved one of its best collection response rates, and more importantly, the best Internet response rate since the Canadian Census’ inception.
The Wave Methodology used for the 2016 Census collection consisted in a series of mailings to the dwellings in order to stimulate self-response, in particular Internet response. This paper will present an overview of the methodology as well as collection results. Statistics Canada is at the forefront of the use of internet-based data collection tools and results of ongoing investigations on paradata analysis to better understand the respondents’ behavior will also be presented.