Tuesday 14th July      Wednesday 15th July      Thursday 16th July      Friday 17th July     

Thursday 16th July, 16:00 - 17:30 Room: HT-104

Going beyond the basics of questionnaire design: new and innovative approaches to instrument design in web surveys 2
Convenor	Dr Femke De Keulenaer (Ipsos )
Coordinator 1	Professor Edith De Leeuw (Utrecht University)
Coordinator 2	Mr Arnaud Wijnant (CentERdata / Tilburg University)

Session Details

Web questionnaires can differ substantially from questionnaires in traditional modes; nonetheless, most web-based instruments are based on classic text-based questionnaire principles. Rather than simply applying the design principles for paper questionnaires, researchers could capitalize on the unique properties of the web interaction. In web surveys, the medium can be fully exploited to produce better ways of asking and answering questions, and to introduce new approaches to surveying (i.e. going beyond “asking questions”).

In this session, we would like to focus on the potential for advanced questionnaire design in web surveys and innovative approaches to surveying respondents. Topics could include, but are not limited to the following:

- The unique properties of web interaction can be used to design web questionnaire interfaces that adapt or tailor themselves to respondents’ behaviour, diagnose respondents’ need for clarification, detect respondents’ lack of effort, etc.

- A difference between web surveys and traditional surveys can be the focus on (audio-)visual communication. The dynamic and graphical nature of the web has led to the creation of a wide range of measurement tools that previously could not be done on paper; examples include card sort tasks, interactive maps and verbal information.

- Web surveys also offer possibilities for innovative approaches to surveying; for example, behavioural experiments have made the switch from asking respondents to report on their behaviour (via survey questions) to actually observing respondent behaviour (e.g. using game-enhanced instruments or facial expression devices).

- This type of innovative approaches to web instrument design, however, can also lead to a variety of unpredicted effects that reduce the quality of web-based surveys. Researchers are also invited to present empirical evaluations and split ballot experiments. Only by fully understanding both the benefits - and the drawbacks – of innovations can we fully exploit the potential of web surveys.

Paper Details

1. Comparing Interface Designs of Database Lookups with Traditional Measurements
Ms Jessica Herzing (German Internet Panel, SFB 884, University of Mannheim and GESIS Leibniz Institute for the Social Sciences, Germany)

Traditional modes of data collection often face the problem of incompleteness in case of long-list factual questions, e.g. country of residence, prescription drugs, etc., as the number of answer categories is limited to space and respondent burden.
To overcome these drawbacks of traditional questionnaire instruments and to improve data collection in terms of data quality and respondent burden, a database query was incorporated in the survey procedure. To test different database interfaces a split-ballot experiment was conducted. Using the example of educational attainment the analysis is based on paradata and reliability checks.

2. Survey Wizard 2.0: Addressing implicit response tasks in a census of businesses
Mr Alfred Tuttle (US Census Bureau)

The U.S. Census Bureau developed an application called Surveyor to facilitate reporting by large enterprises for the Economic Census. Despite its useful features, Surveyor’s lack of a clearly defined path may be a source of respondent burden and potential data quality issues. A new collection system is being developed that will present critical information about the response process when and where it is needed. This presentation will focus on our delineation of the implicit process of completing the Economic Census for large firms, and our steps to make this process more explicit in the new instrument.