ESRA 2017 Programme

Tuesday 18th July      Wednesday 19th July      Thursday 20th July      Friday 21th July     

     ESRA Conference App


Friday 21st July, 11:00 - 12:30 Room: Q2 AUD2

New measurement capabilities of mobile devices

Chair Dr Peter Lugtig (Utrecht University )
Coordinator 1Dr Vera Toepoel (Utrecht University)
Coordinator 2Miss Anne Elevelt (Utrecht University)
Coordinator 3Mr Randall Thomas (GfK Custom Research)
Coordinator 4Dr Frances Barlas (GfK Custom Research)

Session Details

Mobile devices (tablets and phones) can be used as a mode of administration in web surveys. Nowadays, between 10-40% of web surveys are being completed on mobile devices. While there is still much to learn about how to design questionnaires for mobile devices, this session focuses on data that mobile devices can record in addition to survey data. When mobile surveys are taken through a browser, one can the same paradata that can also be recorded on desktops and laptops. Additional data can however also be collected, both through browsers and mobile apps. Examples of such data are GPS data, data from environmental sensors (light, temperature, speed, pictures, sound), and device use data.

In this session we invite presentations that show how sensor data can and should be used in mobile surveys. We particularly would welcome papers on the following topics:
- Successful examples of recording sensor data
- Integrating sensor data and survey data
- Practical issues in designing apps and browsers for recording sensor data
- Ethical issues in recording sensor data
- Asking for consent

Paper Details

1. Making Grid Questions Mobile Friendly - An Impossible Dream?
Dr Frances Barlas (GfK Custom Research)
Dr Nicole Buttermore (GfK Custom Research)
Mr Randall K. Thomas (GfK Custom Research)

Grid questions are used routinely in online surveys when researchers want to assess multiple items using the same response format. Grids typically present elements to be evaluated in rows with responses arrayed in the columns. Grids can be problematic with smaller screen sizes such as those of smartphones. Horizontal (left-right) scrolling is often required to view all response options, making it impossible to view both the item text and all response options at the same time and raising the possibility that respondents will be less likely to select response options not visible on the screen. In the accordion grid format, respondents see the item text listed vertically, then can click on each item to reveal the response scale in a standard single response format. Using this approach, respondents can see the entire response scale for each item, along with the response items on a single screen. However, vertical response presentation has been associated with an increased likelihood of response order effects. Is the accordion grid an effective alternative to the traditional grid format? We conducted two experiments each using over 1,000 cases from KnowledgePanel®, GfK’s U.S. probability-based online panel. With each study, respondents were randomly assigned to complete either an accordion or traditional grid. The time it took for respondents to complete each grid type was comparable, and we found few differences in results across the two types of grids for both dichotomous grids and unipolar scales with three or more response options. We found similar concurrent validity for both grid types when correlating the results with behavioral data and little evidence of response order effects. Respondents provided similar ratings of the ease and accuracy with which they could respond to the traditional and accordion grid scales. Taken together, the results suggest that the accordion grid is a promising alternative to the traditional grid design, especially given the increase in the proportion of online survey respondents who complete surveys using smartphones and other mobile devices.


2. All Thumbs? Improving Measurement for Smartphones.
Professor Randall K. Thomas (GfK Custom Research)
Dr Frances M. Barlas (GfK Custom Research)
Dr Nicole R. Buttermore (GfK Custom Research)

Mobile devices have smaller screens which make survey response selection more difficult, especially on smartphones. In self-administered surveys, grids have evolved to efficiently measure multiple concepts using the same graded response format across a number of different targets, with responses typically in columns and the rating targets to evaluate in rows. Generally, grids allow us to more efficiently assess multiple targets compared to presenting each target is presented on separate screens. The response format that is chosen can vary in terms of semantic labeling - every response can have an associated semantic label (fully labeled) or only the end categories have a semantic label (end labeled). The response format could represent a bipolar scale (ranging from a concept and its antithesis, e.g. like – dislike) or a bipolar scale (with a concept and its absence, e.g. like – do not like). Finally, the response format can vary in terms of the number of response categories used. We summarize 4 studies comparing the use of end-anchored and fully-anchored categories with both unipolar and bipolar scales, holding the number of categories constant. The unipolar fully labeled variant generally appeared to be better as a measure across topics, with better differentiation and validity. Since online surveys are increasingly being taken on mobile devices, the utility of a grid is becoming affected by screen size – grids with more responses are much harder to implement on smartphones. In a series of 2 additional studies we examined how reducing the number of grid response categories affects the measurement of a variety of concepts. We found that response formats with fewer response categories take less time to complete, are easier to complete on mobile devices, and shorter scales show as much validity as formats with more response categories, especially when using the unipolar variant. We examine the ability of the scales to detect smaller differences between rating targets and found some improved differentiation with scales with fewer responses rather than those with more responses.