ESRA 2025 Preliminary Glance Program
All time references are in CEST
Optimizing Grid Questions in Mixed-Mode Surveys Across Devices: Challenges and Solutions |
| Session Organisers |
Dr Ellen Ebralidze (Leibniz Institute for Educational Trajectories (LIfBi))
Mrs Annette Trahms (Institute for Employment Research (IAB))
|
| Time | Thursday 17 July, 15:30 - 17:00 |
| Room |
Ruppert 119 |
Mixed-mode surveys face the challenge of ensuring comparability across different modes and devices, including PCs, tablets, and smartphones. A significant difficulty is optimizing grid question layouts to suit all screen sizes while maintaining consistency across modes.
This session will explore constraints and solutions in designing grid formats. We invite practitioners, especially those involved in large-scale surveys, to share their approaches, focusing on mixed-mode designs, mobile-friendliness, potential question format effects, and lessons learned. Specifically, the following questions can be addressed:
• What modes are used for your survey, and how do they interact? Are you using a concurrent mixed-mode design with two or more modes at the same time, a sequential mixed-mode where you start with one mode and let others follow, or a combination? What is the reference mode in your mix?
• In panel studies, how have mixed-mode designs evolved, and have there been changes in the reference mode?
• Is your grid format mobile-friendly, and does the layout differ across devices? How does it compare to the grid display in other modes you may use (e.g. CATI, PAPI)? What question format effects have you observed or anticipate?
• What compromises have you made, what lessons have you learned?
We welcome case studies from researchers, survey methodologists and survey institute experts on this topic.
Keywords: Mixed-Mode Surveys, Grid Questions,
Papers
Star Rating Scales in Online Grid Questions: Better than Verbally Labelled Scales?
Ms Lea Königer (LMU Munich) - Presenting Author
Optimizing response scales for mobile devices is becoming increasingly important, as the proportion of respondents participating in online surveys via mobile phones continues to grow. This study evaluates the use of 5-star pictorial rating scales compared to traditional verbal Likert scales, focusing on their usability and response quality in online surveys across modes and devices.
We present data from an experiment conducted in an online survey of the German Longitudinal Environmental Study (GLEN), a large-scale probability-based mixed-mode panel started in 2024. In a split-ballot experiment, participants are randomly assigned either a 5-star scale or a conventional five-point verbalized Likert scale to rate residential quality. To assess the measurement quality of the two response formats, we employ two approaches: First, we compare responses over time as the question was included in two waves of GLEN (late 2024 and early 2025). Second, we compare respondents’ subjective ratings with linked objective geospatial neighborhood data.
We benefit from the mixed-mode approach in GLEN: Following a mixed-mode recruitment survey (paper and web), the second survey is conducted exclusively online. This design allows us to analyze differences between response formats both within and across devices and modes. Our findings have important implications for mixed-mode survey design, which often faces the challenge of optimizing instruments for all available modes and devices.
