ESRA logo

ESRA 2027 sessions by theme

Back to Overview of Sessions

We tried and it did not work! Survey experiments that did not yield the desired results

Coordinator 1Mr Michael Blohm (GESIS - Leibniz Institute for the Social Sciences)
Coordinator 2Dr Oshrat Hochman (GESIS - Leibniz Institute for the Social Sciences)
Coordinator 3Dr Jan-Lucas Schanze (GESIS - Leibniz Institute for the Social Sciences)

Session Details

The widespread shift towards self-administered modes of data collection has introduced new challenges in persuading target persons, particularly when relying solely on indirect, written, and one-sided communication methods. This is especially true when striving to convince all subgroups of target persons equally. Issues of representation of ethnic or immigrant minorities, or of individuals with low levels of education, are just two examples of challenges that demonstrate this point. Several methods have been tested to maximize the impact of invitation letters, incentives, mode choice sequences, information leaflets and similar instruments. Some survey programmes have experimented with a combination of postal invitations and face-to-face contact attempts. In our session, we propose to focus on small or large-scale survey experiments, but to limit our view to those experiments that did not yield the desired results, and which therefore might fall victim to publication bias. In our opinion, experiments with null or unexpected results are often overlooked but carry significant meaning for our work. With this session we thus seek to narrow the existing gaps in our field by learning from recent experiences and allowing opportunities to build on these experiences instead of starting over from scratch. We particularly welcome experiments that focus on the response rate and the sample composition, but we are also interested in experiments that meant to improve other aspects of survey data quality.