ESRA 2027 sessions by theme

Back to Overview of Sessions

Session Details

Large language models are increasingly proposed, and commercially marketed, as substitutes for human survey respondents. "Silicon sampling" and "digital twin" approaches promise opinion estimates at a fraction of the cost and turnaround of fieldwork, and a rapidly growing literature evaluates whether LLM-generated responses can replicate human survey data. The evidence so far is mixed. Synthetic samples reproduce some aggregate patterns of opinion, but they underestimate response heterogeneity, misrepresent subgroups, and fail in ways that are invisible from inside the data itself. Whether, when, and for what purposes synthetic respondents are usable remains an open and urgent question for the field.
This session brings together empirical evaluations of LLM survey simulation. We welcome work that maps where synthetic respondents succeed as well as where they fail, identifying the conditions, topics, and populations under which simulation is reliable enough to be useful. We are also particularly interested in work that treats design choices as variables to be tested rather than fixed features. Relevant choices include how synthetic respondents are specified, how responses are elicited, whether standard survey-statistical corrections such as weighting, raking, or MRP improve synthetic estimates, and how accuracy itself should be measured.
We welcome papers that benchmark synthetic data against human ground truth across models, domains, or populations; studies of known failure modes such as variance collapse, suppression of non-attitudes, and demographic essentialism; methodological work on evaluation standards and reporting practices for synthetic survey data; investigations of hybrid designs that combine human and synthetic respondents; and critical or conceptual contributions on the appropriate role of simulation in the survey research toolkit. Contributions from academic, official statistics, and commercial settings are equally encouraged. The session aims to move the debate from whether LLMs can replace respondents to what, precisely, they can be trusted to do.