ESRA 2025 Preliminary Glance Program

All time references are in CEST

Questionnaire translation in a changing world: challenges and opportunities 2
Session Organisers	Dr Alisú Schoua-Glusberg (Research Support Services) Dr Brita Dr. Dorer (GESIS-Leibniz Institute for the Social Sciences) Dr Dorothée Behr (GESIS-Leibniz Institute for the Social Sciences)
Time	Wednesday 16 July, 14:00 - 15:00
Room	Ruppert paars - 0.44

With the world becoming more and more globalised and population demographics changing at an ever higher pace, a good quality level of questionnaire translations is increasingly important to develop reliable cross-cultural data.
The digital transition has also been entering the fields of translation in general and questionnaire translation in particular: While good practice recommendation in cross-cultural survey methodology is still to apply team approaches, making sure to involve appropriately trained and experienced questionnaire translation experts, ideally TRAPD (consisting of the steps Translation, Review, Adjudication, Pretesting and Documentation), digital innovations such as Machine Translation or Artificial Intelligence are more and more entering the fields of (questionnaire) translations. Technical innovations related to questionnaire translations can have manifold forms, such as platforms allowing a smooth handling of the workflow to develop, translate and later field questionnaires in manifold languages. Or, for instance, crowd-based translation schemes become more and more popular and may contribute to translating questionnaires too. Where are the strengths and where the weaknesses of such developments? Which role should AI play in the translation of survey instruments?
This session addresses various aspects of questionnaire translation, whether related to digital innovations or not. We invite papers on various topics related to the translation and interpretation of survey instruments, referring for example to: experiments on new techniques or alternative methods, studying challenges of specific language pairs or translation methods. How to translate certain questionnaire elements, such as translating answer scales, approaches to develop questionnaire translations of minority languages, or comparing effects of different translation quality assessment approaches on survey data.

Keywords: Questionnaire translation, cross-cultural surveys, survey methodology, digitalisation

Papers

Questionnaire Translation in the Fourth European Survey of Enterprises on New and Emerging Risks (ESENER 2024)

Mr Xabier Irastorza (European Agency for Safety and Health at Work (EU-OSHA)) - Presenting Author

The European Agency for Safety and Health at Work (EU-OSHA) completed its fourth wave of ESENER in 2024, interviewing over 41,000 establishments across all activity sectors in 30 countries.
This presentation focuses on the main steps of the translation of the ESENER 2024 questionnaire into 40 national versions:

a) A translatability assessment (TA) of the English master questionnaire.
Experienced translators from three different language families -Croatian, Finnish and Greek- reviewed the draft of the new and modified questions, identifying potential translation, adaptation or cultural issues and providing recommendations for alternative wording. The draft questionnaire reviewed in the TA was the result of cognitive testing interviews that were carried out in three languages different to those in the TA: German, Polish and Spanish. This aimed to ensure a broader language coverage in the pre-testing phase.

b) Two independent translators and one adjudicator per language version.
Following training -for all translators and adjudicators- two independent translations were provided to the adjudicator, who produced a reconciled version to be discussed at a team review meeting –one for each of the 40 national versions. A help desk service was offered throughout the entire translation and adjudication/adaptation process. There was a thorough documentation of each step of the translation using different worksheets of the same Excel file (one file per national versions), including hints, instructions and specific terminology.

c) National expert feedback
Representatives of the national bodies in charge of OSH in each of the 30 countries covered in ESENER 2024 reviewed the questionnaires. The aim was to ensure the appropriateness of the national OSH terminology rather than a linguistic check. The final versions of the questionnaires were sent to proofreading, to check for correctness of the target language.

The entire process lasted from November 2023 to February 2024

Exploring Gendered Representations in Machine Translation

Mr Orfeas Menis–Mastromichalakis (National Technical University of Athens, School of Electrical and Computer Engineering)
Mr George Filandrianos (National Technical University of Athens, School of Electrical and Computer Engineering)
Dr Glykeria Stamatopoulou (Panteion University of Social and Political Sciences, Department of Social Policy)
Professor Dimitris Parsanoglou (National and Kapodistrian University of Athens, Department of Sociology)
Professor Maria Symeonaki (Panteion University of Social and Political Sciences, Department of Social Policy) - Presenting Author
Professor Giorgos Stamou (National Technical University of Athens, School of Electrical and Computer Engineering)

In an increasingly globalised and digitised world, the translation of survey instruments faces both long-standing challenges and emerging opportunities, particularly with the growing influence of automated machine learning and artificial intelligence technologies. We argue that automated translation systems can significantly advance the accuracy, speed and efficiency of translations. This is exemplified by the correct translation of the response category "Fulfilling domestic tasks" in the European Union Labour Force Survey (EU-LFS) questionnaire into Greek—a task where manual translation produced a biased interpretation "She is a housewife", implying domestic work is exclusively performed by women. Other examples also illustrate the transformative potential of these systems in ensuring more neutral and accurate translations. However, challenges persist as automated systems for example can also reinforce inherent biases present in linguistic corpora, such as consistently translating "doctor" in gendered languages predominantly as masculine, reflecting historical patterns rather than contemporary realities. This study examines both the strengths and limitations of automated translation systems, focusing on their handling of gender biases in high-resource languages like English and French, as well as in lower-resource languages such as Greek. Utilising labour statistics from France, Greece, and the UK for the past decade, the research investigates gender variations and their persistence in employment, offering a classification framework for identifying and understanding gender bias in machine translation. This dual perspective highlights the critical advancements achieved by translation systems while also drawing attention to areas requiring refinement to address lingering biases. The findings underscore the need for balanced and informed technological innovations that build on the strengths of current systems while mitigating their limitations. By doing so, this research aims to enhance the accuracy, inclusivity, and reliability of automated translation systems, ultimately contributing to more equitable practices in survey design and beyond.