ESRA logo

Tuesday 16th July       Wednesday 17th July       Thursday 18th July       Friday 19th July      

Download the conference book

Download the program





Tuesday 16th July 2013, 16:00 - 17:30, Room: No. 12

Advancing the field of questionnaire translation - identifying problems, discussing methods, pushing the research agenda. A tribute to Janet Harkness 3

Convenor Dr Dorothee Behr (GESIS)
Coordinator 1Ms Brita Dorer (GESIS)
Coordinator 2Dr Gijs Van Houten (Eurofound)

Session Details

Questionnaire translation is a crucial aspect when it comes to collecting comparable survey data in different countries or among different language groups. In the widest sense, translation should ensure the implementation of 'equivalent' instruments, in different linguistic, cultural and institutional settings.

The quest for equivalence already starts at the questionnaire development stage, in which advance translation can already contribute to a more 'translatable' source questionnaire. It continues in the translation stage through procedures - such as using multiple translators with varying skills and providing them with extensive information and task specification. In the review or assessment stage, committee assessments have been argued to contribute to questionnaire equivalence, more so than the use of 'back translation' which was common practice in many multi-lingual surveys in the past. Pretesting and documentation should round off all these translation procedures (e.g., Harkness 2003).

We invite papers on all aspects related to questionnaire translation. The papers may address more specific (e.g. pertaining to a particular language combination or translation issue) or more general translation issues (e.g., pertaining to scale translation) to draw attention to where problems are located. The papers may also deal with the role of translation in source text development (e.g., advance translation). The papers may equally focus on how challenges are met in terms of methodology (e.g., translation and assessment methods) or software (e.g., translation tools). Last but not least, papers are encouraged which push the research agenda and provide deeper insights into what exactly a good and comparable survey translation is (e.g., effects of different survey translations on the data; translation vs. adaptation; different communication styles across cultures and their translation/adaptation; the notion of "equivalence" itself).


Paper Details

1. Seeking equivalence in translating ends of survey scales: A corpus-based approach

Mrs Danuta Przepiórkowska (University of Warsaw)
Mrs Teresa Zmijewska-jedrzejczyk (Polish Academy of Sciences)

According to the Ask-the-Same-Question model, the most important consideration is to ensure equivalence of translated questionnaires and their components. However, as Janet Harkness pointed out "[...] close translation is a balancing act." (Harkness et. al. 2010, p. 47). Our paper will look at the problems posed by translation of numeric-graphic scales where ends of scales are described verbally. Such scales have been commonly used in many cross-national surveys (e.g. the European Social Survey). In particular, we will analyse verbal descriptors of extreme ends of scales, especially looking at modifying adverbs/adverbial phrases (such as extremely, completely, very etc.). The practice for many surveys has been to use the same modifier at both ends of scales (e.g. extremely bad vs. extremely good, or extremely satisfied vs. extremely dissatisfied). However, translation of those modifiers into other languages has revealed a number of problems (some phrases do not translate well, different modifiers have to be used even if the same word was used in the original English questionnaire etc.). It seems that linguistic issues such as semantic prosody may be at play here: some modifiers work well with 'positive' words but not so well with 'negative' ones and vice versa. Our query into English and Polish linguistic resources (monolingual corpora and dictionaries) seems to confirm this intuition.
In our paper we will discuss the findings and explore if and how corpora may provide a useful source to address the problems mentioned above.


2. COMPETENCIES REQUIRED FOR ADAPTATION OF SURVEY INSTRUMENTS AS PREMISE FOR METHODOLOGICAL QUALITY

Ms Zemyna Pauliukaite (none)

Different authors consider that competencies of survey instruments' translation are an important premise for good measurement properties of translated questionnaire. Incorrectly translated questionnaires can result in the whole research project failure due to the lack of methodological quality. Nevertheless, only few attempts have been made to produce some empirical validations.
The author of this article formulated "a model of 4 competencies", while looking for an answer to a question as to what determined good results of the questionnaires' translation and cross-cultural adaptation. The suggested translation organization technique (based on "the model of 4 competences") stipulates that competencies of a researcher-translator should converge towards the "ideal" profile of two cultural and two academic-professional competencies. That is - the linguistic competence, the cultural competence, the methodological competence in survey, the competence in a measured diagnostic construct. The duality of the competencies is essential - practical and theoretical knowledge of a subject.
The effectiveness of suggested translation organization technique was tested with empirical social research. It was supported with biographical survey and interview with the scholarly group who created, applied and tested the above mentioned technique. Linguistic adaptation of questionnaires and the quality of the cross-cultural transfer was tested with psychometric statistics and methodological re-analysis of five different surveys. The methodological quality of the adapted questionnaires was confirmed from analysis of 665 primary indicators and 76 scales (when N=5516). That allowed empirically recognize that the suggested methods as effective.



3. Can we survey migrants by questionnaires developed for the population of the respective source countries?

Mr Patrick Brzoska (University of Bielefeld, School of Public Health, Dept. of Epidemiology and International Public Health)
Mrs Yuece Yilmaz-aslan (University of Bielefeld, School of Public Health, Dept. of Epidemiology and International Public Health)
Professor Oliver Razum (University of Bielefeld, School of Public Health, Dept. of Epidemiology and International Public Health)

Introduction: Quantitative migrant health researchers often apply questionnaires validated only for the source populations of migrants. However, after years in the host country, migrants may differ in cultural and language characteristics from the source population. This may limit the transferability of questionnaires. Based on a comprehensive analytical framework of equivalence covering conceptual, semantic and measurement characteristics, the transferability of the Turkish version of the Revised Illness Perception Questionnaire (IPQR) (part II) between Turks residing in Turkey and Turkish migrants in Germany is examined as an illustrative example.

Methods: The measurement equivalence of the IPQR was examined through multigroup confirmatory factor analysis based on data from 602 patients surveyed in Turkey and Germany. Conceptual and semantic equivalence were studied by qualitative interviews with patients (n=40 and 58, respectively) and experts (n=6 and 8, respectively) and by systematic literature reviews.

Results: The indicator-factor pattern of the IPQR differed between both samples and only a partial configural model could be established. Subsequent analyses, additionally, revealed differences in factor loadings and intercepts. While there was no evidence of a different conceptual frame between both populations, several sources of semantic nonequivalence could be identified, going back to complex wordings of certain items.

Conclusion: The study illustrates possible areas of nonequivalence in the transfer of questionnaires between source and migrant populations, some of which go back to differences in language usage. They need to be considered in the readaptation of questionnaires for which the analytical framework of equivalence can be a useful tool.


4. Translation Verification in the European Social Survey - a means for achieving equivalent translations in a cross-national survey?

Ms Brita Dorer (GESIS - Leibniz-Institute for the Social Sciences, Mannheim, Germany)
Ms Sally Widdop (Centre for Comparative Social Surveys, City University London)
Mr Rory Fitzgerald (Centre for Comparative Social Surveys, City University London)

Throughout all stages of the cross-national survey life cycle there are threats to measurement equivalence. Errors can occur during the drafting, pre-testing or translation stages of questionnaire design as well as during administration itself.

In order to address potential inaccuracies in its questionnaire translations, the European Social Survey (ESS) has included an additional 'verification' stage in rounds 5 and 6. cApStAn, an external company specialised in linguistic quality control of survey instruments, was appointed to verify a selection of items taken from the translated questionnaires in each participating country. This process has involved verifiers from outside the original translation teams checking the translations for linguistic accuracy and equivalence to the source questionnaire.

This paper will compare the different strategies used to implement the verification process during ESS rounds 5 and 6 and provide an assessment of the contribution of verification to date. The types of translation problems identified by the verifiers in each round will be discussed and implications for both ESS national teams and for source questionnaire designers in general will be assessed. In particular, the overall contribution made by verification to ongoing efforts to improve translation quality of ESS items in terms of (a) general translation issues, (b) language-version-specific translation issues, (c) term-specific translation problems, and (d) equivalence between all translations in this cross-national survey will be discussed.