ESRA 2013 Sessions

Innovations in Measurement Instrument Construction for Web-based SurveysMr Ozan Kuru


Innovations in measurement instrument construction for web-based surveys 1Mr Simon Munzert
Large parts of the increasing research body on web-based surveys deal with coverage, sampling and nonresponse issues, and therefore questions of representativeness. Less frequently discussed are measurement issues which arise from the unique way web-based surveys are conducted. In comparison with other modes, web-based surveys provide a bouquet of new tools and methods which allow for previously unknown flexibility in designing measurement instruments: The respondent can be presented (audio-)visual additional to (or even as a substitute of) verbal information, question and item order can be easily randomized, and valuable paradata like response latencies, key stroke measures or server-side paradata can be collected on-the-fly. These tools may help reduce respondent's burden when answering the questionnaire, but also allow for developing completely new instruments of existing concepts (e.g., visual measures of any kinds of knowledge). Although measuring opinions, facts etc. in an online setting might induce additional measurement bias in comparison with other modes, the web survey toolbox may provide instruments which can and should be used to fight these sources of error.
The goal of this panel is to bring together scholars who make use of new web survey tools to improve existing or construct new measures of a variety of concepts. The focus hereby is not so much on purely stylistic adaptions of the questionnaire layout, but on development of new instruments with methods going beyond ordinary question wording or response scale modifications. Papers to be presented in this session might deal with one of the following topics:
- innovative adaptation of existing or development of new instruments in web-based survey setting by use of unique web survey tools
- usage of web survey paradata to reduce survey error, or as a substantive measure
- studies implementing a cross-validation or MTMM design


Innovations in measurement instrument construction for web-based surveys 2Mr Simon Munzert
Large parts of the increasing research body on web-based surveys deal with coverage, sampling and nonresponse issues, and therefore questions of representativeness. Less frequently discussed are measurement issues which arise from the unique way web-based surveys are conducted. In comparison with other modes, web-based surveys provide a bouquet of new tools and methods which allow for previously unknown flexibility in designing measurement instruments: The respondent can be presented (audio-)visual additional to (or even as a substitute of) verbal information, question and item order can be easily randomized, and valuable paradata like response latencies, key stroke measures or server-side paradata can be collected on-the-fly. These tools may help reduce respondent's burden when answering the questionnaire, but also allow for developing completely new instruments of existing concepts (e.g., visual measures of any kinds of knowledge). Although measuring opinions, facts etc. in an online setting might induce additional measurement bias in comparison with other modes, the web survey toolbox may provide instruments which can and should be used to fight these sources of error.
The goal of this panel is to bring together scholars who make use of new web survey tools to improve existing or construct new measures of a variety of concepts. The focus hereby is not so much on purely stylistic adaptions of the questionnaire layout, but on development of new instruments with methods going beyond ordinary question wording or response scale modifications. Papers to be presented in this session might deal with one of the following topics:
- innovative adaptation of existing or development of new instruments in web-based survey setting by use of unique web survey tools
- usage of web survey paradata to reduce survey error, or as a substantive measure
- studies implementing a cross-validation or MTMM design


Innovations in measurement instrument construction for web-based surveys 3Mr Simon Munzert
Large parts of the increasing research body on web-based surveys deal with coverage, sampling and nonresponse issues, and therefore questions of representativeness. Less frequently discussed are measurement issues which arise from the unique way web-based surveys are conducted. In comparison with other modes, web-based surveys provide a bouquet of new tools and methods which allow for previously unknown flexibility in designing measurement instruments: The respondent can be presented (audio-)visual additional to (or even as a substitute of) verbal information, question and item order can be easily randomized, and valuable paradata like response latencies, key stroke measures or server-side paradata can be collected on-the-fly. These tools may help reduce respondent's burden when answering the questionnaire, but also allow for developing completely new instruments of existing concepts (e.g., visual measures of any kinds of knowledge). Although measuring opinions, facts etc. in an online setting might induce additional measurement bias in comparison with other modes, the web survey toolbox may provide instruments which can and should be used to fight these sources of error.
The goal of this panel is to bring together scholars who make use of new web survey tools to improve existing or construct new measures of a variety of concepts. The focus hereby is not so much on purely stylistic adaptions of the questionnaire layout, but on development of new instruments with methods going beyond ordinary question wording or response scale modifications. Papers to be presented in this session might deal with one of the following topics:
- innovative adaptation of existing or development of new instruments in web-based survey setting by use of unique web survey tools
- usage of web survey paradata to reduce survey error, or as a substantive measure
- studies implementing a cross-validation or MTMM design


Web data collection for probability-based general population surveys 1Professor Peter Lynn
Web survey methodology is well-established for non-probability online panels and for specialist populations where web access can be assumed to be universal and where an available sampling frame includes email addresses. However, for probability-based general population surveys experience of web data collection remains limited. Many such surveys are now considering the inclusion of web within a mixed-mode design, though few have yet incorporated a web element and there is no consensus on the best way to do this. Meanwhile there is a very small but growing number of single-mode probability-based online panels, using different methodologies.
Survey researchers anticipate several potential benefits from the use of web data collection for general population surveys, such as reduced data collection costs and faster data collection. However, there are significant challenges to be overcome. Major themes include:

Sampling and coverage: How can we design surveys, incorporating web, so that they meet the representativity requirements of general population surveys?

Participation and engagement: What must we learn and do to engage participants in web surveys so that we get high, unbiased response and good quality data? In particular, how do we engage with sub-groups crucial to the success of social surveys - those with poorer access to technology and lower skills, those from disadvantaged and minority groups, etc?

Measurement challenges: How can we best capture complex data using the web and what new opportunities (and related research implications) are there for us to capture new kinds of data?

We welcome submissions to this session that address any of the issues faced by probability-based general population surveys with respect to the inclusion of a web-based data collection element. We particularly welcome reports of findings from experimental or developmental work. We also welcome case studies of general population surveys that have added a web element.

3rd co-ordniator Dr. Caroline Roberts, University of Lausanne, caroline.roberts@unil.ch

Web data collection for probability-based general population surveys 2Professor Peter Lynn
Web survey methodology is well-established for non-probability online panels and for specialist populations where web access can be assumed to be universal and where an available sampling frame includes email addresses. However, for probability-based general population surveys experience of web data collection remains limited. Many such surveys are now considering the inclusion of web within a mixed-mode design, though few have yet incorporated a web element and there is no consensus on the best way to do this. Meanwhile there is a very small but growing number of single-mode probability-based online panels, using different methodologies.
Survey researchers anticipate several potential benefits from the use of web data collection for general population surveys, such as reduced data collection costs and faster data collection. However, there are significant challenges to be overcome. Major themes include:

Sampling and coverage: How can we design surveys, incorporating web, so that they meet the representativity requirements of general population surveys?

Participation and engagement: What must we learn and do to engage participants in web surveys so that we get high, unbiased response and good quality data? In particular, how do we engage with sub-groups crucial to the success of social surveys - those with poorer access to technology and lower skills, those from disadvantaged and minority groups, etc?

Measurement challenges: How can we best capture complex data using the web and what new opportunities (and related research implications) are there for us to capture new kinds of data?

We welcome submissions to this session that address any of the issues faced by probability-based general population surveys with respect to the inclusion of a web-based data collection element. We particularly welcome reports of findings from experimental or developmental work. We also welcome case studies of general population surveys that have added a web element.

3rd co-ordniator Dr. Caroline Roberts, University of Lausanne, caroline.roberts@unil.ch

Web data collection for probability-based general population surveys 3Professor Peter Lynn
Web survey methodology is well-established for non-probability online panels and for specialist populations where web access can be assumed to be universal and where an available sampling frame includes email addresses. However, for probability-based general population surveys experience of web data collection remains limited. Many such surveys are now considering the inclusion of web within a mixed-mode design, though few have yet incorporated a web element and there is no consensus on the best way to do this. Meanwhile there is a very small but growing number of single-mode probability-based online panels, using different methodologies.
Survey researchers anticipate several potential benefits from the use of web data collection for general population surveys, such as reduced data collection costs and faster data collection. However, there are significant challenges to be overcome. Major themes include:

Sampling and coverage: How can we design surveys, incorporating web, so that they meet the representativity requirements of general population surveys?

Participation and engagement: What must we learn and do to engage participants in web surveys so that we get high, unbiased response and good quality data? In particular, how do we engage with sub-groups crucial to the success of social surveys - those with poorer access to technology and lower skills, those from disadvantaged and minority groups, etc?

Measurement challenges: How can we best capture complex data using the web and what new opportunities (and related research implications) are there for us to capture new kinds of data?

We welcome submissions to this session that address any of the issues faced by probability-based general population surveys with respect to the inclusion of a web-based data collection element. We particularly welcome reports of findings from experimental or developmental work. We also welcome case studies of general population surveys that have added a web element.

3rd co-ordniator Dr. Caroline Roberts, University of Lausanne, caroline.roberts@unil.ch