ESRA 2017 Programme

Tuesday 18th July      Wednesday 19th July      Thursday 20th July      Friday 21th July     

     ESRA Conference App

Tuesday 18th July, 11:00 - 12:30 Room: F2 103

Power of Survey Research in Evidence-Based Policymaking 2

Chair Dr Young Chun (PSI Institute for Data Science and Interdisciplinary Research )
Coordinator 1Ms Giang Nguyen (ISR Foundation Center for Interdisciplinary Research)
Coordinator 2Ms Clara Kyung (McGill University)

Session Details

Co-Organizers: Cindy Won, Brown University; Hanyuying Wong, Cambridge University; Leying Guan, Stanford University

Evidence-Based Policymaking (EBP), a term coined from the idea of “evidence-based medicine” and spread to all spheres of public policy, involves the use of evidence collected by scientifically rigorous methods, such as randomized controlled trials. The EBP has garnered support from academia and governmental sectors across the Atlantic. For instance, Adrian Smith (1996) introduced the term in his presidential address to the Royal Statistical Society, questioning the current process of ideology-based policymaking and urging for an “evidence-based approach” in public policy. On the other side of the Atlantic, Davies (2004) echoed EBP as the integration of experience, judgement and expertise with “the best available external evidence from systematic research.” Listening to the call from the scientific research community and gleaning from the steps taken by the UK government, the Obama administration kicked off the Evidence-Based Policy Commission in summer of 2016 to institutionalize evidence-based policy development across all federal agencies.

Survey research-based data and survey data linked to other sources of data, such as administrative records and big data, are at the heart of evidence-based policy making. This session invites papers that demonstrate how survey research and administrative records are instrumental to evidence-based policymaking. We encourage submission of interdisciplinary papers that address the following topics:

1) Contributions of survey research applications to evidence-based policy making in public health, economy and education, just to name a few. For example, how survey research has informed the policymaking process, which intends to bring about social, economic, and fiscal benefits?

2) Papers demonstrating the use of administrative records linked to survey data in developing public policy, for example, in healthcare intervention, special education for the disabled, and economic welfare programs.

3) Papers involving experimental design, such as randomized controlled trials, in a broad context of survey research, to advance evidence-based policymaking with case studies (e.g., small business support, education initiatives for low-income children, healthcare programs to prevent or control epidemic disease like TB and Hepatitis).

4) Papers utilizing data visualization, a tool of showing agile comprehension of massive data, to communicate the public messages about policy issues particularly in developing countries. EBP is less implemented in developing countries due to lack of resources and limited civil liberty; the potential for evidence-based policy impact is greater.

Paper Details

1. Using survey research for evaluating new models of integrated health and social care services in England
Mr Bob Erens (London School of Hygiene & Tropical Medicine)
Professor Nicholas Mays (London School of Hygiene & Tropical Medicine)
Dr Mary Alison Durand (London School of Hygiene & Tropical Medicine)
Dr Ties Hoomans (London School of Hygiene & Tropical Medicine)

High quality data, including from surveys, are essential to inform evidence-based policy-making. However, considerable investment goes into developing government policies which are often based on poor evidence, and there is usually little rigorous data on their impact. This is especially the case for complex interventions that have multiple interacting components.

The Integrated Care Pioneer programme in England, launched in 2013, exemplifies a complex intervention. The 25 Pioneer sites are integrating health and social care services which aim to improve patient experiences and outcomes while saving costs. We discuss our use of surveys as part of a 5 year evaluation of the Pioneer programme.

Our evaluation involves an interdisciplinary team carrying out a mixed method study that includes surveys and qualitative data collection alongside the use of linked administrative data (eg, the use of health and social care services). Survey research will be used at two levels: first, at a policy level, to assess the impact of the overall Pioneer programme over its 5-year lifespan; second, to determine the cost-effectiveness of specific interventions Pioneers implement locally to deliver integrated care.

At the policy level, we ask a panel of key informants of managerial staff from all 25 Pioneers to complete an online survey 1-2 times/year. The 25 sites vary considerably in geographical and population size, and service integration typically involves a range of organisations including local authorities, health care providers and the voluntary sector. In order to obtain survey results that are comparable across sites and over time, some of the issues that need to be resolved include deciding the types of individuals to include on the panel, how to deal with the different levels of knowledge about the programme among panel members, and how to handle panel turnover.

For looking at the cost effectiveness of specific interventions, we will distribute postal questionnaires to patients, carers and frontline staff, 2-3 times over an 18 month period, to ask about their experiences of the new service delivery models and to measure their effect on quality of (working) life. Signed consent is required to be able to link survey results with health and social care administrative data, which raises the issue of how best to obtain informed consent as part of a postal survey. We will also survey patients, carers and frontline staff in areas without the service intervention in order to make inferences about the integration of health and social care services.

Evidence-based policy making relies on evidence from rigorous evaluation. Surveys can provide an essential component for evaluating complex interventions, but survey implementation is not necessarily straightforward and needs to be tailored for the specific context and circumstances of the policy (and evaluation) in order to provide high quality data that can be used by policy-makers.

2. Renewable Energy in North Korea: Challenges of Limited Resources and Smart Solutions
Ms Giang Nguyen (ISR Foundation Center for Interdisciplinary Research)
Mr Jun Bin Lee (ISR Foundation Center for Interdisciplinary Research)
Ms Tiffany Lee (ISR Foundation Center for Interdisciplinary Research)
Ms Helene Cho (ISR Foundation Center for Interdisciplinary Research)
Mr Asaph Young Chun (U.S. Census Bureau)

The purpose of the paper is to demonstrate the power of data visualization to complement an evidence-based policymaking approach to addressing the national needs of developing countries. As a case study, we the issue of renewable energy in North Korea. Supply of electricity in north Korea is characterized by widespread dependence on hydroelectric sources, comprising of 60% of total output. Severe droughts in 2015 caused extreme shortages of electricity, with the majority of households connected to the power grid receiving only a few hours of electricity per day. This drought exacerbated and brought to focus the ongoing issue of the North’s precarious electricity situation. It is the objective of this paper to explore the complex problem of and solutions to North Korea’s electricity needs, by leveraging data visualizations and creating evidence-based policy prescriptions to address the problem. Analysis is done at both the regional and national level to form a comprehensive picture. Data is gathered from the North Korean Census, meteorological data sources, and international outreach agencies. To characterize north Korean electricity demand, we first map the population density along regional and demographic characteristics. We then go on to overlay the electrical grid with solar and wind maps to develop integrated information and provide an understanding of the potential of renewable energy in North Korea. We conduct a comparative analysis of the North Korean case with nations facing similar energy issues. We demonstrate that renewable and alternative sources such as wind and solar energy can be used to diversify and improve the resiliency of power supply in North Korea to weather and other shocks. We show the extent to which renewable energy capacity may be developed on the local and regional level. Such an approach may contribute to targeted improvements in electricity supply while circumventing the need for costly investment in grid infrastructure. Data-based evidence from this research may inform energy policy decisions of UN organizations, nongovernmental organizations, and neighboring countries.

3. Using long-term labour market projections for policy consulting
Mrs Caroline Neuber-Pohl (German Federal Institute for Vocational Education and Training)
Mr Tobias Maier (German Federal Institute for Vocational Education and Training)

The economy and labour market are subject to structural change over time. Demographic change, technological progress, and globalisation will frame and steer the behaviour of economic actors. Political planners have a special interest in having some knowledge about the future – be it for budgetary planning, preliminary policy assessments, or the mere identification of possible areas in need of action in the future. More specific, regarding future developments of the labour market, a concern is whether the supply of skills will suffice the demand of the economy, such that growth can spur, or whether there is a possibility of labour shortages.
In this area of research, long-term labour market projections have become a more and more popular tool for policy consulting and as such contribute to the advancement of evidence-based policymaking. The reason for this especially is that evaluating alternative scenarios with a projection model can yield insights on new developments which are expected to be important in the future, however cannot or poorly be traced in past data yet. Examples for this are plentiful; they may grasp topics from digitalized production and polarization of work, to the labour market integration of refugees, or the increased usage of electric cars.
In such analyses, long-term projections can produce broad assessments to give an intuition of the direction and rough size of effects given the scenario formulations. Based on this, areas in which political action may become necessary can be identified and recommendations for policy interventions can be deduced. Furthermore, they visualize the strength of existing interdependencies in the model and by that can give valuable information about the limitations of policies. At the same time, however, there are naturally limitations to the explanatory value of the identified effects which have to be addressed carefully in the outward presentation of results.
In the presentation, we discuss the opportunities but also the challenges of long-term labour market projections in greater detail. We introduce a projection model for the German labour market and display its scope of application to evaluations of future developments and policies. Using this example, we discuss how detailed such a model should be to accurately reflect specific heterogeneities and when too great detail jeopardise the transparency of the results. Further, we highlight the possibilities and limitations of the model to present accurate results and policy proposals.