Transparency in Comparative Social Science Research |
|
| Coordinator 1 | Dr Elena Damian (University of Leuven) |
| Coordinator 2 | Professor Bart Meuleman (University of Leuven) |
| Coordinator 3 | Professor Wim van Oorschot (University of Leuven) |
Transparency is one of the foundations of the scientific method (cfr. Merton’s scientific norm of communalism, 1973) and has two main functions: to enable readers evaluate the validity and reliability of a study’s findings (evaluation transparency) and to conduct direct replications (replicability transparency) (Damian, Meuleman, van Oorschot, forthcoming). Despite its acknowledged importance in academia, very little measures have been taken to encourage greater research transparency. For example, it is still very uncommon for journals to have research transparency guidelines and there are no real incentives for scholars to voluntarily provide clear records of their studies. As a consequence, in the last decade, there has been increasing evidence of failure to replicate experiments (e.g., Open Science Collaboration, 2015), a growing popularity of various questionable research practices (e.g., Fanelli, 2009; Simmons, Nelson & Simonsohn, 2011; John et al., 2012) as well as cases of misconduct, which mostly feature in experimental psychology and medical research.
As cross-national survey research generally analyses large-scale publicly available data sources, this field seems to be relatively well protected against the replication crisis. However, we believe that its specific nature poses some particular threats to transparency. For instance, collecting data in multiple countries is a complex process that comes with many methodological issues. Despite this, researchers are still rarely required to or voluntarily do provide all necessary information on data and preparation procedures. As a result, many of these issues and data limitations are insufficiently reported in studies. Furthermore, although performing secondary analyses on cross-national data is a long and complex process (e.g., operationalisation of theoretical concepts, treatment of missing values, dealing with outliers etc.), many steps performed in this stage remain undocumented. Therefore, disclosure about the data and analytical procedures is crucial to evaluate and replicate this type of research.
In this session, we welcome papers that address the following topics:
(1) Theoretical contributions about transparency issues and/or possible solutions in cross-national survey research or quantitative social sciences research in general
(2) Empirical evidence of current research practices in quantitative social science research
(3) Examples of good practice (e.g., sharing an experience of publishing a transparent substantive study with replication materials and the lessons learned from the process)