Making data reusable: Lessons from replications of impact evaluations
Speakers: Marie Gaarder, director of evaluation and global director for innovation and country engagement, 3ie
Venue: Student Central (Room 3a, 3rd Floor), Malet Street, WC1E 7HY
Time: 12.45 – 14.00 BST
In recent years, efforts to replicate the findings in scientific studies indicate that many results cannot be verified. In other words, reported findings cannot be reproduced using the original dataset and analysis code. The ‘replication crisis’ (as it has come to be known) appears to be a cross-disciplinary challenge. While this has led to a call for more replications, in practice, there are few incentives for doing so. In the international development sector, there is an emphasis on developing interventions and policies that are grounded in rigorous evidence. Given the limited resources available to tackle large-scale challenges, it is imperative to ensure policymaking and programming draw upon lessons learned from evaluations of development interventions. But, given the replication crisis, how reliable is this evidence?
In its role as a producer and synthesiser of evidence, the International Initiative for Impact Evaluation (3ie) funds impact evaluations of development interventions and policies in low- and middle-income countries. In 2018, 3ie embarked on a project to replicate published evaluation results using the data and analysis code submitted by evaluation teams.
Marie Gaarder presented the findings from this effort and discussed lessons learned and possible recommendations for various actors. Lessons included that open data is an ongoing process, policy articulation and enforcement can serve as important steps to re-inforce commitment to transparency, open data begins with informed consent, it is important to conduct replications soon after a project is complete, and build internal capacity to manage replications.
About the speaker
Marie Gaarder provides general leadership, strategic direction and guidance to 3ie’s work in evaluation, synthesis, innovation and country engagement, in addition to overseeing the evaluation and synthesis office. Marie has over 19 years of experience managing operational and research projects with a development focus.
Marie is the co-chair of the International Development Coordinating Group within the Campbell Collaboration, a member of the Research Ethics Review Committee of the Partnership for Economic Policy, and a member of the DFID-CDC Evaluation & Learning Programme Steering Group. Marie holds a PhD in Economics from University College London, an MSc in Economics from London School of Economics and a graduate degree in Political Science, Arabic and Economics from University of Oslo, Norway.