The International Initiative for Impact Evaluation (3ie) and the Berkeley Initiative for Transparency in the Social Sciences (BITSS) team at CEGA share the lessons they have learned when it comes to synthesizing evidence. This work draws on their experience of formulating policy recommendations from academic studies, and their vision of what is needed to unlock further evidence.
Over the past two decades, the volume of rigorous impact evaluation evidence has grown dramatically. This is a major success for the evidence-informed development community. Yet despite this growth, synthesizing this evidence in systematic reviews and meta-analyses that provide clear and reliable findings for decision-making remains slower, more expensive, and more fragmented than it should be.
The challenge today is no longer a lack of evidence. It is the way evidence moves through the system. To change this, 3ie, CEGA, and key partners are joining forces around a commitment to work towards a sustainable evidence infrastructure that provides reliable, trustworthy, and timely evidence to inform evidence-informed decision-making in low- and middle-income countries.
The problem with scattered and poorly organized evidence
Much of today’s evidence infrastructure depends on labor-intensive, manual processes to extract insights. Studies are often difficult to find, inconsistently pre-registered, and published in unstructured formats that make reuse challenging. Key information—on study outcomes, samples, program design, or implementation—is frequently missing, scattered, or locked in PDFs. For example, on average it takes 5-6 hours to comprehensively extract data from a single impact evaluation. And even with this effort, results are sometimes unusable or uncertain because of limitations in reporting.
As a result, organizations working on evidence synthesis spend significant time and resources locating studies and extracting data that could have been captured more systematically from the start. Evidence portals, systematic reviews, and aggregation efforts play a critical role in collating and synthesizing evidence for informing decisions. But they are time and resource intensive, partially due to how research is designed, reported, and shared.
In short, how evidence is currently published and reported creates inefficiencies whereby researchers, and ultimately funders, need to dedicate resources to get data out of papers. These inefficiencies can be addressed by developing a better research pipeline that introduces reporting and data standards earlier in the research lifecycle, producing data in a format that facilitates re-use and synthesis.
Improving the evidence pipeline
Advances in technology, including AI, have the potential to support more rapid synthesis and use of existing research. But these tools can only be as effective as the inputs they rely on. Without structured, consistent, and well-documented research, technology risks automating inefficiencies rather than solving them.
The solution is simple: Improve how studies enter and move through the evidence system. Clearer pre-registration, structured reporting, and responsible data sharing as proposed as part of the SEIC can address these issues and has system-wide benefits. When evidence is more findable and reusable:
- Evidence synthesis becomes faster and less costly
- Learning across studies and contexts improves
- Investments in evidence infrastructure deliver greater public value
Alignment, coordination, and incentives
Of course, if it was easy we would already have a well-coordinated evidence pipeline. Standards and systems alone are not enough. Adoption requires behavior change, coordination, and collaboration. Researchers operate within systems shaped by funders, journals, and institutions. Contractual data-use restrictions, inconsistent journal requirements, and limited support for reporting and data preparation all influence behavior. Addressing these constraints requires collective engagement.
By acting together, organizations can more effectively signal what good practice looks like, demonstrate feasibility, and have constructive, forward-looking conversations with funders and publishers about how their policies can better support public-good evidence.
This is about alignment, not (primarily) enforcement—and about increasing the value of investments in research over the long term.
Launching the Sustainable Evidence Infrastructure Commitment (SEIC)
3ie and CEGA are proud to lead an initiative to strengthen the evidence pipeline, so studies are conducted, reported, and published in ways that facilitate data re-use and synthesis.
Today, we are launching the Sustainable Evidence Infrastructure Commitment (SEIC), whereby the signatories commit to working in partnership to develop a sustainable, reliable, and trustworthy evidence infrastructure for supporting evidence informed decision- making through:
- Developing, adopting, and disseminating common reporting standards for impact evaluations tailored to the social sciences;
- Developing and adopting data-sharing practices that facilitate data re-use from single studies and evidence synthesis;
- Developing a federated platform of registries so that all new impact evaluation studies enter the evidence pipeline in a streamlined manner, with interoperable data;.
- Establishing and contributing to an open access shared ‘data store’ for analysis data from impact evaluation papers.
We’re proud to have the support of some of the major producers and curators of impact evaluations focused on public policy in low- and middle-income countries, including the Center for Open Science, IDInsight, International Food Policy Research Institute (IFPRI), and the Policy Innovation Research Center (PIRC).
This is an early, co-design effort and we invite evidence producers, funders, publishers, registries, and other partners to join our efforts, sign the commitment, and work together to develop the practical standards, systems, and incentives that are needed to turn the commitment into a sustainable evidence infrastructure.
If you are interested in engaging with or learning more about the initiative, please join us here.
