What we offer
At the core of our work, impact evaluations are rigorous studies that measure the effects of international development programmes. We focus on conducting impact evaluations on policy-relevant research questions where credible findings can drive decision-making.

3ie invented evidence gap maps, which provide a visual overview of existing and ongoing studies or reviews in a sector or sub-sector of international development. These maps help policymakers find the most relevant evidence for a given question, thereby improving decision-making.

Our synthesis products, including systematic reviews, integrate findings from multiple different studies which address a common research question. Because they look systematically across the evidence base to see what works and why, systematic reviews and other synthesis products are more reliable for decision-making than results from a single study used in an ad hoc way.

Often, policy decisions must be made quickly. To make sure that decision-makers get the evidence they need when they need it, 3ie has developed a set of helpdesk services and rapid evidence products. Our Rapid Response Briefs and Rapid Evidence Assessments draw from the latest high-quality research to answer policy questions in a fraction of the time of traditional evaluations or systematic reviews.

We work with governments and other organizations to build their capacities in evaluation and evidence-informed decision-making. In addition to providing trainings on a wide range of evaluation methods, we offer ongoing support services to ensure our partners have access to the most up-to-date evidence and evaluation tools.

3ie specialises in increasing access to, demand for and use of evidence by governments, parliaments, programme managers, civil society, programme participants and the media. We do this by emphasising the value of planning and engaging with stakeholders to ensure that evaluations and reviews are relevant and useful. We use robust and effective monitoring to measure evidence use so that we can convey evidence impact on programmes and policies with greater confidence.

Since 3ie was founded, transparency, reproducibility, and ethics (TRE) have been core considerations in our work. We have developed and refined tools and best practices to ensure our studies apply technically rigorous methodologies, transparently share design and analysis decisions, yield computationally reproducible analysis, and incorporate foundational principles of research ethics into design, implementation, and dissemination. 3ie’s Transparency, Reproducibility, and Ethics (TRE) Policy articulates this commitment to TRE best practices.
Evidence hub
3ie’s Development Evidence Portal is the largest-of-its-kind repository of rigorous evidence on what works in international development. This portal includes evaluations and synthesis of studies conducted in low-and middle-income countries. It combines records from 3ie’s Impact Evaluation and Systematic Review repositories, as well as, evidence gap maps.

These provide a visual display of completed and ongoing systematic reviews and impact evaluations in a sector or sub-sector, structured around a framework of interventions and outcomes.

Evidence impact summaries briefly describe how 3ie-supported evidence has informed and influenced decision makers. Each summary highlights verified instances of evidence impact.

We provide funding for replications, conduct in-house replication research and publish guidance on replication methodology. We also provide funding to original authors of 3ie-funded for preparing their raw datasets.

As part of our mandate as a knowledge producer and translator for our main audiences, we publish a range of knowledge products. These include briefs, impact evaluation reports, systematic review reports and summaries, replication papers, evidence gap map reports, scoping reports and working papers.

3ie’s Registry for International Development Impact Evaluations (RIDIE) aims to enhance the transparency and quality of impact evaluation research before it begins.
Our work
- Working with governments
3ie’s evidence programmes support studies to fill critical knowledge gaps in a sector, sub-sector or in an area with limited rigorous evidence. We fund studies under a specific theme or which address a particular question or set of questions in programme areas where our donors want to expand global public knowledge of what works and what does not.

To help address gaps in the understanding of what works and what does not, we fund a variety of studies across this programme area, including interventions focused on insurance, extension, land-use and forestry, and innovation and technology.

3ie supports impact evaluations, systematic reviews and evidence gap maps on education effectiveness that help answer the questions of what works, for whom, why and at what cost.

We fund the production of rigorous evidence on biodiversity and forest conservation programmes, environmental regulations, impact of sustainable fuels, climate change mitigation and adaptation.

3ie is supporting the generation of evidence in areas such as transparency and accountability in natural resource governance. We also fund the production of rigorous evidence on interventions to curb corruption, judicial and civil service reforms, land reforms, public financial management, conflict prevention and peacebuilding, decentralised governance and public service delivery.

We fund the production of rigorous evidence on interventions on HIV and AIDS, immunisation maternal and child health, nutrition and sexual and reproductive health through a number of evidence programmes. Evidence products from these programmes include replication studies, evidence gap maps, systematic reviews and impact evaluations.

We are supporting the generation of rigorous evidence in humanitarian contexts on interventions related to water, sanitation and hygiene, food security, multi-sectoral humanitarian programming and interventions targeting malnutrition.

We fund the production of rigorous evidence on the socio-economic and environmental impacts of public transportation by rail, bus and rapid transit systems, and essential services such as electricity and gas to expand access, foster inclusive growth, and combat climate change through sustainable systems.

3ie, in collaboration with India’s rural development ministry is working to generate rigorous evidence on the impact of the National Rural Livelihoods Mission.

We support policy-relevant studies that contribute to improving our understanding of public expenditure trends and improving the delivery of public goods.

We support impact evaluations to build the evidence base on the effectiveness of interventions that reduce the risks faced by the poor through participation in public works and employment programmes.

3ie’s has two major evidence programmes that support the generation and use of high-quality evidence for informing decision-making in the water, sanitation and hygiene sector

In alignment with our mission, 3ie promotes rigorous, efficient, and ethical use of innovative data sources for impact evaluations, including in those conducted by 3ie, by 3ie research partners, and in the global development community more broadly.

We set up our Replication Programme to address the need for a freely available global public good that helps improve the quality and reliability of impact evaluation evidence used for development decision-making. Replication is the most established method of research validation in science, yet it has not been fully embraced by the research community or development donors, leading to this gap.

3ie produces research – whether in-house or through grants or contracts – that clearly and precisely documents the data and methods used in the analysis and the materials used to conduct the study. We have experienced and highly skilled staff ready to support the transparent production of evaluations, systematic reviews, and other knowledge products. .

Fragility has expensive, long-term consequences and trying to build peace in situations of protracted conflicts is becoming the norm. In 2016, 1.8 billion people – nearly a quarter of the world's population – were living in situations of fragility. Social cohesion is widely considered important in building sustainable peace in fragile contexts.

.

We work with various departments in the Philippines government to develop and fund rigorous evidence useful for policymakers. We also support capacity-building activities for Philippine researchers and support the impact evaluation management framework of the National Economic and Development Authority.

Working in collaboration with the Office of the Prime Minister, the primary aim is to improve developmental outcomes through evidence-informed decision making in Uganda. 3ie is currently supporting evaluation of government programmes around youth livelihood, family planning, public service delivery and local governance, and universal primary education.

3ie and the government of Benin are working on a a multi-year regional initiative that aims to promote the institutionalization of evaluation in government systems across eight countries in West Africa, including: Benin, Burkina Faso, Côte d’Ivoire, Guinea-Bissau, Mali, Niger, Senegal and Togo.

This is a five-year programme (2019-23) that is working on increasing the use of evidence by policymakers in Ghana, Pakistan and Uganda. In partnership with country governments, this programme aims to develop capacity and promote innovation in increasing evidence-informed decision-making. SEDI is funded by the UK Foreign, Commonwealth and Development Office.
Funding
- Open opportunities
View our current funding opportunities for evaluations, systematic reviews and internal replication studies.

About 3ie evidence impact summaries

Read how 3ie verifies and classifies evidence impact

Highlights

Evidence impact

Early Grade Reading Study (EGRS) findings prompted USAID to fund a follow-up evaluation in Mpumalanga Province to test virtual modes of instructional coaching for English as first additional language.
USAID reconfigured its Reading Support Project to extend EGRS implementation and evaluation across all foundation phase classes in two districts of North West province.
In two addresses in 2019, President Cyril Ramaphosa revealed plans to extend the package of reading resources, expert reading coaches and lesson plans found effective in the EGRS to all foundation phase classes.

Factors supporting evidence impact

The EGRS responded to the growing demand for rigorous evidence to improve students’ performance in early grades, which had become an urgent policy priority following South Africa’s low ranking in regional and international learning assessments.
The research team’s credibility, existing networks and experience in the sector and the region helped nurture relationships with a donor community already interested in improving early learning.
Institutionalised processes, such as the National Evaluation Policy Framework, coordinated by the Department of Planning, Monitoring and Evaluation, helped create ownership of evaluation findings and provided the means for ensuring evaluation results reached the Cabinet.
Multiple donors stepped up to support the evaluation findings, broker relationships and advocate for the provincial implementation of EGRS approaches.
The researchers were effective in translating evaluation evidence into briefs, summary reports and multimedia products as part of ongoing engagement with key actors.

Factors constraining evidence impact

Implementing the EGRS findings will require national and provincial buy-in for improving teacher training approaches for home languages and developing a new cadre of reading specialists for instructional coaching.
Provincial budget constraints, with a large proportion of funds committed to personnel costs, make it unlikely that provinces could immediately implement the evaluation recommendations.
Given recent elections and a weak economy, decision makers may prioritise responding to other urgent issues in the education sector, such as upgrading toilets and other school infrastructure.

www.3ieimpact.org/evidence-hub/publications/evidence-use/how-evidence-informing-solutions-south-africas-early-grade
The authors of this brief analyse facilitators and barriers to the use of the findings from the 3ie-supported Early Grade Reading Study.

Title: Improving early grade reading in South Africa

Authors: Stephen Taylor, Jacobus Cilliers, Cas Prinsloo, Brahm Fleisch and Vijay Reddy

Status : Completed September 2018

How evidence is informing solutions to South Africa’s early grade reading crisis

Context

In South Africa, 78 per cent of children in fourth grade cannot read for meaning. Scholars have attributed the reading crisis to apartheid-era policies of vastly unequal and lower-standard schooling systems for a majority of people. They emphasise the vicious cycle of factors that lead to deficient reading, which then contribute to poor learning outcomes. These factors include insufficiently trained teaching cadres focusing on pronunciation and rote memorisation, deficient provincial educational budgets, lack of reading materials at school and home, and challenges facing predominantly poor and illiterate parents.

Although the government prioritised improving reading, there was little evidence regarding which early grade reading interventions worked. In the past, many governmental, non-governmental and academic programmes in different parts of South Africa had implemented diverse interventions to improve these skills. Most of these programmes either had short durations or did not include evaluations of what specifically worked about them, including for whom, how or why they worked. Against this backdrop, demand grew for evidence of the effectiveness of early reading programmes.

Responding to this demand, researchers affiliated with the national Department of Basic Education, the Human Sciences Research Council, the University of Witwatersrand and Georgetown University designed an evaluation to find the best approach to improving reading skills, which they named the EGRS.

The researchers piloted three approaches to improve reading outcomes in a set of early grade classrooms in two districts of North West province. They piloted high-quality Setswana readers and structured lesson plans for teachers of grades 1 and 2, combined with either centralised teacher training or on-site teacher coaching. The third approach they piloted was community-based support to increase parental involvement in children’s literacy. The EGRS sought to identify the most cost-effective of the three approaches to improve home-language reading skills in the early grades.

Evidence

The evaluation found that the on-site coaching approach had larger and statistically significant effects on more dimensions of home-language reading ability than the centralised training or parental involvement approaches. Students of teachers who received on-site coaching were approximately five months of learning ahead of the students in control schools across all measured dimensions of home-language reading proficiency.

Although on-site teacher coaching also had a positive effect on English literacy, the training approach had a significant positive effect on only three dimensions of home-language reading ability: recognition of sounds, recognition of non-words and paragraph reading. The parental involvement approach had to grapple with low parental attendance and had small, statistically insignificant effects.

Larger effect sizes were detected among boys, schools in urban townships, larger schools and communities with higher socio-economic status. Middle-ranked and above-average students in the classes and large classes benefitted the most. However, even initially weak students benefitted as long as they were exposed to two years of the coaching intervention. None of the approaches had a negative effect on the acquisition of home-language reading skills or on the learning of other subjects.

Evidence impacts

USAID has funded follow-up evaluation

Informed by the evidence from the 3ie-supported EGRS, USAID came on board to fund a follow-up evaluation of the teacher training and coaching approaches. The EGRS II evaluation compared two mechanisms to provide expert coaching to teachers in Mpumalanga province. The first mechanism included tablet-based lesson plans for English as first additional language, graded readers and virtual coaching of teachers. The second used the EGRS model of on-site coaches who visit teachers and compared them with virtual coaches who support teachers through calls and messages.

Findings prompted reconfiguration of USAID project

Findings also led USAID to change its Reading Support Project to implement and evaluate a longer duration of the structured learning programme, with centralised training and on-site coaching, in North West province. The parental involvement approach was discontinued due to the insignificant effects found in the EGRS evaluation.

‘The department’s early grade reading studies have demonstrated the impact that a dedicated package of reading resources, expert reading coaches and lesson plans can have on reading outcomes. We will be substantially expanding the availability of these early reading resources across the foundation phase of schooling.’

Cyril Ramaphosa

President

Republic of South Africa

South Africa’s Cabinet has backed implementation of the coaching approach

The president and the Department of Basic Education both announced, on multiple occasions, the government’s plan to implement the EGRS package of reading resources, lesson plans and expert coaches to improve reading comprehension in the foundation phase over the next five years. The government developed an implementation plan, informed by the 3ie-funded evaluation’s findings, that involves multiple stakeholders, including steering committee members inside and outside the government.

Suggested citation

International Initiative for Impact Evaluation (3ie), 2020. How evidence is informing solutions to South Africa’s early grade reading crisis [online summary], Evidence Impact Summaries. New Delhi: 3ie.

If you have any suggestions or updates to improve this summary, please write to influence@3ieimpact.org

Last updated on 11th November 2020