Conn, K. (2014) Identifying effective education interventions in Sub-Saharan Africa: a meta-analysis of rigorous impact evaluations. Graduate School of Arts and Sciences, Columbia University.Link to Source
Headline Findings: a summary statement
Interventions involving pedagogical methods have improved learning outcomes for students in sub-Saharan Africa. Other interventions can work well in complementing the improvements in pedagogical methods.
The author included 56 experimental and quasi-experimental studies, evaluating 66 separate treatments. Forty-one of the studies were from the field of economics, 15 from education and 10 from health. Geographically, 48 per cent of studies were from east Africa, 23 per cent from southern Africa and 29 per cent from west Africa.
Implications for policy and practice
- Pedagogical interventions have the largest effect on learning outcomes (0.918 standard deviations); this effect is larger than that of any of the 11 other interventions analysed by 0.54 standard deviations (p=0.025). The overall effect is smaller when outliers and only high quality studies are included. However, even when controlling for a range of study and intervention level variables, the effect of pedagogical interventions is more than 0.30 standard deviations larger than the other intervention areas combined.
- Long-term teacher mentoring or coaching is the most-effective teacher training intervention (0.249 standard deviations).
- Greater length of time in class results in better test scores in mathematics (0.412 standard deviations), however this result is based on only one study and should be interpreted with caution.
Class size and composition
- There is limited evidence on the effect of class size reductions and peer effects on performance from sub-Saharan Africa. Tracking students by performance can improve outcomes. The impact of class size cannot be deduced from the studies included here.
- Providing textbooks or flipcharts has a limited impact on learning outcomes (0.022 standard deviations).
School meals and supplements
- Provision of school meals and supplements has varying but low effects on performance (0.02-0.09 standard deviations).
- Malaria treatment is the most common health intervention in this set of studies and has a positive effect on cognitive outcomes (0.189 standard deviations).
Student and teacher motivation
- Using incentives to increase student motivation has a greater effect on learning outcomes than incentivising teachers (0.288 standard deviations versus 0.075, P=0.2).
Student or community financial restrictions
- Interventions that reduce the financial burden for students and families, such as cash transfers, elimination of school fees, or providing uniform, have a small effect on learning outcomes (0.036 standard deviations).
School or system accountability
- Interventions that increase school-level and system-level accountability, such as providing information on school performance or funding to the community, have a large effect on student performance (0.147 standard deviations).
- School management interventions have a small effect on learning outcomes (0.016 standard deviations).
- Private schools have an advantage over government schools in Kenya and Nigeria, although this was not the focus of the present analysis.
Implications for further research
The author calls for future research to go beyond the six countries in which most of the research on this topic has been conducted (Kenya, Nigeria, South Africa, Uganda, Burkina Faso and Madagascar). There is also a need to study topics that are relevant to sub-Saharan Africa, such as language of instruction, multi-grade teaching and class size. There is also a need for further study of pedagogical methods (particularly the use of different techniques) and student incentives.
There has been progress in improving access to education in sub-Saharan Africa over the past decade. However, low attendance and information retention continue to be an issue. The results of international assessments suggest that there are significant challenges in ensuring children learn when they get to school, with students demonstrating low levels of proficiency in English and Math for instance. There is therefore a need to identify which programmes may be most effective in strengthening education systems and improve learning outcomes in the region. This review assesses the effectiveness of 12 different types of education intervention on learning outcomes in sub-Saharan Africa.
To assess the available literature and identify which interventions have the largest effect on learning outcomes in sub-Saharan Africa and to evaluate why some interventions are more effective than others.
The author included experimental and quasi-experimental studies of the impact of education, economic or public health interventions on learning outcomes for students in formal education in sub-Saharan Africa. The author searched a broad range of sources to identify published and un-published studies published from 1980 onwards, including JSTOR, Academic Search Complete, Business Source Complete, EconLit, Education Research Complete, ERIC and 25 individual journals and 22 organisational websites. She also conducted citation tracking, searched conference presentations and contacted researchers in the field. The author systematically extracted data and appraised the quality of included studies. She used random-effects meta-analysis to evaluate the relative effect of different types of interventions and explain variation in effect sizes within and across intervention types.
The review has clear inclusion criteria, and includes a comprehensive search of the published and unpublished literature. The author assessed and reported summary statistics for study quality, explicitly detailed data extraction and synthesis procedures, adjusted for clustering errors, and conducted appropriate and relevant subgroup analyses. However, the review has some limitations. The author did not articulate whether two independent researchers screened the full-text articles for inclusion, or describe other methods for avoiding bias in the selection of studies for inclusion. Moreover, it is not clear if studies in other languages than English were included and there is no table with description of a table or summary of the characteristics of the participants, interventions and outcomes for the included studies.