Petrosino, A, Morgan, C, Fronius, T, Tanner-Smith, E, and Boruch, R, 2016. Interventions in developing nations for improving primary and secondary school enrolment of children: a systematic review, 3ie Systematic Review 3. London: International Initiative for Impact Evaluation (3ie).Link to Source
Headline Findings: a summary statement
They find that the interventions increase primary- and secondary-school enrolment, attendance and progression. The interventions also improve maths and language achievement, to a greater extent than standardised test performance and other achievement measures.
The authors included 73 experimental and quasi-experimental studies assessing 36 different interventions.
Implications for policy and practice
For each educational outcome, the authors estimate the average improvement due to the interventions, calculate the 95 per cent confidence interval and tally the number of studies (n) on which the estimate is based. The results are presented as standardised mean differences (which present impacts measured in terms of standard deviations of the outcome variable such as a test score). Their results are as follows:
- School enrolment: SMD=0.18, equivalent to 9% improvement in intervention over control group, meta-analysis of 34 programmes,
- School attendance: SMD=.15, equivalent to 8% improvement, 33 programmes,
- Drop-out: SMD=0.05, or 3% improvement, 18 programmes,
- Progression: SMD=0.13, or 7% improvement, 15 programmes,
- Mathematics achievement: SMD=0.16, or 8% improvement, 25 programmes,
- Language: SMD=0.18, or 9% improvement, 25 programmes,
- Standardised achievement tests: SMD=0.06, or 3% improvement, 10 programmes,
- Other measures of achievement: SMD=0.05, or 3% improvement, 5 programmes.
The authors also classify the interventions into five categories and group the studies accordingly. Across the primary and secondary educational outcomes, they find the average improvement by category as follows:
- New Schools/Infrastructure—i.e. School Repair, New Schools, Community Schools and Road Improvement studies: SMD=0.44, or 21.5% improvement, evidence from 5 programmes.
- Health Care/Nutrition—i.e. Menstrual Cups, Single Deworming, School Meals, Combination Deworming, Malaria Prevention and Asthma/Epilepsy Treatment studies: SMD=0.23, or 11.5% improvement, 12 programmes.
- Economic—i.e. Microfinance, User-Fee Elimination, Voucher, Uniforms, Conditional Cash Transfer, Girls Fellowship, Conditional Food Programme and User-Fee Reduction studies: SMD=0.16, of 8% improvement, 25 programmes.
- Providing Information/Training—i.e. Information on Returns to Education, Community Participation and Empowerment, Family Planning, Providing Report Cards and Early Intervention Programme studies: SMD=0.06, or 3% improvement, 5 programmes.
- Educational Practices/Programmes—i.e. Remedial Education, Funding, English-Language Machines and Activities, Improving School Management, Flip Charts, Teacher Incentives and Computer studies: SMD=0.06, or 3% improvement, 12 programmes.
- Asthma/epilepsy treatment, early intervention, malaria prevention, road improvement and building new schools have the greatest positive effect on educational outcomes (36, 30, 29, 24.5 and 23 per cent, respectively). Four interventions—providing returns to education, community participation and empowerment, microfinance and family planning—had a negative impact on educational outcomes, although many of these findings are based on only one study.
Implications for further research
The authors suggest future reviews should use enrolment measures and learning outcomes as the inclusion criteria (learning outcomes were assessed only when they were available in the included enrolment studies.) They also stress the importance of reviewing interventions with individual and community outcomes in addition to enrolment (health, for example), given such interventions’ cost-effectiveness and appeal to policy makers.
Finally, the authors highlight the need to improve the descriptive validity of primary studies to facilitate future systematic reviews, as many studies did not clearly specify the number of students or level of randomization for instance.
To assess and synthesise the evidence on the effects of school enrolment policies and programs in developing countries on enrolment, persistence and learning outcomes.
The authors included studies assessing the effects of interventions implemented in low- or middle-income countries on school enrolment, attendance, drop-out or progression. To be included studies had to use experimental or quasi-experimental study designs and be published or available before December 2009.
The authors' searched published and unpublished literature using a range of different strategies, including electronic searches of bibliographic databases, hand searching, reference checks, consultation with key experts and internet searches of sources of grey literature. The authors then systematically extracted data from included studies and synthesized findings using meta-analysis.
The review uses appropriate methods to reduce risk of bias in terms of explicit inclusion criteria, independent screening and data extraction, and clear reporting of information about these studies. The authors report which studies are at a high risk of bias and analyse studies separately by methodological quality. However the systematic review has limitations in the approach to meta-analysis of heterogeneous interventions. The authors discuss the heterogeneity between the studies and provide the statistics to demonstrate this heterogeneity, but they do not fully address this in their meta-analysis. This makes it difficult to interpret the findings: it is difficult to draw strong conclusions about the effects of different programmes across a range of contexts, and it is potentially misleading to compare different types of interventions in the way the authors do in one meta-analysis across a broad intervention areas.