Zhang, L., Rozelle, S. and Shi, Y., 2013. Paying for performance in China's battle against anaemia, 3ie Impact Evaluation Report 8. New Delhi: International Initiative for Impact Evaluation (3ie)
Link to Source
The aim of this study is to find new evidence on how to match supply-side incentives with good population health as well as how to estimate the additional benefits of health improvement in other domains. Specifically, this large-scale randomised field experiment was implemented to assess the impact of information, subsidies and financial incentives for principals to reduce anaemia amongst students in rural China.
The study was carried out in 72 randomly selected schools with a sample of 3,944 fourth and fifth graders. The schools were divided at random into four arms: three sets of 15 treatment schools and one set of 27 control schools. Those schools assigned to arm 1 received only information about anaemia, whilst those in arm 2 received information and a subsidy, and arm 3 schools received information, a subsidy and an incentive. Arm 4 schools received no treatment, as they formed the control group. Baseline school, student and household surveys were conducted in October 2009, with follow-up occurring in May 2010.
To assess how students’ haemoglobin (Hb) concentrations and anaemia rates changed in the intervention arms relative to the control arm, the authors used unadjusted and adjusted ordinary least squares regression analysis. They also documented the correlation between changes in anaemia levels and changes in educational outcomes (e.g., grades, attendance and cognitive skills), behavioural responses to each intervention arm (such as principals’ strategies to reduce anaemia), and how the impact of treatment arms varied by the existence of pre-existing financial incentives offered to principals for good average test scores in their schools.
At baseline, it was determined that 24.9 per cent of surveyed students were anaemic. The results show that during the treatment period, students’ Hb levels increased in all four arms, but larger increases were seen in intervention arms relative to the control arm. Unadjusted estimates of Hb changes in each of the intervention arms relative to the control arm, although not statistically different from zero at conventional significance levels, were positive (information, 0.24 Hb; subsidy, 0.22 Hb; incentive, 0.23 Hb). The adjusted results saw a 9 per cent to 23 per cent reduction in the rate of anaemia for the incentive arm. This impact was statistically larger than the impact of the information and subsidy arms on anaemia reduction. Additionally, schools with pre-existing test score incentives in the intervention arms experienced larger Hb gains than did their counterparts without them.
The results of this project offer evidence on the effectiveness of rewarding providers for better health outcomes and show that incentives could be used to promote the use of a broad range of underused supply-side health technologies more effectively. Additionally, the fact that primary school principals with pre-existing incentives for good test scores made more effective use of subsidies to reduce anaemia provides evidence of the importance of recognising interactions between new incentives with other existing incentives.
- Effectiveness of provider incentives for anaemia reduction in rural China: a cluster randomised trial, BMG, July 2012
- Do you get what you pay for with school-based health programs? Evidence from a child nutrition experiment in rural China
- Conducting influential impact evaluations in China: the experience of the Rural Education Action Project, JDEFF
- Teachers’ beliefs of behaviors, learning, and teaching related to minority students, Taylor & Francis Online, Jan 2015
About this impact evaluation
Anaemia - or iron deficiencies - affects nearly a quarter of students from poor rural areas of China. In other words, there are up to 20 to 30 million children with anaemia in the country. Anaemia has been shown to reduce cognitive ability, lead to learning problems and be associated with poor school attendance, bad behaviour and overall poor educational performance. In the past, there have been top-down efforts to reduce anaemia, but, they have had mixed results.
Large scale field experiments, run collaboratively with local and regional educational agencies, will be set up to examine the incentives of school principals in China’s rural primary schools in combating anaemia. The Randomized Control Trials (RCT) approach will facilitate the evaluation and provide policy makers clear and simple evidence of different ways to address this problem. One of the key treatments will examine how incentives can be designed to induce local educators to solve anaemia and improve educational outcomes.
The evaluation will provide evidence on the effectiveness, including cost-effectiveness, of using Pay for Performance incentive mechanisms (in lieu of direct monitoring) to deliver health inputs, to realize health outcomes and to achieve educational impacts.
Read the final report of this project: Paying for Performance in China's Battle Against Anemia
View presentation: Sean Sylvia's presentation ‘Paying for Health: Experimental Evidence from a School-Based Health Program in Rural China’ at the 3ie 2011 impact evaluation conference.
Other relevant publications (published or forthcoming):
Matthew Boswell, Scott Rozelle, Linxiu Zhang, Chengfang Liu, Renfu Luo, Yaojiang Shi, 2011. Conducting influential impact evaluations in China: the experience of the Rural Education Action Project. Journal of Development Effectiveness Vol. 3, Iss. 3, 2011
Katherine Loosley, Sean Sylvia, Prashant Loyalka. 2013. "The Intrinsic Motivation of Teachers and Student Performance: Evidence from Rural China" Working Paper, Stanford University, Palo Alto. (in progress)
Chu Yang, Sean Sylvia, Luo Renfu, Linxiu Zhang, Alexis Medina, Scott Rozelle. 2013. "Does the Health of Classroom Peers Affect Student Achievement?" Working Paper, Stanford University, Palo Alto. (in progress)
Enyan Yang, Sean Sylvia, Linxiu Zhang, Guangrong Ma, Chu Yang, Natalie Johnson, and Scott Rozelle. 2013. “Teacher Contract Status, Centralization and Student Achievement in Rural China,” Working Paper, Chinese Academy of Sciences, Beijing. (in progress)
Luo, Renfu, Linxiu Zhang, Chengfang Liu, Qiran Zhao, Yaojiang Shi, Grant Miller, et al. (2011). “Anaemia among students of rural China’s elementary schools: Prevalence and correlates in Ningxia and Qinghai’s poor counties,” Journal of Health, Population and Nutrition 29(5):471-485.
Sean Sylvia, Renfu Luo, Linxiu Zhang, Yaojiang Shi, Alexis Medina, and Scott Rozelle. (2013). “Do You Get What You Pay for With School-based Health Programs? Evidence from a Child Nutrition Experiment in Rural China,” Economics of Education Review, 37, 1-12.
Sean Sylvia, Renfu Luo, Linxiu Zhang, Alexis Medina, Scott Rozelle, Marcos Vera-hernandez and Grant Miller, 2013. "Multitasking in Public Service Delivery: Experimental Evidence from School-based Health Programs" Working Paper, Stanford University, Palo Alto. (in progress)
Renfu Luo, Grant Miller, Scott Rozelle, Sean Sylvia, and Marcos Vera-Hernandez. 2013. “Incentives and Resources in Public Service Delivery: Evidence from School-based Anemia Reduction programs in Rural China,” Working Paper, Stanford University, Palo Alto. (in progress)
Yunfan Yang, Huan Wang, Linxiu Zhang, Sean Sylvia, Renfu Luo, Yaojiang Shi, Wei Wang and Scott Rozelle. (2013). “The Han-Minority Achievement Gap, Language and Returns to Schools in Rural China” (Submitted to Economic Development and Cultural Change)
Grant Miller, Renfu Luo, Linxiu Zhang, Sean Sylvia, Yaojiang Shi, Patricia Foo, Qiran Zhao, Reynaldo Martorell, Alexis Medina, and Scott Rozelle. (2012). “Effectiveness of Provider Incentives for Anaemia Reduction in Rural China: a Cluster Randomised Trial,” British Medical Journal, 345:e4809.