Mohanan, M, Miller, G, La Forgia, G, Shekhar, S and Singh, K, 2016. Improving maternal and child health in India: evaluating demand and supply strategies, 3ie Impact Evaluation Report 30. New Delhi: International Initiative for Impact Evaluation (3ie).
Link to Source
This study evaluates two policies of India for their impact on maternal health; the Chiranjeevi Yojana (CY) in Gujarat and the Thayi Bhagya Yojana (TBY) in Karnataka. This evaluation also studies the differential impact of incentive contracts based on quality of care (‘inputs’) or on favorable health outcomes (‘outputs’) on provider behaviour, quality of care, and maternal and infant health outcomes.
The Special Census report published by the Registrar General of India in December 2013 reported estimates from the Census’ Sample Registration System that show that maternal mortality ratio (MMR) in India fell from 212 in 2007-09 to 178 in 2010-12. While the decline is encouraging, the national average for India continues to be still higher than that for the rest of South Asia. Over the past decade, the central government and various state governments in India have introduced a range of programmemes aimed at improving maternal health indicators. A central feature of several new programmemes is to encourage pregnant women to use designated medical facilities for their deliveries rather than giving birth at home.
The IMATCHINE (Improving maternal and child health in India: evaluating demand and supply strategies) project was developed in response to these policy challenges and seeks to provide evidence on the effectiveness of these policies in improving maternal and child health.
This report covers three related, but independent, studies that were conducted as part of the IMATCHINE project: Evaluations of the CY and TBY programmemes, and the experimental evaluation of incentive contracts.
- To evaluate the impact of two maternal and child health programmemes in the states of Gujarat (Chiranjeevi Yojana – CY) and Karnataka (Thayi Bhagya Yojana - TBY) on rates of institutional delivery and on maternal and child health outcomes.
- To provide robust empirical evidence to inform policies on how to structure contracts with private providers to incentivise them to improve quality of care and outcomes for maternal and neonatal health.
Maternal mortality rates in South Asia are among the highest in the world. Some Indian states have introduced voucher programmes; through which pregnant women can use BPL (below poverty line) card status to receive free maternity care at designated private maternity hospitals and the contracted providers receives flat payments for each BPL delivery. Others are developing conditional cash transfer programmes to encourage institutional deliveries and also offer cash payments to mothers who deliver their babies in private obstetric facilities.
The study evaluates two such programmes to reduce maternal and infant mortality in the states of Gujarat (Chiranjeevi Yojana) and Karnataka (Thayi Bhagya Yojana) and their impact on maternal and infant health outcomes including institutional delivery rates. The study evaluates performance-based incentive payments to providers (for quality of medical care and health improvement) and tests the effect of these supply-side incentives both with and without the voucher programmes. The studies use a mix of quasi-experimental designs (regression discontinuity, difference-in-difference, before-after) to evaluate the impact of the two programmes on rates of institutional deliveries and on maternal health outcomes in the population. It also includes a large cluster randomised evaluation to assess the impact of provider incentives in Karnataka. Providers will be randomised into one of two incentive contracts or to a control group. Incentive contracts are tied to improvements in quality of care provided during delivery or to improvements in maternal and infant health outcomes in the catchment area served by the providers.
The study uses household surveys to collect data on socioeconomic, human capital, quality-of-life variables (including BPL index components) and as well as information about deliveries, fertility histories, morbidity and mortality (for mothers, infants and children), anthropometrics, birth-related complications, health service use and spending. The surveys will collect data from approximately 36,000 households in Karnataka (two rounds in 180 clusters with 100 mothers in each cluster) and 5,000 households in Gujarat. Additionally, provider surveys will collect data on infrastructure, staffing, provider qualifications and process measures of provider performance. The experimental evaluation of supply-side incentives in Karnataka will be conducted among approximately 600 providers in the 180 clusters, with 60 clusters in each of the three treatment arms.
CY : The objective of the Chiranjeevi Yojana (CY) programmeme is to promote institutional deliveries among women in households below the poverty line, especially in rural areas of Gujarat. In 2006, the government had introduced CY in response to the acute ulack of trained obstetricians in public-sector facilities in rural areas. The policy aimed to leverage the presence of a large and vibrant private sector in healthcare available across the state by contracting with a large number of private-sector providers. There providers were offered INR 1600 as reimbursement per delivery for offering free maternity care to women from poor households. The programmeme was launched in early 2006 in five northern districts of Gujarat, and scaled up to the rest of the state by the end of 2007. By 2012, over 800 private-sector hospitals were part of the programmeme, which helped pay for more than 800,000 deliveries.
TBY : The Thayi Bhagya Yojana (TBY) in Karnataka is similar to the CY in Gujarat. The state contracted with private-sector providers in the six C-category districts of Bagalkot, Bidar, Bijapur, Gulbarga, Koppal and Raichur, and also the district of Chamarajanagar to provide free obstetric care services poor mothers in these areas. One major difference relative to CY was that the TBY paid INR 3000 as reimbursement for each delivery, compared to INR 1600 in CY. In addition, in March 2010, the state also announced the TBY Plus across all the districts in Karnataka, which provided a cash incentive of INR 1000 for women from India’s scheduled caste and scheduled tribes and those who were from poor households, were above 19 years of age. This programme covered women for their first two live deliveries in private hospitals.
Experimental evaluation of performance incentives contracts:
Given growing interest among various national and state governments to contract with the private sector, there is potentially important scope for including explicit rewards for good performance in these contracts. In particular, even if programmes like CY and TBY are a great opportunity to reward contracted providers for improving quality of care and key outcomes of maternal and neonatal health, one important concern is that even if programmes succeed in increasing medical attendance of childbirths and institutional deliveries, doing so may have little impact on actual health outcomes per se if the quality of medical care in rural areas is poor.
In order to provide rigorous empirical evidence on whether supply incentives (pay-for-performance) structured as input-based contracts or output-contingent contracts yield better performance from providers, the study uses a randomised controlled trial among private obstetric care providers in rural Karnataka
CY evaluation: The study analysed the expansion of CY across Gujarat’s districts between 2005 and 2007. By 2013, approximately 800 private-sector hospitals were participating and the programme had helped pay for more than 800,000 deliveries. We collected data on retrospective birth histories and outcomes from 5597 households in all districts in Gujarat as part of this study. The sample included households who had assets within 5 points of the eligibility cut-off to be considered ‘below poverty line’ (BPL) (it does not include the poorest or the richest sub-groups). We combine this data with the rollout dates of the CY programme across districts to implement a difference in differences (DD) analysis. Data from an additional 6484 households from the DLHS-3 in Gujarat were used in parallel analyses.
The team conducted multivariate DD ordinary least-squares regression analyses to determine if changes in our primary outcomes were associated with the staggered introduction of CY across Gujarat’s districts. We relied on the timing of births – as reported by mothers in retrospective birth history collected in both surveys – together with mother’s district of residence to determine if CY had been introduced in the district when a delivery occurred.
TBY evaluation: Similar to the CY evaluation, the study relied on a DD method using data from a retrospective pregnancy history questionnaire that asked for details about each woman’s three most recent births since 2008. The study also uses data collected in each of the study clusters as part of the incentives experiment conducted between December 2013 and August 2014. These are rural areas (at the level of the sub-district) that are predominantly served by private obstetric care providers.
Experimental evaluation of performance incentives contracts: The randomised experiment consisted of two orthogonal treatment arms and a control arm. Eligible rural private obstetric providers were randomly assigned to one of the three arms: (A.) Output-based contracts that reward lower rates of post-partum haemorrhage, pre-eclampsia, sepsis and neonatal mortality; (B.) Input-based contracts that reward better provision of healthcare inputs based on WHO guidelines for obstetric care and (C.) Control contracts that provide same information on best practices as other arms but no financial incentives. The contracts were structured such that providers had the potential to earn approximately INR 150,000 (about USD 2,700 at the time of the contract, equivalent to more than 15 per cent of a mid-level government doctor’s salary and more than double the state per-capita income), to be paid at the end of the intervention period (approximately 1 year).
The study finds that the CY programme had no significant effect on institutional delivery rates or maternal health outcomes. The programme also did not reduce out-of-pocket expenditures significantly between 2005 and 2010. Previous evaluations that found large programme impacts did not account for self-selection of women into hospitals for delivery or for secular increases in institutional delivery over time, thus leading to upward biases in estimates.
For TBY, the study finds that the programme did not have any significant effect on overall rates of institutional deliveries, or on maternal and child health outcomes. Intensive programme implementation led to small reductions in expenditures (INR200) and a 3.5 percentage point increase in delivery at private facilities. Comparing provider perceptions across varying programme intensity districts, there is no evidence that private providers have a clear understanding of the TBY programme or how to best leverage it for their patients.
In the incentive experiment, input incentive contracts reduced rates of post-partum haemorrhage, a leading cause of maternal mortality in India, by 28 per cent, while there was no change caused by output incentives. Part of the explanation appears to be that providers respond less to performance contracts with greater risk that efforts will not be rewarded.
- Millions Saved: Case Studies in Global Health Third Edition
- Read project in focus: Roadblocks to safe deliveries: Evaluating impact on maternal and child health in India
- INVESTING IN GLOBAL HEALTH SYSTEMS SUSTAINING GAINS, TRANSFORMING LIVES, NAP
- Personality Traits and Performance Contracts: Evidence from a Field Experiment among Maternity Care Providers in India, AEA,2017
- Gujarat Chiranjeevi programme, Indian Express, 30 Dec 2013
- Gujarat's maternal health scheme is a failure: Study, Times of India, 30 Dec 2013
- Gujarat's Chirajeevi Yojana fails to deliver, Down to Earth, 2 Jan 2014
- Gujarat Experiments With Expansion of Public Health Insurance, New York Times, 3 April 2014
- Gujarat’s Chiranjeevi Yojana program fails to improve maternal health in India, The Siasat Daily, 12 Dec 2013
- Gujarat's Chiranjeevi Yojana program fails to improve maternal health in India, News Track India, 12 Dec 2013
- Maternal health program in India failing to deliver, study shows, Science Daily, 11 Dec 2013
- Maternal health program in India failing to deliver, study shows, EurekAlert, 11 Dec 2013
- Maternal health program in India failing to deliver, study shows, Medical Express, 11 Dec 2013
- Gujarat's Chiranjeevi Yojana program fails to improve maternal health, Asian News International Press release, 12 Dec 2013
- Gujarat's Chiranjeevi Yojana program fails to improve maternal health in India, Business Standard, 12 Dec 2013
- Gujarat's Chiranjeevi Yojana program fails to improve maternal health in India, ANI News, 12 Dec
- 122 mothers die per one lakh births in Gujrat, DNA News, 11 April 2014