Why does India account for more than half of all impact evaluations in South Asia? What does that reveal about who’s producing evidence in the region and where critical gaps remain? This is the latest entry in our blog series looking at the state of impact evaluation evidence in different regions, based on data from the Development Evidence Portal (DEP). To recap: the DEP is the world’s largest repository of impact evaluations and systematic reviews focused on low- and middle-income countries, and it is updated monthly with new studies.
While South Asia trails regions like Sub-Saharan Africa and East Asia in the number of evaluations, it still represents a substantial body of evidence that offers important insights into development trends in the region—some of which contrast with patterns we observed in our earlier blog on the Middle East and North Africa. Using the World Bank’s regional classifications, it comprises of Afghanistan, Bangladesh, Bhutan, India, Maldives, Nepal, Pakistan, and Sri Lanka. Currently, South Asia is the region with the fourth most impact evaluations (IEs) in the DEP, with 2,742 IEs from the region. That puts South Asia quite a way behind Sub-Saharan Africa (5,843 IEs) and East Asia (5,002), and just a smidge behind Latin America (2,753).
Which countries in South Asia have the most evaluations?
Over half of impact evaluations in South Asia have taken place in India (1,468 IEs). Bangladesh (650), Pakistan (308), and Nepal (226) also have substantial evidence bases, while the remaining countries each have fewer than 100 IEs. To some extent these differences track the population sizes of these countries (“World Development Indicators | DataBank,” 2015), although Nepal is notable for having over three times as many IEs as Sri Lanka, despite similar populations.
Countries in South Asia that have the most impact evaluations
What sectors and interventions are frequently evaluated in South Asia?
The IE literature in South Asia is dominated by evaluations of health sector interventions, which are responsible for almost 40% of all IEs in the region. There is a specific focus on nutrition, reproductive health, maternal health, and child health interventions. The most frequently evaluated health interventions in this region are nutrient supplementation (132 IEs); nutrition education and behavior change (63); nutrient fortification in foods (50); and maternal and infant nutrition care (25). Beyond the health sector, commonly evaluated interventions include access to microcredit (83 IEs), conditional cash transfers (49) and unconditional cash transfers (48). Other sectors in which evaluations are commonly conducted in the region are social protection; agriculture, fishing, and forestry; and public administration. These cover 32% of all IEs produced in SA. Two rarely evaluated sectors in this region are transportation (28 IEs) and information and communications technology (14).
Sectors in South Asia that have most impact evaluations
What projects are being evaluated?
Two large-scale government projects have been evaluated numerous times: the Mahatma Gandhi National Rural Employment Guarantee Act 2005 (MGNREGA), implemented in India; and the Benazir Income Support Program (BISP), implemented in Pakistan. There are 53 IEs that study MGNREGA and 19 IEs that study BISP. MGNREGA provides guaranteed wage employment to rural households with adult members willing to do unskilled manual labor, amounting to at least 100 days in a financial year. It has been implemented throughout rural areas in India. The BISP is also a federal unconditional cash transfer program that aims to reduce poverty in Pakistan, primarily through transfers to women to improve financial independence and decision-making power within the family.
Interestingly, the details of how each project has been implemented have influenced how they have each been evaluated. Researchers have commonly used experimental and quasi-experimental designs to evaluate MGNREGA, whereas BISP has solely been evaluated through quasi-experimental designs like regression discontinuity, which relies on arbitrary cut-off points.
Who writes IE papers focused on South Asia?
Many evaluations in South Asia are authored by researchers based outside the region. Based on institutional affiliations listed on the papers, we have found that the largest proportion of authors are based in the United States, with about 42% of all South Asia IEs having at least one author from the US, followed by India (26% of studies).
Approximately 40% of studies conducted in South Asia have no “local” authors who are based in the country where the study was conducted. The absence of locally-based authors is concerning if you think—as we do—that evaluations are most rigorous and useful when conducted by researchers who are familiar with the local context and are embedded in local research and policymaking institutions.
A diverse range of funding organizations have supported impact evaluation research in this region. These include the Gates Foundation (200 IEs), USAID (155), the World Bank (140), FCDO (136) and 3ie (76). This suggests substantial global interest in supporting evaluation research in South Asia, although the current funding context leaves the persistence of such support in doubt.
Top funding agency who supports impact evaluations
These findings highlight opportunities for funders and policymakers to foster a more balanced and inclusive evidence ecosystem. Supporting locally-led research and expanding evaluations into underrepresented sectors will be essential to strengthening South Asia’s evidence base. We’ll soon be looking at DEP-based insights on other regions, so keep an eye on our website as we explore more regions and research areas in the future.
