Each year, billions of dollars are spent attempting to help individuals affected by humanitarian crises in low- and middle-income countries. Despite the large investments and the critical importance of the mission, relatively few humanitarian assistance efforts undergo rigorous evaluations to test whether they are achieving their intended impacts, and important knowledge gaps persist. A recently-completed impact evaluation conducted by 3ie in the context of the Venezuelan crisis contributes to filling this void. Our study shows that a multipurpose cash transfer program for Venezuelan migrants in Colombia yielded positive effects on migrants' incomes, food security, savings, and labor force participation rates.
Humanitarian crisis in Venezuela
Millions of Venezuelans have been displaced in recent years as a result of economic collapse in the South American country. It is estimated that 1.8 million Venezuelan migrants and Colombian returnees are living in neighboring Colombia , with millions more throughout the region. Many of the displaced suffer from poverty and economic vulnerability, putting pressure on host communities and public services. In response, the ADN Dignidad program has been supporting Venezuelan migrants, Colombian returnees, and members of host communities through multipurpose cash assistance (MPCA) coupled with advisory services on nutrition, legal, and social services. Participating households receive a monthly transfer of approximately $100 for six months, and thus far the program has reached more than 250,000 crisis-affected individuals. The program is implemented by humanitarian organizations Action Against Hunger, the Norwegian Refugee Council and Danish Refugee Council and is funded by the United States Agency for International Development Bureau of Humanitarian Affairs.
Designing an impact evaluation in a crisis context
Conducting rigorous impact evaluations in the context of a humanitarian crisis has some added complexity relative to the development sector at large. For one, the emergency nature of crises means that aid needs to be deployed quickly and reach as many individuals as possible. Taking the time and resources to design a prospective randomized control trial (RCT, the “gold standard” of impact evaluation) and field a baseline survey in these contexts may be challenging. However, as we found in the context of ADN Dignidad, clearly-established and well-implemented rules of operation along with strong monitoring systems offer opportunities to measure program impacts through quasi-experimental approaches. These evaluation strategies are not explicitly experimental, but they mimic the variation in treatment generated by an experiment that is necessary to isolate the effect of an intervention.
In the case of ADN Dignidad, with limited resources and a large number of displaced migrants, MPCA was targeted to the most vulnerable households. The program collected quantitative measures of household vulnerability and converted these measures into indices. Applicants above a pre-established threshold were eligible to receive the program and applicants with lower vulnerability scores were not.
This eligibility rule, which was adopted for operational and equity reasons rather than for purposes of evaluation, provided exactly the kind of quasi-experimental variation needed to implement an impact evaluation using a method known as a “regression discontinuity design” or RDD. Working with the ADN Dignidad team, 3ie ascertained that the necessary conditions for an RDD were met and set out to collect a survey on over 3,100 eligible and ineligible households. The survey was collected in the second half of 2022, interviewing households between 1 to 3 months after they had graduated from the program and were no longer receiving MPCA. Importantly, ADN Dignidad’s high-quality registration and monitoring data provided the necessary information for re-contacting applicant households many months after they had applied for the program. We relied on computer-assisted telephone interviews coupled with a face-to-face data collection strategy that allowed us to reduce the cost of collecting endline data while minimizing the risk of selective sample attrition.
The analysis compares the outcomes of eligible and ineligible households that were close to the program’s eligibility threshold. Since the sample of eligible and ineligible households near the threshold were very similar before the program, ineligible households provide a credible estimate of the “counterfactual” outcomes for program participants in the vicinity of the cutoff. In other words, we were able to control for confounders (such as fluctuations in the labor market, or unobserved preferences of participants) that might influence the outcomes of interest, such as income and food insecurity. Therefore, this research design produces a credible estimate of program impact, at least for the sub-population close to the eligibility threshold. Since those participants were the “least vulnerable” of the eligible population, we might reasonably assume that any effects found in this part of the distribution might be “lower bound” estimates, meaning that impacts for the more vulnerable populations might well be greater or equal to those we are able to detect with this evaluation strategy.
Results: positive effects for cash transfer recipients
Results of the evaluation show significant positive impacts of ADN Dignidad. One to three months after graduating from the program, recipients maintain elevated levels of income and expenditures, translating into large reductions in food insecurity and the use of related coping mechanisms. Specifically, recipients experience a 15 percent increase in income and report per-capita expenditures approximately 10 percent higher than the comparison group. Food insecurity is lower among recipients, and coping strategies decline on several important fronts, including relying on child labor, borrowing or depending on donations from friends and family, and relying on fewer meals.
How are households able to sustain these benefits even several months after MPCA transfers have ended? Data on labor market participation, investments, and savings provide some clues. Participants have higher savings and lower debt, and they report working 3.7 additional hours per week on average. Participants are more likely to have invested in tools for their businesses, and female participants are more likely to own a business. Participants also rely less on remittances, and they remit more to other households. The improved outlook for migrant households reduces their perceptions of discrimination by 20 percent, reduces their perceptions of insecurity by 35 percent, and boosts their subjective life satisfaction by 7.5 percent.
Assuming that increased labor market participation remains stable over time, a simple calculation of return-on-investments suggests that the MPCA transfers produce a positive return on investment in under three years. Specifically, we project that participants will earn approximately $815 in additional income through increased working hours alone. Taken together, the initial evidence on impacts of ADN Dignidad suggest that MPCA is an effective approach to alleviating suffering and improving the wellbeing of displaced Venezuelan migrants, Colombian returnees and members of host communities. This evaluation also opens several important lines of inquiry and uncovers promising areas of future research in the humanitarian aid sector. We hope to address some of these questions, such as the cost-effectiveness and sustainability of impacts in the longer run, through ongoing and future research on the current evaluation sample. Other important operational questions could be explored in future incarnations of MPCA interventions in Colombia and elsewhere. For example, one could imagine experimenting with the duration of benefits, amounts of benefits, targeting rules, and eligibility criteria, to name a few.
In the meantime, evidence from the current evaluation is already paying off. The evaluation results supported the design of a new Early Recovery and Markets Systems component to promote the socioeconomic integration of the target population, aiming to increase their ability to obtain employment, start a business, and access financial services. This updated intervention offers yet another important opportunity for learning.
