Next 1 2 3 4 Previous
Posted 2010-02-17
Using mixed methods for impact evaluation
Mixed methods are the explicit adoption of both quantitative and qualitative methods in the impact evaluation design (
White, 2008).
The debate around ‘mixed methods’, also known as ‘Q-squared in Policy’ has stressed the complementarities of the two approaches. That is the depth from qualitative research and the statistical robustness from quantitative research often inform, correct and augment each other (
Shaffer et al, 2009). Important aspects of this complementarity are that mixed methods for impact evaluation ensure that the research process is used as a way of increasing accountability and empowerment of the poor (
Garbarino and Holland, 2009), and that participatory research offers qualitative insights and generates statistics for relevant dimensions that might otherwise be overlooked (
Chambers, 2007).
Posted 2010-01-10
Updated wikipedia on impact evaluation. To view.
New glossary on impact evaluation now available. 3ie has developed a glossary of key terms in impact evaluation.
Posted 2009-12-18
Evaluating the US Coalition for Evidence-Based Policy
Recent congressional initiatives seek to focus federal funds on interventions that have been found effective in randomized experiments. A new report from the US Government Accountability Office examine the process used by the nonprofit Coalition for Evidence-Based Policy to identify social interventions meeting such standard, and concludes that requiring evidence from randomized studies as sole proof of effectiveness will likely exclude many potentially effective practices.
Posted 2009-12-07
Evaluating HIV/AIDS prevention programs
A meta-analysis showed HIV voluntary counselling and testing (VCT) recipients were significantly less likely to engage in unprotected sex when compared to behaviours before receiving VCT, or as compared to participants who had not received VCT (Denison JA et al, 2008).
However in Malawi, a program that offered door-to-door HIV testing was found to have no impact on preventive behaviour, as measured by condom purchases. Door-to-door voluntary counselling and testing (VCT) campaigns are expensive and not usually targeted at key populations at higher risk, thus lowering the effectiveness of such program. These costs must be weighed against small apparent benefits. Only some participants showed more preventative behaviour after learning their status, and even among them the effect was very small (Thornton R.L., 2008).
An earlier study showed that after two years, girls in schools in Kenya where teachers had been trained in the standard, national HIV/AIDS curriculum were no more likely to prevent pregnancies than girls in comparison schools. Reducing the cost of education by paying for school uniforms reduced dropout rates, teen marriage, and childbearing, and informing girls that men in their 20s, or older, are more likely to be HIV positive than young men led to a dramatic 65 percent drop in childrearing with older men, with no increase in pregnancies with younger men (Duflo et al, 2006, Dupas, 2009).
Posted 2009-11-17
Evaluating the impact of agriculture programs
Over 1 billion people go hungry in the world today. Learning from successes in agricultural development is now more urgent than ever. As heads of state gather in Rome for the World Summit on Food Security, a new book pulls together – for the first time – major successes in agricultural development that have brought millions of people out of hunger over the past 50 years.
"Millions Fed: Proven Successes in Agricultural Development", produced by the International Food Policy Research Institute (IFPRI) and commissioned by the Bill & Melinda Gates Foundation, determines what works in agriculture – what sorts of programs, policies, and investments have had a proven impact on hunger and food security.
It presents 20 case studies of large-scale initiatives that are documented to have substantial, long-term effects. These stories include achievements not just in increasing the yields and production of staple food crops, but also in expanding markets, developing better policies, conserving natural resources, and improving nutrition.
Posted 2009-10-26
Evaluating the impact of Conditional Cash Transfer on children’s poverty
A recent study on Oportunidades, Mexico’s large-scale conditional cash transfer program for poor families (Ozer et al., 2009), found that though the program did not directly address children’s behavior problems, it had a ripple effect in reducing children’s aggressive and oppositional behaviors.
The program was also found to have a significant impact in lowering the level of stress amongst children of mothers with high depressive symptoms (Fernald and Gunnar, 2009).
Earlier, Oportunidades was associated with lower prevalence of obesity and hypertension in rural Mexico (Fernald et al., 2008) suggesting that the educational component of the program was counterbalancing the income effect.
However in Malawi, a one-year schooling impacts from a conditional cash transfer experiment among teenage girls and young women (Baird et al., 2009) showed that a total transfer offer of $5 per household per month induces the average girl to be 10 percentage points more likely to be in school after one year and concluded that such insignificant increase in schooling rates achieved by doubling the total transfer to the household to $10 does not seem cost-effective.
Posted 2009-10-02
Evaluating fair trade interventions
For Kenyan farmers, the impact of fair trade has been positive in terms of higher income satisfaction, higher food consumption and value added effects such as access to services and training.
But a recent impact assessment of fair trade programs for bananas and coffee in Peru, Costa Rica, and Ghana, found that fair trade producers will benefit more from the ability to attract long-term delivery contracts, which give them the assurance and stability of selling their products in large-scale market outlets, than only from the price advantage of selling fair trade products.
Evidence showed that fair trade in Bolivia also had a positive influence on conflict prevention by reversing horizontal inequalities biased against indigenous people.
Posted 2009-09-07
Improving humanitarian impact assessment: bridging theory and practice by Karen Proudlock, Ben Ramalingam and Peta Sandison. The latest ALNAP review of humanitarian action summarises current debates on impact assessment in humanitarian aid, and identifies a series of challenges and approaches to impact assessments.
Focus of humanitarian evaluations is shifting towards analysis of the impact of humanitarian assistance – to understand, in an evidence-based way, how aid ultimately affects the lives and livelihoods of aid recipients. Improved impact assessment could contribute to greatly improved beneficiary participation, more robust needs assessments, and more evidence of what works.
Posted 2009-08-26
Evaluating anti-corruption interventions
A randomized control trial in Indonesia finds introducing universal auditing at village level reduced "missing expenditures" by 8 percent. By contrast, increasing grass-roots participation in the monitoring process only reduced missing wages and had no effect on missing materials expenditures. An earlier study also cautions relying on community based approaches.
Two studies in Madagascar show that an independent media can be an important tool in the fight against corruption. There is evidence that local presence of mass media significantly decreased the capture of cash as well as in-kind interventions. However, the impact of media is conditional on the level of literacy and reach of print and broadcast media in remote areas.
Posted 2009-08-10
Evaluating Nutritional Programs
Two studies of the Nutrition Enhancement Program in Senegal – one using a double difference, and the other propensity score matching - find a significant impact on nutritional outcomes. The program was meant to use randomized placement, and a simple comparison of treatment and control found no significant impact. But the matching methods show differences in the characteristics of the treatment and control, demonstrating that selection was at work, and so teaching the lesson that the integrity of randomization cannot be taken for granted.
A new paper from the World Bank outlines the methodological challenges in evaluating nutrition programs.
.jpg)
.jpg){kind=small}
.jpg){kind=small}
