Three years ago 3ie established the impact evaluation replication programme to highlight the benefits of internal replication studies of impact evaluations in the development sector. The programme soon became the subject of much debate that revolved around the scope, methods and timing of replication research. To get feedback from its stakeholders on the way ahead, 3ie recently hosted a consultation event in Washington DC that brought together both the proponents and the critics of replication. The event featured rich discussions and threw up some important ideas and questions on the future of replication research and the role 3ie could play.
The year 2014 was one of innovation and impact at 3ie. We are filling large knowledge gaps by funding policy-relevant research in new thematic areas. Our studies have informed development policies and improved programming. Read highlights about our work and achievements in our latest annual report.
Networks of researchers and policymakers promoting research evidence in policy are gradually gaining importance in many African countries. The first colloquium of the Africa Evidence Network brought together representatives from these networks, policymakers, practitioners and researchers to discuss the future of evidence-informed policy in the region.
The Executive Director of 3ie, Dr. Howard White will be leaving at the end of 2014. In this open letter, Dr. White reflects on 3ie's journey from being a small start-up with just one person to becoming a strong, successful organisation that has established itself as a leader in the field of evidence-based development.
Annette N. Brown and Anna Heard
One of the reasons we appreciate international days is that they prompt us to pause and reflect on what we’ve been doing in the past year, as well as think about what the next year will bring. On this International AIDS Day, our first reflection is realising how much we have grown our HIV/AIDS programming in 3ie in 2013.
At the opening session of 3ie’s recent Measuring Results conference, Jyotsna Puri, Deputy Executive Director and Head of Evaluation at 3ie, said, “It takes a village to do impact evaluation.” What she meant was that, for an impact evaluation to be successful and policy relevant, research teams need to be diverse and include a mix of disciplines, such as statisticians, anthropologists, economists, surveyors, enumerators and policy experts, as well as use the most appropriate mix of evaluation and research methods.
At 3ie’s recent Measuring Results conference in Delhi, one of the key speakers proposed a ban on new ideas in India. This speaker was not a Luddite, nor was she on a crusade against creativity, nor did she represent the tourism industry to claim that India had already achieved perfection in all dimensions. Rather, she was echoing a sentiment that has been bouncing around classrooms and corridors and cafes and corner-markets in which people discuss improving human welfare: we need to figure out how to implement the ideas we already have.
Shining a light on the unknown knowns: 3ie launches a new registry for prospective registration of impact evaluations
Annette N. Brown and Drew Cameron
Donald Rumsfeld, a former US Secretary of Defense, famously noted the distinction between known knowns (things we know we know), known unknowns (things we know we don’t know), and unknown unknowns (things we don’t know we don’t know). In international development research, these same distinctions exist. There is published evidence that can be used to inform programmes and policies; there are research questions that have yet to be tested and interventions that have yet to be evaluated; and there are, we certainly hope, ideas and innovations that have yet to be developed or discovered. But what about the fourth category? Is there such a thing as unknown knowns?
What works to improve education in developing countries
The low-quality of education in much of the developing world is no secret. The Annual status of education report (Aser), produced by the Indian NGO Pratham, has been documenting the poor state of affairs in that country for several years. The most recent report highlights the fact that more than half of grade five students can read only at grade two level. Similar statistics are available from around the world.
What if scientists directly tested their drug ideas on humans without first demonstrating their potential efficacy in labs? This question sounds hypothetical because we all know that using untested drugs can be potentially dangerous. If we were then to use the same logic, should we not be exercising similar caution with randomized controlled trials (RCTs) of social and economic development interventions involving human subjects?
Reflections from the CGD-3ie workshop on 17 July in Washington, DC
At a public forum on impact evaluation a couple of years ago, Arianna Legovini, head of the World Bank’s Development Impact Evaluation programme (DIME), declared that ‘dissemination is dead’. But her statement does not imply that we should stop the dissemination of impact evaluation findings for influencing policy. Quite the opposite: impact evaluation studies should be published in academic journals, be distilled into effectively formulated policy briefs, be disseminated at workshops and have social media fanfare. Legovini’s point was that this approach was not enough. For influencing policy, we need to move beyond dissemination and have strategies that include ongoing communication and active engagement with key actors. At a minimum, policymakers, programme managers and field staff, need to be involved in the evaluation from the design stage.
Thomas de Hoop
The constraints imposed by an intervention can often make designing an evaluation quite challenging. If a large-scale programme is rolled out nationally, for instance, it becomes very hard to find a credible comparison group. Many evaluators would shy away from evaluating programmes when it is hard to find a plausible counterfactual. Since it’s also harder to publish the findings of such evaluations, there don’t seem to be many incentives for evaluating such programmes. But the evaluation of a large-scale programme provides crucial information to policymakers. Doesn’t this make it important that evaluators take up the challenge of evaluating these types of programmes? If impact evaluators want to influence policy, they have to move beyond evaluating small programmes for which they have optimal control over the design of the intervention.
Déo Gracias-Houndolo and Jyotsna Puri
Economists use a variety of tools to understand impact caused by development programmes. Theories of change, qualitative analysis, quantitative techniques and advanced econometrics are all arrows in the quiver. But are these methods sufficient to ensure high quality impact evaluations?
I recently met with the acting chairperson of the Pakistan Earthquake Reconstruction and Rehabilitation Authority (ERRA). He described in proud detail ERRA’s successful reconstruction efforts after the disastrous 2005 earthquake that claimed more than 70,000 lives. In more muted tones, he described the 2010 floods, which claimed more lives than the 2005 earthquake. The rescue and relief effort was excellent in 2005. It failed miserably in 2010.
Impact evaluation might be seen as a prime example of what leading participatory proponent Robert Chambers has called extractive research. Researchers go out to communities, collect data, scurry back to their ivory towers to analyse and publish their findings. They build up their career and reputation, and then move onto the next study. In doing all of this, the research subjects have received nothing. They don’t even know the research ‘findings’.
Earlier this year I had a knee operation. So I turned to the Cochrane Library to find evidence on whether post-operative physiotherapy really works. What I found was a systematic review, which showed that post-operative physiotherapy has too little benefit to be worthwhile. Those assigned a course of physiotherapy had hardly any greater joint flexibility one year later than those who were not.
My last blog discussed the possible link between the dramatic reductions in crime in the UK and the adoption of evidence-based policing. But the Sunday Times recently ran a front page story which says that rather than solving crime, police are ignoring 850,000 cases a year which go uninvestigated.
We’ve all been there. You’ve just come up with an interesting international development research question, or your NGO or government agency is about to embark on a new project, and you need to find research evidence. You set off in search of the most rigorous kind available – impact evaluations (IEs). But where do you start? Perhaps your research library is expansive, and you have too many portals to choose from. Or maybe your organization is too small to afford annual database access. One thing is for sure, the search for relevant studies can be a time consuming, and frustrating endeavor.
While watching BBC news some time last year, I heard the new head of the Metropolitan Police, (London’s police force) make a very cogent statement of the need to base policy and programmes on evidence.
Where does economics fit on the spectrum of sciences? ‘Hard scientists’ argue that the subjectivity of economics research differentiates it from biology, chemistry, or other disciplines that require strict laboratory experimentation. Meanwhile, many economists try to separate their field from the ‘social sciences’ by lumping sociology, psychology, and the like into a quasi-mathematical abyss reserved for ‘touchy-feely’ subjects, unable to match the rigor required of economics research. However, the dismal science’s poor (and thin) replication record does little to lend credence to the claim that economics is more rigorous than the other ‘social sciences’.
Esther Coren and Birte Snilstveit
On 12th April 2013 street children, NGOs, celebrities, policymakers, businesses and individuals will observe the International Day for Street Children. The theme for this year is Home Street Home – highlighting that for many children across the world the street is their home.
Uganda’s cabinet has just approved a new monitoring and evaluation policy, which will be officially endorsed and disseminated next month. It comes as a positive signal after several donors suspended aid to the Government and provides a solid foundation to boost the country’s commitment to evidence informed policy-making.
Programme implementers often claim that it is unethical and not possible to randomise the assignment of their intervention. In his blog, 3ie’s Executive Director outlines how different designs of randomised controlled trials can overcome all these objections. Randomisation need not be a big deal.
A 3ie-supported study, highlighted as best practice, at the South Asian Evaluation Conclave of Evaluators, shows how active engagement between research teams and funding agencies improves the study design of an impact evaluation. Paromita Mukhopadhyay reports from the conference.
“Literature reviews are like sausages... I don’t eat sausages as I don’t know what goes into them.”
Dean Karlan said this to an unfortunate researcher at a conference recently. The ‘sausage problem’ puts in a nutshell why at 3ie we favour the scientific approach to evidence synthesis -- evidence as encapsulated by the systematic review.
“If anybody tells you to conduct an impact evaluation, tell that person to go to hell!”
This was the comment made by a renowned impact evaluator after Khalid Al Kudair presented the achievements of his NGO Glowork in mobilising the female labour force in Saudi Arabia and promoting substantial changes in the country's labour legislation. The impact evaluator was trying to drive home the point that if a programme is obviously working, why spend the time and money on conducting an impact evaluation? I disagree.
At 3ie, we stress the need for a good theory of change to underpin evaluation designs. Many 3ie-supported study teams illustrate the theory of change through some sort of flow chart linking inputs to outcomes. They lay out the assumptions behind their little arrows to a varying extent. But what they almost invariably fail to do is to collect data along the causal chain. Or, in the rare cases where they do have indicators across the causal chain, they don’t present them as such.
A pitch for an interdisciplinary approach to impact evaluation
Radhika Menon and Thomas de Hoop
A recent analysis of 67 3ie supported evaluations shows that distrust and suspicion among community members can sometimes slam the brakes on the implementation of development programmes. The distrust in almost all cases was sparked off by a ‘rumour’.
3ie is now making an important contribution to filling the evidence gap of what works in development and why. And it is not that evidence is being produced for evidence’s sake. Pre-schools are being rolled out across Mozambique, independent audits are being adopted in Gujarat and discussed with other states, registration requirements for small farmers in West Bengal have been abolished, and the NGO in northern Ghana has gone back to the drawing board. These are just some of the examples of how evidence from 3ie-funded studies is being used to inform better policies. And better policies can improve lives.
Female condoms reduce transmission of sexual diseases including HIV/AIDS. And they put the woman in charge in situations in which the man not be willing to use a condom. But adoption of female condoms remains low.
The chasm between policymakers and researchers is frequently observed but seldom addressed. A little over two weeks ago, 3ie organized a matchmaking market place to bridge this gap between research and policy. The setting for this was the Dhaka Colloquium for Systematic Reviews in International Development. The people who came together for this innovative matchmaking exercise were both ‘users’ and ‘doers’ of systematic reviews.
Using impact evaluation to improve policies and programmes: second generation impact evaluation questions
Conditional cash transfers increase school enrolments and use of health facilities. Community-level water supply does not have health benefits. There is emerging evidence that community-driven development programmes do not increase social cohesion.
These statements can be made with confidence based on the considerable body of evidence from impact evaluations undertaken to answer the question of what works in development. 3ie is now adding this body of evidence as more completed studies are becoming available.
When a group of UK policymakers were surveyed about what they thought about research evidence they said it was verbose, dense, full of jargon, untimely and irrelevant for policy. This is not surprising. Getting technical research evidence to seep into policy thinking has always been a challenge. The fact that the culture of opinion based policy still persists makes it difficult for researchers to reach out to policymakers.
What is the way forward then for systematic reviews? This was the big question that Philip Davies, Deputy Director-Systematic Reviews, 3ie, Kent Ranson from the Alliance for Health Policy and Systems Research, WHO and Howard White, Executive Director of 3ie tackled at the final plenary of the Dhaka Colloquium on Systematic Reviews in International Development. The plenary brought to a close a very successful week of deliberations on the use of systematic reviews in international development among 130 researchers and policymakers from 30 countries across the world.
As major funders of systematic reviews, the UK Department for International Development and 3ie had several insights to share at the Dhaka Colloquium of Systematic Reviews in International Development.
Overall DFID considers its systematic reviews programme to be a success and a valuable addition to its toolbox for evidence based policymaking. Raymond Kennedy, Systematic Review Lead at DFID said that as an evidence product, systematic reviews offer great value for money. “They offer a lot of synthesis for the money invested,” he said.
This thought provoking question was the highlight of the opening plenary of the Dhaka Colloquium of Systematic Reviews in International Development.
Systematic reviews summarise all the evidence on a particular intervention or programme and were first developed in the health sector. The health reviews have a specific audience: doctors, nurses and health practitioners. The audience is also easily able to find the systematic reviews.
Tackling HIV/AIDS: what works?
There has been only a small decline in the prevalence of HIV in the last decade, dropping from 5.9 percent to 5 percent between 2001 and 2009 for those aged 15-49 (UNAIDS, 2010). This decrease, whilst important, does not seem impressive compared to over US$5 billion spent fighting AIDS in low and middle income countries each year (the latest available figure is US$5.1 billion in 2008).
To get a sense of what has been happening on the ground with respect to its funded impact evaluations, 3ie recently carried out a field monitoring visit to four projects. Radhika Menon highlights some of the lessons learned through this visit, and how it gave 3ie the opportunity to learn a lot about the relationship between implementing agencies and researchers, and the challenges involved in implementing an impact evaluation.
A systematic review of daycare programmes in Latin America shows encouraging results both in terms of short and medium term development of children. Based on the evidence, Paola Gadsden, one of the researchers of the study, recommends that policymakers implement daycare interventions and continue to monitor and evaluate the programmes.
In a recent interview to 3ie, Velayuthan Sivagnanasothy, Secretary for Ministry of Traditional Industries and Small Enterprise Development, Government of Sri Lanka, highlighted how impact evaluations have changed the way Sri Lanka looks at development programmes.
Recently the Chris Evans breakfast show on UK’s Radio 2 picked up a news story on a Danish study reporting that half an hour’s exercise a day is better for you than one hour. Like me, the radio presenters were puzzled by this finding and wanted to know more.
A 3ie-supported study shows that, in water-abundant West Bengal, price-based reforms reduced costs for tube well owners but increased them for small farmers buying water. Recommendations from the study to reduce red tape and costs for small farmers to acquire electric pumps were adopted by government.
Early stimulation and micronutrients interventions: the next frontiers to break the cycle of child poverty
Why should we put more money into early childhood development interventions? Does this help children in secondary education? Should we invest in preschool programmes or more in home stimulation or parenting classes? What is most cost-effective? These are key questions that policymakers are grappling with at a time when early childhood development is emerging as a priority issue for many developing countries.
3ie was set up to fill ‘the evaluation gap’, the lack of evidence about ‘what works in development’. Our founding document stated that 3ie will be issues-led, not methods led, seeking the best available method to answer the evaluation question at hand. We have remained true to this vision in that we have already funded close to 100 studies in over 30 countries around the world. And we strongly promote mixed methods, in which the attribution analysis of ‘what works’ is embedded in a larger evaluation framework combining process and impact evaluation, factual and counterfactual analysis. This helps unpack the causal chain to understand why an intervention works or not, or only works for certain people in certain places.
Evidence-based policymaking is important but not always straightforward in practice. The complex reality of policymaking processes means that the availability of high quality research is a necessary, but not sufficient, ingredient for evidence informed policy.
At the recent 12th Annual Colloquium of the Campbell Collaboration, all the vibrant and in-depth discussions conveyed one clear message: we need to get better at bridging the gap between research and policy. Over 150 researchers and policymakers gathered in Copenhagen for the three-day event, which combined training in systematic review methods with plenary sessions focusing on policy and practice. The takeaway messages from the sessions offer a clear and sharp call to action for researchers.
The outcomes of evidence-based policy making are only as good as the evidence used. Impact evaluation methods, which are scientifically rigorous, provide highly useful evidence for policy making. But as with all scientific research, the only way to validate the findings of impact evaluation is to make sure that if we conduct the analysis over again, we yield the same results - that is, to replicate them. A well-known example of replication is the Roodman and Morduch replication of Pitt and Khandker on microfinance. But it is to be hoped and expected that most replications confirm the validity of study findings, which is very important for increasing the credibility of evidence for policy making.
Cash transfers are estimated to reach between 0.75 and 1 billion people (DFID, 2011). Evidence on the effectiveness of these transfers in increasing school enrolment and health care utilization in Latin America (Gaarder, 2010) has spurred the wide-spread growth of similar programmes across many developing countries.
16 year old Kou Yaokang’s family are poor subsistence farmers. They cannot afford to pay for Kou's high school education. Instead of ending his formal education after middle school, Kou Yaokang enrolled in a vocational school. This seemed like a good idea at the time. “The government was providing subsidies for vocational schools, and I thought I could learn new skills to get a good job,” Kou Yaokang told a team from the Rural Education Action Project in December 2011.
780 million people in developing countries lack access to clean water. That's approximately the same number of people that live in the United States and the European Union, combined. Waterborne illnesses such as diarrhea are the leading cause of death among children under five in the developing world. According to the World Health Organization, 1.4 million children die every year from diarrheal diseases attributable to access to poor water and sanitation, and poor hygiene conditions.
A review of evidence on what works in reducing maternal mortality
Every 90 seconds, a woman dies from complications of pregnancy or childbirth. 99 per cent of these deaths are in developing countries (UNFPA, 2011).
Maternal mortality fell by 34 percent in the nineteen years from 1990 to 2008: far short of what is required to achieve the Millennium Development Goal of a reduction of 75 percent by 2015. Some countries are doing better than others, but clearly there are substantial challenges in improving birthing conditions for poor women in sub-Saharan Africa and South Asia. In many countries, childbirth remains a death sentence rather than a blessing.
Abandoned by her family and severely malnourished, four-year-old Castera's story seemed destined for an unhappy ending.
But, this is where the plot changes. Taken in by Alda Mate, the strong, determined village chief of Machalucuane, and enrolled in Save the Children's supported early childhood education program, today Castera is a happy, curious second grader.
Our colleagues at the Overseas Development Institute have written a thought-provoking briefing paper on systematic reviews which offers some insights into the value of this form of analysis, but also finds fault with the approach.
The word ‘evaluation’ has several different meanings in African languages. In the Yoruba language, evaluation is often associated with ‘ayewo’ which means ‘investigation’. The meaning ties in with the cultural concept of evaluation. Many African societies have ‘evaluation’ rooted in their traditions in that they undertake all kinds of ‘investigations’ before they embark on a major project – farming, marriage, travel, assessment of causes and sources of illness.
How can one implement an ‘ideal’ impact evaluation, one that is technically robust, produces useful findings and influences policymakers? The authors of 3ie’s new working paper Behind the scenes: managing and conducting large scale impact evaluations in Colombia address this question by drawing on the experience of implementing evaluations of four major programmes in Colombia.
Nearly a quarter of the 543 elected members of the Indian Parliament have been charged with crimes, including rape and murder (Association for Democratic Reforms, 2009). Civil society movements have made corruption among politicians a topic of national importance using approaches such as voter education campaigns to empower citizens and demand more effective leadership.
Most development practitioners, academics, and decision makers would agree that usually social policy is not redesigned, upgraded and fine-tuned according to evidence. In fact, the past decade has witnessed a growing concern that very little evidence is available on what works in development policy, and that only a small part of this little knowledge is used in social policymaking.
Six million children die of hunger every year. Over 40 per cent of children under five in countries such as Bangladesh, Ethiopia, India, Nepal and Niger are stunted. Children that do survive are more likely to have heart disease, diabetes and renal damage.
Development practitioners and donors are gearing up to the Busan meeting on Aid Effectiveness later this month, which will focus on results. But will this renewed commitment translate into a more systematic use of evidence in policymaking?
Despite progress in recent years, over 30 percent of Bangladesh’s population lives below the poverty line, making it one of the poorest countries in the world (Poverty estimate based on cost of basic needs method, Government of Bangladesh and World Bank, 2011). Many evaluations assess short-term gains from development interventions but few studies have evaluated impact in the long run. Have development programmes in Bangladesh led to sustainable and lasting benefits for people?
Over 70,000 women die every year in India due to complications during childbirth. Several Indian states have developed programmes that encourage pregnant women to deliver their babies in medical facilities (institutional delivery) rather than at home. Poor women can use their Below Poverty Line (BPL) cards as vouchers to receive free maternity care at designated maternity hospitals. Contracted providers, usually an obstetrician or a small maternity facility, receives flat payments for each BPL delivery.
Women's labour force participation is growing - especially in Latin America and the Caribbean - with more than half the women around the world now working outside the home (UN, 2010). As more women take up jobs in developing countries, the demand for daycare services is increasing.
Evidence on the effectiveness of development programmes is now just a click away. 3ie is pleased to announce the launch of the systematic review database, a one-stop shop for synthesized evidence on development programmes. Funded by the UK Department for International Development, this database contains systematic reviews focusing on development interventions in low and middle income countries.
I was recently talking with one of my younger colleagues and she was lamenting something that was going wrong in an impact evaluation she was working on. She was thinking of throwing in the towel and shutting down the work. This reminded me of the horrible feeling in the pit of my stomach when I started doing impact evaluation (as well as research more generally) when something went wrong. Now, of course, I am bald…
Around one million people die each year from malaria and 90 percent of these deaths are in Africa. Malaria kills a child every 45 seconds, killing more children than any other disease.
While a variety of treatments have been used to tackle this deadly disease, the use of insecticide-treated nets, a corner-stone of the approach advocated by the roll-back malaria partnership, has come under close scrutiny following a recent study in Dielmo, Senegal by the Institut de Recherche pour le Developpement, published in the Lancet. This study has grabbed media attention and been the subject of articles in the Telegraph and the Independent, among others, and been covered by the BBC. The researchers found that after the introduction of bed nets, the incidence of malaria attacks fell from 5.5 percent in August, 2008 to 0.4 percent in August, 2010.But the incidence rose back up to 4.6 per cent between September and December, 2010.
Non governmental organisations (NGOs) need credible, reliable feedback on whether their interventions are making a meaningful difference. But they struggle with how they can practically access it. Like all credible research, impact evaluation takes time, resources, and expertise to do well, and many NGOs are not set up to rigorously evaluate the bulk of their work.
Since the 1990s, many multi-lateral and bi-lateral donors have expanded their peace-making and peace-keeping assistance to conflict-affected countries to include peace-building activities. The objective of these interventions is to prevent the conflict from reoccurring and return countries to a stable situation in which the economy can operate.
Evidence based development is treading in the footsteps of evidence-based medicine: innovating, testing, and systematically pulling together the results of different studies to see what works, where and why. Other disciplines as diverse as sports science and management have been going down the same route. Hard Facts, Dangerous Half-Truths, and Total Nonsense: profiting from evidence-based management by Jeffrey Pfeffer and Robert Sutton contains valuable insights for practitioners of evidence-based development.
“Policymakers act when they see an opportunity to tell a good story … They also act when they see an opportunity to avoid criticism. The good news is that the world is changing. Right now we have a growing movement for accountability with social media playing a lead role” commented Felipe Kast, Chile Planning Minister at the 3ie conference “Mind the Gap: From Evidence to Policy Impact.
The results agenda gained momentum in development circles during the 1990s, becoming firmly established with the widespread adoption of the Millennium Development Goals. This focus on results is welcome. Simply measuring success by the volume of spending, or even the number of teachers trained, kilometres of road built and women’s groups formed, is not a satisfactory approach. Input monitoring does not ensure that development spending makes a difference to people’s lives. Spending that makes a difference; that is what we mean by a result. So we would expect this agenda to go hand in hand with impact evaluations. But that has not been the case.
Chile sent a strong signal at the Mind the Gap conference by announcing its membership of the International initiative for Impact Evaluation (3ie). The Planning Ministry has a proven strong commitment to effective development spending with the establishment of the Compass Commission to pilot innovative programs based on impact evaluations.
Brazil, under the leadership of President Lula, has already made some headway in demanding evidence to stop spending tax payers’ money on programs that don’t work and committing to evaluation.
From June 15 to 17, the city of Cuernavaca in Mexico will be the center of discussions on how to use evidence to find effective solutions to critical development problems. Over 400 policymakers, practitioners and researchers from Latin America, as well as Africa and Asia will come together to take stock of what we have learned from evaluations and the way forward for adopting new approaches.
Is aid money used effectively to improve lives? Public opinion surveys in developed countries show that people are very sceptical about the benefits of aid.
We know that the level of education of mothers has a direct effect on children’s health, nutrition and education. Many development programmes are directed at educating mothers for improving their children’s health and education outcomes. The premise here is that a more educated mother will have better knowledge, have healthier behaviour and allocate more resources to her children’s health and education. But, is nutrition and health knowledge learned outside school more effective than literacy and numeracy skills learned in schools?
Every 40 seconds, a child in sub-Saharan Africa dies from a mosquito bite. Every year 781,000 people die of Malaria and most of them are children, infants and pregnant women. While the funding for malaria control has increased to around US $1.8 billion in 2010, it is still much lower than the required resources of $6 billion (WHO, 2010).
Lack of resources, conflict, poor governance, economic and humanitarian crises have impeded progress towards the health-related Millennium Development Goals. Undernutrition continues to be an underlying cause in about one third of all child deaths. Half a million women, most of them in developing countries, still die each year of complications during pregnancy or childbirth (WHO, 2010).
3ie Annual Report offers an insight into 3ie’s work in funding actionable research, informing policy and building capacity.
Poverty can manifest itself in many non-material ways. It could be expressed in feelings of powerlessness, lack of voice, exclusion, shame and break-down of the social fabric. This insight has meant that the role of social cohesion in improving people’s well being has received a lot of attention in recent years. Many governments, multilateral organisations such as the World Bank and other international agencies have encouraged community driven development and curriculum interventions to promote social cohesion.
Nearly one billion people, one-sixth of the world's population, go hungry every day. Several billion suffer from deficiencies in micronutrients like vitamin A or Iron. Hunger and poor nutrition can severely impact people’s health, particularly in the case of women and children.
Does microcredit do more harm than good? Do we know which humanitarian progammes work best in complex situations like Haiti? What are the lessons learned from conditional cash transfer programmes in Latin America and elsewhere? These are some of the key questions to be addressed at the evaluation conference "Mind the Gap: From Evidence to Policy Impact," in Cuernavaca, Mexico, on June 15 -17, 2011.
Do we really need more money to realize our anti-poverty goals? Or is the real need to find more cost effective solutions to development problems? Are the programmes we are carrying out the best ones to achieve our goals?
The pressing issues of farmer debt, climate change and food security reinforce the importance of agriculture as a global priority. This priority is clear from the high demand for impact evaluations of agriculture and farmer-related interventions. 3ie announces today the 16 awardees under the third round of its Open Window grants, half of which are in the area of agriculture and rural development. A total of US $ 6.1 million will be awarded for conducting these 16 rigorous and policy relevant studies.
The evidence-based revolution: Donors and researchers call for rigorous evaluations to ensure that aid works
Each year billions of dollars are spent on development programs with little evidence on whether those programs are having any impact at all on things that matter to poor people, such as poverty, child mortality, maternal health or girl’s education. But academics and policy makers believe that an ‘evidence-based revolution’ is now imminent in international aid policy.
Around 150 Chinese high level policy makers, government officials and researchers participated in a series of events in Beijing organized by 3ie together with the Center for Chinese Agricultural Policy (CCAP) of the Chinese Academy of Sciences (CAS), and Stanford University, who collaborate in 3ie Associate Member, REAP. Over five days, the team offered three separate, but interlocking events: a half day conference, a one and a half day orientation workshop and a three day hands-on training.
Seventy-two million children are missing out on education, according to the Education For All Global Monitoring Report, 2010. Millions of children leave school without having gained basic skills because they enter school late and drop out early. The question then is how does one encourage poor families to send their children to school?
3ie is seeking international impact evaluation expertise to evaluate development programs – ranging from credit guarantees to HIV/AIDS interventions - in Burundi, China, Fiji, Mexico, Mozambique, Sri Lanka, Tanzania and Uganda.
At the recent 3ie-DFID Impact Evaluation Workshop, grantees working in India teams from many institutions, including J-PAL, MIT, IFPRI, Delhi School of Economics, London School of Hygiene and Tropical Medicine, the International Water Management Institute and Pratham, presented the rationale, questions and design for impact studies in the diverse areas of governance, health, education, environment, water and agriculture.
Around 75 million children are still not enrolled in primary school, over a third of children drop out before completing primary school, and many more leave having ‘failed’ (UNESCO, 2009). As the global movement to meet the learning needs of children, youth and adults by 2015 gathers pace, it is time to reflect on whether education for all can also mean quality education for all.
More than five months after the earthquake, over 1.5 million Haitians are still living in makeshift camps and a heightened hurricane season gets under way. While relief efforts are still going on, medium- and long-term recovery strategies are being planned and mechanisms are being put in place to track recovery funds and monitor reconstruction programmes. The marathon of recovery and reconstruction has slowly begun in Haiti.
Every minute around the world, a woman dies in childbirth. Every year, two million babies die on their first day. A majority of these deaths could be prevented with early recognition and proper implementation of required skills and knowledge (Ray and Salihu, 2004). The Millennium Development Goals call for a reduction in maternal mortality by three quarters and child mortality by two thirds (MDG 4 and 5), but this can only be achieved if health care coverage of mothers and newborns is significantly scaled up.
Total funds for 3ie’s second round of Open Window grants goes up to US$ 14.6 million with additional DFID support
The UK Department for International Development (DFID) has just announced additional funding to 3ie for supporting eight high quality impact evaluations across India. This brings the total number of round two open window grants to 30.
Over two thirds of poor people in the world depend on small-scale agriculture. Lack of access to information about markets, vulnerability to the fluctuating demand for commodities such as coffee or cotton, and low margins have dented the profitability of small-scale farmers.
International Women’s Day: Do you know that ...
Quotas for women in politics cost practically nothing and help overcome bias against women leaders in India (Chattopadhyay and Duflo, 2004; Duflo and Topalova, 2004).
The UK Department For International Development (DFID) calls for systematic reviews to promote evidence based policy making
This innovative pilot project aims to increase the use of evidence in shaping and improving the effectiveness of development policy by commissioning over 40 priority systematic reviews in eight thematic areas: Growth and Investment, Governance, Fragile States, Climate and Environment, Social Development, Human Development, Agriculture and Aid Delivery.
The International Initiative for Impact evaluation (3ie) awards grants to 21 cutting edge impact studies which are set to have a significant impact on future policy-making.
3ie unveiled earlier this month the 21 winners for its second round of open window grants. The funds, totalling US$ 10 million, will be channelled towards assessing the impact of key development interventions in countries spanning Africa, Asia and Latin America. The awarded research teams are responding to the need of policy-makers and program managers for real evidence, by applying rigorous methodologies to investigate if development spending is indeed making a difference in people’s lives.
2009 laid a solid foundation in building the evidence base to help policy makers and development agencies invest more in effective programs. This first annual report explains how we are moving forward and constantly learning to do better.
While experts call for increased financing of climate change mitigation and adaptation by hundreds of billions of dollars a year, evidence of the effectiveness of current spending has become more and more essential. Donors will likely remain hesitant to provide additional funding unless it is clear that interventions are reaching their environmental and developmental objectives.
The World Bank says developing nations need US$400 billion per year for mitigation. The UN climate change summit is the deadline for countries to agree on a new global deal. Since funding and mandate for climate change mitigation and adaptation interventions is increasing substantially, there is an urgent need to ensure effective allocation of these resources and for policy makers to know which mitigation policy works, under what circumstances and at what cost.
Information is key in tackling development challenges. 3ie has now launched an online roster of leading professionals from around the world with experience in impact evaluation. This information will help development practitioners and policy makers in designing and conducting impact evaluations of their programs.
Many poor people in Africa and Asia still do not have good access to health services. In sub-Saharan Africa, one-in-six children still die from preventable diseases before their fifth birthday. Two thirds of these deaths could be avoided through low cost and easily available health care (WHO, 2008). If this continues the Millennium Development Goals for maternal and child health will not be achieved, putting millions of lives at risk.
How can we maximize results given the scarce resources allocated to development interventions? How do we find ways of improving performance to produce better results? These were some of the key questions addressed at the Indian Planning Commission in October.
The world is not on track to meet the international sanitation targets set by the Millennium Development Goals. While lack of safe water and sanitation is the world’s single largest cause of illness, 2.5 billion people are still without access to improved sanitation and 1.2 billion have no facilities (United Nations, 2008).
New 3ie Synthetic Review shows hand-washing, sanitation and point of use treatment interventions are most effective in improving health outcomes. However, it is difficult to influence sustainable behavior changes to ensure that water treatment at source translates into long-term health benefits.
Innovative research grants in Africa, Asia and Latin America helping improve lives of millions of people
3ie announced its first round grants of US$4.3 million for better evidence and development effectiveness
The International Initiative for Impact Evaluation (3ie) announced in June the results from its first round of funding for impact evidence. Eighteen high quality impact evaluations have been selected under the 3ie open window, with more than half of the funded projects being in Africa.
For the first time, an extraordinary gathering of 700 people from different continents, disciplines, sectors and methodological traditions shared their experiences and insights on how to evaluate development effectiveness. The Cairo international conference (March 29 to April 2, 2009) brought people from diverse perspectives together to draw lessons learned and engage with different approaches within the real-world of political, financial, and time limitations.
Achieving gender equality in education by 2005 was the first “missed Millennium Development Goal”. Continued failure to achieve that goal puts the other MDGs at risk as female education reduces fertility, child malnutrition and child mortality and promotes higher economic growth (Bruns et al., 2003; Herz, 2006; Klasen, 2002; Sen, 1999; Smith and Haddad, 1999).
The 2011 UNAIDS World AIDS report highlights a 21 per cent decline in the yearly number of new HIV infections and a 2.5 million total reduction in AIDS-related deaths since the peak of the epidemic in the mid-1990s. While these numbers signify success in the fight against this deadly epidemic, a lot more needs to be done to ‘get to zero’: zero new HIV infections, zero discrimination and zero AIDS-related deaths.