The 2015 Global Assembly organised by the International Development Evaluation Association (IDEAS) aims to gather evaluators from across the world to discuss possible solutions to the emerging evaluation challenges. IDEAS invites members and interested evaluation professionals to submit abstracts of evaluation presentations that fit into the overall theme of the conference. Furthermore, IDEAS envisages the following presentation streams that identify further which sustainable development issues: Education, Health, Equity and Gender, Climate Change, Impact, M&E Systems and capacity development. Deadline for abstracts: 15 May 2015.
Watch recording of webinar presentation on productive safety nets gap map by 3ie's Martina Vojtkova.
Harvard Kennedy School programme on Leading successful social programs: using evidence to assess effectiveness
The Harvard Kennedy School is accepting applications for a programme on Leading successful social programs: using evidence to assess effectiveness. This programme is designed for senior leaders within organisations who either manage social programmes or fund programmes designed to improve the well-being of people and communities. Application deadline: 21 April 2015
PEGNet invites papers on the theme: Transformation of developing and emerging economies: Challenges and opportunities for inclusive and sustainable growth for the tenth anniversary conference to be held 8-9 October 2015. Submission deadline 29 May
SciDev.Net's Tapping into Data podcast (1 March 2015) speaks to 3ie's Birte Sniltsveit on the evidence gap maps that have mapped large archives of academic information making them available and easy to navigate, in an effort to inform policymaking around issues such as payment for environmental studies.
A 3ie-funded impact evaluation of the Total Sanitation programme in Odisha, has been cited in an op-ed in the Mint newspaper (20 February).
The challenge for the Modi government is to evaluate what works in development programming before making large spending decisions, says 3ie's Radhika Menon and Howard White in an op-ed for The Hindu newspaper (19 February 2015).
3ie's evidence gap maps, a tool to identify what evidence is available in particular sectors, was launched on 5 February in London. The event was featured in SciDevNet (10 February).
A paper titled Better targeting of farmers as a channel for poverty reduction: a systematic review of Farmer Field Schools targeting (open access) by Daniel Phillips, Hugh Waddington and Howard White published in the Development Studies Journal is among the most downloaded papers in Routledge Social Sciences Journals in 2014.
Another paper titled Quality evidence for policymaking: I’ll believe it when I see the replication (open access) by Annette N. Brown, Drew B. Cameron and Benjamin D. K. Wood published in the Journal of Development Effectiveness is also among the most downloaded papers in Routledge Social Sciences Journals in 2014.
A study on Effectiveness of a rural sanitation programme on diarrhoea, soil-transmitted helminth infection, and child malnutrition in Odisha, India: a cluster-randomised trial co-funded by 3ie has a paper in The Lancet.
The latest issue of the Journal of Development Effectiveness is out. It includes a paper on Mitigating seasonal hunger with microfinance in Bangladesh: how does a flexible programme compare with the regular ones? by Shahidur R. Khandker, M. A. Baqui Khalily and Hussain A. Samad. And a paper on A checklist to avoid pilot failures: lessons from a set of women’s economic empowerment initiatives by Sara Johansson de Silva, Pierella Paci and Josefina Posadas.
Annette N. Brown, 3ie deputy director speaks to Through the Noise, an online platform that interviews thought leaders in different areas in an effort to understand the changing realities.
The future of aid: building knowledge collectively, a Center for Global Development Policy Paper by Ruth Levine and William Savedoff argues that aid agencies are particularly well suited to fund impact evaluations, and can accelerate progress in the developing world by increasing the resources available for evaluation, particularly through an organisation like 3ie.
3ie-funded systematic reviews on Effects of payment for environmental services (PES) on deforestation and poverty in low and middle income countries and Effects of decentralized forest management (DFM) on deforestation and poverty in low and middle income countries have been published in the Campbell Library.
3ie-supported study on Nourishing the future: targeting infants and their caregivers to reduce undernutrition in rural China featured in The Economist (8 January 2015).
The first edition of the ARCO Lab Summer School on Methodologies for Impact Evaluation will be held in Florence 17-19 June 2015. The school will offer a combination of teaching sessions, practical examples and case studies in the afternoon. Special reduced fee for 3ie members. Early bird reduced fee available for those who register before 15 February 2015.
Impact Evaluation Methodological Briefs co-authored by 3ie experts and published by UNICEF's Office of Research. The topics covered under these briefs include randomised controlled trials, quasi-experimental design and methods and theory of change.
A 3ie-supported study on the impact evaluation of the Cheeranjivi programme, a maternal health scheme launched by the Government of Gujarat in India cited in the World Bank Future Development blog (20 November).
3ie deputy executive director Jyotsna Puri was cited in a article titled The logframe dilemma in Devex (20 November 2014). This article is based on the logical framework — or “logframe” — approach to planning, monitoring and evaluation of development programmes is gaining traction among the international aid community. But does this system truly work for beneficiaries?
Ruth Levine, director of global development at Hewlett Foundation and member, 3ie board of commissioners, interviewed by SciDev.Net
"To stay relevant in a changing world, aid agencies must focus their resources on meeting low-income countries’ growing demand for data that can help them assess the impact of development activities," Ruth Levine, director of global development at Hewlett Foundation told SciDev.Net (29 October 2014) during the 3ie Evidence Week in London, 13-17 October.
Howard White was honoured with an award of distinction by the Government of Benin in London on 15 October. Aristide Djidjoho, general director of evaluation at the ministry of public policy analysis presented the award to Howard White on behalf of the president of Benin. The award was presented during the one-day colloquium of 3ie's London Evidence Week.
Djidjoho spoke of White’s significant contribution towards Benin’s development as well as his role in creating a sustainable environment for undertaking impact evaluations of development programmes. He went on to add that impact evaluations are not an end, but a means for achieving effective policies. He hopes that Benin and the whole of Africa will continue to benefit from White’s experience in the field of impact evaluations and evidence-informed decision making.
Djidjoho went on to thank Richard Manning, chair, 3ie board of commissioners and lauded 3ie’s efforts in building effective evaluation systems in several countries. He concluded by saying that through these sustained efforts in trying to strengthen public institutions, we can aim to achieve our common goals of development effectiveness.
Developing country policymakers are increasingly using impact studies to assess development programmes, Howard White, tells SciDev.Net (14 October 2014).
The Making Impact Evaluation Matter conference organised by 3ie in collaboration with the Asian Development Bank and Philippine Institute for Development Studies in Manila, 1-5 September, received wide media coverage.
These include articles in the Asian Journal (3 September), Philippine Daily Inquirer (3 September), InterAkyson.com (3 September), BusinessMirror (3 September), Devex, (3 September), Devex, (4 September), Devex, (4 September), Manila Bulletin (7,September).
Awards for the best presentation at the Making Impact Evaluation Matter conference are:
Best overall presentation:
1st Prashant Loyoka
2nd Abu Shonchoy
2nd Bekele Shiferaw
Best presentation by a young researcher
1st Abu Shonchoy
2nd Nigussie Abadi
Best presentation by a L&MIC researcher
1st Nigussie Abadi
2nd Altantsetseg Batchuluun
3rd Abu Hayat Md. Saiful Islam
3rd Evans Muchiri
1st MacCarthy Honu-Siabi
1st Desiree Manicom
2nd Drew Cameron, 3ie
3rd Martina Vojtkova, 3ie
The Making Impact Evaluation Matter conference in Manila organised by the Asian Development Bank, 3ie and Philippine Institute for Development Studies was featured on Devex (3 September 2014).
3ie-supported study on metering of electric tube wells in West Bengal was featured in a television programme
3ie-supported study on the metering of tube wells in rural West Bengal, titled Does marginal cost pricing of electricity affect the ground water pumping behaviour of farmers? Evidence from India was featured in a television programme Neb Kolkata (30 August 2014).
3ie-supported study on student evaluation in schools in the State of Haryana, India, titled A wide angle view of learning: evaluation of the CCE and LEP programmes in Haryana was featured in Business Standard (31 August 2014), Zee news (31 August 2014), Nagaland Post (31 August 2014), The Hindu (1 September 2014), Webindia123 (1 September 2014), India Today (31 August 2014), Veooz.com (31 August 2014), Samachar.com (31 August 2014), The News (31 August 2014).
A paper titled, Better targeting of farmers as a channel for poverty reduction: a systematic review of Farmer Field Schools targeting, by 3ie researchers Daniel Phillips, Hugh Waddington and Howard White, has been published in Development Studies Research.
The paper is available for download.
A 3ie-supported study evaluating the impact of the vocational education and training programme in rural China has been featured The Economist (23 August).
3ie is happy to announce the winners of the Impact Evaluation Repository challenge. In the last two months, we received 97 study submissions from more than 25 people.
Of the 97 studies submitted, 63 were eligible impact evaluations and were immediately added to the Impact Evaluation Repository. Of these, 32 studies** (and one correction) were published before 2013 and should have been found in our last search (thus are eligible for our $10 Amazon.com gift certificate prizes).
These studies are invaluable in our efforts to improve our systematic search and screening protocol and provide the international development community with the most up-to-date, published impact evaluation evidence available.
We would like to sincerely thank the following people for sending their approved impact evaluation studies to the Impact Evaluation Repository. As always, we encourage you to send any published impact evaluations to email@example.com.
William Savedoff of the Center for Global Development blogs on 3ie's journey from being a start-up to maturity. The appointment of Emmanuel (Manny) Jimenez as the new executive director marks a transition in the organisation.
"With the transition to a new executive director, 3ie will be able to build on past success while drawing on the different qualities and depth of experience and expertise that Manny Jimenez will bring to the post," says Savedoff.
The announcement of the Emmanuel Jimenez as the next 3ie executive director was featured on Hindustan Times (23 July 2014), the Indian national daily.
The Chair of 3ie, Richard Manning, has announced that the 3ie Board of Commissioners has appointed Emmanuel Jimenez to succeed Howard White as the executive director of 3ie in early 2015. Dr Jimenez is currently director, public sector evaluations in the Independent Evaluation Group of the World Bank.
3ie evaluation specialists participated in the Journées Béninoises de l’Evaluation in held in Cotonou, Benin, from 30 June to 2 July. There were 300 participants from Argentina, Mexico, Canada, France, South Africa, Uganda, Togo, Niger and Benin. Representatives from institutions like the West African Development Bank, the African Development Bank, GIZ, CLEAR, UNDP, UNICEF, etc. were also present.
The event received wide media coverage.
This paper by Annette N. Brown, Eric W. Djimeu and Drew B. Cameron reviews the literature on a set of self-tests that share some characteristics with HIV self-tests to determine whether there is any evidence of harm.
The International Rescue Committee requests feed back on 10 gap maps that they have constructed based on systematic reviews. These gap maps are visual representation of existing evidence on topics like agricultural interventions, access to quality education, cash transfers, social and economic empowerment of women.
A 3ie-supported study on Paying for performance in China’s battle against anaemia was featured in The Economist (14 June 2014).
3ie regrets to announce that the planned thematic window on demining has been cancelled. We do not plan to launch a window on demining in the near future.
Nature (30 April 2014) has cited the 3ie systematic review summary on farmer field schools in an article on agricultural innovations to increase yields and protect the planet.
A three-day evidence symposium on what works to improve labour productivity was organised by the International Labour Organization in partnership with J-PAL, Silatech and the Arab Urban Development Institute at the Georgetown University, Doha, 6-8 March 2014.The event brought together policymakers, development practitioners, academics and researchers to present and discuss the evidence base for what works in increasing the employment and productivity of youth. Discussants shared recent findings of impact evaluations from the Middle East and North Africa and other developing regions. Participants were also introduced to evidence from systematic and literature reviews which are important for evidence-based policies.
This paper by Thomas de Hoop, Luuk van Kempen, Rik Linssen & Anouka van Eerdewijk in the Feminist Economist looks at the relationship between participation in women's self-help groups, women's autonomy and subjective well-being in Odisha, India.
The findings suggest that while self-help group membership results in positive effects on women's autonomy, there is also evidence of negative effects on women's subjective well-being in villages with relatively conservative gender norms. Qualitative research suggests that this negative effect is associated with social sanctions after the transgression of gender norms.
Howard White's presentation on Evidence-based policies to reduce poverty at the Independent Evaluation Group, Washington, DC, was featured in Devex (21 April, 2014). The article talks about the importance of systematic reviews in measuring the effectiveness of development programmes.
3ie is part of the Global Open Knowledge Hub, an open-access digital data initiative to help increase online accessibility of development research coming from the global South and boost their decision-making influence.
An article in Sci Dev Net on the Global Open Knowledge Hub mention's 3ie's participation in this initiative.
A 3ie-funded impact evaluation of the Cheeranjivi Yojana, a health insurance programme in the state of Gujarat, India, undertaken by Duke University, was cited in an article titled Gujarat experiments with public health insurance in India Ink, New York Times, 3 April 2014.
Agricultural land tenure reforms have been less effective in Africa than Latin America or Asia, says Steven Lawry and Cyrus Samii in a Guardian op-ed. This is based on a systematic review, The impact of land property rights interventions on investment and agricultural productivity in developing countries. 3ie provided the quality assurance support for this review.
A three-day international workshop on Evaluating Forest Conservation Initiatives, held in Barcelona in December 2013. The workshop brought together about 40 researchers, practitioners and policymakers to discuss the complexities involved in evaluating forest conservation initiatives. Philip Davies, 3ie Deputy Director - Systematic Reviews was as the workshop. Davies has been cited in this blog by the Center for International Forestry Research.
Two papers from the Journal of Development Effectiveness are among the top 25 most read Development Studies papers of 2013 from among a variety of Routledge Development Studies journals. These include, An introduction to the use of randomised control trials to evaluate development interventions by Howard White and How to do a good systematic review of effects in international development: a tool kit by Hugh Waddington, Howard White, Birte Snilstveit, Jorge Garcia Hombrados, Martina Vojtkova, Philip Davies, Ami Bhavsar, John Eyers, Tracey Perez Koehlmoos, Mark Petticrew, Jeffrey C. Valentine, Peter Tugwell.
"To err is human, but to use the word 'error' in a replication study is usually not divine." Guest post on the Development Impact blog by Annette Brown, Deputy Director, Advancement and Impact Evaluation Services, 3ie, and Benjamin Wood, Evaluation Specialist for Replication, 3ie.
Bill Saveoff of Center from Global Development applauds the achievements of 3ie under the leadership of Howard White. He also says it's time to revisit the original visions for 3ie – that all foreign aid and multilateral agencies should contribute 0.01% of their annual disbursements to 3ie in support of impact evaluation.
3ie-supported impact evaluation of a maternal health programme in Gujarat, India, shows that the much-touted Chiranjeevi Yojana, launched in 2006 to reduce maternal and infant mortality in the state, hasn’t had any significant impact on institutional delivery rates or maternal health outcomes.
An article by Prof Michael Greenstone of the Massachusetts Institute of Technology in the New York Times cites a 3ie-supported study on reforming the environmental audit system in industrial units in Gujarat.
Howard White was interviewed on the Story of Aid in the Rear Vision programme on ABC Radio.
Howard White writes about the lessons 3ie has learned through the experience of conducting and managing impact evaluations in the past decade in the Impact magazine published by Population Services International.
An article in The Hindu newspaper (dated 31/10/2013) on the 3ie-ASCI conference on Measuring Results.
Howard White, 3ie Executive Director, was part of a discussion on evaluation of social sector programmes on Rajya Sabha TV with Ajay Chhibber, Director General, Independent Evaluation Office, Government of India; Thoriq Ali Luthfee, Minister of Health, Maldives and Biraj Patnaik, Principal Advisor, Right to Food in the office of Commissioners to Supreme Court. Anchor: Girish Nikam
Two 3ie-supported studies cited in the Economist. These include Girl power: cash transfers and adolescent welfare. Evidence from a cluster-randomized experiment in Malawi, by Sarah J. Baird, Ephraim Chirwa, Jacobus de Hoop, Berk Özler and Relative effectiveness of conditional and unconditional cash transfers for schooling outcomes in developing countries: a systematic review, by Sarah Baird, Francisco H. G. Ferreira, Berk Özler, Michael Woolcock.
Girl power: cash transfers and adolescent welfare. evidence from a cluster-randomized experiment in Malawi
This 3ie-supported study now has a paper in the NBER working paper series.
This study summarises evidence from short-term impacts of a cash transfer programme on the empowerment of adolescent girls in Malawi during and immediately after the two-year intervention.
Getting children into school is only part of the education battle. We must also ensure they learn once they are there. Blog by Howard White in the Guardian
The University of East Anglia is offering a one-year MSc course in Impact Evaluation in International Development.
The course has been designed for students who are interested in designing and implementing development projects and programmes and/or in researching development effectiveness, and who need to develop and enhance their skills for undertaking high-quality rigorous impact evaluations.
3ie is offering a scholarship for this course.
This 3ie-supported study now has a paper in the NBER Working Paper Series.
This study evaluates the impact of a programme that provides pre-fabricated housing to members of extremely poor population groups in Latin America. Findings show that a positive effect on general well-being of the people. There is also evidence of improvement in children's health in two countries.
A 3ie study on the impact of daycare in Brazil shows that it has the potential to improve household welfare especially for the poor. Household income was went up (8%), as did the labour supply of the carer (usually the mother). Children also fared better in terms of cognitive and anthropometric outcomes.
This paper by Marie Gaarder and Jeannie Annan argues that it is both possible and important to carry out impact evaluations even in settings of violent conflict, and it presents some examples from a collection of impact evaluations of conflict prevention and peacebuilding interventions. This Policy Research Working Paper is published by the Independent Evaluation Group, World Bank.
The Campbell Collaboration training videos were recorded at the May 2013 Campbell Colloquium. The introductory videos are intended for researchers conducting systematic reviews as well as for policymakers interested in evidence-based policies. The advanced methods / applied topics videos are for researchers carrying out systematic reviews.
Do conditions moderate the effects of cash transfer programs? Preliminary findings from a systematic review
The preliminary findings from the systematic review indicates that both conditional cash transfer (CCT) and unconditional cash transfer (UCT) programmes improve school enrolment: 23% in the case of UCT programmes and 41% for CCT programmes.
An impact evaluation of the loans offered by Compartamos, the largest microlender in Mexico shows that loan recipients grew their business revenues and expenses, were happier, more trusting, had greater household decision power, and were better able to manage liquidity and risk. However, there was little evidence that loans had an impact on building wealth like household income, business profits, or consumption.
The study adds to the evidence that microcredit is generally beneficial, but not necessarily transformative.
The Research Department of Agence Française de Développement (AFD) has published the Ex Post collection of the following papers: Insuring health or insuring wealth? An experimental evaluation of health insurance in rural Cambodia and Sky impact evaluation, Cambodia, 2010, village monographs
Ian Ramage and David Levine, co-authors of these studies will be speaking at the Health Financing in Developing Countries Conference (Financement de la santé dans les pays en développement) on 27 and 28 May at AFD.
This report gathers together examples which attempt to explain how and why change happens as a result of cash transfers (CTs). It first presents a selection of theories of change about how cash transfers are expected to work in general, drawn from academic literature, and then a selection of theories as used in a few specific cash transfer programmes.
Pocast of presentation by Jyotsna Puri at the 28th Active Learning Network for Accountability and Performance in Humanitarian Action (ALNAP) meeting.
In a video interview, Phil Davies talks about the importance of timing for researchers wanting to engage and inform policymakers. He talks about 3ie GapMap as a visual and engaging tool for understanding what is known and what isn't.
In Part 2 of the video, he stresses the importance of physical access, encouraging efforts towards ensuring that policymakers are aware of, and can get hold of, the kinds of evidence that they need to make good decisions.
A DFID guidance note on the best ways to assess evidence in international development. It offers some rules on:
- understanding different types of empirical research evidence
- appreciating the principles of high quality evidence
- considering how the context of research findings affects the way that staff may use them
- understanding how to make sense of inconsistent or conflicting evidence
Findings from a 3ie-funded study on the impact of a conditional cash transfer programme in Nicaragua cited in this Inter-American Development Bank (IADB) blog.
Theme: What evidence-based development has to learn from evidence-based medicine
Speaker: Chris Whitty
Theme: What we have learned from 3ie's experience in evidence- based development
Speaker: Howard White
Howard White discusses different designs of randomized control trials and addresses criticisms of RCTs which are mostly argued to rest on misunderstandings of the approach.
The standard approach to policymaking and advice in economics implicitly or explicitly ignores politics and political economy, and maintains that if possible, any market failure should be rapidly removed. This NBER working paper by Daren Acemoglu and James A. Robinson argues why this conclusion may be incorrect.
This CGD Working Paper by Tessa Bold, Mwangi Kimenyi, Germano Mwabu, Alice Ng'ang'a, and Justin Sandefur looks at how a Kenyan education programme proven successful in a randomised controlled trial, failed to have similar outcomes when implemented.
See blog by Justin Sandefur Finding what works in development: what is the what?)
Esther Duflo discusses a 3ie-supported study on Gujarat pollution control in an interview to Yale Insights.
Markus Goldstein highlights the findings from two 3ie-supported studies -- a systematic review on the impact of daycare programmes and the impact evaluation of Save the Children's early childhood development programme in Mozambique.
"The purpose of this document is to advance the Foundation’s existing work so that our evaluation practices become more consistent across the organization.We hope to create more common understanding of our philosophy, purpose,
and expectations regarding evaluation as well as clarify staff roles and available support."
Howard White, 3ie Executive Director, discusses “closing the evidence gap”, randomised control trials, and the value of impact evaluation with Dereck Rooken-Smith of ODE, AusAID.
Louise Shaxson, Research Fellow, ODI on how "Pressure to demonstrate concrete impacts on public policy is encouraging researchers to make grand claims about what we/they are likely to achieve."
Rosalind Eyben, Research Fellow at IDS and Chris Roche, Associate Professor, La Trobe University kick off a discussion on the implications of evidence-based approaches.
Nina Cromeyer Dieke shares tips and lessons development community can learn from mainstream media. The 3ie policy impact toolkit finds a mention in the story.
Berk Ozler on the pros and cons of using surveys to measure impact.
"...So, evaluating a large government program using an unrelated routine government survey may be fine (although I suspect that they too will have biases depending on what the respondents think the survey is for, how large, important, and ‘in the news’ the intervention is, etc.), but evaluating your own experiment that aims to change some behavior by asking study participants whether they have changed that behavior is unacceptable."
How can systematic reviews contribute evidence for policy? Blogs on this page take up the debate on conducting and using systematic reviews.
Tracey Koehlmoos, adjunct professor at George Mason University, Washington DC, and adjunct scientist at ICDDR,B blogs on sessions at the Dhaka Colloquium on Systematic Reviews.
"...Perhaps the most controversial session that I have attended so far was provocatively named “Rapid reviews: opportunity or oxymoron?” 3ie deputy director, Phil Davies presented on “rapid evidence assessment” and their place in the pantheon of evidence synthesis efforts aimed at informing policy making. Serious questions remain about rapid reviews being biased compared to systematic reviews—and how the process would even allow the developers of these reviews to recognize any biases. However, Davies pointed out that “all evidence is probabilistic.” ..."
"This discussion paper aims to support appropriate and effective use of impact evaluations in AusAID by providing AusAID staff with information on impact evaluation. It provides staff who commission impact evaluations with a definition, guidance and minimum standards."
3ie recently participated at an IFAD learning event on impact evaluations for environmental and climate change interventions.
William Savedoff blogs on the challenge that impact evaluation poses for organisations. "...Other times, the concerns reflect an unwillingness to clearly state their goals, be explicit about their theories of change, or put their beliefs about what works to an objective test. Yet, this is exactly what is at stake with evaluation: are you willing to be proven wrong?"
"Potential biases can arise when collecting qualitative data, in deciding which questions are asked, in what order, how they are asked and how the replies of the respondents are recorded.
"There can also be biases in how the responses are interpreted and analyzed, and finally which results are chosen for presentation. Of course quantitative data and analysis is also prone to bias, such as sampling bias and selection bias. But methodologies have been developed to explicitly deal with these biases. Indeed evaluation designs are judged on precisely how well they deal with these biases."