Capacity strengthening

Overview

Achieving the Sustainable Development Goals across low- and middle-income countries (L&MICs) requires the capacity to implement—and evaluate—development policies across in the regions where they're most needed. 3ie is a global organization working with a range of partners in L&MICs, and our recent evaluation capacity strengthening activities span Africa, South America, South Asia and South-east Asia. Recent trainings have reached the African Population and Health Research Centre (APHRC), Makerere University, several UN agencies in Sudan, and the Departments of National Planning in Colombia, India and the Philippines.

Over the course of the pandemic, 3ie has developed and provided virtual trainings on a range of topics. A typical curriculum would include modules on a range of impact evaluation methodologies, mixed-method research designs, research transparency, the development of an appropriate theory of change, the use of big data and machine learning for causal inference, and cost analyses. For higher-level audiences, 3ie also has a module on how to decide on the proper impact evaluation approach – and when not to do one. We recently launched a new module on how to integrate process evaluations into impact evaluations to learn the nuances of the mechanisms which underlie the quantitative findings.

3ie’s evidence synthesis trainings teach when and how to create evidence gap maps (EGMs) and conduct systematic reviews. We also deliver modules on evidence uptake and how to engage with stakeholders to understand the context within which an evaluation is carried out. 3ie’s evidence uptake and use training on contribution tracing enables stakeholders to learn how to rigorously evaluate whether and in what way their evaluation made an impact.

Evaluation capacity building in Africa

3ie’s experiences with the West Africa Capacity-building and Impact Evaluation (WACIE) program and our collaboration with the UN Peacebuilding Fund in Sudan have provided some useful lessons on the best ways of building evaluation capacity in contexts where it is lacking. We have found that a holistic approach that fits impact evaluation training into people's existing work is most effective, especially when combined with active follow-up. Building capacity is most effective when integrated into real-world work that policymakers, project managers, and evaluation professionals are already engaged in – i.e., when we show people impact evaluation tools, methods, and findings relevant to their work.

The WACIE program is a multi-year regional initiative that aims to promote the institutionalization of evaluation in government systems across eight countries of the West African Economic and Monetary Union: Benin, Burkina Faso, Côte d’Ivoire, Guinea-Bissau, Mali, Niger, Senegal and Togo. More than 500 actors in the evaluation ecosystem have benefited from capacity-building in impact evaluation through this program.

3ie’s partnership with the CLEAR Center for Francophone Africa is a particularly successful example of our collaborative approach, which enabled the implementation of the Regional Program for Training and Support to Impact Assessment (Programme Régional de Formation et d’Appui à l’Évaluation d’Impact, PREFA-EI). The initiative supported 20 members of public administrations—of countries with impact evaluation projects in the agriculture and rural development sector—who were coached in the development of impact evaluation protocols in four projects.

Recent examples of our capacity-strengthening work

Workshops with Development Monitoring and Evaluation Office (DMEO), NITI Aayog, India

Description: 3ie conducted workshops for 60 participants from across departments of DMEO and Niti Aayog about the challenges involved in choice of methods and designs for impact evaluation of policies and programs in August 2021. These sessions were followed by trainings and a research seminar covering a range of topics, including the need to differentiate between monitoring and evaluation, steps to develop an M&E framework and the importance of a counterfactual and theory of change.

Impact Evaluation Methodology Training with African Population and Health Research Centre (APHRC)

Description: 3ie developed an impact evaluation methodology training program to deliver to a group of 11 researchers at the APHRC. This work was part a series of trainings funded by the Hewlett Foundation that the staff were taking to strengthen their capabilities in monitoring and evaluation. The three modules broadly covered: introduction to impact evaluation, experimental designs, and quasi-experimental designs. Each module incorporated case studies featuring 3ie studies, Stata exercises, and interactive presentations. These trainings were delivered between November 2020 and January 2021.

Due to the success of the trainings, APHRC reached out to 3ie to deliver additional trainings. In August 2021, 3ie delivered additional trainings in economic evaluation and the use of remote sensing in impact evaluation. The latter component fit in with 3ie's participation in the Geo4Dev initiative. Read more about it here.

Impact Evaluation Methodology Training with Makerere University

Description: Along with CEGA at UC Berkeley, 3ie developed an impact evaluation methodology training program for a group of 40 participants. The participants included faculty and staff from Makerere University as well as policymakers with little previous exposure to M&E from various Ugandan government agencies, including the Office of the Prime Minister, Ministry of Health, Ministry of Agriculture, Animal Industry and Fisheries, and the Ministry of Education and Sports.

Impact Evaluation Methodology Curriculum Preparation with CLEAR-AA/University of Zambia

Description: 3ie developed five modules for University of Zambia faculty to use in their impact evaluation courses. We also incorporated a “training of the trainers” where 3ie staff walked through each module with the faculty to underscore the key learning messages from each section. The modules covered topics such as theory of change, managing an impact evaluation, analyzing the results, and reporting results from an impact evaluation. This work was funded by CLEAR AA.

Impact Evaluation Methodology Training with Education Endowment Fund/eBASE

Description: 3ie delivered impact evaluation methodology trainings to the eBASE team in Cameroon, with funding from the Education Endowment Fund. These trainings were delivered virtually and were accompanied by an in-person workshop in May 2021 that guided participants through creating a theory of change and developing survey instruments.

Econometrics and Big Data/Machine Learning Training with Department of National Planning, Colombia

Description: 3ie worked with the Department of National Planning (DNP) of Colombia to train over 30 DNP staff in impact evaluation methodology, research transparency, and the use of big data/machine learning for causal inference. These trainings were delivered in July 2021 and funded by the Global Evaluation Initiative.

Impact Evaluation Methodology Training with Peacebuilding Fund partners

Description: As part of 3ie's participation in the PeaceFIELD initiative, 3ie collaborated with several UN agencies to train in-country implementing partners and local UN staff in impact evaluation methodology. The October 2021 virtual training for partners on the UN Peacebuilding Fund's Darfur projects introduced impact evaluations, presented a refined approach to using a theory of change, taught experimental and quasi-experimental methods for impact evaluation, and provided practical guidance on impact evaluation operations and planning. As a result of this training, UN partner M&E staff updated their existing monitoring tools to provide the more detailed data necessary for impact evaluation design. The training also informed and deepened 3ie’s ongoing collaboration with the International Organization for Migration (IOM) on designing the endline survey for the impact evaluation.

Humanitarian Impact Evaluation Methodology Training

Description: Funded by USAID’s Bureau of Humanitarian Assistance, 3ie is collaborating with Save the Children, Tango, and Causal Design to train implementing partners, policymakers, and humanitarian researchers in impact evaluation methodology. The trainings, developed in 2022, are expected to be delivered in early 2023. For policymakers and implementing partners, these will provide an introduction to impact evaluation and why we need evaluations in the humanitarian sectors. For humanitarian researchers, these trainings will complement other efforts to build capacity in conducting impact evaluations.

Past examples

• As part of the Uganda Evidence Program, 3ie worked  with the Office of the Prime Minister to improve government capacity to commission, manage, quality-assure and use evaluation evidence. Targeted skills development included how to develop theories of change that can support theory-based evaluation and writing effective policy briefs
• As part of our Philippines Evidence Program, 3ie worked with the Department of Labour and Employment to improve government capacity through workshops on designing and conducting impact evaluations, analyzing data and using survey software
• 3ie partnered with CLEAR-AA, the African Evaluation Association, Twende Mbele and national partners on a bold regional initiative to strengthen, harmonize and agreed on standards for evaluation training and postgraduate education
• With the African Evidence Network, the African Evaluation Association, CLEAR-AA, and evaluation providers and funders in the region, 3ie led a stakeholder dialogue about what works and what does not to develop evaluation, systematic review, and evidence mapping capacity and use
• As part of the Strengthening Evidence Use for Development Impact (SEDI) program, 3ie worked with partners and country teams from Ghana, Pakistan and Uganda to deliver a strong, innovative, gender-responsive and inclusive MLA system.
• 3ie and the Campbell Collaboration co-organized two capacity-building workshops on conducting systematic reviews for approximately 80 researchers, mainly from India, in Delhi. Our work on monitoring and measuring evidence use has been presented at key forums and within networks at the regional and national levels. 3ie experts organized capacity-development activities at the Global Evidence Summit in Cape Town on evidence production and its use.
• 3ie coordinated capacity-development activities to increase research transparency for the International Fund for Agricultural Development’s Research and Impact Assessment division in Rome. 3ie experts focused on survey sampling, systematic reviews and Survey Solutions (the World Bank’s surveying software). The division plans to implement several new transparency policies based on 3ie’s recommendations.
• 3ie, jointly with the Community of Evaluators-Nepal and International Centre for Integrated Mountain Development, organized a three-day workshop on designing, managing and conducting impact evaluations in Nepal. The participants included members of the local evaluation community, development professionals and government officials.

Want to learn more? Drop us a line at trainings@3ieimpact.org for a personalized consultation.