Integrating cost evidence in research and evaluation

Public resources have competing usages, and efficient allocation of resources for development projects is key to improving social outcomes. Policymakers need information on both effectiveness and costs to support prioritization of scarce resources, budget optimization and project management. 3ie’s Cost Evidence Group designs and implements rigorous cost analysis, including as an integrated part of impact evaluation, provides training on measuring and analyzing costs, and supports partners in better use of cost evidence in decision-making.

 

Integrating cost evidence in research and evaluation

3ie has advocated for the inclusion of cost evidence in research by contributing to multilateral costing initiatives, developing publicly available guidance materials and trainings, and producing cost evidence to inform decision-making. This followed the work of Brown and Tanner (2019) who identified cost evidence as a critical gap in impact evaluation research. 

Our work aims to promote the use of cost evidence in evaluation both internally within 3ie and externally among policymakers, funders, and researchers. 

Key objectives of 3ie’s Cost Evidence Group:

  • Design and implement rigorous cost analysis, including as an integrated part of impact evaluation
  • Provide training on measuring and analyzing costs
  • Maintain ‘living’ guidelines and tools for implementing cost analysis as part of impact evaluation
  • Assess the quality of cost analyses
  • Advocate for better use of cost evidence in impact evaluation and related evidence approaches, including by contributing to sector-wide costing communities of practice, and 
  • Support 3ie partners to identify cost analysis applications that meet their unique needs and context

Evaluating the outcomes of interventions is only one side of the coin; the other, equally crucial, aspect involves analyzing the costs associated with achieving these outcomes. 3ie champions the integration of detailed cost analysis in impact evaluations to enhance decision-making and foster transparency and accountability.

Our approach:

  • Cost analysis plays a pivotal role in informing policy and programmatic decisions. By measuring and analyzing the costs involved in achieving specific outcomes, stakeholders are equipped with a clearer understanding of the value generated by different interventions to optimize resource allocation.
  • Rigorous cost analyses provide a detailed account of how resources are utilized to inform programmatic decisions such as program budgeting, design, and scale-up. 
  • By systematically analyzing and reporting costs, 3ie advocates for a culture of accountability where decisions are made based on evidence of cost-effectiveness, reinforcing the commitment to prudent and responsible use of resources.

Some of the initiatives we are working on include:

  • Piloting our approach on 3ie Evaluation and Evidence, Policy and Learning (EPL) projects
  • Standards for assessing CEA in evidence synthesis

 

3ie has developed a handbook that lays out guidelines providing impact evaluators with tools and ideas on how to design, implement and report metrics of cost-effectiveness in experimental and quasi-experimental impact evaluation designs. The 3ie guide enables you to identify the most appropriate way to collect cost data as part of the evaluation after accounting for uncertainty, missing data, and sensitivity analyses.

We continue to refine and iterate these guidelines based on piloting and user feedback; to that end, we’re currently expanding our scope to include other common CEA applications, such as scale-up decision-support, using incremental cost-effectiveness ratio (ICER) thresholds, and budget impact analysis. We welcome all inquiries, critiques, questions and suggestions. Reach out to a 3ie cost specialist to unpack and calculate the actual costs or true costs of your project at info@3ieimpact.org. 

Vox | We’re finally figuring out if foreign aid is any better than handing out cash

In this article, 3ie’s Daniel Handel underscores the need for evidence-based aid programming, highlighting ongoing efforts to enhance evaluation quality and foster evidence-driven decision-making within USAID and beyond.

Learn more about 3ie’s Commitment to Evidence in this blog.

Barriers to integrating cost-effectiveness analysis & cost-benefit analysis into impact evaluations

This session, as part of 3ie's virtual Evidence Weeks Webinars, has discussions based on comparisons between practices in the Health Sector and International Development. What mechanisms should the field of international development put in place to advance both the methods and practice of conducting CEA? How would this work in international development? What accommodation should be made for the many sectors included in International Development?