If economists are the ‘dismal scientists’ always harping on about limited resources, evaluators are often considered the ‘pessimist scientists’ who only see the half-empty glass, rather than the half-full one. The end of the year is, however, a perfect time to break with that tradition. It is a time to reflect and appreciate progress, promising initiatives, and flickers of hope. Evidence suggests that positive reinforcement can be effective in ensuring that positive developments or behaviors are maintained.
A bit of scene-setting before we share the promising news. Over the last 15 years, the body of rigorous effectiveness studies—and therefore the evidence available for decision-making—has increased from just a few hundred to over 10,000 (searchable in 3ie’s Development Evidence Portal). Unfortunately, the appropriate and timely use of this new evidence has not taken off as much as it should have—especially problematic since failure to use evidence can cost lives and money, and allows for the waste of precious taxpayers’ resources.
To tackle this and other challenges to evidence use, in late 2020, the Center for Global Development (CGD) convened a Working Group on New Evidence Tools for Policy Impact, bringing together government policymakers, multilateral organizations, bilateral aid agencies, and NGOs. The group was tasked with ‘reviewing recent progress and remaining challenges in the field and formulating recommendations for how to realize the full potential of impact evaluations and other evidence tools as essential elements of evidence-informed policymaking’. i
One key challenge identified in the group’s final report is that multilateral and bilateral development institutions often lack institutional incentives, consistent signals, and role modelling from leadership on the importance of learning and evidence use. The authors note: ‘Professional success is still too often measured by project approval and disbursements, as opposed to learning from, acting on, and sharing of evidence […] Even when evidence generation is prioritized, decision-makers may overlook the methods that are most appropriate and relevant to answering specific policy questions’. ii
With that background, on to the cause for celebration. Two recent developments in Norway signal that the leadership of the country and Norad are committed to incentivizing the generation and use of evidence. Norway’s recently released 2023 national budget states: ‘Norwegian development cooperation must maintain high quality and be knowledge-based. The ministry, in collaboration with Norad, has ongoing work to strengthen and further develop systems and methods for better measurement and analysis of results. To contribute to a more knowledge-based implementation of aid programmes, impact evaluation and formative research can be integrated into relevant programmes.’ iii
In addition to this more general directional document, the Norwegian government recently launched its food security policy. It emphasizes the importance of building on evidence from impact and process evaluations to ensure that the efforts have the largest possible positive effect.
So, what does this all mean? It shows that Norway’s Ministry of Foreign Affairs and Norad realized that key to making aid portfolio managers prioritize the generation and use of evidence is to make it easier for them to do so. How do you do that? You make funding evaluative research an expectation rather than something they need to argue. You include it as part and parcel of good development aid management rather than viewing it as something that takes away funds from the ‘real work’ which may or may not actually be working well.
In parallel to these signals and the lowering of barriers, Norad also rightly realized that incentives and signals alone would not suffice. In collaboration with the Development Learning Lab and 3ie, the agency has launched the first-ever Impact Evaluation Incubator: a three-day hands-on training workshop for Norwegian aid recipient organizations interested in using impact evaluations as a learning tool in their projects. The workshop, held in early December, was very well-received and saw highly engaged participants. Norad has committed to making this a recurrent training effort.
On behalf of CGD’s Working Group and 3ie, we congratulate Norwegian aid leaders for these initiatives, and look forward to seeing similar progress at USAID where a new chief economist and impact evaluator has just been appointed, and to see other bilateral aid agencies begin to follow suit.
This blog was authored by members of CGD’s working group: Marie Gaarder (3ie), Amanda Glassman (Center for Global Development) and Ruth Levine (IDInsight).
i Kaufman, Julia, Amanda Glassman, Ruth Levine, and Janeen Madan Keller. Breakthrough to Policy Use: Reinvigorating Impact Evaluation for Global Development. Washington, DC: Center for Global Development, 2022, p13.
ii Ibid. p10.
iii Free translation of Norway’s 2023 national budget, page 74.
Through approaches that enable development actors to innovate and improve development practice, we are getting better at development. I don't think anyone would argue that we have "solved" best development practice however. So we collectively need to invest in trying new things, create evidence on whether it works, act on that evidence to improve our programing, then "rinse and repeat". Lets celebrate our progress and inspire all to seek ways we can make development work more effective, and create the evidence that supports improved policy making.