As director of the Hewlett Foundation’s Global Development and Population Program and new 3ie board chair, Ruth Levine is a tireless and passionate advocate for the importance of having and using high-quality evidence to inform development policies. Emmanuel Jimenez, 3ie executive director, caught up with her over email recently, and here is an edited version of their conversation.
1. What has changed about evaluation in the last ten years?
The most important achievement has been a widespread realisation that it’s possible to rigorously measure impact. There has been a tremendous increase in the availability of strong evidence about programme effectiveness and innovative approaches to measuring the impact of interventions far outside some of the more conventional areas, like health and education.
We ask harder questions now. In an earlier era, it was extremely unusual to think that you could rigorously and credibly answer the question: “Does this programme do what we were hoping it would?” We usually didn’t even ask. We were satisfied counting inputs and activities and then making a judgment call about whether a given programme was working. Pioneering academics working on impact evaluation showed us that it actually is possible, under certain conditions, to estimate impact, which has increased our collective appetite for answering hard questions.
We know more.
Through 3ie’s work to identify priority questions and fund evaluations, and through the work that others have done to conduct rigorous evaluations, we have immensely more evidence about programme effectiveness than we had five to eight years ago.
3ie’s databases contains thousands of impact evaluations and hundreds of systematic reviews of evidence on development policies and programmes. Although we still have gaps, we have a far more robust evidence base in many areas in which governments and donors make the largest investments.
Finally, we are seeing tremendous innovation. Those who are working on impact evaluation have stretched themselves in impressive and creative ways to measure impact in new sectors. For instance, we now have studies that estimate programme impacts in gender equality, climate change, and strengthening of democratic institutions – areas that we once thought were too challenging from a methodological perspective.
2. What challenges remain for 3ie and others working to help improve the lives of people living in poverty?
I think of 3ie as a unique part of a larger ecosystem of organisations and individuals working to advance the use of evidence for policymaking. 3ie brings together those who wish to use impact evaluation findings with those who conduct impact evaluations. It mobilises and focuses resources for questions that matter in the real world, and paves the way for research transparency practices to ensure good quality. 3ie also works closely with governments, and NGOs to strengthen the systems through which they prioritise evaluations and use findings.
There are many exciting challenges ahead. Two that stand out are political, rather than technical. The first is to sustain the commitment by funders to the effectiveness agenda, which includes, but is broader than, investing in high-quality, meaningful evaluation. In times of austerity, we should all care much more about applying the best available evidence about what works and what doesn’t. Right now, though, enthusiasm for evidence-informed decision-making is waning, and that may have some unfortunate spillovers for funding impact evaluations.
The second one is to build on the existing momentum in low- and middle-income countries for using the best available evidence to design public programmes and make adjustments when warranted.
"We see many governments expressing more and more interest and capacity to focus on technical issues in public policy – and one of the most important is programme effectiveness. Therefore, the impact evaluation community has an unmissable opportunity to respond to this interest comprehensively."
3. How would you characterise your style of leadership as board chair?
I guess we’ll see!
As a board chair, I’ll have to be conscious of finding the right balance in multiple ways. I will have the responsibility to support the organisation whole-heartedly while at the same time asking some tough questions. I’ll try to be sufficiently tuned in to the details of 3ie’s work to be helpful with strategy and fundraising, while keeping away from the sorts of decisions that are management’s responsibility. All board members work on achieving this balance, and I’m responsible for creating the conditions for everyone to work together to benefit 3ie’s mission. I’ll also take advantage of every opportunity to remind people that we are part of one of the most important movements in the world today: one that is fighting for facts and openness.
4. Is it unusual for a Hewlett Foundation staff member to sit on a grantee organisation’s board?
In general, we are not permitted to sit on grantee boards. As a funder, we believe we already have significant influence and want to avoid excessively driving an organisation’s agenda and operations. In this case, we made an exception because of 3ie’s board structure. There is a board seat for private funders, and last year I was asked to take that on. I will hold that position along with being board chair. During the time I serve as 3ie’s board chair, I will not make any funding recommendations concerning 3ie to the foundation.