We are capping our 10th anniversary celebrations with a two-part blog from our founding and current executive directors. Their reflections capture 3ie's journey from starting up to taking off and cruising ahead.
Even though it’s been over three years since I joined 3ie, I was still fascinated to read Howard White’s reflections on how it all started. A question that has often come up for me as well is about why I’m in Delhi. As Howard said, the vision of an organisation having its locus of authority be in the Global South, is something that attracted me as well. Howard’s reflections also hinted at how 3ie has continuously evolved to meet changing needs. In fact, our ability to learn and adapt is one of our core strengths. So, we’re regularly drawing on our first decade of experience to take us beyond our take-off stage and help us cruise through new challenges.
Where we are headed on evidence use
One such challenge is evidence uptake and use to inform decision-making. A study of our databases shows that there is now a much larger number of rigorous studies but there is not a lot that is known about how evidence has been used, if at all. 3ie’s grant programmes have stressed policy relevance and promotion of evidence uptake through various measures – scoring criteria for policy relevance and tools such as the stakeholder engagement and evidence uptake plans that grantees implement through the entire study period.
A key question that has emerged is, when and how do we know when evidence uptake happens? From the beginning, 3ie has been addressing this challenge. We are leaders in our sector in terms of developing effective evidence use monitoring and measuring tools and methods. We have evidence use briefs that document how context, actors and other mechanisms contribute to or limit the use of evidence in policies and programme. Our evidence use team is also using methods such as contribution tracing with Bayesian testing to help ensure we are rigorous about making claims about the contributions of 3ie-funded studies and reviews to decision-making. We will continue to share what we are learning in this important area through publications and multimedia resources.
As our experience shows, producing evidence does not guarantee that anyone is going to use it. As an economist I still remember how we used the artificial construct of the ‘benevolent dictator’ to come up with ‘optimal’ policies or interventions: If only the decision-makers knew what to do, they’d do the right thing. If only that were true in the real world. Many have written (and some have won Nobel prizes) on why decisions are not made only on rational choices and on the basis of hard evidence. Decisions about policies and programmes are always influenced by many factors. So, it is crucial that evidence be accessible, not only to policymakers and programme implementers, but also to policy or programme beneficiaries themselves and civil society that seeks to influence and the media reporting out to wider society. This is why 3ie will be investing in our impact evaluation and systematic review repositories of evidence on development effectiveness, to make them easier to search and more accessible and useful to decision-makers and influencers, as well as researchers.
Moving the needle on research transparency and ethics
Another challenge is helping to navigate through the larger amount of evidence. Howard’s blog already mentioned the need for synthesis, which I can only heartily endorse. But another emerging lesson is that when research is transparent, it is more likely to be credible and thus have more influence. It also helps improve the quality of the research if one knows it will come under greater scrutiny. This is one reason why 3ie has been stressing research transparency. As 3ie’s grantees know, we require their study reports and the associated data to be publicly available. We have also begun to conduct push button replications of 3ie-funded studies as part of our final quality assurance before we accept final study reports.
3ie has been an active member of global discussions about improving research ethics since 2015. This work has always been linked to our commitment to improving research transparency. We recently hosted a dialogue in India on research transparency that validated the position we have taken on ethical issues. While it is the researcher’s responsibility to share their findings and data widely, research subjects also have a right to participate in research, be part of analysis and validation of findings and to have full access to the results of the study, including the report.
Moreover, when data are released, it is crucial for researchers, funders and users to ensure the privacy of those whose data are being collected. These are issues that concern us all – after all, it is upsetting when we find out that those who collect data on us (such as Facebook or Amazon), provide the data either willingly (when they sell market information) or unwillingly (when they are hacked) to others. But it is also our moral responsibility to protect poor and vulnerable populations, who unlike the rest of us, do not choose to patronise the data collectors. We intend to continue our dialogue on ethics and evaluation in the future.
Building partnerships for capacity development
3ie’s work on the ground in several countries has brought into focus the importance of developing capacity, not only of individuals who generate and use evidence, but also of institutions. While not all countries are ready to undertake and absorb the lessons from impact evaluations, many are now undertaking some kind of evaluation. 3ie has helped developing country governments think about how to situate impact evaluations and rigorous studies within their broader evidence system. Some of this work has informed the development of national evaluation policies, such as in Uganda.
As part of our response, 3ie has been customising our work at country level and in our thematic evidence programmes to match what’s needed. We start by mapping existing evidence, scoping what key actors think are priority evidence needs and synthesising existing evidence, when possible. If new evidence is required, we now support initial process and formative evaluations that inform the feasibility and strengthen the design of eventual impact evaluations.
Because capacity development is complex and involves some aspects that go beyond our niche, as well as being hard work, we have enlisted knowledgeable partners and begun to mobilise our own members in this effort. We are revamping our membership model to facilitate south-south institutional interactions. We are also co-creating programmes with some of our members. For example, in an ambitious programme, 3ie is partnering with the Benin government on the West Africa Capacity-building and Impact Evaluation (WACIE) programme. This multi-year initiative will promote the institutionalisation of evaluation in government systems across eight members of the West African Economic and Monetary Union: Benin, Burkina Faso, Côte d’Ivoire, Guinea-Bissau, Mali, Niger, Senegal and Togo.
Despite the dramatic progress over the past 15 years, there are still huge evidence gaps in development that need to be filled and more effective and concerted efforts required to build national capacities to produce and use evidence that meets their needs. 3ie has demonstrated, in our short history, many important contributions to filling those gaps. And as we look forward, we are applying lessons we are learning and adapting how we work to be effective leaders and partners in capacity development and global advocacy. The next ten years promises to be as exciting as the first ten.
To read our founding executive director, Howard White's blog, click here.
