Writing on the What Work’s World Bank Group blogsite last July, Lauren Kelly and Marie Gaarder called for a “wide debate” about the important issues raised by the increasing trend of development agencies to use third parties to carry out monitoring, data collection and other work in fragile and conflicted-affected locations too risky for their own staff. Last week, CDA reposted Kelly and Gaarder’s blog to help spur more discussion. Similarly, we are doing the same on our Evidence Matters blogsite.
The use of third party monitoring in development projects has gained increasing prominence in recent years, driven by demand for more robust accountability mechanisms. Typically, the goal is to bring an independent perspective on project performance by using outside parties who are neither direct beneficiaries or part of the project’s management structures.
Increasingly, however, third party monitoring is being used more as a risk management tool in countries considered too volatile for development agencies to deploy their own staff. This raises important quality, ethical, and technical issues that deserve a wider debate.
First, it is useful to understand the broader context driving this trend. Around 20 percent of the world’s poor live in fragility and conflict-affected situations and the number is expected to double by 2030. As a result, global attention and development aid is increasingly being focused on supporting prevention and early response interventions to address the drivers of conflict. To this end, the International Development Association – the part of the World Bank that helps the world’s poorest countries – has committed to doubling its financing for fragile and conflict affected states. At the same time, restrictions and security concerns around deploying its own staff in fragile countries, have pushed the World Bank to increasingly rely on various forms of third party monitoring techniques to collect data and to report project results.
Third Party Monitoring fills a critical evidence gathering gap.
Without some form of project verification, development partners, including the Bank Group would not be able to justify or account for financial support to conflict affected areas. Third party monitoring has been an integral feature of the Bank’s Afghanistan portfolio, for example, enabling real time data collection and course correction. Local and expat subject matter experts, contracted by executing agencies, have promoted accountability through thousands of site visits and local data gathering. Geared mainly towards verifying infrastructure quality, this tool has helped to pinpoint problem areas and validate field results. Similar techniques have been deployed in Iraq, Yemen and Mali as well.
But third party monitoring carries risks and limitations.
Recent research conducted by the Secure Access in Volatile Environments (SAVE) program, funded by UKAID, suggests that third party monitoring works best when it is used as a last resort, in conjunction with agencies’ internal monitoring systems, and when it is limited to exceptional areas with constrained access. Third Party Monitoring is mainly used to gather output data: it has limited ability to measure outcomes or impacts or to determine why or how a project can achieve better results. As pointed out by Gaarder and Annan (2013), it is both possible and important to carry out impact evaluations even in conflict-affected settings. The Bank has commissioned influential evaluations in conflict affected areas, like the Randomized Impact Evaluation of Afghanistan’s National Solidarity Program. It has also innovated with the use of high-frequency surveys in Somalia to establish statistical baselines and to monitor cost-effectiveness. Third Party Monitoring should supplement, rather than come at the expense of, these deeper more intensive data analysis methods that have been implemented by the Bank in fragile and conflict affected areas.
It should also not come at the expense of learning whether aid may be contributing to, or exacerbating, conflict.
Unfortunately, little is known about how we can deliver aid to conflict-affected areas without exacerbating conflict. A recent systematic review of the causal impact of development aid on violence in countries affected by civil war found that aid in conflict zones is more likely to exacerbate violence rather than to dampen it. Two recent papers summarizing the aid experience in Afghanistan and Iraq, respectively, found small program size to be an important factor for success. In the Afghanistan paper, small-scale programs were found to be relatively more successful than large-scale ones because the latter tended to create unrealistic expectations and were more subject to corruption and targeting by insurgents, undermining their near-term effectiveness. Recent evidence from Iraq suggests that smaller projects that are coordinated with security forces tend to be more successful (Berman et al.). More research on this important topic will hopefully allow donors to design aid interventions in conflict-affected areas that effectively improve livelihoods without exacerbating conflict.
Ethical questions remain unresolved.
One area that needs close attention is what ethical protocols donors apply to third party monitoring. Should third party monitors be asked to conceal their identity in an effort to promote their safety? What is the responsibility of the donor in the event that a third party is identified or, worse, detained or killed? Eighty percent of the estimated 208 aid workers killed, kidnapped or seriously wounded worldwide in 2016 were local – a figure that points to the risks involved. Other questions pertain to equity. In severely conflict affected settings, what is fair pay? And, for the sake of data accuracy, what protocols need to be put in place to mitigate the risks associated with conflict of interest, or collusion, between the monitor and the community?
As the donor community ratchets up its investment and attention towards assisting the world’s most vulnerable populations living in the most insecure places, it will need to widely debate the means by which it supports information flow both in and out of these areas. The need to know the effect that aid is having on the ground in these environments is imminent. Lives are depending on it.
Marie Gaarder joined 3ie in September 2017 as the head of evaluation and global director for country engagement and innovation.
