Even if development projects are highly effective at a small scale, many are abandoned before they can have the type of large-scale impacts needed to alleviate poverty. To learn more about how projects can grow past the pilot phase, nearly 300 people joined 3ie’s expert panel on scaling up interventions, the last of its Virtual Evidence Weeks events. Plans to scale up need to be developed early and take into account both the capacity of implementing institutions and the wide range of people who might be affected, panelists said.
During the pilot phase of any intervention, planning should already be underway for a pathway to scale it up, said Johannes F Linn, a non-resident senior fellow at the Brookings Institution. That pathway may require adapting the intervention as it transitions to a larger scale.
“Scaling up is about achieving the desired and appropriate scale of development impact,” Linn said. “It’s not about scaling the particular solution… you may have to change the solution.”
Funding organizations often focus on smaller-scale projects in part because of their limited resources, said Ruth Levine, the incoming CEO of IDinsight.
“All funders recognize that they themselves do not have sufficient resources to really solve any consequential problems by themselves,” Levine said. “There’s an attraction to some small-scale activity that you can then imagine would then be scaled up by others.”
While the idea of such a handoff may look beautiful when written in a proposal, it seldom comes to pass, she said.
“We have very few instances where an NGO-incubated idea has then been scaled by government,” Levine said. “One reason that we fail is that we engage in too much magical thinking … we underestimate the complexities, except in retrospect when we can list them all out as reasons our dreams didn’t come true.”
Even if a program is shown to be effective at a small scale via rigorous research, these findings may not apply to a scaled-up version of the same program, said Musfiq Mobarak, professor of economics at Yale University.
“There’s a disconnect between the research we do at pilot scale and the policy question we’re trying to answer,” Mobarak said. “At scale, the effect of the program will look different than what you get at a pilot scale.”
There are whole new categories of effects at large scale, he said. Many of these changes, like shifts in market prices or wages, are what economists call “general equilibrium effects.”
“Unintended consequences of these programs at scale may not show up at the pilot [stage],” Mobarak said.
Another important issue is the differing levels of capacity between the organizations that implement small-scale studies and the institutions that implement large-scale programs, panelists agreed.
“Scaling up in itself is based not only on the evidence from a pilot program, but also on the capacity of the scaling-up agency, and also the political will to scale up,” said Bidisha Barooah, a senior evaluation specialist at 3ie.
These differences in capacity have been evident in 3ie’s work on rural livelihoods programming in India, where some programs are already scaling up, Barooah said. (For more on that work, see blog posts here and here.)
“Our experience showed that taking the program to scale meant that the quality of implementation changed, because the institutions that were responsible for taking the program to scale had different capacities than what was there in the pilot phase,” Barooah said. “The challenge typically comes when the agency that is doing the pilot is different than the agency that is doing the scale-up.”
Finally, scaling up is only one side of the equation. Because budgets are often fixed, especially in the context of the COVID-19 crisis, scaling up any one initiative requires shifting resources from somewhere else, said Rachel Turner, director for economic development at the United Kingdom’s Department for International Development (DFID).
“There’s scaling and de-scaling, and the day-to-day decisions are as much about what you stop doing and where you take the money out, as much as what you grow,” Turner said.
To watch the whole discussion, click here.
