© www.flickr.com

How can one implement an ‘ideal’ impact evaluation, one that is technically robust, produces useful findings and influences policymakers? The authors of 3ie’s new working paper Behind the scenes: managing and conducting large scale impact evaluations in Colombia address this question by drawing on the experience of implementing evaluations of four major programmes in Colombia.

Colombia’s institutional structure for monitoring and evaluation is often cited as an example for evaluation agencies in other countries. The 16 year old National System for Evaluation of Public Sector Performance (SINERGIA) in Colombia has spent US$ 15 million on the implementation of 61 evaluations (Departamento Nacional de Planeación, 2010). Over a third of these evaluations were impact evaluations, including the studies discussed in this paper: Familias en Acción Conditional Cash Transfer programme, the Jovenes en Acción youth labor training program, the Hogares Comunitarios de Bienestar nursery program, and the fiscal transfers policy Sistema General de Participaciones.

The Jovenes en Acción evaluation was considered rigorous and unbiased, and the results showed very positive effects from the intervention. However, neither the government officials nor the programme’s managers used the results of the impact evaluation. The fact that the original programme had already been turned into a new intervention before the results were out showed that there was low buy-in for the evaluation among policymakers. In contrast, the findings from the Familias en Acción evaluation were useful in protecting the government from political actors who were critical of the programme. The results of this evaluation were widely disseminated through political and academic seminars, news conferences, and several meetings among high-ranking government officials.

Read More...