Community of Evaluators- South Asia Conference (COE-SA)

6-9 June, 2017 Thimpu, Bhutan

The Community of Evaluators – South Asia (CoE-SA) organised the 4th Evaluation Conclave in Bhutan from 6 – 9 June 2017. The Conclave 2017 was a collaborative event of CoE – SA, the Gross National Happiness Commission (GNHC) of the Royal Government of Bhutan, and the Evaluation Association of Bhutan. 3ie was a major sponsor for this event. The theme for the conference was ‘Well-being and sustainable development – new frontiers in evaluation’. 

Conclave 2017 focused  largely on the relationship between well-being and sustainable development with a view to demonstrate the importance of evaluations in these areas. Conclave 2017 was a great forum for government, civil society, donors and evaluation fora to deliberate on a number of fronts.

3ie staff participants organised two skill development workshops, five panels and presented one paper. Emmanuel Jimenez spoke at the inaugural keynote panel and moderated the second keynote session. 3ie supported eight bursaries, including ones to ensure grantees under our financial inclusion and improving adolescents’ lives window could share their experiences with a wider audience. Take a quick look at the 3ie participation at the Evaluation Conclave 2017 through storify link below. 


Interviews with experts at Evaluation Conclave 2017


3ie sponsored skills development workshops and demonstrations (6-7 June)

Evaluating evidence uptake and use: tips for monitoring, measuring and reporting (WS-15)
Key people: Beryl Leach & Kanika Jha
Venue: City Hotel
Date: 6 June
Time: 09:30 to 12:30
More and more, development sector donors are demanding to see evidence of uptake and use of research, particularly evaluations. To be able to do this effectively and systematically it is important to know what contributes to promoting evidence uptake and what counts for use? For 3ie, this means knowing what to monitor and measure about stakeholder engagement and communication and why it is important for improving evidence use? As an evaluator, what can I do if I only come in at the end of an intervention and need to measure the use of evidence about a programme? What makes sense to measure? Through interactive discussions and exercises, participants will learn about how and where to look for evidence of uptake and use. They will get an overview of methods for capturing and interpreting the contribution of research findings to decision making about development programming. They will learn about different ways to report on evidence use, depending on the targeted audiences. This workshop draws extensively on 3ie’s own work in developing effective systems for promoting and monitoring stakeholder engagement and evidence uptake and use from over 150 impact evaluations and 35 systematic reviews.


Using Natural Experiments to Evaluate Policies and Interventions [DEMO -2]
Francis Rathinam, Neeta Goel, 3ie
Venue: City Hotel
Date: 7 June
Time: 13:30- 15:00
: Prospective impact evaluation techniques such as randomised control trials (RCT) may provide the strongest evidence of a causal effect, but they may not be feasible or even ethical in some cases. For example, RCTs cannot be conducted on a nationwide policy, or when beneficiaries self-select into the programme, or when a programme / policy has already been implemented. In these cases, natural experiments can serve as the method of choice. Natural experiments compare a group of people affected by an event beyond their control with a comparable group of people who are not affected by the event. These studies rely mostly on secondary data, and hence are cost-effective and easier to execute. In this demonstration skills workshop, we will discuss a few representative natural experiments, their empirical strategy, data requirements and sensitivity tests required to ensure internal validity.  Because these studies are retrospective, we will also discuss strategies to identify and design natural experiments.

3ie sponsored pre-formed panels (8-9 June)

Keynote Panel
Venue: Le Meridian Hotel
Date: 8 June
Time: 08:30 to 10:30
Moderator: Nancy MacPherson, Panellists: Emmanuel Jimenez
[Fostering governance, accountability, credibility and transparency], Natalia Kosheleva [Bridging gender and complexity to ensure high quality of gender-responsive evaluation], Penny Hawkins [Innovative Methods and Partnerships], Robert McLean [Evaluation with Values], Adeline Sibanda [The Role of Partnerships in Strengthening Voices, Tools and Methods from the Global South].


Innovations in evaluation methods: Evidence from behavioral science interventions to promote latrine use in rural India (PP-4) International Initiative for Impact Evaluation (3ie)
Panelists: Neeta Goel, Shaon Lahiri
Venue: Le Meridien Hotel
Date: 8 June
Time 11:00-12:30

Abstract: Considerable strides have been made in India during the MDG period in tackling the problem of open defecation, but 61% of the 1.3 billion population still reports defecating in the open. Though the government of India has enacted a national sanitation program, guaranteeing toilets for all by 2019, latrine construction does not necessarily translate to latrine use. In order to generate a body of evidence on how to promote latrine use in rural India, where open defecation is most concentrated, 3ie awarded nine grants to research teams for the design and implementation of low-cost, behavioral interventions conducted over three months in diverse geographies of rural India. The interventions drawn upon behavioral science, comprising psychology, sociology, economics, anthropology and other social sciences. The panel will showcase the results of the 9 studies, highlighting innovative methods of incorporating behavioral science into evaluation.


Empowering adolescents: Evidence from 3ie’s Improving Adolescents’ Lives thematic window
(PP-3)  Moderator: Diana Milena Lopez-Avila, Panelists: Alexandra Avdeenko, Sancheeta Ghosh, Renu Singh
Venue: Le Meridien Hotel
Date: 8 June
Time: 13:30-15
Unicef’s regional program “Improving Adolescents’ Lives in South Asia” aims at reducing and preventing child marriage through empowering adolescents, boys and girls.  To generate evidence on whether the program is generating and impact on improving adolescents’ lives, 3ie is supporting 3 impact evaluations in India, Pakistan and Afghanistan. This panel brings together researchers and implementers to discuss the challenges at targeting adolescents and ways to measure their empowerment. The panel brings, particularly, the experiences from the impact evaluations in India and Pakistan. Results from the baseline data from the impact evaluation in Pakistan will also be presenting, together with the different methods, qualitative and quantitative, used to capture the data.


Engaging with evidence: Do financial inclusion programmes have an impact on poverty reduction? (PP-10) Moderator: Priyanka Dubey, Panelists: Maren Duvendack, Giovanna Prennushi
Venue: Le Meridien Hotel
Date: 8 June
Time: 13:30-15:00
Access to basic financial services - savings, credit, insurance and payment can help spur economic activity and improve the lives of the poor. However, evidence from interventions that aim to improve access to finance to the poor in low and middle income countries are limited. This panel will share some of the recent evidence from rigorous impact evaluations in South Asia. There are three broad key themes that this panel will discuss: the ultimate impact of financial inclusion interventions on income, consumption, health, education, self-employment, business activities, and livelihoods; and challenges in evaluating real world, large scale financial inclusion programmes, and the lessons for the government, civil society organisations and policy makers on implementation and evaluation challenges.


Learning from impact evaluations of community engagement approaches and programmes (PP-14)
Panelists: Santanu Pramanik, Sudip Mahapatra, Radhika Menon (Moderator)
Venue: City Hotel
Date: 8 June
Time: 15:30- 17:00
  A wide variety of approaches and programmes have been used to engage communities for making them in-charge of their own development. This panel discussion will draw on specific examples to take stock of what we are learning from the impact evaluations of community engagement approaches and programmes. The discussion will focus on the theory of change of participatory interventions, the appropriateness of various impact evaluation methods and indicators used for assessment and the existing evidence on the effectiveness of community engagement approaches and programmes.


Keynote Panel
Moderator: Emmanuel Jimenez Panellists: Nancy MacPherson [What do we do when evidence does not matter?], Jo Puri [Another extinction - what is the evidence saying about environment sustainability?], Katherine Hay [When equity and evidence come last, what is the role of evaluation?], A K Shiva Kumar [Establishing violence-free societies for children: Challenges in evaluation], John Gargani [I don’t want to change the world, I want to make it better: The growing responsibility of evaluation to improve the lives of people]
Venue- Le Meridien Hotel
Date: 9 June
Time- 08 30 – 10 30


EGM paper presentation
Do group-based livelihood programmes have an impact on poverty? An Evidence Gap Map
(Presented by Avantika Bagai)  
Venue: Le Meridien
Date: 9 June
Time: 11:00- 12:30           
  Evidence Gap Map (EGM) are thematic collections of impact evaluations and systematic reviews that measure the effects of international development policies and programmes. These maps present a visual overview of available evidence from completed and ongoing studies and systematic reviews in a sector or sub-sector in terms of the types of programmes (or interventions) evaluated and the outcomes measured. This EGM reviews all group-based rural livelihoods programmes, in low and middle income countries (LMICs), to learn what initiatives/actions/strategies/outcomes) makes programmes work, and why. Livelihood programmes typically include financial inclusion, self-employment and skill training that target the poor. This assessment of available information becomes critical in the wake of many LMICs adopting large-scale public policies that advocate the promotion of livelihoods through groups.

Photo©Ganesh Dhamodkar, Wikimedia

Scroll to Top