7-9 February, 2018, Silver Oak, India Habitat Centre, New Delhi, India
EvalFest 2018 is the first national conference aimed at promoting a culture of evaluation since the successful India Evaluation Week 2015 conference held in Delhi. 3ie was a major sponsor of the three-day event. Around 200 contributors participated in this event, many of whom were state government officials interested in knowing more about evaluation. The discussion was heightened by the presence of high-level experts in evaluation, which set the tone for the event. EvalFest was jointly organised by the Evaluation Community of India, Oxfam India, Centre for Media Studies, International Institute of Health Management Research and many others.
Highlights from 3ie’s sessions
Ethical standards in social research and evaluation
Chair: PN Vasanti, director, CMS
Panelists: Beryl Leach, director and head, policy and advocacy, 3ie; and Sushanta K. Banerjee, senior director, research and evaluation, Ipas Development Foundation
Panellists and the audience identified a number of ways in which we could improve the application of ethical standards in evaluation in India. Beryl Leach provided an overview of the important dialogues going on at the global level about improving ethics in evaluation, using examples from DFID’s review in 2016, 3ie’ own work and a UK-based dialogue on ethics in impact evaluation spearheaded by the Centre for Development Impact. Sushanta Banerjee provided an overview of institutional review boards in India, detailing where improvements need to be made. Both underscored the importance of values-based ethical standards where accountability and roles are shared and exist throughout the research cycle. Ethics are hard to define and standards are hard to agree. The first step toward effective ethics policies and processes is to promote an ongoing dialogue.
Emmanuel Jimenez, executive director, 3ie gave the first keynote on the importance of ‘value’ in evaluation and how it is geared toward seeking truth to create a just society. He also stressed the importance of ethics in evaluation and spoke about the need to place the interests of communities at the forefront, as opposed principal investigators or others commissioning an evaluation. Marco Segone (United Nations Population Fund) and Joseph Barnes (ImpactReady) then talked about ‘visibility’ and ‘voice’ in evaluations respectively.
High-quality evaluation: the importance of stakeholder engagement
Panelists: Harini Kannan, senior research manager, J-PAL South Asia; Sudha Narayanan, associate professor, Indira Gandhi Institute of Development Research; Sudipta Mondal, director, Monitoring and Evaluation, Project Concern International, Chair: Beryl Leach, director and head, policy, advocacy and communication, 3ie and Rituu B Nanda, facilitator, online community of practice on gender and evaluation, Institute of Social Studies Trust.
This session combined a talk show portion and small groups discussing the challenges of effective stakeholder engagement. Each panellist briefly shared lessons about their work in different contexts. Thereafter audience members proceeded to form groups based on their interest, to discuss different questions put forward by the panellists. Some of the discussion centred around ethical dilemmas in conducting evaluations with respect to different stakeholders, where it was discussed that it is imperative to analyse power relations between the implementing agency, researchers and beneficiaries. Another group discussed the barriers of engaging with policymakers, and highlighted the importance of mapping out the priorities and motivations of policymakers and other stakeholders involved in designing and conducting evaluations. A different group discussed the importance of involving local communities in the evaluations, and called for a ‘shift’ of the approach that considers the community as a ‘target’ group as opposed to a ‘stakeholder’. Questions around timing, content and evaluator credibility were also discussed.
Deconstructing innovations in evaluation
Chaired by Neeta Goel, senior evaluation specialist at 3ie, this session included keynote speeches Emmanuel Jimenez (Manny) and Shobhini Mukherji, executive director, J-PAL South Asia, on their experiences with using innovative practices in evaluation. The panel discussed the use of various innovative tools in data collection. Manny gave the example of 3ie-funded impact evaluation of the Khushi Baby- an intervention which provided a digital vaccination record in a wearable pendant or a sticker on routine immunisation in Rajasthan. He also referred to using drones to collect data in Philippines. He also spoke about how formative evaluations can increase the effectiveness of impact evaluations. Shobhini Mukherjee gave other examples from the evaluation of the ongoing Direct Benefit Transfer project in collaboration with the Tamil Nadu government, and various other innovative approaches to evaluation of projects that J-PAL is undertaking.