Evidence 2018 is a regional conference organised by Africa Evidence Network (AEN) to raise awareness of evidence-informed decision-making and highlight the growing demand for evidence amongst decision makers. Approximately 350 evidence producers and users, including participants from governments, academia, think tanks, NGO’s, donors attended this conference.
3ie satellite sessions
3ie, as a co-sponsor, actively participated throughout the conference. We had awarded eight bursaries to support regional participation.
Invisibility and evidence: time’s up for evidence that doesn’t consider gendered drivers of inequality
Chair: Beryl Leach, director and head of policy and advocacy, 3ie
Panellists: Adeline Sibanda, president, African Evaluation Association; Mark Abrahams, editor-in-chief, African Evaluation Journal and senior consultant, Southern Hemisphere.
This talk show style session engaged the audience on persistent challenges of and barriers to improving the gender and equity responsiveness of research and evaluation. The session began by the group interrogating whether or not they agreed that gender is a cross-cutting determinant in poverty. Everyone agreed that the starting point for the discussion would be that gendered social norms always matter. Not looking at them adequately in development harms women and girls and can harm men and boys.
Adeline Sibanda emphasised that sex-disaggregated data is the first necessary step, but that is not looking at gender, which requires incorporating gender analysis frameworks into methodologies. Researchers often cite cost considerations as a reason to omit gender analysis in their studies. She suggested that this can be mitigated by ensuring that cost of carrying out this analysis is incorporated in the budgets and fighting with donors to fund it. AfrEA’s experience shows that just providing training on gender analysis does not overcome the gender blind culture in evaluation. She cautioned against setting up separate gender task forces or groups in evaluation, as they become marginalised.
Mark Abrahams shared insights on staff composition at the African Evaluation Journal. The composition of the board remains inequitable, with one-third more men than women. In terms of authors, there are more women than men. However, women prefer to work collaboratively in producing articles as opposed to men producing more single author articles. Men also outnumber registered women reviewers of the journal. He stated that the publishing sector has become complicit in perpetuating inequality. There is an urgent need to restructure institutions in order to ensure they are more equitable.
3ie’s Beryl Leach encouraged the audience to reflect on entrenched power dynamics that Adeline and Mark had been illustrating that dictate the research, evaluation and publishing sector. She emphasised the need to see gender and equity responsiveness as standards for ethical and quality research and evaluation. The session concluded with each participant sharing their key takeaways, as well as committing to one action they will take when they go home that is more gender responsive. Actions included doing more reading, training staff on gender analysis, refusing to take any evaluation work that is not gender responsive and improving participation of women in the AEJ and looking at how to improve the review process.
Strengthening evidence use in Uganda: what’s working and what’s to
Chair: Radhika Menon, senior policy and advocacy officer, 3ie
Panellists: Josephine Watera, Uganda Parliament M&E Unit; Rhona Mijumbi-Deve, Africa Centre for Systematic Reviews and Knowledge Translation; Timothy Lubanga, OPM, Uganda
3ie’s Radhika Menon chaired a talk-show style session on Uganda’s success in institutionalising evaluation capacity and using evidence to inform policymaking. The panellists credited the Ugandan government for investing in creating a strong institutional infrastructure, which supports an effective enabling environment for evidence production and use. This includes identifying evidence champions in the government, creating the right frameworks and allocating sufficient resources to M&E.
Panellists discussed 3ie’s role in promoting evidence-informed policymaking, especially in the public sector. 3ie-supported evaluations have led to programmatic changes and a new plan to restructure a number of public sector agencies. They also discussed challenges to using evidence, including a lack of access to high-quality data for policymakers, and a gap in communication between producers and users of evidence.
Measuring evidence use: the value of contribution tracing
Presenters: Kanika Jha Kingra, evidence impact manager, Kirthi V. Rao, consultant- research associate and Radhika Menon, senior policy and advocacy officer
This session drew on 3ie’s experience with applying contribution tracing to measure evidence uptake and use from a portfolio of completed impact evaluations and systematic reviews. The presenters started by focusing on the myriad factors, apart from evidence, that influence decision-making. They underlined the importance of strengthening early and ongoing engagement to improve the relevance and usefulness of evidence being generated. These approaches have improved 3ie’s processes for monitoring and measuring evidence use, and improved the rigour of validating evidence use claims from 3ie-supported studies.
Participants worked in groups to identify uptake or use from research that they have commissioned or been part of. They were encouraged to identify ways to monitor and measure changes with the tools and processes they already have. Participants queried how this methodology can be adapted to inform their own work, without it being resource intensive. The presenters emphasised the value of collecting relevant monitoring data from the very beginning to ensure that it is cost-effective and so that evidence use claims can be validated more robustly.
Joint AEN-3ie workshops
Networking between Francophone and Anglophone Africa: bridging a post-colonial divide
Chair: Beryl Leach, director and head, policy and advocacy, 3ie
Panellists: Deo-Gracias Houndolo, Evaluation Specialist, 3ie;John Njovu, AfrEA board member; Olfa Soukri Cherif, Member of Parliament, Tunisia; Siziwe Ngcwabe, Africa Evidence Network senior manager
This was the first-evers ession on barriers and challenges to networking between Francophone and Anglophone Africa. The most prevalent reason perpetuating the barriers was language—the majority English speakers in the region do not learn French. French speakers often have to learn English as adults due to work or career demands. There was also a discussion on how institutional structures vary between these countries, such as differences in the way governments operate because of their colonial legacies. The panellists and audience members highlighted how institutions can work together to find cost-effective ways, such as using voluntary translators, simultaneous translations at conferences, and using Swahili as a common language to create an environment to facilitate networking and collaboration. There was a strong call for organisations producing publications to offer them in both languages.
Capacity development to use evidence in decision-making: working toward partnerships and building space to use approaches that work
Chair: Ruth Stewart, chairperson, Africa Evidence Network
Speakers: Adeline Sibanda, president, African Evaluation Association; Beryl Leach, director and head, policy and advocacy, 3ie; Caitlyn Blaser, senior M&E technical specialist, CLEAR-Anglophone Africa; Emily Hayter, programme specialist, Evidence for Policy at INASP.
Participants in this session shared multiple perspectives to take forward the regional dialogue on developing capacity to produce and use evidence. The panellists highlighted the need to focus on not just developing capacity, but also co-producing evidence and the value of learning from all actors involved in this process. Speakers shared various examples from the region on their efforts to embed systems within legislatures and government, to improve understanding of and therefore use of evidence. There was an acknowledgement that it isn’t the lack of skills among decision makers, but the absence of opportunities to network, learn from peers and engage to access rigorous evidence. The audience also highlighted the need to move beyond just conducting workshops, to carrying out assessments and diagnostics to be responsive to the needs of decision makers.
Understanding HIV research in Sub-Saharan Africa: Analysis of 300 HIV studies from the Impact Evaluation Repository (IER)
Presenter: Jorge Miranda, data analyst, 3ie
As part of a panel on evidence in healthcare, 3ie’s Jorge Miranda presented key findings from an analysis of impact evaluations of HIV programming included in 3ie’s Impact Evaluation Repository. He used several charts and infographics to show the top ten countries producing these evaluations, top researchers in the field and the change in publications on HIV over time. He also explained the inclusion criteria and methodology, shared results of topic extraction, network analysis using authorship and citation data.
Strengthening the supply side of evidence-informed policymaking
Chair: Beryl Leach, director and head of policy and advocacy, 3ie
Speakers: Aloysius Ssennyonjo, project manager, Makerere University School of Public Health, Uganda; Nasreen Jessani , senior technical advisor, Local Public Health Policy and Health Systems, South Africa; Violet Murunga , PhD student at the University of Liverpool/African Institute for Development Policy, Kenya
In this session, Aloysisus Ssennyonjo shared three interrelated approaches being used in the EU-funded, SPEED partnership to achieve universal health coverage in Uganda: multi-stakeholder engagements to understand problems and identify solutions, policy implementation monitoring through conducting a policy implementation barometer, and capacity building for faculty, students and partner institutions.
Nasreen Jessani of the Centre for Evidence-based Healthcare introduced the centre’s two-day elective course on knowledge translation. This course is an excellent example of the importance of using adult learning principles for this type of capacity development. Mixing researchers and decision-makers in the workshops improved value to participants and having an experienced facilitator is critical for effective learning. Enhancing knowledge and skills ‘just in time’ rather than ‘just in case’ has demonstrated increased potential for immediate relevance, uptake and sustainability.
Over the past decade, numerous interventions aimed at enhancing knowledge translation in L&MICs have been tested. However, most efforts have focused on policymaker perspectives. Violet Murunga (PhD student at the Liverpool School of Tropical Medicine) shared findings from her literature review in this area. There is a need for more studies and interventions focusing on understanding and addressing the barriers for researchers.