Despite being highly effective in preventing avoidable disease and death among children, routine child immunization remains well below targets. Child immunization has been especially hit hard by the COVID-19 pandemic, reversing years of gains. In 2021, 25 million children missed essential vaccines, a high not seen since 2009. Putting immunization back on track, especially in low- and middle-income countries (L&MICs), is an urgent priority.
In keeping with this year’s World Immunization Week’s theme of The Big Catch-up, 3ie’s forthcoming Systematic Review of systematic reviews aims to support efforts to increase routine immunization outcomes in L&MICs. The Review, the largest of its kind, goes over a considerable amount of evidence synthesized in 24 systematic reviews to understand what really works in these contexts to address the enablers and barriers related to immunization access, vaccine delivery, and demand. For more insight, we asked the authors of this forthcoming Review of reviews – Monica Jain and Maren Duvendack – about the main highlights and actionable findings.
We are talking about a systematic review (SR) of systematic reviews. Since systematic review findings are already ready to be used for decision-making, what is the value in further systematically synthesizing available systematic reviews?
There are several benefits of conducting a review of systematic reviews. First, it allows us to look at the evidence base as a whole systematically using a uniform conceptual framework. For example, in our review, we collated learning from similar interventions by using a uniform intervention framework. Second, in cases where more than one systematic review exists on a particular intervention type, a review of reviews compares and contrasts their findings to provide more clarity on the evidence base. For instance, in this case there were many SRs on immunization programs and some had discordant findings. Third, we can analyze the strengths and weaknesses of the existing evidence base through a review of reviews. The exercise can help in identifying areas of improvement, including identifying critical evidence synthesis gaps and highlighting if systematic reviews are dated.
Could you touch on the main findings and takeaways from the Review of reviews for implementers and policymakers?
Overall, the evidence base is highly varied in terms of scope, intervention types, and outcomes. Our findings suggest large positive and statistically significant effects for caregiver-oriented interventions, i.e., interventions that seek to promote routine immunization by engaging with caregivers. These effective caregiver-oriented interventions include one-time sensitization and education campaigns as well as written messages to caregivers. Among health-systems oriented interventions, home visits by health workers seem to have positive effects. However, the evidence is mixed for pay-for-performance schemes and inconclusive for contracting out routine childhood immunization services to non-governmental providers. For community-oriented interventions, the effects are positive and statistically significant but small.
For policymakers and implementers, the evidence base provides clear answers on the effectiveness of caregiver-oriented interventions and, to a certain degree, community-oriented interventions whereas the evidence on health systems-oriented interventions is not as informative.
In the process of systematically reviewing reviews, did you find key areas where the available evidence was insufficient or where there is a need for more research?
There are large gaps in the evidence base, especially on critical interventions such as monetary incentives to caregivers, which have received a lot of attention in recent years. Although interventions building skills, capacity and motivation of formal health workers are of particular policy relevance and L&MIC governments are increasingly adopting and expanding interventions leveraging digital technologies like new Health Management Information Systems or dashboards, there are considerable evidence gaps in these intervention types too. In future research, it would be important to better understand the complementarity of interventions as well, because we do not yet know which combinations of interventions work best in terms of improving child immunization outcomes.
Any thoughts on why the evidence gaps, especially on policy-relevant health systems interventions, exist and how future research can fill them?
These evidence gaps could be related to the feasibility or challenges of carrying out impact evaluations of health interventions that are usually implemented at higher administrative levels. For example, an electronic immunization register intervention in India would be implemented at least at the block level or even higher at the district level for reasons of financial and technical viability. It becomes harder to find valid counterfactuals at higher administrative levels and it also becomes more costly to collect data across wider geographical areas. This does not mean evaluations of such interventions are not possible. It requires more planning, thought, resourcing as well as collaboration with the existing health administration to carry them out. One can also explore using administrative data for such evaluations if these are available and of high quality.
How are you taking the learning from this work forward?
We presented key findings from the project at an online seminar organized by the Gates Foundation. Our presentation slides and video are publicly available for those who are interested. Recently, we submitted a manuscript to a peer-reviewed public health journal. Anybody interested in learning more about this work can also contact us directly.
