Claims about what might improve or harm our health are everywhere. Some of these claims are reliable, but many are not. People often don’t know how to tell the difference. Making decisions based on unreliable claims wastes resources and can result in unnecessary suffering. This problem was exacerbated by the COVID-19 pandemic, which was accompanied by an “infodemic”— an overload of information, including false or misleading information.
To make well-informed decisions, people must be able to think critically about what claims to believe and what to do. Critical thinking about what works and making informed choices depends on understanding and applying basic principles that can help us recognise unreliable claims and reliable evidence. Many curricula for primary and secondary schools are now competence-based and include critical thinking as a key competence. However, few educational interventions to teach these principles have been rigorously evaluated. To help address this gap, we developed and evaluated digital educational resources to teach some of these concepts in lower-secondary schools in Kenya, Rwanda, and Uganda.
Developing lesson plans and resources
We found that competence-based curricula in all three countries include critical thinking as a key competence, but no learning goals related to critical thinking about health. Teachers lacked prior knowledge and skills, and we did not identify any existing educational resources in use for teaching critical thinking about health. Many schools lacked ICT equipment.
We used the Informed Health Choices Key Concepts as a framework for determining the content of the resources. The 49 concepts in the framework are principles for evaluating the trustworthiness of claims and making informed choices. A panel of curriculum specialists, teachers, and researchers prioritized 17 of the concepts to include in the resources. After testing prototypes of the resources, we decided on nine concepts to include in a set of lessons that could be taught in a single school term.
To inform decisions about which teaching strategies to use in the resources, we reviewed the findings of 326 systematic reviews of teaching strategies, and we summarized the findings for 37 strategies that we considered most relevant. We tested using different strategies. In the final version of the resources, we used 12 of these teaching strategies.
The final version of the resources ("Be smart about your health") includes 10 lesson plans, a teachers’ guide, and extra resources. There are two versions of each lesson plan: a blackboard version for use in classrooms equipped only with a blackboard and a projector version that includes illustrated presentations. The resources are designed so that they can be used flexibly and can be adapted to different contexts. The resources also include training materials for teachers. The educational intervention that we evaluated consisted of a 2-3-day teacher training workshop and the ten 40-minute lessons.
We developed a test to measure students’ and teachers’ ability to understand and apply the nine concepts. The test includes two multiple-choice questions for each concept. We validated the test in all three countries. A panel of curriculum specialists, teachers, and researchers determined the cut-off for a passing score (at least 9 of 18 questions answered correctly).
Outcomes and findings
We evaluated the effects of the intervention in randomized trials in each of the countries. Altogether, 244 schools (11,344 students) took part in the three trials. Overall, 33% more students and 32% more teachers had a passing score in the intervention schools compared to the schools where we did not intervene. In total, 3397 (58%) of 5846 students and 118 (97%) of 122 teachers in the intervention schools had a passing score. Other outcomes also favoured the intervention.
We interviewed teachers, students, and parents from the intervention schools and collected other information about perceptions and experiences of the intervention. Factors that may have facilitated implementing the intervention and could facilitate scaling it up include the design of the resources and the perceived value of the lessons. Factors that impeded implementation of the intervention and could impede scaling it up are inadequate time to prepare for and teach the lessons, the lessons not being in the curricula or national examinations, and a lack of printed materials for students. Perceived benefits of the intervention include students and teachers understanding the concepts and using them in their daily lives. Other potential benefits include students being more confident, thoughtful, and open minded. Potential adverse effects include conflicts with parents and other students, misunderstanding the lessons, and misapplying what was learned.
The educational resources can be accessed, downloaded, and used in contexts with minimal resources. More information about this project and related work can be found here.
Watch the recent 3ie Evidence Dialogue webinar, where we presented this research and discussed its implications with a panel of policymakers.
