Conservation Agriculture Evaluation Project in Northern Ghana: a formative evaluation using a framed field experiment

Our expertise
3ie invented evidence gap maps to improve decision-making around where to make investments in producing more evidence or synthesising existing evidence. Since then, we have pioneered further advancements, including developing our interactive online map platform and innovating evidence mapping to broaden its use in development decision-making.

3ie funds and quality assures formative and impact evaluations of development programmes in low- and middle-income countries. These evaluations provide crucial evidence on what works, for whom, why and at what cost. On our website, we host the largest-of-its-kind impact evaluation repository that includes summaries of both 3ie-funded and other studies.

We are global leaders in producing and assuring the quality of theory-based systematic reviews of the effectiveness of development interventions. 3ie continues to innovate and improve synthesis and systematic review methodologies and the uptake and use of synthesised evidence.

3ie specialises in increasing access to, demand for and use of evidence by governments, parliaments, programme managers, civil society, programme participants and the media. We do this by emphasising the value of planning and engaging with stakeholders to ensure that evaluations and reviews are relevant and useful. We use robust and effective monitoring to measure evidence use so that we can convey evidence impact on programmes and policies with greater confidence.

We set up our replication programme to address the need for a freely available global public good that helps improve the quality and reliability of impact evaluation evidence used for development decision-making. Replication is the most established method of research validation in science, yet it has not been fully embraced by the research community or development donors, leading to this gap.

3ie’s evidence programmes and services help build technical capacity to commission and conduct rigorous evaluations, produce evidence gap maps, conduct evidence synthesis and use evidence. We also work with L&MIC governments to build effective monitoring and evaluation systems. 3ie’s bursary programme supports L&MIC policymakers, programme managers and researchers to participate in specialised training and international events.

3ie has always been strongly committed to research transparency and open access to data. We are proud to be a leader in the growing movement to improve global standards for research transparency.

On request, 3ie provides services to partners for supporting the generation and use of evidence to inform their development policies and programmes. We commission and quality assure evidence gap maps, evaluations and syntheses as well as provide training.

3ie plays a unique role in promoting collaboration among researchers, policymakers and development programme managers at country, regional and global levels. Our global advocacy for evidence-informed action helps ensure decision makers have quality evidence about what works when they need it.
Evidence hub
3ie’s Development Evidence Portal is the largest-of-its-kind repository of rigorous evidence on what works in international development. This portal includes evaluations and synthesis of studies conducted in low-and middle-income countries. It combines records from 3ie’s Impact Evaluation and Systematic Review repositories, as well as, evidence gap maps.

These provide a visual display of completed and ongoing systematic reviews and impact evaluations in a sector or sub-sector, structured around a framework of interventions and outcomes.

Evidence impact summaries briefly describe how 3ie-supported evidence has informed and influenced decision makers. Each summary highlights verified instances of evidence impact.

We provide funding for replications, conduct in-house replication research and publish guidance on replication methodology. We also provide funding to original authors of 3ie-funded for preparing their raw datasets.

As part of our mandate as a knowledge producer and translator for our main audiences, we publish a range of knowledge products. These include briefs, impact evaluation reports, systematic review reports and summaries, replication papers, evidence gap map reports, scoping reports and working papers.

3ie’s Registry for International Development Impact Evaluations (RIDIE) aims to enhance the transparency and quality of impact evaluation research before it begins.
Our work
3ie’s evidence programmes support studies to fill critical knowledge gaps in a sector, sub-sector or in an area with limited rigorous evidence. We fund studies under a specific theme or which address a particular question or set of questions in programme areas where our donors want to expand global public knowledge of what works and what does not.

To help address gaps in the understanding of what works and what does not, we fund a variety of studies across this programme area, including interventions focused on insurance, extension, land-use and forestry, and innovation and technology.

3ie supports impact evaluations, systematic reviews and evidence gap maps on education effectiveness that help answer the questions of what works, for whom, why and at what cost.

We fund the production of rigorous evidence on biodiversity and forest conservation programmes, environmental regulations, impact of sustainable fuels, climate change mitigation and adaptation.

3ie is supporting the generation of evidence in areas such as transparency and accountability in natural resource governance. We also fund the production of rigorous evidence on interventions to curb corruption, judicial and civil service reforms, land reforms, public financial management, conflict prevention and peacebuilding, decentralised governance and public service delivery.

We fund the production of rigorous evidence on interventions on HIV and AIDS, immunisation maternal and child health, nutrition and sexual and reproductive health through a number of evidence programmes. Evidence products from these programmes include replication studies, evidence gap maps, systematic reviews and impact evaluations.

We are supporting the generation of rigorous evidence in humanitarian contexts on interventions related to water, sanitation and hygiene, food security, multi-sectoral humanitarian programming and interventions targeting malnutrition.

We fund the production of rigorous evidence on the socio-economic and environmental impacts of public transportation by rail, bus and rapid transit systems, and essential services such as electricity and gas to expand access, foster inclusive growth, and combat climate change through sustainable systems.

3ie, in collaboration with India’s rural development ministry is working to generate rigorous evidence on the impact of the National Rural Livelihoods Mission.

We support policy-relevant studies that contribute to improving our understanding of public expenditure trends and improving the delivery of public goods.

We support impact evaluations to build the evidence base on the effectiveness of interventions that reduce the risks faced by the poor through participation in public works and employment programmes.

3ie’s has two major evidence programmes that support the generation and use of high-quality evidence for informing decision-making in the water, sanitation and hygiene sector

In alignment with our mission, 3ie promotes rigorous, efficient, and ethical use of innovative data sources for impact evaluations, including in those conducted by 3ie, by 3ie research partners, and in the global development community more broadly.

.

We work with various departments in the Philippines government to develop and fund rigorous evidence useful for policymakers. We also support capacity-building activities for Philippine researchers and support the impact evaluation management framework of the National Economic and Development Authority.

Working in collaboration with the Office of the Prime Minister, the primary aim is to improve developmental outcomes through evidence-informed decision making in Uganda. 3ie is currently supporting evaluation of government programmes around youth livelihood, family planning, public service delivery and local governance, and universal primary education.

3ie and the government of Benin are working on a a multi-year regional initiative that aims to promote the institutionalization of evaluation in government systems across eight countries in West Africa, including: Benin, Burkina Faso, Côte d’Ivoire, Guinea-Bissau, Mali, Niger, Senegal and Togo.

This is a five-year programme (2019-23) that is working on increasing the use of evidence by policymakers in Ghana, Pakistan and Uganda. In partnership with country governments, this programme aims to develop capacity and promote innovation in increasing evidence-informed decision-making. SEDI is funded by the UK Foreign, Commonwealth and Development Office.

.

3ie funds internal replications of influential or innovative impact evaluations of financial interventions on mobile money, cash transfers, bank deposits, and other financial service interventions targeted towards underserved and unbanked populations in developing countries.

3ie funds internal replications of influential or innovative impact evaluations of biomedical, behavioural, social, and structural HIV prevention and treatment interventions to improve the evidence base in low- and middle-income countries.
Funding
- Open opportunities
View our current funding opportunities for evaluations, systematic reviews and internal replication studies.

Publication Type: Other evaluations

Country: Ghana

3ie evidence programme: Agricultural Insurance Evidence Programme
Author(s): Kate Ambler, Alan de Brauw, Nicole Gargano, Mike Murphy, Usamatu Salifu
Institutional affiliation(s): International Food Policy Research Institute, International Food Policy Research Institute, Innovations for Poverty Action, International Food Policy Research Institute, Innovations for Poverty Action
Grant-holding institution: International Fund for Agricultural Development (IFAD)
Main implementing agency: Ghana Agricultural Sector Investment Programme (GASIP)
Sex disaggregation: Yes
Gender analysis: No
Equity focus: Yes, Heterogeneity Analysis
Study type: Formative evaluation

Context

While conservation agriculture (CA) provides public benefits, farmers adopting CA do not usually reap immediate private benefits. Since it takes time for organic matter to build up in the soil, productivity gains are not immediately realised. Farmers adopting CA, therefore, typically face an immediate cost for a potentially uncertain future benefit; hence, adoption rates in Africa have been low.

Given that there is a public good component to CA and potential private benefits to adopters in the medium-term, one method of reducing farmers’ perceived risk of CA techniques is to provide incentives for adopting CA. Within this context, Innovations for Poverty Action (IPA) and International Food Policy Research Institute (IFPRI) collaborated with The Ghana Agricultural Sector Investment Programme (GASIP), a programme within Ghana’s Ministry of Food and Agriculture (MoFA), to design and implement a framed field experiment (FFE) that incentivised farmers to take up CA practices.

Intervention design

The goal is to promote CA by educating farmers on how to best implement CA practices and encouraging the uptake of CA techniques. This study used an FFE to test whether incentives or information about peers affects the adoption of minimal soil disturbance (MSD).

In the FFE, farmers made decisions about adopting MSD over ten rounds representing agricultural seasons. Each ‘season’ was framed in three stages: a first stage in which farmers decided whether to adopt conventional practices or MSD for that season (with MSD requiring an associated cost for weeding); a second stage in which they received a random draw that determined whether the rainfall for that season was ‘normal’ or ‘poor’ and a third stage in which they received a payout for their ‘harvest’ for that season. The rainfall probabilities were designed to reflect conditions in the study areas, and the payouts were structured to reflect that adopting CA has short-term costs but long-term payoffs if adopted for multiple seasons consecutively.

For each round, farmers were also given information about the adoption decisions of a hypothetical peer in that ‘season’, and about that peer’s yields for that season.

Theory of change

The theory of change behind this incentive-based intervention to catalyse CA adoption followed two main mechanisms: (i) incentives could act as a nudge to access available information, which will improve knowledge of CA techniques and reduce uncertainty surrounding adoption and (ii) incentives may provide the financial compensation that will allow the financial benefits of transitioning to CA agriculture to outweigh its cost.

The peer-observation component posits that farmers will be more likely to adopt CA if others in their peer group are doing so, especially if farmers can observe positive results enjoyed by peers who adopt CA.

Incentives, peer observation, or their combination may lead to greater rates of CA adoption.

Evaluation design and methodology

A small randomised experiment was conducted in late 2019 and early 2020 to assess the effects of incentives and peer information on CA adoption. The experiment is ‘framed’ in such a way that participants choose between different options explicitly labelled as agricultural practices, rather than abstract choices.

The first stage of the research was a qualitative study to understand the feasibility of providing incentives for adopting CA techniques and to inform the design of a baseline survey instrument for an eventual impact evaluation of the GASIP programme once it is implemented. The research was conducted in districts GASIP was targeting and in villages where farmers had some familiarity with CA from previous programmes.

The study sample for the baseline survey and the FFE included FBO members in the 66 communities who were direct beneficiaries of the GASIP programme in 2018 or targeted to become GASIP beneficiaries in 2019. The sample design anticipated 20 beneficiary farmers in each community. Ultimately, 1,324 individuals from the sample frame of 1,328 completed the experiment.

Primary evaluation questions

The focus of the study was to evaluate the best means of promoting adoption of the CA technique. The study also looked into factors helping or hindering adoption in the long term. Formative evaluations are crucial for proper design of large impact evaluations for future.

There are two main research hypotheses. First, if farmers receive incentives for adopting CA techniques promoted through extension agents, their adoption of CA techniques will increase. Second, if farmers observe farmers in their community or surrounding area practicing CA techniques, they will become more likely to also adopt CA.

Primary findings

Farmers who were exposed to the incentives in the FFE were 7.6 per cent more likely to choose MSD and around 8 per cent more likely to achieve production gains built into the experiment.
There was a continued higher take up of CA even after incentives were withdrawn. This suggests that targeting incentives over a fixed period could help farmers in adopting CA practices, as they cross hurdles of costs and uncertainty associated with adoption. MSD adoption was particularly notable and was sustained.
A positive effect was found among treated individuals who were told that a peer had successfully adopted CA over a long time period, suggesting that, for example, a demonstration plot might be effective if maintained for a long time period.
The focus groups also suggested that incentives could improve CA adoption. When specifically asked about incentives in focus groups, farmers generally agreed that incentives would increase the likelihood of them giving CA a chance. Farmers cited fertiliser and herbicide as potentially strong incentives. Durable goods like hand seeders or wheelbarrows were also commonly cited as incentives that would be effective, although the authors note that it is unclear how a one-time transfer would incentivise maintaining adoption over a sustained period.