Geospatial impact evaluations offer deeper insights but also come with their unique set of challenges. As part of the GeoField community of practice, we are finding ways to unpack and address these by collaborating with individuals and organizations working across sectors. To that end, 3ie, AidData, and DevGlobal recently co-hosted a workshop with GIS experts to address key issues and best practices in response to technical questions received from participants before and during the session.
Opportunities and complexities
Increased availability of and access to high-frequency, granular geospatial data has allowed us to expand on insights from traditional evaluations and conduct them more frequently, efficiently, and cost-effectively. This has been particularly crucial in cases where data previously did not exist or is difficult to obtain through traditional data collection methods. At the same time, geospatial data can be misinterpreted or misrepresented in many ways. For example, NDVI, a common proxy indicator for vegetation cover and crop yield, is not always the best measure for these outcomes, and the most relevant proxy indicator may change based on the context (e.g., SAVI is more sensitive to vegetation cover in dry regions).
Similarly, feeding training data from one region into a Machine Learning (ML) model to classify land use in an entirely different region is likely to be inaccurate and misleading. While some aspects of these technical issues are relatively clear-cut to address, many are less obvious. For instance, we can check how well our ML models perform in different contexts, particularly when incorporating context-specific on-the-ground data for validation, but what level of accuracy is “good enough”?
Another aspect that makes geospatial evaluations particularly challenging is that they are by nature interdisciplinary and require a strong technical understanding of not only econometrics and statistics, but also geospatial analysis and the specific sector and context. Therefore, while we may know how to deal with an issue within one discipline, it is likely approached differently in another, and the overlap is not immediately clear. For example, spatial autocorrelation in econometrics is typically addressed by clustering standard errors. However, with extremely granular environmental data without clear administrative or social boundaries on which to cluster data, how should we address any existing autocorrelation?
Addressing key challenges
During the recent Geospatial Technical Workshop, experts from various fields convened virtually to discuss and provide one another with feedback on practical technical issues faced in implementing their geospatial impact evaluations in multiple sectors.
Technical questions were submitted and shared before the event, and participants had the opportunity to jump into the conversation to suggest ways forward during Q&A and breakout sessions. The topics covered included best practices, approaches, and ideas for:
- Dealing with spatial autocorrelation when we have very granular data and no clear level of aggregation (e.g., administrative boundaries).
- Exporting pixel-level data of a specific resolution, and accounting for cloud cover.
- Accurately predicting rice yields using ML.
- Utilizing advanced techniques (e.g., hyperspectral imaging and Unmanned Aerial Vehicle imagery)
- Defining treatment and control areas, using buffers.
The shared commitment and foresight to engage with the questions demonstrate that shying away from these issues is not an option. While there were no clear-cut answers to many questions, the session takeaways (read the discussion summary here) included some new techniques to try, a better understanding of the gaps in our collective knowledge, and better connections with the individuals and organizations that are eager to push the technical frontier—and learn and advance together.
Asking tough questions and getting answers in this format is not easy, but the value of bringing a wide range of experts and policymakers into one room to understand and tackle them is immense. Keep an eye out for future events and workshops where we can reconnect, or feel free to contact info@3ieimpact.org to discuss or propose your ideas for an event.
