Measurement of latrine use in rural India

Study type: Other evaluations
Country: India

3ie evidence programme: Promoting Latrine Use in Rural India
Authors: Sujatha Srinivasan, Divya Mary, Ajaykumar Tannirkulam
Institutional affiliations: IFMR LEAD, Krea University, Chennai, India
Grant-holding institution: Institute for Financial Management and Research
Main implementing agency: Not applicable

Sex disaggregation: Yes
Gender analysis: Yes
Equity focus: No

Context

The Sustainable Development Goals made ending open defecation a priority. To this end, the Indian government has led a massive sanitation programme, Swachh Bharat Abhiyan - Gramin, to improve latrine access and use. However, there is growing awareness that access and use are not the same. 3ie supported the impact evaluations of four behaviour science-informed interventions to promote latrine use among latrine owning households in four states in rural India – Bihar, Odisha, Karnataka and Gujarat. These studies also identified local barriers to latrine use. However, to understand the effectiveness of efforts to support latrine use, the behaviour must be accurately measured. The measurement of latrine use has been challenging as it is prone to inaccuracy due to social desirability and recall bias. Therefore, 3ie commissioned a fifth study, termed the measurement study, to compare reported rates of latrine use between an individual-level, balanced question and a household-level imbalanced question.  The study was carried out amongst a sub-sample in the four impact evaluation study areas.

Primary evaluation questions

1. Is there a difference in reported latrine use rates when an individual-level or a household-level question is used?
2. Was there a difference in the change in reported rates of latrine use in intervention vs control sites?

Evaluation design and methodology

The measurement team interviewed 10-30 households in 22-26 villages in each of the four states, before the interventions were implemented and approximately one year later. At baseline, households were randomly asked one of two questions to measure latrine use:

1. ‘I have seen that some people defecate in the open, and some people use the latrine. Now I want to ask about where you and your family members defecate. The last time [NAME] defecated, did [NAME] defecate in the open or use the latrine?’ Answer codes were latrine, open and somewhere else (potty, nappy, etc.)
2. ‘What kind of toilet facility do members of your household usually use?’ Answer codes were the same as in the National Family Health Survey (NFHS) and did not require a household roster. These include flush to piped sewer system, flush to septic tank, flush to pit latrine, flush to somewhere else, flush to don’t know where, ventilated improved pit or biogas latrine, pit latrine with slab, pit latrine without slab or open pit, twin pit or composting toilet, dry toilet, and no facility or uses open space or field.

At baseline and endline, latrines were inspected for signs of use such as the storage of other items, the pan being sealed, and slippers being present. Basic demographic data such as gender, age, education and asset ownership were collected at baseline and endline. Surveys also included indirect questions meant to elicit truthful responses to the sensitive questions of latrine use and open defecation behaviours. Only the individual-level question was asked at endline. Surveys were adapted somewhat between states. They were translated into local languages. Due to seasonality, two additional questions were added in the baseline Gujarat survey regarding latrine use during non-monsoon seasons.

Baseline data from households which received the individual- or household-level questionnaire were first compared descriptively using chi-square and t-tests to establish balance in the sample distributions. Differences in reported latrine use between the two question types were also examined using a t-test. Further descriptive analysis was done at both baseline and endline to determine patterns in latrine use by demographic traits. A difference-in-difference analysis was conducted to examine changes in reported latrine use among households that responded to the individual-level questionnaire at both baseline and endline. Standard errors were adjusted to account for village level clustering.

Primary findings

Households asked the individual- and household-level questionnaire were similar at baseline. Based on latrine observations, 92% of the existing latrines were in use in Bihar and 87% were in use in Gujarat; however, only 65% and 70% were in use in Karnataka and Odisha, respectively. While these values were similar for Bihar (90%) and Odisha (70%) at endline, the proportion of latrines that appeared to be in use dropped considerably in Gujarat (79%) and increased in Karnataka (71%).

At baseline, reported open defecation was 20 percentage points lower in response to the household-level questionnaire than the individual-level questionnaire. The lowest rates of open defecation reported through the individual-level questionnaire were observed in Bihar, with 15% of the population reporting open defecation. In contrast, in Karnataka, 56% of the population reported open defecation in response to the individual-level question. However, these values were 3% and 19% according to the household-level questionnaires. At endline, only 29% of the sample reported open defecation. This ranged from 15% in Bihar to 46% in Karnataka. The adjusted difference-in-difference estimates show that reported latrine use increased more in the intervention that control sites in Bihar. However, there was no statistically significant effect of the interventions in the other three states.

Implications

This analysis demonstrates the effect that framing can have on reported rates of latrine use. Reported rates of use vary considerably by the type of question asked to measure use. Therefore, when measuring latrine use, either for evaluation or monitoring purposes, both researchers and policymakers must carefully consider the framing of their questions. It may be necessary to further develop tools that can be used to rapidly, accurately, and non-intrusively measure latrine use behaviours.

This analysis also provides an evaluation of the four behaviour science-informed interventions to support latrine use. Authors here find that the intervention in Bihar successfully improved latrine use behaviour, but the other interventions did not have significant effects on latrine use. These results are not consistent with the results of the analyses presented by the individual study teams.

The teams from Odisha and Karnataka found that the intervention meaningfully improved latrine use. The team from Gujarat reports small, but statistically significant, improvements in latrine use associated with the intervention as well. However, the Bihar team reported no effect of the intervention. Despite using the same individual-level question to measure latrine use as the impact evaluations, there are some differences in the study designs which may account for these differences. The samples were not the same in the analyses considered by the individual study teams and by the measurement team. Furthermore, the statistical methods used to estimate changes in latrine use were different across each of the studies.