Whether you are a research funder, decision maker or researcher, keeping up with the ever expanding evidence base is not easy. Over 2600 impact evaluations and 300 systematic reviews assessing the effects of international development interventions have been completed or are ongoing to help answer that question and understand how, why and at what cost.  Despite this increase in quality evidence, more evidence is needed, which is why funders and researchers continue to fund and produce new research.

“The 3ie replication process differs in important ways from the standard research community-led peer-review process in academic journals. We have been explicitly instructed by 3ie staff not to discuss our experiences with the replication process at any length in this note, including our views on the weaknesses of their current system and the review standards they employ.

I delivered a talk at 3ie’s Delhi Seminar Series on a recently published PLoS ONE paper) and follow-up research. This project was a randomised experiment evaluating the potential for text messages to remind malaria patients to complete their treatment course of antimalarial medication.

Suppose you toss a coin and it comes up heads. Do you conclude that it is a double-headed coin? No, you don’t. Suppose it comes up heads twice, and then a third time. Do you now conclude the coin is double-headed? Again, no you don’t. There is a one in eight chance (12.5 per cent) that a coin will come up heads three times in a row. So, though it is not that likely, it can and does happen.