International Initiative for Impact Evaluation

Q&As with 3ie Executive Director, Howard White

The Centre for Global Governance interview on his nomination
http://www.cgdev.org/content/general/detail/15102

One year on

Q: What do you see as 3ie's biggest achievements in this first year?

The Cairo impact evaluation conference. This was a major international event with 700 participants, which brought together a wide-range of views on impact evaluation. 3ie was one of the leadorganizers.

3ie has also made substantial progress with other areas of our work. We have issued two calls for proposals, as a result of which we are now financing seven synthetic reviews and 18 primary studies. The Journal of Development Effectiveness has been launched, as has the website, which we are now updating on a regular basis.

Since January 3ie has been operating out of GDN’s Delhi office and we now have in place an excellent core staff to run the program.

Q: What have been the main challenges you've faced in setting-up the grant program?

What is most significant is the challenge we have not faced.

A major worry I had was that there would not be enough high quality proposals for our grant program. But, we received 78 proposals in response to the first call under the Open Window, the vast majority of which were excellent studies. Some of these innovative research projects are evaluating:
- the use of mobile phones to help mornitor patients' compliance to Tuberculosis treatment in Karachi;
- the distribution of cooking oil to compensate for dowry to delay adolescent marriage in remote parts of Bangladesh;
- early childhood development centres in Mozambique;
- a community driven development pilot in post-conflict Sierra Leone;
- a slum-housing upgrading in Peru.
I was sorry we could not finance more of them than we did.

But, there have been challenges, the most significant of which has been the time taken to achieve start up – getting the office set up and staff in place, so that we can get on with our work of supporting the production of better evidence of what works in development and why.

Q: How many evaluations do you expect to fund in the first year of operation and over the next three years?

Each call for proposals for new impact evaluation studies will finance 15-20 studies. During 2009 we hope to make four calls for proposals, though that number is still under review. I anticipate that we will make awards for at least 40 new studies this year, but it may exceed 70.

From 2010 on, 3ie will be issuing three calls for proposals a year, and so financing somewhere between 45 and 60 new studies. So after the first four years, 3ie will have funded around 200 new studies.

Q: Can you give an example of the kind of impact studies 3ie is likely to fund?

3ie’s mandate is to finance impact studies of social and economic development interventions in low and middle income countries. The studies we finance have to satisfy several additional criteria. The intervention being evaluated should be important – meaning either that it affects a lot of people or that it has a big impact on those it does affect. The topic should be one that is of interest to policy makers. And the study design has to be rigorous.

Examples from our first round include a program to increase the age of marriage in Bangladesh, a daycare program in Mexico, and scholarships for Ghanaian secondary school students.

Q: How will 3ie grant program be evaluated and when?

3ie’s Board has responsibility for oversight of 3ie’s operations, which includes commissioning independent evaluations of our work. It is envisaged that a comprehensive evaluation will be undertaken every 3-5 years.

In the meantime, we will be monitoring our progress against a set of process indicators, which will be published in annual report and evaluation will be publicly available.