What is Giving Compass?
We connect donors to learning resources and ways to support community-led solutions. Learn more about us.
Giving Compass' Take:
• In this 80,000 Hours podcast, economic expert Dr. Eva Vivalt discusses the possible problems with trying to replicate outcomes in global development: The results are often inconclusive.
• Does this mean we can't learn anything from evidence-based aid reforms? Not necessarily, but the date collection and testing must be more precise.
If we have a study on the impact of a social program in a particular place and time, how confident can we be that we’ll get a similar result if we study the same program again somewhere else?
Dr. Eva Vivalt is a lecturer in the Research School of Economics at the Australian National University. She compiled a huge database of impact evaluations in global development — including 15,024 estimates from 635 papers across 20 types of intervention — to help answer this question.
Her finding: not confident at all.
The typical study result differs from the average effect found in similar studies so far by almost 100%. That is to say, if all existing studies of an education program find that it improves test scores by 0.5 standard deviations — the next result is as likely to be negative or greater than 1 standard deviation, as it is to be between 0-1 standard deviations.
She also observed that results from smaller studies conducted by NGOs — often pilot studies — would often look promising. But when governments tried to implement scaled-up versions of those programs, their performance would drop considerably.
Read the full article about the flaws in evidence-based development by Robert Wiblin at 80000hours.org.