Giving Compass’ Take:
• Urban Institute examines effective practices of impact measurement and why repeated evaluations may be necessary, especially if a program is being tried in different contexts.
• This is a good reminder that the hard, important work of evaluation is never really over. In what ways can we move towards better data-gathering methods?
Strong evidence on which social programs work is crucial for effective policymaking, as it ensures that limited government resources go to programs that deliver results. Impact evaluations, which use randomized controlled trials, or quasi-experimental designs, contribute to this evidence base. But what if an untested social policy or program appears so intuitive and has a theory of change so compelling that government officials are highly confident in its ability to deliver the promised outcomes? Is there still a benefit in doing a rigorous evaluation?
The answer is yes. Many social policies and programs were assumed to be effective but were eventually proven not to be. Impact evaluations show us which programs to invest in and which programs not to invest in.
The Cambridge-Somerville Youth Study is one example of this unexpected result. Initiated in the late 1930s, this longitudinal study has been recognized as the first randomized evaluation of a social program. The program it studied — mentoring for at-risk boys and young men from low-income backgrounds — appeared to have obvious and intuitive merit. Mentees remembered their experiences fondly, and many believed the program improved their lives.
But repeated analyses, comparing outcomes for those who were mentored with those who were not, published in 1948, 1959, and 1978, suggested the program was not effective. In fact, the 1978 study found that on several measures — including alcoholism, multiple criminal offenses, and stress-related diseases — the treatment group fared worse than the control group. Compounding these results, the study found that the longer someone was in the intervention program, the worse the outcomes they experienced. It was stunning but compelling evidence that this social program, which had appeared promising, was actually detrimental.
These stories underscore the importance of evaluating programs that are new or are being applied in new ways. But what if a program already has a strong evidence base? Is another impact evaluation necessary?
Read the full article on social program impact evaluations by Will Schupmann and Matthew Eldridge at Urban Institute.
Since you are interested in Impact Philanthropy, have you read these selections from Giving Compass related to impact giving and Impact Philanthropy?
Looking for a way to get involved?
If you are looking for opportunities to learn and connect with others interested in the topic of Impact Philanthropy, take a look at these events, galas, conferences and volunteering opportunities aggregated by Giving Compass.
Are you ready to give?
Impact Philanthropy is an important topic. Other members found these Giving Funds, Charitable Organizations and Projects aggregated by Giving Compass to be relevant to individuals with a passion for Impact Philanthropy.