Giving Compass’ Take:
• Mary Beech explores the concept of external validity: When organizations examine other organizations’ successes as guidance for programs.
• How strong are the measurement tools we use in general? How much can Randomized Controlled Trials (RTCs) make a difference in evaluating impact?
• Read more about the role of Randomized Control Trials in social progress.
When assessing program impact, researchers often focus on evaluating the causal relationship between two variables and the accuracy of those results. This concept is called “internal validity,” which asks the question: What’s the likelihood that this relationship actually exists? Another question equally important to policymakers is whether the observed results can be generalized to other settings, which is referred to as external validity. In other words, if we were to change characteristics such as time, setting, or population, would the causal relationship still exist?
There are a number of scenarios in which the external validity question is relevant. Take, for example, a county that conducted a rigorous five-year randomized control trial (RCT) to determine the impact of a family housing program. The evaluation found that the program improves adult and child well-being.
Based on this finding, the government wants to
- Continue to fund the program for the next five years
- Scale the program to new neighborhoods
- Select additional service providers to deliver the same program model
- Expand the program to serve homeless veterans in addition to families
As the social sector continues to identify and scale promising program models, we must think critically about how to best generalize and apply learnings. Exclusive use of RCTs is not scalable as programs expand. Continual monitoring, feedback, and learning builds on the prior experimentation and can enhance external validity as a program scales.
Read the full article on external validity by Mary Beech at Third Sector Capital Partners.
If you are looking for more articles and resources for Impact Philanthropy, take a look at these Giving Compass selections related to impact giving and Impact Philanthropy.
How will the original finding translate to these new settings? Given funding constraints, it’s unrealistic to think that we can conduct a full randomized control trial (RCT) evaluation for each program in every community.
Looking for a way to get involved?
If you are looking for opportunities to learn and connect with others interested in the topic of Impact Philanthropy, take a look at these events, galas, conferences and volunteering opportunities aggregated by Giving Compass.
Are you ready to give?
If you are interested in Impact Philanthropy, please see these relevant Issue Funds, Charitable Organizations or Projects where you can get involved.