In order to find out what difference a program per se makes, you need to know what would have happened if the program had not existed (this is called the counterfactual). Hence, the methodology of the randomized controlled trial (RCT). In RCTs, you take a group of people and select half by lottery to receive a program, while the other half serves as a comparison group.

RCTs can only be used in situations where the provision of a particular good or service can be randomized. And this requirement rules out a lot of important ways of helping people that are worth thinking about.

RCTs are really good at answering questions of causality: X causes Y. They are also expensive, take a long time, and require randomization.

  1. An RCT cannot measure the impact of an RCT itself.
  2. An RCT cannot readily measure the impact of R&D type work.
  3. Finally, an RCT cannot measure value.

Read the full article on randomized controlled trials by Matt Spitzer at Medium