Imagine that there's a very low probability something's going to work, let's say a perpetual motion machine. We have strong reason to believe that it won't work. Then it doesn't make sense to invest a lot of resources in building the perpetual motion machine on a mass scale and having a factory to build that.

Now let's say it's a somewhat higher probability, so still low but somewhat higher. Let's say there's a new approach to teaching math or language, that there's no evidence for but based on theoretical grounds we think it might be pretty effective. Well, it might not yet be time to implement that at scale because perhaps there are negative effects of doing so. But it probably does make sense to do some sort of staged investment in evidence.

Let's say it's a higher probability. I'm going to put a flossing as an example here because I just saw a newspaper article about it. The newspaper article pointed out, and I know nothing about flossing so let me clarify this, it said that the RCT evidence for this is not strong but dentists continue to recommend it. Well, my guess would be that the dentists, I don't know and I haven't investigated this, but my guess would be that the dentists probably know what they're talking about and have some reason to believe this. So, probably the best thing to do for your health is to continue flossing for now. But, we should probably also collect additional evidence.

But let me talk about something on the other side of that line. Let's say there is evidence for it, enough for a new drug to get past the FDA. Then probably if you have the disease you should consider taking the drug, or take the drug if it's recommended for other medical reasons.

But does that mean we should stop collecting evidence? Almost certainly not.

Read the full transcript on evidence by Michael Kremer at Effective Altruism