You must not fool yourself, and you are the easiest person to fool.
Richard Feynman
What works and what doesn’t? What’s true and what isn’t? This is a surprisingly hard question to answer. Consider drug discovery, the field I work in. Most drugs that pharmaceutical companies start development on turn out not to work. That’s not because big pharma is bad at developing drugs, but because it’s hard to find any therapeutic that works. This includes drugs for which common sense, and all of the evidence, suggested that they should work.
Last week, the pharmaceutical company Novo Nordisk published the results of a large clinical trial for Ziltivekimab in patients with cardiovascular disease. Ziltivekimab inhibits the protein IL-6. People who are born with genetic variants that result in IL-6 activity have less inflammation, which in turn leads to less cardiovascular disease. All of this is well understood, and most people, including myself, thought that replicating the effect of the genetic variants with Ziltivekimab would have the same protective effect. Yet in Novo Nordisk’s clinical trial, Ziltivekimab turned out not to be more effective than placebo. The jury is still out about what exactly went wrong.
There is no way around trials to determine what works and what doesn’t. Common sense, or even studying the problem very closely, is a poor substitute. That’s not just true for pharmaceuticals, but also for any other intervention, including interventions meant to modify behaviors. Adam Mastroianni on Experimental History has a list of examples of interventions that look like they should work, but don’t. Mastroianni goes on to praise epistemic humility, an underappreciated concept despite regular admonitions by the world’s greatest minds to remain humble.
It’s important to stop at humility, and not to confuse it with nihilism or despair. There are things that work. There are many drugs that are efficacious, and there are social interventions that really work. Fortunately, science, when applied correctly, gives us the tools to make that call.
