Beyond Belief: How Evidence Shows What Really Works

by Helen Pearson

Synopsis:

The remarkable story of the global movement championing the idea that evidence, not opinions, should guide our decisions.

Today, more and more people around the globe are using scientific evidence to figure out what works—in health, government and business as well as conservation, schools and parenting. This wasn’t always the case. This book tells the story of the evidence revolution—a worldwide movement that promotes evidence-based thinking—and shows how it can help us all, especially in an age of alternative facts.

For many years, most medical advice was based on doctors’ opinions and conventional wisdom, not solid science. Helen Pearson describes how evidence-based medicine swept the world in the 1990s—becoming the predominant form of medicine practiced today—and how the idea that evidence should guide decisions is quietly transforming a host of other fields as well. Do police patrols reduce crime? Do performance appraisals boost job performance? Do welfare programs help the poor? Do smaller classes aid learning? Do smartphones harm teenagers? At a time when science is under attack and questionable claims run rampant, Pearson underscores the importance of evidence in all facets of our lives, empowering each of us to sift fact from falsehood and misinformation from the truth.

I’ve been meaning to read Helen Pearson’s Beyond Belief for a while, since I saw it reviewed positively in New Scientist; given my studies, I know a fair amount about evidence-based medicine, but I have to admit that my university was pretty down on the use of meta-analyses (we weren’t allowed to reference them) and synthesis (like meta-analyses) get short shrift in academia. Being a synthesiser and not a researcher, I’ve stalled out after completing my master’s in two fields now.

(Which is okay by me in a sense; I love getting a broader and broader base of knowledge. If synthesis had been valued more in my first university, I probably wouldn’t have swapped from English literature to natural sciences, but I think that was good for me — and I’m looking forward to whatever branch of academia claims me next. No regrets here.)

Anyway, to get to the point, Pearson covers the evidence-based “revolution” across various fields. Each time it’s clear that true evidence-based decision making has unfortunately been thin on the ground. We like to think we intuitively use evidence in our decision making and do “what works”, but human minds are rife with bias and we don’t look for much evidence before concluding what works. It’s frightening how new the drive to evidence-based treatments is — it wasn’t a new concept 35 years ago (Semmelweiss, Lind, the streptomycin trials, etc) but it was one that was not applied as much as it really ought to be.

Some of the chapters are about things that aren’t super interesting/relevant to me, but they still help to make it clear how the practice works and how people have implemented it, so I don’t recommend just reading the chapter on evidence-based medicine or evidence-based education and calling it there. The whole book builds up an overall picture that is very useful.

My one quibble is that in a couple of footnotes she recommends the use of an LLM (trained specifically on a narrow corpus of relevant evidence) for finding evidence relevant to planned interventions in particular fields. I have questions about how effective and accurate those LLMs are, and would in fact like to see the evidence — quite apart from having serious qualms about using LLMs at all.

In fairness to her, the chapter about implementing evidence-based practices in your own life does mention that LLMs are not reliable and you should always ultimately read the papers they surface rather than taking the output at face value. I question, though, how well that works over just completing the search yourself and maintaining awareness of the things you do in order to narrow your search and find the information you need. But… again. Show me the evidence. And again, there are serious problems with LLMs in general.

I do actually know a certain amount about evidence-based practices, owing to my science degrees, and have read some of the papers she references and books by the people she discusses, so I don’t think she’s been led astray by an LLM in her own writing, at least, but it’s concerning and spoils the book somewhat.

Rating: 4/5 (“really liked it”)