Goldfinger's pan-man-scan: Available at a hospital near you

As my regular readers know all too well, I think a lot about our proneness to cognitive biases and how they impact our healthcare choices. I have been reading Sam Harris's book The Moral Landscape (Free Press, 2010), where he tries to lay a scientific foundation behind our values and beliefs. It is rather a dense book, full of linguistic convolution and intellectual contortions. And although I do not agree with Harris on some of his points (I think he would say that my disagreement stems more from my preconceived notions and values than from the weakness of his arguments), there are some very enlightening ideas in the book.

On page 123 he discusses bias:

...we know that people often acquire their beliefs about the world for reasons that are more emotional and social than strictly cognitive. Wishful thinking, self-serving bias, in-group loyalties, and frank self-deception can lead to monstrous departures from the norms of rationality. Most beliefs are evaluated against a background of other beliefs and often in the context of an ideology that a person shares with others.

Doesn't this echo my assertion that scientific knowledge acquisition tends to be unidirectional? What occurs to me is that the combination of these emotional and ideological factors along with our cognitive predispositions gangs up to steer us toward doom. Too nihilistic? Well, that's not what I am going for. Bear with me, and I will show you how it is driving us into bankruptcy, particularly where the healthcare system is concerned.

We know a lot about how the human brain works now. Creatures of habit, for the purpose of conserving energy, we develop many mental shortcuts that drive our decisions. Think about them like the impact of talking on your cell phone while driving: most of the time under usual circumstances you can pay enough attention to both driving and talking. But on occasion something unexpected comes along, and you find yourself plowed into the backside of an eighteen-wheeler, your face firmly pressed into an airbag, if you are lucky. So are these mental shortcuts: most of the time they serve us well, or at least don't get us into trouble, while just when we least expect it, the truth ambushes us and we suffer from an error in this habitual way of deciding.

Let's bring it around to medicine. Back in 1978, the dark ages, an intriguing paper was published by Casscells and colleagues in the New England Journal of Medicine. Here is what they did:

We asked 20 house officers, 20 fourth-year medical students and 20 attending physicians, selected in 67 consecutive hallway encounters at four Harvard Medical School teaching hospitals, the following question: "If a test to detect a disease whose prevalence is 1/1000 has a false positive rate of 5 per cent, what is the chance that a person found to have a positive result actually has the disease, assuming that you know nothing about the person's symptoms or signs?"

Here are the results:

Eleven of 60 participants, or 18 per cent, gave the correct answer. These participants included four of 20 fourth-year students, three of 20 residents in internal medicine and four of 20 attending physicians. The most common answer, given by 27, was 95 per cent, with a range of 0.095 to 99 per cent. The average of all answers was 55.9 per cent, a 30-fold overestimation of disease likelihood.

Since we are all experts on the positive predictive value of a test, I hardly have to go through the calculation. Briefly, though, and just to be complete, among 1,000 people, 1 has the disease and additional 50 test falsely positive. The PPV then is 1/51 = 2%. This is the chance that a person found to have a positive result actually has the disease. Yet nearly 1/2 of the responders said 95%. What did you estimate? Be honest. And this was Harvard! What chance do the rest of us, mere mortals, stand before such formidable cognitive traps? We might as well throw in the towel and prepare for an onslaught of tail-chasing physicians and patients spinning their gears in pursuit of false positive results. Because once you let a positive finding out of the bag... Well, you get the picture. I would wager that this is at least in part what has hijacked our healthcare systems, its finances and our reason. After all, as Sam Harris pointed out, it doesn't take much for us to form beliefs: no rational thinking required. How can I be so sure, you ask?

Casscells and his team in their paper bring up a phrase "the pan-man scan." They use it to refer to a battery of 25 screening tests and the chances that all of them come back normal if the person is perfectly healthy. Care to take a guess? Twenty-eight per cent. That's 28%! This means that out of 10 patients who walk into the office healthy, 7 will walk out with an abnormal finding and either be assigned a disease or be sent for further testing, even if it is only a repeat of the previously abnormal test. Now think back to your last "routine physical." Did you get "routine" blood work, cholesterol test, urinalysis? How many separate values were run? What was your risk of having an abnormal value and what ensued when you did?

The point here is once again to think about the pre-test probability of the disease and the characteristics of the test. Without this information there is no way to make an informed decision about a). whether or not to bother getting the test, and b). what to make of the result. But how do you rewire clinicians' and the public's brains to get out of these value-laden habitual errors that drive us to all kinds of bad behaviors?

Well, a psychologist from Northwestern University in Chicago by the name of Bodenhausen studies the human brain's proclivity for stereotyping. How is this relevant to medical decision making? Well, in medicine we tend to think in stereotypes. What I mean is that when a middle-aged male smoker with diabetes and hypercholestrolemia comes to the ED with a crushing substernal chest pain, we invoke the stereotype of an acute MI patient, and in this case such stereotyping allows us to make some good choices very rapidly. Where we fall into a trap is in the example above, where we stereotype a patient with a positive screening test into a group with the disease. While it is natural for our habit-addicted brains to do so, it lands us in hot water, individually and as a society. Well, Bodenhausen in this paper (subscription required) in the journal Medical Decision Making states plainly that there are certain circumstances that predispose us to stereotyping: complex decisions, cognitive overload, and happy, angry and anxious moods. Do you recognize any of these risk factors for stereotyping in medicine? I thought so.

So, what do we do? Here are a 5 potential solutions to consider:
1. Go back and think some more.
Metacognition training early and often would be a great addition to the medical curriculum.
2. Get our medical appointment out of the microwave
Give the clinician more cognitive time. Not possible, you say? Pity! That is the simplest answer.
3. Educate the public about these cognitive thought-traps and rational approaches to making medical decisions.
4. If we are so committed to the proliferation of medical stuff, then we really need better HIT infrastructure. We need bedside decision aids that automatically calculate the patient's probability of having a disease given their pre-test probability, as well as posterior probability adding the test characteristics into the mix. Having these data before ordering tests might reduce some of the waste and tail-chasing. It may also decrease some of the harm that comes with over-utilization in our dogged pursuit of a definitive diagnosis.
5. Finally, the cost-benefit equation of innovations in medicine must begin to incorporate these cost to cognition. They may be higher than we have admitted so far.

 

Evidence of harm

We alluded recently to some of the perils of our healthcare system -- the errors, the unnecessary deaths, healthcare-associated infections, and even the idea that we have not done a good job accounting all of the harm that goes with our interventions. It occurred to me that, since you are not immersed in this stuff all day every day the way that I am, it might be helpful actually to put the issue in perspective through transparent data. Mind you, these data are not easy to gather or clean or interpret. So, the point estimates that are widely cited may be somewhat off. But think about it: even the problem is only one-half the magnitude reported, it is still mammoth!  


When we search the Centers for Disease Control and Prevention web site for latest statistics on mortality, we get to this page, where the familiar numbers confront us. These numbers do not change drastically year after year, and indicate that in 2007 the top four cause of death in the US remained cardiovascular disease (616,067), cancer (562,875), stroke (135,952), and lung disease (127,924). 

Here is something shocking, though: in the CDC's hierarchical accounting of the causes of death in the US, one of top three causes is incorrect. And this baffles me because for a decade we have known the third leading cause of death in the US to be something other than stroke. The cause is what I call fulminant iatrogenesis. What is fulminant iatrogenesis? Well, in Greek, the word for "doctor" is "iatre". In our medicalese we have adopted this root to describe complications arising from healthcare as "iatrogenic". We all know that there are adverse events, mistaken identities and such, and we even have an idea that there are somewhere in the range of 50,000-100,000 deaths attributed to hospital care itself, thanks to a well publicized report from the Institute of Medicine from 1999 with a memorable title of "To Err is Human". And since then we have heard echoes of other rather large numbers of deaths due to hospital-acquired infections, also somewhere in the vicinity of 100,000. But details and holistic picture of these numbers, whether there are there overlaps or are they really as outrageously high as they seem, have not been clarified well by the popular press. But is has been addressed in the medical literature, a decade ago. And while we continue to hear a constant din of hoofbeats of the safety police, it is still questionable what effect they have had.           

According to a paper that can be considered vintage at this point, appearing in JAMA in 2000 by Barbara Starfield, MD, MPH, from Johns Hopkins University in Baltimore, our US healthcare system is not so healthy. An accounting of deaths that can be attributed to various forms of contact with the healthcare system is shocking:  

Iatrogenic deaths/year in US hospitals

12,000: unnecessary surgery

7,000: medication errors in hospitals

20,000: other errors in hospital

80,000: nosocomial infections in hospitals

106,000: non-error medication adverse events

These numbers add up to 225,000 deaths/year from iatrogenic causes, and this broadly represents the third leading cause of death in the US after cardiovascular disease and cancer, and accounting for twice as many deaths as the CDC-listed 3rd cause, cerebrovascular disease. And please note that these numbers are actually far lower than those presented in the 1999 IOM for hospital deaths related to errors (48,000 -- 99,000), and at least a bit lower than the most recent CDC estimates for deaths from hospital-acquired infections (99,000). I call this "fulminant iatrogenesis". According to Starfield, what the above sum total of 225,000 fails to take into account is the additional 199,000 deaths related to adverse effects of outpatient treatment. So, putting these together, there are 424,000 iatrogenic deaths in the US annually. Now, this is something to shout about! We know that in the US we do not do a very good job following evidence, even when there is such available, and even when it is relatively solid. So, with all of the caveats and care that I regularly discuss on this site, we really need to do a better job streamlining processes and implementing best available evidence as appropriate. This frightening number of nearly 1/2 million deaths due to contact with the healthcare system needs to become branded onto our national psyche. What a travesty that in a nation that outspends every other in healthcare, that has possibly the worst access to care of all developed nations, we should have such a staggering number of potentially preventable deaths from iatrogenic causes.  How many lives does this kind of system have to be responsible for saving in order to balance this risk-benefit equation?   

Another premier cause of death hidden in the CDC numbers is smoking. In fact, back in the late 1990s the estimate was that the annual death toll in the US from smoking-related causes is 440,000. Of course this is not on top of the causes of death listed, but overlaps with many of the proximal causes listed. Add to this our growing obesity epidemic, and the new data that by 2050 we can expect 1 in 3 Americans to have diabetes mellitus, and you have yourself a tidy public health catastrophe. 

Now, some people may consider this to be a "tu quoque" logical fallacy. Or that this is a straw man. They may say that we are already applying evidence-based methods to address this monster. And they will, of course, be correct, to an extent. The methods that we are applying are not always evidence-based and the incentives system growing up around these measures is far from the latest and greatest in what social sciences tell about human nature and successful behavior change. Most importantly, 10 years later the evidence indicates that we have not moved this hospital death meter, not one little bit. 

So, my response to criticism of my logic is: yeah, whatever. A straw man this massive could eat Tokyo. To me these are startling numbers. We need everyone committed to science and evidence to participate in finding solutions, and constantly reflect on the medical errors that are so pervasive and so deadly. If you pay attention to the evidence of harm, there is no escaping the conclusion that our house is indeed made of glass.