How to make safer decisions in medicine

I love when an article I read first thing in the morning gets me to think about itself all through my morning chores and then erupts into a blog post. So it was with this little gem in the statistical publications "Significance." The author suggests making gambling safer by placing realistic odds estimates right on the poker machines in casinos. He even goes through the generation of the odds of winning and losing and how much based on really transparent assumptions. In fact, what he has in effect constructed is a cost-benefit model for the decision to engage in the game of poker on these machines. Seems pretty simple, right? Just a few assumptions about how long the person will play, some objective inputs about the probabilities, and PRESTO, you have a transparent and realistic model of what is probable.

In medicine, there is a discipline known as Medical Decision Making, and what it does is exactly what you see in the "Significance" article: its practitioners construct risk- (and, hence, cost-) benefit models for decisions that we make in medicine. To be sure,these turn out to be rather more complex, since the inputs for them have to come from a large and complete sampling of the clinical literature addressing the risks and the benefits. But that's the meat; the skeleton upon which this meat hangs is a simple decision tree with "if this then that" arguments. In this way these models synthesize everything that we know about a specific course of action and put it together into a number driven by probability.

They usually go something like this. We have a group of women between 40 and 49 years of age with no apparent risk factors for breast cancer. What is the risk-benefit balance for mammography screening in this specific age layer? One way to approach this is to take a hypothetical cohort of 1,000 women who fit this description and put it through a decision tree. The first decision node here is whether to perform a screening or not. What follows are limbs stretching out toward particular outcomes. Obviously, some of these outcomes will be desirable (e.g., saving lives), while some will be undesirable, ranging from worry about false positive results to unnecessary surgery, chemotherapy, radiation, and even death. Because these outcomes are so heterogeneous, we try to convert everything to monetary costs per quality of life (quality because there are outcomes worse than death, as it turns out). But what underlies all of these models is the mathematics derived from clinical studies, not pulled out of thin air. This is the most useful synthesis of the best evidence available.

To be sure, MDM models are rather more complicated than the poker example. They require a little more undivided attention to follow and understand. Furthermore, I personally did not get a whole lot of exposure to them in my training, but perhaps that has changed. Like anything to do with probability, these models tend to be off-putting in a society that has consigned itself to wide-spread innumeracy. And doctors are certainly not immune from misunderstanding probability. Yet without them perceptions rule, and our healthcare becomes a reckless gamble. In our ignorance we collude to build profits that come with medicalizing small deviations from the perceived normality. Sadly, the primary interests that drive these profits are not usually doing so with probabilistic forethought either, but rather on the basis of red hot conviction that they are right.

Doctors and e-patients need to lead a radical transformation in how we handle decisions in healthcare. It is very clear that willful ignorance has not served us well, and we are all too easily led into panic about every pimple. Resilience can only come when we question our assumptions. Alas, our intuitive brain is almost certain to mislead us when faced with complex information; why else would we need explicit odds listed on poker machines? The absurd complexity of information in medicine deserves no less. It's time to start the probability revolution!

If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. 


Thank you for your support!

My solution to the healthcare crisis

Here is my talk from Ignite Boston last night -- I solved the healthcare crisis!

Zilberberg igniteboston2012

View more PowerPoint from murzee

If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. 

Thank you for your support!

The probabiltiy dozen of participatory medicine

Yesterday my rant about uncertainty and probability got quite a bit of play in cyberspace, and I am glad.

Uncertainty is ubiquitous. We consider the odds of rain when choosing what to wear. We do (or at least we should do) a quick mental risk-benefit analysis before buying a burger at Quickie-Mart. We choose our driving routes to work based on the probability of encountering heavy traffic. We do this mental calculus subconsciously but reliably, mostly getting it right. What is odd, though, is that there are certain parts of our lives where we expect complete and utter certainty. I will not get into the political aspects of this fallacy, but I do want to continue down this line of reasoning about healthcare.

As I said yesterday, and many many times in the past, the only certain thing about medicine is uncertainty. And here is what I want you to understand deeply: the amount of uncertainty is much greater than you think. So, every time you say to yourself "I think there is a lot of uncertainty in this information," multiply it by 100, and then you may get close to just how uncertain most information is.

And again, I want to emphasize that this uncertainty gets magnified in the office encounter. So, what is the solution, short of having everyone understand the totality of evidence? Yesterday I said that the solution is to teach probability early and often, and this is indeed the best long-range answer. But is there anything we can do in the short-term? The answer, of course, is yes. And here is what it is.

Everyone needs to learn what questions to ask. Instead of nodding your head vigorously to everything your doctor says, put up your hand and ask how certain s/he is that s/he is on the right track. Here is a dozen questions to help you have this conversation:

1. What are the odds that we have the diagnosis wrong?
2. What are the odds that the test you are ordering will give us the right answer, given the odds of my having the condition that you are testing me for?
3. How are we going to interpret results that are equivocal?
4. What follow-up testing will need to happen if the results are equivocal?
5. What are the implications of further testing in terms of diagnostic certainty and invasiveness of follow-up testing?
6. If I need an invasive test, what are the odds that it will yield a useful diagnosis that will alter my care?
7. If I need an invasive test, what are the odds of an adverse event, such as infection, or even death?
8. What are the odds of missing something deadly if we forgo this diagnostic testing?
9. What are the odds that the treatment you are prescribing for this condition will improve the condition?
10. How much improvement can I expect with this treatment if there is to be improvement?
11. What are the odds that I will have an adverse event related to this treatment? What are the odds of a serious adverse event, such as death?
12. How much will all of this cost in the context of the benefit I am likely to derive from it?

And in the end, you need to understand where these odds are coming from -- the clinician's gut or evidence or both? I prefer it when it integrates both, which, I believe, was the original intent of evidence-based medicine.

Perhaps for some of us this is a stretch: we don't like numbers, we are intimidated by the setting, the doc may be unhappy with the interrogation. But it is truly incumbent on all of us to accept the responsibility for sharing in these clinical decisions. I believe that the docs of today are much more in tune with shared decision-making, and understand the value of participatory medicine. And if they are not, educate them. Ultimately, it is your own attitude to risk, and not just the naked data and the clinician's perceptions of your attitude that should drive all of these decisions.  

Knowledge is empowering, and empowerment is good for everyone, patient and clinician alike. As patients, taking control of what happens to us in a medical encounter can only bring higher odds of a desirable outcome. For physicians, a cogent conversation about their recommendations may help safeguard against future litigation, not to mention augment the satisfaction in the relationship.

And thus starting to discuss probabilities explicitly is very likely to get us to a better place in terms of both quality and costs of medical care. And in the process it may very well train us how to make better decisions in the rest of our lives.

I would love to hear about your experiences discussing probability, be it in a medical or non-medical setting. And as always, thanks for reading.


If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. 

Thank you for your support!

Does number needed to treat help with rational decision-making?

Here is the perfect illustration of how irrational and emotional the issue of breast cancer is. Take the current maelstrom over the USPSTF's new screening mammography recommendations, which now advise against routine screening for women between the ages of 40 and 49 and change the recommended interval for women 50 to 74 years old from yearly to biennial screening. Let's focus on the number needed to invite (a diagnostic test's analogue of the number needed to treat, NNT). The NNT of mammography for a woman in her 40s is nearly 2,000, meaning that we need to screen 2,000 women to prevent 1 breast cancer death. Similarly, among women in their 50s, this number is about 1,300.

Let's not even talk about what the implications of over-diagnosis and over-treatment may be in all these women; I have written about this in the past here and here. Let's just focus on costs. An average cost of a mammogram is ~$100. So, multiplying the 2,000 NNT by $100 yields $200,000 per life saved. Again, if this were the only cost (and again, we are staying away from costs of repeat testing of false positives, invasive diagnostic testing and potential over-treatment and its attendant complications), I would say that it might be reasonable, especially when you take into account the number of years that can be saved for a woman in her 40s.

Now, let's look at the only therapy on the market that reduces mortality in patients with severe sepsis, drotrecogin alfa (activated). Its NNT is 16. That's right, it takes treating 16 patients to prevent 1 sepsis death. Given that a course of this drug costs ~$10,000, the cost to save 1 life is $160,000, or not that different from screening mammography in the 40-49 age group. Though the drug cost is 2 logs higher than that for mammography, the total population is about 2 logs less, so the total costs may be comparable. Yet, there is no battle going on for the use of drotrecogin alfa, and is has been all but abandoned by the ICUs in the US, mostly due to its expense.

So, without making any kind of a value judgment or a politically motivated statement, is this not a double standard? Is this not irrational and selective? Is this a result of a disease with a strong lobby versus one that does not have a patient advocacy group (mostly because 50% of these patients die in the hospital)? Or is it that mammography is perceived as prevention while drugs are disease treatment?

I am really not sure what the answer is to this apparent double standard. I also will refrain from proselytizing about the willingness to pay and whose money and the potential harm and even death due to over-diagnosis and over-treatment. But people, we do need to confront our irrational demons of inconsistency. On the other hand, if we cannot make these allocation decisions rationally as individuals, don't you think we would benefit from a body whose sole purpose it is to do this transparently and in an evidence-based manner?