How to game mortality data

There is a great illustration in this BMJ article of what I discuss in Chapter 2 of my book: the type of mortality that matters. In the figure below from the paper, note that, as the new diagnoses of each of the cancers rise (green lines), the attendant "Deaths" (red lines) stay unchanged. If you look at the Y-axis of each graph, it tells us that the unit of measurement is "Rate per 100,000 people." So the red lines represent population mortality.

"Population mortality" means that the denominator for this value is all people in the population who are at risk for the disease in question. This means that for prostate cancer, for example, we include only those men who have a prostate and exclude all women and men who have had a prostatectomy. Population mortality stands in contradistinction to case fatality. The latter is defined as deaths among all the people diagnosed with the disease. So, for prostate cancer, case fatality would be deaths among all men who have been diagnosed with prostate cancer.

It is not difficult to see how case fatality is a somewhat circular, even self-referential, measure of our diagnostic prowess, but says very little about how well we are doing with disease treatment. If we have tests that are capable of picking up the most minute of diseases, those that are not likely to cause death in the first place, then the denominator becomes inflated with this noise, while the numerator, the actual fatalities, does not change. This leads of course to an apparent reduction in deaths from the disease, but a reduction that is an artifact of overdiagnosis. Population mortality, on the other hand, cannot be gamed this way, as you can see in the figure above. This is the only mortality that gives us honest feedback without a bias about how we are doing with our early detection and other interventions. In the case of the cancers in the figure, the answer is "not so well."

You can learn more about this in Chapter 2, where I even have a figure that illustrates this critical distinction.        

If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. Thank you for your support!

Three wrong reasons to dismiss the azithromycin data

There is a lively discussion over at KevinMD about the azithromycin study in the New England Journal of Medicine, which I blogged about here. A certain ethos has emerged in the comments that bears unpacking. There are 3 distinct points that I am feeling unsetlled by:

#1. As a retrospective cohort study it should be ignored.
This is an intellectually indefensible position in my mind -- it is a study that uses a set of methods developed and validated for this type of data. Yes, it's complex; yes, it's messy; no, it's not to be ignored. The discipline of pharmacoepidemiology relies heavily on observational data. To expect anything more is to indulge in misapprehensions that a). it is feasible to run a RCT to detect such rare signals, and 2). that a RCT like that would give us a definitive answer.

#2. They really had to add a lot of zeros to the denominator to make the numerator seem impressive.
This is a baseless accusation, since 1 million prescriptions is not that difficult to generate. Z-pak has been on the market for over a decade, and according to this article, last year 55 million prescriptions for azithro were handed out in the US. So, just a back-of-the-envelope calculation for excess deaths per year at this rate is well over 2,000. And this is just in one year! So, as a safety signal this is not something to be trivialized.

#3. A concern that this information will keep patients from doctors' offices and delay needed treatment.   
I find this to be rather a hollow concern (though I am sure that the person putting it forward believes it wholeheartedly). As another commenter pointed out, the overuse and misuse of antibiotics is completely out of control! And yes, cardiac deaths from azithromycin are but a small part of the issue, where the elephant in the room is the evolution of resistance. It is not just these latest data that should keep patients as far away as possible from unnecessary healthcare encounters, seeing as how these encounters are the third leading cause of death in the US. Why aren't we worried that this entire monster is keeping patients away? And quite frankly, why isn't it?

So, all in all, I am very glad that Rob Lamberts chose to blog the study, and the discussion has been worthwhile. The comments have really confirmed for me that it is not only the lay public, but also healthcare professionals, who have a hard time interpreting data. And when a study is somewhat challenging, it is generally easier to let our cognitive biases run amok.      

If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. Thank you for your support!

Septic shock doesn't need my rescue bias

Rescue bias and auxiliary hypothesis bias are tempting distractions when it comes to dealing with data that are counter to our preconceived notions. In general, the role of our cognitive biases in all kinds of discourse, including clinical and scientific, is under appreciated. For this reason I devoted fully two chapters of my book to cognitive biases.

Which makes it even more frightening that, as I read this NEJM paper on the failed drotrecogin PROWESS-SHOCK study, I am still looking for reasons why it failed, other than that just doesn't work. After so many years of trials and tribulations with Xigris, and hopes that this lone therapy ever to have been approved for treating this deadly disease, I am having a hard time letting go.

Yet the rial was meticulous, as all trials designed and overseen by B. Taylor Thompson are -- he is just a brilliant clinical researcher that we all need to learn from. The design considered all the right issues a priori -- multiple interim looks at the data, a possible need to increase the power, heterogeneous treatment effect, and others. Although there were a few numerical imbalances between the treatment and the placebo groups -- blood cultures were positive 4% more frequently and the offending pathogen overall was 5% more likely to be identified in the Xigris than in the placebo group -- I have to accept that these are not the reasons for the observed failure to improve either the 28-day or the 90-day mortality.

I have seen and even blogged about these data before -- the press release had some of them, but, more importantly, Taylor presented them at the Society of Critical Care Medicine meeting back in January. So, this is nothing new. Yet reading all of the details in the pages of the NEJM brings back that pre-conscious cognitive wall that I have to climb over in order to get to reality. And the reality is that the drug just did not work.

But the reality is also that sepsis patients have a better chance of surviving this deadly assault than they did 10 years ago. In the original PROWESS study 28-day mortality in the placebo group was 31%, and these were all kinds of sepsis patients. In the current study, among most severe sepsis patients, those with septic shock, the 28-day mortality was on the order of 25% in both arms -- that's a staggering reduction in this most ill septic population! We really need to appreciate this. In part this trend may be due to some evolution of the disease-host interaction. But in part it has to be because of the concerted effort to understand sepsis better, to study various treatment options, and, most importantly, to implement these learnings at the bedside. In no small part we owe these advances in sepsis care to the short life of drotrecogin alpha.

The data are clearly anticlimactic, and for this reason the current paper has no sex appeal -- just look at the (lack of) press coverage about it. Yet, this is a clear example of countering the traditional publication bias: Here is a manufacturer-funded negative study published in a premier peer-reviewed journal. I realize that it was always a high-profile undertaking, but let's pause for a moment to enjoy this small victory for those who have railed against the spread of biased information in the medical literature.

I don't know if and when another therapy will emerge courageous enough to brave the rough waters of sepsis and septic shock. But one thing is clear: Xigris has served out its purpose, and no amount of rescue bias from me will save it. Rest in peace.    

If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. Thank you for your support!

"Imagine the Future of Medicine" Project

Yesterday's post hit a nerve: Who is planning the future of medicine? I don't mean the futuristic visions of single individuals, or even the palpable excitement that permeates rooms filled with health IT discussions. I am referring to an integrated vision for what we want to deliver, a vision that can then inform our actions. I am talking about strategy rather than tactics, as pointed out by Patrick Jonas in the comments here. Would you build a house without first developing a blue print? Why not? After all, we can all get excited about IKEA kitchen cabinets and stainless steel bathroom appliances. But if we don't first work with a team of people to think about the role of the structure informing its design, all we will have in the end is a pile of (shiny) rubble. So why is it that we are so averse to building a blue print for medicine? Or is it that we think it is politically impossible?

If I had to choose one most important thing I carried away from the Health Foo Camp, it would be this (as embodied by Regina Holliday): "Never doubt that a small group of thoughtful, committed people can change the world. Indeed it is the only thing that ever has." And in the age of the internet, which allows groups of committed individuals to come together in fractions of a second, there is no excuse not to try.

So, here is what I propose: How about we start having that discussion here? I will accept blog posts on the theme of "Imagine the Future of Medicine" to be posted on this site. I don't just want to hear from doctors, or those who have previously made their views known (though I encourage you to contribute too). I want to hear from everyone who has something to contribute to the discussion. I want to hear from patients, pharmacists, engineers, architects, nurses, professors, story tellers, artists, poets, anyone who can articulate their vision for what medicine of the future can and should be.

Before we start, we need to lay out the values that should inform everything in medicine. From were I sit, "do no harm" is still the most important one of all. Equally important seems to be access to safe, humane and compassionate care aligned with the patient's values, and that is guaranteed to all. Did I miss something important? Remember, we are going for high level here to refer back to when we recommend more specific details to make sure they fit this overarching purpose.

I don't want to micromanage this. I want to free-range it and see where it goes. Heck, I am not even sure that I will get any responses from all you busy people who have more important daily duties to fulfill. But I want to try. We seem to be so fixated on the trees that we are not even sure what forest we want to end up in. Let's paint that forest.

There are a couple of rules to keep in mind:
1. I will not publish any profanities, personal attacks or marketing pitches.
2. To keep the reading manageable, I ask to limit your posts to 500 words. If you feel that you really need an extra 100 or 200 for an extremely important message, I will consider that.
3. I reserve the right to do some light editing for grammar, syntax and clarity.
4. If you are unable to make yourself clear, please, request editorial help before you send in your contribution.

So, let's make Margaret Mead proud and see where this field experiment ends up.

You can e-mail me your contributions for review at healthcareetcblog AT gmail DOT com                

If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. Thank you for your support!

Free-range thinkers create Foo for thought

I cite this favorite quote from Max Planck in my book (and every chance I get):

A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it.

I think this applies to all walks of life, not just science. Yet sometimes an argument so compelling comes along that, though reluctantly at first, one by one the old guard drop at its feet. This is what happened to me this weekend at the Health Foo Camp in Cambridge, MA.

First, what is Health Foo? Well that was my first question when I received an invitation to attend this strangely named meeting. A Foo Camp is something put together by O'Reilly, the pioneering digital media group. Started 12 years ago, these meetings are thematic gatherings of "Friends of O'Reilly," hence "Foo," intended to bring together a diversity of thought about a specific field. The camp that I attended was the second such gathering in the healthcare space, supported in part by the Robert Wood Johnson Foundation, and held at Microsoft's New England Research and Development Center in Cambridge. How can I ever thank O'Reilly, RWJF and Microsoft for this mind-shifting event?

As I mentioned in my previous post, the attendee roster was so full of luminaries that I frankly wasn't sure that the invitation had not ended up in my Inbox by mistake. But mistake or not, what a privilege to attend! I spent the weekend getting to know the faces and the substance behind such familiar names as Regina Holliday, Paul Levy, Alan Greene, Ted Eytan, Susannah Fox, Gilles Frydman and others. And what still has my mind spinning is my conversations with people I don't normally interact with -- computational scientists, game designers, food advocates and international public health movers and shakers.

The most risky aspect of this meeting was the very essence of its success: we were to free-range. No agenda was set; space, food and company were provided. The resulting sessions ran the gamut from the usual nerd porn of probability to such far-reaching topics as memory and the role of faith, poetry and the arts in medicine (my personal favorite, where I got to play in the sandbox of participatory painting led by Regina. Take that, left brain!)

I have to say I spent a part of the weekend in a bit of a fog. What is gamification of medicine? What does "deep modularity" mean? But the full impact of such diversity of knowledge did not hit me until I was heading West on the Turnpike away from the meeting in the direction of home. It felt like a deep air pocket, and for a moment I couldn't catch my breath.

My epiphany was this: I have been sitting in my office and analyzing, writing and thinking about how to slow down this juggernaut of digitalization in healthcare. My logic has been to identify the problems, particularly the overdiagnosis and overtreatment and the attendant harm, all in the context of an obscene price tag, and to say that not only does medicine not need the radical digital revolution that is being imposed on it, but the very definition of medicine needs to change. What I failed to consider is that medicine is a module that needs to fit into the rest of what we call our modern life. So, slamming on the brakes in hopes of stopping this locomotive before it squashes the medical system is the wrong approach.

I can hear what you are thinking. "There goes another one." "She drank the Kool-aid." "If she gets all starry-eyed about technology, there is no hope for the rest of us." Well, I am not a fan of Kool-aid, but I do like Shakespeare. My realization is as follows: If we don't start thinking about what we want medicine to be, we will continue getting medicine that looks and works like a Rube Goldberg machine, a conglomeration of unrelated levers that may or may not achieve the desired results. The stakes are too high, stakeholders too many, and the resources being used staggering.

The digital revolution will continue its break-neck pace regardless of my opinions. In its quest to take over the world, it will continue to advance into every aspect of medicine. But instead of positioning it as a confrontation between Godzilla and King Kong, I have decided that a more constructive way is to start to imagine what medicine can and should be in the future. The current model is moribund, if not altogether dead. Nature abhors a vacuum, and unless we fill it with something that heals and nurtures that has come out of a concerted multidisciplinary blueprint, it will continue to grow into a hydra that will eventually swallow us.

I know, I miss the slide rule too. But we did not get to be on Twitter by chaining ourselves to the old paradigms. Regina Holliday taught me, among so many other things this weekend, the word "chaordic." It is a neologism that combines the ideas of chaos and order into one force of nature. I think there has been enough chaos in the relentless penetration of technologies into medicine. I am tired of screaming at at the back of the digital monster like a crazy lady. We need to get ahead of it with an open mind and even some excitement, and start imagining where we can direct it for the better health of the public. Our magical thinking will not change the fact that this tidal wave will destroy us if we lack the imagination to ride it. A gathering like the Health Foo Camp is what fuels that imagination. Let's grow and harness it!


If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. Thank you for your support!

Why I have a propensity to believe the azithromycin data

Before I take off for the Health Foo Camp in Cambridge, MA, where I will be rubbing elbows with such luminaries as Susannah Fox, Ted Eytan, e-Patient Dave deBronkart, Regina Holliday, Nick Christakis and Nancy Etcoff, I thought I'd say a few words about one of my favorite topics: antibiotics. In particular I want to talk about azithromycin. A lot has been said about it this week already in the wake of the NEJM study (alas, the full paper is behind a pay wall) indicating an increase in the risk of cardiac death among patients exposed to this drug compared to those either not getting antibiotics or those receiving a different class of antimicrobials. And a lot more will be said in the future as the FDA reviews this risk. This is not specifically what I wanted to talk about. I wanted to focus on the methods.

The study was done meticulously, as far as I can see. It was a large Medicaid database from Tennessee (this may present a generalizability problem, of course, though I cannot think of a specific reason why it would). The study design was a retrospective cohort, which, as we already know, sets the study up for all kinds of biases. So, why should we believe anything the study showed? And particularly given many people's distaste for invoking causality from observational data?

There are at least 2 reasons why we should pay attention to these results. The first is that we are talking about the ultimate harm: death. When it comes to harm, my philosophical approach leans in the direction of caution. What this means scientifically is that I accept a much lower threshold for the certainty that the data convey than when it comes to evidence of benefit. This is an extension of the precautionary principle, where the burden of proof of safety now lies with the drug.

But there is a second, possibly more important reason that I am inclined to believe the data. The reason is called succinctly "propensity scoring." This is the technique that the investigators used to adjust away as much as feasible the possibility that factors other than the exposure to the drug caused the observed effect. In my book I briefly discuss propensity scoring in Chapter 21. And here is what I say:

Propensity scoring is gaining popularity as an adjustment method in the medical literature. A propensity score is essentially a number, usually derived from a regression analysis, describing the propensity of each subject for a particular exposure. So, in terms of smoking, we can create a propensity score based on other common characteristics that predict smoking. We take advantage of the presence of some of these characteristics also in people who are non-smokers to yield a similar propensity score in the absence of this exposure. In turn, the outcome of interest can be adjusted in several ways for the propensity for smoking. One common way is to match smokers to non-smokers based on the same (or similar) propensity scores and then examine their respective outcomes. This allows us to understand the independent impact of smoking on, say, the development of coronary artery disease. 

And if you are able to access Table 1 of the paper, you will see that their propensity matching was spectacularly successful. So, although it does not eliminate the possibility that something unobserved or unmeasured is causing this increase in deaths, the meticulous methods used lower the probability of this.

One final word about azithromycin. There are data that suggest that macrolides (the class of drugs that includes erythromycin, clarithromycin and azithromycin) are actually associated with improved outcomes in the setting of community-acquired pneumonia, or CAP. This is why these drugs are in the CAP treatment guideline. The point is that again, as in everything, the benefit of using azithromycin in any individual case will have to be weighed against this newly-identified risk.

  

If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. Thank you for your support!

Blind and toothless: The future of middle class in America?

Since it's Friday, we'll steer clear of academic subjects and just tell a story. This is a story of a good friend of mine who works in an editorial position for an interdisciplinary humanities journal. This journal is based at a prestigious private liberal arts institution, and thus she is on staff at that institution. This story is about her, but also about how well the Massachusetts health law is working for people like her. I will call her Jo.

Jo is in her early 50s, a single mom of two lovely teenage boys. Their father who is separated from Jo, although in the picture, does not provide any child support for various reasons. Thus she carries sole financial responsibility for her family.

Well, you say, she is on the journal's editorial staff, she is loaded, right? Nah, surely you are not so naïve as to think that this is a high-paid position at a humanities publication. Let's just say that she is not the 1%; why, she is not even in the top one-half. Would you believe me if I told you that a high-ranking editorial staffer gets a salary that is only 2x above the federal poverty level threshold? So much for those wealthy elite academic East Coast types! But she does get her healthcare insurance through her employer, and this is fortunate, right? Well, here is the rub.

Jo has a complex health condition that affects her entire body, including skin, teeth, eyes, heart, lungs and viscera. Despite this, she is optimistic, upbeat and one of the most patient parents I have ever met. But here is what gets her goat: it is the very fact that she gets her health insurance through her employer! How can this be?

Here is how. The cost of her basic restrictive HMO insurance through her employer is over $5,000 per year. And this flat rate is not affected by the salary level of the employee; in other words, the janitor and the university president get to pay the same amount of money for the same plan. In addition to this, her office visit copay is $20, and her medication co-pays are between $10 and $30 for a month's worth of medication. As you can imagine, for someone with a complicated chronic condition, these co-pays can add up quickly, and they do, to a shocking $2,500 per year. And this is before any allowance for the boys' medical needs or dental or eye care for any of them. Once you add everything up, Jo spends over 1/4 of her entire not-so-stellar income on healthcare. But if we do add dental and eye expenses into the mix (remember, her condition affects these organ systems as well), her healthcare expenditure comes to 40-50% of her annual income! And this is just for draconian restrictions of an HMO! AND, this does not cover all of the time that she spends on the phone finding providers that take her insurance and on schlepping miles away to see that single specialist that the HMO will pay for.

But wait, you say, you live in Massachusetts, the land of socialism, gay marriage and healthcare for all. Why can't she just dump this lousy and expensive employer-provided coverage and go to the Mass Connector, where everyone is equal and all get what they need for what they can pay? And furthermore, you say, isn't the PPACA, the new healthcare law of the land that is fashioned after Romneycare in MA, going to take the choice away from people and MAKE them get insurance through these exchanges rather than through their employers? Jo must be an idiot not to be taking advantage of this communist healthcare state! Hmmm, let's see now.

Jo has had a number of discussions with the staff at the Connector. And yes, you are right in that, if she switched to one of their plans, she would qualify for a plan very similar to her present one for about 1/5 of what she pays now. And the co-pays? Why those would shrink to $0 to $10. How's that for a deal? But here is the catch: According to people she has spoken with at the Connector, a person who has health insurance offered through her employer is not permitted by law to take advantage of the Connector deals if the employer plan offers coverage that meets the minimum standard in the law. And hers does. Ironic, isn't it?

So Jo goes on struggling with the financial burden of her crappy and expensive employer-provided insurance, only now she has to pick and choose: Can she afford to replace her failing dental bridge ($4,700) or should she just choose something so frivolous as feeding her children instead? This is a choice that is not a choice at all, is it? And if these are the choices that we can expect with the PPACA, well, then, we have the law that we deserve: one that will make sure that the investors in the "healthcare" marketplace continue to get handsome returns. But start getting used to people who cannot see their computer screens showing up to work without their teeth!        


If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. Thank you for your support!

Press coverage of health data: Just like Pharma's DTC?

I am warning you now: This is going to be a rant.

Yesterday's Wall Street Journal had a front page story on observational data, and how researchers are growing more concerned about its accuracy even as the volume of such research is growing exponentially. And then today, there was this from multiple news outlets:

The HealthDay story quotes the senior author of the study thusly [emphasis mine]:

"The results of our research allow us to definitively answer the question of whether jogging is good for your health," Peter Schnohr, chief cardiologist of the long-term Copenhagen City Heart Study, said in a news release from the European Society of Cardiology. "We can say with certainty that regular jogging increases longevity. The good news is that you don't actually need to do that much to reap the benefits."

And then at the very end it says this:

The study was slated for presentation Thursday at a meeting of the European Association for Cardiovascular Prevention and Rehabilitation, called EuroPRevent2012, in Dublin.

Data and conclusions presented at meetings should be considered preliminary until published in a peer-reviewed medical journal.

So, let recap. WSJ says that observational data are of concern because they can be tough to confirm, so we should be skeptical. HealthDay and others insist that this observational study shows definitively that jogging prolongs life, so what are we supposed to believe?

The problem and the disconnect, I believe are not with the studies themselves. The problem is with the way these stories are reported and the end result of that: Just like direct-to-consumer advertising from Pharma, these stories call us to immediate action, even when the results are preliminary. We rail against Pharma's DTC, but take this kind of press coverage as a given. This type of reporting, where half-baked data are presented as the final word, disappointingly enabled by the investigators themselves (who doesn't want 15 minutes of fame?), makes observational data look like something they are not. On the one hand, we are told that here is the result. On the other, after some contemplation and peer review, we realize that the study did not show what it was said to have showed. Bingo, the sweeping conclusion is that all observational studies are bad and biased, so let's just throw out the baby with the bath water.

The press are doing a huge disservice to the public and to science itself by presenting everything in such black-and-white terms. We know that it is the initial message that grabs attention; hence "jogging adds years to your life." To make the next message stick, something powerful needs to be cooked up; hence, "Analytical Trend Troubles Scientists." Saying that "well, we overstated what the jogging study showed" isn't nearly as sexy. How about we back up a bit and say it like it is: the devil is in the details, and those details don't make nifty headlines.

I am grateful to Gary Schwitzer for slapping this kind of sloppy reporting, but he cannot eliminate it alone. We all must speak out against it. We abhor Pharma's DTC marketing practices. Why do we give the press a free pass for the same behavior? Cover accurately or don't cover at all!  

If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. Thank you for your support!