Recession as an experiment in value of treatments

This is going to be short and sweet.

Pharmalot reported two days ago that abandonment rate for all new prescriptions for brand-name drugs has reached nearly 10%. This should worry Pharma, but not for the reasons that they think. Yes, this means at least a short-to-medium term hit to their revenues. But more importantly, if this trend persists for a meaningful amount of time, it provides an opportunity for healthcare researchers to establish the true value (or lack thereof) of many of the routine drugs prescribed to the US population. What I mean is, we can examine what happens to health trends, and even to specific diseases, while patients are not taking the medications that are considered so vital for their well-being. If we do not see a rise in untoward events, or, better yet, if we see that our public's health is actually better without these meds, that will deal a significant blow to the idea of "a pill for everything". Alternatively we may learn that these meds are truly vital, and this finding should certainly impact our debate about access to them.

Of course, there will be objections. The methods for mining these effects may not be solidly hypothesis-testing, the time in question may not be enough to detect any meaningful changes, and, our favorite, the causality will be difficult to establish. All of the this notwithstanding, this period of time should provide a very interesting natural experiment in the overall value of our treatments today.

Hat tip to @AHCJ_Pia for a ink to this story here.    

Does disproving a statistical null automatically render the clinical null disproved?

A good friend of mine lost her mother to pancreatic cancer recently. The whole process from diagnosis to her death took 6 weeks. And despite wonderful care from a palliative medicine team, the process proved grueling to her family. And no wonder: how do you assimilate a loved one's going from healthy to dead in six weeks? Of course, my friend's family made all the right choices, forgoing aggressive treatment in favor of maximizing their mother's comfort and quality of life. Their experience made me think of the new generation of cancer treatments, experienced by my father in his dying days, and how it all fits in the healthcare debate. 

Much of the debate around the healthcare reform has centered around the unsustainable cost trajectory, as well as the value of evidence in improving the quality and effectiveness of care. The Congress has appropriated a substantial amount of dollars for comparative effectiveness research (CER) intended to provide data comparing one treatment to another, rather than just a treatment to a placebo. This in turn has sparked the debate as to whether to include cost-effectiveness in this comparison. I am definitely in the camp that believes it should be included for the reasons so eloquently outlined by Weinstein and Skinner. At the same time this MSNBC article made me re-evaluate its need, at least in some special cases. Let's peel back the cabbage, specifically looking at Tarceva in advanced pancreatic cancer, exactly what my friend's mother was diagnosed with.

Tarceva, or erlotinib, is a kinase inhibitor manufactured by Genentech, indicated for the treatment of some cases of non-small cell lung carcinoma, and most recently approved by the FDA for advanced pancreatic cancer. Reading the package insert, it becomes clear that the FDA-approved 100 mg dose of this drug, if given in combination with gemcitabine, prolongs median survival by <2 weeks, from 6 months in gemcitabine+placebo arm to 6.4 months in the gemcitabine+Tarceva arm, for a p=0.028. This difference imparts statistical significance at the conventionally set p<0.05, and therefore renders Tarceva better than placebo. Period. 

Delving a tad more deeply into the peer-reviewed publication of the phase 3 trial, one gleans a few other facts. I quote:

Survival and Response

The final analysis was conducted after 486 deaths (239 on erlotinib and gemcitabine and 247 on placebo and gemcitabine). Overall survival was significantly longer in the erlotinib and gemcitabine arm with an estimated HR of 0.82 (95% CI, 0.69 to 0.99; P = .038; log-rank test stratified for performance status, extent of disease, and pain score at baseline; Fig 1A). Median survival times were 6.24 months versus 5.91 months for the erlotinib and gemcitabine versus placebo and gemcitabine groups with 1-year survival rates of 23% (95% CI, 18% to 28%) and 17% (95% CI, 12% to 21%), respectively (P = .023). A multivariate Cox regression analysis showed that erlotinib treatment (HR, 0.82; 95% CI, 0.69 to 0.99; P = .04) and female sex (P = .03) were significantly associated with longer overall survival. While there was an imbalance in male:female ratio between the arms, the treatment effect remains significant when adjusted for sex.

 Results of subgroup analyses of survival by baseline stratification factors and other factors such as sex, race, pain intensity score, and age are displayed in Figure 2.

Progression-free survival was significantly longer in the erlotinib and gemcitabine arm than the placebo and gemcitabine arm with an estimated HR of 0.77 (95% CI, 0.64 to 0.92; P = .004; log-rank test stratified for performance status, extent of disease, and pain score at baseline; median, 3.75 months v 3.55 months; Fig 1B).

So, what do we have overall? We have a hazard ratio of dying that very nearly crosses 1.0, thus coming perilously close to not disproving the null; we have a prolongation of median survival by 1/3 of a month, and a progression-free median survival prolongation by 1/5 of a month.

But given my fondness for Gould's "The Median is not the Message" essay, let's practice full disclosure and look at the tail of the Kaplan-Meyer curves above. As the text points out,

...1-year survival rates of 23% (95% CI, 18% to 28%) and 17% (95% CI, 12% to 21%), respectively (P = .023).

Indeed, these are significant differences, both statistically and clinically. Within the trial this represents the difference in favor of survival for 18 additional patients, thus rendering the cost of roughly $1.5 million for 1 year of life saved by my back-of-the envelope calculation. Of course, this difference is not adjusted for confounders, so it is difficult to say of the number is real of under- or over-estimated. Because the adjusted analysis is given as a hazard ratio of death, I cannot calculate the corresponding adjusted cost.

So, for me this begs the following question (and I would love to hear the thoughts from my colleagues who proudly proclaim being science-based and thus eschewing placebo effect as a valid way to get a therapeutic response): Is what we are seeing here real or is this in fact equivalent to a placebo effect? Is the median survival prolongation of <2 weeks indeed a real effect that means something to the patient and the clinician, or is it just clinical noise, if you will? In other words, should "disproving" the statistical null by default disprove the clinical null? Or does the bar for disproving the clinical null need to be set just a tad higher than a statistically significant increase in life expectancy of 2 weeks?

Obviously, no one knows a priori who will do better and who will not. So, without a crystal ball, it is one's values and preferences that have to drive these decisions. But does the society have a say in any of this, as we struggle with equitable distribution of a limited resource? Is $1.5 million for 1 year of life a good societal investment? And what is the quality of this life? And given that at least 1/2 of all treated patients get far less than extra 2 weeks of life, how do we strike the balance between a reasonable expectation of a response and a false hope?

My friend's family based their choices on their mother's wishes and their values and utilities for her comfort. The choice they made was very different from that made by my parents with regard to my father's palliation. Both were right for the respective families. But one may have been far too costly, both financially and emotionally.    

Healthcare reform: so much more than costs

The opposition to healthcare reform is fanning fears of rationing as a way of controlling escalating costs and broadening access. A better use of our collective energies would be to identify and eliminate the waste and harm produced by the current healthcare system. We are well aware of over 1 million hospitalizations and the nearly 100,000 deaths associated with hospital-acquired infections, at a cost of approximately $40 billion annually. We have this information partly because the data infrastructure within hospitals exists to track these events. Many more potential adverse consequences of healthcare remain unrecognized and unquantified, particularly those that occur outside of hospitals, where the majority of healthcare takes place. We hear distant echoes of alarms sounded by researchers, but because there is no integrated picture of the full magnitude of the harm, we remain complacent. We must admit that harm exists, develop systems to track and quantify it, and finally eliminate it. The time for this is now; here is why.

Why would harm be pervasive in what has been called by some “the best healthcare system in the world”? Theoretically, risk aversion, by promoting a culture of over-testing, over-diagnosis and over-treatment, is fraught with paradoxically noxious potential. Beyond theory, an example of how the “better-safe-than-sorry” approach can be detrimental plays out in screening mammography. Despite the fact that the US Preventive Services Task Force says to commence breast cancer screening at the age of 40, the risk and benefit balance of this recommendation is unclear. A recent study from Europe, for example, noted that for every 2,000 women screened over a 10-year period, one cancer death is averted. The trade-off is that 2-10 other women receive invasive and toxic treatments for a cancer that would not have become life-threatening. What is even more disturbing is that we do not know how many of these 10 women are consigned to life-long illnesses or an early death due to the complications of this unnecessary treatment. Similar concerns exist for prostate cancer screening and for many other diagnostic and therapeutic modalities we employ every day: the drug and device approval process focuses more on short-term benefits and less on long-term risks. Therefore, the full extent of the harm remains unknown.

Until recently, the heretical talk of harmful effects of healthcare was relegated to the fringes of the medical world. Fortunately, this conversation has now penetrated into the mainstream, with a paper in a recent issue of The New England Journal of Medicine laying out the research framework to explore this problem. Unfortunately, the research agenda, which still requires much clarification and planning, though harnessing the will to fill this knowledge gap, will be hard-pressed to find a way. This is because of the heterogeneous and still mostly paper-based medical record-keeping by the majority of US practitioners, precluding meaningful recognition and aggregation of relevant data.

This lack of knowledge gives the public a rosy view of healthcare: high benefits with virtually no harms. Getting a handle on the full picture of the balance of benefits with harms and waste, however, is critical for optimizing outcomes. Although data to quantify it are presently scant, the timing of the emergence of this research agenda, coincident with the major push to adopt electronic health data platforms along with the recent allocation of funding to the comparative effectiveness research (CER), is fortuitous. Electronic data, if captured accurately and uniformly, are a robust source of complete information on long-term outcomes, and the well-established framework for CER is a natural fit for incorporating these data to inform policy decisions.

If healthcare reform is to achieve its goals of universal access to good quality healthcare at a reasonable price, we must not miss this opportunity to align the research into harm with our ability to generate useful data from electronic sources to feed smart policy decisions through CER. Although historically our healthcare system has enjoyed very little integrated planning, this is an opportunity to draw up a sensible blueprint for a successful future. If we do it right, we will be not only saving money, but also addressing such an important societal concern as human lives. 

On "10 Steps to Better Health Care" in NYT

Here is a comment I posted on the NYT site in response to Gawande, Berwick, Fisher and McClellan's article "10 Steps to Better Health Care". Thought it was worth posting here because it brings together many of the arguments that I have been making piece-meal in multiple posts.

A great piece from a venerable group indeed, thanks! Clearly we are at a tipping point, since we have known about regional variations from Drs. Wennberg's and Fisher's work for nearly 30 years and have done nothing about it till now. The trajectory of growth of our national healthcare expenditures combined with the abysmal quality and worsening access have gotten us to this political nexus where inaction is not an option.

The two central problems that are driving the situation are 1). the profit motive and 2). fear of malpractice. The profit motive for all involved in the system has created an incentive to diagnose more disease to treat with therapies developed based on the market size rather than on real needs, to perform more procedures and to build a giant bureaucracy ostensibly for the sake of checks and balances, but in reality to increase the bottom line. As for malpractice fears, because our society simplistically thinks that more care is better care, doing more is the default.

In my mind, the Mayo system works well precisely because the profit motive has been removed for those on the ground. Take the market out of healthcare, and you have a model for success. Unfortunately, single payer has very little chance of passing, as it is a political third rail in the US. And tort reform? Well, we know that no one wants to tackle that either, for so many reasons. So, best we can do is to come up with a way of defining and promoting value-based healthcare (think comparative effectiveness). Within that we really have to quantify the harms inherent in overuse and misuse, so that we can start changing this culture of more is better.

For more on these issues see the following posts on this blog: http://evimedgroup.blogspot.com/2009/08/putting-sticker-price-on-healthcare.html http://evimedgroup.blogspot.com/2009/07/convenient-failure.html http://evimedgroup.blogspot.com/2009/07/is-useless-harmful.html

Putting a sticker price on healthcare

Let's say you are in the market for a car. You have settled on a GM model (unlikely, I know), and now you are shopping around for the best price. There are 2 dealerships that are equidistant from your home, and they both have the model you want in the color and with all the accessories that your heart is set on. The only difference is that the dealer on your right is insisting on charging you $5,000 more than the one to your left. What do you do? Absurd, you say? OK, how about if the dealer to the right is throwing in a rust-proofing treatment that the dealer on the left is refusing to give you. Are you willing to spend an extra $5,000 on rust-proofing? Still absurd?

Well, when it comes to cars, these decisions are pretty straight-forward. Not so in healthcare, right? Healthcare is full of arcane and esoteric concepts that only experts can unravel, and the public has to be vigilant not to get duped into getting less than they deserve. The reality is that we have not asked for the same accountability from our healthcare "dealers" as we expect from car vendors. By trusting the experts to know what they are doing, and because of our love affair with new stuff, we have been willing to pay a Rolls Royce price for a used Oldsmobile. Just because a technology is new and touts itself as being better, it is not necessarily so. Would you buy a car only because of its manufacturer's claims of safety? Would you not want this confirmed by independent reviewers, such as Consumer Reports, before investing your trust and dollars?

Admittedly, healthcare is more complicated. But the real monkey wrench is not what you think it is, the complexity of each individual human organism's interaction with its world. Rather, the monkey wrench is the matrix of interconnectedness within the healthcare industry, running from the manufacturers to healthcare providers, to the Food and Drug Administration, to the Insurance companies. This complex web is based on money in the way of profits, reimbursements, salaries and executive bonuses, and it does not explicitly include patient interests. It is this very complexity that keeps us on guard.

So, into this swamp of special interests, the administration is introducing the concept of comparative effectiveness research (CER) for sensible policy development. The reality of CER is much less sinister that what the Republicans will have you believe: its intent is in essence to compare values of interventions aimed at the same outcome. Going back to the analogy of cars, the CER is meant to quantify explicitly how much the rust-proofing is worth, and to tell us whether $5,000 is too much to pay for it. The idea is that, once this comparative value has been determined, we can start making our healthcare decisions more rationally than we are forced to do today.

Americans have a legacy of being sensible consumers. While the administration is proposing demystifying the value proposition of healthcare decisions, the Republicans are promoting further obfuscation. They do not want us to be able to make value-based choices in healthcare, and we absolutely must ask why. It is that simple: the choice is not between whether grandma lives or dies (after all, there is still 100% risk of death for all of us eventually), the choice is essentially between giving away the $5,000 to the car dealer for a useless rust-proofing or whether we can use that money for something of real value to ourselves and our community.