Today I was going to tell you the tale of my son's broken wrist (he is fine now, this happened in January, but the insurance issues are fascinating), but I got distracted thinking about another fascinating subject that many do not understand well: confounding by indication. I especially started thinking about it in the context of how decisions and policies are made, and how not having the right data at the right time leads to this "Titanic effect" for a technology. What do I mean by this? Well, let me explain.
Some say the Titanic sank simply because of poor preparation -- not enough life boats, not enough training on the evacuation procedure, in other words "not enough imagination" to plan for a catastrophe. It was derailed in its course by an entirely predictable natural calamity that had not been planned for adequately, even though the risk was obvious in retrospect. Was this just on of those "unintended consequences" that could have been avoided with more clear vision? Perhaps, but the Titanic is, ahem, water under the bridge. But we can focus on some more mundane and current potential missteps and make some guesses.
Let's talk about medical technologies, and drugs in particular. Let us say that there is a new sepsis drug that has been tested among patients with sepsis but without organ failure. This drug appears to prevent organ failure in a fraction of the treated patients, and also reduces mortality by 6%. The only obstacle to widespread use of this drug is its acquisition cost, which is much higher than what the hospital's critical care pharmacist is used to paying for other drugs. Because of this high cost, the drug, despite being on the formulary, gets administered only to those patients who have developed not one, but two organ failures. The savvy pharmacist looks at the outcomes of these patients and, after comparing them to those of the patients who did not receive the drug, concludes that the new sepsis drug, instead of saving lives, actually kills. The P&T committee discusses this, dumps the drug from the formulary and other hospitals follow suit. What's wrong with this picture?
Several fallacies are at work here, including an overly broad inference of causality and bias. But the most important lesson is to do with confounding: because of its apparent expense, the drug has been niched into a population of patients who a). were not the ones that exhibited the evidence of benefit in the trials, and b). have a very high risk of mortality at baseline. So, not only is it not valid to conclude that the drug killed these patients, but it is not even valid to say that the drug does not work -- it may well work in the populations that it was shown to work in, but not in this, much more ill, population. You see the difference? It is like saying that you umbrella failed to keep you dry when you opened it only after you already got soaked.
So confounding by indication is one reason that drugs "fail" -- they are given to people who are by definition not going to do well, and the confirmation bias pushes us to say see, it's expensive and doesn't work. So how do we overcome this phenomenon and make sure that appropriate patients get access to useful technologies? I believe I have a very simple answer: don't squeeze the toothpaste out of the tube if you don't want to have to cram it back in. Huh?
In other words, do what I always advocate: be ready with the relevant data before the train leaves the station, before the cat gets out of the bag, before the horse gets out of the barn. It is very well known that cognitive biases, once established, are difficult to overcome. The pharmacist's first concern is for being able to use his very limited resources efficiently, and to guard from spending his monthly budget on a potentially useless intervention in a single patient only to be left with no resources to care for all of the other patients. Yet many manufacturers at launch send their reps to the pharmacist with two virtually unrelated stories: one about efficacy and the other about the acquisition price and its impact on his budget. When the drug is expensive, the efficacy pales in comparison to the price tag, and the pharmacist has no choice but to restrict the use of the drug, thereby consigning it to failure by confounding by indication. Sound familiar?
Is there a way to avoid this scenario? I think so. It is self-evident that you have to have good data. The surprising thing is that good data are necessary, but not sufficient: the timing of these data is critical as well. It is easier to help people form an opinion where none exists than to change one that is already there. So, to be successful, the manufacturer with a good technology must have a coherent effectiveness and cost-effectiveness proposition right out of the gate. Not only that, but it is imperative to help the clinician understand what patients might benefit from the technology (no, not all patients should be on your drug). This is the kind of a collaboration that will ultimately benefit all stake holders: 1). Appropriate patients will get the opportunity at better outcomes, 2). The pharmacist will understand up front the value proposition and the potential scope of use, and 3). The manufacturer will profit from providing a beneficial service. Isn't this the intent of all this drug development?
If all this seems all too obvious, it is because this is not rocket science. But why, then, do I see so many companies get into trouble with this very scenario? Is it just the case of "best laid plans" or is it a real blind spot that needs to be illuminated? You tell me. Given the investment that goes into drug development, I think it makes sense to approach this gap earnestly, instead of just shuffling the deck chairs on the Titanic.
If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status.
Thank you for your support!
Showing posts with label cost-effectiveness. Show all posts
Showing posts with label cost-effectiveness. Show all posts
Tuesday, April 24, 2012
Friday, July 22, 2011
Whose perspective?
After a long hiatus filled with travel, work and lack of inspiration to write anything, I have chosen this arguably hottest day of the year to venture forth again. But I will make this brief, as it seems that everything that needs to be thought and said has already been thought and said. Yet who is listening?
Anyhow, to suspend my natural cynicism, I want to talk about perspective. No, not the perspective that makes parallel lines converge in the distance, but the one that gets lost in many of our political, civic, business, and, yes, even scientific discussions. I am talking about my perspective, your perspective, societal perspective, etc. I was inspired to write this because of these tweets by Gary Schwitzer to Kaiser Health News about a story on their web site:
Anyhow, to suspend my natural cynicism, I want to talk about perspective. No, not the perspective that makes parallel lines converge in the distance, but the one that gets lost in many of our political, civic, business, and, yes, even scientific discussions. I am talking about my perspective, your perspective, societal perspective, etc. I was inspired to write this because of these tweets by Gary Schwitzer to Kaiser Health News about a story on their web site:
Linking out to the story, I learned that a consulting arm of Disney is teaching hospitals about hospitality. Since there is going to be a financial incentive for these hospitals to deliver good customer service, many are feeling that an investment in this type of training will help them maximize these new reimbursements. Hurrah and ta-da!
Well, Gary likes to burst these one-sided bubbles, and so he rightfully asked about the costs. What was baffling to me was the response by the KHNews who did not seem to appreciate the importance of reporting the costs or the various perspectives that these costs represent. So, this seemed like a teachable moment, and here is the teaching.
In outcomes research, we are always interested in understanding the perspective for both the costs and the benefits of interventions. In health outcomes these perspectives are broadly represented by the patient, the provider, the hospital, the payer, the employer, the manufacturer, the society, to name a few. These are just some of the examples of the usual stakeholders involved in healthcare decisions. Because our healthcare is such a fragmented disaster, many of these perspectives find themselves at odds with one another. Just think of the patient who wants to get what she perceives as a life-saving treatment that in reality has a 1% chance of helping at a cost of $600,000 per treatment course. From her perspective, since she is insured, this investment is well worth the cost. For a payer, however, this means $600,000 (multiply this by 100 in order to determine the cost to save 1 life) that cannot be spent on something else that can help more people more predictably. And if this payer is the taxpayer, the societal perspective enters the picture, where we have to decide what amount of money is worth spending on possibly saving one life -- is $60 million reasonable? Perhaps. But these are not simple questions, and, as such, do not have simple answers. In addition, all conversations that we hear or engage in have multiple perspectives. This is why a black-and-white approach is so divisive: it generally emphasizes two diametrically opposed perspectives.
So, next time you hear about death panels or Mickey Mouse teaching hospitals how to maximize their revenue, consider the broader implications from may different perspectives. Chances are you will find yourself agreeing with more than one point of view. And when this happens, you will know that you have learned an important lesson and can now start engaging in more nuanced and thus productive debates, many of which will shape our society's future.
h/t to @garyschwitzer for this KHN story
Thursday, January 6, 2011
National Healthcare Expenditures, 2009 (In pictures)
Well, it's that time of the year again: CMS has given us the accounting of our National Healthcare Expenditures (NHE) in a paper published in Health Affairs. I am sure you have already heard that the spending only went up by 4% this year over last, an all-time low.
At the same time, we have achieved the highest ever NHE as a proportion of the GDP (17.6%) and as expenditures per capita ($8,086). But the GDP proportion is a somewhat deceptive number on the one hand, as the GDP has suffered a substantial drop from its 2008 value of $14.4 trillion to $14.1 trillion in 2009. On the other hand, this implies that healthcare is eating into the rest of our expenditures on life. At the same time the per capita expenditures have continued their relentless rise.
Let us look at the components of the NHE individually and see what they can tell us.
As usual, the bulk of the expenditures went to personal health care (85%). Public health got a measly 3% of the total NHE, and this continues to be one of our gravest misappropriations. You may recall that about a year ago I did a post where I cited some startling statistics about some broad categories of causes of premature death in the US. Access to medical care accounted for a measly 10% of those, and the rest were attributable to behavior, genetics, environment and social factors. So, while, by inference, fixing medicine may impact 10% of these premature deaths, in reality 97% of the entire NHE goes to medicine rather than to potentially more impactful public health interventions. And the real travesty is that, despite these astronomical expenditures, we are still losing 1,000 lives per day to our broken healthcare system.
Looking a bit more closely at the "personal health" category, we see that, just as in years past, hospital costs and professional services comprise the bulk of this spending.
The "professional services" category, 81% of which is physician and other clinical services, is a bit murky. Yet, without too many leaps of faith we can say that if this expenditure buys us better preventive care, it may be a cost-effective area. At the same time we know that we can make this area a lot more efficient by streamlining and realigning incentives to promote better health rather than more care. Hospital expenditures, on the other hand, are a juggernaut that without a doubt requires containing. It is very likely that exchanging our inflated personal healthcare budgets for well placed public health funding along with reimbursement reform and improved end-of-life decisions, could substantially alter this category of spending.
One final data point that interested me was the breakdown of what are considered investments in the healthcare system. This broadly includes government-funded research and allocations for structures and equipment. Now, I am not sure what "structures and equipment" means, so, if any of my readers know, please, enlighten me. I do know what "research" means, however, and am rather disappointed about this breakdown. What I do not understand is, given that structures and equipment should have some kind of a half-life and not be replaced annually, how it is that this budget also grows consistently year-over-year at a steady rate? Would love to get more details on this.
To be sure, the total research expenditure of $45 billion is nothing to sneeze at. The big question is, however, are we spending it on the right research. I am not at all sure that the answer is yes, given that we still struggle with the same issues at the bedside that we have been struggling with for over a decade. But more on this later.
At the same time, we have achieved the highest ever NHE as a proportion of the GDP (17.6%) and as expenditures per capita ($8,086). But the GDP proportion is a somewhat deceptive number on the one hand, as the GDP has suffered a substantial drop from its 2008 value of $14.4 trillion to $14.1 trillion in 2009. On the other hand, this implies that healthcare is eating into the rest of our expenditures on life. At the same time the per capita expenditures have continued their relentless rise.
Let us look at the components of the NHE individually and see what they can tell us.
As usual, the bulk of the expenditures went to personal health care (85%). Public health got a measly 3% of the total NHE, and this continues to be one of our gravest misappropriations. You may recall that about a year ago I did a post where I cited some startling statistics about some broad categories of causes of premature death in the US. Access to medical care accounted for a measly 10% of those, and the rest were attributable to behavior, genetics, environment and social factors. So, while, by inference, fixing medicine may impact 10% of these premature deaths, in reality 97% of the entire NHE goes to medicine rather than to potentially more impactful public health interventions. And the real travesty is that, despite these astronomical expenditures, we are still losing 1,000 lives per day to our broken healthcare system.
Looking a bit more closely at the "personal health" category, we see that, just as in years past, hospital costs and professional services comprise the bulk of this spending.
The "professional services" category, 81% of which is physician and other clinical services, is a bit murky. Yet, without too many leaps of faith we can say that if this expenditure buys us better preventive care, it may be a cost-effective area. At the same time we know that we can make this area a lot more efficient by streamlining and realigning incentives to promote better health rather than more care. Hospital expenditures, on the other hand, are a juggernaut that without a doubt requires containing. It is very likely that exchanging our inflated personal healthcare budgets for well placed public health funding along with reimbursement reform and improved end-of-life decisions, could substantially alter this category of spending.
One final data point that interested me was the breakdown of what are considered investments in the healthcare system. This broadly includes government-funded research and allocations for structures and equipment. Now, I am not sure what "structures and equipment" means, so, if any of my readers know, please, enlighten me. I do know what "research" means, however, and am rather disappointed about this breakdown. What I do not understand is, given that structures and equipment should have some kind of a half-life and not be replaced annually, how it is that this budget also grows consistently year-over-year at a steady rate? Would love to get more details on this.
To be sure, the total research expenditure of $45 billion is nothing to sneeze at. The big question is, however, are we spending it on the right research. I am not at all sure that the answer is yes, given that we still struggle with the same issues at the bedside that we have been struggling with for over a decade. But more on this later.
Monday, November 29, 2010
Why are we still paying tobacco executives to kill us?
A few days ago I blogged my dissatisfaction with the coverage of the NLST trial by "The Health Show" on NPR. On the show, the host interviewed Dr. Regina Vidaver, the head of the National Lung Cancer Partnership, and the interview proceeded along predictably sensationalized lines of popular health reporting. For my substantive criticisms you can refer to my previous post. What has followed my posting of the piece is what is unusual, and perhaps tends to get lost in the heat of criticism. Dr. Vidaver promptly contacted me to schedule a time to talk. I just got off the phone with her and felt compelled to write a follow-up (with her consent, of course).
The conversation centered around some of my major criticisms of the interview: not emphasizing enough smoking cessation as the main intervention needed to reduce lung cancer mortality, the issue of false positive CT findings followed by the ensuing potentially invasive work-up with its on occasion adverse consequences, the costs in the setting of finite resources. Dr. Vidaver pointed out, and this did not surprise me, that her strong initial statement on the need to fund smoking cessation did not make it into the final segment. Neither did any discussion of the potential for false positive findings and their consequences. She also articulated to me that, although she and her organization believe that smoking prevention and cessation remain the single most effective public health approach to curtailing lung cancer mortality, they also believe that reliable screening tools are necessary, particularly since never smoking or quitting smoking does not guarantee that one will not die of lung cancer. In fact, she hopes for the development of an intermediate marker to help risk stratify those population members, be it smokers or non-smokers, who are at a heightened risk for lung cancer and who might then be candidates for close and regular radiographic screening. As for the issues of cost, she indicated that her organization does not get involved in this sticky policy issue.
So, this interaction has raised a couple of interesting points for me. First, how much control does the guest, or one being interviewed in general, have over the final content of the interview? I worried about this myself when in 2008 one of my papers on Clostridium difficile epidemiology in the US hospitals garnered some press attention. Out of the blue, I was contacted by Mike Stobbe from the Associated Press, who was interested in asking me some questions. Being largely inexperienced in talking to the press, and unaware of Mike's sophistication and integrity, I was nervous. This made me quite cautious about how I represented our data, as well as other relevant science. For about 48 hours after the paper went live, I received a constant stream of calls and e-mails requesting newspaper and radio interviews. I was even interviewed by Jon LaPook, the CBS health reporter. Throughout that experience I was exceedingly circumspect, and may have missed an opportunity to drive home some messages firmly. This, my friends tell me, is the difference between science and advocacy. So, while I am currently creating the nexus between the two, at that time I clearly chose to keep them separate.
But what about journalism and advocacy? Were the show's host and producers unwittingly engaging in advocacy to get people to accept the screening paradigm to the exclusion of other, possibly more sensible, interventions? Or was it just that there is nothing new and shiny about the smoking cessation message, and they just did not want their listeners to turn the dial? I do not know the answers to these questions, as they have not volunteered their comments. But you can bet that I will be that much more vigilant of their reporting in the future.
The second point raised for me by my interaction with Dr. Vidaver is the issue of costs. I completely understand why a disease advocacy organization would want to steer clear of addressing this third rail of healthcare policy. The impression among most of my fellow Americans is that bringing costs into the equation diminishes the value of one's life. Yet, our lives are constantly being priced, and rather more crudely and expediently than health economics dictates. Paradoxically, no one seems to mind the fact that our politicians and pundits price our lives every day by prioritizing economic interests (such as the entire sector of the economy powered by tobacco) above human lives. How many lives lost to this addiction pay for the jobs and the salaries of Philip Morris executives? Why is this not seen for what it is: a trade-off between a citizen's life and profit for a producer of poison? How is asking the difficult cost of care questions, where the trade-off is often between prolonging suffering at the close of life and redeploying these resources toward preventing disease, so much more deplorable than paying tobacco companies to kill us?
These are odd contradictions, if you ask me, and all emblematic of our predictably irrational human nature. At the same time, these issues will not go away, and just because they are difficult does not mean we should bury our heads in the sand to avoid them. Nothing replaces a cogent national discussion to get at the much needed solutions. Yet, isn't it time we just said no to tobacco? Would this not eliminate the huge policy headaches of how to finance screening for lung cancer and deal with the avalanche of false positive results and ensuing complications among perhaps as many as 10% of the US population? Does it not make more sense to eliminate 85% of all lung cancer deaths by getting rid of the poison than to eliminate 0.3% of all lung cancer deaths while adding untold hundreds of billions of dollars to our already mammoth healthcare bill, not to mention causing further escalation of healthcare-associated injury and death by chasing false positives?
The conversation centered around some of my major criticisms of the interview: not emphasizing enough smoking cessation as the main intervention needed to reduce lung cancer mortality, the issue of false positive CT findings followed by the ensuing potentially invasive work-up with its on occasion adverse consequences, the costs in the setting of finite resources. Dr. Vidaver pointed out, and this did not surprise me, that her strong initial statement on the need to fund smoking cessation did not make it into the final segment. Neither did any discussion of the potential for false positive findings and their consequences. She also articulated to me that, although she and her organization believe that smoking prevention and cessation remain the single most effective public health approach to curtailing lung cancer mortality, they also believe that reliable screening tools are necessary, particularly since never smoking or quitting smoking does not guarantee that one will not die of lung cancer. In fact, she hopes for the development of an intermediate marker to help risk stratify those population members, be it smokers or non-smokers, who are at a heightened risk for lung cancer and who might then be candidates for close and regular radiographic screening. As for the issues of cost, she indicated that her organization does not get involved in this sticky policy issue.
So, this interaction has raised a couple of interesting points for me. First, how much control does the guest, or one being interviewed in general, have over the final content of the interview? I worried about this myself when in 2008 one of my papers on Clostridium difficile epidemiology in the US hospitals garnered some press attention. Out of the blue, I was contacted by Mike Stobbe from the Associated Press, who was interested in asking me some questions. Being largely inexperienced in talking to the press, and unaware of Mike's sophistication and integrity, I was nervous. This made me quite cautious about how I represented our data, as well as other relevant science. For about 48 hours after the paper went live, I received a constant stream of calls and e-mails requesting newspaper and radio interviews. I was even interviewed by Jon LaPook, the CBS health reporter. Throughout that experience I was exceedingly circumspect, and may have missed an opportunity to drive home some messages firmly. This, my friends tell me, is the difference between science and advocacy. So, while I am currently creating the nexus between the two, at that time I clearly chose to keep them separate.
But what about journalism and advocacy? Were the show's host and producers unwittingly engaging in advocacy to get people to accept the screening paradigm to the exclusion of other, possibly more sensible, interventions? Or was it just that there is nothing new and shiny about the smoking cessation message, and they just did not want their listeners to turn the dial? I do not know the answers to these questions, as they have not volunteered their comments. But you can bet that I will be that much more vigilant of their reporting in the future.
The second point raised for me by my interaction with Dr. Vidaver is the issue of costs. I completely understand why a disease advocacy organization would want to steer clear of addressing this third rail of healthcare policy. The impression among most of my fellow Americans is that bringing costs into the equation diminishes the value of one's life. Yet, our lives are constantly being priced, and rather more crudely and expediently than health economics dictates. Paradoxically, no one seems to mind the fact that our politicians and pundits price our lives every day by prioritizing economic interests (such as the entire sector of the economy powered by tobacco) above human lives. How many lives lost to this addiction pay for the jobs and the salaries of Philip Morris executives? Why is this not seen for what it is: a trade-off between a citizen's life and profit for a producer of poison? How is asking the difficult cost of care questions, where the trade-off is often between prolonging suffering at the close of life and redeploying these resources toward preventing disease, so much more deplorable than paying tobacco companies to kill us?
These are odd contradictions, if you ask me, and all emblematic of our predictably irrational human nature. At the same time, these issues will not go away, and just because they are difficult does not mean we should bury our heads in the sand to avoid them. Nothing replaces a cogent national discussion to get at the much needed solutions. Yet, isn't it time we just said no to tobacco? Would this not eliminate the huge policy headaches of how to finance screening for lung cancer and deal with the avalanche of false positive results and ensuing complications among perhaps as many as 10% of the US population? Does it not make more sense to eliminate 85% of all lung cancer deaths by getting rid of the poison than to eliminate 0.3% of all lung cancer deaths while adding untold hundreds of billions of dollars to our already mammoth healthcare bill, not to mention causing further escalation of healthcare-associated injury and death by chasing false positives?
Monday, September 27, 2010
Does disproving a statistical null automatically render the clinical null disproved?
A good friend of mine lost her mother to pancreatic cancer recently. The whole process from diagnosis to her death took 6 weeks. And despite wonderful care from a palliative medicine team, the process proved grueling to her family. And no wonder: how do you assimilate a loved one's going from healthy to dead in six weeks? Of course, my friend's family made all the right choices, forgoing aggressive treatment in favor of maximizing their mother's comfort and quality of life. Their experience made me think of the new generation of cancer treatments, experienced by my father in his dying days, and how it all fits in the healthcare debate.
Tarceva, or erlotinib, is a kinase inhibitor manufactured by Genentech, indicated for the treatment of some cases of non-small cell lung carcinoma, and most recently approved by the FDA for advanced pancreatic cancer. Reading the package insert, it becomes clear that the FDA-approved 100 mg dose of this drug, if given in combination with gemcitabine, prolongs median survival by <2 weeks, from 6 months in gemcitabine+placebo arm to 6.4 months in the gemcitabine+Tarceva arm, for a p=0.028. This difference imparts statistical significance at the conventionally set p<0.05, and therefore renders Tarceva better than placebo. Period.
Delving a tad more deeply into the peer-reviewed publication of the phase 3 trial, one gleans a few other facts. I quote:
Survival and ResponseThe final analysis was conducted after 486 deaths (239 on erlotinib and gemcitabine and 247 on placebo and gemcitabine). Overall survival was significantly longer in the erlotinib and gemcitabine arm with an estimated HR of 0.82 (95% CI, 0.69 to 0.99; P = .038; log-rank test stratified for performance status, extent of disease, and pain score at baseline; Fig 1A). Median survival times were 6.24 months versus 5.91 months for the erlotinib and gemcitabine versus placebo and gemcitabine groups with 1-year survival rates of 23% (95% CI, 18% to 28%) and 17% (95% CI, 12% to 21%), respectively (P = .023). A multivariate Cox regression analysis showed that erlotinib treatment (HR, 0.82; 95% CI, 0.69 to 0.99; P = .04) and female sex (P = .03) were significantly associated with longer overall survival. While there was an imbalance in male:female ratio between the arms, the treatment effect remains significant when adjusted for sex.

Results of subgroup analyses of survival by baseline stratification factors and other factors such as sex, race, pain intensity score, and age are displayed in Figure 2.

Progression-free survival was significantly longer in the erlotinib and gemcitabine arm than the placebo and gemcitabine arm with an estimated HR of 0.77 (95% CI, 0.64 to 0.92; P = .004; log-rank test stratified for performance status, extent of disease, and pain score at baseline; median, 3.75 months v 3.55 months; Fig 1B).So, what do we have overall? We have a hazard ratio of dying that very nearly crosses 1.0, thus coming perilously close to not disproving the null; we have a prolongation of median survival by 1/3 of a month, and a progression-free median survival prolongation by 1/5 of a month.
But given my fondness for Gould's "The Median is not the Message" essay, let's practice full disclosure and look at the tail of the Kaplan-Meyer curves above. As the text points out,
...1-year survival rates of 23% (95% CI, 18% to 28%) and 17% (95% CI, 12% to 21%), respectively (P = .023).Indeed, these are significant differences, both statistically and clinically. Within the trial this represents the difference in favor of survival for 18 additional patients, thus rendering the cost of roughly $1.5 million for 1 year of life saved by my back-of-the envelope calculation. Of course, this difference is not adjusted for confounders, so it is difficult to say of the number is real of under- or over-estimated. Because the adjusted analysis is given as a hazard ratio of death, I cannot calculate the corresponding adjusted cost.
So, for me this begs the following question (and I would love to hear the thoughts from my colleagues who proudly proclaim being science-based and thus eschewing placebo effect as a valid way to get a therapeutic response): Is what we are seeing here real or is this in fact equivalent to a placebo effect? Is the median survival prolongation of <2 weeks indeed a real effect that means something to the patient and the clinician, or is it just clinical noise, if you will? In other words, should "disproving" the statistical null by default disprove the clinical null? Or does the bar for disproving the clinical null need to be set just a tad higher than a statistically significant increase in life expectancy of 2 weeks?
Obviously, no one knows a priori who will do better and who will not. So, without a crystal ball, it is one's values and preferences that have to drive these decisions. But does the society have a say in any of this, as we struggle with equitable distribution of a limited resource? Is $1.5 million for 1 year of life a good societal investment? And what is the quality of this life? And given that at least 1/2 of all treated patients get far less than extra 2 weeks of life, how do we strike the balance between a reasonable expectation of a response and a false hope?
My friend's family based their choices on their mother's wishes and their values and utilities for her comfort. The choice they made was very different from that made by my parents with regard to my father's palliation. Both were right for the respective families. But one may have been far too costly, both financially and emotionally.
Tuesday, April 6, 2010
Evidence: What the bleep do we really know?
So I know that my blogging has fallen off a bit, and I am sure you are all sorely disappointed (or not). Let me try to explain why.
There are several things going on to nudge me toward the next rung in the evolution of my thinking about healthcare and evidence. The event most responsible for this re-examination of my assumptions is the previously-mentioned illness of a loved one, along with my need to be closely involved with his care decisions. I think that I have generally adequately voiced my frustration with how decisions are made in our healthcare system, and where the switches for these choices should be as opposed to where they are. I am even more convinced now that by the time the physician and the family are considering expensive options with marginal effectiveness, the cat is already out of the bag: how can either the distraught family or the committed healthcare provider not consider those, despite shaky evidence of the value?
But this is not all. I have actually been cogitating the entire way we do evidence and evidence-based medicine. When we invoke evidence, we generally talk about some scientific study's findings, or a group of studies either showing similar or differing results. Let's keep stepping back and looking at the components. The scientific community, based on some statistical and other methodological considerations has come to a consensus around what constitutes valid study designs. This consensus is based on a profound understanding of the tools available to us to answer the questions at hand. The key concept here is that of "available tools". As new tools become available, we introduce them into our research armamentarium to go deeper and further. What we need to appreciate, however, is that "deeper" and "further" are directional words: they imply the same direction as before, only beyond the current stopping point. This is a natural way for us to think, since even our tools are built on the foundation of what has been used previously.
So, what emerges is a picture of being emotionally committed to a certain way of viewing the question, the processes of answering it and the actual answers. And what if by narrowing ourselves to this one particular way of examining the world, to this one particular way to collect and interpret the evidence, we have effectively ignored all other possibilities?
Let me try to clarify what I mean a little further. We spend our days walking through waves. Some of these waves we can detect through our own senses: sound, light, for example. Others we need special external "receptors" to detect, such as radio and micro-waves. Is it possible that there are some other, heretofore unknown waves (or other phenomena) that are around us that we are unable to attune to because of our biology? Is it kooky to think that this is possible, or is it simply blind to walk away from such possibility?
The answer may be that it is both. Nevertheless, it is highly likely that there are many biological phenomena that are not amenable to being examined through our narrow prism of current methodology. We as scientists and clinicians need to be open to this possibility. The Buddha said that both preachers and scholars are blind because they constantly commit themselves to partially-seen truths. Given the shambles in which we find not only our nation's healthcare system, but also the health of its citizens, this would be a great time for this epiphany to penetrate our collective psyche and strive toward a broader view of possibilities. Let's give up the arrogance of ignorance in favor of the humbleness of enlightenment.
And these are my latest thoughts. I am not saying that they are the only way or that they are even remotely correct. But here they are nevertheless. Do with them what you will.
There are several things going on to nudge me toward the next rung in the evolution of my thinking about healthcare and evidence. The event most responsible for this re-examination of my assumptions is the previously-mentioned illness of a loved one, along with my need to be closely involved with his care decisions. I think that I have generally adequately voiced my frustration with how decisions are made in our healthcare system, and where the switches for these choices should be as opposed to where they are. I am even more convinced now that by the time the physician and the family are considering expensive options with marginal effectiveness, the cat is already out of the bag: how can either the distraught family or the committed healthcare provider not consider those, despite shaky evidence of the value?
But this is not all. I have actually been cogitating the entire way we do evidence and evidence-based medicine. When we invoke evidence, we generally talk about some scientific study's findings, or a group of studies either showing similar or differing results. Let's keep stepping back and looking at the components. The scientific community, based on some statistical and other methodological considerations has come to a consensus around what constitutes valid study designs. This consensus is based on a profound understanding of the tools available to us to answer the questions at hand. The key concept here is that of "available tools". As new tools become available, we introduce them into our research armamentarium to go deeper and further. What we need to appreciate, however, is that "deeper" and "further" are directional words: they imply the same direction as before, only beyond the current stopping point. This is a natural way for us to think, since even our tools are built on the foundation of what has been used previously.
So, what emerges is a picture of being emotionally committed to a certain way of viewing the question, the processes of answering it and the actual answers. And what if by narrowing ourselves to this one particular way of examining the world, to this one particular way to collect and interpret the evidence, we have effectively ignored all other possibilities?
Let me try to clarify what I mean a little further. We spend our days walking through waves. Some of these waves we can detect through our own senses: sound, light, for example. Others we need special external "receptors" to detect, such as radio and micro-waves. Is it possible that there are some other, heretofore unknown waves (or other phenomena) that are around us that we are unable to attune to because of our biology? Is it kooky to think that this is possible, or is it simply blind to walk away from such possibility?
The answer may be that it is both. Nevertheless, it is highly likely that there are many biological phenomena that are not amenable to being examined through our narrow prism of current methodology. We as scientists and clinicians need to be open to this possibility. The Buddha said that both preachers and scholars are blind because they constantly commit themselves to partially-seen truths. Given the shambles in which we find not only our nation's healthcare system, but also the health of its citizens, this would be a great time for this epiphany to penetrate our collective psyche and strive toward a broader view of possibilities. Let's give up the arrogance of ignorance in favor of the humbleness of enlightenment.
And these are my latest thoughts. I am not saying that they are the only way or that they are even remotely correct. But here they are nevertheless. Do with them what you will.
Monday, January 11, 2010
Evidence-based airport security?
Backscatter scanners cost $150,000-$180,000 a piece. Congress has appropriated funding for 450 scanners to be placed in the US airports. By my simple back-of-the-envelope calculation we are talking $67,500,000-$81,000,000 of our taxpayer dollars. How do we wrap our brains around the value proposition of this investment?
Well, in order to do this we need to know something about what outcome we are trying to affect and the impact of this machine on this outcome of interest. Seems like the most sensible outcome of interest is deaths averted due to airplane-related terrorist activity. According to Nate Silver at fivethirtyeight.com, in the first decade of the 21st century, factoring out the 9/11 fatalities, there were about 200 deaths related to violent incidents on board of commercial aircraft in the entire decade. So, assuming that the scanners' effectiveness is 100% (that is that it can prevent any violent act aboard an airplane that would result in any number of deaths), the cost to avert one death is $337,500-$405,000 over 10 years (this is not taking into account either inflation or discounting for future events).
If we had a medical technology with the same cost-effectiveness profile, would we think it reasonable to pay for it? More importantly, would we get congressional appropriations to pay for it even in the absence of any effectiveness data? What we know and what we think we know about this are widely divergent. What we know is that the scans can detect certain culprits of potential acts of terrorism. What we do not know is whether this level of detection will indeed result in aversion of death. After all, had we not detected prior to the Detroit flight that Umar Farouk Abdumutallab was a potential threat to US citizens? We had, but the information was not acted upon. Similarly with the scanners, they may have efficacy in detecting the threat, but how this information is utilized is sure to impact their effectiveness as far as the endpoint of interest: death prevention. So, reaching for a new expensive and potentially more invasive technology in this case is a bit like blaming one's failure to treat a cancer detected on a mammogram on unavailability of the more sensitive MRI technology.
There seem to many points of faulty logic in this undertaking of heightened airport security. Not only do we need to re-examine critically our assumptions, but also we have to assess soberly whether the investments we have made to date are in the right places. Remember the cold war? It was all about getting a leg up in the nuclear arms race. "You build a better missile, we will build a missile shield. Then you will build an even better missile that will penetrate the shield." And so on, and so on. It is much like laws, upon which clever corporate lawyers rely for loop holes in favor of their clients' misdeeds.
As much as anyone, I would like international terrorism, just like ventilator-associated pneumonia and other hospital-acquired complications, to become a "zero event". Unfortunately, I am all too keenly aware that wishing something to be true only makes it so in fairy tales. The dearth of evidence to support many of the expensive anti-terrorism interventions is concerning. Poor logic, erroneous assumptions and unjustified inferences have been driving our decisions for too long. The public should demand the same level of evidentiary support for astronomical "anti-terrorism" appropriations that we do for healthcare.
Well, in order to do this we need to know something about what outcome we are trying to affect and the impact of this machine on this outcome of interest. Seems like the most sensible outcome of interest is deaths averted due to airplane-related terrorist activity. According to Nate Silver at fivethirtyeight.com, in the first decade of the 21st century, factoring out the 9/11 fatalities, there were about 200 deaths related to violent incidents on board of commercial aircraft in the entire decade. So, assuming that the scanners' effectiveness is 100% (that is that it can prevent any violent act aboard an airplane that would result in any number of deaths), the cost to avert one death is $337,500-$405,000 over 10 years (this is not taking into account either inflation or discounting for future events).
If we had a medical technology with the same cost-effectiveness profile, would we think it reasonable to pay for it? More importantly, would we get congressional appropriations to pay for it even in the absence of any effectiveness data? What we know and what we think we know about this are widely divergent. What we know is that the scans can detect certain culprits of potential acts of terrorism. What we do not know is whether this level of detection will indeed result in aversion of death. After all, had we not detected prior to the Detroit flight that Umar Farouk Abdumutallab was a potential threat to US citizens? We had, but the information was not acted upon. Similarly with the scanners, they may have efficacy in detecting the threat, but how this information is utilized is sure to impact their effectiveness as far as the endpoint of interest: death prevention. So, reaching for a new expensive and potentially more invasive technology in this case is a bit like blaming one's failure to treat a cancer detected on a mammogram on unavailability of the more sensitive MRI technology.
There seem to many points of faulty logic in this undertaking of heightened airport security. Not only do we need to re-examine critically our assumptions, but also we have to assess soberly whether the investments we have made to date are in the right places. Remember the cold war? It was all about getting a leg up in the nuclear arms race. "You build a better missile, we will build a missile shield. Then you will build an even better missile that will penetrate the shield." And so on, and so on. It is much like laws, upon which clever corporate lawyers rely for loop holes in favor of their clients' misdeeds.
As much as anyone, I would like international terrorism, just like ventilator-associated pneumonia and other hospital-acquired complications, to become a "zero event". Unfortunately, I am all too keenly aware that wishing something to be true only makes it so in fairy tales. The dearth of evidence to support many of the expensive anti-terrorism interventions is concerning. Poor logic, erroneous assumptions and unjustified inferences have been driving our decisions for too long. The public should demand the same level of evidentiary support for astronomical "anti-terrorism" appropriations that we do for healthcare.
Tuesday, January 5, 2010
The FDA and drug costs: A health services researcher's epiphany
Happy New Year to everyone, and apologies for not posting for some time. You probably thought I was on vacation, but in fact a family illness has kept me from writing. For reasons of confidentiality, I will not be disclosing the details of it for the moment, but will share certain salient points this experience is clarifying for me.
Take expensive medications, for example. In the last few weeks, the anonymous cost-effectiveness equation of certain interventions has taken on a uniquely personal dimension. Thinking about using a prohibitively expensive medication with limited effectiveness in the abstract derives a much clearer answer than in the case of a loved one. What is a couple of extra months of life and functionality worth to you? And what if it is indeed more than a couple of months, since there are many individual exceptions to average values? Who am I to dissuade my family member from accepting this treatment course, even though I am keenly aware of its fiscal and clinical ramifications?
Here is what I am beginning to think. If even someone as close to health economics as I am cannot say "no", when put in the situation personally, to a therapy that I would academically consider cost-ineffective, how can a lay person faced with such a choice say "no"? If they cannot be expected to say "no", who should? In the spirit of patient empowerment I would certainly not advocate substituted judgment; that is, I would not advise the physician to make this decision unilaterally. And truly, although everyone is interested these days in having MDs take economic aspects into consideration, there is a potential conflict of this stance with the "do no harm" priority.
So, humanistically speaking, who should make this judgment? As you know, I do not believe that we can let the market drive these choices, unless down the road we are willing to spend 100% of our GDP on healthcare! This means that someone does have to say "no". Following the chain of development and marketing of technologies, since neither the MDs nor the patients (or their surrogates) are in a great position to do this, the decision must reside upstream. The most proximate upstream entity is the payor, but look at the political maelstrom that a discussion of limiting payment for existing interventions has precipitated in the US. The FDA is the next stop in this space-time continuum, second only to the manufacturer. This to me seems to be the logical final stop for the decision bus. Beyond the manufacturers themselves, who would be well-served not to invest untold dollars into bringing to market marginally effective exorbitantly priced technologies, the agency is probably the most sensible point in the pathway to close the door to products with questionable value. Today, the FDA is not empowered to consider the value proposition; this role is left to the payor. The FDA can only make judgments on efficacy and safety of technologies under evaluation. Could we change that? Would we want to? Is this the right solution?
In the meantime, we will do the trial of this expensive therapy and see what happens. Maybe it will buy more time than we think. Then again, maybe not.
Take expensive medications, for example. In the last few weeks, the anonymous cost-effectiveness equation of certain interventions has taken on a uniquely personal dimension. Thinking about using a prohibitively expensive medication with limited effectiveness in the abstract derives a much clearer answer than in the case of a loved one. What is a couple of extra months of life and functionality worth to you? And what if it is indeed more than a couple of months, since there are many individual exceptions to average values? Who am I to dissuade my family member from accepting this treatment course, even though I am keenly aware of its fiscal and clinical ramifications?
Here is what I am beginning to think. If even someone as close to health economics as I am cannot say "no", when put in the situation personally, to a therapy that I would academically consider cost-ineffective, how can a lay person faced with such a choice say "no"? If they cannot be expected to say "no", who should? In the spirit of patient empowerment I would certainly not advocate substituted judgment; that is, I would not advise the physician to make this decision unilaterally. And truly, although everyone is interested these days in having MDs take economic aspects into consideration, there is a potential conflict of this stance with the "do no harm" priority.
So, humanistically speaking, who should make this judgment? As you know, I do not believe that we can let the market drive these choices, unless down the road we are willing to spend 100% of our GDP on healthcare! This means that someone does have to say "no". Following the chain of development and marketing of technologies, since neither the MDs nor the patients (or their surrogates) are in a great position to do this, the decision must reside upstream. The most proximate upstream entity is the payor, but look at the political maelstrom that a discussion of limiting payment for existing interventions has precipitated in the US. The FDA is the next stop in this space-time continuum, second only to the manufacturer. This to me seems to be the logical final stop for the decision bus. Beyond the manufacturers themselves, who would be well-served not to invest untold dollars into bringing to market marginally effective exorbitantly priced technologies, the agency is probably the most sensible point in the pathway to close the door to products with questionable value. Today, the FDA is not empowered to consider the value proposition; this role is left to the payor. The FDA can only make judgments on efficacy and safety of technologies under evaluation. Could we change that? Would we want to? Is this the right solution?
In the meantime, we will do the trial of this expensive therapy and see what happens. Maybe it will buy more time than we think. Then again, maybe not.
Subscribe to:
Posts (Atom)




