Wednesday, September 29, 2010

Disruptive innovation in healthcare: Overcoming HTE

I have been working on a talk for the American College of Chest Physicians (Chest) annual meeting. The session is on alternative research study designs, and I chose to talk about the N of 1 trials. I think that you can get an idea of why I chose this topic from reading my previous posts here, here and here. Doing my research, I came upon a term "heterogeneous treatment effect", or HTE, that is well worth exploring. I think that every clinician who has ever seen patients is familiar with this effect, but let us trace its explanation.

This excellent article in the UK Independent summarizes the premiss with some scathing comments from none other than GSK's chief geneticist, Dr. Allen Roses:
"The vast majority of drugs - more than 90 per cent - only work in 30 or 50 per cent of the people," Dr Roses said. "I wouldn't say that most drugs don't work. I would say that most drugs work in 30 to 50 per cent of people. Drugs out there on the market work, but they don't work in everybody."
There is even a table presented with response rates by therapeutic area, though the reference(s) is(are) not cited, so, please, take with a grain of salt:
Response rates
Therapeutic area: drug efficacy rate in per cent
  • Alzheimer's: 30
  • Analgesics (Cox-2): 80
  • Asthma: 60
  • Cardiac Arrythmias: 60
  • Depression (SSRI): 62
  • Diabetes: 57
  • Hepatits C (HCV): 47
  • Incontinence: 40
  • Migraine (acute): 52
  • Migraine (prophylaxis): 50
  • Oncology: 25
  • Rheumatoid arthritis: 50
  • Schizophrenia: 60
In essence, what Dr. Roses was referring to is the phenomenon of HTE, described aptly by Kent et al. as the fact that "the effectiveness and safety of a treatment varies across the patient population". The authors preface it by saying that 
Although “evidence-based medicine” has become the dominant paradigm for shaping clinical recommendations and guidelines, recent work demonstrates that many clinicians’ initial concerns about “evidence-based medicine” come from the very real incongruence between the overall effects of a treatment in a study population (the summary result of a clinical trial) and deciding what treatment is best for an individual patient given their specific condition, needs and desires (the task of the good clinician). The answer, however, is not to accept clinician or expert opinion as a replacement for scientific evidence for estimating a treatment’s efficacy and safety, but to better understand how the effectiveness and safety of a treatment varies across the patient population (referred to as heterogeneity of treatment effect [HTE]) so as to make optimal decisions for each patient.
Ah, so it is not your imagination: when someone brings an evidence-based guideline to you, and insists that unless you comply 95% of the time, you are providing less than great quality of care, and you say "this does not represent my patients", you are actually not crazy. To be sure, a good EBPG will apply to most patients encountered with the particular condition. But the devil, of course is in the details. As I have already pointed out, we impose statistical principles onto data to whip it into submission. When we do a good job, we acknowledge the limitations of what measures of central tendency provide us with. But so much of the time I see physicians relying on the p value alone to compare the effects, that I am convinced that the variation around the center is mostly lost on us. And further, how does this variance help a clinician faced with an individual patient who has at best a probability of response on some continuum of a population of probabilities? And more importantly, what will this individual patient's risk-benefit balance be for a particular therapy?

I think what I am walking away with after thinking about this issue is that it is of utmost importance to understand what kind of data have gone into a recommendation. What is the degree of HTE in the known research, and specifically, what is known about the population that your patient represents. The less HTE and the more knowledge about the specific subgroups, the more confident you can be that the therapy will work. Ultimately, however, each patient is a universe onto herself, since no two people will share same genetics, environmental exposures, chronic condition profile or other treatments, to name just a few potential characteristics that may impact response to therapy.

This is the reason that we need better trials, where people are represented more broadly, leading to an increase in external validity. To make this information useful at the bedside, we need a priori plans to analyze many different subgroups, as that will give clinicians at least some granularity so desperately needed in the office. And while pharmacogenomics may be helpful, I am sure that it will not be the panacea for reducing all of this complexity to zero.

Until technology gives us a better way (assuming that it will), where possible, a systematic approach to treatment trials should be undertaken. Later I will blog about N of 1 trials, which, though not appropriate in every situation, may be quite helpful in optimizing treatment in some chronic conditions. With the advent of health IT, these trials may become less daunting and, in aggregate, provide some very useful generalizable information on what happens in the real world. Each clinician will need to take some ownership in advancing our collective understanding of the diseases s/he treats. This may truly be the disruptive innovation we are all looking for to improve the quality of care not just to please the bureaucrats, but to promote better health and quality of life.      



  1. Another great post. Thanks for showing me this. You may have addressed this in previous posts, but I have wondered recently why published studies do not present all their raw data openly on the internet, surely the capability exists. From what I've seen this has begun to occur modestly eg. the BMJ has done this for some data in some studies. But why not all? There are still discussions like this one in Pediatrics 2009; 123:e544 where someone makes a comment and the authors go away and retorture their data. Why not make the data available to all? It would show up the curse of data mining if authors were obliged to report all their data, not only those consistent with the authors' desired outcomes (although I suppose it might encourage some data mining by anyone looking at the data, aaargh). I saw this obligation once recommended for drug trials at least.

    I agree that Health IT has the potential to greatly assist this particular issue of differing patient responses. When I'm asked why Health IT is so very slow to develop (in Australia at least) I don't have a good answer; there are privacy concerns but I don't think that is the only reason and I'm sure much better technology is available than that which we use. Bit of a mystery for me that.

    I always have a political ear out and the item from The Independent presented great information; they also just couldn't help angling it as a reflection on the dastardly deeds of big pharma.

  2. TheyKilledKenny,

    Yeah, posting the data is what the Open Data movement is about. People like Pew Research are doing it - no ax to grind there.

    Marya, so glad to meet you - thanks for visiting You popped up at a perfect moment. I sure hope we-all can wake up "the establishment," whoever that is.

  3. Hi, Dave! Great o have you join this discussion. Janice McCallum had some very interesting things to say in her comments to today's post. Would be very interested in your reaction to the conversation.


  4. Almost 2 years later, we've met and I've read your book and now I just ran across this post again, because I linked to it in a comment on another blog. The social media circle is complete!

    And today, almost 2 years later, I think this is one of the most important single factoids for EVERYONE in healthcare to realize - clinicians and patients alike. Amazing - the illusion of certainty is a huge balloon that needs to get popped.

  5. (Here's the comment, over on the RA Warrior blog ...