Disruptive innovation in healthcare: Overcoming HTE

I have been working on a talk for the American College of Chest Physicians (Chest) annual meeting. The session is on alternative research study designs, and I chose to talk about the N of 1 trials. I think that you can get an idea of why I chose this topic from reading my previous posts here, here and here. Doing my research, I came upon a term "heterogeneous treatment effect", or HTE, that is well worth exploring. I think that every clinician who has ever seen patients is familiar with this effect, but let us trace its explanation.

This excellent article in the UK Independent summarizes the premiss with some scathing comments from none other than GSK's chief geneticist, Dr. Allen Roses:

"The vast majority of drugs - more than 90 per cent - only work in 30 or 50 per cent of the people," Dr Roses said. "I wouldn't say that most drugs don't work. I would say that most drugs work in 30 to 50 per cent of people. Drugs out there on the market work, but they don't work in everybody."

There is even a table presented with response rates by therapeutic area, though the reference(s) is(are) not cited, so, please, take with a grain of salt:

Response rates

Therapeutic area: drug efficacy rate in per cent

• Alzheimer's: 30
• Analgesics (Cox-2): 80
• Asthma: 60
• Cardiac Arrythmias: 60
• Depression (SSRI): 62
• Diabetes: 57
• Hepatits C (HCV): 47
• Incontinence: 40
• Migraine (acute): 52
• Migraine (prophylaxis): 50
• Oncology: 25
• Rheumatoid arthritis: 50
• Schizophrenia: 60

In essence, what Dr. Roses was referring to is the phenomenon of HTE, described aptly by Kent et al. as the fact that "the effectiveness and safety of a treatment varies across the patient population". The authors preface it by saying that 

Although “evidence-based medicine” has become the dominant paradigm for shaping clinical recommendations and guidelines, recent work demonstrates that many clinicians’ initial concerns about “evidence-based medicine” come from the very real incongruence between the overall effects of a treatment in a study population (the summary result of a clinical trial) and deciding what treatment is best for an individual patient given their specific condition, needs and desires (the task of the good clinician). The answer, however, is not to accept clinician or expert opinion as a replacement for scientific evidence for estimating a treatment’s efficacy and safety, but to better understand how the effectiveness and safety of a treatment varies across the patient population (referred to as heterogeneity of treatment effect [HTE]) so as to make optimal decisions for each patient.

Ah, so it is not your imagination: when someone brings an evidence-based guideline to you, and insists that unless you comply 95% of the time, you are providing less than great quality of care, and you say "this does not represent my patients", you are actually not crazy. To be sure, a good EBPG will apply to most patients encountered with the particular condition. But the devil, of course is in the details. As I have already pointed out, we impose statistical principles onto data to whip it into submission. When we do a good job, we acknowledge the limitations of what measures of central tendency provide us with. But so much of the time I see physicians relying on the p value alone to compare the effects, that I am convinced that the variation around the center is mostly lost on us. And further, how does this variance help a clinician faced with an individual patient who has at best a probability of response on some continuum of a population of probabilities? And more importantly, what will this individual patient's risk-benefit balance be for a particular therapy?

I think what I am walking away with after thinking about this issue is that it is of utmost importance to understand what kind of data have gone into a recommendation. What is the degree of HTE in the known research, and specifically, what is known about the population that your patient represents. The less HTE and the more knowledge about the specific subgroups, the more confident you can be that the therapy will work. Ultimately, however, each patient is a universe onto herself, since no two people will share same genetics, environmental exposures, chronic condition profile or other treatments, to name just a few potential characteristics that may impact response to therapy.

This is the reason that we need better trials, where people are represented more broadly, leading to an increase in external validity. To make this information useful at the bedside, we need a priori plans to analyze many different subgroups, as that will give clinicians at least some granularity so desperately needed in the office. And while pharmacogenomics may be helpful, I am sure that it will not be the panacea for reducing all of this complexity to zero.

Until technology gives us a better way (assuming that it will), where possible, a systematic approach to treatment trials should be undertaken. Later I will blog about N of 1 trials, which, though not appropriate in every situation, may be quite helpful in optimizing treatment in some chronic conditions. With the advent of health IT, these trials may become less daunting and, in aggregate, provide some very useful generalizable information on what happens in the real world. Each clinician will need to take some ownership in advancing our collective understanding of the diseases s/he treats. This may truly be the disruptive innovation we are all looking for to improve the quality of care not just to please the bureaucrats, but to promote better health and quality of life.      

 

"Clinical Decision Making: Evidence Is Never Enough"

A senior resident, a junior attending, a senior attending, and an emeritus professor were discussing evidence-based medicine (EBM) over lunch in the hospital cafeteria.
"EBM," announced the resident with some passion, "is a revolutionary development in medical practice." She went on to describe EBM's fundamental innovations in solving patient problems.
"A compelling exposition," remarked the emeritus professor.
"Wait a minute," the junior attending exclaimed, also with some heat, and presented an alternative position stating that EBM merely provided a set of additional tools for traditional approaches to patient care.
"You make a strong and convincing case," the emeritus professor commented.
"Wait a minute," the senior attending exclaimed to her older colleague, "their positions are diametrically opposed. They can't both be right."  

The emeritus professor looked thoughtfully at the puzzled physician and, with the barest hint of a smile, replied, "Come to think of it, you're right too."

So begins the Users' Guide to Medical Literature issue #XXV, "Principles of Applying the Users' Guides to Patient Care", published 10 years ago, almost to the day in JAMA (JAMA 2000;13:284;1290-6). I am enjoying re-reading it, as I now appreciate the authors' evolving positions acknowledging the complexities of modern practice and the need to hold together apparent dualities. They acknowledge their continued commitment to the scientific rigor of EBM:

In 1992, in an article that provided a background to the Users' Guides, we described EBM as a shift in medical paradigms.² In contrast to the traditional paradigm, EBM acknowledges that intuition, unsystematic clinical experience, and pathophysiologic rationale are insufficient grounds for clinical decision making, and stresses the examination of evidence from clinical research. The philosophy underlying EBM suggests that a formal set of rules must complement medical training and common sense for clinicians to effectively interpret the results of clinical research. Finally, EBM places a lower value on authority than the traditional paradigm of medical practice.

However, they point out that the complexities of the world often call for more than one way of approaching problems:

While we continue to find the paradigm shift a valid way of conceptualizing EBM, as the scenario suggests, the world is often complex enough to invite more than 1 useful way of thinking about an idea or a phenomenon. In this article, we describe the 2 key principles that clinicians must grasp to be effective practitioners of EBM. One of these relates to the value-laden nature of clinical decisions; the other to the hierarchy of evidence postulated by EBM. 

A few paragraphs into the paper, we come to a surprising heading, which serves as the title for today's post: "Clinical Decision Making: Evidence Is Never Enough". Wow, is this not a bit heretical, especially coming from the cradle of EBM? What do the authors mean by this? To illustrate this they develop three scenarios, each involving a patient with pneumococcal pneumonia, a disease for the treatment of which plenty of strong evidence exists. Yet, depending on the context of the patient's situation, treatment decisions may or may not be straightforward. Their examples of a terminal cancer patient who herself may forgo treatment, an elderly demented nursing home patient with no family, whose treatment decisions have to be made solely by the clinicians, and a 30-year-old mother of two, whom only Lord Voldemort would consign to non-treatment. The point being that evidence of effectiveness, in this case for the treatment of pneumococcal pneumonia, is necessarily applied to the particular circumstance at the bedside. In other words, some value judgments are employed to apply the existing evidence to a trade-off decision. Quantifying this process is far from a clear science:

Acknowledging that values play a role in every important patient care decision highlights our limited understanding of eliciting and incorporating societal and individual values. Health economists have played a major role in developing a science of measuring patient preferences.^14-15 Some decision aids are based on theassumption that if patients truly understand the potential risks and benefits, their decisions will reflect their preferences.¹⁶ These developments constitute a promising start. Nevertheless, many unanswered questions concerning how to elicit preferences, and how to incorporate them in clinical encounters already subject to crushing time pressures, remain. Addressing these issues constitutes an enormously challenging frontier for EBM.

Switching gears somewhat to delve into some of the pragmatic issues, the authors talk about the hierarchy of evidence, acknowledging the superiority of systematic clinical observation over ad hoc anecdotal evidence as a tool to reduce common threats to inference validity. However, they loudly and clearly state that in practice one size does not fit all:

Clinical research goes beyond unsystematic clinical observation in providing strategies that avoid or attenuate the spurious results. Because few, if any, interventions areeffective in all patients, we would ideally test a treatment in the patient to whom we would like to apply it. Numerous factors can lead clinicians astray as they try to interpret the results of conventional open trials of therapy, which include natural history, placebo effects, patient and health worker expectations, and the patient's desire to please.

... and offer the following hierarchy of evidence for individual treatment decisions:

Table 1. A Hierarchy of Strength of Evidence for Treatment Decisions

So, for a patient encounter, the most robust kind of evidence is the N of 1 trial! Not too coincidentally, I came to the same conclusion here. With a few caveats, the authors confirm the feasibility of this undertaking:

N of 1 RCTs are unsuitable for short-term problems; for therapies that cure (such as surgical procedures); for therapies that act over long periods of time or prevent rare or unique events (such as stroke, myocardial infarction, or death); and are possible only when patients and clinicians have the interest and time required. However, when the conditions are right, N of 1 RCTs are feasible,^24-25 can provide definitive evidence of treatment effectiveness in individual patients, and may lead to long-term differences in treatment administration.²⁶

They go on to say that applying results based on group data may be fraught with erroneous assumptions:

When considering any source of evidence about treatment other than N of 1 RCTs, clinicians are generalizing from results in other people to their patients, inevitably weakening inferences about treatment impact and introducing complex issues of how trial results apply to individuals.

They again stress the importance of the individual encounter and making collaborative decisions in the context of the patient's values:

Thus, knowing the tools of evidence-based practice is necessary but not sufficient for delivering the highest-quality patient care. In addition to clinical expertise, the clinician requires compassion, sensitive listening skills, and broad perspectives from the humanities and social sciences. These attributes allow understanding of patients' illnesses in the context of their experience, personalities, and cultures.

They rightfully treat scientific evidence as the foundation of practice, but not the whole of it. Indeed, they conclude that

A continuing challenge for EBM, and for medicine in general, will be to better integrate the new science of clinical medicine with the time-honored craft of caring for the sick.

I really like the tempered tone of this paper, paying homage not only to the science, but also to the art of medicine. It is clear that the authors have evolved with the field, and if these giants can evolve, so can we. Clearly, in the art of medicine, "evidence is never enough".