What-based medicine?

Evidence-based medicine started out with Archie Cochrane in 1972 and really hit big time with David Sackett, David Eddy and others in the '90s. While initially meant to ensure equitable use of a limited resource through evidence-based decisions, EBM now drives reimbursement, quality metrics, MD grades, and opinion. As with market success of a drug thrust into a generally healthy population, where safety signals are bound to overshadow any measurable benefit (think Vioxx), there are likewise unintended consequences to such widespread adoption, or we might even say bastardization, of EBM.

Here is how I see it. EBM has given the opportunity to develop markets, vast markets of diseases that would not have existed or been given the time of day without it. "Diseases" such as pre-diabetes, pre-hypercholesterolemia, pre-hypertension, pre-osteoporosis, all have been brought to our consciousness through the engine of EBM. The "E" in EBM is what is so poorly defined, and your E may not be my E, as keenly illustrated in several recent editorials in both the popular and scientific journals. Where we have arrived is at the lowest common denominator of E: statistical significance. The argument goes that a response statistically different from that to a placebo constitutes evidence of efficacy. Indeed, this is what our regulatory agencies rely on in their deliberations of market approvals. Once on the market, the Wild West of the marketing and prescribing giant is rarely held to a higher standard than the proof obtained in the laboratory of a clinical trial. Yet, those of us familiar with the methods realize that a randomized controlled clinical trial, no matter how well executed, is but the beginning of gathering evidence of effect. Just look at a typical trial's list of inclusion and exclusion criteria. It is no wonder that the worn meme of "not my patient" is so prevalent among docs being handed this new "evidence".

Several mechanisms have been developed to overcome this limitation of pre-approval trials. One is a meta-analysis, where many similarly designed trials contribute the data to arrive at a point estimate for the effect in a much larger population. The issue with this is that it in no way helps us understand what the intervention does in the real world to a real individual patient. In fact, a meta-analysis can make us a bit more certain about the consistency of the magnitude of this effect, but will in no way diverge from what is seen in the controlled environment of a trial. And we know that in most instances therapies lose in the magnitude of their effectiveness when tested in the real world setting.

Another interesting discussion happening in the literature is about heterogeneous treatment effect (HTE), which I have discussed here. The idea is that a study reports the aggregate findings through measures of central tendency, such as the means and medians, as well as some of the scatter around them. What these numbers fail to tell us is how an individual patient can be expected to respond to a particular intervention. The inherent inter-individual variability in disease and response, as well as intra-individual variability in these factors over time pose major challenges in interpreting the single curve presented in a clinical trial report. Although methodologically rigorous subgroup analyses have been proposed as a potential solution to this conundrum, they are a thorn in the side of statisticians due to their implications for probability of errors, and, even when done well, are still not likely to get down to the level of granularity needed at the bedside. And of course, if a trial is not representative of the population that the intervention will eventually address in the real world, then all of these machinations will yield rather limited information.

So, what are we left with? We are still left with "E" that is far from perfect, yet it is this "E" that drives the business of medicine. I was reminded of this by this op-ed from Gil Welch from Dartmouth, where he mentions screening mammography as a metric that feeds into publicly reported MD grades. Clearly, the issue of screening mammography is not at all settled, yet the "evidence" is driving an important policy.

I hope you, my reader, do not think that I am an opponent of EBM. I have seen the dirty underbelly of business-based medicine, where an expedient diagnosis of a common ailment provides a steady revenue stream from subsequent treatment and complications associated with it. I have witnessed willfullness-based medicine and laziness-based medicine. I have also seen medicine practiced with compassion and utmost respect for the nexus of evidence and the individual patient. It is the dogmatic know-it-all approach to our evolving understanding of human health that gets us into trouble every time. Patients are not happy, clinicians are not happy, and even the bureaucrats are beginning to feel the sting of EBM's unbridled success. Men like Cochrane, Eddy and Sackett understood the nuances of the clinical encounter and advocated that it take place within the context of EBM, not that EBM replace these nuances. The bastardization of their intent is palpable and fraught. We will be reaping the fruit of their success for a long time to come.