Tipping a sacred cow: QI under the microscope

So much media and journal space has been devoted to financial conflicts of interest, particularly within and related to pharma and device manufacturers, that to write any more about it may be redundant. On this site we have also intermittently addressed COI from other perspectives, such as financial interest of the members of the American College of Radiology in maintaining mammography screening status quo, thinly veiled in its own version of the pernicious "death panel" language. We have also spoken a bit about the non-financial COI. And even though we are so very much aware of COI's potential to lurk around every corner, there are still some surprises.

Take the sacred cow of "quality improvement" in healthcare. Even the name, much like the "pro life" moniker, suggests that it is untouchable in its purity and nobility of purpose. So necessary is it because of the epic magnitude of morbidity and mortality attributed to healthcare itself, that the billions of dollars spent on it seem unquestionably justified. Indeed, much like our public education system, the QI movement garners higher and higher allocations simply due to the sheer face validity of the assumption that more of it is better. And the most fascinating aspect is that, in our current zeal for sensible economic allocation through evidence, QI, much like education, appears immune to scrutiny. This is the very definition of politics driving policy.

I return to the case of ventilator-associated pneumonia, or VAP, as the poster child for this movement. I have already alluded to the fact that definitionally VAP is a slippery slope: its diagnosis varies based not only on the tools used to diagnose it, but also depending on who is doing the diagnosing. Yes, indeed, what one clinician calls VAP another may call absence of VAP. I have also dissected the weak evidence behind some of the strongest purportedly evidence-based recommendations aimed at VAP prevention. But what if VAP itself is the wrong endpoint? What if we are spending untold dollars and other resources on a futile pursuit?

Do you feel yourself bristling yet? If you said "yes", it is a normal response I get from my colleagues and people who read my scholarly papers. Because how can anyone be against QI? Well, I am not against QI. I am simply against sanctifying QI as a sacred cow and thus shielding it from a sensible and rational evaluation.

So, if you are over the initial shock, allow me to explain myself. I am sure you have heard of surrogate endpoints. Here is a definition from Wikipedia:

In clinical trials, a surrogate endpoint (or marker) is a measure of effect of a certain treatment that may correlate with a real clinical endpoint but doesn't necessarily have a guaranteed relationship. The National Institutes of Health (USA) defines surrogate endpoint as "a biomarker intended to substitute for a clinical endpoint".^[1][2]

Surrogate markers are used when the primary endpoint is undesired (e.g., death), or when the number of events is very small, thus making it impractical to conduct a clinical trial to gather a statistically significant number of endpoints. The FDA and other regulatory agencies will often accept evidence from clinical trials that show a direct clinical benefit to surrogate markers. ^[3]

This begs the question of what constitutes a "real" clinical endpoint. Well, in my simplemindedness I think of them as endpoints that matter to the patient or in the long run. So, death, disability, quality of life, functionality, these are the real endpoints. Something that alters one's life or threatens it is a real endpoint. Thus, blood pressure and cholesterol are surrogate endpoints, since they usually, but not always, correlate with the risk of a myocardial infarction or death. But what if such a correlation did not exist? Furthermore, what if a cholesterol level was measured with, say, tea leaves, and therefore was subject to a tremendous variation in detection? Would we then spend hundreds of billions of dollars on trying to alter this factor or would we calmly and rationally walk away and look for something that truly impacts the real outcome of a heart attack or death? I think I am making my point fairly clearly.


Let me explain why I think that VAP is but a surrogate outcome, and, given its diagnostic challenges, not a sensible one in the least. VAP by definition occurs in patients on mechanical ventilation (breathing machine), whose quality of life is fairly badly damaged in the short term. The literature would suggest that not all VAP impacts mortality adversely, but some forms of VAP indeed do, particularly VAP that develops late in the course of illness. So in this VAP does correlate with a real endpoint. Also, there is very little doubt that getting VAP prolongs one's dependence on mechanical ventilation, and increases the duration of the stay in the ICU and hospital overall. So, this can be considered not a very good, albeit real, outcome. An additional point to remember is that VAP engenders the use of additional, usually broad spectrum, antibiotics, putting both the individual and the society at risk for such unwanted consequences as the emergence of highly resistant microorganisms.


So, even though VAP is a surrogate endpoint, it certainly seems to fit the bill for something we would want to prevent. But here is the monkey wrench in this argument: what seem to be great surrogate endpoints do not always end up correlating with clinical reality. The association of VAP with morbidity and mortality has been detected in mostly retrospective observational studies. Trials of VAP prevention rarely, if ever, report any endpoint other than VAP. And, given how elusive VAP diagnosis is, there is plenty of room for gamesmanship so pervasive in the real world to make any data fit our preconceived hypotheses and political needs. 


So, what is my point? My point is that if QI wants to be a science, it needs to be subject to the same rules that all other science is guided by. Since we do not even know how much money we are spending on the ubiquitous QI efforts (likely hundreds of billions), and since we are not sure what they are accomplishing (see my many prior posts on the lack of validity of current claims in VAP prevention), we need to pause and ask ourselves whether the cheering alone justifies such an investment. I hate to say it, but can we really trust those with most to lose, financially and politically, if in reality QI does little more than lather the masses, to be the oracles of truth about the results of these efforts? The cognitive biases alone should disqualify them from being the arbiters of their own success. So, if we do not want to continue to indulge the principle of diminishing returns in QI, we need to take a sober look at what we have invested and what this investment has accomplished. Then and only then can we claim to practice evidence- rather than politics-or dogma-based policy.

Why we need collaborations across healthcare sectors

I want to digress from our recent focus on methods and talk a bit about conflict of interest (COI for short). There has been a lot in the press lately about doctors taking money from the biopharmaceutical manufacturers, and doctors inserting unnecessary hardware into patients' hearts and spines. All of this has been happening against the background of a low hum of an ongoing discussion of what constitutes a COI, how much is too much and for what (for example, can a doc who takes research and education dollars from a manufacturer with an interest in anticoagulation sit on a committee that develops the guidelines for prevention of thromboembolic disease?), and how to mitigate these ubiquitous and pesky COIs.

In some ways watching this discussion has been amusing, while in others it has been downright sad. Medical journals, while insisting that advertising money is OK to take (presumably because the editorial and marketing offices are separated by some sort of a fire wall), though professional societies should not be able to take this tainted education money. Professional societies, on the other hand, are running away from the accusations by tightening their continuing medical education (CME) criteria and scrambling to replace the lavish budgets derived from pharma to develop their coveted evidence-based practice guidelines. And while all the pots are calling all the kettles black, academic researchers are being barred from collaborating with the industry on research projects, and industry researchers are being precluded from presenting their data at professional society meetings. While all the time the public is being whipped into lather about these alleged systematic transgressions, and forced to cheer for the ensuing retribution.

But, like many things in life, and especially stuff that we discuss on this blog, this issue is neither black nor white. Don't take me wrong: I am not condoning the egregious excesses of greed demonstrated by some members of my hallowed profession. If you have been reading my blog for some time, you know that I do not dispute the shameful reality of many breeches of public trust. I am an ardent supporter of exposing these breeches and of harsh punishments that they deserve. This is not what I am talking about here.

I am much more concerned about the one-sided story that we have been hearing about pharma-academic collaborations. Because of the persecutory nature of public opinion, some institutions are now shying away from such collaborations. This attitude is akin to navigating a treacherous road while looking in the rearview mirror. Yes, there have been transgressions, yes there has been greed and even scientific fraud in the name of money. Does this mean that we need to stop everything and come up with an entirely new way of managing these risks? Absolutely! Does this mean that we have to get rid of all pharma-academic collaborations? Absolutely not! In my humble opinion, erecting non-scaleable walls between these two groups is a big mistake. Here is why.

First, let me make a disclaimer: I do have active ongoing collaborations with multiple manufacturers. I do not take speaking or other promotional money, but limit myself to consulting and research grant funding. I also do a good deal of unfunded research, and I have never taken a penny for any of my blogging or blogging-related activities. And here is the crux of the matter: In this world of über-subspecialization, with the expertise being demographically and geographically diffuse, how can we afford not to collaborate across different types of organizations with different types of capabilities? Can we really afford to leave all of therapeutic development in the hands of organizations whose overarching purpose is to make money? And equally importantly, can we afford to continue this fragmented model of medical development without any thought to integration of the needs of all of the stake holders? I think not. Just as we are reaping the fruit of electronic medical record development in isolation from the end-user, so this isolation of research effort will lead to even less coherence in medicine. And unless we are ready to socialize our entire healthcare system, it seems naïve to expect that this one sector will acquiesce and start working outside of our coveted free market for the good of humankind alone.

My readers know that I am not an industry apologist. On the contrary, I have said many times that there has been bad behavior across all the sectors of healthcare, starting with biopharma. But if we want to advance rather than stagnate and regress, we need robust collaborations. We also need higher ethical standards and greater professionalism to keep public's health as our top priority.

There is COI everywhere, and, while financial COI is most visible, it is the more hidden COI that is most insidious. An hidden COI can be intellectual, reputational, ego-driven, career-mediated, etc. It is incumbent on us all in this complex world to ask questions and mitigate any ill effects of any cognitive biases, including those created by COI. Ultimately, as I have begun to realize of late, nothing will replace an educated and empowered patient: This is the only model that can provide appropriate checks and balances for our oftentimes misaligned and perverse incentives, both academic and economic.

How much for your vote, Senator?

It is not a secret that large sums of money have passed from the pharmaceutical and device manufacturers into physicians' hands. This money has been paid not only for such socially beneficial pursuits as research and consulting, but also for lavish gifts and junkets. And interestingly, despite disingenuous assertions to the contrary by many a stake holder, these gifts generated a return on investment -- surprise! It turns out that the spender companies could count on improving their prescription volumes in return for their magnanimity. Well, of course this is really no surprise. After all, we all know that money talks. What has been less clear over the years is whether small, almost inconsequential gifts, might also be influential in changing practices. Social sciences provided us with an answer to this: even gifts of small value create fertile soil for payback. OK, then, the answer became clear: remove all manufacturer influence from the day-to-day world of medicine. And so it has happened that pharmaceutical reps are no longer allowed to bring lunch or pens or pads of paper with the name of their wares on them to the hospitals or offices. Only educational gifts of a certain value are accepted. I will refrain from opining on the journals' and professional societies self-absolution from such rules, as my views on that are beyond the scope of the current post.

Is this removal of potential temptation bad? For someone who can argue each side with equal aplomb, the question is irrelevant: it just is. What interests me a lot more is this: how is it that a 25-cent pen can sway my brethren's prescribing practices, but $1 million in campaign contributions leaves a politician impartial to the contributor's cause? Take Senator Lieberman's claim that $1 million in campaign contributions from the health insurance lobby has not affected how he votes. See for yourselves, in this NYT piece from December 2009:

Campaign finance advocates have attacked Mr. Lieberman as “an insurance industry puppet,” suggesting that he wants to protect private health insurers from competition because he has received more than $1 million insurance company campaign contributions since 1998.

During his 2006 re-election campaign, Mr. Lieberman ranked second in the Senate in insurance industry contributions. Connecticut is a hub of the insurance business, with about 22,000 jobs specifically in health insurance, according to an industry trade group.

In the interview, Mr. Lieberman dismissed assertions that he was doing the industry’s bidding. “It’s hogwash and it’s weak,” he said, noting that he had often sided against the companies. 

Are we really supposed to believe that? What about Blanche Lincoln, when she asserts that $1/2 million in oil money that her campaign has taken has not swayed her legislative priorities, is she for real?

A spokesperson for the Lincoln campaign says that campaign contributions play no role in the senator's public policy decisions.

"This ad is nothing new, just more lies from another outside group seeking to malign Sen. Lincoln's record and bully voters into their agenda," spokeswoman Katie Laning Niebaum told CBS News. 

If a 25-cent pen can hijack a doctor's prescribing practice, how can these sums of money not be hijacking our democracy? And upon whom does the burden of proof fall in this situation?

Here is my solution: less advertising, less mud slinging, less dirty money (a.k.a. special interest contributions). Naïve? Maybe. But totally necessary. Let's do what medicine has done and, a-la Nancy Reagan's advice, "Just say 'NO'!" It's time to bring back our democracy!

Conflict of interest in continuing medical education

I know I've been on a conflict-of-interest kick lately, and this post will continue in that vein. I have to thank the New England Journal of Medicine for the fodder, which also leads me back to my old assertion: healthcare should NOT be done for profit.

In this week's NEJM, there is a piece from the Office of Inspector General of the US Department of Health and Human Services titled "The Agenda for continuing Medical Education -- Limiting Industry's Influence". In this article Morris and Taitsman lay out the issues and potential solutions. And while the problems are quite apparent, sensible solutions are scarce. The problems may be summarized as the undue influence by the Industry stake holders on the content of physician education. To understand this mouthful, one has to be familiar with the structure of the CME establishment.

The overseeing body for CME accreditation is the Accreditation Council for Continuing Medical Education. ACCME's stated Mission is

...the identification, development, and promotion of standards for quality continuing medical education (CME) utilized by physicians in their maintenance of competence and incorporation of new knowledge to improve quality medical care for patients and their communities.

In their 2008 report the ACCME quantified the total income for the 725 CME providers they accredit to be $2.4 billion. Who are the CME providers? According to the NEJM article, in 2007 they included

...270 physician membership organizations, 150 for-profit medical-education and communication companies, 123 medical schools, 93 hospitals and health care systems, 38 other nonprofit organizations, 15 government entities, 14 insurance and managed-care companies, and 33 providers that were not classified.

It is interesting to note that all of these organizations derive significant revenue from Industry CME funding. While most suspicion of undue influence centers around the for-profit MECCs, it is a fact that MD membership organizations rely heavily on CME funding, in addition to registration fees, to bank-roll their annual congresses. So, the idea is that, since ACCME accredits CME providers and not their programs on the ground, these providers may cater to their Industry clientele by structuring CME programs so as to optimize the chance of being funded. In other words, the relationship between CME providers and Industry is seen as too cozy.

Some solutions are offered by the authors. The one they seem to favor most is a compromise between today's mechanisms of allowing Industry to target specific programs they want to fund and the extreme of removing Industry funding from the CME space altogether: allowing companies to pay dollars into a common pool, which in turn is to be used by a third party to pay for CME programs deemed worthy and without either credit to or input from a company with an interest in the specific area being covered.

So, let's go with this solution. This means that for a private manufacturer the choice now becomes either to play in the common sandbox without any guarantee of a return on their investment (granted, even today the ROI is not supposed to enter into the CME funding calculus), or to get out of the CME funding game and invest elsewhere. If too many manufacturers should choose the latter, the whole CME game will be in trouble, and the physicians will need to pay for their own continuing education. In addition, if you think that meeting registration is expensive now ($400 to $800 in my experience), think what professional societies will have to charge once there is no Industry funding! The whole paradigm as it stands now may tumble.

I personally do not think that this solution is viable. Why not expect the manufacturers to be good corporate citizens and continue to contribute vast sums of money even without direct tangible benefits to them? Well, the answer is surprisingly simple: economic theory. The entire foundation of our capitalist free market theory is the idea of selfish utilities. Simply put, this theory maintains that people will act in their own best interests; this constitutes the rational decision making as promoted by free market economists. The fact that this is a theory that has never been put to a test does not keep our economy, philosophically and pragmatically, from being mired in greed and selfishness. What is the mission of the Boards of Directors of Industry? To generate and maximize profit. Period. Being a good corporate citizen is acceptable only as a by-product of this mission. And this is not a negative judgment of the Industry philosophy; this is the direction backed even by most liberal left-leaning economists in the US today.

OK, then why would we expect Pharma and device manufacturers to "donate" money that will not let them get ahead of their competition? We shouldn't. Having chosen to throw the dice of healthcare on the roulette wheel of free market competition, is it not hypocritical of us to ask that sector to play by different rules? Does it not make more sense to take healthcare out of the for-profit game altogether? I don't know about you, but to me the message is quite clear: by demanding a private industry to give money without an expectation of any ROI is completely at odds with the mission proscribed by free market economic theory. De-profitizing (yes, in fact, it is a neologism; any problem with that?) healthcare is the only solution that I see to remove this dissonance. This is one baby that needs to be thrown out with the bath water.