Healthcare, etc.

Physician Payment Sunshine Act: More marginal thinking

2012-01-23T09:25:00.000-05:00

It never ceases to amaze me how we gravitate to the margins in our thinking: margins seem to have a centrifugal force that is nearly impossible to overcome in today's political discourse. Yet the truth almost always lies at the center, the place that does not generate Op-Eds or produce votes.

I have said this before, and I will say it again: industry-physician relationship is not all bad or all good, there is no one within this relationship that is all bad or all evil, and it does not always benefit or always harm patients! The truth, of course, is somewhere in the middle. Contrary to Stossel's thesis, there is plenty to worry about with respect to corruption promoted by the big money exchanging hands between Pharma and doctors. On the other hand, just because there are instances of corruption and its consequences, not all interaction, financial or otherwise, is counterproductive. I am the first to admit that the much-touted innovation in medicine is rare, and we have largely given up its pursuit in favor of predictable markets and returns. Yet without a robust and transparent collaboration between industry and practitioners there is not only little hope of innovation, but any innovation that may stand a chance is likely to be irrelevant.

Yes, I agree with Stossel that the new reporting regulation is overly punitive and will inevitably result in undue administrative burden. But it would be disingenuous of me to disagree with the fundamental idea that there needs to be at least some degree of transparency in the financial dealings between industry and clinicians, if only to avoid the appearance by the docs of serving two masters.

As in everything in life, the devil is in the details. And it is these details that get buried by the gravitational pull of peripheral thinking and discourse. The solution? How about we stop paying attention to these marginal fallacies and start putting our heads together for real to solve these significant problems? How about we start a rational discussion about what is best for the people and not for the corporations or the economy or reputations? The discussion has been subverted by extremism. It is time to give in to the centripetal pull of reason.

3 ways to sink a new drug

2011-12-22T10:02:00.000-05:00

I don't just rant about methods and evidence -- in my work life I also rant about health economics and outcomes! This is why I was so interested in this post by the health economist Ulf Staginnus called
"New Models for Market Access." I want to give a hat tip to Healtheconomics.com for pointing me here.

The thesis of the article is that we need to refocus our discussion from market access to true innovation in the biopharma sector. There are some priceless quotes here, like this one, for example:

It is amusing, at least to me, to see the continued flood of articles, consultant presentations, blogs, congress announcements, workshops, summits, reorganizations, speeches, etc. all over the place, basically suggesting how the industry just needs to throw a few more people with fancy titles here and there, coupled with slight organizational changes, onto the problem and involve stakeholders and—guess what?!—actually talk to patients and perhaps even payers and all of a sudden, like Alice in Wonderland, everything will be good, after all.
The uncomfortable truth is, it won't be. All this “noise” is only good for one thing, paying the bills of the consultants, which is fine, too, as I have been one myself so I can understand. But it will not address the problem the research-based pharmaceutical industry and its employees are facing. Without a substantial increase in R&D productivity, the pharmaceutical industry's survival (let alone its continued growth prospects), at least in its current form, is in great jeopardy.

Don't you love it? It is hard to disagree. He also calls for more of a focus on the long-term returns than the short-term (duh!), as well as more internal honesty, or having the courage to stand up to the pathologic internal enthusiasm about a late-stage product that will obviously go nowhere. And all of this is on target.

He has this to say about health economics and outcomes and such:

Of course, you need experience in areas such as HE, outcomes research, pricing, economics, policy, advocacy, etc. and all needs to work in sync and early on and with the payer in mind and, yes, most people have understood that by now. So the problem is essentially not in the capabilities, although some are more advanced than others, but rather in the company cultures.

And this, I think, is where I have to disagree with him. In my experience there are glaring deficits in the approach to HEOR within biopharma, though of course there are exceptions to the rule. It starts with the fact that disease burden, and especially its costs, are initially assessed through less than, shall we say, rigorous methods. I have seen this critical information get pieced together through market "research", where 5-10 "thought leaders" are asked for their opinions, and the quantification is based on this tiny non-representative sample of nothing more than guesses. This is a shame because the data usually exist which can give a much more bona fide estimate of the extent of the problem.

The second problem is that articulating the value proposition of a nascent technology is usually an afterthought. In fact it is self-evident that drug pricing must be fed using the information on the burden of disease, and the impact the new technology can make in mitigating such burden. Unfortunately, time and time again I see companies backing into a price simply in reaction to what their Boards perceive the returns should be. And frequently this is based on the overly optimistic market projections flowing from, you guessed it, market "research."

So, the direct result of all this short-sightedness and business as usual is that even innovative useful products are driven into oblivion because there is no realistic look at what the technology is worth or where best to use it. And fixing the problem after the drug or device is on the market is a much bigger challenge for several reasons. First, the acquisition costs of new technologies are bound to be higher than of those already in use. This puts them at a disadvantage in that they get niched into populations that have much greater burdens of illness and therefore less of a chance of doing well. In other words, they are used as a last ditch therapy, which very rarely ends well. Ironically, these are usually not the populations who were studied in the pre-approval studies, and thus the use turns out to be off-label. But here is the real problem: When these technologies are in the "kitchen sink" category, they will almost always end up looking worse in terms of the outcomes than their older counterparts. And to the untrained eye, or an eye who does not have the time to discern the truth, particularly in the setting of perceived high expenses on the new product, this rings the death toll for the drug. But the reality, of course, is that the abysmal outcomes are the result of confounding by indication, where the drug was inappropriately given to those patients who were very unlikely to benefit from in the first place. But you see how this early lack of attention to the articulation of appropriate populations and health economic data can snowball into failure of a promising therapy.

So, if you want your drug to fail, do the opposite of what I recommend below. In other words DO NOT
1. Develop your market understanding
Do it not from the opinions of a handful of "experts" -- experts will rarely tell you the truth. Instead, do epidemiology studies to understand your population and its subpopulations so as to get the most reasonable idea of the disease.
2. Start thinking about the value proposition early
At the end of a successful Phase 2 program is a good time to do this. The surprise to most companies is how little HEOR studies cost in comparison with their clinical trials program. Yet, as you can see from above, this drop in the ocean can make or break a product.
3. Focus on transparent pricing methods
When pricing the technology, be very very sure that you have all of the ducks in a row, meaning:
a. do understand your market
b. do understand the burden and costs of the disease
c. do understand how your product impacts these costs
d. do price the product to reflect this balance
It is truly embarrassing to have to admit that your price reflects nothing more than the greed of your investors. Trust me, you will not score points with your customers.

Staginnus makes one other important point which I generally agree with:

And let's face it, if you need a major workshop and intensive external “coaching” to help define the value of your product … well, there actually is little to none. If it was really good, it would have been obvious from the start. So maybe we ought to stop beating around the bush and move on if there is nothing to be done anymore.

There is a nuance here, however, as in most things. Given what I have said above, products are more likely to gut than sell themselves. So, while I agree that you do not need a throng of consultants in suits and hair gel to pollute your offices, you do need to understand how to articulate this value, even when it appears obvious.

When end of life is not

2011-12-13T14:05:00.000-05:00

Twenty years ago, I helped save a man's life.

So begins this New York Times essay by Peter Bach, MD, where he talks about the inadequacy of resource use at the end of life as a policy metric. Now, I am not very fond of policy metrics, as most of you know. So, imagine my surprise when I found myself disagreeing vehemently with Peter's argument. Well, to be fair, I did not disagree with him completely. I only disagreed with the thesis that he constructed, skillfully yet transparently fallaciously (wow, a double adverb, I am going to literary hell!) Here is what got me.

He describes a case of a middle-aged man who was experiencing a disorganized heart rhythm, which ultimately resulted in dead bowel and sepsis. The man became critically ill, the story continues, but three weeks later he went home alive and well. This, Dr. Bach says, is why end of life resource utilization is a bad metric: if this guy, who had a high risk of dying, had in fact died in the hospital, the resources spent on his hospital care would have been considered wasted by the measurement. And I could not agree more that lumping all terminal resource use under one umbrella of wasteful spending is idiotic. Unfortunately, knowingly or not, Peter presented a faulty argument.

The case he used as an example is not the case. Indeed it is a straw man constructed for the cynical purpose of easy knock-down. When we talk about futile care, we are not referring to this middle-aged (presumably) relatively healthy guy, no. We are talking about that 95-year-old nursing home patient with advanced dementia being treated in an ICU for urosepsis, or coming into the hospital for a G-tube placement because of no longer being able to eat or drink. We are talking about patients with advanced heart failure and metastatic cancer, whose chances of surviving for the subsequent three months are less than 25%. And yes, we are also talking about some middle-aged guy with gut ischemia, sepsis and worsening multi-organ failure whose chances of surviving to hospital discharge are close to nil; but in his case, instead of being clear from the beginning, the situation evolves.

So, yes, the costs of end of life care, and specifically hospitalizations, are staggering. But more importantly, among patients with terminal illnesses like metastatic cancer, advanced heart failure and dementia, hospitalizations and heroic interventions at the end of life cause unnecessary pain and suffering, and without much, if any, benefit in return. Their families and caregivers suffer as well, and many studies suggest that these caregivers are not interested in prolonging suffering, provided they are aware of the prognosis. Unfortunately, just as many studies suggest that communication between doctors and patients' families about these difficult issues is less than stellar.

So, let me play the devil's advocate and pretend that I support end of life resource utilization as a quality metric. If I did, I certainly would not be interested in depriving Dr. Bach's middle-aged acutely ill patient of the chance to survive. In fact, my aim would be to make sure that we align resource use with where it can do most good, and turn away from interventions that are apt merely to prolong dying.

Lessons from Xigris

2011-11-22T15:28:00.001-05:00

I have been wanting to write for a while about the demise of Xigris, but work and other commitments have stalled my progress. But it is time.

Here is my disclosure: I have received research funding from BioCritica, a daughter company of Eli Lilly, the manufacturer of Xigris. I also happen to know well and hold in high esteem the depth of knowledge and integrity of several colleagues who worked on Xigris internally at Lilly.

But on to the story. Xigris has had a short and bumpy life. When the PROWESS study, the Phase III Xigris trial, was first published in the NEJM in 2001 [1], it was the first therapy to succeed in sepsis, reducing mortality by 6% from about 31% to about 25%, yielding the number needed to treat of 16. This was huge, as so many trials to date had failed, and no progress had been made in sepsis management for years. These data opened the door to the FDA approval, despite a hung advisory committee, where equal numbers of members voted for and against approval. The controversy centered on concerns for bleeding complications, as well as some protocol changes during the trial and a switch in the manufacturing process. The latter concern was allayed by the Agency's detailed analysis and the finding of equivalence. There was a signal in a subgroup analysis that the drug might have been most effective among the most ill patients with a high probability of death, but not in their less ill counterparts. And despite the fact that the pivotal trial was not specifically performed in these patients, the approval for use specified just such a population.

So, despite the controversy, the drug was approved, though several post-marketing commitment studies were mandated. ENHANCE [2, 3] was an international study whose findings broadly confirmed the safety and efficacy of the drug, while the ADDRESS study [4], done in patients at low risk for death, was terminated early for lack of efficacy.

It seemed that PROWESS ushered in an era of positive results in sepsis. Shortly after its publication, other studies on the use of early goal-directed therapy [5], low-dose steroids [6] and tight glucose control [7] appeared in high impact journals, and the years of failure in sepsis management seemed to be over.

In the meantime, and amid further controversy [8], Lilly supported the creation of the Values, Ethics and Rationing in Critical Care (VERICC) Task Force [9, 10], in addition to giving funding for the international Surviving Sepsis Campaign (SSC), which has resulted in the evidence-based practice guideline for sepsis management [11, 12] and an implementation program for the sepsis bundles, jointly sponsored by the SSC and the Institute for Healthcare Improvement [13]. The latter 2-year program enrolled over 15,000 patients world-wide, and achieved a doubling of bundle compliance from 18% to 36% with a concurrent drop in adjusted mortality of 5%. Because of several methodological issues and the lack of transparency about what it took to implement the bundle, it has never been clear to me a). whether there was causality between the bundle and mortality, and b). whether this effort was cost-effective.

But that aside, Xigris continued to stir up controversy, and there were still safety concerns. Some very well done observational studies, however, continued to confirm its effectiveness and safety in the real world setting [14]. Yet the final trial, PROWESS-SHOCK (done because of fears of an increase in bleeding complications), where patients in septic shock received Xigris as a part of their early management, brought doom. It was this study, whose preliminary results appeared in the press release from October 25, 2011, that prompted Lilly to pull the drug off the world market, since no difference in the 28-day mortality was detected between placebo and Xigris arms. Ironically, the preliminary reports indicate that no excess bleeding was noted in the treatment arm.

So, after roughly 10 years and millions of dollars, Xigris disappeared. But what can we learn from its story? There are many lessons that we should carry away, some about the way we do research, some about marketing practices, but all of them are about the need for a higher level of conversation and partnership. The biggest elephant in this room is whether a manufacturer should be allowed to fund guideline development. It is a complicated issue, particularly given our native proneness to cognitive bias, but in my opinion yes. This certainly cannot be done in a quid pro quo way. Perhaps this is naïve but should it not simply be a question of good data? And why wouldn't a manufacturer give money for the development of sensible guidelines without strings attached when the data are good?

Unfortunately, to me, Xigris is the poster child for how broken our research enterprise is, as I have discussed in this JAMA commentary [15]. Until all stake holders start talking to each other and arriving at common, useful and achievable goals, this is a story that will repeat itself again and again. The fact that regulatory trials, with all of their expensive and flashy internal validity, concern themselves only with statistical issues and care nothing about what happens in the real world is a travesty on many levels. The fact that it costs nearly $1 billion to bring a drug to market means that only big Pharma can bankroll such a gamble, and in return must demand big profits. The fact that this $1 billion fails to bring us studies that help clinicians and policy makers understand fully how to optimize the use of a drug once it is on the market is inexcusable. What we need is more intellectually honest discussions leading to novel pragmatic ways to answer the relevant questions in a timely manner and without bankrupting the system.

So, does the obvious financial interest mean that manufacturers should stay out of these discussions? I happen to think that they need a prominent place at the table. I actually think that the current fiasco is largely the result of too little interaction and too little cross-pollination of ideas: when we all sit around the table a nod in agreement, there is little progress. Deeper and novel understanding is built on disagreement and debate. Therefore, to leave the manufacturers out would invite further irrelevance. The bottom line is that we are all conflicted, and, according to the editors of PLoS, non-financial conflicts of interest, though more subtle and difficult to discern, may present an even bigger threat to much of what we do [16]. Elbowing out a party with an obvious conflict may have the unintended consequence of leaving some of the more insidiously conflicted others to run the show. And although we can argue whether profit is the healthiest driver for performance in healthcare, the reality is that our entire healthcare "system" is built around profit-making. Therefore it is disingenuous to single out one player over others.

On the positive side, the halo effect around Xigris brought a ton of attention to sepsis and its management. As Wes Ely conjectured in this piece, our improved understanding of sepsis (largely due to all the attention Xigris brought to it, in my opinion), is probably what rendered the drug useless in PROWESS-SHOCK. So, after all the hype, the noise and the hoopla, what is left is a company less one drug and hundreds of millions of dollars, and a disease area with a whole lot of what amounted to public health investment, with a vastly improved understanding of the disease state. How much is this benefit worth?

References

[1] Bernard GR, Vincent JL, Laterre PF, etal: Efficacy and safety of recombinant human activated protein C for severesepsis. N Engl J Med 2001; 344:699–709
[2] Bernard GR, Margolis BD, Shanies HM, etal. Extended Evaluation of Recombinant Human Activated Protein C United StatesTrial (ENHANCE US). A Single-Arm, Phase 3B, Multicenter Study of DrotrecoginAlfa (Activated) in Severe Sepsis. Chest 2004;125:2206-16
[3] Vincent JL, Bernard GR, BealeR et al. Drotrecogin alfa (activated) treatment in severe sepsis from theglobal open-label trial ENHANCE: further evidence for survival and safety andimplications for early treatment. Crit Care Med 2005;33: 2266-77
[4] Abraham E, Laterre P-F, Garg R, et al. DrotrecoginAlfa (Activated) for Adults with Severe Sepsis and a Low Risk of Death. New EnglJ Med 2005;353:1332-1341
[5] Rivers E, Nguyen B, Havstad S, et al.Early goal-directed therapy in the treatment of severe sepsis and septic shock.N Engl J Med 2001;345:1368-1377
[6] Annane D, Seville B, Charpentier C, etal. Effect of treatment with low doses of hydrocortisone and fludrocortisone onmortality in patients with septic shock. JAMA 2002;288:862-871
[7] van den Berghe G, Wouters P, Weekers F,et al. Intensive insulin therapy in the critically ill patients. N Engl J Med 2001;345:1359-1367

[8] Eichacker PQ, Natanson C, Danner RL.Surviving Sepsis – Practice Guidelines, Marketing Campaigns and Eli Lilly. NEngl J Med 2006;355:1640-2

[9] Sinuff T, Kahnamui K, Cook DJ, et al. Rationingcritical care beds: A systematic review. Crit Care Med 2004;32:1588-97

[10] Truog RD, Brock DW, Cook DJ, et al. Rationingin the intensive care unit. Crit Care Med 2006;34:958-63

[11] Dellinger RP, Carlet JM, Masur H, etal: Surviving Sepsis Campaign guidelines for management of severe sepsis andseptic shock. 2004;32:858-73

[12] Dellinger RP, Levy MM, Carlet JM, etal: Surviving Sepsis Campaign: International guidelines for management ofsevere sepsis and septic shock: 2008. CritCare Med 2008;36:296-327. Erratum in Crit Care Med 2008;36:1394-96

[13] Levy MM, Dellinger RP, Townsend SR, etal. The Surviving Sepsis Campaign: Results of an international guideline-basedperformance improvement program targeting severe sepsis. Crit Care Med 2010;38:367-74

[14] Lindenauer PK, Rothberg MB, NathansonBH, et al. Activated protein C and hospital mortality in septic shock: Apropensity-matched analysis. Crit Care Med 2010;38:1101-7

[15] Zilberberg MD. The clinical researchenterprise: Time for a course change? JAMA 2011;305:604-5

[16] The PLoS Medicine Editors (2008) Making Sense of Non-Financial Competing Interests. PLoS Med 5(9): e199. doi:10.1371/journal.pmed.0050199

Massachusetts' unwinnable gamble

2011-11-21T13:52:00.001-05:00

It is ironic how, just a few days following the startling (?) confirmation by the Robert Wood Johnson Foundation-funded research that an ounce of prevention is indeed worth a pound of cure, the Massachusetts legislature with reckless abandon ushered in yet another mechanism for the erosion of public health: legalized gambling. Really, I have nothing against a little gambling. The issue is that this legislative move does not just open the door to a trickle of small local gambling operations. No, what it does is turn the crank to open a fire hose of "big box" gambling establishments descending upon our state. And it is not just anywhere in the state: it is in the Western part, far removed from the back yards of the legislators who are salivating over the projected licensing and tax revenues.

But I don't want to get into the NIMBY aspect of this misguided bill. I would rather stick to the real issue: selling us out to raise short-term revenue. The move projects 15,000 new jobs (menial with no benefits mostly), $40 million annually in tax income, on top of $85 million licensing fees from each of the three casinos, all this in addition to construction investment and the like. Already the bill allocates $50 million to overhauling healthcare reimbursements in the state. As well, there is a $25 million provision to shore up research into and prevention of problem gambling. And even people who are in staunch opposition to legalizing gambling seem appeased by this provision, which they say makes it the best bill of its kind. But we still have to ask, if prevention is better than cure, why settle for good mitigation strategies when we have the best prevention available to us already: keep casinos out!

Some of you will probably say that I am naive. After all, reason fades when we are talking about such big bucks for the state coffers. Well, just because this kind of a trade-off is something we have come to expect from our politicians does not mean that we should tolerate it. Others will bring up the old free will argument. No, I am not against people exercising their personal decision making, but haven't you read "Nudge?" We are all deeply flawed human beings, and in the face of temptation we fail miserably! And since we know that casinos increase the risk of problem gambling, why not just steer clear of them altogether? This is simply not a winnable gamble.

I hope that some of you are hearing echoes of the food-obesity debate. We deem it an individual rather than a societal problem, and look how well we have done mitigating the obesity epidemic! There is no rocket science here, and it is disingenuous to say that we do not understand the causes of obesity. Human physiology has not changed over a couple of generations, no. What has changed is our constant access to high-calorie cheap concoctions that pass for food; what has changed is our limited access to physical activity; and what has changed is the degree to which we as a society are willing to sign on to corporate and political propaganda designed to get votes and make money at the expense of our health.

So, am I shocked that this casino bill is likely to become law? Not at all. Am I surprised that the public is allowing this to happen in a pathetic perversion of personal freedom? Of course not. Am I going to shut up about what a mistake this is? You bet I am not. And in a decade I will say "I told you so." But I am sure that then, not unlike now, no one will be listening.

An open letter to my past and future students

2011-09-29T17:34:00.000-04:00

As most of my readers know, I teach Public Health graduate students and the University of Massachusetts, sometimes on campus, and sometimes online. This is an open letter to all of my past and future students.

First, I want to thank everyone in my June course who took the time to complete the evaluation -- the feedback is very helpful to me. I also want to thank the whole class (and all my previous and future classes) for the privilege of learning together with you. Looking at this and previous rounds of evaluations made me realize that I need to make a public statement that all students contemplating taking my courses can read before they commit.

My evaluations tend to be bimodal -- a peak around "love", a (smaller, thankfully) peak around "hate", and mostly a trough in between. Oddly, the reasons for the love and the hate are the same: not too much structure, not too much interim formal performance evaluations, a lot of opportunities for discussions and questions. Everyone seems to appreciate my effort to make the material interesting and relevant, but a substantial number do not seem to like it.

I am actually fascinated by the convergence of the reasons for liking and disliking. By way of inference, I am going to suggest that where you fall out on the teaching should tell you more about your learning style than how I teach. To clarify, let me spell out my philosophy of teaching.

Whether I teach on campus or online, I limit my classes to graduate students. The reason for this is not that I do not think that undergrads cannot handle the material. Rather it is because I believe nothing replaces time with a topic to develop a depth of understanding and discussion about it. So, I view my classes as incubators of ideas. I do not see myself as the oracle delivering answers. My role is to get you excited about the questions. Furthermore, it is not my questions that should excite you, but the questions that you come to at the limits of your knowledge seen through the prism of the class material and discussions.

To be sure, I realize that this not a comfortable place for many. Most of us glide through an educational system that convinces us that there is a single correct answer, and, after teaching us to parrot it compliantly, punishes us if we stray. So, swimming in the sea of questions, seemingly answering them only to realize that the answers lead to further questions is disquieting. Yet it is at this edge that we gain access to the next level of understanding of our universe. Here, the feedback is not about an arbitrary letter, but about the exuberance of ideas, discussion generated and the richness of asking the questions.

I cannot tell you how much I love the learning environment that we create together. I gain something from each and every one of you, and I hope that each of you walks away with at least one idea that is new. What I suggest to you if you are a potential student considering taking a course with me in the future is to contemplate the boundaries of your own comfort zones in learning. If you like the feeling of vertigo that you get when old dogmatic answers are shattered and uncertainty reigns, take my classes. If you are worried about how it might feel, but curious to try, I will meet you where you are and help you weave a net to cradle your fall. But if you know that uncertainty cripples you, that you would rather have a map for every step of the way, my classes may be the wrong stop along your educational path at this time. But perhaps sometime in the future?

Again, thanks to everyone for enriching my learning. I miss you and look forward to future opportunities for exchanging ideas.

Clinician as the Politbureau of medicine?

2011-09-23T12:08:00.000-04:00

Do you think that medicine in the US is centralized? I do, but not in the way that we generally understand centralization. And furthermore, it is this centralization that I believe is making the idea of shared decision making so intimidating to some. Here is what I mean.

If you read management texts, centralization refers to an organization that is run predominantly top-down. In other words, a couple of oligarchs at the top of the ladder make all the decisions without consulting anyone below. In this way all the power is concentrated in the hands of the few. In an antithesis to this, in a decentralized organization, grassroots input and initiatives are incorporated into the fabric of the organization. And while in the times of a great crisis, when rapid decisions are necessary, the benefits of centralization may outweigh its risks, during normal day-to-day operations, such unilateral power can result in obviously negative consequences, from discontent among the employees to making the wrong choices. Furthermore, as organizations grow in size, it gets that much more difficult to run them effectively within the centralized paradigm.

Now, let us look at medicine. The traditional model of the doctor-patient relationship relies on the clinician to know what is right for the patient: take this pill and don't worry about the side effects, dear. Now, clearly, when someone shows up to the emergency room in septic shock, there is very little room for a democratic process; we want the doctor to do rapidly what needs to be done to save the patient. But this is a catastrophic exception to the rule of what modern medicine cares for. From pre-diabetes to pre-hypertension to "borderline cholesterol" to osteopenia to mild depression, these are the "diseases" that are prevalent in the office of the 21st century. None of these is particularly urgent or life-threatening. And if we are honest with ourselves, even a devastating diagnosis of cancer does not demand an instantaneous intervention: in the vast majority of cases there is ample time for discussion and contemplation. So, the centralized approach is the wrong way to go. Thus enter the robust discussion about shared decision making.

Another reason that centralization of medical decisions is crumbling is the expanding patient panels that clinicians need to engage with in order to stay solvent, all within the context of increasing compliance and regulatory burdens along with decreasing reimbursements. Without an equal growth in one's cognitive ability to multi-task, this escalating imbalance is creating a rising risk for unilateral decisions to be plain wrong.

So, in my mind, this is yet another argument for all parties to embrace shared medical decision making to the extent we as patients are willing and able to do so. Because what is the alternative?

Why patient lab data should be liberated, with a few caveats

2011-09-21T08:48:00.000-04:00

I am admittedly not an expert on health IT, but I am a firm believer in the empowerment of patients to be the driver of her/his health decision making. So this whole discussion about lab data being available directly to the patient is of great interest to me. But it does seem like yet another instance of the two sides coming together not to listen to each other but to be heard by the other side. And as well know, this works so well for any relationship!

Each side's view is represented roughly thusly:
Patients -- these are my data and I have the right to access them as soon as they are available.
Doctors -- we are worried that the sheer volume, complexity and irrelevance of (much) of the data will make it confusing and unnecessarily alarm the patient
Both arguments are valid, of course. But it is important to ask what lurks below the visible portion of each iceberg.

Let's take the patient view. Why do I want immediate access to my data? Well, obviously, because it is mine, it represents the results of testing on my body, and the record should belong to me. I should be able to access it freely whenever I damned well please. I am also more than a little exasperated with having to wait sometimes days to hear from my doctor's office about a result that has been available for a while, but was buried under the reams of paperwork on the MD's desk or his/her assigning a low priority to my data. And I am most exasperated when my lab results get lost or otherwise never make it to me at all. Perhaps if I have direct and unfettered access, this will make thing more efficient for me as an individual.

The doc's view, on the other hand, is that the patient does not necessarily understand what the notation of "low"connotes in reference to, say, total bilirubin, or how to interpret the RDW data. Even more importantly, what if there is an outrageously abnormal value for some important test? Surely the patient will desire an immediate explanation of it and its implications.

So, clearly, both sides have valid concerns. I do think that those of access predominate, as ethically it just makes sense. But for a non-medical person, looking at a lab sheet is like trying to read information about yourself in Chinese: your success in understanding is largely dependent on your ability to read and understand Chinese. So, before that horse leaves the barn, we should think through how to execute this most sensibly. For example, perhaps it is not sensible to have the lab computer directly vomit all of the inane values that no one really looks at right to the patient's account. And backing up a step, perhaps it is time for our lab use to be driven not by the lab equipment packages and processes, but to test only for factors that are of value. If I want to know the patient's creatinine, maybe the other 6 components of the Chem-7 should not be run, or at least not reported. And obscure values like the ones I mentioned above, e.g., RDW, MCHC, etc., should only be available when the situation actually makes them useful, and not just distracting.

I can see a potential positive unintended consequence of this development as well: maybe clinicians will be less trigger-happy ordering all kinds of labs for all kinds of oblique reasons. Maybe, just maybe, this apprehension about the patient's access to all the labs will result in more Bayesian thinking in the office and a lot less shot-gunning. Finally, it will not be all patients that choose to access their data. Let us hope that the selection bias does its job and assures that only those who are truly ready to be educated and empowered decide to do so.

All in all, I am looking forward to the liberation of my lab data. What I worry about is all the calls I will be getting from friends and family to help them understand them. All the same, I will do my part for the education and empowerment that absolutely needs to happen for this to be a successful and meaningful change.

Eminence or evidence, or how not to look like a fool when reporting your own data

2011-09-20T10:42:00.000-04:00

A study presented a the ICAAC meeting was reported by the Family Practice News that piqued my interest. Firstly, it is a study on C. difficile infection treatment, and secondly it is counter to the evidence that has accumulated to date. So, I read the story very carefully, as, alas, the actual study presentation does not appear to be available.

Before I launch into the deconstruction of the data, I need to state that I do have a potential conflict of interest here. I am very involved in the CDI research from the health services and epidemiology perspective. But equally importantly, I have received research and consulting funding from ViroPharma, the manufacturer of oral Vancocin that is used to treat severe CDI.

And here is an important piece of background information: the reason the study was done. The recent evidence-based guideline on CDI developed jointly by SHEA and IDSA recommends initial treatment with metronidazole in the case of an infection that does not meet severe criteria, while advocating the use of vancomycin for severe disease. We will get into the reasons for this recommendation below.

OK, with that out of the way, let us consider the information at hand.

My first contention is that this is a great example of how NOT to conduct a study (or how not to report it , or both). The study was a retrospective chart review at a single VA hospital in Chicago. All patients admitted between 1/09 and 3/10 who had tested positive for C. difficile toxin were identified and their hospitalizations records reviewed. A total of 147 patients were thus studied, of whom 25 (17%) received vancomycin and 122 (83%) metronidazole. It is worth mentioning that of the 122 initially treated with vancomycin, 28 (23%) were switched over to metronidazole treatment. The reasons for the switch as well as their outcomes remain obscure.

The treatment groups were stratified based on disease severity. Though the abstract states that severity was judged based on "temperature, white blood cell count, serum creatinine , serum albumin, acute mental status changes, systolic blood pressure<90, requirement for pressors," the thresholds for most of these variables are not stated. One can only assume that this stratification was done consistently and comported with the guideline.

Here is how the severity played out:

Nowhere can I find where those patients who were switched from metronidazole to vancomycin fell in these categories. And this is obviously important.

Now, for the outcomes. Those assessed were "need for colonoscopy, presence of pseudomembranes, adynamic ileus, recurrence within 30 days , reinfection > 30 days post therapy, number of recurrences >1, shock, megacolon, colon perforation, emergent colectomy, death." But what was reported? The only outcome to be reported in detail is recurrence in 30 days. And here is how it looks:

The other outcomes are reported merely as "M was equivalent to V irrespective of severity of illness (p=0.14). There was no difference in rate of recurrence (p= 0.41) nor in rate of complications between the groups (p=0.77)."
What the heck does this mean? Is the implication that the p-value tells the whole story? This is absurd! In addition, it does not appear to me from the abstract or the FPC report as if the authors bothered to do any adjusting for potential confounders. Granted, their minuscule sample size did not leave much room for that, but a lack of attempt alone invalidates the conclusion.

Oh, but if this were only the biggest of the problems! I'll start with what I think is the least of the threats to validity and work my way to the top of that heap, skipping much in the middle, as I do not have the time and the information available is full of holes. First, in any observational study of treatment there is a very strong possibility of confounding by indication. I have talked about this phenomenon previously here. I think of it as a clinician noticing something about the patient's severity of illness that does not manifest as a clear physiologic or laboratory sign, yet is very much present. A patient with this characteristic, although looking to us on paper much like one without a disease that is that severe, will be treated as someone at a higher threat level. In this case it may translate into treatment with vancomycin of patients who do not meet our criteria for severe disease, but nevertheless are severely ill. If present, this type of confounding blunts the observed differences between groups.

The lack of adjustment for potential confounding of any sort is a huge issue that negates any possibility of drawing a valid conclusion. Simply comparing groups based on severity of CDI does not eliminate the need to compare based on other factors that may be related to both the exposure and the outcome. This is pretty elementary. But again, this is minor compared to the fatal flaw.

And here it is, the final nail in the coffin of this study for me: sample size and superiority design. Firstly, the abstract and the write-up say nothing of what the study was powered to show. At least if this information had been available, we could make slightly more sense out of the p-values presented. But, no, this is nowhere to be found. As we all know, finding statistical significance is dependent on the effect size and variation within the population: the smaller the effect size and the greater the variation, the more subjects are needed to show a meaningful difference. Note, I said meaningful, NOT significant, and this they likewise neglect. What would be a clinically meaningful difference in the outcome(s)? Could 11% difference in recurrence rates be clinically important? I think so. But it is not statistically significant, you say! Bah-humbug, I say, go back and read all about the bunk that p-values represent!

One final issue, and this is that a superiority study is the wrong design here, in the absence of a placebo arm. In fact, the appropriate design is a non-inferiority study, with a very explicit development of valid non-inferiority margins that have to be met. It is true that a non-inferiority study may signal a superior result, but only if it is properly designed and executed, which this is not.

So, am I surprised that the study found "no differences" as supported by the p-values between the two treatments? Absolutely not. The sample size, the design and other issues touched on above preclude any meaningful conclusions being made. Yet this does not seem to stop the authors from doing exactly that, and the press from parroting them. Here is what the lead author states with aplomb:

"There is a need for a prospective, head-to-head trial of these two medications, but I’m not sure who’s going to fund that study," Dr. Saleheen said in an interview at the meeting, which was sponsored by the American Society for Microbiology. "There is a paucity of data on this topic so it’s hard to say which antibiotic is better. We’re not jumping to any conclusions. There is no fixed management. We have to individualize each patient and treat accordingly."

OK, so I cannot disagree with the individualized treatment recommendation. But do we really need a "prospective head-to-head trial of these two medications"? I would say "yes," if there were not already not 1 but 2 randomized controlled trials addressing this very question. One by Zar and colleagues and another done as a regulatory study of the failed Genzyme drug tolevamer. Both of the trials contained separate arms for metronidazole and vancomycin (the Genzyme trial also had a tolevamer arm), and both stratified by disease severity. Zar and colleagues reported that in the severe CDI group the rate of clinical response was 76% in the metronidazole-treated patients versus 97% in the vancomycin group, with the p=0.02. In the tolevamer trial, presented as a poster at the 2007 ICAAC, there was an 85% clinical response rate to vancomycin and 65% to metronidazole (p=0.04).

We can always desire a better trial with better designs and different outcomes, but at some point practical considerations have to enter the equation. These are painstakingly performed studies that show a fairly convincing and consistent result. So, to put the current deeply flawed study against these findings is foolish, which is why I suspect the investigators failed to mention anything about these RCTs.

Why do I seem so incensed by this report? I am really getting impatient with both scientists and reporters for willfully misrepresenting the strength and validity of data. This makes everyone look like idiots, but more importantly such detritus clogs the gears of real science and clinical decision-making.

Adventures with American or Flying, American Style

2011-09-19T12:38:00.000-04:00

So, we all know that the time for a doctor's appointment is merely a suggestion, not a mandate, as the doctor is hardly ever on time. We have even started of necessity to apply this theory to air travel. But to end up arriving 5 hours late and... to the wrong city? Well, this was a new one on me. Here is what happened.

I went to ICAAC for the day yesterday to present some of our data on predictors of a mixed skin and soft tissues infection. If it had not been a podium presentation, I would have considered skipping the whole meeting, since it was my son's birthday. Instead, I decided to swoop in for the day.

I chose American Airlines, as it is one of the few carriers that can get me to Chicago without a lay-over. The flight there was fine, and I had plenty of time to get myself to McCormick Convention Center, have lunch with a colleague, get to the session and do my thing. My schedule was such that I had to get in the cab immediately after presenting to get to my flight, which I did.

Getting to the airport was a challenge, as after 3:00 PM the perennial Chicago traffic jams are just a fact of life. But get there I did, with time to spare. The lines at the check-in counters were daunting, and luckily I was able to find a self-service kiosk, where I swiped my credit card. With a predictable automaticity I went to press the "Print boarding pass" icon when something caught my attention: this did not look my itinerary! I had been meant to fly to Bradley International Airport on the 5:25 PM flight. Now the screen was trying to push someone else's trip on me that went from Chicago to Bradley via Dallas-Fort Worth. Luckily there was an option on the screen to decline the itinerary as not owned by me, which I did. Alas, the second try resulted in the same baffling error.

A lovely young man with the American Airlines uniform on standing next to my kiosk noticed my confusion and asked how he could help. I pointed to the screen. He smiled broadly and sweetly and delivered the bad news that my flight had been canceled, and I was being re-routed via Texas, and that instead of getting to Hartford around 8:00 PM, I would be getting there well after midnight. The shock prevented me from stopping him from printing out the boarding pass. Before I had recovered my ability to speak, he furrowed his brow while examining the pass. Glancing at his watch, which read 3:30 PM, seemingly speaking to himself, he said "I wonder why they put you on a 3:05 flight when it is already 3:30?" As I was catching my breath, he continued "Oh, this flight is tomorrow afternoon!"

With these words my hopes of seeing my son before the end of the day of his birth hopelessly vanished. Yet I was not to be defeated yet. Although I was told that a later flight to Hartford was still happening, but was already overbooked, I was not ready to give up. Instead, I asked how close to Hartford they could get me on the same night. Turned out that there was a 5:05 to Boston, which was now going to be delayed until 6:30, but there was a coveted seat available and did I want it. Well, given my choices, I desperately wanted it, and luckily got it. So, now, instead of getting home by 10:00 PM I was looking at getting into Boston at a yet undetermined time, renting a car to get myself to Bradley, picking up my car from the garage there and driving home. If luck was with me, I would be home before 3:00 AM. Well, at least I would not have to go to Texas. Tomorrow.

Now that I had indefinite time before boarding, I treated myself to a latte and a book, Paolo Coelho's The Alchemist. It felt somehow luxurious to be suspended in this space without time, where I could just concentrate on the rich story and language and question my Personal Legend, though I was pretty sure that awaiting a late flight at O'Hare only to land in the wrong city and then drive for hours to get home was not it. Nevertheless, peculiarly, this departure from the rush of traveling felt like a little spa break, albeit in the midst of a chaotic throng craning their necks for the arrival of the aircraft.

Well, we finally were airborne a little after 8:00 PM, the flight was uneventful, I rented a car, got to Bradley before Avis closed (no, they are not open 24 hours), got into my car and got myself home just before 3:00 AM. All in all, it could have been a lot worse (I could have had to go through Dallas... today). And everyone was so frightfully nice, especially the young man with large brown eyes and a feline manner at O'Hare. The epiphany for me was that I could not even be angry -- there was no one or nothing to be angry at, and anger would have been disempowering, impotent. Instead, the lesson was that this is air travel in the 21st century -- crowded, uncertain, thoroughly unappealing. The only thing one can do, aside from avoiding it, is to accept it for what it is and go on. Though I have to say that being screwed is much nicer with a smile.

So I got my son a tracfone...

2011-09-14T11:44:00.000-04:00

Wanted to post some follow up to my absurd interaction with AT&T, the beginning of which can be found here. Briefly, all I wanted to do was add one more line to my family plan for $9.99 per month. Instead of making it simple, the web site demanded personal information (including my social security number) for the purpose of doing a credit check. Now, I already have a monthly family plan for which I pay over an order of magnitude more than $9.99, and on time. Yet now they wanted to subject me to an additional credit check. Well, I said no and wrote the post in question.

What was interesting and even encouraging to me was that I got the following comment from an AT&T customer service rep, and responded in kind:

attcathyw said...: Hi Marya- I am with AT&T and saw your post. We would like to assist with and answer any questions/concerns that you may have. Please email your contact information to one of our managers at attcatherinew@att.com and include your name in the subject line. Thanks.; September 7, 2011 4:12 PM
Marya Zilberberg said...: Thanks to all of you who tweeted and facebooked (as a verb?) me about this post. My impression that there is growing dissatisfaction out there with the way business is done is being confirmed.I wanted to update you all on what the follow-up has been. I e-mailed Cathy (see comment above) yesterday. Last night I got this message from her coworker:"Hi Dr. Zilberberg,Cathy is out of the office so I am reaching out on her behalf. In order for me to research your wireless account further, I would need either the account number or the mobile number in question. In additionally, a good contact number for you would be great.Once I receive the information, I will partner with a wireless customer service manager to see what we can possibly do.I do hate to see the frustration in your post. While I cannot guarantee, I will do my best to see what can be done to provide the extra line without the credit verification. Of course we will not make any changes without your consent.Please email your contact information, account number or mobile number to me at attannellem@att.com and include your name in the subject line. Please provide a best time to contact you. Thanks and have a good evening!Thanks,ATTAnnelle M"OK, so again I am the one wasting my time having to contact yet another person because the original individual I was told to contact for customer service is not there? Wow!Nevertheless, this morning I e-mailed Annelle and let her know that I simply do not have the time to keep explaining the issue, and that unless she can do what I need without wasting my time, she should go ahead and let me know by mail. I reiterated what the issues were and also that this is a corporate culture problem that needs to be rethought in light of its ethical implications.Will let you know what happens.; September 8, 2011 10:43 AM

The following correspondence ensued between me and Annelle (emphasis mine; numbers x'ed out to protect my privacy, which I am still interested in maintaining, despite Mark Zuckerberg's assertions to the contrary):

September 8: me

Dear Annelle,

Thank you for your e-mail. Unfortunately, I do not have the time to spend explaining (again!) what I need and waiting for you to see what can be done without any guarantee of the outcome. I hope that you see my point about the futility of a credit check for a $10/month line, since for the last 6 years I have paid my approximately $xxx monthly mostly on time. Additionally, no credit checks were necessary for the last x family lines added, and I was also able to increase my service by about $xx without a credit check. And this does not even get into the minimum $20 for the texting feature on my son's line -- one of my family lines has 50 messages for $2.99, but this option no longer seems to be available.

I realize that you are trying to help me, and I thank you for it. My cell number is xxx-xxx-xxxx. But as I said I simply do not have the time to spend on these endless phone calls with your company. If you can resolve something for me, great, let me know via e-mail. If not, it is fine, as I plan to visit my local AT&T store shortly.

The bigger issue is how user-unfriendly this whole process has become, and it is your marketing executives that need to do some soul searching about whether this is an ethical or sustainable way to continue doing business.

Thanks once again.

September 8: Annelle

Hello Dr. Zilberberg,
I appreciate your candid response and your patience. I know your time is valuable therefore I will keep this brief...
What kind of wireless device were you interested in for your son? I know from your email that you want to keep your same plan and to place him on texting pay per use at a cost of .20 per text (incoming/outgoing, each are charged per text).
Please advise and I will continue to work on this end. Hopefully you will not have to travel to the store if I can get this done for you.
Thanks so much!
Annelle M

September 8: me

Annelle,

Thank you for following up.
I want to get him a very simple flip phone that is free or very cheap. It does not have to have a camera or any other fancy features.

Thank you once again.

September 8: Annelle

Good evening Dr. Zilberberg,

After great effort, I have found that ALL entities (including the store) will require a credit verification for new lines of service even though the monthly cost is not very great. The other option, without performing the credit verification, is prepaid.
I know this is disappointing and I hate to be the bearer of bad news. However, if you do consent to the credit verification, I can have a nice phone ordered for you son at no cost sent directly to your home in roughly 2 days. There are approximately 15 different devices available for new activations such as samsung solstice 2, palm pixi, sharp fx, pantech impact, samsung strive, etc.
Please kindly respond and let me know how you would like to proceed either postpaid or prepaid. I will be looking for your response.
Thanks!

Annelle M

September 9: me
Dear Annelle,

Thank you for your diligence.
There is absolutely no way I am consenting to disclosing this highly sensitive information for a $9.99/month fee. Furthermore, my impression was that AT&T charges one month ahead, so there is no credit involved. Is this not correct?

This feels like a frivolous usurpation of what is reasonable. And given the frequent reports of confidential data breaches, I am not interested in subjecting myself to this risk. I will have to withhold this portion of my business form AT&T and go with the pay-as-you-go scheme.

Thank you. I would appreciate if this issue were brought to the attention of the management of AT&T so that a more reasonable policy can be developed.

This was the last correspondence, and I purchased a Tracfone for him. Researching it made me think seriously why more people are not using it and why we continue to allow ourselves to be strong-armed by these mammoth corporations who in a purely Orwellian example of doublespeak want to convince us that handing over the reins to them is a good thing for us. Tracfone: no activation fee, no contract, no intrusive credit check, pay as you go, full texting, web and e-mail capabilities, reasonable rates. What more does one need, especially as an emergency use phone for a kid?

So, thank you, AT&T, I am eternally impressed with and grateful for your customer service.

Supersize me, the AT&T way

2011-09-06T17:21:00.000-04:00

File this under "etc."

I had one simple goal -- add a line to my AT&T family wireless plan for my son. I know, I know, I did check other carriers' plans and deals, since my contract with AT&T ran out long ago. But for various reasons I decided to stay with them. Why trade a known headache for an unknown one? Anyway, I went online with every intention of being finished in the space of 10 minutes. How naïve...

I wanted to get a basic plan, with just voice and minimal text messaging -- say 20 texts per month -- so that he can reach us in case of an emergency, but not abuse this disembodied conversation mode. And you would think that it would be easy to get this, right? Well, not so much. The only text option that came up on the screen was unlimited texting for $20/month. This was infinitely more than I needed, so I initiated a chat with a representative. He was effusively polite, and every time I volunteered information or responded to a question he thanked me very much for sharing this information. After several volleys, in which I thought I had conveyed my dilemma clearly and succinctly, he came back with "So, if I am understanding you correctly, you wish to change the text messaging plan on your phone." I did a rapid-fire "No, no, no," trying to get ahead of his rogue fingers as I imagined them poised to hit "change plan." You see, I am still traumatized from a recent experience with the cordial AT&T customer service representatives who "helped" me with an issue.

Just to give you the flavor for that episode, in the process of resolving a signal issue, they implemented such changes in my texting and data plans as to require several weeks of phone calls with the AT&T business office, a call and a correspondence with the Attorney General of Massachusetts, and a follow-up call with AT&T on the heels of their communication with the AG. So, no, I am not interested in having them "help" me with plan changes. I politely excused myself from the chat and decided to tackle it on my own.

But first I chose to distract myself from the problem at hand by doing another task that I had meant to do: increase my monthly minutes. This I was able to do without any glitches, and my success encouraged me to try again with the new line. Perhaps I missed something the first time?

Having considered my choices at this point, I made the decision that I would pay for a limited number of text messages for my son, at $.20/message, and this would take care of things. Smugly congratulating myself on such a creative solution, I went to complete the purchase. After addressing just a couple of minor issues stemming from the fact that my billing address is a PO Box and not a street address, and because lately everyone except the USPS has decided that I have played a joke on them by giving a non-existent address (don't get me started on the joys of living in rural America), I was almost home. I just needed to input my credit card information and... Wait a second! What's this? A credit check consent? I had to give them my social security number and consent to a credit check? Because I tacked an additional $9.99 monthly service fee to my (much larger than that) bill? After being a customer for nearly 6 years? After being able to up my monthly minutes by more than $9.99/month, without being subjected to a credit check?!!

Well, I did what anyone in my position would have done: I ignored this prompt hoping that it was optional. But it wasn't. Fill it in or else take yourself to a brick-and-mortar store to get this settled. Which is what I am choosing to do. But there is a larger moral here.

How did we get to this place, where our anti-trust protections have resulted in basically two gargantuan corporations essentially screwing the public in any way they see fit? Why do they get to dictate the devices and services that I need to purchase? How is this a free market? And how is it that this technology, whose intent is to make our lives so much easier, made me go through a bewildering amount of useless machinations only to end up with what? An offer to take my social security number and subject me to a credit check? Really?

And lest AT&T feel singled out by my rant, this is the trend with many events and purchases in life. Home insurance, for example, which, despite rising premiums, does not give you a penny towards rebuilding a retaining wall that collapses in a flood. In fact, the system is set up in such a way as to require you to file a claim, get it rejected and give the company the reason to fire you as a customer for filing too many claims. Health insurance (I don't have to remind you the galloping pace of the rise in those premiums), which covers less and less every year. Cable companies, computer manufacturers, automobile vendors, they are the ones that seem to know better than I what it is that I need, and they constantly and with impunity wrestle me into straightjackets of their packages. Where am I, the customer in all this? This old familiar strategy to maximize returns has been so successful in the food business that its legacy is the obesity epidemic, proliferation of chronic disease and shortening life spans. Is this really how we want to continue?

I will get that line for my son, and I will get only what I need. I would prefer not to be so dependent on this stuff; alas, I am. But mark my words, there is enough bad taste building among my fellow humans to start exploring alternatives. I only wish that the government were really in the business of protecting its citizens from unethical practices rather than pandering to the highest bidder. I am ready to stop being viewed as a giant walking ROI potential, and start being respected as a citizen and a human. How about you?

You want to know #6?

2011-09-01T18:44:00.000-04:00

Actually, it should really be #1. I am referring to the list I blogged yesterday of my top 5 reasons for rejecting a manuscript. The most important reason, which I failed to mention is...

... drum roll, please...

6. No "Limitations" paragraph
This is something that no manuscript should neglect, as every study, even the most well designed and executed randomized controlled trial, has limitations. So, in every paper that I write, my third paragraph from the end is devoted to the laundry list of limitations. And it should not be merely a laundry list, no. Each limitation mentioned needs to be put into the context of how it may have influenced the results, directionally and magnitudinally (oh, whatever), if applicable.

So, no limitations paragraph, no "Accept" from me!

My top 5 reasons for rejecting a manuscript

2011-08-31T11:51:00.001-04:00

Here are five manuscript transgressions that make me hit "Reject" faster that you can blink. The first four in particular do not instill confidence in what you actually did in the study.

1. Matched cohort masquerading as a case-control
This happens quite a bit with papers submitted to third and fourth tier journals, but watch out for it anywhere. The authors claim to have done a matched case-control study, where there is indeed matching. However, the selection of participants in the study is based on the exposure variable, rather than the outcome. Why is this important? Well, for one, the design informs the structure of the analyses. But even more fundamentally, I am really into definitions in science because they allow us to make sure we are talking about the same thing. And the definition of a case-control study is that it starts with the end -- that is to say, the outcome defines the case. So, if you are exploring whether a Cox-2 inhibitor is associated with mortality from heart disease, do not tell me that your "cases" were defined by taking the Cox-2 and controls were the ones that did not take it. If you are enrolling based on exposure, even if you are matching on such variables as age, gender, etc., this is still a COHORT STUDY! It is a different story that this may not be the most efficient way to answer the particular example question, and a real case-control might be better. In order to call your study case-control, you need to define your cases as those who experienced the outcome, death in our example, making the controls those that did not die. I know that this explanation leaves a thick shroud over some of the very important details of how to choose the counterfactual, etc., but that is outside the scope here. Just get the design right, for crissakes

2. Incidence or prevalence, and where is the denominator?

I cannot tell you how annoying it is to see someone cite the incidence of something as a percentage. But as annoying as this is, it alone does not get and automatic rejection. What does is when someone tells me this "incidence" in a study that is in fact a matched cohort. By definition, matching means that you are not including the entire denominator of the population of interest, so whatever the prevalence of the exposure may seem to be in a matched cohort is the direct result of your muscling it into this particular mold. In other words, say you are matching 2:1 unexposed to exposed and the exposure is smoking, while the outcome of interest is the development of lung disease. First, if you are telling me that 10% of the smokers developed lung disease in the time frame, please, please, call it a prevalence and not an incidence. Incidence must incorporate a uniform time factor in the denominator (e.g., per year). And second, do not tell me what the "incidence" of smoking was based on your cohort -- by definition in your group of subjects smoking will be experienced by 1/3 of the group. Unless you have measure the prevalence of smoking in the parent cohort BEFORE you did your matching, I am not interested. This is just stupid and thoughtless, so it definitely gets an automatic reject (or a strong question mark at the very least).

3. Analysis that does not explore the stated hypothesis
I just reviewed a paper that initially asked an interesting question (this is how they get you to agree to review), but turned out to turn the hypothesis on its head and ended up being completely inane. Broadly, the investigators claimed to be interested in how a certain exposure impacts mortality, a legitimate question to ask. As I was reading through the paper, and as I could not make any heads or tails out of the Methods section, it slowly began to dawn on me that he authors went after the opposite of what they promised: they started to look for the predictors of what they set up as the exposure variable! Now, this can sometimes still be legit, but the exposure variable needs to be already recognized as somehow relating to the outcome of interest (hey, surrogate endpoints, anyone?). This was not the case here. So, please, authors, do look back on your hypothesis once in a while as you are actually performing the study and writing up your results.

4. Stick to the hypothesis, can the advertising
I recently rejected a paper that asked a legitimate question, but, in addition to doing a shoddy job with the analyses and the reporting, did the one thing that is an absolute no-no: it reported on a specific analysis of the impact of a single drug on the outcome of interest. And yes, you guessed it, the sponsor of the study was the manufacturer of the drug in question. And naturally, the drug looked particularly good in the analysis. I am not against manufacturer-sponsored studies, and even those that end up shedding positive light on their products. What I am against is random results of random analyses that look positive for their drug without any justification or planning. So, all of this notwithstanding, the situation might have been tolerable, had the authors made a credible case for why it was reasonable to expect this drug to have the salutary effect, citing either theoretical considerations or prior evidence. They of course would have had to incorporate it into their a priori hypothesis. Otherwise this is just advertising, a random shot in the dark, not an academic pursuit of knowledge.

5. Language is a fraught but important issue
I do not want to get into the argument about whether publishing in English language journals brings more status than in non-English language ones. This is not the issue. What I do want to point out, and this is true for both native and non-native English speakers, is that if you cannot make yourself understood, I do not have either time or the ability to read your mind. If you are sending a paper into an English language journal, do make your arguments clearly, do make sure that your sentence structure is correct, and do use constructions that I will understand. It is not that I do not want to read foreign studies, no. In fact, you have no idea just how important it is to have data from geopolitically diverse areas. No, what I am saying is that I volunteer my time to be on Editorial Boards and as a peer reviewer, and I just do not have the leisure to spend hours unraveling the hidden meaning of a linguistically encrypted paper. And even if I did, I assure you, you are leaving a lot to the reviewer's idiosyncratic interpretation. So, please, if you do not write in English well, give your data a chance by having an editor take a look at your manuscript BEFORE you hit the submit button.

Botox and empathy: Less is more

2011-08-26T12:28:00.000-04:00

I am kind of stuck on this whole Botox-empathy thing. A recent study from researchers at Duke and UCLA implied that people who get Botox to attenuate their wrinkles also seem to attenuate their empathic ability. Somehow their inability to mimic others' facial expressions impairs the firing of their mirror neurons and they top feeling empathy. Wow!

But think of it -- Botulinum toxin, arguably one of the most potent poisons known to humans, is being used essentially recreationally as a drug, quite possibly an addictive one. Who thought this was a good idea? OK, don't answer that.

To be sure, the same toxin in a therapeutic preparation can help people with paralysis release painful contractures, and this is a wonderful advance. Just as morphine is a terrific pain reliever under the right circumstances. But used recreationally? Everyone is aware of the havoc it can wreak, both personally and societally. So, how did we justify allowing this most potent of all poisons to be injected into perfectly healthy (and beautiful, I might add) aging faces?

File this under "Go figure." Another opportunity for "less is more."

Side effects: The subject must become the scientist

2011-08-25T11:43:00.000-04:00

A few weeks ago someone I know, a normally robust and energetic woman, began to feel fatigued and listless, and had some strange sensations in her chest. She presented to her primary care MD, who obtained an EKG and a full panel of blood tests. The former showed some non-specific changes, while the latter was entirely normal. Although reassured, she continued to experience malaise. When she fetched her EKG, she received a copy with the computer interpretation indicating that, in its wisdom, the program could not rule out a heart attack. Given that her symptoms continued, and now anxiety was piled on top, she presented to the ED, where a heart attack was excluded, and she was scheduled for a stress test. In the subsequent weeks the symptoms continued off and on, and the stress test turned out to be negative for coronary disease. Great, mazel tov!

What I failed to mention was that just prior to the onset of her symptoms, she had been started on 5-fluorouracil cream for a basal cell skin cancer. And while she did not commit my current device of omission with her doctors (including the dermatologist who prescribed the drug), all denied her constellation of symptoms as a potential side effect. And granted, when I looked it up, there was no mention of anything like fatigue and listlessness. So, does it mean that it is not within the realm of the possible that this drug was responsible?

Not at all. And here is why. Our adverse event reporting is essentially a discretionary system. Here is what the FDA says about their Adverse Event Reporting System (AERS):

Reporting of adverse events from the point of care is voluntary in the United States. FDA receives some adverse event and medication error reports directly from health care professionals (such as physicians, pharmacists, nurses and others) and consumers (such as patients, family members, lawyers and others). Healthcare professionals and consumers may also report these events to the products’ manufacturers. If a manufacturer receives an adverse event report, it is required to send the report to FDA as specified by regulations.

What this means is that, when a patient complains to a doctor of a symptom, even when its onset is in obvious proximity to a particular medication, the doctor is not compelled to report it. The most an average physician will do is look up the known AE profile of the drug and at best look up its interactions with other medications. But one is not generally inclined to use one's imagination (and the constraints of the shrinking appointments spread across exponentially growing cognitive loads conspire against it too) to entertain the possibility that the current problem is related. And yet since many AEs are particularly rare, the knowledge about them must necessarily rely on scrupulous reporting by the prescribers into a central repository. This is what is missing: not the repository, but the impetus to report.

So, when we go looking up side effects of a given medication, we must take the information for what it is: a woefully incomplete list of what has been experienced by other patients. And when someone asks "Do statins make you stupid," instead of denying the possibility, we should just admit that we don't know. Because once drugs are released by the FDA into the wild of our modern healthcare, by relying on others' reports of AEs we become inadvertent enablers of our ignorance about them.

My friend's symptoms abated after she finished the course of the 5-FU cream. None of the MDs bothered to report her symptoms to the AERS, and nor did she. I am not even sure that any of the players were aware of the possibility. Oh, well, an opportunity lost. We need to feel responsible for gathering this knowledge. The subject must be empowered to become the scientist; this is the only way we can get the full picture of the harm-benefit balance of our considerable and unruly pharmacopeia.

If you want to report a possible side effect of a medication, this FDA web page will guide you through the process.

Counterfactuals: I know you are, but what am I?

2011-08-17T11:14:00.000-04:00

It occurs to me that as we talk more and more about personalized medicine, the tension between the need for individual vs. group data is likely to intensify. And with it, it is important to have the vocabulary to articulate the role for each.

Scientific method, in order to disprove the null hypothesis, demands highly controlled experimental conditions, where only a single exposure is altered. While this is feasible when dealing with chemical reactions in a beaker, and even, to a great extent, with bacteria and single cells in a petri dish, the proposition becomes a whole lot more complicated in higher order biology. In this way, the phrase "all things being equal" must really apply to the individuals or groups under study.

We call this formulation "the theory of counterfactual," and it is defined in the following way by the researchers at the University of North Carolina (see slide #3 in the presentation):

Theory of Counterfactuals
–The fact is that some people receive treatment.
–The counterfactual question is: “What would have happened to those who, in fact, did receive treatment, if they had not received treatment (or the converse)?”
–Counterfactuals cannot be seen or heard—we can only create an estimate of them.
–Take care to utilize appropriate counterfactual

So, essentially what it means is figuring out what would have happened to, for example, Uncle Joe if he had not smoked 2 packs of cigarettes per day for 30 years. Now, our complexity as the human organism makes it impossible (so far) to replicate Uncle Joe precisely in the laboratory, so we must settle for individuals or groups of individuals that resemble Uncle Joe in most if not all identifiable ways in order to understand the isolated effect of heavy smoking on his health outcomes.

So, you see the challenge? This is why we argue about the validity of study designs to answer clinical questions. This is why a randomized controlled trial is viewed as the pinnacle of validity, since in it, just by the sheer force of randomness in the Universe, we expect to get two groups that match in every way except the exposure in question, such as a drug or another therapy. This is why we work so hard statistically in observational studies to assure that the outcome under examination is really due to the exposure of interest (e.g., smoking), "all other things being equal."

But no matter how we slice this pie, this equality can only be approached, but never truly reached. And this asymptotic relationship of our experimental design to reality may be OK in some instances, yet not nearly precise enough in others. We just cannot know the complete picture, since we only have partial information on how the human animal really works. And this is precisely what makes our struggle to infer causality problematic, and precisely what introduces uncertainty into our conclusions.

What is the answer? Is it better to rely on individual experience or group data? As always, I find myself leaning inward toward the middle. Because an individual's experience is prone to many influences, both internal, such as cognitive biases, and external, such as variations in response under different circumstances, it is not valid to extrapolate this experience to a group. In the same vein, because groups represent a conglomeration of individual experiences, smoothing out the inherent variabilities which ultimately determine the individual results, study data are also difficult to apply to individuals. For this reason medicine should be the hybrid of the two: the make-up of the patient can partly fit into the larger set of persons with similar characteristics, yet also jut out into the perilous territory of idiosyncratic individuality. This is precisely what makes medicine so imprecise. This is precisely the tension between the science and the art of medicine. Because "counterfactuals cannot be seen or heard," Uncle Joe!

Medicine and the internet: Harnessing the yottabytes

2011-08-16T12:48:00.000-04:00

What if medicine in the US is just like the internet? What if it is just as difficult to separate the chaff from the wheat in medicine as it is on the web?

Both the curse and the blessing of the web is its accessibility. This means that anyone's voice can be heard. And it also means that anyone's voice can be heard. So, we are just as likely to stumble upon drivel as we are on information gold. And what takes time and skill is separating the two into neat piles, one to be ruthlessly discarded, and the other cherished for how it enriches us. To be sure without the web we might not have had access to either, and it is the egalitarian nature of the internet that gives us such a variety of sources in our information diet.

Now, let's look at medicine. Every day we hear about how much noise there is in the field, and this noise is difficult, if not impossible, to separate from the signal. Some signals are becoming much clearer, and they tell us that by being too egalitarian in medicine, we have likely been causing great harm. Take, for example, PSA and mammography screenings. The drumbeat of harm associated with these highly non-specific tests and the resultant chase after false positive results, is getting deafening, and rightfully so. Every day we hear that researchers have uncovered a breakthrough mechanism or treatment, and we hear with increasing frequency that a treatment previously thought to be sacrosanct is a bunch of rubbish. What gets lost among all this noise is the possibility of a true breakthrough in disease management or treatment or cure.

Think how hard it is to separate general valuable content from bunk on the web. Now, think of the logs of increase in the levels of difficulty of this task in medicine, where difficult concepts are further shrouded in the opaque cloth of arcane and obfuscating terminology. In fact, it is so difficult, that the class previously designated as the interpreters of this information for the lay public, physicians, are unable to keep up. There is a need for a whole new class of interpreters now -- researchers and patient advocates. And while this is good for the market and the economy, since it creates jobs that had not existed before, it begs a more critical evaluation vis a vis its impact on public's health. It also begs the question of the value of this gadgetry and information glut in medicine -- what is truly the wheat and what is the chaff? And what happens when you continuously try to drink from a fire hose? And do we turn down the stream, or is there another way?

Is it feasible to limit this stream of idea and information generation? Furthermore, is it sensible to do so? Many worry that putting limitations on this is tantamount to stifling innovation. But what is innovation? The most pertinent definition to the current discussion in the Merriam-Webster dictionary is "a new idea, method or device." Nowhere does the definition incorporate the value of this idea, method or device. Perhaps it is left to the free market to determine this value and ultimate use of such innovation. Well, in a market that claims to be free, but is filled with cynical machinations in the form of favoritism, subsidies and pricing games, is objective value really what is valued? And indeed, given the complexity of these "innovations", is it even possible for the end-user to judge their value, even if the market were free?

Yet, even despite all these challenges to establishing the value of innovation on the back end, I am not sure that centrally limiting idea generation is either feasible or right. In the case of ideas on the web, I have come to the conclusion that such microblogging platforms as Twitter can be invaluable filters of information, where my network of favorite tweeters whom I follow faithfully provides me with the wheat that has already been cleaned, yet not always overprocessed. Is this possible in medicine? I know that the FDA and CMS are supposed to provide some filtration for such medical information and interventions, but each is statutorily handcuffed and gagged not to stray beyond their legislative agendas. Therefore, a value filter should not be a body beholden to the letter of the law, or to political or financial interests. It needs to be driven by the spirit of scientific curiosity, objective evaluation and pragmatism. Most importantly, it must be open to a conversation that incorporates respectful dissent and many different perspectives.

Twitter arose out of the drive to share information, and it has shaped itself as a tool for developing value in the gargantuan and ever-growing world of yottabytes. Perhaps it is citizen bloggers and tweeters, including e-patients and clinicians and researchers and writers and others, who will ultimately solve this information glut in medicine by extracting the kernel of usefulness from this morass of vegetation. Harnessing this power systematically and accurately is the next challenge of our information age.

Because ultimately, for human cognition and health, less is more. And we are still human.

Quality measures: Process, outcome, or both?

2011-07-29T11:41:00.000-04:00

In the last week I wrote about our quality improvement, or QI, efforts in healthcare. And although there is a burgeoning field representing itself as the "science" of QI, I question much of its scientific validity. As always, VAP is my poster child for these discussions, where neither the definition of the condition itself nor its prevention efforts are subject to much scientific scrutiny. This makes VAP have a surreal, ghost-like quality: now you see it, now you don't. And this alone makes it difficult to assess prevention efforts. Much as in the heated mammography debate, where passionate anecdote prevails, the sanctity of the QI rubric blunts the usual critical approach to the data.

So, the central point that I made in this post was essentially to devalue the VAP eradication efforts as not grounded in solid scientific evidence. What has occurred to me, however, is that this position may be in fact at odds with a realization I blogged about here and here, wherein I agreed with Dan Arieli's suggestion that outcomes in the real world, where they are influenced by so much randomness, are not the thing to reward. It would be much more rational to reward best efforts at best results, thus the process rather than the outcome. So, here is the apparent contradiction: On the one hand I agree that outcomes may be too unpredictable, being that they are influenced by too many factors that are not in our control, yet I am also advocating that we start measuring such outcomes as antibiotic use associated with VAP and its reduction. What gives?

Well, on the one hand, I am OK with contradiction; life is full of instances where we have to hold conflicting information and feelings together. But as a scientist it is my predisposition to analyze (which literally means splitting into smaller, more manageable chunks), so I have given this ostensible paradox more thought. What I came up with is that measuring process is the right thing to do, but only under very specific conditions. Avedis Donabedian, who is considered the father of quality science, introduced the triad of structure-process-outcome as the backbone of quality science. This relationship certainly lends validity to the "process" metrics as surrogates for "outcome." But the condition that has to be met is that there be an actual correlation between the process and the said outcome. If there is no such solid correlation, then we are simply going through the motions, doing a rain dance to cause rain.

So, what I have said about VAP prevention in particular is that we are nowhere near being able to say that the recommended processes correlate with any changes in meaningful clinical outcomes. And because the data on these interventions are so weak, throwing massive resources behind implementing them is irrational and resembles religious fervor more than scientific pragmatism.

It is entirely understandable that we would jump on this bandwagon so rapidly, given the magnitude of harm in our healthcare system combined with the need to reign in the healthcare spending. But there is a more subtle point to be made here too. It relates to the fertile soil of our American psyche, where doing something is always perceived as better than thinking about our course of action, which is frequently referred to with contempt as "doing nothing." In the end, this crisis response mentality is good in a crisis, but potentially detrimental in the long term: we are unlikely to be altering meaningful outcomes, and we are spending billions of dollars on interventions lacking evidence.

So, I stand behind both of my assertions and maintain that they are not mutually exclusive. Yes, outcomes are subject to much randomness; yes, processes known to alter these outcomes are the sensible measures of our efforts to improve quality; and yes, these processes need first to be rigorously validated for their impact on the outcomes in question. Anything short of this pathway is not just a waste of our collective resources, but a manipulation of the public trust. And that is as far from the intent of science as it can get.

Health surveys: Run the other way!!!

2011-07-27T20:06:00.000-04:00

Sometimes when I get an unsolicited call about answering survey questions, I feel a karmic obligation to participate; after all, if everyone said no to everything, I would not have any data to analyze. So, for this very reason, I just got off the phone with a poor young woman conducting a survey who called me randomly. The survey had to do with healthcare delivery, and she had no idea what she was getting herself into. First, I queried her whom the survey was for. She proceeded to tell me that she did not have that information specifically, but gave me a general idea of who the customers tend to be. Then she launched into the survey questions.

Now, I realize that they all have to ask the same questions the same way in order not to bias the data. But man, who writes these questions? "What would you say is the reputation of the cardiac surgery program at thus-and-such a hospital in your area: a). good locally, 2). good locally and state-wide, 3). good locally, statewide and regionally, 4). good locally, statewide, regionally and nationally, 5). good locally, statewide, regionally, nationally and internationally, or 6). not good at all?" Well, what the heck do you mean by "reputation"? You mean what is the gossip about Dr. Smith in my community? Or do you mean what kind of care they provide in terms of timeliness, evidence, shared decision making, post-operative complications, what? Then came "if you or your family member needed a cardiac procedure, how comfortable would you be going to this facility? 1). very comfortable, 2). somewhat comfortable, 3). somewhat uncomfortable, and 4). not at all comfortable?" How the heck should I know? I have not researched all the local facilities, I have not checked on their outcomes, I have not interviewed all of their cardiac surgical teams (yes, including anesthesia), I do not know what their infection control track records are, and, most importantly, how willing they are to treat me as an individual rather than a source of income. And then, for every hospital she mentioned (and there were quite a few), she went through the same litany of meaningless questions.

And then she asked me if I am familiar with some of the well-known quality-rating organizations. And she included US News and World Report Hospital Ratings! And I don't even believe the CMS got it anywhere near right!!! Oy! What do the answers to these questions from someone who is not steeped in the data mean anyway? If researchers and providers have not arrived at the appropriate metrics for quality, how meaningful are the lay public's opinions on these matters?

And finally, a group of questions that let the cat out of the bag as to the purpose of the survey. She told me a story first, of a large regional medical center in the area building a new multi-million dollar state-of-the-art cardiac care facility. Sexy new equipment, individual patient rooms, targeted and individually-tailored treatment plans, all the buzzwords of the brave new world of medicine. And then she asks me would I be comfortable going to this facility. What am I supposed to say? I have no idea! How do I tell this poor child that the questions are written in an absurd way and smack of marketing? How do I explain to her that this facility will probably need to recoup their capital investment, and, therefore, has a conflict of interest when it comes to caring for me? How do I teach her that this is the problem with American medicine, this very over-reliance on reputations and expertise to tell us to over-indulge in interventionism at the expense of our health and budgets?

Anyway, I will not belabor this further. My advice to survey fielders: If you want to market to the gullible, go ahead and call people randomly and ask your market-building question. And if a person tells you she is a physician and a health services researcher to boot, run, don't walk, the other way.

Tipping a sacred cow: QI under the microscope

2011-07-26T12:07:00.000-04:00

So much media and journal space has been devoted to financial conflicts of interest, particularly within and related to pharma and device manufacturers, that to write any more about it may be redundant. On this site we have also intermittently addressed COI from other perspectives, such as financial interest of the members of the American College of Radiology in maintaining mammography screening status quo, thinly veiled in its own version of the pernicious "death panel" language. We have also spoken a bit about the non-financial COI. And even though we are so very much aware of COI's potential to lurk around every corner, there are still some surprises.

Take the sacred cow of "quality improvement" in healthcare. Even the name, much like the "pro life" moniker, suggests that it is untouchable in its purity and nobility of purpose. So necessary is it because of the epic magnitude of morbidity and mortality attributed to healthcare itself, that the billions of dollars spent on it seem unquestionably justified. Indeed, much like our public education system, the QI movement garners higher and higher allocations simply due to the sheer face validity of the assumption that more of it is better. And the most fascinating aspect is that, in our current zeal for sensible economic allocation through evidence, QI, much like education, appears immune to scrutiny. This is the very definition of politics driving policy.

I return to the case of ventilator-associated pneumonia, or VAP, as the poster child for this movement. I have already alluded to the fact that definitionally VAP is a slippery slope: its diagnosis varies based not only on the tools used to diagnose it, but also depending on who is doing the diagnosing. Yes, indeed, what one clinician calls VAP another may call absence of VAP. I have also dissected the weak evidence behind some of the strongest purportedly evidence-based recommendations aimed at VAP prevention. But what if VAP itself is the wrong endpoint? What if we are spending untold dollars and other resources on a futile pursuit?

Do you feel yourself bristling yet? If you said "yes", it is a normal response I get from my colleagues and people who read my scholarly papers. Because how can anyone be against QI? Well, I am not against QI. I am simply against sanctifying QI as a sacred cow and thus shielding it from a sensible and rational evaluation.

So, if you are over the initial shock, allow me to explain myself. I am sure you have heard of surrogate endpoints. Here is a definition from Wikipedia:

In clinical trials, a surrogate endpoint (or marker) is a measure of effect of a certain treatment that may correlate with a real clinical endpoint but doesn't necessarily have a guaranteed relationship. The National Institutes of Health (USA) defines surrogate endpoint as "a biomarker intended to substitute for a clinical endpoint".^[1]^[2]

Surrogate markers are used when the primary endpoint is undesired (e.g., death), or when the number of events is very small, thus making it impractical to conduct a clinical trial to gather a statistically significant number of endpoints. The FDA and other regulatory agencies will often accept evidence from clinical trials that show a direct clinical benefit to surrogate markers. ^[3]

This begs the question of what constitutes a "real" clinical endpoint. Well, in my simplemindedness I think of them as endpoints that matter to the patient or in the long run. So, death, disability, quality of life, functionality, these are the real endpoints. Something that alters one's life or threatens it is a real endpoint. Thus, blood pressure and cholesterol are surrogate endpoints, since they usually, but not always, correlate with the risk of a myocardial infarction or death. But what if such a correlation did not exist? Furthermore, what if a cholesterol level was measured with, say, tea leaves, and therefore was subject to a tremendous variation in detection? Would we then spend hundreds of billions of dollars on trying to alter this factor or would we calmly and rationally walk away and look for something that truly impacts the real outcome of a heart attack or death? I think I am making my point fairly clearly.

Let me explain why I think that VAP is but a surrogate outcome, and, given its diagnostic challenges, not a sensible one in the least. VAP by definition occurs in patients on mechanical ventilation (breathing machine), whose quality of life is fairly badly damaged in the short term. The literature would suggest that not all VAP impacts mortality adversely, but some forms of VAP indeed do, particularly VAP that develops late in the course of illness. So in this VAP does correlate with a real endpoint. Also, there is very little doubt that getting VAP prolongs one's dependence on mechanical ventilation, and increases the duration of the stay in the ICU and hospital overall. So, this can be considered not a very good, albeit real, outcome. An additional point to remember is that VAP engenders the use of additional, usually broad spectrum, antibiotics, putting both the individual and the society at risk for such unwanted consequences as the emergence of highly resistant microorganisms.

So, even though VAP is a surrogate endpoint, it certainly seems to fit the bill for something we would want to prevent. But here is the monkey wrench in this argument: what seem to be great surrogate endpoints do not always end up correlating with clinical reality. The association of VAP with morbidity and mortality has been detected in mostly retrospective observational studies. Trials of VAP prevention rarely, if ever, report any endpoint other than VAP. And, given how elusive VAP diagnosis is, there is plenty of room for gamesmanship so pervasive in the real world to make any data fit our preconceived hypotheses and political needs.

So, what is my point? My point is that if QI wants to be a science, it needs to be subject to the same rules that all other science is guided by. Since we do not even know how much money we are spending on the ubiquitous QI efforts (likely hundreds of billions), and since we are not sure what they are accomplishing (see my many prior posts on the lack of validity of current claims in VAP prevention), we need to pause and ask ourselves whether the cheering alone justifies such an investment. I hate to say it, but can we really trust those with most to lose, financially and politically, if in reality QI does little more than lather the masses, to be the oracles of truth about the results of these efforts? The cognitive biases alone should disqualify them from being the arbiters of their own success. So, if we do not want to continue to indulge the principle of diminishing returns in QI, we need to take a sober look at what we have invested and what this investment has accomplished. Then and only then can we claim to practice evidence- rather than politics-or dogma-based policy.

Whose perspective?

2011-07-22T11:20:00.001-04:00

After a long hiatus filled with travel, work and lack of inspiration to write anything, I have chosen this arguably hottest day of the year to venture forth again. But I will make this brief, as it seems that everything that needs to be thought and said has already been thought and said. Yet who is listening?

Anyhow, to suspend my natural cynicism, I want to talk about perspective. No, not the perspective that makes parallel lines converge in the distance, but the one that gets lost in many of our political, civic, business, and, yes, even scientific discussions. I am talking about my perspective, your perspective, societal perspective, etc. I was inspired to write this because of these tweets by Gary Schwitzer to Kaiser Health News about a story on their web site:

Linking out to the story, I learned that a consulting arm of Disney is teaching hospitals about hospitality. Since there is going to be a financial incentive for these hospitals to deliver good customer service, many are feeling that an investment in this type of training will help them maximize these new reimbursements. Hurrah and ta-da!

Well, Gary likes to burst these one-sided bubbles, and so he rightfully asked about the costs. What was baffling to me was the response by the KHNews who did not seem to appreciate the importance of reporting the costs or the various perspectives that these costs represent. So, this seemed like a teachable moment, and here is the teaching.

In outcomes research, we are always interested in understanding the perspective for both the costs and the benefits of interventions. In health outcomes these perspectives are broadly represented by the patient, the provider, the hospital, the payer, the employer, the manufacturer, the society, to name a few. These are just some of the examples of the usual stakeholders involved in healthcare decisions. Because our healthcare is such a fragmented disaster, many of these perspectives find themselves at odds with one another. Just think of the patient who wants to get what she perceives as a life-saving treatment that in reality has a 1% chance of helping at a cost of $600,000 per treatment course. From her perspective, since she is insured, this investment is well worth the cost. For a payer, however, this means $600,000 (multiply this by 100 in order to determine the cost to save 1 life) that cannot be spent on something else that can help more people more predictably. And if this payer is the taxpayer, the societal perspective enters the picture, where we have to decide what amount of money is worth spending on possibly saving one life -- is $60 million reasonable? Perhaps. But these are not simple questions, and, as such, do not have simple answers. In addition, all conversations that we hear or engage in have multiple perspectives. This is why a black-and-white approach is so divisive: it generally emphasizes two diametrically opposed perspectives.

So, next time you hear about death panels or Mickey Mouse teaching hospitals how to maximize their revenue, consider the broader implications from may different perspectives. Chances are you will find yourself agreeing with more than one point of view. And when this happens, you will know that you have learned an important lesson and can now start engaging in more nuanced and thus productive debates, many of which will shape our society's future.

h/t to @garyschwitzer for this KHN story

In honor of my father

2011-06-18T20:56:00.001-04:00

My relationship with my Dad was not the easiest: we were so alike that all of his flaws were magnified in my estimation and bugged me disproportionately. But it was not always this way. When I was a kid in Odessa, we were really close. He told me tall tales to get me to eat my dinner, he took me for walks, he told me stories about our city. He instilled his love of Odessa in me, a love which I had forgotten until just the last couple of weeks.

I finally went back to Odessa after a 35-year absence. Just thirteen when we left, I was a reluctant party to our emigration. Despite my resistance, once we came to the US, I assimilated most successfully and repressed every notion of being from Odessa -- accent, uniquely Odessit attitude toward life, everything. Until I went back 10 days ago.

I arrived on a plane from Istanbul on Wednesday morning. The airport is a tiny one-story building with an attached airfield. Its only concession to modernity is buses that cart passengers between the building and the aircrafts. Stepping out of the plane, I sensed something achingly familiar in the light, the smells and sounds of the city. And this sense was to persist through the three days that I spent there.

Odessa, known as the "Pearl of the Black Sea", is the ultimate planned city. Established by the order of Catherine the Great in 1794, its role was to be a door to the southern trade routes, leading easily to the Ottoman Empire, as well as the ports of Western Europe. The architecture of the city, its art and literature, reflect the liberal, permissive attitude fostered by its founding planners that made it a civic and commercial success prior to the October Revolution of 1917. Now, emerging from her 70-year repression, Odessa greeted me with open arms.

The love for the city I learned from my father permeates everything in Odessa: Odessits love their city, despite her myriad flaws and imperfections. In this they are not given to extremes of thinking her either perfect or abominable. Their attitude is one of reverence and understanding. Some of her streets are perfect, while others are at the edge of ruin, and she still smiles and winks to us knowing that she transcends these minor details.

I went to the most treasured of places in Odessa, the Opera Theater. I sat in the sixth row, close enough to smell the make up. It was Rigoletto, and as Gilda was singing her aria, I could feel my Dad next to me, nodding and humming along with the music, as he always used to do. And I finally understood his years of silent longing for this most unpretentiously beautiful of all cities I have ever been to. Because if you ever visit Odessa, it will happen to you too.

So, on this, my first Father's day without you, I give you my love for our beautiful city.

Remember: Social media is social

2011-05-25T15:28:00.000-04:00

A few days ago, several news outlets reported on Newt Gingrich's unfortunate turn of phrase about our President. In a speech in Georgia he referred to Obama as "the most successful food stamp president in modern American history." Promptly, David Gregory of NBC News called Newt on the potential racist connotations of this remark, thus precipitating a maelstrom of "liberal media" accusations from the usual suspects (here and here). And so it goes, possibly a careless figure of speech, possibly racially charged code, but everyone is now yelling and screaming about whether or not it was OK to say and subsequently to accuse.

There is a much bigger issue at stake than just the utterance of these words. While they offend me and make me think that they were spoken deliberately to elicit racial tensions, the point is that, in an obtuse sort of a way, they can be defended as non-racial in nature. Unfortunately, the speaker's intention is not the issue any longer, so much as the fact that these words can easily be construed as racially inciting by large swathes of the population. And this is an important point: once squeezed out, much like the toothpaste from its tube, these words cannot be forced back. They will exist in perpetuity and continue to elicit visceral reactions.

Such is the nature of public discourse. Which brings me to the discussion on Bryan Vartabedian's site about the etiquette of healthcare professionals on Twitter. The example of the particular Twitter stream from a healthcare professional who tweets anonymously under a nom de plume, precipitated a spirited discussion (see over 100 comments) about the propriety and professionalism of her messages in this public forum. While many saw her behavior as at the very least undesirable, some MDs and, more concerning, medical students, did not see a thing wrong with her eructations. Furthermore, she responded via Twitter that she was quite surprised by this tongue lashing and did not know what was so offensive in her messages.

But that is just it! Enough readers saw her words as being disrespectful and even contemptuous of patients, regardless of her intent. And this is the crux of the matter: the distance between the output, and its intent, and its interpretation by the listener or reader can be vast, especially when the message is reduced to 140-character snippets. This is why when people lament that communication is the biggest obstacle in all human relations, they are right. And different types of communication deserve different levels of scrutiny.

So, here is my bottom line. Whether we intend to be hurtful or disrespectful in our tweets or racially divisive, as in our political discourse, is immaterial. In an open forum, if we are concerned with being empathic and careful of others' feelings, we should think twice (or more) about what we put out there. Because extending this blogger's thought to all communications, Twitter, like a tattoo, is forever. And everyone is looking.

All harm and no benefit: Wake up and smell the pus!

2011-05-23T17:42:00.001-04:00

Ideas are a little bit like a pimple: There needs to be enough pressure that builds up to cause one to erupt into consciousness. So it has been with my thinking around value of certain consumables. And the more I think about it, the more I am convinced that the law of diminishing returns is scowling at us from its perch. Here are 3 examples that I hope will drive this point home.

Take cigarettes. They cost money to manufacture, they cost a boat load of money to buy, yet what do they produce that is of value? A rise in the teen's coolness quotient? A brief relaxation effect? Appetite control? Sultriness? A tax base? All of this is so squashed by the sheer size of the risk of disease and death that cigarettes cause as to make any of these potentially perceived "benefits" infinitesimally small, thus bringing the harm in the harm-benefit ratio into the range of infinity. So, huge investment, personal and societal, for a negative return.

Next comes food. I am sorry, but I am going to stop wringing hands and apologizing for being convinced that there is in fact unequivocal scientific proof (yes, I did say "scientific proof") that our gargantuan and infinitely dysfunctional food production system is poisoning our public. Obesity is but one manifestation of this diseased production. The potentially more catastrophic consequences include accelerated deforestation with the consequent extinctions and climate change, which is rapidly moving from the realm of abstract future into the concrete present. Many have written about the science behind all this, and I will not belabor it here. Yet it is clear that this is another example of ever-increasing investment and subsidies made at the peril of environmental and public health. So, again, we have a cigarette-like situation, where the numerator of harm overwhelms the denominator of benefit to such and absurd degree that it threatens to stamp us out, like a drunken Godzilla strolling down a city street, pulverizing everything to dust.

And of course, we must bring in "healthcare." I put it in quotes because there is so little health in this healthcare paradigm. It is striking to me how the "prevention" conversation has been hijacked by the vocabulary of "screening." And don't you practically feel like a criminal refusing some screening test or another, or worse yet, refusing to comply with your annual check-up? At the same time, every day we hear about harm due to overzealous search for what is wrong with us. Healthy people walk into a place of "healthcare" delivery and walk out at best with a handful of pills and a bunch of ICD-9 codes attached to them, or at worst end up dead (or so close to it that they wish the job had been completed). I am sure you all remember the paper in the Archives of Internal Medicine that precipitated this post, where a healthy woman ended up needing a heart transplant because of the dogged search for a diagnosis that did not exist. More recently, several lay press articles, including this one, have highlighted data on an increase in potentially life-threatening post-biopsy infections among men undergoing a prostate biopsy. And the kicker is that the PSA test, which is what usually leads to a biopsy, has been so convincingly characterized as completely unreliable that it is difficult to believe that anyone still submits himself to it. And to top it all off, Archives of Internal Medicine has just published this paper on screening for heart disease in asymptomatic people (this means schmos like you and I who hang out in our offices without any heart symptoms). And, not surprisingly, they found an increase in the diagnosis and treatment of heart disease among those screened compared to those not screened, and absolutely no impact on the outcomes! All harm or potential harm and no benefit! We should be outraged, but instead we demand more intervention.

Given our peaceful acceptance of pushing (yes, pushing) harmful stuff on the public without any pretense at benefit, just as the brazen strategies of the tobacco and monocultured foods have done for so long, it is not surprising that we are willing to dive head first into this quicksand of harm. It attests to how effective the PR industry's brainwashing is. But really, isn't it time to wake up and smell the pus?

In praise of not knowing

2011-05-20T15:43:00.000-04:00

Recently I had the occasion to tell my 10-year old an old secret: until I was into my forties, I had a strong belief that the rest of the people in the world knew something I did not know. I don't mean just about stuff I do not know, but about everything! It was unnerving, anxiety-provoking and self-defeating. Until one day I had the epiphany that most humans feel this way, not just me. So, be humbled by not knowing and move on.

Yet even more recently this line of self-examination has led me to the conclusion that I end up saying "I don't know" a lot. I read a definitive tweet from someone I respect, and I say to myself "I don't know"; I read a new paper in a journal and say, "Gee, I don't know", I hear a political speech, and I walk away saying, "I just don't know." Is it that I am an idiot, or intellectually lazy? Perhaps. But what is occurring to me more and more lately is that what we are convinced of today will be much less certain and obvious tomorrow, barring some truly sacred cows. This is called growth, and as far as I can tell is a desirable development.

On the other hand, saying "I don't know" sometimes means that it just does not make sense to take sides. I know that we have to apply current knowledge and not wait for perfect information, but I still do not see getting all polar about stuff. Most of the time we act like there are only two possibilities, and they are diametrically opposed to one another. Well these are false dichotomies promoted by our educational system, which drills into us the idea that there are only two answers to any question: the right one and the wrong one. What if this is untrue? What if we change the way we think about the world, and instead of seeing only the black and the white, the left and the right, the correct and the incorrect, we start really seeing the entire continuum of possibilities? What a fantastic variety of solutions we might stumble upon to our perennial questions!

A nice mind game could be trying to think about stuff without using words. Can we do that? It is thoroughly difficult, yet it is language that seems to bracket our conceptual understanding of the world within and around us. Take the word "race" or "gender", for example. These are human-made and defined terms, which are meant to distinguish rather than merge. Yet just think how uncomfortable we can be made by a person with an ambiguous gender identity, say. Why? Because he/she does not fit into our preconceived dichotomy? Uncertainty is uncomfortable, and dichotomies cure uncertainty. But I am not sure that nature is all that into dichotomies.

The human brain is wired for "belonging." I believe it is for this reason we gravitate to our respective extreme corners of thinking and being, instead of meeting somewhere in the isle. The isle is an uncomfortable place, yet that is where we must aim to be. All the borders we have created are imaginary separations. Instead we can reposition them as the glue that unifies that which lies to either side.

Here is to not knowing more!

Of pigs, Babel and totalitarianism

2011-05-19T12:32:00.000-04:00

Here is a weird thought: Our current food production is closer to the Soviet collectivization than to the free enterprise model. Outrageous? Not really.

I had avery interesting weekend. As some of you may know, I am finally going back to Odessa (no, not the one in Texas) after a 35-year hiatus -- we left when I was 13. Since I am only spending 3 days there, I have been doing a lot of soul-searching to figure out what I need to get out of the trip. Serendipity struck last Sunday, when in the midst of a gloomy morning I went online to search for something to do in the Valley, indoors, and came upon this. Now, although most of you probably have never heard of Babel, he was a very well known and respected author of the Soviet era. A Jew in peri-revolutionary Odessa, he was a master story teller, best known as the bard of the colorful life of the Moldavanka, a district of Odessa particularly rich in poverty and Jewish gangsters. My father was a great scholar of Babel, and I remember hearing his stories from a very young age. I also remember passing by his house in the streets of Odessa, with a modest plaque marking its historic lineage.

But Babel became personal for me when in 1996 my parents brought me a gift from their trip to our home town, a book of Babel's short stories with an inscription from my cousin's family. This inscription pointed me specifically to page 19 of the tome. As my eyes focused on the small print, and as I started to skim the text, I shortly came upon the following sentence: "None other than Dr. Zilberberg operated on him..." Yes, my great-grandfather was a fairly well respected surgeon in Odessa at the turn of the 20th century, and Babel in fact mentions him in several other of his works, including this autobiographical sketch. Unfortunately, most of what I know about that Dr. Zilberberg has been passed down through the idealized prism of the family lore. Yet I am thoroughly intrigued by the claims that his professional ethics forbade him from turning anyone away, and he cared for the rich Jewish ladies alongside the mafiosi of Moldavanka with the same level of professionalism. And for this, the legendary gang leader Mishka Japonchik, who incidentally is rumored to be related to me on my mother's side, but that is a story for a different time, is known to have afforded him protection from the rogue elements of his gang. The story is well told by my father in this essay, albeit in Russian.

But I digress. So I immediately got myself out the door to get to the program and sat, along with about 100 other attendees, riveted by Andrei's reading of his grandfather's work. After the intermission he read the story that features my great-grandfather, and of course in the Q&A I came out as the great-granddaughter of the great surgeon. Because of this I was invited to join a group of locals and Andrei for a memorable dinner. It was at this dinner that an insightful comment from the brilliant hostess clarified the mindset for my trip. And it was at this dinner that Andrei started to remind us about his grandfather's journalistic work about the collectivization movement in the nascent Soviet Union. It is at this dinner that I got a very clear picture of how our current food production is eerily totalitarian. Imagine if you will being a small farmer at the turn of the 20th century in Russia. You are by no means well off, and all you have is the land and what it gives you after hours and hours and hours of backbreaking loving tending. Imagine now that in comes a new regime, claiming to be for the workers and peasants, and now considers you a land-owner, a kulak, a member of the bourgeoisie, and takes away your farm (the term used is "raskulachit'" or de-kulakize) and puts it in the hands of the collective. Now the big whole owns your meager part and you are left with nothing.

Now let us think about what has transpired in the US over the last century. Let us look at the meat industry specifically. Its deplorable practices at the turn of the 20th century were chronicled brilliantly by Upton Sinclair, and I will only mention that, according to Eric Schlosser's writing, in 1917, 5 largest meatpacking companies owned 55% of the market. On the heels of Sinclair's book, and prompted by the appalling confirmation by the government investigation of Sinclair's claims, the government embarked on a regulatory voyage culminating in the 1920s with the anti-trust legislation, meant to ensure that no monopoly (or oligopoly) would control any market in our nation, thus precluding companies from getting big enough to control our free markets. Well the result of this was a sharp decline in the market share for each of the giants. Yet this was transient, and, as Schlosser points out on page 162, today the top 4 meatpacking giants control over 80% of the market. What happened? Well, we call it "deregulation." But is it really all that different from collectivization? Not that much -- it is still the few controlling the many, and the many going bankrupt and having no leverage to improve their lot. But, you say, it is not the government, but private interests that are at play, so this makes it all better, right? Well, are you sure about that? Who is pilling the strings of our congressional representatives? And these strings are only going to get stronger, thanks to Citizen's United decision from the Supremes. So, while the politicians are foaming at the mouth calling Obama a socialist and a communist, we are careening head first away from democracy and into that social structure we contemptuously call totalitarianism.

Simplistic? I don't think so. Think about it: a pig with lipstick on is still a pig.

When do diagnostic tests improve mortality?

2011-05-10T10:12:00.000-04:00

I thought this post, originally published last May, was worth revisiting apropos this paper that came out in this week's Archives of Internal Medicine. Below I discuss some of the data in the paper, as they were presented in an abstract at a meeting last year, as the context for understanding various mortality statistics.

The question the title of this post poses is well worth asking, particularly as we argue about the merits of mammography screening. The USPSTF has really stirred up the hornet's nest with this one, and the politicians cannot help but get on their populist pulpit, ignoring the facts completely. Oh well, what else is new?

But the question remains: do screening or diagnostic tests that are more sensitive save lives? A great talk on pulmonary embolism detection and outcomes by a recent graduate from the Dartmouth group at the American Thoracic Society last week prompted me to clarify this. We all hear that mortality from many diseases has decreased over the last few decades. But is this true? In order to answer this question, one has to ask what is meant by mortality. Even people well versed in epidemiology and biostatistics occasionally blur the lines between mortality and case fatality, and to our question the distinction is critical. Case fatality is defined as the proportion of patients with the disease that dies, while mortality is a population-based measure, a proportion of all of the population at risk for the disease that dies. The difference lies in our old friend the denominator, which will always keep us honest.

Let's go through a simple example to illustrate this concept. Let's pretend that the total number of cases of disease D diagnosed using stone-age test T 30 years ago was 100 in a population of 10,000 people. Of these cases, 90 died, giving us the case fatality of 90% and mortality of 9 per 1,000 population. Now, we have a new test for D, a super-Doppler-MRI-PET-cyberscan called über-T, a much more sensitive test than the old "gold standard" test T. And now we detect 1,000 cases of D in the population of 10,000 people. Of the 1,000 cases detected by über-T, 90 have died. The case fatality now has decreased dramatically from 90% to 9%, and we can pat ourselves on the back for a job well done, right? Not so fast, the population mortality from disease D has remained a steady 9 per 1,000 population!

So, what does this mean? Does it mean that über-T, which costs 2 orders of magnitude more than its predecessor, is worthless? Well, decide for yourselves. What it means to me is that the additional cases detected by über-T, though finding earlier stage disease, thus increasing the denominator for the case-fatality calculation, has had no impact on the numerator and therefore has not in fact improved the only mortality that matters: population mortality related to the disease.

So, next time a politician tells you how well we are doing with technological innovation in disease management, ask this simple question: Has all the money and innovation really altered the important outcomes, or is this all smoke in mirrors, a mirage created by our irrational belief that technology is our salvation? This may be an uncomfortable epiphany for some. But think about the 900 excess cases of the pseudo-disease diagnosed in our example above -- how many people could have been saved becoming a chronically ill person, how many complications of follow-up procedures could have been avoided, and yes, how much money could have been spent on something other than healthcare? And asking these questions may help us to identify technological advances that actually improve our lives, as opposed to those that merely create attractive business opportunities and stimulate the economy.

How many diseases does it take?

2011-05-06T10:35:00.000-04:00

It is not a secret that I dislike tobacco companies. Intensely. I do not see the point of allowing them to sell a product whose value is all in the negative. I am appalled that we are looking for expensive ways to diminish lung cancer mortality before considering a complete ban on this disease promotion apparatus. Yet this story in the LA Times got my goat. Briefly, a woman who has smoked for years and has had smoking-related obstructive lung disease since 1989 decided to sue tobacco companies after developing lung cancer in 2003. The suit has been making the rounds in various levels of courts, since the defendants asserted that she had exceeded the 2-year statute of limitations following the onset of her smoking-caused disease, referring to the 1989 COPD diagnosis. However, the California State Supreme Court has ruled that she can still sue the manufacturers, since she filed her suit within two years of the lung cancer diagnosis. So, why am I bothered?

Well, here is the thing: once you develop lung disease, followed by periodontal disease, as this woman did, had she really remained unaware that cigarettes are bad? That they cause problems? Is it really possible to live in our world and NOT be aware that tobacco kills? And if she was aware and continued to smoke, whose responsibility is it that she developed lung cancer, hers or the manufacturer's? Well, you say, but the tobacco companies are unethical and lied about making cigarettes more addictive by adding undisclosed ingredients. So, how are we, the consumers, to know? Well, this is pretty simple: We have free will, don't we? And if you have the free will, you have to exercise some will power, no? Is this not what the human condition is all about? Consider what would happen if we just let all of our desires run rampant. At the simplest level, who would want to get up early and do back-breaking work to produce food for our communities? And why contain anger at town hall meetings, when my humanity tells me to get into a brawl? These are basic ways in which we conquer our instincts and do what we need to do to live in a society with human beings and other organisms. But what is peculiar is that we have not extended these exercises of will to the area of consumerism. In other words, it seems to me that whichever way the market, and more importantly marketing, goes, so goes the perceived need for personal will and responsibility. Ergo, smoking despite warnings of its dire effects is OK, since the poor soul is addicted, and she can always sue on the back end, while the murderous tobacco CEOs and investors walk away with the profits. I don't know, I think it is embarrassing to give up your will that way personally.

There are two nuances to this view that I want to express. First, I do believe that cigarette companies are unethical, cruel and in debt to us, but the debt that needs to be paid is to the society, not to individuals. It is a debt to our public health that requires complete withdrawal of their product from the market and a large monetary compensation to promote healthy habits among human beings. Second, I believe that there are shades of this personal vs. societal responsibility balance that are important. Take, for example, food options for an inner city youth who lives in poverty. He may want to exercise his free will to get better nutrition than a $1.25 meal at McDonald's offers, or spend his $1.25 on an apple instead of a bag of potato chips, but for this he has to go across town, a trip that he does not have the means to undertake. This, folks, is where this young man's personal responsibility needs to be supplemented with societal commitment to equity.

So, should this unfortunate smoker with severe and life-threatening sequelae of tobacco abuse be able to sue the producer of the poison, even if she knowingly took the poison? I guess as a society we have decided that this is OK, but as an individual I am dubious. Yet it really is in the interest of our common health and wealth to punish and eliminate producers of such poisons as a society. Relying on individuals to do this job is just a perpetuation of the idea that we are not responsible for our actions. And furthermore, this becomes but a small pimple on this giant's ass, a nuisance, and not a necrotizing fasciitis that is required to kill it once and for all.

Invalidated Results Watch, Ivan?

2011-04-02T09:35:00.000-04:00

My friend Ivan Oransky runs a highly successful blog called Retraction Watch; if you have not yet discovered it, you should! In it he and his colleague Adam Marcus document (with shocking regularity) retractions of scientific papers. While most of the studies are from the bench setting, some are in the clinical arena. One of the questions they have raised is what should happen with citations of these retracted studies by other researchers? How do we deal with this proliferation of oftentimes fraudulent and occasionally simply mistaken data?

A more subtle but no less difficult conundrum arises when papers cited are recognized to be of poor quality, yet they are used to develop defense for one's theses. The latest case in point comes from the paper I discussed at length yesterday, describing the success of the Keystone VAP prevention initiative. And even though I am very critical of the data, I do not mean to single out these particular researchers. In fact, because I am intimately familiar with the literature in this area, I can judge what is being cited. I have seen similar transgressions from other authors, and I am sure that they are ubiquitous. But let me be specific.

In the Methods section on page 306, the investigators lay out the rationale for their approach (bundles) by stating that the "ventilator care bundle has been an effective strategy to reduce VAP..." As supporting evidence they cite references #16-19. Well, it just so happens that these are the references that yours truly had included in her systematic review of the VAP bundle studies, and the conclusions of that review are largely summarized here. I hope that you will forgive me for citing myself again:

A systematic approach to understanding this research revealed multiple shortcomings. First, since all of the papers reported positive results and none reported negative ones, there is a potential for publication bias. For example, a recent story in a non-peer-reviewed trade publication questioned the effectiveness of bundle implementation in a trauma ICU, where the VAP rate actually increased directionally from 10 cases per 1,000 MV days in the period before to 11.9 cases per 1,000 MV days in the period after implementation of the bundle (24). This was in contradistinction to the medical ICU in the same institution, which achieved a reduction from 7.8 to 2.0 cases per 1,000 MV days with the same intervention (24). Since the results did not appear in a peer-reviewed form, it is difficult to judge the quality or significance of these data; however, the report does highlight the need for further investigation, particularly focusing on groups at heightened risk for VAP, such as trauma and neurological critically ill (25).

Second, each of the four reported studies suffers from a great potential for selection bias, which was likely present in the way VAP was diagnosed. Since all of the studies were naturalistic and none was blinded, and since all of the participants were aware of the overarching purpose of the intervention, the diagnostic accuracy of VAP may have been different before as compared to after the intervention. This concern is heightened by the fact that only one study reports employing the same team approach to VAP identification in the two periods compared (23). In other studies, although all used the CDC-NNIS VAP definition, there was either no reporting of or heterogeneity in the personnel and methods of applying these definitions. Given the likely pressure to show measurable improvement to the management, it is possible that VAP classification suffered from a bias.

Third, although interventional in nature, naturalistic quality improvement studies can suffer from confounding much in the same way that observational epidemiologic studies do. Since none of the studies addressed issues related to case mix, seasonal variations, secular trends in VAP, and since in each of the studies adjunct measures were employed to prevent VAP, there is a strong possibility that some or all of these factors, if examined, would alter the strength of the association between the bundle intervention and VAP development. Additional components that may have played a role in the success of any intervention are the size and academic affiliation of the hospital. In a study of interventions aimed at reducing the risk of CRBSI, Pronovost et al. found that smaller institutions had a greater magnitude of success with the intervention than their larger counterparts (26). Similarly, in a study looking at an educational program to reduce the risk of VAP, investigators found that community hospital staff were less likely to complete the educational module than the staff at an academic institution; in turn, the rate of VAP was correlated with the completion of the educational program (27). Finally, although two of the studies included in this review represent data from over 20 ICUs each (20, 22), the generalizability of the findings in each remains in question. For example, the study by Unahalekhaka and colleagues was performed in the institutions in Thailand, where patient mix and the systems of care for the critically ill may differ dramatically from those in the US and other countries in the developed world (22). On the other hand, while the study by Resar and coworkers represents a cross section of institutions within the US and Canada, no descriptions are given of the particular ICUs with respect to the structure and size of their institutions, patient mix or ICU care model (e.g., open vs. closed; intensivists present vs. intensivists absent, etc.) (20). This aggregate presentation of the results gives one little room to judge what settings may benefit most and least from the described interventions. The third study includes data from only two small ICUs in two community institutions in the US (21), while the remaining study represents a single ICU in a community hospital where ICU patients are not cared for by an intensivist (23). Since it is acknowledged that a dedicated intensivist model leads to improved ICU outcomes (28, 29), the latter study has limited usefulness to institutions that have a more rigorous ICU care model.

OK, you say, maybe the investigators did not buy into my questions about the validity of the "findings." Maybe not, but evidence suggests otherwise. In the Discussion section on page 311 they actually say

While the bundle has been published as an effective strategy for VAP prevention and is advocated by national organizations, there is significant concern about its internal validity.

And guess what they cite? Yup, you guessed it, the paper excerpted above. So, to me it feels like they are trying to have it both ways -- the evidence FOR implementing the bundle is the same evidence AGAINST its internal validity. Much like Bertrand Russell, I am not that great at dealing with paradoxes. Will this contradiction persist in our psyche, or will sense prevail? Perhaps Ivan and Adam need to start a new blog: Invalidated Results Watch. Oh? Did you say that peer review is supposed to be the answer to this? Right.

Another swing at the windmill of VAP

2011-04-01T11:24:00.000-04:00

Sorry, folks, but I have been so swamped with work that I have been unable to produce anything cogent here. I see today as a gift day, as my plans to travel to SHEA were foiled by mother nature's sense of humor. So, here I am trying to catch up on some reading and writing before the next big thing. To be sure, I have not been wasting time, but have completed some rather interesting analyses and ruminations, which, if I am lucky, I will be able to share with you in a few weeks.

Anyhow, I am finally taking a very close look at the much touted Keystone VAP prevention study. I have written quite a bit about VAP prevention here, and my diatribes about the value proposition of "evidence" in this area are well known and tiresome to my reader by now. Yet, I must dissect the most recent installment in this fallacy-laden field, where random chance occurrences and willful reclassifications are deemed causal of dramatic performance improvements.

So, the paper. Here is the link to the abstract, and if you subscribe to the journal, you can read the whole study. But fear not, I will describe it to you in detail.

In its design it was quite similar to the central line-associated blood stream infection prevention study published in the New England Journal in 2006, and similarly the sample frame included Keystone ICUs in Michigan. Now, recall that the reason this demonstration project happened in Michigan is because of their astronomical healthcare-associated infection (HAI) rates. Just to digress briefly, I am sure you have all heard of MRSA; but have you heard of VRSA? VRSA stands for vancomycin-resistant Staphylococcus aureus, MRSA's even more troubling cousin, vancomycin being a drug that MRSA is susceptible to. Now, thankfully, VRSA has not yet emerged as an endemic phenomenon, but of the handful of cases of this virtually untreatable scourge that has been reported, Michigan has had plurality of them. So, you get the picture: Michigan is an outlier (and not in the desirable direction) when it comes to HAIs.

Why is it important to remember Michigan's outlier status? Because of the deceptively simple yet devilishly confounding concept of regression to the mean. The idea is that in an outlier situation, at least some of the effect is due to random luck. Therefore, if the performance of an extreme outlier is measured twice, the second time it will be closer to the population mean just by pure luck alone. But I do not want to get too deeply into this somewhat muddy concept right now -- I will reserve a longer discussion of it for another post. For now I would like to focus on some of the more tangible aspects of the study. As usual, two or three features of the study design reduce substantially the likelihood that the causal inference is correct.

First feature is the training period. Prior to the implementation of the protocol, which by the way consisted of the famous VAP bundle, which we have discussed on this blog ad nauseam, there was intensive educational training of the personnel on a "culture of change", as well as the proper definitions of the interventions and outcomes. It is at this time that the "trained hospital infection prevention personnel" were intimately focused on the definition of VAP that they were using. And even though the protocol states that the surveillance definition of VAP would not change throughout the study period, what are the chances that this intensified education and emphasis did not alter at least some of the classification practices?

Skeptical? Good. Here is another piece of evidence supporting my stance. A study from Michael Klompas from Harvard examined inte-rater variability in the assessment of VAP looking at the same surveillance definition applied in the Keystone (and many other) study. Here is what he wrote:

Three infection control personnel assessing 50 patients for VAP disagreed on 38% of patients and reported an almost 2-fold variation in the total number of patients with VAP. Agreement was similarly limited for component criteria of the CDC VAP definition (radiographic infiltrates, fever, abnormal leukocyte count, purulent sputum, and worsening gas exchange) as well as on final determination of whether VAP was present or absent.

And here is his conclusion:

High interobserver variability in the determination of VAP renders meaningful comparison of VAP rates between institutions and within a single institution with multiple observers questionable. More objective measures of ventilator-associated complication rates are needed to facilitate benchmarking and quality improvement efforts.

Yet, the Keystone team writes this in their Methods section:

Using infection preventionists minimized the potential for diagnosis bias because they are trained to conduct surveillance for VAP and other healthcare-associated infections by using standardized definitions and methods provided by the CDC in its National Healthcare Safety Network (NHSN).

Really? Am I cynical to invoke circular reasoning here? Have I convinced you yet that CAP diagnosis is a moving target? And as such it can be moved by cognitive biases, such as the one introduced by the pre-implementation training of study personnel? No? OK, consider this additional piece from the Keystone study. The investigators state that "teams were instructed to submit at least 3 months of baseline VAP data." What they do not state is whether this was a retrospective collection or a prospective one, and this matters a little. First, retrospective reporting in this case would be a lot more representative of what has been, since these rates of VAP are already recorded for posterity and cannot presumably be altered. On the other hand, if the reporting is prospective, I can still conceive of ways to introduce a bias into this baseline measure. Imagine, if you will, that you are employed by a hospital that is under scrutiny for a particular transgression, and that you know the hospital will look bad if you do not demonstrate improvement following a very popular and "common-sense" intervention. Might you be a tad more liberal with identifying these transgressive episodes in your baseline period that after the intervention has been instituted? This is a subtle, yet all too real conflict of interest, which, as we know so well, can introduce a substantial bias into any study. Still don's believe me? OK, come to my office after school and we will discuss. In the meantime, let's move on.

The next nugget is in the graph in Figure 1, where VAP trends over the pre-specified time periods are plotted (you can find the identical graph in this presentation on slide #20). Look at the mean, rather than the median line. (The reason I want you to look at the mean is that the median is zero, and therefore not credible. Additionally, if we want to assess the overall impact of the intervention, we need to be embracing the outliers, which the median ignores). What is tremendously interesting to me is that there is a precipitous drop in VAP during the period called "intervention", followed by much smaller fluctuations around the new mean across the subsequent time periods. This to me confirms the high probability of reclassification (and Hawthorne effect), rather than an actual improvement in VAP rates, as the cause of the drop.

Another piece of data makes me think that it was not the bundle that "did it." Figure 2 in the paper depicts the rates of compliance with all 5 of the bundle components in the corresponding time periods. Again, here as in the VAP rates graph, the greatest jump in adherence to all 5 strategies is observed in the intervention period. However, there is still a substantial linear increase in this metric between the intervention period and through to 25-27 months period. Yet, looking back at the VAP data, no such robust commensurate reduction is observed. While this is somewhat circumstantial, it makes me that much more wary of trusting this study.

So, does this study add anything to our understanding of what bundles do for VAP prevention? I would say not, and it actually muddies the waters. What would have been helpful to see is whether any of the downstream outcomes, such as antibiotics administration, time on the ventilator and length of stay were impacted. Without impacting these outcomes, our efforts are Quixotic, merely swinging at windmills, mistaking them for a real threat.

Is this double dipping? A new bipartisan House bill on oncology reimbursements

2011-03-04T09:59:00.001-05:00

Here is another gem from the House of Representatives: a bipartisan bill to increase Medicare reimbursements to community oncology practices. While at first glance this seems like a reasonable idea, this detail is puzzling:

The so-called "prompt pay" legislation excludes certain discounts extended to wholesalers when calculating Medicare reimbursements and is strongly supported by oncologists.

Confused? Met too. Here is how I understand it. Many community oncology practices have set up infusion clinics, where they administer intravenous chemotherapy on site to their patients. To stock these infusion centers they deal with drug manufacturers and distributors to purchase the drugs at wholesale prices. The bigger the buy, the bigger the manufacturer discount. To the best of my knowledge these discounts are proprietary information, guarded like state secrets. Yet despite these discounts, the clinics charge Medicare a premium for the drugs themselves as well as for the service of administering. The way this legislation looks to me is that it will completely eliminate any reduction in reimbursement related to these discounts. Double dipping, anyone?

Now, I have many friends who are oncologists, and this is really not a slur against them. But these infusion clinics have always represented a cash cow for these practices. And who would not want to have a steady source of income to maintain a robust practice and have some money left over for a life? Again, this is not an indictment of community oncology practices. If, however, one takes an external perspective, this bill becomes something of an anathema to improving efficiency of healthcare delivery. If the reimbursement rates for administering these already exorbitantly expensive drugs improve further, will it not become even more difficult for an oncologist to tread the fine line of the conflict of interest between treatment only when it is in the patient's best interest and treatment for income optimization? Again, I want to point out that I am not singling out oncologists, as it is a part of the human condition to rationalize our selfish decisions by putting them in an altruistic light. And given the amount of uncertainty about who might respond to these drugs, it is easy to convince oneself that a trial of a therapy may be a reasonable idea, with the reimbursements providing a nudge in that direction.

A couple of quotes from the sponsors of the legislation are also worth reprinting:

"On any legislation today, you have to find a way to pay for it. And like any legislation, that's an issue with this one," Whitefield said.

"But to be truthful, because of the oncologist groups and patient groups and others, we think that there may be some provisions in the healthcare bill that passed last year that we may be able to utilize some of those funds for this. All of it's about healthcare, and if we can convince people that this is more important than the others then we can do it."

On any legislation today? You mean it has not always been like this? I guess we have all gotten used to credit as a life style, and now it is time to pay the piper.

Now, what about this: "Because of the oncologist groups and patients groups and others..."? Are they saying what I think they are saying? That because groups are likely to benefit are saying that this is vital, it is in fact vital? I also have to wonder who those "others" are. Hmmm, I wonder...

And this: "All of it's about healthcare, and if we can convince people that this is more important than others then we can do this." OK, so the statement is so grammatically abominable and non-sensical that I can interpret it any way I like. And it seems to me that they are implying that increasing these already hefty reimbursements is more important than stuff like paying for prenatal care and immunizations to the poor? And other essential services to the Medicare population? Well, if this is not the a poster child for why we need to be articulating the value of healthcare, I do not know what is.

Why easy is not always good

2011-03-03T14:22:00.000-05:00

My mother-in-law is a typesetter. She will not read a book unless it is not only appealing in its content, but also pleasing to the eye. When I was in medical school, she did quite a bit of work for medical textbook publishers. Comparing books typeset by her to what I was grinding through on a daily (and nightly basis) incensed her: unwieldy tables appearing three pages away from the corresponding text, small letters crammed to capacity onto oversized pages, few illustrations -- all baffling, annoying (and easily fixable) transgressions against readability. Yet, like all budding docs of all generations, I plowed through these morasses of knowledge without giving its readability much thought -- this was just what you did to get to your goal.

Yesterday I was listening to a program where the author Amy Chua was interviewed about her (ahem) embattled autobiography Battle Hymn of the Tiger Mother. Ms. Chua, though evenly humored throughout the interview, was on the defensive nearly the entire time, explaining how the intent of her opus has been grossly misunderstood by the public, thanks to attacks by critics on her parenting style. And granted, looking at the book as a parenting manual through the prism of our Western parenting norms is a bit disturbing. Yet putting its events in a culturally appropriate context, as well as looking at the content as a narrative rather than a guide, leads to completely different conclusions.

Why am I bringing up Amy Chua's interview after talking about my conquest of the unreadable? Well, it seems that ease is what we have come to expect from everything. What I mean by this is that not only do we expect easily readable texts, but we also expect people to present themselves in such a way as to make it easy for us to like them. Why else change your appearance through life-threatening eating disorders and grueling surgeries, get coached on how to make friends and influence people, and comment on how unlikable some of our female politicians are? Is this not a triumph of form over substance?

Amy Chua clearly bucks this trend in her book and is paying the price. But what worries me is that we are all paying a price. By creating another false dichotomy of "she is nice" or "he is nasty", we have eschewed a more realistic view of our human foibles. We are all nice sometimes and nasty at others. Yet this dichotomy has proven supremely fruitful to our political discourse, where for 30 years this new reality has been taking root. And it has born fruit, so that now people who do not hold similar opinions to ours are summarily dismissed as "nasty" or idiotic, and we are satisfied to surround ourselves with "nice" like-minded sycophants. How primitive it renders our political and social interactions!

Ms. Chua's immigrant parents' philosophy resonated with my upbringing. Coming from lands of uncertainty and deprivation, as immigrants, our parents subscribed to Maslow's pyramid and taught us that economic security trumped everything else. This is why only certain career choices were acceptable, while others were relegated to the back burner of a hobby. These choices were not about ease, but about doing what we were taught was the right thing. As John Adams said:

I must study Politicks and War that my sons may have liberty to study Mathematicks and Philosophy. My sons ought to study Mathematicks and Philosophy, Geography, natural History, Naval Architecture, navigation, Commerce, and Agriculture, in order to give their Children a right to study Painting, Poetry, Musick, Architecture, Statuary, Tapestry, and Porcelaine.

We all set priorities, and some of them may not be easy. I myself still read books even if they are not all that well presented; my priorities are content and writing style, though, to be sure, I do not frown upon the beauty of the visual form. I even enjoy characters who in, their multidimensionality, are a challenge to like. And I have learned in the rest of my life to enjoy people who do not necessarily hold easy or quick appeal for me, yet in the long run prove to add unimaginable richness to my life. Nietzsche coined the famous quote "What does not break you will make you stronger." In all aspects of our lives, while, based on Nietzsche's statement, adversity is a sufficient but not necessary road to strength, pushing ourselves a little bit out of our stuporous ease may prove to be one timely remedy.

The value of a test

2011-03-03T11:43:00.000-05:00

Reading this vintage paper on C diff from the Archives of Pediatric and Adolescent Medicine, I came upon this irresistible conclusion:

Priceless!

Quality or value? A measure for the 21st century

2011-03-03T10:02:00.000-05:00

Fascinating, how in the same week two giants of evidence-based medicine have given such divergent views on the future of quality improvement. Here (free subscription required), Donald Berwick, the CMS administrator and founder and former head of the Institute for Healthcare Improvement, emphasizes the need for quality as the strategy for success in our healthcare system. But here, one of the fathers of EBM, Muir Gray, states that quality is so 20th century, and we need instead to shine the light on value. So, who is right?

Well, let's define the terms. The Merriam-Webster dictionary defines quality as "the degree of excellence." The same source tells us that value is "a fair return or equivalent in goods, services or money for something exchanged." To me "value" is a holistic measure of cost for quality, painting a fuller picture of the investment vis-a-vis the returns on this investment. What do I mean by that?

Simply put, the idea behind value is to establish what is a reasonable amount to pay for a unit of quality. Let's take my used 1999 VW Passat as an example. If my mechanic tells me that it needs to have some hoses replaced, and it will cost me under $100, and the car will run perfectly, I will consider that to be a good value. However, if my transmission has fallen out in the middle of Brookline Ave. in Boston (really happened to me once, many years ago and with a different car), and it will cost me $5,000 to fix, I may say that the value proposition is just not there, particularly given that the car itself is worth much less than $5,000. Given that my budget is not unlimited, I have to make trade-off decisions about where to put my money, so I may instead spend the money on another used Passat that has good prospects.

But in medicine, we routinely avoid thinking about value. There seems to be an overall impression that if it out there on the market, and especially if it is new, it is good and I am worth all of it. This impression is further enabled by the fact that CMS has no statutory power to make decisions based on value of interventions -- they are legislatively mandated to turn a blind eye to the costs. Does this make sense? How toothless is our comparative effectiveness effort likely to be if it has to ignore half of the story?

Let us now look at my favorite sticky wicket, ventilator-associated pneumonia, or VAP. Now, the IHI bundle aimed at eliminating VAP consists of 5 points of intervention: 1). semi-recumbent positioning, 2). daily screen for readiness to get off mechanical ventilation, 3). daily sedation vacation, 4). prophylaxis against GI bleeding, and 5). prevention of clots. As I have mentioned before elsewhere, adherence of 95% to all these measures is deemed compliance and may be ultimately used as a quality measure by payers to determine levels of reimbursement. And while each of these interventions is basically "motherhood and apple pie", applying them blindly and in toto to 95% of intubated patients may be a strategy for disaster. But what is even clearer is that, in order to implement this and all of the other quality improvement strategies, systems need to be put in place that will safeguard against failing to implement these quality measures. The time and resource expenditures needed to institute and maintain these systems, which have not been described in great enough detail as far as I am concerned, have never been quantified. So, what we are left with is a bunch of interventions that, while looking OK individually in clinical trials (until you really start looking at them critically), are likely providing small, if any, gains in quality at the margins, whose investment-return equation has not even been disclosed, let alone balanced. And because budgets are necessarily limited, as are clinicians' time and cognitive capacities, we need to select a sensible menu of interventions from this practically unlimited feast.

This is the quality conundrum, a clear case of chasing our tails to achieve perfection at the expense of good enough. And while no one in their right mind will argue with the language of improved quality in healthcare, I do think that Muir Gray and his camp are on to something that has been a long time coming. At this time of shrinking budgets, competing priorities and tightening resources, does it not make sense to look at value as a package deal, rather than merely at quality in isolation from its context? Instead of being bombarded by ever-increasing volume of quality measures coming from many directions, would it not be more sensible to prioritize these interventions based on the value that they bring rather than merely on their projected outcomes benefits, so frequently estimated based on data that have very little applicability to the real world? Let's start asking the question: how much quality and at what price? Without paying attention to this critical balance, we will not only bankrupt the system, but also worsen outcomes paradoxically, as we continue to overwhelm clinicians with infinite minutia that may or may not be generating helpful outcomes.

So, in my book, Muir Gray: score; Berwick: keep trying.

Top 5 this week

2011-02-27T10:38:00.000-05:00

#5: Redefining compassion: "A spiritual technology"
#4: CMS never events: Evidence of smoke in mirrors?
#3: The rose-colored glasses of early trial terminatio...
#2: Guidelines: What really constitutes level I eviden...

Drum roll for #1 post of the week: New treatments: What benefits at what costs

Thanks for stopping by and reading!

Guidelines: What really constitutes level I evidence?

2011-02-25T13:47:00.000-05:00

There has been some interesting buzz in the blogosphere about where evidence-based guideline recommendations come from, and I wanted to add a little fuel to that fire today.

As you know, I think a lot about the nature of evidence, about the "science" in clinical science, and about pneumonia, specifically ventilator-associated pneumonia or VAP. Last week I wrote here and here about a specific recommended intervention to prevent VAP consisting of semi-recumbent, as opposed to supine, positioning. This recommendation, one of 21 maneuvers aimed at modifiable risk factors for VAP, had level I evidence behind it. Given my recent deconstruction of this level I evidence, consisting of a single unblinded RCT in a single academic urban center in Spain, and given that we already know that level I data represent a very small proportion of all the evidence behind guideline recommendations, I got curious about this level I stuff. How is level I really defined? Is there a lot of room for subjective judgment? So, I went to the source.

In its HAP/VAP guideline, the ATS and IDSA committee define the levels of evidence in the following way:

Level I (high)

Level II (moderate)

Level III (low)

Evidence comes from well conducted, randomized controlled trials

Evidence comes from well designed, controlled trials without randomization (including cohort,patient series, and case-control studies). Level II studies also include any large case seriesin which systematic analysis of disease patterns and/or microbial etiology was conducted,as well as reports of new therapies that were not collected in a randomized fashion

Evidence comes from case studies and expert opinion. In some instances therapy recommendations come from antibiotic susceptibility data without clinical observations

So, well conducted, randomized controlled trials. But what does "well conducted" mean? Seems to me that one person's well conducted may be another person's garbage. Well, I went to the text of the document for clarification:

The grading system for our evidence-based recommendations waspreviously used for the updated ATS Community-acquired Pneumonia(CAP) statement, and the definitions of high-level (Level I),moderate-level (Level II), and low-level (Level III) evidenceare summarized in Table 1 (8).

OK, then. We have to go to reference #8, or the CAP guideline to get to the bottom of the definition. And here is what that document states:

Therefore, in grading the evidence supporting our recommendations,we used the following scale, similar to the approach used in therecently updated Canadian CAP statement (46): Level I evidencecomes from well-conducted randomized controlled trials; LevelII evidence comes from well-designed, controlled trials withoutrandomization (including cohort, patient series, and case controlstudies); Level III evidence comes from case studies and expertopinion. Level II studies included any large case series in whichsystematic analysis of disease patterns and/or microbial etiologywas conducted, as well as reports of new therapies that were notcollected in a randomized fashion. In some instances therapy recommendationscome from antibiotic susceptibility data, without clinical observations,and these constitute Level IIIrecommendations.

Again, we are faced with the nebulous "well-conducted" descriptor with no further defining guidance on how to discern this quality. I resigned myself to going to the next source citation, #46 above, the Canadian CAP statement:

We applied a hierarchical evaluation of the strength of evidence modified from the Canadian Task Force on the Periodic Health Examination [4]. Well-conducted randomized, controlled trials constitute strong or level I evidence; well-designed controlled trials without randomization (including cohort and case-control studies) constitute level II or fair evidence; and expert opinion, case studies, and before-and-after studies are level III (weak) evidence. Throughout these guidelines, ratings appear as roman numerals in parentheses after each recommendation.

Another "well-conducted" construct, another reference, another wild goose chase. The reference #4 above clarified the definition for me thus:

OK, so, now we have "at least one properly randomized controlled trial." So, having gotten to the origin of this broken telephone game, it looks like proper randomization trumps all other markers for a well-done trial. The price of such neglect is giving up generalizability, confirmation, appropriate analyses, and many other important properties that need to be evaluated before stamping the intervention with a seal of approval.

And this is just one guideline for one syndrome. The bigger point that I wanted to illustrate is that, even though we now know that only 14% of all IDSA guideline recommendations have so-called level I evidence behind them, what is dubious is the value and validity of assigning this highest level of evidence to these recommendations, given the room for subjectivity and misclassification. So, what does all of this mean? Well, for me it means no foreseeable shortage of fodder for blogging. But for our healthcare policy and our public's health? Big doo-doo.

New treatments: What benefits at what costs

2011-02-24T11:05:00.000-05:00

Yesterday brought quite a bit of press coverage to a small biotechnology company in Cambridge called Vertex. All this attention was spurned by their gene therapy trial results in cystic fibrosis. The treatment, aimed at a genetic mutation present in about 4% of all CF sufferers, was able to improve the volume that a patient can force out of his lungs in 1 second by over 10%, from about 65% to 75%. Matthew Herper of Forbes on his blog, while being duly impressed by the results, also cautioned that the annual price tag for this medicine is likely to reach $250,000 per patient. So, what does all of this mean in the context of our ongoing national discussion about the value of therapies? Well, let's break things down a bit.

First, let's talk about CF. This is a genetic disorder that essentially makes mucus very sticky. Among its many effects, in its most familiar manifestation this mucus plugs up the airways making it difficult to breathe and predisposing the person to frequent and serious lung infections. When I was a resident back in the early '90s, I remember a devastating case of a young man in his late teens with CF whom we all knew so well from his frequent admissions for exacerbations. Though he was pretty high on the lung transplant list, he ended up succumbing to a devastating pneumonia in our ICU, leaving behind a devoted sister who had been fortunate enough to benefit from a transplant several years earlier. This was a typical course in those days: a brief life punctuated by frequent exacerbations, hospitalizations, antibiotics, gastrointestinal complications, and early death in the second or at best third decade of life with very little hope of procreation. Over the last 20 years things have changed dramatically in the treatment of CF: fewer exacerbations, much lengthened life expectancy and a good chance of having children. Yet we cannot attribute most of these changes to dramatic new breakthrough therapies. To be sure, while there have been tweaks to how we give antibiotics and how pancreatic enzymes are administered to replace the digestive enzymes that the pancreas in CF is unable to produce, most of the progress can be attributed to the increased attention to detail and the advent of almost ruthless care coordination at specialty centers. As a Fellow in the '90s I participated in a clinic where CF patients were transitioning from care by pediatric Pulmonologists to that by adult doctors. The CF specialist running this clinic did not only know all of his patients and their family members by names, but was available 24/7 to them and to his staff for consultation. This is the kind of dedication and vigilance necessary to improve the outcomes in CF.

Now, let's talk about the lesion addressed in the Vertex trial. The type of chronic lung disease caused by CF is called "obstructive." Simply put, it makes exhaling the air in the lungs difficult to do. On lung testing one manifestation of obstruction is the amount of air one is able to force out of his lungs in the first second of the effort, and this is called the FEV1, or forced expiratory volume in 1 second. Another important measure of the degree of obstruction is the amount of air that this volume expired in 1 second represents as a proportion of all of the air in the lungs that can be expired, known as the FVC or forced vital capacity. We say that if the FEV1/FVC ratio is under 75%, then obstruction is present. The size of FEV1 helps us understand how bad the obstruction is.

With this as a background, the primary outcome in many obstructive lung disease trials is the improvement in the FEV1. In the specific trial discussed, the average starting FEV1 in the intervention group was about 65%, which falls in the mild-to-moderate category of obstruction. What this means in terms of symptoms can vary widely. The 10% absolute improvement seen in the intervention group resulted in the average FEV1 of about 75% after treatment, definitely representing fairly mild obstruction (generally FEV1 over 80% is considered to be in the normal range). And this truly is impressive. However, equally interesting is the information that is not in the press coverage, largely based on press releases and sound bites from company executives, since the peer reviewed study is not available at this point. We are not told, but led to assume that, the control group started out on average with a similar deficit in lung function. We are informed that the treatment patients were 55% less likely than placebo patients to have an exacerbation of their disease, yet we do not know what the absolute numbers are; that is we are not told what proportion in each group had an exacerbation, how frequently or how severely. So, this 55%, in the absence of context, while an attention grabber, is not a substantive number. Herper does tell us that there was a remarkable difference in the weight gain (a desirable outcome in the CF population), on average 6.8 lb in the treatment vs. 0.9 lb in the placebo group. This is truly impressive, though it would be even more so if I knew that the trial was double blind, a piece of information I did not notice in any of the reports. Some of the reports have also alluded to symptomatic improvement in shortness of breath, though nowhere did I see this quantified.

The most important piece of data, however, is conspicuously absent from all the stories. What is the proportion of patients who responded to therapy? Why is this important? Well, we know that far from everyone responds to every treatment that they ostensibly qualify for; this is referred to as the heterogeneous treatment effect, or HTE. It is very likely that the 10% improvement in the FEV1 represents at once an inflated estimate referent to those non- or under-responders and a muted one for those patients with a terrific response. The question of a minimal clinically significant change in the FEV1 has haunted the lung trials community for a long time now. Yet, without setting some threshold for a minimum FEV1 improvement that correlates with a meaningful improvement in symptoms, one cannot quantify how well the drug works and hence articulate its value. This is crucial when trying to justify the ostensibly exorbitant price tag anticipated for this drug. How many patients will we need to treat in order to have one of them respond meaningfully with an improvement not just in a laboratory number, but also in their lives? If this targeted drug produces a desirable response even in 50% of all patients with the specific mutation it targets, then it means that we need to spend $500,000 annually to obtain a meaningful improvement in symptoms in one CF patient. But what if it only works this way in 20%? Then we will need to treat 5 patients with this drug to obtain 1 meaningful response at the price of $1.25 million annually. This becomes a bit more daunting, particularly given that the costs will have to be covered through some kind of public or pooled funds and given that this is one of many therapies in the pipeline likely to come with a similar conundrum.

I am not implying that improving a single life is not worth $1.25 million annually. In fact, it may well be a bargain. My point is that these are the serious discussions we need to have as a society, so that when the time comes to make these choices, the discussion will not be subverted by a few loud voices sensationalizing "death panel" slogans. Manufacturers need to know that they should disclose full data, not just selective tidbits that highlight benefits only, but also those difficult pieces of information that shed light on their costs. On our part, we need to understand the gargantuan effort and resources these companies expend to tame these elusive wild therapies that hold so much more promise in the abstract than they end up embodying.

We tread a fine line here. Information and how we assimilate it are the next frontier for cogent decision making. We need to get educated about this now because this train is leaving the station regardless of how we feel about it.

The rose-colored glasses of early trial termination

2011-02-15T11:10:00.000-05:00

The other day I did a post on semi-recumbent positioning to prevent VAP. The point I wanted to make was that an already existing quality measure for a condition that is well on its way to becoming a CMS "never event" is based on one unreplicated single-center small unblinded randomized controlled trial that was terminated early for efficacy. In my post I cited several issues with the study that question its validity. Today I want to touch upon the issue of early termination, which in and of itself is problematic.

What is early termination? It is just that: stopping the trial before enrolling the pre-planned number of subjects. First, it is important to be explicit in the planning phases about how many subjects will need to be enrolled. This is known as the power calculation and is based on the anticipated effect size and the uncertainty in this effect. Termination can happen for efficacy (the intervention works so splendidly that it becomes unethical not to offer it to everyone), safety (the intervention is so dangerous that it becomes unethical to offer it to anyone) or for other reasons (e.g., the recruitment is taking too long, etc.).

Who makes the decision to terminate early and how is the decision made? Well, under the best of circumstances, there is a Data Safety Monitoring Board, a body that is specifically in place to look at the data at certain points in the recruitment process and look for certain pre-specified differences between groups. This DSMB is fire-walled from both the investigators and the patients. The interim looks at the data should be pre-specified by the protocol also, as the number of these looks actually influences the initial power calculation, since the more you look, the more differences you are likely to find by chance alone.

So, without going into too much detail on these interim looks, understand that they are not to be taken lightly, and their conditions and reporting require full transparency. To their credit, the semi-recumbent position investigators reported their plan for one interim analysis upon reaching 50% enrollment. Neither the Methods section nor the Acknowledgements, however, specify who was the analyst and the decision-maker. Most likely it was the investigators themselves that ended up taking the look and deciding on the subsequent course of action. And this itself is not that methodologically clean.

Now, let's talk about one problem early termination. This gargantuan effort led by the team from McMaster in Canada and published last year in JAMA sheds the needed light on what had been suspected before: early termination leads to inflated effect estimates. The sheer massiveness of the work done is mind boggling -- over 2,500 studies were reviewed! The investigators elegantly paired meta-analyses of truncated RCTs with meta-analyses of matched but nontruncated ones, and compared the magnitude of the inter-group differences between the two categories of RCTs. Here is one interesting tidbit (particularly for my friend @ivanoransky):

Compared with matching nontruncated RCTs, truncated RCTs were more likely to be published in high-impact journals (30% vs 68%, P<.001).

But here is what should really grab the reader:

Of 63 comparisons, the ratio of RRs was equal to or less than 1.0 in 55 (87%); the weighted average ratio of RRs was 0.71 (95% CI, 0.65-0.77; P <.001)(FIGURE2). In 39 of 63 comparisons (62%), the pooled estimates for nontruncated RCTs were not statistically significant. Comparison of the truncated RCTs with all RCTs (including the truncated RCTs) demonstrated a weighted average ratio of RRs of 0.85; in 16 of 63 comparisons (25%), the pooled estimate failed to demonstrate a significant effect. [Emphasis mine]

The authors went on to conclude the following:

In this empirical study including 91 truncated RCTs and 424 matching nontruncated RCTs addressing 63 questions, we found that truncated RCTs provide biased estimates of effects on the outcome that precipitated early stopping. On average, the ratio of RRs in the truncated RCTs and matching nontruncated RCTs was 0.71. This implies that, for instance, if the RR from the nontruncated RCTs was 0.8 (a 20% relative risk reduction), the RR from the truncated RCTs would be on average approximately 0.57 (a 43% relative risk reduction, more than double the estimate of benefit). Nontruncated RCTs with no evidence of benefit—ie, with an RR of 1.0—would on average be associated with a 29% relative risk reduction in truncated RCTs addressing the same question.

So, what does this mean? It means that truncated RCTs do indeed tend to inflate the effect size substantially and to show differences by chance alone where none exists.

This is concerning in general, and specifically for our example of the semi-recumbent positioning study. Let us do some calculations to see just how this effect inflation would play out in the said study. Recall that microbiologically confirmed pneumonia occurred in 2 of 39 (5%) semi-recumbent cases and in 11 of 47 (23%) supine cases. The investigators calculated the adjusted odds ratio of VAP in the supine compared to semi-recumbent to be 6.8 (95% CI 1.7 - 26.7). This, as I mentioned before is an inflated estimate as odds ratios tend to be with frequent events. Furthermore, I obviously cannot do the adjusted calculation, as I would need the primary patient data for this. What we need is the relative reduction in VAP due to the intervention being investigated anyway, which is the reciprocal of what we have. So, I can derive the unadjusted relative risk thusly: (2/39)/(11/47) = 0.22. Now, if the RCT truncation alone reduces this risk by 29%, then if the trial had been allowed to go to completion, this relative risk would have been ~0.3. In this range, the difference does not seem all that impressive. But as all of the threats to validity we discussed in the original post begin to chisel mercilessly away at this risk reduction, the 29% inflation becomes a proportionally bigger deal.

Well, that does it.

Redefining compassion: "A spiritual technology"

2011-02-14T13:09:00.002-05:00

This is a TEDxUN talk by Krista Tippett. It is fantastic!
If you are in a rush, just go to around minute 10:00 or so. But really the whole talk is well worh considering.

Top 5 this week

2011-02-13T08:00:00.001-05:00

#5: Reviewing medical literature, part 5: Inter-group ...
#4: Intervention in ICU reduces hospital mortality, bu...
#3: CMS never events: Evidence of smoke in mirrors?
#2: Medical decision making: More signal less noise, p...

And #1 post of the week is: Evidence and profit: An unhealthy alliance

CMS never events: Evidence of smoke in mirrors?

2011-02-11T12:00:00.000-05:00

Let me tell you a fascinating story. In 1999, I was still fresh out of my Pulmonary and Critical Care Fellowship, struggling for breath in the vortex of private practice, when a cute little paper appeared in the Lancet from a great group of researchers in Spain, describing a study performed in one large academic urban medical center's two ICUs: one respiratory and one medical. Its modest aim was to see if semi-recumbent (partly sitting up) compared to supine (lying flat on the back) positioning could reduce the incidence of that bane of the ICU, ventilator-associated pneumonia (VAP). The study was a well done randomized controlled trial, and the investigators even went so far as to calculate the power (the number needed to enroll in order to detect a pre-determined magnitude of effect [in this case an ambitious 50% reduction in clinically suspected VAP]), and this number was 182 based on the assumption of a 40% VAP prevalence in the control (supine) group. The primary endpoint was the prevalence (percentage of all mechanically ventilated [MV] patients developing) and the secondary the incidence density (number of cases among all MV patients spread over all the cumulative days of MV [patient-days of MV]) of clinically suspected VAP, based on the CDC criteria, while microbiologically confirmed VAP (also rigorously defined) served as the secondary endpoint.

Here is what they found. The study was stopped early due to efficacy (this means that the intervention was so superior to the control in reaching the endpoint that it was deemed unethical after the interim look to continue the study), enrolling only 86 patients, 39 in the intervention and 47 in the control groups. And here are the results for the primary and secondary outcomes:

So, this is great! No matter how you slice it, VAP is reduced substantially; there is a microbiologically confirmed prevalence reduction of nearly 6-fold (this is unadjusted for potential differences between groups; and there were differences!). Well, you know what's coming next. That's right, the "not so fast" warning. Let's examine the numbers in context.

First of all, if we look at the evidence-based guideline on HCAP, HAP and VAP from the ATS and IDSA, the prevalence of VAP is generally between 5 and 15%; in the current study the control group exceeds 20%. Now, for the incidence density, for years now the CDC has been keeping and reporting these numbers in the US, and the rate in patients comparable to the ones in the study should be around 2-4 cases per 1,000 MV days. In this study, no matter how you slice it, clinically or microbiologically, the incidence density is exceedingly high, more in line with some of the ex-US numbers reported in other studies. So, they started high and ended high, albeit with a substantial reduction.

Second of all, there is a wonderful flow chart in the paper that shows the enrollment algorithm. One small detail has always been somewhat obscure to me: the 4 patients in the semi-recumbent group that were excluded from analysis due to reintubation (this means that they were taken off MV, but had to go back on it within a day or two), which was deemed a protocol violation. Now, you might think that 4 patients is a pretty small number to worry about. But look at the total number of patients in the group: 39. If the excluded 4 all had microbiologically confirmed VAP, that would bring our prevalence from 5% to 14% (6 out of 43). This would certainly be a less than 6-fold reduction in VAP.

Thirdly, and this I think is critical, the study was not blinded. In other words, the people who took care of the patients knew the group assignment. So what, you ask. Well remember that VAP is a pretty difficult, elusive and unclear diagnosis. So, let us pretend that I am a doc who is also an investigator on the study, and I am really invested in showing how marvelous semi-recumbent positioning is for VAP prevention. I am likely to have a much lower threshold for suspecting and then diagnosing VAP in the comparator group than in my pet intervention group. And this is not an indictment of anyone's judgment or integrity; it is just how our brains are wired.

Next, there were indeed important differences between groups in their baseline risk factors for VAP. For example, more patients in the control (38%) than in the intervention (26%) group were on MV for a week or longer, the single most important risk factor for developing VAP. Likewise, the baseline severity of illness was higher in the control than the intervention group. To be sure, the authors did statistical analyses to adjust these differences away, and still found an adjusted odds ratio of VAP among the supine group to be 6.8, with the 95% confidence interval between 1.7 and 26.7. This is generally taken to mean that, on average, the risk of VAP increases nearly 7-fold for supine position as opposed to semi-recumbent, and if the trial was repeated 100 times, 95 of those times this estimate would fall between a 1.7 and a 26.7-fold increase. OK, so we can accept this as a possible viable strategy, right?

But wait, there is more. Remember what we said about the odds ratio? When the event happens in more than 10% of the sample, the odds ratio vastly overestimates the risk of this event. 28.4% anyone?

Now, let's put it all together. A single center study from a Spanish academic hospital, among respiratory and medical ICU patients, with a minuscule sample size, yet halted early for efficacy, an exceedingly high baseline rate of VAP, a substantial number of patients excluded for a nebulous reason, unblinded and therefore prone to biased diagnosis, reporting an inflated reduction in VAP development in the intervention group. It would be very easy to write this off as a flawed study (like all studies tend to be in one way or another) in need of confirmatory evidence, if it were not so critical in the current punitive environment of quality improvement. (By the way, to the best of my knowledge, there is no study that replicates these results). The ATS/IDSA guideline includes semi-recumbent positioning as a level I (highest possible level of evidence) recommendation for VAP prevention, and it is one of the elements of the MV bundle, as promoted by the Institute for Healthcare Improvement, which demands 95% compliance with all 5 elements of the bundle in order to get the "compliant" designation. And even this is not the crux of the matter. The diabolical detail here is that CMS is creeping up on making VAP into one of their magical "never" events, and the efforts by hospitals will most assuredly be including this intervention. So, ICU nurses are already expected to fall in step with this deceptively simple yet not-so-easily executable practice.

And this is what is under the hood of just one simple level I recommendation by two reputable professional organizations in their evidence-based guidelines. One shudders to think...

Evidence and profit: An unhealthy alliance

2011-02-09T10:18:00.000-05:00

My JAMA Commentary came out this week, and I am getting e-mail about it. It seems to have resonated with many docs who feel that the research enterprise is broken and its output fails them at the office. But what I want to do is tie a few ideas together, ideas that I have been exploring on this blog and elsewhere, ideas that may hold the key to our devastating healthcare safety problem.

The last four decades can be viewed as a nexus between the growth of evidence-based medicine (EBM) on the one hand, and the unbridled proliferation of the biopharmaceutical industry and its technologies. The result has been rapid development, maximization of profit, and a juggernaut of poorly thought-out and completely uncoordinated research geared initially at regulatory approval and subsequently to market growth. It is not that the clinical research has been of poor quality, no. It is that our research tools are primitive and allow us to see only slivers of reality. And these slivers are prone to many of our cognitive biases to boot. So, the drive to produce evidence and the drive to grow business colluded to bring us to where we are today: inundated with evidence of unclear validity, unbalanced with regard to where the biggest difference to public health can be made. Yet we are constantly poked and prodded by the eager bureaucracy to do better at implementing this evidence, while the system continues to perform in a devastatingly suboptimal fashion, causing more deaths every year than strokes.

A byproduct of this technological and financial race has been the rapid escalation of healthcare spending, with the consequent drive to contain it. The containment measures have, of course, had the "unintended consequence" of increased patient volume for providers and of the incredible shrinking appointment, all just to make a living. The end-result for clinicians and patients is the relentless pressure of time and the straight jacket of "evidence-based" interventions in the name of quality improvement. And in this mad race against the clock and demoralization, very few have had the opportunity to think rationally and holistically about the root causes of our status quo. The reality is that we are now madly spinning our wheels at the margins, getting bogged down in infinitesimal details and losing the forest for the trees (pardon all of the metaphor mixing). Our evidence-based quality improvement efforts, while commendable, are like trying to plug holes in a ship's hull with bandainds: costly and overall making little if any difference.

But if we step back and stop squinting, we can see the big picture: stagnated and outdated research enterprise still rewarding spending over substance, embattled clinicians trying to stay afloat, and a $2.5 trillion healthcare gorilla feeding the economy at the expense of human lives. Will technology fix this mess? Not by itself, no. Will more "evidence" be the answer? No, not if we continue to generate it as usual. Is throwing more money at the HHS the solution? I doubt it. A radical change of course is in order. Take profit out of evidence generation, or at least blunt its influence (this will reduce the clutter of marginal, hair-splitting technologies occupying clinicians' collective consciousness), develop new tools for better patient care rather than for maximizing the bottom line, give clinicians more time to think about their patients' needs rather than about how to maintain enough income to pay for the overhead, these are some of the obvious yet challenging solutions to the current crisis. Challenging because there needs to be political will to implement them. And because we are currently so invested in the path we are on that it is difficult and perhaps impossible to stray without losing face. But what is the alternative?

Medical decision making: More signal less noise, please!

2011-02-08T11:46:00.001-05:00

It's official, I'm a country bumpkin! Driving in Boston last week I was distracted, annoyed, made anxious and confused by the constant traffic, billboards and signs. Even highway markings confused me, particularly one indicating a detour to Storrow Drive East, which never materialized. Despite the fact that I know the geography of Boston like the back of my hand, I nearly went down the wrong streets multiple times, including driving the wrong way on some one-way roads. Yes, I am now the menace I used to save my prize driving language for in my younger days.

But it seems that over the years of my living away, there has been a sharp increase in the information thrown at me from all directions, accompanied by a decline in places to rest my gaze without suffering the perseveration of conscious processing. And while the value of this information is at best questionable, the sum total of this overstimulation is clearly confusion, wrong road choices and possibly a reduction in the safety of my driving. This whole experience reminded me of Thomas Goetz's distaste for how medical results are reported. If you have not seen him preach about it, you really should. Here is his excellent TED talk on the subject.

It is ironic that during this overwhelming city visit I also had the chance to speak to a doctor about "routine" preoperative testing and its value. Before surgery, it is recommended that a patient get a screening evaluation. Yet the components of this evaluation vary widely, and may include blood work, urinalysis, electrocardiogram, a chest X-ray and the like. Although evidence suggests that most of the points of this evaluation are useless at best, many institutions continue to order a shotgun panel of preoperative testing for everyone. This one-size-fit-all medicine results in reams of useless and distracting information, a high frequency of abnormal findings of questionable significance, a potential for harm, worry and needless healthcare spending. In my particular conversation I asked the anesthesiologist what the pre-test probability for someone with my characteristics was for a useful chest X-ray result, for example, and whether the fancy electronic medical record used by the hospital could help her determine this. While the answer to the former question was "probably exceedingly low", the answer to the latter was a definitive "no." So, given some elementary thinking, it became clear that a patient like me should not in fact be subjected to a chest X-ray, since any pathology found on one would likely represent a false positive finding, which would nevertheless require potentially invasive follow-up. And guess what? By focusing on the particular individual in the office, rather than all comers, we could have gone through the entire menu of the possible preoperative tests "routinely" ordered and eliminated most if not all of them. But my bet is that not all patients, not even all e-patients, either know or are able to initiate this type of a critical discussion. And yet what tests to obtain, if any, should always be a thoughtful and individualized decision. To approach testing in any other way is to risk generating noise, distraction and harm.

And this brings me back to Thomas Goetz's idea of redesigning how test results are reported. I love his idea. But to me what needs to happen before making the data patient-friendly, is making the decision-making provider-friendly. So, great idea, Mr. Goetz, but let us move it upstream, to the office, where the decision to get chest X-rays, cholesterols and urinalyses is made, and help the doctor visualize her patient's risk for a disease being present, the characteristics of the test about to be ordered, the probability of a positive test result, and all the downstream probabilities that stem from this testing, so as to put a positive test result in the context of the individual's risk for having the disease. Because getting the results of tests that perhaps should never have been obtained in the first place is following the GIGO principle. It is generating noise, distraction and detours going wrong way down one-way roads. And when applied to medicine, these are definitely unwelcome metaphors.

Intervention in ICU reduces hospital mortality, but by how much?

2011-02-02T09:58:00.007-05:00

Addendum #2, 12:09 PM EST, 2/2/11:
So, here is the whole story. Stephanie Desmon, the author of the JH press release, e-mailed me back and pointed me to Peter Pronovost as the source for the 10% reduction information. I e-mailed Peter, and he got back to me, confirming that

"The 10 percent is the rounded differences in differences in odds ratios"

Moral of the story: The devil is in the details.

And speaking of details, I must admit to an error of my own. If you look at the figure reproduced below, I called out the wrong points. For adjusted data, you need to look at the open circles (for the intervention group) and squares (for the control group). In fact, the adjusted mortality went from about 20% at baseline to 16% in the 13-22 months interval for the Keystone cohort, while for the control group it went from a little over 20% to a little under 18%. This makes the absolute reduction a tad more impressive, though there is still less than a 2% absolute difference between the reduction seen in the intervention vs. the control group, leaving all of my other points still in need of addressing.

Addendum #1, 11:00 AM EST, 2/2/11:
I just found what I think is the origin of the 10% mortality reduction rumor in this press release from Johns Hopkins. I just e-mailed Stephanie Desmon, the author of the release, to see where the 10% came from. Will update again should I hear from either Maggie Fox or Stephanie Desmon.

Remember the Keystone project? A number of years ago when we started to pay close attention to healthcare-associated infections (HAI), and hospitals started to take introspective looks at their records, it turned out the the ICUs in the state of Michigan for one reason or another had very high rates of HAIs. As this information percolated through our collective consciousness, the stars aligned in such a away as to release funding from the AHRQ in Washington, DC, for a group of ICU investigators at the Johns Hopkins University School of Medicine in Baltimore, MD, headed by Peter Pronovost, to design and implement a study employing IHI-style (Boston, MA) bundled interventions to prevent catheter-associated blood stream infections (CABSI) and ventilator-associated pneumonia (VAP) across the consortium of ICUs in MI. Whew! This poly-geographic collaboration resulted in a landmark paper in 2006 in the New England Journal of Medicine, wherein the authors showed that the bundled interventions directed by a checklist aimed at CABSI were indeed associated with a satisfying reduction of CABSI. Since 2006 the ICU community has been eagerly awaiting the results of the VAP intervention from Keystone, but none has come out. When there is a void of information, rumors fill this void, and plenty of rumors have circulated about the alleged failure of the VAP trial.

I do not want to belabor here what I have written before with regard to VAP and its prevention, and what makes the latter so difficult, and how little evidence there really is that the IHI bundle actually does anything. You can find at least some of my thoughts on that here. But why am I bringing up the Keystone project again anyway? Well, it is because Pronovost's group has just published a new paper in BMJ, and this time their aim was even more ambitious: to show the impact of this state-wide QI intervention on hospital mortality and length of stay. This is a really reasonable question, mind you, since, we could argue that, if the intervention reduces HAI, it should also do something to those important downstream events that are driven by the particular HAI, namely mortality and LOS. But here are a couple of issues that I found of great interest.

First, as we have discussed before, whether or not VAP itself causes death in the ICU population (that is patients die from VAP), or whether VAP tends to attack those who are sicker and therefore more likely to die anyway (patients die with VAP) remains unclear in our literature. There is some evidence that late VAP may be associated with an attributable increase in mortality, but not early, and these data need to be confirmed. Why is this important? Because if VAP does not impart an increase in mortality, then trying to decrease mortality by reducing VAP is just swinging at windmills.

So, let's talk about the study and what it showed as reported in the BMJ paper. You will be pleased that I will not here go through the traditional list of potential threats to validity, but take the data at face value (well, almost). The authors took an interesting approach of comparing the performance of all eligible ICUs regardless of whether they actually chose to take part in the project. Of all the admissions examined in the intervention group, 88% came from Keystone participants. This is a really sound way to define the intervention cohort, and it actually biases the data away from showing an effect. So, kudos to the investigators. The comparator cohort came from ICUs in the hospitals surrounding Michigan, those that were not eligible for Keystone participation. One point about these institutions also requires clarification: I did not see in the paper whether the authors actually looked at the control hospitals' QI initiatives. Why is this important? Well, if many of the comparator hospitals had successful QI initiatives, then one could expect to see even less difference between the Keystone intervention and the control group. So, again, good on them that they biased the data against themselves.

This is the line of thinking that brings me to my second point. Reuters' Maggie Fox covered this paper in an article a couple of days ago, an article whose ~~byline~~ lede (thanks for the correction, @ivanoransky) floored me:

(Reuters) - A U.S. program to help make sure hospital staff maintain strict hygiene standards lowered death rates in intensive care units by 10 percent, U.S. researchers reported on Monday.

Mind you, I read the article before delving into the peer-reviewed paper, so my surprise came out of just knowing how supremely difficult it is to reduce ICU mortality by 10% with any intervention. In the ICU we celebrate when we see even a 2% absolute mortality reduction. So, it became obvious to me that something got lost in translation here. And indeed, it did. Here is how I read the data.

There are multiple places to look for the mortality data. One is found in this figure:

Now, look at the top panel and focus on the solid circles -- these depict the adjusted mortality in the Keystone intervention group. What do you see? I see mortality going from about 14% at the baseline to about 13.5% at implementation phase to about 13% at 13-22 months post implementation. I do not see a 10% reduction, but at best about a 1% mortality advantage. What is also of interest is that the adjusted mortality in the control group (solid squares) also went down, albeit not by as much. But almost at every point of measurement it was lower already than in the intervention group.

Then there is this table, where the adjusted odds ratios of death are given for the two groups at various time points:

And this is where things get interesting. If you look at the last line of the table, the adjusted odds ratios indeed look impressive, and, furthermore, the AOR for the intervention group is lower than that for the control group. And this is pleasing to any investigator. But what does it mean? Well it means that the odds of death in the intervention group went down roughly by 24% (give-or-take the 95% confidence interval) and by 16% in the control group,each compared to itself at baseline. This is impressive, no?

Well, yes, it is. But not as impressive as it sounds. A relative reduction of 24% with the baseline mortality of 14% means an absolute reduction in mortality of 14% x 24% = 3.4%. But, you notice that we did not actually observe even this magnitude of mortality reduction in the graph. What gives? There is an excellent explanation for this. It is a little known fact to the the reader (and only slightly more so to the average researcher and peer reviewer) that the odds ratio, while a fairly solid way to express risk when the absolute risk is small (say, under 10%), tends to overestimate the effect when the risk is higher than 10%. I know we have not yet covered the ins and the outs of odds ratios, relative risks and the like in the "reviewing literature" series, but let me explain briefly. The difference between odds and risk is in the denominator. While the denominator for the latter is the entire cohort at risk for the event (here all patients at risk for dying in the hospital), that for the former is that part of the cohort that did not experience the event. See the difference? By definition, the denominator for the odds ratio is smaller than for the relative risk calculation, thus yielding a more impressive, yet inaccurate, reduction in mortality.

Bottom line? Interesting results. Not clear if the actual intervention is what produced the 1% mortality reduction -- could have been secular trends, regression to the mean or Hawthorne effect, to name just a few alternatives. But regardless, preventing death is good. The question is were these improvements in mortality sustained after hospital discharge, or were these patients merely kept alive so that they could die elsewhere? Also, what is the value balance here in terms of resources expended on the intervention versus the results that may not even be due to the particular intervention in question?

All of this is to say that I am really not sure what the data are showing. What I am sure of is that I did not find any evidence of a 10% reduction in mortality reported by Reuters (I did e-mail Maggie Fox and at this time still awaiting a reply; will update if and when I get it). In this time of aggressive efforts to bend the healthcare expenditures curve we need to pay attention to what we invest in and the return on this investment, even if the intervention is all "motherhood and apple pie."

The beautiful uncertainty of science

2011-02-01T17:53:00.000-05:00

I am so tired of this all-or-nothing discussion about science! On the one hand there is a chorus singing praises to science and calling people who are skeptical of certain ideas unscientific idiots. On the other, with equal penchant for eminence-based thinking, are the masses convinced of conspiracies and nefarious motives of science and its perpetrators. And neither will stop and listen to the other side's objections, and neither will stop the name-calling. So, is it any wonder we are not getting any closer to the common ground? And if you are not a believer in the common ground, let me say that we are only getting farther away from the truth, if such a thing exists, by retreating further into our cognitive corners. These corners are comfortable places, with our comrades-in-arms sharing our, shall we say, passionate opinions. Yet this is not the way to get to a better understanding.

Because I spend so much time contemplating our larger understanding of science, the title "Are We Hard-Wired to Doubt Science" proved to be a really inflammatory way to suck me into thinking about everything I am interested in integrating: scientific method, science literacy and communication and brain science. The author, on the heels of doing a story on the opposition to smart meters in California, was led to try to understand why we are so quick to reject science:

But some very intelligent people I interviewed had little use for the existing (if sparse) science. How, in a rational society, does one understand those who reject science, a common touchstone of what is real and verifiable?
The absence of scientific evidence doesn’t dissuade those who believe childhood vaccines are linked to autism, or those who believe their headaches, dizziness and other symptoms are caused by cellphones and smart meters. And the presence of large amounts of scientific evidence doesn’t convince those who reject the idea that human activities are disrupting the climate.

She goes on to think about the different ways of perceiving risk, and how our brains play tricks on us by perpetuating our many cognitive biases. In essence, new data are unable to sway our opinion because of rescue bias, or our drive to preserve what we think we know to be true and to reject what our intuition tells us is false. If we follow this argument to its logical conclusion, it means that we just need to throw our hands up in the air and accept the status quo, whatever it is.

I happen to think that the author missed an opportunity to educate her readers about why we need to come to a better understanding and how to get there. The public (and even some of my fellow scientists) needs to understand what science is and, even more importantly, what it is not.

First, science is not dogma. Karl Popper had a very simple litmus test for scientific thinking: He asked how you would go about disproving a particular idea. If you think that the idea is above being disproved, then you are engaging in dogma and not science. The essence of scientific method is developing an hypothesis from either a systematically observed pattern or from a theoretical model. The hypothesis is necessarily formulated as the null, making the assumption of no association the departure point for proving the contrary. So, to "prove" that the association is present you need to rule out any other potential explanation for what may appear to be an association. For example, if thunder were always followed by rain, it might be easy to engage in the "post hoc ergo propter hoc" fallacy and conclude that thunder caused rain. But before this could become a scientific theory, you would have to show that there was no other explanation that would disprove this association.

So, the second point is that science is driven by postulating and then disproving the null hypotheses. By definition, an hypothesis can only be disproved if we 1). the association exists, and 2). the constellation of phenomena is not explained by something else. And here is the third and critical point, the point that produces equal parts frustration and inspiration to learn more: That "something else" as the explanation of a certain association is by definition informed only by what we know today. It is this very quality of knowledge production, the constancy of the pursuit, that lends the only certain property to science, the property of uncertainty. And our brains have a hard time holding and living with this uncertainty.

The tension between uncertainty and the need to make public policy has taken on a political life of its own. What started out as a modest storm of subversion of science by politics in the tobacco debate, has now escalated into a cyclone of everyday leveraging of the scientific uncertainties for political and economic gains. After all, how can we balance the accounting between the theoretical models predicting climate doom in the future and the robust current-day economic gains produced by the very pollution that feeds these models? How can we even conceive that our food production system, yielding more abundant and cheaper food than ever before, is driving the epidemic of obesity and the catastrophe of antimicrobial resistance? And because we are talking about science, and because, as that populist philosopher Yogi Berra famously quipped, "Predictions are hard, especially about the future," the uncertainty of our estimates overshadows the probability of their correctness. Yet by the time the future becomes present, we will be faced with potentially insurmountable challenges of a new world.

I have heard some scientists express reluctance about "coming clean" to the public about just how uncertain our knowledge is. Nonsense! What we need under the circumstances is greater transparency, public literacy and engagement. Science is not something that happens in the bastions of higher education or behind the thick walls of corporations. Science is all around and within us. And if you believe in God, you have to believe that God is a scientist, a tinkerer, always looking for a more elegant solution. The language of science that may seem daunting and obfuscatory. Yet do not be afraid -- patterns of a language are easy to decipher with some willingness and a dictionary. Our brains are attuned to the most beautiful explanations of the universe. Science is what provides them.

Self-determination is predicated upon knowledge and understanding. Abdicating our ability to understand the scientific method leaves us subject to political demagoguery. Don't be a puppet. We are all born scientists. Embrace your curiosity, tune out the noise of those at the margins who are not willing to engage in a sensible dialogue, leave them to their schoolyard brawling. And likewise, leave the politicians, corporate interests, and, alas, many a journalist, and start learning the basics of scientific philosophy and thought. Allow the uncertainty of knowledge excite and delight you. You will not be disappointed.

Reviewing medical literature, part 5: Inter-group differences and hypothesis testing

2011-01-31T11:24:00.000-05:00

Happy almost February to everyone. It is time to resume our series. First I am grateful for the vigorous response to my survey about interest in a webinar to cover some of this stuff. Over the next few months one of my projects will be to develop and execute one. I will keep you posted. In the meantime, if anyone has thoughts or suggestions on the logistics, etc., please, reach out to me.

OK, let's talk about group comparisons and hypothesis testing. Scientific method that we generally practice demands that we articulate an hypothesis prior to conducting a study which will test this hypothesis. The hypothesis is generally advanced as the so-called "null hypothesis" (or H0), wherein we express our skepticism that there is a difference between groups or an association between the exposure and outcome. By starting out with this negative formulation, we set the stage for "disproving" the null hypothesis, or demonstrating that the data support the "alternative hypothesis" (HA, or the presence of the said association or difference). This is where all the measures of association that we have discussed previously come in, and most particularly the p value. The definition of the p value once again is "the probability that the found inter-group difference, or one that is greater than what was found, would have been found under the condition of no true difference." Following through on this reasoning, we can appreciate that the H0 can never be "proven." That is, the only thing that can be said statistically when no difference is found between groups is that we did not disprove the null hypothesis. This may be because there truly is no difference between the groups being compared (that is the null hypothesis approximates reality) or because we did not find the difference that in fact exists. The latter is referred to as the Type I error, and can be present for various reasons, the most common of which is a sample size that is too small to detect statistically significant difference.

This is a good place to digress and talk a little about the distinction between "absence of evidence" and "evidence of absence." The distinction, though ostensibly semantic, is quite important. While "evidence of absence" implies that studies to look for associations have been done, done well, published, and have consistently shown the lack of association between a given exposure and outcome or a difference between two groups, "absence of evidence" means that we have just not done a good job looking for this association or difference. Absence of evidence does not absolve the exposure from causing the outcome, yet so often it is confused with the definitive evidence of absence of an effect. Nowhere is this more apparent than in the history of the tobacco debate, which is the poster child for this obfuscation. And we continue to rely on this confusion in other environmental debates, such as chemical exposures and cell phone radiation. One of the most common reasons for finding no association when one exists, or the type I error, is, as I have already mentioned, a sample size that is too small to detect the difference. For this reason, in a published study that fails to show a difference between groups it is critical to assure that the investigators performed the power calculation. This maneuver, usually found in the Methods section of the paper, lets us know that the sample size is adequate to detect a difference if one exists, thus minimizing the probability of type I error. The trouble is that, as we know, there is a phenomenon called "publication bias." This refers to the scientific journals' reluctance to publish negative results. And while it may be appropriate to reject studies prone to type I error due to poor design (although even these studies may be useful in the setting of a meta-analysis, where pooling of data overcomes small sample sizes), true negative results must be made public. But this is a little off topic.

I will ask you to indulge me in one other digression. I am sure that in addition to "statistical significance" (this is simplistically represented by the p value), you have heard of "clinical significance." This is an important distinction, since even a finding that is statistically significant may have no clinical significance whatsoever. Take for example a therapy that cuts the risk of a non-fatal heart attack by 0.05% in a certain population. This means that in a population at a 10% risk for a heart attack in one year, the intervention will bring this risk on average to 9.95%. And though we can argue whether or not this is an important difference, at the population level, this does not seem all that clinically important. So, if I have the vested interest and the resources to run the massive trial that will give me this minute statistical significance, I can do that and then say without blushing that my treatment works. Yet, statistical significance always needs to be examined in the clinical context. This is why it is not enough to read the headlines that tout new treatments. The corollary to this is that the lack of statistical significance does not equate to the lack of clinical significance. Given what I just said above about type I error, if the difference appears significant clinically (e.g., reducing the incidence of fatal heart attacks from 10% to 5%), but does not reach statistical significance, the result should not be discarded as negative, but examined as to the probability of the type I error. This is also where Bayesian thinking must come into play, but I do not want to get into this now, as we have covered these issues in previous posts on this blog.

OK, back to hypothesis testing. There are several rules to be aware of when reading how the investigators tested their hypotheses, as different types of variables require different methods. A categorical variable (one characterized by categories, like gender, race, death, etc.) can be compared using the chi square method if there is an abundance of events or the Fisher's exact test when values are scant. A normally distributed continuous variable (e.g., age is a continuum that is frequently distributed normally) can be tested using the Student's t-test, while one that has a skewed distribution (e.g., hospital length of stay, costs), requires testing with the Mann-Whitney U-test or the Wilcoxon rank-sum test or the Kruskall-Wallis test. Each of these "non-parametric" tests is appropriate in the setting of a skewed distribution. You do not need to know any more than this: the test for the hypothesis depends on the variable's distribution. And recognizing some of the situations and test names may be helpful to you in evaluating the validity of a study.

One final frequent computation you may encounter is survival analysis. This is often depicted as a Kaplan-Meier curve, and does not have to be limited to examining survival. This is a time-to-event analysis, regardless of what the event is. In studies of cancer therapies we frequently talk about median disease-free survival between groups, and this can be depicted by the K-M analysis. To test the difference between times to event, we employ the log-rank test.

Well, this is a fairly complete primer for most common hypothesis testing situations. In the next post we will talk a little more about measures of association and their precision, types I and II errors, as well as measures of risk alteration.

Soliciting contributions: "Healthcare professional as e-patient" series

2011-01-28T11:30:00.000-05:00

I am contemplating a series of posts arising from my own recent experience as an e-patient to help the broader e-patient community navigate the stormy medical waters with a bit more comfort. I am looking for other healthcare professionals who have had their own experiences as an e-patient that may be instructive for non-healthcare professionals as patients to contribute to the series. Namely, I am most interested in helping people establish better communication lines and channels with their healthcare providers. I am not looking for a comprehensive description of every aspect of your encounter, but rather one specific point that may be particularly instructive. If you have more than one to share, that is great too, we can do that as well. No bitching or moaning, just lessons that we can all learn from.

Issues I would like to touch upon range from how to bring out risk-benefit balance to how to feel OK about confronting your physician with dissenting information to how best to communicate (not everyone is good on e-mail, for example), given our individual styles and time constraints.

I think contributions from healthcare professionals may be very valuable, as we can see both sides of the coin, so to speak.

Would love feedback from both, healthcare professionals and e-patients on what would be valuable. If you are interested in contributing, please, either let me know in the comments section or e-mail me at healthcareetcblog@gmail.com. If you have an idea for a post, please, be very specific about what your theme is, as I will make decisions based on how relevant it is to what I am envisioning.

This is jut a thought at this point, but seems like there may be something to it. Looking forward to your ideas.

The price of marginal thinking in healthcare policy

2011-01-27T17:49:00.000-05:00

I find it fascinating how our brains have this propensity to latch on to what is at the margins at the expense of seeing the bulk of what sits in the center. This peripheral only vision is in part responsible for our obscene healthcare expenditures and underwhelming results.

I have blogged ad nauseam about the drivers of early mortality in the US. In one post I reproduced a pie chart from the Rand Corporation, wherein they show explicitly that a mere 10% of all premature deaths in the US can be attributed to being unable to access medical care. The other 90% is split nearly evenly between behavioral, social-environmental and genetic factors, of which 60%, the non-genetic drivers, can be modified. Yet instead of investing the bulk of our resources in this big bucket of behavioral-environmental-social modification, we put 97% of all healthcare dollars towards medical interventions. This investment can at best produce marginal improvements in premature deaths, since the biggest causes of the effect in question are being all but ignored.

A couple of other striking examples of this marginal magical thinking have surfaced in a few recent stories covered with gusto in the press. One of the bigger ones is the obesity epidemic (oh, yes, you bet it was intended), and its causes. This New York Times piece with its magnetic headline "Central Heating May Be Making Us Fat" entertains the possibility that because of the more liberal use of heat in our homes we are no longer engaging our brown fat, which is a furnace for burning calories. And this is all well and good and fascinating, in a rounding out sort of a way. And it is just as interesting to hear that lack of sleep may be contributing to our expanding waistlines. But it is also baffling that we are still expending these enormous amounts of energy (OK, this one was not intended) on finding the silver bullet, when the target is not a supernatural being, but a super-sized expectation. Is it really that mysterious that we are fatter now than we were 20 years ago, when our current portion sizes are 70% bigger and we spend our days worshipping at the temple of the screen, in all its manifestations? While I am all for learning as much as we can, what we need right now is immediate action to abrogate this escalating epidemic, and I think we can all agree that the way to do it is not through lowering house temperatures. Plenty of behavioral research is available to inform our strategies to get people to eat less and move more. Let's start translating it into practice rather than latch on to one marginal magical idea after another.

And finally, I have to touch upon lung cancer, of course. The current fodder for this was provided by the Washington Post with this story about the growing advocacy among lung cancer patients for early detection. You may recall that recently I did several posts on the heels of the large NCI-sponsored study National Lung Screening Trial (NLST) whose purpose was to understand whether early detection of lung cancer in heavy smokers may improve lung cancer survival. I do not wish to go into all of the specifics of this study and my interpretation of the results -- you can find my thoughts on this study in particular and on screening in general here. What I do want to reiterate is that 85% of all lung cancer is caused by a single exposure: smoking. And guess what? The same behavioral strategies that can help people stop overeating can be deployed towards smoking cessation. Yet, instead of spending 85% of all expenditures on smoking cessation efforts, we prefer to allocate it to early detection. My point is that we need both, but the balance has to be informed by pragmatism, not the marginal magical thinking.

And so it goes that the Pareto principle is bleeding into our healthcare policy decisions -- this is the steep price of the marginal magical thinking. What will it take to get the blinders off and face up to the idea that some intervention points are just more impactful than others? Marginal panaceas will improve our lives, but only at the margins. And without being addressed, the big elephants in the room are likely to stampede us.

SoMe in medicine: It's about communication, stupid!

2011-01-27T11:13:00.000-05:00

My generation of doctors was almost proud of its paternalistic overbearing know-it-all archetype, with the my-way-or-the-highway attitude to patient care. Even today there are inter-specialty fights in medicine that demonstrate these entrenched and seemingly fundamental, albeit willfully exaggerated, differences of opinion and clinical approach. It used to be, and still is to an extent, a badge of honor for an internist to disagree with a surgeon, for a pulmonologist to recommend a course of action diametrically opposed to that suggested by the infectious diseases specialist, and for everyone to disparage neurologists (apologies to my neuro friends). The extent of the discussion with patients as modeled by some of my senior colleagues was to say "You have this, and I am giving you this prescription, and see you in 2 months." And even today, I have observed the best of doctors still respond to a cogent "why?" question from a patient with a "because this is how we do it" answer.

My peers' lack of communication skills is the stuff of urban legends. Yet here we are at what seems like a pivotal moment for so many aspects of medicine -- science, healthcare system, communication technologies -- where effectively communicating outside the profession is a make-or-break proposition. Along these lines, in this BBC documentary Sir Paul Nurse, the head of the Royal Society, examines the societal forces that are coalescing to bring "Science Under Attack." The unifying message that comes out of his inquiry is that other less informed parties with political agendas are co-opting the discussion. Yet there is a distinct lack of the antidote of countervailing communication by scientists in terms that are understandable to the lay public. Nurse's battle cry is that scientists need to do a better job communicating their craft themselves, and not just to each other.

In some ways the prevailing elitism of medicine in the 20th century set the stage for the backlash we are experiencing today. The erosion of trust in the profession, commodification and consequent devaluation of medicine, while multifactorial at their root, could no doubt have been mitigated with better communication. Yet, great communicators rarely choose medicine as the path.

And this brings me to the contentious topic of the role of social media in medicine. For many of the early adopters, the question is no longer "should we", but "how best to." But my sense is, that physicians engaging in social media are still a minority. I am not even sure what proportion of MDs are amenable to communication via e-mail with their patients, though these data may be out there. So, for what seems to me as the majority of MDs who are not sold on e-mail, Twitter, Facebook, blogging or Quora, the value must not be that obvious. This makes me wonder if there are certain unifying characteristics of these docs, one being lack of perceived value of communication outside the profession across all media, including in-person contact.

I am friends with many docs on Twitter and in the blogosphere. The vast majority of them have shown themselves to be patient-centric, knowledgeable and collaborative, the kind of people I would not hesitate to send a loved one to. Yet, this is a skewed sample born out of a selection bias. These are the people who are interested and confident in their ability to communicate outside medicine. These are the people to whom medicine is a humanistic pursuit, where communities of patients and doctors strengthen the discussion of how to transform our system and the patient encounter. My guess is, and this is purely unscientific, that many of those who are skeptical of social media are also skeptical of communication itself, or just do not see the value of it in the equation of providing good patient care within the crushing time constraints of today's healthcare.

So my point is this: before social media tools can be expected to diffuse broadly into the medical community, the value of all communication needs to become clear to physicians in general. At this moment of increasing societal skepticism of science and of usefulness and integrity of the medical profession, against the backdrop of healthcare changes and increasingly unfiltered media noise, willingness and skills to communicate clearly may be as useful to today's doctors as a stethoscope. Once communication becomes the backbone of all medicine, tweets and blog posts are sure to start flowing freely from the fingers of physicians everywhere. And that will be good for the patients, the science and the healthcare system.

Webinar survey results

2011-01-26T10:26:00.000-05:00

Last week I posted a survey link to gauge interest in and potential content for a webinar on how to review medical literature critically. I had a great response, and wanted to share the data with you.

The web page got 302 hits, resulting in 82 survey responses. This is a 27% rate of response, which certainly sets the results up to be biased and non-generalizable. But what the heck? I was looking to hear from people with some interest in this, not all-comers. So, here are the questions and the aggregated answers.

Q1: "I am thinking about creating a webinar based on some of the posts I have done on how to review medical literature. Would this be of interest to you?
R1: 82 people responded, of whom 81 (99%) answered "yes".

Q2: Are you a healthcare professional/researcher, an e-patient, or just an innocent bystander?
R2: 82 responses, 60 (72%) healthcare professionals/researchers, 5 (6%) e-patients, 17 (21%) innocent bystanders

Q3: Why do you feel the need to understand how to review medical literature
R3: This was a free text field, and I got 73 responses. Of these, many had to do with gaining a better understanding of the subject in order to help others (patients, clients, trainees) learn how to read and understand medical literature.

Q4: This question was only for those who responded "yes" to being a healthcare professional/researcher: Do you engage in journal peer review as a reviewer?
R2: Of 60 responses, only 7 (12%) were "yes".

Q5: Similar to Q4, this question was for only those who responded "yes" to Q4: Have you had formal training on how to be an effective peer reviewer?
R2: All 7 responded, of whom only 2 (29%) had formal training through a journal or a professional society, The remaining 5 (71%) have gained pertinent knowledge through reading about it. None of the responders got any reviewing courses during their medical training. Although the sample size is small, the responses are revealing and go along with my experience.

Q6: This question was targeted to only those responders who identified themselves as e-patients: How technical do you want the webinar information to get?
R2: All 5 e-patients answered this question, of whom 2 were comfortable with some degree of technicality, while the remaining 3 were comfortable with a greater degree of it.

Q7: This question was for all responders who expressed interest in having a webinar: Would you want one session or multiple sessions?
R2: Of the 80 responders, 21 (26%) felt that 1 session would suffice, 40 (50%) would be amenable to up to 3 sessions, and 11 (14%) would do up to 5 sessions. The remaining 8 (10%) of the responders chose "other", where their replies ranged from "no clue" to "as many as you see fit" to "let's start an ongoing discussion."

Q8: This was for those who would prefer a single session: How long should the session be?
R2: Of the 20 responses, 10 (50%) indicated 1 hour, while the majority of the rest indicated 2 hours.

Q9: If you are a part of an institution, do you think this would be of interest to your institution?
R9: 70 people responded, with 39 (56%) saying "yes" and 31 (44%) saying "no".

Q10: This was for those responding "yes" to Q9: What type of an institution are you a part of?
R10: All 39 people responded, and there was a range of institutions from medical schools to hospitals to government organizations to academic libraries. What was interesting here was that none of the "yes" responses to Q9 came from anyone in Biopharma or a professional organization or a patient advocacy organization. This I found surprising.

Overall, I am very pleased with the response. I am grateful to Janice McCallum (@janicemccallum on Twitter) for spreading the word to a lestserv of medical librarians. It certainly looks like there is enough interest in a webinar, and now I have to figure out how to execute one. If anyone has ideas, please, let me know in comments here or via e-mail.

Thanks again to all who took the time to respond!

Mirror neurons and the need for slow medicine

2011-01-25T10:28:00.000-05:00

How long does it take for a silence to become uncomfortable? 5 seconds? 20 seconds? A minute? Students of education are taught to give a child roughly 20 seconds to answer a question posed to him. How long do teachers actually give? About 5 seconds, if that. Now sit there and count out 20 Mississippis and see what an astonishingly long time it seems. Why, what if a web page takes that long to load on your browser? This becomes a major technological tragedy for most of us. The point is that 20 seconds is a longer time than we appreciate.

Now, let's talk about empathy. Yes, empathy. This seeming non-sequitur has a solid connection. How do we like to experience empathy? Silent attentive listening is a great example of empathic engagement. When we talk with out friends about emotionally charged topics, we do not want them to respond with "yeah, yeah", and move on rapidly to the next topic, do we? So, empathy takes time and engagement. And when 20 seconds of silence seems like a long time, imagine it in a doctor's office, following a hard revelation or an emotional response by the patient. Can you? Are you counting the Mississippis?

Well, it is no wonder that doctors miss opportunities to express empathy to their patients. In a study from Canada, where oncologists were recorded during patient encounters, these doctors seized fewer than 1 in 4 opportunities to respond to their patients with empathy; the other 3 chances they squandered on discussing clinical information. And this is a pity, as is rightfully acknowledged by the investigator quoted in the article. His conjecture for why docs miss these opportunities to be empathic has to do with their apparently erroneous idea that it takes too much time, and his guidance is the following:

Showing empathy does not mean a doctor has to feel what his or her patient is feeling, Buckman says. Rather, it means acknowledging patients’ fears and other emotions.

“It is perfectly OK for the doctor to remain detached, but it is not OK to talk detached,” he says. “Acknowledging what a patient is feeling is not the same as feeling it yourself.”

Well, I have to respectfully disagree. Here is the meaning of the word "empathy" from the trusted Merriam-Webster dictionary:

2
: the action of understanding, being aware of, being sensitive to, and vicariously experiencing the feelings, thoughts, and experience of another of either the past or present without having the feelings, thoughts, and experience fully communicated in an objectively explicitmanner; also : the capacity for this

And in fact, looking to brain science to guide us on how we are wired to accomplish this, we realize that by definition empathy implies non-detachment, and, in fact, involves feeling what the other is feeling. Empathy is mediated by the so-called mirror neurons, residing in the cingulate gyrus of the brain. The great neurobiologist VS Ramachandran thinks that the discovery of these neurons is to the study of human behavior what the discovery of DNA was to biology. It has been said that mirror neurons help "dissolve the 'self vs. other' barrier." It is these neurons that make us feel others' pain, literally and figuratively. So, putting ourselves in the other person's shoes and "feeling what the patient is feeling" is truly the sine qua non of empathy.

So, if the docs' intuition is correct, and empathy does mean non-detachment and time (after all 20 seconds represents 3% of a 10-minute appointment), how does the medical profession go about relishing and leveraging the other 3 opportunities for empathy instead of throwing them away? I agree with the point of the article that medical students should be taught empathic communication. At the same time, we learn by example, and if harried mentors continue to skirt these issues in the office because they are running two hours behind schedule already, the students will get the point loud and clear. The bigger issue is the incredible shrinking appointment, which is not only likely driving up healthcare costs and the frequency and intensity of testing, with its attendant adverse events, but is eroding the opportunity for a meaningful therapeutic relationship. After all, if the doctor herself provides a therapeutic benefit, is this not of utmost importance?

In short, this is another argument for slow medicine, an argument that should not be weakened by the detachment reasoning. My guess is that it is our biologic imperative as humans to exercise our mirror neurons avidly and often, and being forced to blunt their firing may be yet another path to demoralization. And is the medical profession not already demoralized enough?

Top 5 this week

2011-01-23T10:15:00.000-05:00

#5: Do private ICU rooms really reduce HAIs?
#4: Data mining: It's about research efficiency.
#3: To guideline or not to guideline, that is the ques...
#2: Reviewing medical literature, part 1: The study qu...

#1: A webinar survey -- Please, take this brief survey to help me gauge
interest in and content for a possible webinar on how to read and review
medical literature.

Thanks for visiting and reading!

A webinar survey

2011-01-21T13:40:00.000-05:00

Hi, folks,

I am conducting a survey to see how much interest there may be in a webinar on reviewing medical literature. This should take no more than 10 minutes of your time and would be enormously helpful to me to a). gauge interest and b). create appropriate content.

Thank you so much for doing this!
To get to the survey, click on this url: http://qtrial.qualtrics.com/SE/?SID=SV_bKEvpW0cEjYW5da

To guideline or not to guideline, that is the question in... pneumonia?

2011-01-20T10:57:00.001-05:00

Addendum 1/20/11, 1:27 PM
I want to add something to this, since I have been reflecting on the data more. It turns out that about 3/4 of all patients had an organism isolated felt to be causative of their pneumonia. Among these patients, over 80% in each group received empiric treatment that covered the pathogen. This means that 4 out of 5 patients in both groups received appropriate antibiotic coverage. What the authors skimmed over briefly is to talk about de-escalation. De-escalation is the guideline recommended strategy which entails reducing the spectrum of treatment after culture results become available to only those antibiotics that cover what has grown out. So, if, say, a patient is being empirically treated for Pseudomonas aeruginosa with double coverage, and the culture grows our MRSA and no Pseudomonas, the two anti-pseudomonal drugs should be stopped immediately. The investigators state that they did apply a de-escalation protocol, and that by day 3 50% and by day 5 75% were essentially de-escalated. The fact that they state this in the Discussion section makes me think that this was inserted in response to a reviewer. It is a pity that they did not include de-escalation in their stratified analysis, as it may be at least somewhat explanatory for the findings.

I always felt that there was something intangible and intuitive about my assessments of the critically ill for whom I cared. I could not always explain why I thought one particular patient was more ill than the next, but there was that little something that I must have noticed out of the corner of my eye, and if I tried too hard to focus on it, it would disappear like a puff of smoke. Yet, docs make these pre-conscious assessments all the time. And though these hints drive treatment choices, they are distinctly difficult to quantify scientifically.

A new paper that was just published in The Lancet Infectious Diseases online is a great illustration of what happens when our analyses fail to account for these intuitions. The phenomenon is referred to as "confounding by indication", and it is the perennial plague of observational clinical research. Just to summarize, the study was an observational study of guideline implementation for the treatment of healthcare-associated pneumonia among ICU patients. The central guideline was that for the choice of empiric antibiotics selection. The initial choice of antibiotics, even before the definitive results of cultures are available, is based on the clinician's best guess at what organism(s) may be causing the pneumonia. Among these severely ill patients, the risk of having a bug that is resistant to many antibiotics is higher than for patients who come from the community with pneumonia, and this propensity drives the recommendation for a broader antibiotic coverage for these cases. It has been shown by us and many others that missing this initial opportunity to cover the bug(s) adequately subjects patients to a doubling or even trebling of the risk of death, regardless of whether the coverage is broadened later to include the culprit organism(s).

Back to the study. The four academic medical center that participated in it enrolled 303 eligible patients, of whom 129 were treated with antibiotic combinations that comported with the guideline recommendations (guideline compliant treatment) and 174 received other combinations that did not fit the guideline recommendations (guideline non-compliant). To their surprise, the investigators discovered that 28-day survival was actually higher in the non-compliant group than in the compliant one. And even after doing a great job of adjusting for many potential factors that made the groups different, this paradoxical disparity persisted, with an overall near-doubling in the hazard of death at 28 days in the compliant as opposed to the noncompliant group. Now, this is a fine how-do-you-do! So, does this mean that the guideline is actually killing people by advocating broader coverage? Well, not so fast.

First, I have to acknowledge that I may be engaging in rescue bias right now. Having said this, taking biological plausibility into account, the findings are very likely explained by confounding by indication. Namely, the docs who choose, say, dual rather than single therapy against gram-negative bacteria may be pre-consciously incorporating some intangible patient data into their choices, data that are not well represented by either laboratory values or disease severity scoring systems. I know this is a bit "soft" and maybe even "touchy-feely", but ask any doc, and s/he will confirm this phenomenon.

On the other hand, to be fair and balanced, I do have to agree that there may be other explanations. These include the possibility that our guideline recommendations, never really prospectively validated, may be wrong. Perhaps there is something about the untoward effects of these broad spectrum regimens that is at play. Maybe it is as simple as the "no free lunch" principle, and that even in the situation of covering appropriately broadly, introducing additional drugs increases not only their benefits, but also the risks associated with them. Finally, I have to acknowledge the possibility that we just have no clue what any of this means because our understanding of how antibiotics work in the setting of these types of pneumonia is flawed.

Now, let's put all of this in the context of our multiple discussions about data and knowledge on this web site. Several factors suggest that my initial explanation is correct. The bulk of the evidence points to the fact that skimpy early coverage increases the risk of death. Also, over a century of understanding and the durability of the germ theory imply that antibiotics are important in treating serious bacterial infections. So, the pre-test probability of the validity of the finding in the paper is pretty low. This is not to say that the study should not inject caution and self-examination into how we treat severe pneumonia; it absolutely should! This is also a place where we definitely need well designed interventional studies to confirm (or debunk) what we think we know to be true. In the meantime, as we often intone on this blog, let us not throw the baby out with the bath water.

Disclosure: I have done a lot of work in this area, so I have a potential intellectual COI with the study. Also, at least some of my research has been funded by the manufacturers of some of the antibiotics included in the guidelines.

Data mining: It's about research efficiency.

2011-01-19T09:47:00.001-05:00

I have taken a little break from my reviewing literature series -- work has superseded all other pursuits for a little while. But I did want to do a brief post today, since this JAMA Commentary really intrigued me.

First thing that interested me was the authors. Now, I know who Benjamin Djulbegovic is -- you have to live under a rock as an outcomes researcher not to have heard of him. But who is Mia Djulbegovic? It is an unusual enough surname to make me think that she is somehow related to Benjamin. So, I queried the mighty Google, and it spat out 1,700 hits like nothing. But only one was useful in helping me identify this person, and that was a link to her paper in BMJ from 2010 on prostate cancer screening. On this paper (her only one listed on Medline so far), she is the first author, and her credentials are listed as "student", more specifically in the Department of Urology at the University of Florida College of Medicine in Gainesville, FL. The penultimate author on the paper is none other than Benjamin Djulbegovic, at the University of South Florida in Tampa, FL. So, I am surmising from this circumstantial evidence that Mia is Benjamin's kid who is either a college or a medical student. Why does this matter? Well, there seem to be so few papers in high impact journals that are authored by people without an advanced degree, let alone in the first position, that I am in awe of this young woman, now with two major journals to her name -- BMJ and JAMA. This is evidence that parental mentorship counts for a lot (assuming that I am correct about their relationship). But regardless, kudos to her!

Secondly, the title of the essay really grabbed me: what is the "principle of question propagation", and what does it have to do with comparative effectiveness research (CER) and data mining? Well, basically, the principle of question propagation is something we talk about here a lot: questions beget questions, and the further you go down any rabbit hole, the more detailed and smaller the questions become. This is the beauty and richness of science as well as what I have referred to as "unidirectional skepticism" of science, meaning that a lot of the time, building on existing concepts, we just continue down the same direction in a particular research pursuit. This is why Max Planck was right when he said

A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it.

So, yes, we build upon previous work, and continue our journey down a single rabbit hole our entire career. Though of course there are countless rabbit holes all being explored at the same time. It is really more of a fractal-like situation than a single linear progression. What is clear, as the authors of the Commentary point out, is that this results in the ever-escalating theoretical complexity of scientific concepts. What does this have to do with anything? This, the authors state, argues for continued use of theory driven hypothesis testing, given that medical knowledge will forever be incomplete. And this brings them to data mining.

Here is where I get a little confused and annoyed. They caution the powers that be from consigning all clinical research to data mining, at the expense of more rigorous studies to pursue hypothesis testing. They argue that mining data that already exist is limiting precisely because it is constrained by the scope of our current knowledge, and that we cannot use these data to generate new associations and new treatment paradigms. They further state that emerging knowledge will require updating these data sets with new data points, and this, according to the authors

...creates a paradox, which is particularly evident when searching for treatment effects insubgroups—one of the purported goals of the IT CER initiative. As new research generates new evidence of the importance for tailoring treatments to a given subpopulation of patients, the existing databases will need to be updated, in turn undermining the original purpose to discover new relationships via existing records.

Come agin? And then they say that "consequently, the data mining approach can never result in credible discoveries that will obviate the need for new data collection". Mmhm, and so? Is this the punch line? Well, OK, they also say that because of all this we will still need to do hypothesis testing research. Is this not self-evident?

I don't know about you, but I have never thought that retrospective data mining would be the only answer to our research needs. Rather, the way to view this type of research is as an opportunistic pursuit of information from massive repositories of existing data. We can look for details that are unavailable in the interventional literature, zoom in on the potentially important bits, and use this information to inform more focused (and therefore pragmatically more realistic) interventional studies.

Don't take me wrong, I am happy that the Djulbegovics published this Commentary. It is really designed more as an appeal to policy makers, who, in their perennial search for one-size-fit-all panaceas, may misinterpret our zeal for data mining as the singular answer to all our questions. No indeed, hypothesis testing will continue. But using these vast repositories of data should make us smarter and more efficient at asking the right questions and designing the appropriate studies to answer them. And then generate further questions. And then answer those. And then... Well, you get the picture.

Top 5 this week

2011-01-16T11:02:00.000-05:00

#5: Reviewing medical literature, part 1: The study qu...
#4: Reviewing medical literature part 3 continued: thr...
#3: Reviewing medical literature, part 3: Threats to v...
#2: Reviewing medical literature, part 2b: Study desig...

And #1 post of the week is: Do private ICU rooms really reduce HAIs?

Reviewing medical literature, part 4: Statistical analyses -- measures of central tendency

2011-01-14T11:50:00.000-05:00

Well, we have come to the part of the series you have all been waiting for: discussion of statistics. What, you are not as excited about it as I am? Statistics are not your favorite part of the study? I am frankly shocked! But seriously, I think this is the part that most people, both lay public and professionals, find off-putting. But fear not, for we will deconstruct it all in simple terms here. Or obfuscate further, one or the other.

So, let's begin with a motto of mine: If you have good results, you do not need fancy statistics. This goes along with the ideas in math and science that truth and computational beauty go hand in hand. So, if you see something very fancy that you have never heard of, be on guard for less than important results. This, of course, is just a rule of thumb, and, as such, will have exceptions.

The general questions I like to ask about statistics are 1). Are the analyses appropriate to the study question(s), and 2). Are the analyses optimal to the study question(s). The first thing to establish is the integrity and completeness of the data. If the authors enrolled 365 subjects but were only able to analyze 200 of them, this is suspicious. So, you should be able to discern how complete the dataset was, and how many analyzable cases there were. A simple litmus test is that if more than 15% of the enrolled cases did not have complete data for analysis or dropped out of the study for other reasons, the study becomes suspect for a selection bias. The greater the proportion of dropouts, the greater the suspicion.

Once you have established that the set is fairly complete, move on to the actual analyses. Here, first thing is first: the authors need to describe their study group(s); hence, descriptive statistics. Usually this includes so-called "baseline characteristics", consisting of demographics (age, gender, race), comorbidities (heart failure, lung disease, etc.), and some measure of the primary condition in question (e.g., pneumonia severity index [PSI] in a study of patients with pneumonia). Other relevant characteristics may be reported as well, and this is dependent on the study question. As you can imagine, categorical variables (once again, these are variables that have categories, like gender or death) are expressed as proportions or percentages, while continuous ones (those that are on a continuum, like age) are represented by their measures of central tendency.

It is important to understand the latter well. There are three major measures of central tendency: mean, median and mode. The mean is the sum of all individual values of a particular variable divided by the number of values. So, mean age among a group of 10 subjects would be calculated by adding all 10 individual ages and then dividing by 10. The median is the value that occurs in the middle of a distribution. So, if there are 25 subjects with ages ranging from 5 to 65, the median value is the one that occurs in subject number 13 when subjects are arranged in ascending or descending order by age. The mode, a measure used least frequently in clinical studies, signifies, somewhat paradoxically, the value in a distribution that occurs most frequently.

So, let's focus on the mean and the median. The mean is a good representation of the central value in a normal distribution. Also referred to as a bell curve (yes, because of its shape), or a Gaussian distribution, in this type of a distribution there are roughly equal numbers of points to the left and to the right of the mean value. It looks like this (from wikimedia.org):

For a distribution like the one above it hardly matters which central value is reported, the mean or the median, as they are the same or very similar to one another. Alas, most descriptors of human physiology are not normally distributed, but are more likely to be skewed. Skewed means that there is a tail at one end of the curve or the other (figure from here):

For example, in my world of health economics, many values for such variables as length of stay and costs spread out to the right of the center, similar to the blue curve in the right panel of the above figure. In this type of a distribution the mean and the median values are not the same, and they tell you different things. While the median gives you an idea of the central tendency of the entire distribution, the mean will tell you the central tendency of the majority of the distribution that is tightly clustered at the end opposite the tail. For a distribution similar to the one in the right panel, the mean will underestimate the central measure.

To round out the discussion of central values, we need to say a few words about scatter around these values. Because they represent a population and not a single individual, measures of central tendency will have some variation around them that is specific to the population. For a mean value, this variation is usually represented by standard deviation (SD), though sometimes you will see a 95% confidence interval as the measure of the scatter. Variation around the median is usually expressed as the range of values falling into the central one-half of all the values in the distribution, discarding the 25% at each end, or the interquartile range (IQR 25, 75) around the median. These values represent the stability and precision of our estimates and are important to look for in studies.

We'll end this discussion here for the moment. In the next post we will tackle inter-group differences and hypothesis testing.

Reviewing medical literature part 3 continued: threats to validity

2011-01-13T10:29:00.000-05:00

As promised, today we talk about confounding and interaction.

A confounder is a factor related to both, the exposure and the outcome. Take for example the relationship between alcohol and head and neck cancer. While we know that heavy alcohol consumption is associated with a heightened risk of head and neck cancer, we also know that people who consume a lot of alcohol are also more likely to be smokers, and smoking in turn raises the risk of H&N CA. So, in this case smoking is a confounder of the relationship between alcohol consumption and the development of H&N CA. It is virtually impossible to get rid of all confounding completely in any study design, save for possibly in a well designed RCT, where randomization presumably assures equal distribution of all characteristics; and even there you need an element of luck. In observational studies our only hope to deal with confounding is through statistical manipulation we call "adjustment", as it is virtually impossible to chase it away any other way. And in the end we still sigh and admit to the possibility of residual confounding. Nevertheless, going through the exercise is still necessary in order to get closer to the true association of the main exposure and the outcome of interest.

There are multiple ways of dealing with the confounding conundrum. The techniques used are matching, stratification, regression modeling, propensity scoring and instrumental variables. By far the most commonly used method is regression modeling. This is a rather complex computation that requires much forethought (in other words, "Professional driver on a closed circuit; don't try this at home"). The frustrating part is that, just because the investigators did the regression, does not mean that they did it right. Yet word limits for journal articles often preclude authors from giving enough detail on what they did. At the very least they should tell you what kind of a regression they ran and how they chose the terms that went into it. Regression modeling relies on all kinds of assumptions about the data, and it is my personal belief, though I have no solid evidence to prove it, that these assumptions are not always met.

And here are the specific commonly encountered types of regressions and when each should be used:
1. Linear regression. This is a computation used for outcomes that are continuous variables (i.e., variables represented by a continuum of numbers, like age, for example). This technique's main assumption is that the exposure and outcome are related to each other in a linear fashion. The resulting beta coefficient is the slope of this relationship if it is graphed.
2. Logistic regression. This is done when the outcome variable is categorical (i.e., one of two or more categories, like gender, for example, or death). The result of a logistic regression is an adjusted odds ratio (OR). It is interpreted as an increase or a decrease in the odds of the outcome occurring due to the presence of the main exposure. Thus, a OR of 0.66 means that there is a 34% reduction in the odds (used interchangeably with risk, though this is not quite accurate) of the outcome due to the presence of the exposure. Conversely, a OR of 1.34 means the opposite, or a 34% increase in the odds of the outcome if the exposure is present.
3. Cox proportional hazards. This is a common type of a model developed for a time to event, also known as "survival analysis" (even if not done for survival per se as the outcome). The resulting value is a hazard ratio (HR). For example, if we are talking about a healthcare-associated infection's impact on the risk of remaining in the hospital longer, a HR of, say, 1.8 means that a HAI increases the risk of being in the hospital by 80% at any time during the hospitalization. To me this tends to be the most problematic technique in terms of assumptions, as it requires that the risk of an even stays constant throughout the time frame of the analysis, and how often does this hold true? For this reason the investigators should be explicit about whether or not they tested for the assumption of proportional hazards and whether this was met.

Let's now touch upon the other techniques that help us to unravel confounding. Matching is just that: it is a process of matching subjects with the primary exposure to those without in a cohort study or subjects with the outcome to those without in a case-control study, based on certain characteristics, such as age, gender, comorbidities, disease severity, etc.; you get the picture. By its nature, matching reduces the amount of analyzable data, and thus reduces the power of the study. So, is is most efficiently applied in a case-control setting, where it actually improves the efficiency of enrollment.

Stratification is the next technique. The word "stratum" means "layer", and stratification refers to describing what happens to the layers of the population of interest with and without the confounding characteristic. In the above example of smoking confounding the alcohol and H&N CA relationship, stratifying the analyses by smoking (comparing the H&N CA rates among drinkers and non-drinkers in the smoking group separately from the non-smoking group) can divorce the impact of the main exposure from that of the confounder on the outcome. This method has some distinct intuitive appeal, though its cognitive effectiveness and efficiency dwindle the more strata we need to examine.

Propensity scoring is gaining popularity as an adjustment method in the medical literature. A propensity score is essentially a number, usually derived from a regression analysis, giving the propensity of each subject for a particular exposure. So, in terms of smoking, we can create a propensity score based on other common characteristics that predict smoking. Interestingly, some of these characteristics will be present also in people who are not smokers, yielding a similar propensity score in the absence of this exposure. Matching smokers to non-smokers based on the propensity score and examining their respective outcomes allows us to understand the independent impact of smoking on, say, the development of coronary artery disease. As in regression modeling, the devil is in the details. Some studies have indicated that most papers that employ propensity scoring as the adjustment method do not do this correctly. So, again, questions need to be asked and details of the technique elicited. There is just no shortcut to statistics.

Finally, a couple of words about instrumental variables. This method comes to us from econometrics. An instrumental variable is one that is related to the exposure but not the outcome. One of the most famous uses of this method was published by a fellow you may have heard of, Mark McClellan, where he looked at the proximity to a cardiac intervention center as the instrumental variable in the outcomes of acute coronary events. Essentially, he argued, the randomness of whether or not you are close to a center randomizes you to the type of treatment you get. Incidentally, in this study he showed that invasive interventions were responsible for a very small fraction of the long-term outcomes of heart attacks. I have not seen this method used that much in the literature I read or review, but am intrigued by its potential.

And now, to finish out this post, let's talk about interaction. "Interaction" is a term mostly used by statisticians to describe what epidemiologists call "effect modification" or "effect heterogeneity". It is just what the name implies: there may be certain secondary exposures that either potentiate or diminish the impact of the main exposure of interest on the outcome. Take the triad of smoking, asbestos and lung cancer. We know that the risk of lung cancer among smokers who are also exposed to asbestos is far higher than among those who have not been exposed to asbestos. Thus, asbestos modifies the effect of smoking on lung cancer. So, to analyze those smokers exposed to asbestos together with those who were not will result in an inaccurate measure of the association of smoking with lung cancer. More importantly, it will fail to recognize this very important potentiator of tobacco's carcinogenic activity. To deal with this, we need to be aware of the potentially interacting exposures, and either stratify our analyses based on the effect modifier or work the interaction term (usually constructed as a product of the two exposures, in out case smoking and asbestos) into the regression modeling. In my experience as a peer reviewer, interactions are rarely explored adequately. In fact, I am not even sure that some investigators understand the importance of recognizing this phenomenon. Yet, the entire idea of heterogeneous treatment effect (HTE) and our pathetic lack of understanding of its impact on our current bleak therapeutic landscape, is the result of this very lack of awareness. The future of medicine truly hinges on understanding interaction. Literally. Seriously. OK, at least in part.

In the next installment(s) of the series we will start tackling study analyses. Thanks for sticking with me.

Reviewing medical literature, part 3: Threats to validity

2011-01-12T13:30:00.000-05:00

You have heard this a thousand times: no study is perfect. But what does this mean? In order to be explicit about why a certain study is not perfect, we need to be able to name the flaws. And let's face it: some studies are so flawed that there is no reason to bother with them, either as a reviewer or as an end-user of the information. But again, we need to identify these nails before we can hammer them into a study's coffin. It is the authors' responsibility to include a Limitations paragraph somewhere in the Discussion section, in which they lay out all of the threats to validity and offer educated guesses as to the importance of these threats and how they may be impacting the findings. I personally will not accept a paper that does not present a coherent Limitations paragraph. However, reviewers are not always, as, shall we say, hard assed about this as I am, and that is when the reader is on her own. Let us be clear: even if the Limitations paragraph is included, the authors do not always do a complete job (and this probably includes me, as I do not always think of all the possible limitations of my work). So, as in everything, caveat emptor! Let us start to become educated consumers.

There are four major threats to validity that fit into two broad categories. They are:
A. Internal validity
  1. Bias
  2. Confounding/interaction
  3. Mismeasurement or misclassification
B. External validity
  4. Generalizability
Internal validity refers to whether the study is examining what it purports to be examining, while external validity, synonymous with generalizability, gives us an idea about how broadly the results are applicable. Let us define and delve into each threat more deeply.

Bias is defined as "any systematic error in the design, conduct or analysis of a study that results in a mistaken estimate of an exposure's effect on the risk of disease" (the reference for this is Schlesselman JJ, as cited in Gordis L, Epidemiology, 3rd edition, page 238). I think of bias as something that artificially makes the exposure and the outcome either occur together or apart more frequently than they should. For example, the INTERPHONE study has been criticized for its biased design, in that it defined exposure as at least one cellular phone call every week. Now enrolling such light users can really result in such a small exposure as not to be able to detect any increase in adverse events. This is an example of a selection bias, by far the most common form that bias takes. Another example of a frequent bias is encountered in retrospective case-control studies where people are asked to recall distant exposures. Take for example middle-aged women with breast cancer who are asked to recall their diets when they were in college. Now, ask the same of similar women without breast cancer. What you are likely to get is the effect, absent in women without cancer, of seeking an explanation for the cancer that expresses itself in a bias in what women with cancer recall eating in their youth. So, a bias in the design can make the association seem either stronger or weaker than it is in reality.

I want to skip over confounding and interaction at the moment, as these threats deserve a post of their own, which is forthcoming. Suffice it to say here that a confounder is a factor related to both, the exposure and the outcome. An interaction is also referred to as effect modification or effect heterogeneity. This means that there may be population characteristics that alter the response to the exposure of interest. Confounders and effect modifiers are probably the trickiest concepts to grasp. So, stay tuned for a discussion of those.

For now, let us move on to measurement error and misclassification. Measurement error, resulting in misclassification, can happen at any step of the way: it can be in the primary exposure, a confounder, or the outcome of interest. I run into this problem all the time in my research. Since I rely on administrative coding for a lot of the data that I use, I am virtually certain that the codes routinely misclassify some of the exposures and confounders that I deal with. Take Clostridium difficile as an example. There is an ICD-9 code to identify it in administrative databases. However, we know from multiple studies that it is not all that sensitive or all that specific; it is merely good enough, particularly for making observations over time. But even for laboratory values there is a certain potential for measurement error, though we seem to think that lab results are sacred and immune to mistakes. And need I say more about other types of medical testing? Anyhow, the possibility of error and misclassification is ubiquitous. What needs to be determined by the investigator and the reader alike is the probability of that error. If the probability is high, one needs to understand whether it is a systematic error (for example, a coder always more likely than not to include C. diff as a diagnosis) or a random one (a coder is just as likely to include as not to include a C diff diagnosis). And while a systematic error may result in either a stronger or a weaker association between the exposure and the outcome, a random, or non-differential, misclassification will virtually always reduce the strength of this association.

And finally, generalizability is a concept that helps the reader understand what population the results may be applicable to. In other words, will the data be applied strictly to the population represented in the study? If so, is it because there are biological reasons to think that the results would be different in a different population? And if so, is it simply the magnitude of the association that can be expected to be different or is it possible that even the direction could change? In other words, could something found to be beneficial in one population be either less beneficial or even more harmful in another? The last question is the reason that we perseverate on this idea of generalizability. Typically, a regulatory RCT is much less likely to give us adequate generalizability than a well designed cohort study, for example.

Well, these are the threats to validity in a nutshell. In the next post we will explore much more fully the concepts of confounding and interaction and how to deal with them either at the study design or study analysis stage.

Do private ICU rooms really reduce HAIs?

2011-01-11T09:53:00.000-05:00

We have known for quite some time now that the patient's environment in a hospital matters to his/her outcomes. The concept of biophilia was applied by Roger Ulrich back in the 1980s to surgical patients in a series of experiments. Famously, this work showed that looking out your hospital room's window on a bunch trees is associated with better and less eventful post-operative recovery than staring at a brick wall, for example. We have also known for some time that some of the hospital-associated delirium can be mitigated by having the patient dwell in a room with a window and be exposed to the diurnal light changes.

Another, perhaps even more tangible outcome that can be modified by hospital design is the spread of hospital-acquired infections. This week a paper in the Archives of Internal Medicine from the group in Quebec, who brought us detailed reports of the devastating multihospital hypervirulent Clostridium difficile outbreak in the last decade, generally confirms the effectiveness of private ICU rooms in containing the spread of HAIs. There are some interesting details to point out.

For example, the intervention hospital appears to have had a higher proportion of medical patients than the control institution. Why is this important? Well, medical patients generally experience more chronic and therefore longer stays in the ICU. This gives them a greater opportunity for exposure to HAIs than their surgical counterparts. On the other hand, we know that VAP, for example, an infection very likely to be caused by one of the resistant organisms listed in Table 2 of the paper, happens much more frequently in trauma ICUs than medical ICUs.

Second, the unadjusted ICU length of stay shows some interesting results, depicted in the graph below:

So, while at the intervention hospital the raw ICU LOS has remained stable, at the comparator institution it has been slowly creeping up. Of course, the investigators adjusted for all kinds of factors that may influence this outcome, and showed that there may be a (marginal) reduction in the ICU LOS in association with the switch to private rooms. The authors note that the adjusted average ICU LOS fell by 10%, though under similar circumstances in other similar investigations there is a 95% chance that this would fall somewhere between 0% and 19% reduction. So, under the best of circumstances, if we get a 20% reduction in the 5-day ICU LOS, this translates to about 1 day. And given that transfer timing is more likely to be driven by the availability of ward beds than by the patient's clinical readiness, I question whether this is truly a staggering reduction. Additionally, if you read on, you will realize that there is very little reason to believe that this maximal reduction in ICU LOS is unlikely to be achieved by an average institution. In fact, even the 10% seen on average in this investigation may be a bar that is too high in other less well organized ICUs.

It is important to remember a couple of things: 1). In some circumstances there is unlikely to be any reduction in the ICU LOS; 2). Since LOS is not a normally distributed function, the mean value underestimates the true measure of central tendency in this outcome (this is due to the typically long right tail present in this distribution); and 3). This investigation, though not strictly speaking experimental, was done at 2 academic institutions with highly organized infrastructure and what looks like closed model ICUs (a dedicated specialized team of critical care professionals caring for all ICU patients). For this reason, a similar intervention at a less stringently streamlined institution is unlikely to produce the same magnitude of results.

But the mere fact that the rates of exogenous transmission of pathogenic organisms were reduced is itself encouraging. At the same time, by focusing on carriage rates and not just clinical infections, the authors may be overstating the clinical significance of the observed reduction. Additionally, one of the issues that does not appear to have been addressed explicitly has to do with the availability of sinks: In the intervention unit there was a plethora of sinks, missing in the pre- period and also not available in the comparator hospital. Is it possible then that simply putting in more sinks would accomplish the same for a lot less money?

And this brings me to my next issue with the paper -- cost effectiveness. Now, according to the AHA annual survey of US hospitals, the average age of the physical plant is on the order of 10 years. Given the rapid pace of change in medicine, this may well signal a time for capital investments in plant improvements. And surely from the patient's and family's perspective, private rooms are preferable. However, one must ask the pesky question of the return on such an investment in this era of much needed fiscal restraint in medicine. If the same outcomes of reducing the spread of infectious organisms can be achieved with merely adding sinks, this may be a less drastic and more immediately feasible intervention well worth considering.

Reviewing medical literature, part 2b: Study design continued

2011-01-10T08:00:00.002-05:00

To synthesize what we have addressed so far with regard to reading medical literature critically:
1. Always identify the question addressed by the study first. The question will inform the study design.
2. Two broad categories of studies are observational and interventional.
3. Some observational designs, such as cross-sectional and ecological, are adequate only for hypothesis generation and NOT for hypothesis testing.
4. Hypothesis testing does not require an interventional study, but can be done in an appropriately designed observational study.

In the last post, where we addressed at length both cross-sectional and ecologic studies, we introduced the following scheme to help us navigate study designs:

Let's now round out our discussion of the observational studies and move on to the interventional ones.

Case-control studies are done when the outcome of interest is rare. These are typically retrospective studies, taking advantage of already existing data. By virtue of this they are quite cost-effective. Cases are defined by the presence of a particular outcome (e.g., bronchiectasis), and controls have to come from a similar underlying population. The exposures (e.g., chronic lung infection) are identified backwards, if you will. In all honesty, case-control studies are very tricky to design well, analyze well and interpret well. Furthermore, it has been my experience that many authors frequently confuse case-control with cohort designs. I cannot tell you how many times as a peer-reviewer I have had to point out to the authors that they have erroneously pegged their study as a case-control when in reality it was a cohort study. And in the interest of full disclosure, once, many years ago, an editor pointed out a similar error to me in one of my papers. The hallmark of case-control is that the selection criteria are the end of the line, or the presence of a particular outcome, and all other data are collected backwards from this point.

Cohort studies, on the other hand, are characterized by defining exposure(s) and examining outcomes occurring after these exposures. Similar to case-control design, retrospective studies are opportunistic in that they look at already collected data (e.g., administrative records, electronic medical records, microbiology data). So, although retrospective here means that we are using data collected in the past, the direction of the events of interest is forward. This is why they are named cohort studies, to evoke a vision of Caesar's army advancing on their enemy.

Some of the well known examples of prospective cohort studies are The Framingham Study, The Nurses Study, and many others. These are bulky and enormously expensive undertakings, going on over decades, addressing myriad hypotheses. But the returns can be pretty impressive -- just look at how much we have learned about coronary disease, its risk factors and modifiers from the Framingham cohort!

Although these observational designs have been used to study therapeutic interventions and their consequences, the HRT story is a vivid illustration of the potential pitfalls of these designs to answer such questions. Case-control and cohort studies are better left for answering questions about such risks as occupational, behavioral and environmental exposures. Caution is to be exercised when testing hypotheses about the outcomes of treatment -- these hypotheses are best generated in observational studies, but tested in interventional trials.

Which brings us to interventional designs, the most commonly encountered of which is a randomized controlled trial (RCT). I do not want to belabor this, as RCT has garnered its (un)fair share of attention. Suffice it to say that matters of efficacy (does a particular intervention work statistically better than the placebo) are best addressed with an RCT. One of the distinct shortcomings of this design is its narrow focus on very controlled events, frequently accompanied by examining surrogate (e.g., blood pressure control) rather than meaningful clinical (e.g., death from stroke) outcomes. This feature makes the results quite dubious when translated to the real world. In fact, it is well appreciated that we are prone to see much less spectacular results in everyday practice. What happens in the real world is termed "effectiveness", and, though ideally also addressed via an RCT, is, pragmatically speaking, less amenable to this design. You may see mention of pragmatic clinical trials of effectiveness, but again they are pragmatic in name only, being impossibly labor- and resource-intensive.

Just a few words about before-and after studies, as this is the design pervasive in quality literature. You may recall the Keystone project in Michigan, which put checklists and Peter Pronovost on the map. The most publicized portion of the project was aimed at eradication of central line-associated blood stream infections (CLABSI) (you will find a detailed description in this reference, Pronovost et al. N Engl J Med 2006;355:2725-32). The exposure was a comprehensive evidence-based intervention bundle geared ultimately at building a "culture of safety" in the ICU. The authors call this a cohort design, but the deliberate nature of the intervention arguably puts it into an interventional trial category. Regardless of what we call it, the "before" refers to measurement of CLABSI rates prior to the intervention, while the "after", of course, is following it. There are many issues with this type of a design, ranging from confounding to Hawthorne effect, and I hope to address these in later posts. For now, just be aware that this is a design that you will encounter a lot if you read quality and safety literature.

I will not say much about the cross-over design, as it is fairly self-explanatory and is relatively infrequently used. Suffice it to say that subjects can serve as their own controls in that they get to experience both the experimental treatment and the comparator in tandem. This is also fraught with many methodologic issues, which we will be touching upon in future posts.

The broad category of "Other" in the above schema is basically a wastebasket for me to put designs that are not amenable to being categorized as observational or interventional. Cost effectiveness studies frequently fall into this category, as do decision and Markov models.

Let's stop here for now. In the next post we will start to address threats to study validity. I welcome your questions and comments -- they will help me to optimize this series' usefulness.

Top 5 this week

2011-01-09T12:41:00.000-05:00

#5: Guest post: How our brains are wired to advance sc...
#4: National Healthcare Expenditures, 2009 (In picture...
#3: Reviewing medical literature, part 1: The study qu...
#2: Radium, dopamine and innovation: Name your poison

Drum roll...
#1: Gaol fever and intercessory prayer: Redefining the...

Reviewing medical literature, part 2a: Study design

2011-01-07T15:38:00.001-05:00

It is true that the study question should inform the study design. I am sure you are aware of the broadest categorization of study design -- observational vs. interventional. When I read a study, after identifying the research question I go through a simple 4-step exercise:
1. I look for what the authors say their study design is. This should be pretty easily accessible early in the Methods section of the paper, though that is not always the case. If it is available,
2. I mentally judge whether or not it is feasible to derive an answer to the posed question using the current study design. For example, I spend a lot of time thinking about issues of therapeutic effectiveness and cost-effectiveness, and a randomized controlled trial exploring efficacy of a therapy cannot adequately answer the effectiveness questions.
If the design of the study appears appropriate,
3. I structure my reading of the paper in such a way as to verify that the stated design is, in fact, the actual design. If it is, then I move on to evaluate other components of the paper. If it is not what the authors say,
4. I assign my own understanding to the actual design at hand an go through the same mental list as above with the current understanding in mind.

Here is a scheme that I often use to categorize study designs:

As already mentioned, the first broad division is between observational studies and interventional trials. An anecdote from my course this past semester illustrates that this is not always a straight-forward distinction to make. In my class we were looking at this sub-study of the Women's Health Initiative (WHI), that pesky undertaking that sank the post-menopausal hormone replacement enterprise. The data for the study were derived from the 3 randomized controlled trials (RCT) of HRT, diet and calcium and vitamin D, as well as from the observational component of the WHI. So, is it observational or interventional? The answer to this is confusing to the point of pulling the wool over even experienced clinicians' eyes, as became obvious in my class. To answer the question, we need to go back to definitions of "interventional" and "observational". To qualify as an interventional, a study needs to have the intervention be a deliberate part of the study design. A common example of this type of a study is the randomized controlled trial, the sine qua non of drug evaluation and approval process. Here the drug is administered as a part of the study, not as a background of regular treatment. In contradistinction, an observational study is just that: an opportunistic observation of what is happening to a group of people under ordinary circumstances. Here no specific treatment is predetermined by the study design. Given that the above study looked at multivitamin supplementation as the main exposure, despite its utilization of the data from RCTs, the study was observational. So, the moral of this tale is to be vigilant and examine the design carefully and thoroughly.

We often hear that observational designs are well suited to hypothesis generation only. Well, this is both true and false. Some studies actually can test hypotheses, while others are relegated to generation only. For example, cross-sectional and ecological studies are well suited to generating hypotheses to be tested by another design. To take a recent controversy as an example, the debunked link between vaccinations and autism initially gained steam from the observation that as the vaccination rates were rising, so was the incidence of autism. The type of a study that shows two events changing at the group/population level either in the same or in the opposite direction is called "ecologic". Similar types of studies gave rise to the vitamin D and cancer association hypothesis, showing geographic variation in cancer rates based on the availability of sun exposure. But, as demonstrated well by the vaccine-autism debacle, running with the links from ecological studies is dangerous, as they are prone to a so-called "ecological fallacy". It occurs when, despite the finding in groups of a linked change of the two factors under investigation, there is absolutely no connection between them at the individual level. So, don't let anyone tell you that they tested an hypothesis in an ecological study!

Similarly in cross-sectional studies an hypothesis cannot be tested, and, therefore, causation cannot be "proven". This is due to the fundamental property of "a snapshot in time" that defines a cross sectional study. Since all events (with few minor exceptions) happen at the same time, it is not possible to assign causation to the exposure-outcome couplet. These studies can merely help us think of further questions to test.

So, to connect the design back to the question, if a study purports to "explore a link between exposure X and outcome Y", either an ecologic or a cross-sectional design is OK. On the other hand, if you see one of these designs used to "test the hypothesis that exposure X causes outcome Y", run the other way screaming.

We will stop here for now, and in the next post will continue our discussion of study designs. Not sure yet if we can finish it in one more post, or if it will require multiple postings. Start praying to the goddess of conciseness now!

Reviewing medical literature, part 1: The study question

2011-01-07T10:41:00.001-05:00

Let's start at the beginning. Why do we do research and write papers? No, not just to get famous, tenured or funded. The fundamental task of science is to answer questions. The big questions of all time get broken down into infinitesimally small chunks that can be answered with experimental or observational scientific methods. These answers integrated together provide the model for life as we understand it.

Clearly, the question is the most important part of the equation, and this is why in my semester-long graduate epidemiology course on the evaluative sciences we spend fully the first four to five weeks talking about how to develop a valid and answerable question. The cornerstone of this validity is its importance. Hence, the first question that we pose is: Is the study question important?

This is a bit of a loaded question, though. Important to whom? How is "important" defined? This is somewhat subjective, yet needs to be scrutinized nevertheless. In the context of an individual patient, the question may become: Is the study question important to me? So, importance is dependent on perspective. Nevertheless, there are questions upon whose importance we can all agree. For example, the importance of the question of whether our current fast-food life style promotes obesity and diabetes is hard to dispute.

Regardless of how we feel about the importance of the question, we must first identify the said research question. At least some of the time you will be able to find it in the primary paper, buried in the last paragraph of the Introduction section. Most of the questions we ask relate to etiologic relationships ("etiology" is medicalese for "causation"). Now, you have heard many times that an observational study cannot answer a causal question. Yet, why do we bother with the time, energy and money needed to run observational studies? Without getting too much into the weeds, philosophers of science tell us that no single study design can give us unequivocal evidence of causality. We can merely come close to it. What does this mean in practical terms? It means that, although most observational studies are still interested in causality rather than a mere association, we have to be more circumspect in how we interpret the results from such studies than from interventional ones. But I am jumping ahead.

Once we have identified and established the importance of the question, we need to evaluate its quality. A question of high quality is 1). clear, 2). specific, and 3). answerable. The question that I posed above regarding fast food and obesity possesses none of these characteristics. It is too broad and open to interpretation. If I were really posing a question in this vein, I would choose a single well defined exposure (consuming 3 cans of soda per day) influencing a single outcome (10% body weight gain) over a specific period of time (over 30 weeks). While this is a much narrower question that the one I proposed above, it is only by answering bundles of such narrow questions and putting the information together that we can arrive at the big picture.

A general principle that I like to teach to my student is the PICO or PECOT model (I did not come up with it, but am its avid user). In PICO, P=population, I=intervention or exposure, C=comparator, and O=outcome. The PECOT model is an adaptation of the PICO for observations over time, resulting in P=population, E=exposure, C=comparator, O=outcome, T=time. These models can help not only pose the question, but to unravel the often mysterious and far from transparent intent of the investigators.

Once you have identified the question and dealt with its importance, you are ready to move on to the next step: evaluating the study design as it relates to the question at hand. We will discuss this in the next post.