VAP: A case of mistaken identity?

This week in my class we are talking about systematic reviews and meta-analyses. As in the past, I assigned this excellent example published a couple of years ago by Canadian friends:

BMJ. 2007 Apr 28;334(7599):889. Epub 2007 Mar 26.

Oral decontamination for prevention of pneumonia in mechanically ventilated adults: systematic review and meta-analysis.

Chan EY, Ruest A, Meade MO, Cook DJ.

Department of Nursing Services, Tan Tock Seng Hospital, Singapore. ee_yuee_chan@ttsh.com.sg

Comment in:

• BMJ. 2007 Apr 28;334(7599):861-2.

Abstract

OBJECTIVE: To evaluate the effect of oral decontamination on the incidence of ventilator associated pneumonia and mortality in mechanically ventilated adults.

DESIGN: Systematic review and meta-analysis.

DATA SOURCES: Medline, Embase, CINAHL, the Cochrane Library, trials registers, reference lists, conference proceedings, and investigators in the specialty.

REVIEW METHODS: Two independent reviewers screened studies for inclusion, assessed trial quality, and extracted data. Eligible trials were randomised controlled trials enrolling mechanically ventilated adults that compared the effects of daily oral application of antibiotics or antiseptics with no prophylaxis.

RESULTS: 11 trials totalling 3242 patients met the inclusion criteria. Among four trials with 1098 patients, oral application of antibiotics did not significantly reduce the incidence of ventilator associated pneumonia (relative risk 0.69, 95% confidence interval 0.41 to 1.18). In seven trials with 2144 patients, however, oral application of antiseptics significantly reduced the incidence of ventilator associated pneumonia (0.56, 0.39 to 0.81). When the results of the 11 trials were pooled, rates of ventilator associated pneumonia were lower among patients receiving either method of oral decontamination (0.61, 0.45 to 0.82). Mortality was not influenced by prophylaxis with either antibiotics (0.94, 0.73 to 1.21) or antiseptics (0.96, 0.69 to 1.33) nor was duration of mechanical ventilation or stay in the intensive care unit.

CONCLUSIONS: Oral decontamination of mechanically ventilated adults using antiseptics is associated with a lower risk of ventilator associated pneumonia. Neither antiseptic nor antibiotic oral decontamination reduced mortality or duration of mechanical ventilation or stay in the intensive care unit.

The meta-analysis looks at effectiveness of oral care in preventing VAP. Of interest was the overall finding that VAP could indeed be prevented, but preventing it altered neither mortality nor such hospital utilization parameters as duration of mechanical ventilation (MV) or ICU length of stay (LOS).

The study has precipitated a vigorous discussion in class. I will excerpt below some of my responses to the students' questions (all right, so I feel a little tacky quoting myself, but perish the thought I should be accused of plagiarizing anyone, even myself).

One of the students brought up CMS never events (you know, those hospital-acquired conditions that CMS will no longer pay for because they should never happen), and presented me with an opportunity to talk about the subtleties of VAP diagnosis within that context:  

I could not agree more that prevention is critical. The question of whether we can prevent VAP 100% of the time is a little more complicated, however. For one, we are not even sure how to diagnose VAP. Applying the CDC's surveillance definition results in rates of VAP that are quite different from invasive diagnostic testing data. Applying the same definitions to different populations results in rates that are vastly different. Furthermore, diagnostics are driven by somewhat arbitrary thresholds for bacterial counts that may not have the greatest sensitivity or specificity. So, when you are dealing firstly with the wild west of the patient and disease interaction, then add the muddy diagnostic issues to the stew, and season everything with variable processes of care, the issue to me, at least, becomes a little less straight forward. 

Then, when I asked them what is the use of preventing VAP when it does not impact such important outcomes as mortality or LOS, I got some great answers, appropriately ranging from "you are full of s**t" to "OK, my intuition tells me VAP is good to prevent, but here we have no reason for it". Some even referred to these outcome as patient-oriented, so this was a great teachable moment. So, I responded, reiterating:

I am personally a great believer in prevention, but a). it has to be sensible prevention and not just a convenient conglomeration of poorly tested modalities, and b). not everything can be prevented. There are a couple of points to make here.

CMS has not curtailed payment for VAP for the reasons that I outlined above -- what exactly constitutes VAP, what are the best preventive strategies, and exactly how good are they. CMS, on the other hand, HAS stopped paying for completely preventable errors (yes, there are such things), such as leaving instruments inside patients during surgery or administering the wrong unit of blood. These are process errors for which zero tolerance is reasonable. 

Now, on to VAP. From the Chan MA we are under the impression that there is no reason to prevent VAP, since there is no difference in patient-oriented outcomes. First, let me challenge your notion that LOS and MV duration are patient-oriented outcomes. I would argue that the patient cares less about this than about comfort, quality of life, post-critical illness functional status and the development of PTSD after the ICU stay, to name a few. These are rarely measured in RCTs. So, even if the LOS and mortality are not altered by VAP prevention, there may be other perfectly valid reasons to prevent it. Not to mention curbing the use of antibiotics to curtail the spread of resistance. 

One final point to make. Each of the studies was not powered for mortality difference. Having said that, combining the data into a very respectable total number of >3,000 patients analyzable for mortality, this should have been enough to at least show an important trend, if there was one. And in fact the VAP literature is fraught with controversy on whether VAP imparts attributable mortality or not. The LOS issue is even more complex, however. Because LOS is an infinitely variable outcome, an RCT powered to capture this difference would have to be enormously large (a measly 2000 patients would not do). However, the epidemiologic and outcomes literature abounds with data on attributable LOS and $$ due to VAP.

So, here is a valuable MA that shows that there are sound strategies to prevent at least some cases of VAP, but by implication does not justify the effort for its prevention. Here is a situation, where policy requires expert analysis. 

Bottom line, MAs are useful and dangerous at the same time. Their results, if not examined carefully and in context can worsen rather than improve care. And the MA that I assigned is actually of great quality!

I posted this to underline how tricky applying evidence can be. Particularly in an area where there is so much diagnostic confusion. This of course does not mean that we should not strive to understand things better. On the contrary, this calls for more integration of knowledge in a multidisciplinary fashion.

Fools rush in where wise people fear to tread...