Is VAP prevention woo science?

My students and I are continuing with the VAP theme, this week exploring the development and implementation of evidence-based practice guidelines (EBPG) through the ATS/IDSA HAP/VAP/HCAP guideline. To continue what we started last week, we are specifically talking about VAP prevention. In this EBPG, there are over 20 suggested maneuvers to prevent VAP, most of them based on level I or II evidence. One lively thread in the discussion deals with the logistics of implementing so many interventions. The complexity of codifying and introducing durably these many processes was duly acknowledged. And although I used the word "bundle" once so far, I have not alluded yet to the IHI's effort to simplify the process. As some of you know, I have been somewhat critical of our current CMS Administrator's approach to quality and safety improvements, and I have even engendered the wrath of some colleagues by publishing this evidence-driven criticism in a review paper (yes, quoting myself again):

    Given that the MV bundle simply represents a conglomeration of some of the recommended practices, it is still important to evaluate how it performs as a VAP preventive strategy for several reasons. First, for example, it is possible that one or two of the interventions chosen to be included into the bundle drive most of the VAP preventive benefit. If this is the case, then it may be inefficient to include the remaining elements, as they may divert the necessary implementation resources from the elements that truly matter. One example of seemingly simple, cheap, yet rarely attainable goal is compliance with head of the bed elevation. Some studies have indicated that a variety of reasons preclude this goal from being achieved 85% of the time, and even call its effectiveness into question (³¹). Understanding which elements of the bundle drive improvements in which populations may deprioritize head of the bed elevation as a goal to be achieved across the board. Alternatively, it may help to make a more forceful argument to improve compliance with this recommendation. Second, other evidence-based recommendations included in the EBPG, but not the bundle, may impart a greater magnitude of VAP prevention, thus once again making the current approach inefficient. For example, some educational strategies incorporating more broadly the EBPG recommendations have been demonstrated to effect a substantial reduction in the rates of VAP (^{27, 32}). Third, neither the expenditures associated with building the infrastructure for bundle implementation nor the potential return on such investment has been explicitly quantified. In general, the recent disappointing results of two meta-analyses of studies evaluating the impact of a rapid response team on hospital outcomes should serve as a cautionary note for adoption of any new process, even one with a great deal of face validity, that has not undergone rigorous testing as a whole (^{33, 34}). More importantly, in the absence of such rigorous validation scoring requiring nearly complete compliance with these processes (e.g., 95% compliance advocated in the case of the MV bundle) as a quality measure would be misguided.

The 95% compliance refers to the IHI's stipulation that an institution reach this level of implementation of all the components of the bundle in order to be considered compliant.

Although several studies out there have demonstrated that by applying a group of evidence-based preventive strategies at least some cases of VAP can be prevented, none has addressed the potential hierarchy of or interactions between the components. What is clear from the literature, however, is that a concomitant educational effort is necessary to make the guideline stick.

And how much of the effect is actually due to Hawthorne effect rather than the result of the specific intervention? I could of course argue that Hawthorne effect is not something to avoid if it helps reduce VAP rates, but then I might get accused of advocating "woo". Perish that thought!