When the black box of evidence is empty

A close friend of mine gave birth a few days ago to her second child. My friend is a slight woman whose infant came out at 6 lb 9 oz, even though she was calculated to be at 36 weeks' gestation. The infant and the mom have done great with absolutely no complications.

This is why, when I was talking to the Dad this morning, I was shocked to hear that the baby had to undergo a test I had never heard of: something to do with putting the child in its car seat and monitoring its heart rate and oxygen saturation for one hour. Mind you, my youngest is 9 years old, and I had never hear of this test, so I looked it up.

Turns out, this is a test formally known as a "pre-discharge car seat challenge". I went to the American Academy of Pediatrics web site to look for their take on it, and here it is:

Hospitals should develop a policy to ensure provision of a period of observation in a car safety seat before hospital discharge for each infant born at <37 weeks' gestation to monitor for possible apnea, bradycardia, or oxygen desaturation.

I went on a circuitous journey to trace the origin of this recommendation. Turns out, it is based on a consensus statement from an APP committee from 1996, the reference for which is this paper from 1993 about implementation of the 1990 AAP recommendations. Looking at the 1990 version, however, does not help in the least, as there is no mention at all of the test in question, or anything to do with infants under 37 weeks' gestation. So, this is where the dubious evidence trail ends. Period. No evidence near as I can see. [If someone knows of a more complete statement that does include the information I am interested in, please, point me to it].

Why did I go after this piece of information? Well it seemed to me to arise out of the "more is better" fallacy, and I wanted to understand a) the evidence that led to the recommendation, b) how the test alters practice, and finally, and most importantly, c) does this practice impact the desired outcomes (i.e., prevent something bad).

So, I went to one of the most trusted sources in evidence synthesis, the Cochrane Collaborative, to see if they have examined this issue. Lo' and behold, I found this 2006 review, whose findings were summarized thusly:

There is no evidence that undertaking a pre-discharge "car seat challenge" benefits preterm infants. The "car seat challenge" assesses whether preterm infants who are ready for discharge home are prone to episodes of apnoea (stopping breathing), bradycardia (slow heart rate), or desaturation (low oxygen levels) when seated in their car seat. However, it is not clear whether the level of oxygen desaturation, apnoea, or bradycardia detected in the car seat challenge is actually harmful for preterm infants. Additionally there is concern that the use of the car seat challenge may cause undue parental anxiety about the safety of transporting their infant in a car seat. Despite these uncertainties, and despite the widespread use of the test, we have not identified any randomised controlled trials that assessed whether undertaking a car seat challenge is beneficial or harmful to preterm infants.

So, here is a kid, born at supposedly 36 weeks gestation, but looking and acting like a full-term infant, having to sit in her car seat for 1 hour to undergo monitoring that has absolutely no evidence behind it. Putting aside the inconvenience, the false positives, the parental anxiety, let's think about the costs. I could not find published reimbursement rates for this useless test. However, if you think about the person-power, the equipment used, and the potential prolongation of hospitalization that it may induce, the price tag is likely not trivial.

This case is instructive to me. As I have said before, a pitiful minority of medical practice has good evidence behind it. It is precisely this type of testing, done to comply with what looks to be a self-propagating yet outdated recommendation, that healthcare reformers in Washington need to be addressing. Furthermore, when evidence is kept in a black box, we are likely to find that the box is indeed empty.a,

bradycardia, or oxygen desaturation. An appropriate

hospital staff person should conduct the observation.

Hospitals should develop policies to