Wednesday, May 30, 2012

How to game mortality data

There is a great illustration in this BMJ article of what I discuss in Chapter 2 of my book: the type of mortality that matters. In the figure below from the paper, note that, as the new diagnoses of each of the cancers rise (green lines), the attendant "Deaths" (red lines) stay unchanged. If you look at the Y-axis of each graph, it tells us that the unit of measurement is "Rate per 100,000 people." So the red lines represent population mortality.

"Population mortality" means that the denominator for this value is all people in the population who are at risk for the disease in question. This means that for prostate cancer, for example, we include only those men who have a prostate and exclude all women and men who have had a prostatectomy. Population mortality stands in contradistinction to case fatality. The latter is defined as deaths among all the people diagnosed with the disease. So, for prostate cancer, case fatality would be deaths among all men who have been diagnosed with prostate cancer.

It is not difficult to see how case fatality is a somewhat circular, even self-referential, measure of our diagnostic prowess, but says very little about how well we are doing with disease treatment. If we have tests that are capable of picking up the most minute of diseases, those that are not likely to cause death in the first place, then the denominator becomes inflated with this noise, while the numerator, the actual fatalities, does not change. This leads of course to an apparent reduction in deaths from the disease, but a reduction that is an artifact of overdiagnosis. Population mortality, on the other hand, cannot be gamed this way, as you can see in the figure above. This is the only mortality that gives us honest feedback without a bias about how we are doing with our early detection and other interventions. In the case of the cancers in the figure, the answer is "not so well."

You can learn more about this in Chapter 2, where I even have a figure that illustrates this critical distinction.        

If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. Thank you for your support!


  1. Great post, Marya! Glad to have found your blog, can't wait to read the book. Do your cover children's health issues?

  2. Polly, thanks for visiting and for your comment. I very rarely address pediatric issues, as I am much more well versed in adult medicine. Is there something in particular that you had in mind?

  3. Hi Marya,
    You're right in the technical sense, but only to a point.

    From the perspective of someone who's 50 and has say, invasive breast cancer - the general population mortality data are irrelevant. These curves don't provide information on likely survival with or without treatment, or with one treatment instead of another. The same applies to anyone who has any cancer or disease - it could be HIV, or tuberculosis, or renal failure - that shortens life expectancy.

    Wouldn't informed decisions for an individual be based on the survival probability of others with that condition?

  4. Elaine, of course you are correct from the point of view of the patient with the disease. I am speaking from the perspective of population screening, however, and here the denominator of the entire population at risk is crucial. My point is that to make a case for effectiveness of early detection, one needs to show a drop not in case fatality, but in population mortality.

  5. I'm not so sure, Marya.

    Take, for example, a rare, treatable tumor like acute lymphoblastic leukemia (ALL). Detecting the disease and giving curative treatment might have no meaningful impact on the population's survival curve, but definitely saves a high proportion of lives among those who have it.

    Which begs the question - is it worth the costs of research and drugs for a small group of people, among a large population, with potentially curable illnesses?

  6. Elaine, I think we are talking about two slightly different things. First, a population would not get screened for a rare tumor -- it simply would not be cost-effective. Second, if you are able to impact the numerator (the number of deaths from ALL in your example), then even if the denominator is a certain number in the population (say 100,000), the population mortality will still reflect this reduction in deaths.

  7. OK - If you want to talk about screening:

    Take invasive breast cancer in women under 50, which affects approx. 1 in 70. If appropriate treatment doubles survival among those women at 15 years after detection, you wouldn't see that benefit in your overall population mortality curve.

    Note - I'd choose a 15 year time point (or longer) for evaluation of screening these younger women, because it's only after a while that you see a clear benefit of treating early-stage disease (i.e. removal of small tumors before they've spread). Also, at 15 years out, the lead-time bias would be negligible, if any.

  8. Elaine, I think I understand your point -- I think that you are talking about an increase in the median survival. And what you are saying is that these women are still dying of cancer, but later, is this correct? If so, there is also now an opportunity for them to die of other causes (competing risks). So, altogether, if the prolongation of life due to early detection is that substantial, it should still produce a reduction in the age-adjusted population mortality.

  9. Not quite. Rather, I'm saying that some (at least a third, possibly 50%) of the women would not be dying of breast cancer. The reason they would not be dying of cancer is because their tumors were removed years earlier and didn't spread.

    I don't think I can take this discussion further on your excellent blog, Marya. With respect for your work and insights,