How can the same test cut prostate cancer deaths and at the same time not save lives? This is counter-intuitive. Yet I hope that a regular reader of this blog is not surprised at all. For the rest of you, here is a clue to the answer: competing risks.
What's competing risks? It is a mental model of life and death that states that there are multiple causes competing to claim your life. If you are an obese smoker, you may die of a heart attack or diabetes complications or a cancer, or something altogether different. So, if I put you on a statin and get you to lose weight, but you continue to smoke, I may save you from dying from a heart attack, but not from cancer. One major feature of the competing risks model that confounds the public and students of epidemiology alike is that these risks can actually add up to over 100% for an individual. How is this possible? Well, the person I describe may have (and I am pulling these numbers out of thin air) a 50% risk of dying from a heart attack, 30% from lung cancer, 20% from head and neck cancer, and 30% from complications of diabetes. This adds up to 130%; how can this be? In an imaginary world of risk prediction anything is possible. The point is that he will likely die of one thing, and that is his 100% cause of death.
Before I get to translating this to the PSA data, I want to say that I find the second paragraph in the Results section quite problematic. It tells me how many of the PSA tests were positive, how many screenings on average each man underwent, what percentage of those with a positive test underwent a biopsy, and how many of those biopsies turned up cancer. What I cannot tell from this is precisely how many of the men had a false positive test and still had to undergo a biopsy -- the denominators in this paragraph shape-shift from tests to men. The best I can do is estimate: 136,689 screening tests, of which 16.6% (15,856) were positive. Dividing this by 2.27 average tests per subject yields 6,985 men with a positive PSA screen, of whom 6,963 had a biopsy-proven prostate cancer. And here is what's most unsettling: at the cut-off for PSA level of 4.0 or higher, the specificity of this test for cancer is only 60-70%. What this means is that at this cut-off value, a positive PSA would be a false positive (positive test in the absence of disease) 30-40% of the time. But if my calculations are anywhere in the ballpark of correct, the false positive rate in this trial was only 0.3%. This makes me think that either I am reading this paragraph incorrectly, or there is some mistake. I am especially concerned since the PSA cut-off used in the current study was 3.0, which would result in a rise in the sensitivity with a concurrent decrease in specificity and therefore even more false positives. So this is indeed bothersome, but I am willing to write it off to poor reporting of the data.
Let's get to mortality. The authors state that the death rates from prostate cancer were 0.39 in the screening group and 0.50 in the control group per 1,000 patient-years. Recall from the meat post that patient-years are roughly a product of the number of subjects observed by the number of years of observation. So, again, to put the numbers in perspective, the absolute risk reduction here for an individual over 10 years is from 0.5% to 0.39%, again microscopic. Nevertheless, the relative risk reduction was a significant 21%. But of course we are only talking about deaths from prostate cancer, not from all other competitors. And this is the crux of the matter: a man in the screening group was just as likely to die as a similar man in the non-screening group, only causes other than prostate cancer were more likely to claim his life.
The authors go through the motions of calculating the number needed to invite for screening (NNI) in order to avoid a single prostate cancer death, and it turns out to be 1,055. But really this number is only meaningful if we decide to get into death design in a something like "I don't want to die of this, but that other cause is OK" kind of a choice. And although I don't doubt that there may be takers for such a plan, I am pretty sure that my tax dollars should not pay for it. And thus I cast my vote for "doesn't."
Very nicely written. Concise and to the point.
ReplyDeleteThanks for stopping by and commenting, Skeptical!
ReplyDeleteDr. Zilberberg: I think you and others should note that the European study -- and all other studies of prostate cancer screening -- are vastly underpowered to detect statistically significant reductions in all-cause mortality. With a sample size of 80,000 or so each in the treatment and control group, and an overall mortality rate in this sample of around 20%, there simply is no way that this study can detect a reduction in the overall mortality rate of the size of the prostate cancer mortality effect, which, as you note, is about 0.1%. With a sample of this size, there simply is no way to detect the small relative change from 20.0% to 19.9%. You CAN detect the larger relative change from 0.5% to 0.4% in prostate cancer mortality. To have sufficient power to look at overall mortality effects of prostate cancer screening, we would need a sample size of over 5 million, which we will never have. I present more detailed calculations in a comment at another blog, prostate cancer infolink, so I won't repeat those calculations here. But the bottom line is: the lack of statistically significant effects on all-cause mortality is a red herring. It is of no importance because the existing studies are way too small to ever be able to detect such an effect.
ReplyDeleteTim Bartik
Slight correction: I meant to say you need a sample size of over 4 million to have adequate power for overall mortality effects, not 5 million. The same qualitative point remains: The European study is way too small to shed much light on this issue.
ReplyDeleteTim, thanks for your thoughtful comment. Of course you are correct. But herein lies the difference between statistical and clinical significance. Let us pretend that the observed "reduction" in overall mortality from 18.5 to 18.2 per 1,000 person-years is real. For a single individual this represents a drop of 0.3% in the risk of death over 10 years. Let's not forget that it also comes at a cost, individually (through testing of the false positives and the potential for complications) and societal (the NNI is 333 plus the follow-up testing). And let's also remember that this was in a randomized trial, where the effect sizes tend to be a lot better than in the wilds of real life. I think I will stick with my original vote.
ReplyDeleteMarya:
ReplyDeleteBut the benefit-cost analysis of screening largely does not rest on the costs of the blood tests, or the percentage reduction in the risk of death. The benefit-cost analysis largely rests on the costs of prostate cancer treatment induced by treatment versus the lives saved. Of these costs, I suspect the largest costs are the significant side effects.
A fairly large percentage, 23%, of the men in the European study who were screened and diagnosed with prostate cancer ended up choosing only watchful waiting throughout the follow-up period. Actually, this percentage was much higher than for the control group, for which the percentage of those diagnosed in watchful waiting was only 16%, as the control group on average was diagnosed with much later-stage prostate cancer, for which watchful waiting (or active surveillance) would be inappropriate.
If one compares the ratio of the number of extra men who were screened and ALSO thereby received some treatment other than watchful waiting, to the reduced number of prostate cancer deaths, one gets a number of about 21 from the latest study. That's the number that would drive any benefit-cost analysis: 1 life saved vs. 21 men who have a 50% chance of side-effects PLUS the costs to the health care system of this "definitive" treatment. I suspect that a serious benefit-cost analysis would find that whether or not the screening's effects pass a benefit-cost test would be a close call.
Now, the European study found that 60% of the screened men had low-risk cancer. Why then did only 23% opt for watchful waiting? If we could change treatment decisions conditional on screening, we could probably get a much more favorable benefit-cost ratio for screening. We could lower the ratio of men treated with "definitive treatment" to lives saved by quite a bit with greater use of active surveillance.
In other words, I think the real issue in prostate cancer is not the screening, but the treatment decisions conditional on screening.
Tim, but is it not true that the watchful waiting is temporary, until they need definitive treatment? In this case we are merely delaying the same fate rather than eliminating it. And if there is no need for treatment down the road, then the screening clearly led to over-diagnosis.
ReplyDeleteIn the study, with a 10 plus year average follow-up, the 23% is the group that ONLY received watchful waiting. I am excluding folks who received watchful waiting PLUS something else.
ReplyDeletePresumably if their PSA tests and biopsy scans suggest that the prostate cancer is indolent, there will be no need to ever do definitive treatment. If their prostate cancer progresses, they will need definitive treatment, and I think most of the analyses of treatment outcomes suggests that such a treatment response to the cancer showing signs of aggressiveness is a highly medically appropriate as well as economically efficient approach.
If we wait until the prostate cancer shows some sign of not being indolent, the treatment is far less likely to be unnecessary to prevent death than if we immediately use definitive treatment on all identified cancers.
Tim, I appreciate the discussion here and on my blog. One thing I think you're missing is that while watchful waiting is associated with fewer side effects than surgery or radiation, it isn't associated with no side effects. One of the reasons that so many men drop out of watchful waiting (or active surveillance) protocols is that they're scared to death that they might die from prostate cancer, favorable statistics notwithstanding. They are less likely to choose treatment for actual indications such as rising PSA, worsening histology, etc. It's also well documented that men with "indolent" cancers suffer more psychological distress, worry about cancer, report worse sexual function, and even may have a higher risk of suicide than comparable-age men (who probably also have indolent cancer, just not diagnosed). Multiplied over millions of men, this is a lot of harm that isn't accounted for in measurements of direct side effects of curative treatment.
ReplyDeletePart I of answer: Kenny: Certainly knowing you have prostate cancer is a cost. But the relevant issue is how big it is relative to other costs.
ReplyDeleteIt seems to me that in any serious benefit-cost analysis of prostate cancer screening, the BIG items to debate are:
(1) The value to be assigned to a saved life, or the value of extended life years, which is likely to be a large dollar number, which will vary with a man's health and age.
(2) The economic costs of definitive treatment.
(3) The probability and dollar costs of the side effects of definitive treatment.
(4) The ratio of definitive treatments needed to save one life.
Now, a full analysis would also have to consider: (5) costs of PSA tests; (6) costs of biopsies; (7) costs of medical side effects of biopsies and their value;, and (8) what you mentioned, the costs of simply KNOWING you have prostate cancer. However, I think it is highly probable that items 5 through 8, even collectively, will not as important as items (1) through (4).
Continued in part 2
Part 2 of answer: Consider the following back of the envelope calculation, where I steal and round up some numbers from the European study and from a 2010 study by Zhang ( http://www.ise.ncsu.edu/bdenton/Papers/pdf/Zhang-2010.pdform ) We need 1000 men to be screened to save one life. The value of life is assumed to be $5 million. Therefore, the benefit of the screening per man screened is $5M/1000=$5,000. What are the costs per man screened:
ReplyDeleteIf each man has 3 screening tests at a cost of $50 each, a cost per man screened of $150. Obviously trivial in relation to benefit.
If 15% of the men have a biopsy at a cost each of
$500, then the biopsy financial cost per man screened is $500 times 150/1000= $75 per man screened = still not big relative to value of life per man screened.
If we assume that the possible infection side-effects of biopsy are severe enough and frequent enough to say be triple the financial costs of the biopsy, that is just another $225 per man screened. Still not big relative to value of life.
Now, suppose that for each man whose life is saved, we have 21 extra men who receive "definitive" treatment other than watchful waiting, and an extra 33 men who discover they have prostate cancer.
For the first category, assume the financial cost of "definitive" treatment are $25,000 per man treated. This cost per man screened are $25,000 times 21/1000=about $525 per man screened. Now we're starting to get a big number.
Suppose further than 10 out of the 21 men have serious side effects. I don't think it's crazy to put a cost on serious side effects of $100,000 or so. The cost per man screened is then $100,000 times 10/1000= $1,000. Another big number.
Now, what you want to do is add an EXTRA cost of simply knowing you have prostate cancer, say the cost for the man of doing watchful waiting. What is that cost? If serious side-effects are valued at $100,000, I have to assume that the costs of watchful waiting are considerably less than that. Otherwise, why do watchful waiting. Say the costs of WW/AS are $10,000 per man in WW. Then the costs of THAT per man screened are $10,000 times 33/1000= $330.
If I'm doing my math right, I've added up to costs of $2305 vs. benefits of $5000. Of that $2305, $1525 is due to the financial and psychological side-effect costs of definitive treatment, or about two-thirds. These relative benefits vs. costs largely depend on the ratio of definitive treatments to lives saved, and the benefits/costs assigned to lives saved and definitive treatment.
Now, I think this is a close call, because obviously we might decide that $5 million is too high a value of life, or we want to adjust downwards and instead look at the number of remaining years of life left and do a value per life year. Furthermore, there might be men who would place a much higher negative value on the side effect costs.
My own view: if the number of definitive treatments needed per life saved is around 20, this is a close call that depends on individual values. If it is 10, I think the screening clearly makes sense for most men. If it is 50, then screening probably does not make sense for most men.
So, IF the European study is right, and IF we can adopt treatment protocols to lower the treatments needed per life saved, THEN screening probably makes sense.
Obviously if the European study is wrong, and the U.S. study is right, you can throw out all these numbers. If we don't save any lives from screening, we have all costs and no benefits.