Plagiarism and advertising and COI, oh my!

Update #4, 7:45 AM Eastern, Sunday, June 17 (Happy Father's Day, everyone!)
As of this morning, the HealthWorks Collective has taken down the story as well. What is interesting to me is that neither OneMedPlace nor HealthWork Collective has put any explanation on their respective site, and the reader is essentially consigned to finding "ERROR 404." I sure hope that this is not either of the organization's attempt to sweep the whole thing under the rug. 


Update #3, 6:00 PM Eastern (and last for tonight, I hope)
Another e-mail from Matt Margolis, in which he requested that I let my readers know that 

...we took down our piece after an internal discussion, before you and I made any contact. Hence my request to remove a reference to us.

Still no answer on Aviisha -- perhaps it will appear in the "similar piece" they are planning to publish next week. looking forward to it. 


Update #2, 4:35 PM Eastern
Another exiting update for y'all. A few hours ago I got a message from a Matt Margolis at OneMedPlace informing me that the company has taken the post down after realizing their mistake. Well, why don't I just share the whole message (emphasis mine)?

Hello Marya,

Thank you for reaching out to us. We quickly realized there was an error in attribution and have since pulled the story. We would appreciate at this time that you do not include our name or our article in your post. However, we will be publishing a similar piece next week that will incorporate some of the facts in the NYT story. At which time, I am happy to answer any questions to help give depth to your piece.

Best,

Matt Margolis

Managing Editor, OneMedPlace

I responded asking for "more depth" with respect to whether Aviisha is a client of OneMedPlace. I will let you know what I hear when I hear it. And by the way, the HealthWorks Collective still has the story on its page. 


Update #1, 1:45 PM Eastern
As of right now, the OneMedPlace story page has been taken down. No one from OneMedPlace has communicated with me at this point. The story is still up on the HealthWorks Collective site here. 

About four years ago I decided that it was time to learn more about the recent abundant advances in brain science. Since then, I have read avidly on our neurobiology, behavioral economics and decision science. I have learned about our predictable irrationality, biophilia, heuristics and biases, and our drive to create linear explanations for phenomena where none exists. I also learned about priming, where subtle messages delivered prior to a task's completion (did you know that you can, for example, get kids to improve more on their exams by commending their hard work than their native brilliance?) influence the outcome of the task.

It is in this context of (some) understanding about how the human brain assembles its information pathways that I read this story from yesterday's OneMedPlace News (also reprinted by The HealthWorks Collective here under the byline of Herina Ayot, the Managing Editor for OneMedPlace). This story about a correlation between severe sleep apnea and cancer, starts out thusly:

Two new studies have found that people with sleep apnea, a common disorder that causes snoring, fatigue and dangerous pauses in breathing at night, have a higher risk of cancer. The new research marks the first time that sleep apnea has been linked to cancer in humans.

About 28 million Americans have some form of sleep apnea, though many cases go undiagnosed.

[...] For sleep doctors, the condition is a top concern because it deprives the body of oxygen at night and often coincides with cardiovascular disease, obesity, and diabetes.

All of this is true and truly concerning. The next two paragraphs state

In light of the recent studies, Aviisha Medical Institute, LLC is taking $200 off the cost of its home sleep test, which was originally $449.49, and offering free assessments for the duration of May. The special offer is intended to encourage the public to get tested for sleep apnea and raise awareness about the deadly consequences of untreated apnea. Studies estimate that 85% of sleep apnea sufferers don’t know they have the condition.

One may speculate that other diagnostic technologies developers may promote offers in light of this newfound cancer correlation, as well.

This is when I got a little uncomfortable thinking that this is an advertisement rather than a story. And the final statement really got my hackles up:

Although the study did not look for it, study author Dr. Miguel Angel Martinez-Garcia, of La Fe University and Polytechnic Hospital in Spain, speculated that treatments for sleep apnea like continuous positive airway pressure, or CPAP, which keeps the airways open at night, might reduce the association.

And how does sleep apnea cause cancer?

Lead author Dr. F. Javier Nieto, chair of the Department of Population Health Sciences at the University of Wisconsin School of Medicine and Public Health, commented that five times the risk of cancer is more than just a statistical anomaly. Previous studies in animals have shown similar results, while other studies have linked cancer to possible lack of oxygen or anaerobic cell activity over long periods of time, therefore, it’s possible poor breathing fails to oxygenate the cells sufficiently. 

But then even more happened. From the very beginning of the article, I had sensed something familiar in it. It was my recollection that I had seen this story before about the two studies presented at the American Thoracic Society last month. Dutifully clicking on the link provided in the first paragraph of Ayot's story, I found myself on the NYT's "Well" blog reading the post from May 20, 2012, by Anahad O'Connor. Here is how it starts:

Two new studies have found that people with sleep apnea, a common disorder that causes snoring, fatigue and dangerous pauses in breathing at night, have a higher risk of cancer. The new research marks the first time that sleep apnea has been linked to cancer in humans.

About 28 million Americans have some form of sleep apnea, though many cases go undiagnosed. For sleep doctors, the condition is a top concern because it deprives the body of oxygen at night and often coincides with cardiovascular disease, obesity and diabetes.

And then, disappointingly, toward the end of the post:

Although the study did not look for it, Dr. Martinez-Garcia speculated that treatments for sleep apnea like continuous positive airway pressure, or CPAP, which keeps the airways open at night, might reduce the association.

A couple of things shocked me (in addition to the final statement about CPAP):
1). Ayot's story was almost verbatim (with the exception of the Aviisha advertisement) reprinted from O'Connor's story. There did not seem to be an attribution, unless linking the the original post counts as one. Please, someone who is well versed in this, tell me if this is an acceptable way to attribute. I'll tell you now that in the academic circles this would be (and has been) called plagiarism. As you may recall, I am pretty sensitive to this, having had my work plagiarized recently.
2). The thinly veiled advertisement (inserted into the body of this story which is practically copied from the NYT word for word). The advertisement would not bother me if it had stayed on the OneMedPlace web site -- after all they seem to be a PR agency. But is was reprinted on the HealthWorks Collective's site as a legitimate news item, and I don't believe that HWC is a purveyor of advertorials.

I browsed the web site of Ayot's employer OneMedPlace to see if there is evidence that Aviisha is a client, but did not find any, though my search was admittedly perfunctory. I have reached out to the company for a comment and to understand whether a conflict of interest may exist with Aviisha, but have not heard from them at this time. I will update the post if and when I hear from them.

So drawing on my limited understanding of how the brain works, here is what I am thinking this piece aims to accomplish:
1). Create an awareness of a condition that is apparently common and largely undiagnosed, and do it using words from a high-impact publication
2). Prime the reader with the idea that we understand how it causes cancer (if only in laboratory animals, maybe)
3). Set up Aviisha (and "other diagnostic technology developers") as solution providers
4). Create a linear path to CPAP as the answer

But this is just my uneducated guess at how the human brain may perceive this story. Of course, I could just be playing right into my cognitive biases.   

If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. Thank you for your support!

How to game mortality data

There is a great illustration in this BMJ article of what I discuss in Chapter 2 of my book: the type of mortality that matters. In the figure below from the paper, note that, as the new diagnoses of each of the cancers rise (green lines), the attendant "Deaths" (red lines) stay unchanged. If you look at the Y-axis of each graph, it tells us that the unit of measurement is "Rate per 100,000 people." So the red lines represent population mortality.

"Population mortality" means that the denominator for this value is all people in the population who are at risk for the disease in question. This means that for prostate cancer, for example, we include only those men who have a prostate and exclude all women and men who have had a prostatectomy. Population mortality stands in contradistinction to case fatality. The latter is defined as deaths among all the people diagnosed with the disease. So, for prostate cancer, case fatality would be deaths among all men who have been diagnosed with prostate cancer.

It is not difficult to see how case fatality is a somewhat circular, even self-referential, measure of our diagnostic prowess, but says very little about how well we are doing with disease treatment. If we have tests that are capable of picking up the most minute of diseases, those that are not likely to cause death in the first place, then the denominator becomes inflated with this noise, while the numerator, the actual fatalities, does not change. This leads of course to an apparent reduction in deaths from the disease, but a reduction that is an artifact of overdiagnosis. Population mortality, on the other hand, cannot be gamed this way, as you can see in the figure above. This is the only mortality that gives us honest feedback without a bias about how we are doing with our early detection and other interventions. In the case of the cancers in the figure, the answer is "not so well."

You can learn more about this in Chapter 2, where I even have a figure that illustrates this critical distinction.        

If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. Thank you for your support!

More on "mammography saves lives" story

A story in HealthDay with the title of "Two Studies Find Routine Mammography Saves Lives" talks about studies presented at the annual meeting of the American Society for Clinical Oncology (ASCO) European Breast Cancer Conference (EBCC). The studies, both from the Netherlands, allegedly showed that mammography does indeed save lives. If true, these data would contrast with the preponderance of evidence that has stirred up such a ruckus recently about the utility of mammography, complete with references to rationing and death panels. But let's look at what's reported in today's story more closely.

Cutting through all the definitive bravado, here is a little piece of science that was reported:

Compared with the pre-screening period 1986 to 1988, deaths from breast cancer among women aged 55-79 fell by 31 percent in 2009," Jacques Fracheboud, a senior researcher at the Erasmus University Medical Center in Rotterdam, said in a meeting news release. We found there was a significant change in the annual increase in breast cancer deaths: before the screening program began, deaths were increasing by 0.3 percent a year, but afterwards there was an annual decrease of 1.7 percent," he added. "This change also coincided with a significant decrease in the rates of breast cancers that were at an advanced stage when first detected.

Note, the reference is to deaths from breast cancer without any mention of all-cause mortality. (You can read about why the latter is important here.)

The report next states that over the first 20 years of the screening program

... 13.2 million breast cancer screening examinations were performed among 2.9 million women (an average of 4.6 examinations per woman), resulting in nearly 180,000 referral recommendations, nearly 96,000 biopsies and more than 66,000 breast cancer diagnoses.

So, doing the math, I come up with about 31% false positive rate at the biopsy stage (that's 96,000 biopsies minus 66,000 positives for cancer, all divided by the 96,000 total biopsies). If we use the 180,000 "referral recommendations" as our denominator of all positive tests, and stick with the 66,000 true positive rate, then the false positives grow to (180,000-66,000)/180,000 = 0.63, or 63%. If we spread the 33,000 false positives over the 13.2 million examinations, that equates to 0.25% chance for a false positive. Yet the report goes on to say that (emphasis mine):

For a woman who was 50 in 1990 and had 10 screenings over 20 years, the cumulative risk of a false-positive result (something being detected that turned out not to be breast cancer) was 6 percent.

Six percent? This is clearly a place where my high school math teacher's mantra of "show your work" is applicable.

The next piece of information that I would like to understand better is this:

Over-diagnosis (detection of breast tumors that would never have progressed to be a problem) occurred in 2.8 percent of all breast cancers diagnosed in the total female population and 8.9 percent of screening-detected breast cancers."

How exactly was this computed? Again a case for "show-your-work."

And then there is this (emphasis mine):

Regular screening "decreases deaths by over 30 percent, [with] limited harm and reasonable costs. Additionally, cancers are detected at an earlier stage, which means not only decreased mortality but also morbidity; the patient may not have to have chemotherapy or a mastectomy," she noted.

OK, so, if I got it right, it is breast cancer mortality that is decreased by 31%, not all-cause mortality. This really should have been spelled out more clearly, not to mention that the actual, or absolute, reduction likely pales in comparison to this relative drop. And what about diagnosing earlier stage disease? Lead time bias, anyone?

The second study was a computer model, and I will not go through it at this time as I need to move on to other work. But you get the picture: the numbers given in the report are limited and at times they don't add up. Mixing up cancer mortality with all-cause mortality leads to erroneous conclusions. And finally, forgoing reporting on the absolute risk reduction in favor of the inflated relative reduction is not helpful for understanding the true risks involved.

One final thought: Yes, I do have cognitive biases, and it is difficult for me to avoid them. I happen to fall into the camp that thinks screening for sublclinical diseases, at least in our current technological setting, is disease mongering. At the same time, I would like to think that if the data really showed a significant benefit without great risks, I would give them a second look.

The bottom line is this: at least for me, the report confused the issue more than it has clarified. Perhaps the study, once published, will answer all of the questions that I have posed adequately. But at this stage, it is a shame that such strong statements as...

"These results show why mammography is such an effective screening tool," said one U.S. expert, Dr. Kristin Byrne, chief of breast imaging at Lenox Hill Hospital in New York City. She was not involved in the new research.

 and this...

"We are convinced that the benefits of the screening program outweigh all the negative effects," Fracheboud said.

 ... are not backed up by appropriate evidence.

h/t to @ElaineSchattner for the story


If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. 

Thank you for your support!

Aspirin or bias?

Update 4PM Eastern, 3/21/12

Wanted to append a couple of thoughts I tweeted earlier about these studies, in case you don't follow me on Twitter or just missed them:

Around 2:30 PM Eastern:

And then closer to 3:00 PM Eastern: 

Thoughts?


There is a fascinating review by Cancer Research UK of the new and old aspirin data with respect to its effects on cancer and cardiovascular complications in the context of a heightened risk for bleeding. The review is full of fabulous information about what we know and the uncertainties that remain, all with practical suggestions at the end, so go there and read it.

But here is what I wanted to highlight in the graph that I am reproducing from a screen shot:

There is something very interesting going on here. Just as the risk of bleeding begins to drop, so does the risk of developing cancer. This could be a complete coincidence, but perhaps not. An alternative explanation is that people who already have cancer, though it may not yet be diagnosed, may be at a higher risk for a bleeding complication. Those who develop a bleeding complication presumably are taken off aspirin. But remember, they may already be harboring a cancer that will rear its head in the near future. But what about those who do not bleed and therefore are able to tolerate aspirin for a longer time? They also seem to have a drop in their risk of incident cancer. But of course this may have nothing to do with aspirin's preventing cancer, so much as with its ability to unmask a cancer that is already present and essentially weed them out from the future risk pool for cancer development. And when you weed out those at a higher risk for clinical cancer, by definition you have a group with a lower than standard risk, creating the potential for a selection bias. Make sense?

Conversely, the risk of a cardiac event starts to increase roughly at the same time as the risks for cancer and bleeding begin to drop. This to me suggests confirmation that aspirin may prevent cardiovascular events early in the course of taking it. Furthermore, given my hypothesis above about aspirin's weeding out those with an early cancer, perhaps its cardiovascular impact is for some reason limited to those with an early cancer or with another reason for aspirin-induced bleeding.

All-in-all the data do not convince me to start taking aspirin -- I am still at odds with Dr. Agus on that. The selection bias that I described above may very well mean that aspirin's role is not as a cancer prevention, but more likely as a sort of a stress test for those with a subclinical cancer. So we are left again with the the chicken-and-egg question. But isn't that, after all, what makes science exciting?

Would love to know what others think -- does this make sense? Are there other possible explanations?      

If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. 


 Thank you for your support!

Endometriosis and cancer: When a "breakthrough" may not be all that

There is an interesting new study that was just published online at the Lancet Oncology. It is a pooled analysis of a bunch of case-control studies to explore the association between endometriosis and certain types of ovarian cancer. Here is the abstract:

Background

Endometriosis is a risk factor for epithelial ovarian cancer; however, whether this risk extends to all invasive histological subtypes or borderline tumours is not clear. We undertook an international collaborative study to assess the association between endometriosis and histological subtypes of ovarian cancer.

Methods

Data from 13 ovarian cancer case—control studies, which were part of the Ovarian Cancer Association Consortium, were pooled and logistic regression analyses were undertaken to assess the association between self-reported endometriosis and risk of ovarian cancer. Analyses of invasive cases were done with respect to histological subtypes, grade, and stage, and analyses of borderline tumours by histological subtype. Age, ethnic origin, study site, parity, and duration of oral contraceptive use were included in all analytical models.

Findings

13 226 controls and 7911 women with invasive ovarian cancer were included in this analysis. 818 and 738, respectively, reported a history of endometriosis. 1907 women with borderline ovarian cancer were also included in the analysis, and 168 of these reported a history of endometriosis. Self-reported endometriosis was associated with a significantly increased risk of clear-cell (136 [20·2%] of 674 cases vs 818 [6·2%] of 13 226 controls, odds ratio 3·05, 95% CI 2·43—3·84, p<0·0001), low-grade serous (31 [9·2%] of 336 cases, 2·11, 1·39—3·20, p<0·0001), and endometrioid invasive ovarian cancers (169 [13·9%] of 1220 cases, 2·04, 1·67—2·48, p<0·0001). No association was noted between endometriosis and risk of mucinous (31 [6·0%] of 516 cases, 1·02, 0·69—1·50, p=0·93) or high-grade serous invasive ovarian cancer (261 [7·1%] of 3659 cases, 1·13, 0·97—1·32, p=0·13), or borderline tumours of either subtype (serous 103 [9·0%] of 1140 cases, 1·20, 0·95—1·52, p=0·12, and mucinous 65 [8·5%] of 767 cases, 1·12, 0·84—1·48, p=0·45).

Interpretation

Clinicians should be aware of the increased risk of specific subtypes of ovarian cancer in women with endometriosis. Future efforts should focus on understanding the mechanisms that might lead to malignant transformation of endometriosis so as to help identify subsets of women at increased risk of ovarian cancer.

Funding

Ovarian Cancer Research Fund, National Institutes of Health, California Cancer Research Program, California Department of Health Services, Lon V Smith Foundation, European Community's Seventh Framework Programme, German Federal Ministry of Education and Research of Germany, Programme of Clinical Biomedical Research, German Cancer Research Centre, Eve Appeal, Oak Foundation, UK National Institute of Health Research, National Health and Medical Research Council of Australia, US Army Medical Research and Materiel Command, Cancer Council Tasmania, Cancer Foundation of Western Australia, Mermaid 1, Danish Cancer Society, and Roswell Park Alliance Foundation.

Alas, I do not have access to the full article (paywall), so cannot go though it in a detailed way. Nevertheless, we can try to put these findings in perspective. So, briefly, the investigators put together data from many case-control studies and discovered that the risk of some, though not all, ovarian cancers was 2-3 times higher in the presence of endometriosis than in its absence, and concluded that clinicians should be aware of this increase in risk. Fair? But you know I am going to deconstruct it, right? Here we go.

By now we all understand what a case-control study is, right? It is a study where cases (those patients with the disease of interest) are compared to controls (subjects who are in all ways the same as the cases with the exception that they do not harbor the disease in question). So the study identifies subjects with an outcome, and follows them backward to the exposure that is of interest vis-a-vis this outcome. These studies are notoriously difficult to do well, particularly when it comes to the choice of a control, and very few do it well in my experience. I cannot comment on the current conglomeration of 13 of them, so will not venture a guess on whether some or how many of them may lead us astray. Though these studies are difficult, for various reasons they are the way to go when examining an uncommon outcome. So the choice of the design is legit.

No, let's examine the risk for misclassification for both the disease and the exposure. I think you will agree that a case of ovarian cancer is difficult to misclassify, so I will not pick on this as a potentially major threat to the validity. But what about endometriosis? This is a chameleonic condition that is probably way under-recognized. Its symptoms and signs are varied and, unless the studies required a look "inside," which I sincerely doubt they did -- note, the abstract states that this exposure was self-reported -- there is a very real and grave threat to validity here. Ever heard of recollection bias? If such exists, then more women with cancer are likely to report symptoms that may be indicative of endometriosis than those without cancer. So, the observed increase in the risk of ovarian CA in the presence of endometriosis may be due to just that -- a recollection bias.

These limitations notwithstanding, the authors felt that the study was a breakthrough (emphasis mine):

"This breakthrough could lead to better identification of women at increased risk of ovarian cancer and could provide a basis for increased cancer surveillance of the relevant population, allowing better individualization of prevention and early detection approaches such as risk-reduction surgery and screening,” lead author Celeste Leigh Pearce, at the University of Southern California, Los Angeles, said in a journal news release. 

What does Dr. Pearce mean by "increased cancer surveillance?" Is she talking about screening women with endometriosis because of this possibly heightened risk? And if so, is this really wise? Let's simulate some of these numbers.

Let us suppose that endometriosis does indeed increase the risk of ovarian cancer 2-3-fold. This means that the incidence now goes from 13 per 100,000 women up to 39 per 100,000. Let us now also assume that there is a test that is 99% sensitive (able to identify ovarian CA when it is present) and 99% specific (able to demonstrate that no ovarian CA is present when it is not present). Recall that at the population incidence of ovarian CA, the USPSTF does not recommend screening due to a very high risk of a false positive. The question is does this 3-fold elevation in risk change the positive predictive value of screening substantially enough for it now to be recommended? I think I know the answer, but let's go through the exercise anyway, just to be explicit.

	Disease present	Disease absent	Total
Test+	39	1,000	1,038
Test-	0	98,961	98,962
Total	39	99,961	100,000

So, the corresponding positive predictive value is... drum roll, please... 3.7%. This means that out of 100 women who have tested positive, fully 96 have a false positive result and are now likely to be subjected to invasive procedures. If we imagine that a test can have near-perfect specificity of 99.99% (no test that I know of can come close to this in any consistent way), still 20% of all positive results are false positives. So, is this indeed food for screening thought? I really don't think so, particularly given that the current risk calculation is likely a gross over-estimate.

So, there you have it. I don't think I am engaging in hyperbole when I say that "breakthrough" is very likely an overstatement.                  
  

Is it the science or is it the message?

As you know, I have been thinking a lot about cognitive biases lately. Apropos this, I have had this nagging thought that even smart people (I flatter myself by including myself in this group) cannot escape this trap. What sent me down this rabbit hole is a tweet I saw in my stream from someone whom I respect very much, a science writer of the highest caliber with an international reputation. This person said:

I'm skeptical of her; hearing same from many others: Ntl Acad of Sci researcher: "Disconnect: Truth Abt Cell Phone Rads http://bit.ly/dxpTtG

Now, there is really nothing fundamentally wrong with this statement -- we seek out the opinions of those we trust and align ourselves with them in the absence of evidence to the contrary; this is how friendships and other relationships tend to work. But what is striking to me is the disparity of effort to seek out real evidence when it comes to stuff we agree with vs. stuff that we do not like.

The link is to a radio interview with Devra Davis, whom I have mentioned on several occasions on this blog. I read her book on cancer with great interest, and the history that she told in a measured and well documented way sent shivers up and down my spine. Yes, there was in fact a conspiracy to suppress the science behind the link between smoking and cancer. She described the anatomy of this conspiracy in minute detail, well researched and referenced. The continued legacy of the tobacco strategy is our callousness toward the bulk of chemicals on the market and their health effects, our allegiance leaning more towards saving the economy than lives.

In her new book, which I have not yet read, Davis turns her quill to the cell phone and electronics industry. If this book is as well researched and footnoted and damning as the Cancer book, we should all pay attention. Of course, I can see how the message can become quite unpopular -- the devices that we all rely on, even are addicted to, the devices that make the world go round and add trillions of dollars to our beloved economy, are on trial! Radiation from cell phones, brain cell death and proliferation, especially in children -- not a comfortable line of logic to follow. Yet, if we are to be balanced and scientific about it, should we not examine the science that this author presents, rather than knee-jerk into the ad hominem?

Now, I am singling out this particular tweet unfairly -- the person who wrote it probably did not give much thought to the words. But this is an influential individual, and, as I already mentioned, we often join an opinion bandwagon for reasons other than rational thought or preponderance of evidence, so one influential individual may blind the less critical masses to the very opinion nature of this remark. This is a potential broken telephone game, which will end up in "oh, her, she is a crank, a pseudoscientist and not credible". In fact, the only questions of her credibility that I could find was in comments from listeners like this:

I'm really disappointed that Leonard didn't ask this woman any tough questions, and pretty much just let her rant using his show as her soapbox. I just looked up her CV, and this woman does not have one degree in science. Her background is in sociology and history. Where's the science?

Well, if one bothers really to look up her credentials, this is what we find:

Dr. Davis holds a B.S. in physiological psychology and a M.A. in sociology from the University of Pittsburgh, 1967. She completed a Ph.D. in science studies at the University of Chicago as a Danforth Foundation Graduate Fellow, 1972  and a M.P.H. in epidemiology at the Johns Hopkins University as a Senior National Cancer Institute Post-Doctoral Fellow, 1982.  She has also authored more than 190 publications in books and journals ranging from the Lancet and Journal of the American Medical Association to Scientific American and the New York Times and blogs in Freakonomics for the New York Times, Huffington Post and elsewhere.

And her honors and appointments span the entire page. So, in terms of pedigree, she is impeccable, she has authored more respectable peer-reviewed publications than many giants I know, and she has been on all the "right" commissions and boards.

So, why are we skeptical of her? I think we need to be very honest with ourselves here and really examine our objections. And I would go even further to say that we not only need to examine the science, but we need to be cognizant of whether we are examining this science with the same amount of ferocious scrutiny as we examine science that aligns with our preconceived notions of the world. For it is this magnitude of ferocity ("ferocity quotient"?) that will tell us whether we are biased or completely objective. Ultimately, the question each of us needs to answer for herself is this: Is it the science or is it the message?

Please, don't kill the messenger!

     

More thoughts on lung cancer screening

Today I want to continue in the vein of yesterday's post and discuss a slightly different aspect of the NLST study. But first, a story. I was at a meeting a couple of weeks ago, where we were discussing antibiotics in pneumonia. We were talking about mortality, and one of the docs said something that became a constant mosquito buzz in my brain, which I have been unable to swat away. He said that antibiotics never claimed to reduce all-cause mortality, only mortality associated with infection. Now, on the face of it, the statement makes perfect sense, particularly as you think mechanistically. But it also makes an important point from the perspective of population epidemiology. And it just happens to be related to the NLST study.

Yesterday we talked about the characteristics of the screening tests and in what populations it may make sense. But all of these current and future decisions rest on the results for the primary outcome, which was mortality from lung cancer, reaching an impressive 20% relative reduction. But what about all-cause mortality, and why should we care? Well, saving people from cancer deaths is great, and this is what early screening can mechanistically do. But it is only great if they are not at risk of dying of something else and can derive a meaningful benefit from a life not terminated by an undetected cancer. Unfortunately, as you well know, life does not work this way, and there are many other reasons that people die -- old age, cardiovascular disease, and the like. In epidemiology we call these "competing risks", and it is these risks that sometimes dilute the excitement over the effectiveness of our interventions to a Pyrrhic victory.

Let us apply this to the NLST, or at least what we know today. The results reported are so cursory at this time, that we have to make a lot of educated guesses. But when has that ever stopped us? The report, among other data, states "all-cause mortality was 7% lower" in the CT than in the CXR screening group. The report fails to say whether this is a relative or an absolute reduction, and, as you know, this is a critical distinction. It is critical because, if my calculations are correct, the 20% relative reduction in the cancer mortality detected represents only a 0.3% absolute reduction in deaths from cancer. Given a couple of other tid-bits peppering the report, a few numbers can be back-calculated:

So, again, if my calculations are correct, the all-cause mortality is ~6.4% in the CXR group and ~5.5% in the CT group, so no way can there be an absolute 7% reduction in mortality, and the absolute reduction then turns out to be 0.9% over the 5-year follow-up. And here is what it looks like graphically:

There actually is a reduction in non-cancer deaths in the CT arm. Why this is remains unclear so far, but may well be due to the observation bias, as the study was not blinded. If this is the case, the differences should smooth out over further follow-up period.

But here is my bottom line. Since we live in the world of competing risks, the question becomes "What is the value of screening with CT scans to prevent 0.3% of cancer-related deaths over 5 years?" And how much of this 0.3% benefit is due to lead time bias? This question cannot be answered yet, as we do not have the accounting of either the adverse events related to screening and follow-up, or of the financial costs, or of the patient utilities for either strategy. What we can guess is that the risks have to be miniscule in order not to overwhelm the very small benefit detected, even if it is real.

I am personally not that happy that this report was released the way it was -- no mention of absolute risk reductions, no explicit disclosure of the denominators, no hint as to what the comparative risks of the strategies are (both long- and short-term), and most importantly what implications either strategy has for the patients' quality of life. I guess all of these questions will be answered in the peer-reviewed publications that will result from this study. Fully and transparently. Without any politics or obfuscation. I am waiting with baited breath.

CT screening for lung CA: an epidemiology primer

Amid the hype surrounding it, there is an opportunity to ponder the new NCI CT screening study that reduced lung CA deaths by 20% over the 5-year follow-up. The study itself was interesting, as it was a randomized controlled trial of the screening. This means that, although the possibility of lead time bias remains as the explanation for some of the detected difference (this refers to the fact that in the CT scan group the cancers were diagnosed earlier and therefore the deaths may still occur but simply beyond the five-year follow-up period), this is less likely than in any other design. So, the findings, while intriguing, need to be considered with the modicum of caution as recommended by the NCI, particularly given the high number needed to screen to save one life (300), and the high number of false positives requiring invasive testing. Not to mention all the radiation exposure.

So, while the analyses of the trial continue, some groups, such as the Steelworkers' Union, have already jumped on the side of demanding screening for their members. Is this altogether unreasonable and premature? Well, while it may be premature, I do not think that it is altogether unreasonable. And this has to do with screening test characteristics, some of which are immutable and others exquisitely dependent on the population being screened.

The test characteristics that tell us how well a test detects disease are sensitivity, specificity, predictive positive value and predictive negative value. Sensitivity refers to how well the test picks up the disease that is really there, while specificity tells us whether we can trust that a positive test signifies the presence of the particular disease in question. We also calculate a positive predictive value (PPV, or how much we can trust that a positive test really signifies the presence of the disease), and a negative predictive value (NPV, or how sure are we that a negative test result really means the absence of the disease).

In epidemiology, where we are always concerned with denominators, we like to construct so-called 2x2 tables to illustrate. Here is an example:  

 
The values in the squares signify the following: A = true positives, B = false positives, C = false negatives, D = true negatives.

And while both sensitivity and specificity of a test are intrinsic characteristics of the test itself and are not subject to change by the population risk of the disease, not so for PPV and NPV. These measures are sensitive to the pre-test probability of the disease being present. That is, the PPV and NPV of a test are enhanced if the population is enriched for persons with a higher likelihood of the disease.

This brings me to the Steelworkers' Union demand for CT screening. Here is their rationale:

"Millions of workers have been exposed to asbestos, silica, chromium, arsenic, beryllium, cadmium, nickel and combustion products — and all of these exposures are firmly established as causes of human lung cancer.

Union health and safety leaders and others need to meet in the very near future in Washington DC to devise a strategy for assuring that high risk workers are among the first to obtain the benefits of this new screening method."

And here is why it is not altogether unreasonable. Now, it is well appreciated that these workers have high rates of smoking. According to the CDC, smoking alone increases the risk of lung cancer 10-20-fold. It is also well appreciated that occupational exposure to asbestos, for example, is associated with an enormous increase in the risk of lung cancer. More importantly, however, together with smoking these exposures gang up as a team to help each other to promote cancer much more effectively than if each were working alone. This is called potentiation, and it results in a multiplicative increase in lung cancer risk when both smoking and occupational exposure exist. Therefore, the population represented by the Steelworkers' Union is indeed enriched for the risk of lung cancer development, and may indeed be the population where both the PPV and the NPV are enhanced beyond what we see in smokers only. This means a higher degree of certainty that a positive CT finding truly represents cancer, as well as increased comfort that a negative test is not simply missing a cancer that is there. As a corollary, this means a potentially optimized risk-benefit balance for this screening test.

So, while we are awaiting further data from the trial, the Steelworkers' Union may be on to something.    

    

Does disproving a statistical null automatically render the clinical null disproved?

A good friend of mine lost her mother to pancreatic cancer recently. The whole process from diagnosis to her death took 6 weeks. And despite wonderful care from a palliative medicine team, the process proved grueling to her family. And no wonder: how do you assimilate a loved one's going from healthy to dead in six weeks? Of course, my friend's family made all the right choices, forgoing aggressive treatment in favor of maximizing their mother's comfort and quality of life. Their experience made me think of the new generation of cancer treatments, experienced by my father in his dying days, and how it all fits in the healthcare debate. 

Much of the debate around the healthcare reform has centered around the unsustainable cost trajectory, as well as the value of evidence in improving the quality and effectiveness of care. The Congress has appropriated a substantial amount of dollars for comparative effectiveness research (CER) intended to provide data comparing one treatment to another, rather than just a treatment to a placebo. This in turn has sparked the debate as to whether to include cost-effectiveness in this comparison. I am definitely in the camp that believes it should be included for the reasons so eloquently outlined by Weinstein and Skinner. At the same time this MSNBC article made me re-evaluate its need, at least in some special cases. Let's peel back the cabbage, specifically looking at Tarceva in advanced pancreatic cancer, exactly what my friend's mother was diagnosed with.

Tarceva, or erlotinib, is a kinase inhibitor manufactured by Genentech, indicated for the treatment of some cases of non-small cell lung carcinoma, and most recently approved by the FDA for advanced pancreatic cancer. Reading the package insert, it becomes clear that the FDA-approved 100 mg dose of this drug, if given in combination with gemcitabine, prolongs median survival by <2 weeks, from 6 months in gemcitabine+placebo arm to 6.4 months in the gemcitabine+Tarceva arm, for a p=0.028. This difference imparts statistical significance at the conventionally set p<0.05, and therefore renders Tarceva better than placebo. Period. 

Delving a tad more deeply into the peer-reviewed publication of the phase 3 trial, one gleans a few other facts. I quote:

Survival and Response

The final analysis was conducted after 486 deaths (239 on erlotinib and gemcitabine and 247 on placebo and gemcitabine). Overall survival was significantly longer in the erlotinib and gemcitabine arm with an estimated HR of 0.82 (95% CI, 0.69 to 0.99; P = .038; log-rank test stratified for performance status, extent of disease, and pain score at baseline; Fig 1A). Median survival times were 6.24 months versus 5.91 months for the erlotinib and gemcitabine versus placebo and gemcitabine groups with 1-year survival rates of 23% (95% CI, 18% to 28%) and 17% (95% CI, 12% to 21%), respectively (P = .023). A multivariate Cox regression analysis showed that erlotinib treatment (HR, 0.82; 95% CI, 0.69 to 0.99; P = .04) and female sex (P = .03) were significantly associated with longer overall survival. While there was an imbalance in male:female ratio between the arms, the treatment effect remains significant when adjusted for sex.

 Results of subgroup analyses of survival by baseline stratification factors and other factors such as sex, race, pain intensity score, and age are displayed in Figure 2.

Progression-free survival was significantly longer in the erlotinib and gemcitabine arm than the placebo and gemcitabine arm with an estimated HR of 0.77 (95% CI, 0.64 to 0.92; P = .004; log-rank test stratified for performance status, extent of disease, and pain score at baseline; median, 3.75 months v 3.55 months; Fig 1B).

So, what do we have overall? We have a hazard ratio of dying that very nearly crosses 1.0, thus coming perilously close to not disproving the null; we have a prolongation of median survival by 1/3 of a month, and a progression-free median survival prolongation by 1/5 of a month.

But given my fondness for Gould's "The Median is not the Message" essay, let's practice full disclosure and look at the tail of the Kaplan-Meyer curves above. As the text points out,

...1-year survival rates of 23% (95% CI, 18% to 28%) and 17% (95% CI, 12% to 21%), respectively (P = .023).

Indeed, these are significant differences, both statistically and clinically. Within the trial this represents the difference in favor of survival for 18 additional patients, thus rendering the cost of roughly $1.5 million for 1 year of life saved by my back-of-the envelope calculation. Of course, this difference is not adjusted for confounders, so it is difficult to say of the number is real of under- or over-estimated. Because the adjusted analysis is given as a hazard ratio of death, I cannot calculate the corresponding adjusted cost.

So, for me this begs the following question (and I would love to hear the thoughts from my colleagues who proudly proclaim being science-based and thus eschewing placebo effect as a valid way to get a therapeutic response): Is what we are seeing here real or is this in fact equivalent to a placebo effect? Is the median survival prolongation of <2 weeks indeed a real effect that means something to the patient and the clinician, or is it just clinical noise, if you will? In other words, should "disproving" the statistical null by default disprove the clinical null? Or does the bar for disproving the clinical null need to be set just a tad higher than a statistically significant increase in life expectancy of 2 weeks?

Obviously, no one knows a priori who will do better and who will not. So, without a crystal ball, it is one's values and preferences that have to drive these decisions. But does the society have a say in any of this, as we struggle with equitable distribution of a limited resource? Is $1.5 million for 1 year of life a good societal investment? And what is the quality of this life? And given that at least 1/2 of all treated patients get far less than extra 2 weeks of life, how do we strike the balance between a reasonable expectation of a response and a false hope?

My friend's family based their choices on their mother's wishes and their values and utilities for her comfort. The choice they made was very different from that made by my parents with regard to my father's palliation. Both were right for the respective families. But one may have been far too costly, both financially and emotionally.    

Requiem for my father

My father passed away about two weeks ago after battling a brain tumor for some time. Initially diagnosed with an extensive inoperable mass one and one-half years ago, upon presenting with a focal seizure, he did well with only one medication for seizure control for about ten months. Around Christmas of 2009, however, he landed in the hospital in status epilepticus that took three days to control. After these days of florid hallucinations alternating with pharmacologically achieved stupor, he came out of it remarkably cognitively intact, still able to quote poetry and sing Italian opera arias in their entirety. Although he was now unable to use his legs, he eagerly accepted the prospect of getting back on his feet by working hard in physical therapy. And though this never came to be, he managed to survive for additional seven months.

But my Dad was not your average guy. A survivor of World War II, an immigrant from the Soviet Union, a retired professor of mechanical engineering, he was at once a brilliant and an infuriating man. His stubbornness knew no bounds, and, even though he did not have the affinity for competitive sports, his drive to win was strong. He was a survivor by his nature and by his experience. So, when he was faced with the choice of a rapid decline and death versus some form of treatment for his tumor, he opted for the latter. As he and my mother put it, "doing nothing was not an option". So, when offered 6 weeks of radiation (5 days/week, with the risk of cognitive decline, the prospect of which terrified him as much as dying) together with oral and intravenous chemotherapy, he bravely accepted all.

As the days and the weeks of his treatment wore on, he would give me regular updates on how many radiation sessions he had completed and how many were still left to go. He suffered side effects with minimal complaining and never lost hope that the treatment would help him achieve a measure of meaningful recovery. This was not to be.

While I accompanied my parents to many of his appointments, I did not go to them all. And for this reason, I cannot say for sure what was said in some of them, and how different the content might have been from what they actually heard. But I did make the point of talking to every one of the specialists on the phone after they saw my father. They were all appropriately measured in their enthusiasm and recommendations while talking with me. But did they have the same demeanor with my family? Did my parents, both highly educated but neither medically savvy, take their enthusiasm for palliation as a promise of a cure? More importantly, are similar offers to try to prolong life by well-meaning and compassionate physicians to their desperate patients routinely misinterpreted on a grand scale by thousands as promises of cure? Is it possible that most people accepting the wonders of Avastin and other high tech interventions do not really internalize the reality that this and similar wonder drugs may give them merely two months of additional life? And further, do they think that these added months will be "normal" life?

Well, in my father's case, he got superb care at every institution he went to. His doctors and nurses were knowledgeable, compassionate, humane; in other words they were everything that healthcare professionals should be. Yet their eagerness to help a desperate and scared man may have inadvertently registered as false hope for results that were unlikely. My father tolerated his treatments, but over time began to exhibit cognitive decline, at first subtle, then more pronounced and made exquisitely more painful by his over-achiever baseline. He eventually succumbed to aspiration pneumonia after we made the painful decision to provide him with comfort care only.

After seven months of this most intense good bye, my mother now understands end-of-life issues better than many healthcare professionals. She is a seasoned and experienced decision-maker in a grave situation. But this experience comes at a steep price and a little too late, since I hope she is never faced with having to apply it again to a highly personal situation. She will not regret any of the choices that she and my Dad made in this journey -- they were all correct for them. But I have once again to question the wisdom and utility of making available treatments that make a difference at the slightest of the margins, that cost the society enormous resources and exact such high emotional costs from patients and their families.

Death is a natural conclusion to a life. Since we are not that good at "just sitting there", the vocabulary of "do nothing" needs to change to "helping patients die with dignity and comfort". The manipulative rhetoric of "death panels" has to be replaced with empathy and compassion. We are not above the laws of the Universe, we are not above the laws of physics and biology. We have to reposition our societal attitude toward death as an avoidable nuisance. And we have to ask: what are we prolonging in many of the cases that we treat at the end of life -- life or death? In his better days, my Dad might have asked an analogous question: are we living longer or does it just seem longer? And then he would have chuckled. Rest in peace, Dad.