ACOG's dysmenorrhea FAQs: Evidence of propaganda?

I have been looking up information on endometriosis for a friend of mine, and came upon this from the American College of Obstetricians and Gynecologists:

So I bit and started reading. And about half way through my reading it I realized that this really reminds me of how they taught literature in the my native USSR. The teaching consisted of stock interpretations of the great authors' works through the prism of Communist Party propaganda. In this interpretation all of the writers' messages railed against the monarchy, and all exhortations were for the purpose of freeing the proletariat. No teacher ever dared to disagree, and no student was expected to question.

Why, you ask, do these ACOG FAQs on dysmenorrhea remind me of my schooling in the old country? Well, glad you asked. Check out this gem, for example:

That's it. No follow-up questions? Good!

But really let's take it from the top. So, OK, there is the pelvic exam. I can deal with that because I am used to that as the default for anything going on "down there." Then there is the ultrasounds exam. I guess I can deal with that too because there has been so much in the news about pelvic ultrasound, and that seems to be what is done to get a better look at what is down there. A laparoscopy? Wait, isn't that a surgical procedure? Yeah, they even say it's a surgery, and it's done to get a "look inside the pelvic region." Hmmm, this sounds pretty serious. How come they don't say anything here, in these FAQs, about what they are looking for, how good this surgery is at finding it, what the chances that what they find is responsible for my dysmenorrhea, what is the treatment and how successful it is at alleviating my symptoms of dysmenorrhea, and whether or not there are alternative interventions?

(Does anyone really ask the patients what their FAQs are or are they generated by the clinicians based on what they think should be important to the patient? Or even worse, based on what they think they can give a perfunctory answer to? Just from reading these Qs and As I think it's the latter.)

You get my point. This formulation of information is beyond useless. It seems paternalistic in its "there there, dear, we will take care of everything" attitude. Perhaps I am out of touch. Perhaps women, patients in general, don't want to go beyond what their doctor tells them to do. But I happen to think that it is these FAQs that are out of touch. Granted, I am a "difficult" patient, as even a pelvic exam, let alone ultrasound and surgery, meets with questions around the evidence of its effectiveness. But even if you have only completed ePatient 101, you should know enough to ask about something as serious as a laparoscopy! How can anyone be expected to just acquiesce and, sighing, say "yes, I guess I have to have surgery." This "FAQ" is completely absurd in its willful lack of useful information. And if you read the rest of the document, you will find many places where this is true as well.

I know that some of you will read this and click away saying "oh, there she goes again." But I think you need to rethink your apathy. After all, there are well over 200,000 deaths (and possibly even more than 400,000) annually in the US that happen unnecessarily just from contact with our "healthcare" system. If you can avoid the avoidable, is it not incumbent upon you to be fully informed? You may think that all these recommendations are evidence-based, and there is not a whole lot of wiggle room in how to proceed. Well you are wrong if you think so, since the evidence, even when it is available, is rarely, if ever, unequivocal. And furthermore, in medicine no benefit comes without a risk. Are you sure you want your doctor to make these decisions for you? How is it that people who are not even willing to take wardrobe advice from their mothers wade so enthusiastically into these high-risk medical adventures with their eyes and ears closed?

I wrote Between the Lines to show just how imprecise and uncertain the science of clinical medicine is. But beyond that, I wanted to provide you with tools at least to ask the right questions. So, please, go and ask. And insist that you be included in the FAQ processes. Otherwise, we are just wasting terabytes on propaganda.            

If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. Thank you for your support!

Peeling the cabbage of "works" in treatment interventions

What exactly does it mean when we say that a treatment works? Do we mean the same thing for all treatments? Are there different ways of assessing whether and how well a treatment works? I am sure you've guessed that I wouldn't be asking this question if the answer were simple. And indeed, the answer is "it depends."

What I am talking about is examining outcomes. I did a post a couple of years ago here, where I use the following quote from a Pharma scientist:

"The vast majority of drugs - more than 90 per cent - only work in 30 or 50 per cent of the people," Dr Roses said. "I wouldn't say that most drugs don't work. I would say that most drugs work in 30 to 50 per cent of people. Drugs out there on the market work, but they don't work in everybody."

Here is that word "work" again. What does this mean? well, let's take such common condition as heart disease. What does heart disease do to a person? Well, it can do many things, including give him/her such symptoms as a chest pain, shortness of breath, dizziness and palpitations, to name a few. These symptoms may have at least two sets of implications: 1) they are bothersome to the individual, and in this way may impair his/her enjoyment of life, and 2) they may signal either a present or a future risk of a heart attack. Why are heart attacks important? Well, they are important because one may kill the person who is having it, or one (or several) may weaken the heart to the point of a substantial disability and thus a deterioration in the quality of life. So, there certainly seems to be a good rationale to prevent heart disease either from happening in the first place or from at least worsening when it's already established.


Now, what's available to us to prevent heart disease? Well, some think that lowering one's cholesterol is a good thing. OK, let's go with that. What is the sign that the statins (cholesterol-lowering drugs) "work"? What would it look like if it was about lowering the cholesterol? Say, your total cholesterol is 240. You go on a statin and in 6 months your total cholesterol is 238. Your cholesterol was lowered, it worked! Well, yes, but if you are asking what this 2-point drop really accomplishes, you are beginning to understand the meaning of "work." So, just intuitively we can say that there needs to be a certain, perhaps "clinically significant," drop in the total cholesterol in order for us to say that the drug "worked." 


Great! Now we are sidling up to the real issue: What constitutes a "clinically significant" drop in cholesterol? Is it some arbitrary number that looks high enough? Probably not. How about some drop that correlates to a drop in the risk of the actual condition we are trying to impact, heart disease? Say, a 40-point drop, or getting to below 200, may be the right threshold for the "works" judgment. Ah, but there is yet another question to ask: How often does this type of a drop lead to a reduction in heart disease? Is it always (not likely), or is it the majority of the time (rarely) or at least some of the time (most likely in clinical medicine)? And what portion of that time do we consider satisfactory -- 60%? 40%? 20%? 2%? 


Let me bring just one more layer into this discussion. Many people walk with heart disease and don't know that they have it. Some of these people are destined to have a heart attack and/or die from it. Many others are likely to die from something else before they ever experience any symptoms or signs of their heart disease. This raises the question of whether the statins' ability to reduce cholesterol and hence reduce the risk of heart disease is enough to say that the drugs "work." Perhaps "work" means that by lowering cholesterol (say in the majority of those who take it) they reduce the risk of hear disease in some proportion of those who are at risk for it, and among that proportion whose risk is reduced they also reduce the risk of a heart attack in a few, and of death in even fewer. 


So, to sum up, "works" is a loaded term. For the case we are discussing, there is what I call a "dwindle" effect, where the main outcome, cholesterol lowering, is likely to show a somewhat robust result. On the other hand, this (surrogate) outcome itself is not all that interesting when divorced from what we really care about -- symptoms, heart attacks and death. And I haven't even gone into the side of the equation where the patient gets to decide what "work" means for him/herself. The layers of the possible "works" are a cabbage that we all need to peel when discussing treatment plans with our clinicians and when reading news about new technologies.                     
 

If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. Thank you for your support!

Blind and toothless: The future of middle class in America?

Since it's Friday, we'll steer clear of academic subjects and just tell a story. This is a story of a good friend of mine who works in an editorial position for an interdisciplinary humanities journal. This journal is based at a prestigious private liberal arts institution, and thus she is on staff at that institution. This story is about her, but also about how well the Massachusetts health law is working for people like her. I will call her Jo.

Jo is in her early 50s, a single mom of two lovely teenage boys. Their father who is separated from Jo, although in the picture, does not provide any child support for various reasons. Thus she carries sole financial responsibility for her family.

Well, you say, she is on the journal's editorial staff, she is loaded, right? Nah, surely you are not so naïve as to think that this is a high-paid position at a humanities publication. Let's just say that she is not the 1%; why, she is not even in the top one-half. Would you believe me if I told you that a high-ranking editorial staffer gets a salary that is only 2x above the federal poverty level threshold? So much for those wealthy elite academic East Coast types! But she does get her healthcare insurance through her employer, and this is fortunate, right? Well, here is the rub.

Jo has a complex health condition that affects her entire body, including skin, teeth, eyes, heart, lungs and viscera. Despite this, she is optimistic, upbeat and one of the most patient parents I have ever met. But here is what gets her goat: it is the very fact that she gets her health insurance through her employer! How can this be?

Here is how. The cost of her basic restrictive HMO insurance through her employer is over $5,000 per year. And this flat rate is not affected by the salary level of the employee; in other words, the janitor and the university president get to pay the same amount of money for the same plan. In addition to this, her office visit copay is $20, and her medication co-pays are between $10 and $30 for a month's worth of medication. As you can imagine, for someone with a complicated chronic condition, these co-pays can add up quickly, and they do, to a shocking $2,500 per year. And this is before any allowance for the boys' medical needs or dental or eye care for any of them. Once you add everything up, Jo spends over 1/4 of her entire not-so-stellar income on healthcare. But if we do add dental and eye expenses into the mix (remember, her condition affects these organ systems as well), her healthcare expenditure comes to 40-50% of her annual income! And this is just for draconian restrictions of an HMO! AND, this does not cover all of the time that she spends on the phone finding providers that take her insurance and on schlepping miles away to see that single specialist that the HMO will pay for.

But wait, you say, you live in Massachusetts, the land of socialism, gay marriage and healthcare for all. Why can't she just dump this lousy and expensive employer-provided coverage and go to the Mass Connector, where everyone is equal and all get what they need for what they can pay? And furthermore, you say, isn't the PPACA, the new healthcare law of the land that is fashioned after Romneycare in MA, going to take the choice away from people and MAKE them get insurance through these exchanges rather than through their employers? Jo must be an idiot not to be taking advantage of this communist healthcare state! Hmmm, let's see now.

Jo has had a number of discussions with the staff at the Connector. And yes, you are right in that, if she switched to one of their plans, she would qualify for a plan very similar to her present one for about 1/5 of what she pays now. And the co-pays? Why those would shrink to $0 to $10. How's that for a deal? But here is the catch: According to people she has spoken with at the Connector, a person who has health insurance offered through her employer is not permitted by law to take advantage of the Connector deals if the employer plan offers coverage that meets the minimum standard in the law. And hers does. Ironic, isn't it?

So Jo goes on struggling with the financial burden of her crappy and expensive employer-provided insurance, only now she has to pick and choose: Can she afford to replace her failing dental bridge ($4,700) or should she just choose something so frivolous as feeding her children instead? This is a choice that is not a choice at all, is it? And if these are the choices that we can expect with the PPACA, well, then, we have the law that we deserve: one that will make sure that the investors in the "healthcare" marketplace continue to get handsome returns. But start getting used to people who cannot see their computer screens showing up to work without their teeth!        


If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. Thank you for your support!

Five ways to tame the risk-benefit-uncertainty troika

There was a story on NPR this morning that sent me in a radical direction. It discussed the increase in use of brachytherapy for localized breast cancer. The idea is that this is a concentrated dose delivered much more locally and rapidly (over 5 days) than the conventional external beam radiation (over 6 weeks). The issue is that it has not been tested rigorously in a randomized controlled trial yet, and some oncologists are concerned about its outcomes as they compare to the conventional approach. One of the concerns stems from an increase in the rates of subsequent mastectomies, which are double in brachytherapy relative to conventional. At the same time, there are clear benefits, not the least of which is the period of exposure and the hassle associated with daily trips for radiation for 6 weeks.

Several points popped up in my head in response to the story:
1. When discussing this predominantly women's disease, the expert voices mostly heard from were male (5 of the 6 doctors quoted). Does this matter? Not sure.
2. The priorities addressed by the experts were the traditional outcomes -- survival, recurrence, metastasis. The priorities described by the patient were about her time and convenience today.
3. The concern about the procedure stems from the observed doubling of the need for mastectomy within 5 years among patients treated with brachytherapy, an event "rare no matter what what kind of radiation women got."

So what's my point? Do I think that more rigorous testing is not indicated? Not at all; we need a more rigorous evaluation of the technique. No, what I am wondering is at what point should a procedure like this (or any intervention, for that matter) become available to patients as an option to be considered? Should its availability be determined in a dark room by bespectacled men around a conference table, or should it be put on the menu of choices, along with its risks, benefits and uncertainties, as soon as it looks safe enough, whatever that looks like?

The larger question this raises is what is the degree of uncertainty that we are willing to accept around interventions that become available, be it a drug or a procedure or a device? How do we incorporate patients' priorities for outcomes that are important to them into these decisions? Remember ACT-UP and how they moved the FDA to make more rapid decisions about treatments for HIV/AIDS? Have we swung too far in the opposite direction today, whereby we want a virtual guarantee of safety before a technology is approved?

I offer these 5 potential question to help with making these decisions:
1. How severe/deadly is the disease in question?
2. What is (are) the known potential benefit(s) of the intervention?
3. What is (are) the known potential risk(s) of the intervention?
4. What uncertainties bracket this risk-benefit equation?
5. How does the patient feel about the extent of this risk-benefit-uncertainty balance in the context of her condition?

I think that the first four are the questions that the FDA struggles with every day. They are the gatekeepers for the availability of new technologies, and, therefore, for the relevance of the fifth question. What I am wondering is whether it is not better to start bringing a lot more of the public perspective to the discussion much earlier, so that the patient can have the option of evaluating more choices sooner. I know I may be treading on thin ice here, but I am ignoring any market forces or special interests for the moment. The question I am asking is "In the best of all possible worlds, where no one is trying to sell you anything, when is the best time to give the patient an opportunity to accept or reject an intervention, given the risk-benefit-uncertainty profile?"

Bottom line: There are no guarantees. Just because something is available on the market does not mean that it is completely safe or completely effective. Most importantly, it does not mean that we come even close to being certain about these attributes. As a corollary, just because there are uncertainties about the risks and benefits of an intervention, does it mean that it should not be available as an option for a patient? My guess is that there is a balance of this troika of properties that may be optimal on average, but I am also guessing that that this average balance will miss a substantial volume of outliers. Just as some people thrive on the thrill of bungee jumping while others clamp down just at the mere thought of it, so some patients may surprise us with their position on this risk-benefit-uncertainty continuum.

I apologize if my argument is not clear -- I am definitely thinking about this stuff actively. The one thing I am absolutely sure of is this: Unless the public and clinicians are educated about how to have these conversations, we will always have to rely on and, consequently, blame someone else for making decisions for us.

If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. 


Thank you for your support!

How to make safer decisions in medicine

I love when an article I read first thing in the morning gets me to think about itself all through my morning chores and then erupts into a blog post. So it was with this little gem in the statistical publications "Significance." The author suggests making gambling safer by placing realistic odds estimates right on the poker machines in casinos. He even goes through the generation of the odds of winning and losing and how much based on really transparent assumptions. In fact, what he has in effect constructed is a cost-benefit model for the decision to engage in the game of poker on these machines. Seems pretty simple, right? Just a few assumptions about how long the person will play, some objective inputs about the probabilities, and PRESTO, you have a transparent and realistic model of what is probable.

In medicine, there is a discipline known as Medical Decision Making, and what it does is exactly what you see in the "Significance" article: its practitioners construct risk- (and, hence, cost-) benefit models for decisions that we make in medicine. To be sure,these turn out to be rather more complex, since the inputs for them have to come from a large and complete sampling of the clinical literature addressing the risks and the benefits. But that's the meat; the skeleton upon which this meat hangs is a simple decision tree with "if this then that" arguments. In this way these models synthesize everything that we know about a specific course of action and put it together into a number driven by probability.

They usually go something like this. We have a group of women between 40 and 49 years of age with no apparent risk factors for breast cancer. What is the risk-benefit balance for mammography screening in this specific age layer? One way to approach this is to take a hypothetical cohort of 1,000 women who fit this description and put it through a decision tree. The first decision node here is whether to perform a screening or not. What follows are limbs stretching out toward particular outcomes. Obviously, some of these outcomes will be desirable (e.g., saving lives), while some will be undesirable, ranging from worry about false positive results to unnecessary surgery, chemotherapy, radiation, and even death. Because these outcomes are so heterogeneous, we try to convert everything to monetary costs per quality of life (quality because there are outcomes worse than death, as it turns out). But what underlies all of these models is the mathematics derived from clinical studies, not pulled out of thin air. This is the most useful synthesis of the best evidence available.

To be sure, MDM models are rather more complicated than the poker example. They require a little more undivided attention to follow and understand. Furthermore, I personally did not get a whole lot of exposure to them in my training, but perhaps that has changed. Like anything to do with probability, these models tend to be off-putting in a society that has consigned itself to wide-spread innumeracy. And doctors are certainly not immune from misunderstanding probability. Yet without them perceptions rule, and our healthcare becomes a reckless gamble. In our ignorance we collude to build profits that come with medicalizing small deviations from the perceived normality. Sadly, the primary interests that drive these profits are not usually doing so with probabilistic forethought either, but rather on the basis of red hot conviction that they are right.

Doctors and e-patients need to lead a radical transformation in how we handle decisions in healthcare. It is very clear that willful ignorance has not served us well, and we are all too easily led into panic about every pimple. Resilience can only come when we question our assumptions. Alas, our intuitive brain is almost certain to mislead us when faced with complex information; why else would we need explicit odds listed on poker machines? The absurd complexity of information in medicine deserves no less. It's time to start the probability revolution!

If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. 


Thank you for your support!

The probabiltiy dozen of participatory medicine

Yesterday my rant about uncertainty and probability got quite a bit of play in cyberspace, and I am glad.

Uncertainty is ubiquitous. We consider the odds of rain when choosing what to wear. We do (or at least we should do) a quick mental risk-benefit analysis before buying a burger at Quickie-Mart. We choose our driving routes to work based on the probability of encountering heavy traffic. We do this mental calculus subconsciously but reliably, mostly getting it right. What is odd, though, is that there are certain parts of our lives where we expect complete and utter certainty. I will not get into the political aspects of this fallacy, but I do want to continue down this line of reasoning about healthcare.

As I said yesterday, and many many times in the past, the only certain thing about medicine is uncertainty. And here is what I want you to understand deeply: the amount of uncertainty is much greater than you think. So, every time you say to yourself "I think there is a lot of uncertainty in this information," multiply it by 100, and then you may get close to just how uncertain most information is.

And again, I want to emphasize that this uncertainty gets magnified in the office encounter. So, what is the solution, short of having everyone understand the totality of evidence? Yesterday I said that the solution is to teach probability early and often, and this is indeed the best long-range answer. But is there anything we can do in the short-term? The answer, of course, is yes. And here is what it is.

Everyone needs to learn what questions to ask. Instead of nodding your head vigorously to everything your doctor says, put up your hand and ask how certain s/he is that s/he is on the right track. Here is a dozen questions to help you have this conversation:

1. What are the odds that we have the diagnosis wrong?
2. What are the odds that the test you are ordering will give us the right answer, given the odds of my having the condition that you are testing me for?
3. How are we going to interpret results that are equivocal?
4. What follow-up testing will need to happen if the results are equivocal?
5. What are the implications of further testing in terms of diagnostic certainty and invasiveness of follow-up testing?
6. If I need an invasive test, what are the odds that it will yield a useful diagnosis that will alter my care?
7. If I need an invasive test, what are the odds of an adverse event, such as infection, or even death?
8. What are the odds of missing something deadly if we forgo this diagnostic testing?
9. What are the odds that the treatment you are prescribing for this condition will improve the condition?
10. How much improvement can I expect with this treatment if there is to be improvement?
11. What are the odds that I will have an adverse event related to this treatment? What are the odds of a serious adverse event, such as death?
12. How much will all of this cost in the context of the benefit I am likely to derive from it?

And in the end, you need to understand where these odds are coming from -- the clinician's gut or evidence or both? I prefer it when it integrates both, which, I believe, was the original intent of evidence-based medicine.

Perhaps for some of us this is a stretch: we don't like numbers, we are intimidated by the setting, the doc may be unhappy with the interrogation. But it is truly incumbent on all of us to accept the responsibility for sharing in these clinical decisions. I believe that the docs of today are much more in tune with shared decision-making, and understand the value of participatory medicine. And if they are not, educate them. Ultimately, it is your own attitude to risk, and not just the naked data and the clinician's perceptions of your attitude that should drive all of these decisions.  

Knowledge is empowering, and empowerment is good for everyone, patient and clinician alike. As patients, taking control of what happens to us in a medical encounter can only bring higher odds of a desirable outcome. For physicians, a cogent conversation about their recommendations may help safeguard against future litigation, not to mention augment the satisfaction in the relationship.

And thus starting to discuss probabilities explicitly is very likely to get us to a better place in terms of both quality and costs of medical care. And in the process it may very well train us how to make better decisions in the rest of our lives.

I would love to hear about your experiences discussing probability, be it in a medical or non-medical setting. And as always, thanks for reading.


If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. 

Thank you for your support!

How medicine is like quantum physics

When a patients goes to his doctor to get fixed a pivotal triad of presentation-diagnosis-treatment ensues. The three steps are as follows:
1. History, physical examination and a differential diagnosis
When the patient shows up with a complaint, a constellation of symptoms and signs, a good clinician collects this information and funnels it through a mesh of possibilities, ruling certain conditions in and others out to derive the initial differential diagnosis.
2. Diagnostic testing
Having gone through the exercise in step 1, the practitioner then decides on appropriate diagnostic testing in order to narrow down further the possible reasons for the person's state.
3. Treatment
Finally, having reviewed all the data, the clinician makes the therapeutic choice.

These three steps seem dead simple, and we have all experienced them, either as patients or clinicians or both. Yet the cause for the current catastrophic state of our healthcare system lies within the brackets of each of these three little domains.

The cause is our failure to acknowledge the vast universe of uncertainty dotted sparsely with the galaxies of definiteness, all shrouded in false confidence. And while the cause and the way to address it are conceptually simple, the remedy is not easy to implement. But I am jumping ahead; first I have to convince you that I have indeed discovered the cause of this ruin.

Let's examine what goes on in step 1, the compilation of history and physical to generate a differential diagnosis. This is usually an implicit process that takes place mostly at a subconscious level, where the mind makes connections between the current patient and what the clinician has learned and experienced. What does that mean? It means that the clinician, within the constraints of time and the incredible shrinking appointment, has to listen, examine, elicit and put together all of the data in such a way as to cram them into a few little diagnostic boxes, many of which contain much more material than a human brain can hold all at once, even if that brain is at the right tail of human cognition (or not). What overtakes at this step is a bunch of heuristics and biases. Have we talked about those enough here? Just to review, heuristics are mental shortcuts that can serve us well, but can also lead us astray, particularly under conditions of extreme uncertainty, as in a healthcare encounter. If you want to learn more about this, read Kahneman, Slovic and Tversky's opus "Judgement under uncertainty: Heuristics and biases." As for cognitive biases, I will not belabor them, as there is enough material about them on this web site and elsewhere to overload a spaceship.

The picture that emerges at this step is one of fragments of information gathered being fit into fragments of studies and experience, stirred with mental shortcuts and poured into a bunch of baking tins shaped like specific diagnoses. Is there any room in this process for assigning objective probabilities to any of these events? Well, there is an illusion of doing so, but even this step is done by feel, rather than by computation. So while there is some awareness of a probabilistic hierarchy, it is more chaos than science. Given this picture, it's a wonder it actually works as well as it does, don't you think?

The next step in this recipe is the diagnostic workup. What ensues here is utter Wild West, particularly as new technologies are adopted at breakneck speed without any thought to the interpretation of data that they are capable of spitting out. Here the confusion of the first step gets magnified exponentially, just as it seduces us into further illusion of certainty. The uncertainties in arriving at the differential get multiplied by the imperfections of diagnostic tests to give the encounter truly quantum properties: you may know the results or you may know the patient, but you may not know both at the same time. What I mean is what I have always said on this blog: no test is perfect, and because of this simple truth, unless we know the pre-test probability of the disease in a particular patient, as well as the characteristics of the test, we have no idea about the context of these results. Taking them at face value, as we know, is a grave error.

What follows these results is frequently more diagnostic hit-or-misses, as the likelihood of harm and escalating expenditures without any added value rises. Then comes the treatment, with its many uncertainties and the potential for adverse events, and what are we left with? A pile of costly and deadly steaming manure. So, what's a doc to do?

I think that there is a very simple solution to this, and in its simplicity it will be incredibly hard to implement: education. And I don't just mean medical education. Everything that I have talked about in this post echoes back to the concept of probability. In the secondary education, at least as I remember it, probability is left to Advanced Math. By the time a student becomes eligible to take this course, she has been made to feel that she does not have the facility for math, and that, furthermore, math is boring and useless. So, while my friends in education may have a much better idea of what percentage of kids leave high school having been exposed to some probability, my guess is that it is woefully small. And those that do get exposure to it walk out of class perfectly able to bet on a game of craps or a horse race, but no clue how to apply these ideas to the world they live in.

And so those who progress into healthcare and those who don't have heard the word "probability," but cannot quite understand how it impacts them beyond their chance of winning the lottery. And unfortunately, I have to tell you that, if I relied on what I learned in medical school about probability, well, let's just say it is highly improbable that we would be having this discussion right now. This is why I do now and will for the foreseeable future harp on all of these probabilities, so that when you are faced with your own medical decisions, you will at least know the right questions to ask.

I know I need to wrap this up -- I saw that yawn! Here is the bottom line. First, we need to acknowledge the colossal uncertainties in medicine. Once we have done so, we need to understand that such uncertainties require a probabilistic approach in order to optimize care. Finally, such probabilistic approach has to be taught early and often. All of us, clinicians and patients alike, are responsible for creating this monster that we call healthcare in the 21st century. We will not train it to behave by adding more parts. The only way to train it is to train our brains to be much more critical and to engage in a conversation about probabilities. Without this shift a constructive change in how medicine is done in this country is, well, improbable.              

If you like Healthcare, etc., please consider a donation (button in the right margin) to support development of this content. But just to be clear, it is not tax-deductible, as we do not have a non-profit status. 

Thank you for your support!

Three central questions about medical technologies

Every day my inbox gets filled with announcements for and invitations to attend all kinds of conferences. While many of them are of the traditional medical education sort, more and more I hear about meetings where new technologies and gadgetry in healthcare are the focus. And while the former feature healthcare professionals speaking medicalese from the stage to rapt audiences of other healthcare professionals in dimly lit halls, the latter capture their audience' imaginations with the promise of the future, in all its glitter and glory. And naturally, the latter are what attract techies and patients alike, steamrolling over our staid medieval medical conventions. I heard that the HIMSS conference in Las Vegas last month attracted 37,000 attendees! And the excitement was palpable even through Twitter feeds. This is clearly the preferred way to effect public engagement with healthcare.

But here is the thing: medical advances happen much more slowly than the speed of technology development. That is why, year after year, we go to our professional society meetings and have deja vus all over again. Year after year we hear the same people present the same studies, sometimes, if we are lucky, with a slightly different twist. The last group of breakthroughs I heard about at one of our critical care meetings was over a decade ago. And we have had to backpedal from that quite a bit with the removal of Xigris from the market, and with the realization that tight glucose control had to be used with extreme caution, so as not to kill more critically ill patients than it was meant to save.

This disconnect between the glacial pace of true progress in the clinical sciences and the lightening speed of technological progress raises some obvious questions about our assumptions. If we are not making dramatic breakthroughs in medicine every day, what is this breakneck pace of technology innovation delivering, save for the glitter and a seductive promise of health and wealth? Is there evidence supporting this promise? You might counter by saying that we have years, maybe even decades, of translational catching up to do, bringing all the advances from the bench to the bedside. I would have to say that the magnitude of such advances, as well as clinicians' resistance to them, may have been overstated. You might also point out that the vast advances in computing capabilities have not penetrated sufficiently into our healthcare system, and there I cannot disagree. But whether bringing these advances into clinic without careful planning will improve our health or our healthcare finances remains in question.

Moreover, I see numerous downsides to rushing ahead without thinking through what we are rushing toward. Ostensibly, the light at the end of this bright technological tunnel is better health. What concerns me is that the journey has become a sort of an end in itself: the sheer beauty of the tunnel has itself become the prize. To regain our compass, we need to ask three tough questions, each of them central to understanding the impending advent of too much out-of-context information:
1. Do we really want to walk around with sensors (pdf document, see page 16)? Personally, I find 24/7 monitoring of our vital functions to be a depressing prospect. Furthermore, I sincerely doubt that this is a better (or more cost-effective) way to achieve health than through focus on public health and socioeconomic equity.
2. What is the use of having your genome in your pocket?
How is that going to help us at the stage where all we can do is identify certain levels of risk (bracketed by broad intervals of uncertainty) in isolation from all the influences that modify that risk?
3. Do we want an epidemic of false positive findings and pseudo-disease?
We have enough trouble interpreting positive findings from medical screening tests. Do we really want the public falling prey to the anxiety and over-testing that false positives bring? As a healthcare system, can we sustain such an avalanche? As clinicians are we able to mange these screening snafus?
(I have done so many posts on this issue that you can barely navigate this site without stumbling over their debris).

All of the above is not sexy or shiny, and it brings in the ugly four-letter word "risk" to balance the discussion of the holy grail of benefit. So, how do we sell it to people so hypnotized by technology? I am asking this question quite seriously. I, and many of you out there, really would like people to become more cognizant of these nuances, but how do we accomplish this? Do we need to change our byzantine approach to medical meetings to start attracting a broader audience for our messages? If so, let's get started. Perhaps other forums that are already exceedingly successful can teach us how. I commend TEDMED (which I am attending as a Front-Line Scholar) for starting to bring this important viewpoint to their audiences. Now, how about having HIMSS, Health 2.0 and others join to clarify this other edge of the medical sword? Perhaps we can prevent the wild pendulum swings by thinking things through now. And think how much more credibility we all will have if, by thinking and debating now, we minimize the unintended consequences later?                            

Clinician as the Politbureau of medicine?

Do you think that medicine in the US is centralized? I do, but not in the way that we generally understand centralization. And furthermore, it is this centralization that I believe is making the idea of shared decision making so intimidating to some. Here is what I mean.

If you read management texts, centralization refers to an organization that is run predominantly top-down. In other words, a couple of oligarchs at the top of the ladder make all the decisions without consulting anyone below. In this way all the power is concentrated in the hands of the few. In an antithesis to this, in a decentralized organization, grassroots input and initiatives are incorporated into the fabric of the organization. And while in the times of a great crisis, when rapid decisions are necessary, the benefits of centralization may outweigh its risks, during normal day-to-day operations, such unilateral power can result in obviously negative consequences, from discontent among the employees to making the wrong choices. Furthermore, as organizations grow in size, it gets that much more difficult to run them effectively within the centralized paradigm.

Now, let us look at medicine. The traditional model of the doctor-patient relationship relies on the clinician to know what is right for the patient: take this pill and don't worry about the side effects, dear. Now, clearly, when someone shows up to the emergency room in septic shock, there is very little room for a democratic process; we want the doctor to do rapidly what needs to be done to save the patient. But this is a catastrophic exception to the rule of what modern medicine cares for. From pre-diabetes to pre-hypertension to "borderline cholesterol" to osteopenia to mild depression, these are the "diseases" that are prevalent in the office of the 21st century. None of these is particularly urgent or life-threatening. And if we are honest with ourselves, even a devastating diagnosis of cancer does not demand an instantaneous intervention: in the vast majority of cases there is ample time for discussion and contemplation. So, the centralized approach is the wrong way to go. Thus enter the robust discussion about shared decision making. 

Another reason that centralization of medical decisions is crumbling is the expanding patient panels that clinicians need to engage with in order to stay solvent, all within the context of increasing compliance and regulatory burdens along with decreasing reimbursements. Without an equal growth in one's cognitive ability to multi-task, this escalating imbalance is creating a rising risk for unilateral decisions to be plain wrong.

So, in my mind, this is yet another argument for all parties to embrace shared medical decision making to the extent we as patients are willing and able to do so. Because what is the alternative?          

Why patient lab data should be liberated, with a few caveats

I am admittedly not an expert on health IT, but I am a firm believer in the empowerment of patients to be the driver of her/his health decision making. So this whole discussion about lab data being available directly to the patient is of great interest to me. But it does seem like yet another instance of the two sides coming together not to listen to each other but to be heard by the other side. And as well know, this works so well for any relationship!

Each side's view is represented roughly thusly:
Patients -- these are my data and I have the right to access them as soon as they are available.
Doctors -- we are worried that the sheer volume, complexity and irrelevance of (much) of the data will make it confusing and unnecessarily alarm the patient
Both arguments are valid, of course. But it is important to ask what lurks below the visible portion of each iceberg.

Let's take the patient view. Why do I want immediate access to my data? Well, obviously, because it is mine, it represents the results of testing on my body, and the record should belong to me. I should be able to access it freely whenever I damned well please. I am also more than a little exasperated with having to wait sometimes days to hear from my doctor's office about a result that has been available for a while, but was buried under the reams of paperwork on the MD's desk or his/her assigning a low priority to my data. And I am most exasperated when my lab results get lost or otherwise never make it to me at all. Perhaps if I have direct and unfettered access, this will make thing more efficient for me as an individual.

The doc's view, on the other hand, is that the patient does not necessarily understand what the notation of "low"connotes in reference to, say, total bilirubin, or how to interpret the RDW data. Even more importantly, what if there is an outrageously abnormal value for some important test? Surely the patient will desire an immediate explanation of it and its implications.

So, clearly, both sides have valid concerns. I do think that those of access predominate, as ethically it just makes sense. But for a non-medical person, looking at a lab sheet is like trying to read information about yourself in Chinese: your success in understanding is largely dependent on your ability to read and understand Chinese. So, before that horse leaves the barn, we should think through how to execute this most sensibly. For example, perhaps it is not sensible to have the lab computer directly vomit all of the inane values that no one really looks at right to the patient's account. And backing up a step, perhaps it is time for our lab use to be driven not by the lab equipment packages and processes, but to test only for factors that are of value. If I want to know the patient's creatinine, maybe the other 6 components of the Chem-7 should not be run, or at least not reported. And obscure values like the ones I mentioned above, e.g., RDW, MCHC, etc., should only be available when the situation actually makes them useful, and not just distracting.

I can see a potential positive unintended consequence of this development as well: maybe clinicians will be less trigger-happy ordering all kinds of labs for all kinds of oblique reasons. Maybe, just maybe, this apprehension about the patient's access to all the labs will result in more Bayesian thinking in the office and a lot less shot-gunning. Finally, it will not be all patients that choose to access their data. Let us hope that the selection bias does its job and assures that only those who are truly ready to be educated and empowered decide to do so.

All in all, I am looking forward to the liberation of my lab data. What I worry about is all the calls I will be getting from friends and family to help them understand them. All the same, I will do my part for the education and empowerment that absolutely needs to happen for this to be a successful and meaningful change.        

Side effects: The subject must become the scientist

A few weeks ago someone I know, a normally robust and energetic woman, began to feel fatigued and listless, and had some strange sensations in her chest. She presented to her primary care MD, who obtained an EKG and a full panel of blood tests. The former showed some non-specific changes, while the latter was entirely normal. Although reassured, she continued to experience malaise. When she fetched her EKG, she received a copy with the computer interpretation indicating that, in its wisdom, the program could not rule out a heart attack. Given that her symptoms continued, and now anxiety was piled on top, she presented to the ED, where a heart attack was excluded, and she was scheduled for a stress test. In the subsequent weeks the symptoms continued off and on, and the stress test turned out to be negative for coronary disease. Great, mazel tov!

What I failed to mention was that just prior to the onset of her symptoms, she had been started on 5-fluorouracil cream for a basal cell skin cancer. And while she did not commit my current device of omission with her doctors (including the dermatologist who prescribed the drug), all denied her constellation of symptoms as a potential side effect. And granted, when I looked it up, there was no mention of anything like fatigue and listlessness. So, does it mean that it is not within the realm of the possible that this drug was responsible?

Not at all. And here is why. Our adverse event reporting is essentially a discretionary system. Here is what the FDA says about their Adverse Event Reporting System (AERS):

Reporting of adverse events from the point of care is voluntary in the United States. FDA receives some adverse event and medication error reports directly from health care professionals (such as physicians, pharmacists, nurses and others) and consumers (such as patients, family members, lawyers and others). Healthcare professionals and consumers may also report these events to the products’ manufacturers. If a manufacturer receives an adverse event report, it is required to send the report to FDA as specified by regulations. 

What this means is that, when a patient complains to a doctor of a symptom, even when its onset is in obvious proximity to a particular medication, the doctor is not compelled to report it. The most an average physician will do is look up the known AE profile of the drug and at best look up its interactions with other medications. But one is not generally inclined to use one's imagination (and the constraints of the shrinking appointments spread across exponentially growing cognitive loads conspire against it too) to entertain the possibility that the current problem is related. And yet since many AEs are particularly rare, the knowledge about them must necessarily rely on scrupulous reporting by the prescribers into a central repository. This is what is missing: not the repository, but the impetus to report.

So, when we go looking up side effects of a given medication, we must take the information for what it is: a woefully incomplete list of what has been experienced by other patients. And when someone asks "Do statins make you stupid," instead of denying the possibility, we should just admit that we don't know. Because once drugs are released by the FDA into the wild of our modern healthcare, by relying on others' reports of AEs we become inadvertent enablers of our ignorance about them.

My friend's symptoms abated after she finished the course of the 5-FU cream. None of the MDs bothered to report her symptoms to the AERS, and nor did she. I am not even sure that any of the players were aware of the possibility. Oh, well, an opportunity lost. We need to feel responsible for gathering this knowledge. The subject must be empowered to become the scientist; this is the only way we can get the full picture of the harm-benefit balance of our considerable and unruly pharmacopeia.

If you want to report a possible side effect of a medication, this FDA web page will guide you through the process.

Counterfactuals: I know you are, but what am I?

It occurs to me that as we talk more and more about personalized medicine, the tension between the need for individual vs. group data is likely to intensify. And with it, it is important to have the vocabulary to articulate the role for each.

Scientific method, in order to disprove the null hypothesis, demands highly controlled experimental conditions, where only a single exposure is altered. While this is feasible when dealing with chemical reactions in a beaker, and even, to a great extent, with bacteria and single cells in a petri dish, the proposition becomes a whole lot more complicated in higher order biology. In this way, the phrase "all things being equal" must really apply to the individuals or groups under study.

We call this formulation "the theory of counterfactual," and it is defined in the following way by the researchers at the University of North Carolina (see slide #3 in the presentation):

Theory of Counterfactuals

–The fact is that some people receive treatment.

–The counterfactual question is: “What would have happened to those who, in fact, did receive treatment, if they had not received treatment (or the converse)?”

–Counterfactuals cannot be seen or heard—we can only create an estimate of them.

–Take care to utilize appropriate counterfactual

So, essentially what it means is figuring out what would have happened to, for example, Uncle Joe if he had not smoked 2 packs of cigarettes per day for 30 years. Now, our complexity as the human organism makes it impossible (so far) to replicate Uncle Joe precisely in the laboratory, so we must settle for individuals or groups of individuals that resemble Uncle Joe in most if not all identifiable ways in order to understand the isolated effect of heavy smoking on his health outcomes.

So, you see the challenge? This is why we argue about the validity of study designs to answer clinical questions. This is why a randomized controlled trial is viewed as the pinnacle of validity, since in it, just by the sheer force of randomness in the Universe, we expect to get two groups that match in every way except the exposure in question, such as a drug or another therapy. This is why we work so hard statistically in observational studies to assure that the outcome under examination is really due to the exposure of interest (e.g., smoking), "all other things being equal."

But no matter how we slice this pie, this equality can only be approached, but never truly reached. And this asymptotic relationship of our experimental design to reality may be OK in some instances, yet not nearly precise enough in others. We just cannot know the complete picture, since we only have partial information on how the human animal really works. And this is precisely what makes our struggle to infer causality problematic, and precisely what introduces uncertainty into our conclusions.

What is the answer? Is it better to rely on individual experience or group data? As always, I find myself leaning inward toward the middle. Because an individual's experience is prone to many influences, both internal, such as cognitive biases, and external, such as variations in response under different circumstances, it is not valid to extrapolate this experience to a group. In the same vein, because groups represent a conglomeration of individual experiences, smoothing out the inherent variabilities which ultimately determine the individual results, study data are also difficult to apply to individuals. For this reason medicine should be the hybrid of the two: the make-up of the patient can partly fit into the larger set of persons with similar characteristics, yet also jut out into the perilous territory of idiosyncratic individuality. This is precisely what makes medicine so imprecise. This is precisely the tension between the science and the art of medicine. Because "counterfactuals cannot be seen or heard," Uncle Joe!          

Medicine and the internet: Harnessing the yottabytes

What if medicine in the US is just like the internet? What if it is just as difficult to separate the chaff from the wheat in medicine as it is on the web?

Both the curse and the blessing of the web is its accessibility. This means that anyone's voice can be heard. And it also means that anyone's voice can be heard. So, we are just as likely to stumble upon drivel as we are on information gold. And what takes time and skill is separating the two into neat piles, one to be ruthlessly discarded, and the other cherished for how it enriches us. To be sure without the web we might not have had access to either, and it is the egalitarian nature of the internet that gives us such a variety of sources in our information diet.

Now, let's look at medicine. Every day we hear about how much noise there is in the field, and this noise is difficult, if not impossible, to separate from the signal. Some signals are becoming much clearer, and they tell us that by being too egalitarian in medicine, we have likely been causing great harm. Take, for example, PSA and mammography screenings. The drumbeat of harm associated with these highly non-specific tests and the resultant chase after false positive results, is getting deafening, and rightfully so. Every day we hear that researchers have uncovered a breakthrough mechanism or treatment, and we hear with increasing frequency that a treatment previously thought to be sacrosanct is a bunch of rubbish. What gets lost among all this noise is the possibility of a true breakthrough in disease management or treatment or cure.

Think how hard it is to separate general valuable content from bunk on the web. Now, think of the logs of increase in the levels of difficulty of this task in medicine, where difficult concepts are further shrouded in the opaque cloth of arcane and obfuscating terminology. In fact, it is so difficult, that the class previously designated as the interpreters of this information for the lay public, physicians, are unable to keep up. There is a need for a whole new class of interpreters now -- researchers and patient advocates. And while this is good for the market and the economy, since it creates jobs that had not existed before, it begs a more critical evaluation vis a vis its impact on public's health. It also begs the question of the value of this gadgetry and information glut in medicine -- what is truly the wheat and what is the chaff? And what happens when you continuously try to drink from a fire hose? And do we turn down the stream, or is there another way?

Is it feasible to limit this stream of idea and information generation? Furthermore, is it sensible to do so? Many worry that putting limitations on this is tantamount to stifling innovation. But what is innovation? The most pertinent definition to the current discussion in the Merriam-Webster dictionary is "a new idea, method or device." Nowhere does the definition incorporate the value of this idea, method or device. Perhaps it is left to the free market to determine this value and ultimate use of such innovation. Well, in a market that claims to be free, but is filled with cynical machinations in the form of favoritism, subsidies and pricing games, is objective value really what is valued? And indeed, given the complexity of these "innovations", is it even possible for the end-user to judge their value, even if the market were free?

Yet, even despite all these challenges to establishing the value of innovation on the back end, I am not sure that centrally limiting idea generation is either feasible or right. In the case of ideas on the web, I have come to the conclusion that such microblogging platforms as Twitter can be invaluable filters of information, where my network of favorite tweeters whom I follow faithfully provides me with the wheat that has already been cleaned, yet not always overprocessed. Is this possible in medicine? I know that the FDA and CMS are supposed to provide some filtration for such medical information and interventions, but each is statutorily handcuffed and gagged not to stray beyond their legislative agendas. Therefore, a value filter should not be a body beholden to the letter of the law, or to political or financial interests. It needs to be driven by the spirit of scientific curiosity, objective evaluation and pragmatism. Most importantly, it must be open to a conversation that incorporates respectful dissent and many different perspectives.

Twitter arose out of the drive to share information, and it has shaped itself as a tool for developing value in the gargantuan and ever-growing world of yottabytes. Perhaps it is citizen bloggers and tweeters, including e-patients and clinicians and researchers and writers and others, who will ultimately solve this information glut in medicine by extracting the kernel of usefulness from this morass of vegetation. Harnessing this power systematically and accurately is the next challenge of our information age.

Because ultimately, for human cognition and health, less is more. And we are still human.              

Soliciting contributions: "Healthcare professional as e-patient" series

I am contemplating a series of posts arising from my own recent experience as an e-patient to help the broader e-patient community navigate the stormy medical waters with a bit more comfort. I am looking for other healthcare professionals who have had their own experiences as an e-patient that may be instructive for non-healthcare professionals as patients to contribute to the series. Namely, I am most interested in helping people establish better communication lines and channels with their healthcare providers. I am not looking for a comprehensive description of every aspect of your encounter, but rather one specific point that may be particularly instructive. If you have more than one to share, that is great too, we can do that as well. No bitching or moaning, just lessons that we can all learn from.

Issues I would like to touch upon range from how to bring out risk-benefit balance to how to feel OK about confronting your physician with dissenting information to how best to communicate (not everyone is good on e-mail, for example), given our individual styles and time constraints.

I think contributions from healthcare professionals may be very valuable, as we can see both sides of the coin, so to speak.

Would love feedback from both, healthcare professionals and e-patients on what would be valuable. If you are interested in contributing, please, either let me know in the comments section or e-mail me at healthcareetcblog@gmail.com. If you have an idea for a post, please, be very specific about what your theme is, as I will make decisions based on how relevant it is to what I am envisioning.

This is jut a thought at this point, but seems like there may be something to it. Looking forward to your ideas.

Reviewing medical literature part 3 continued: threats to validity

As promised, today we talk about confounding and interaction.

A confounder is a factor related to both, the exposure and the outcome. Take for example the relationship between alcohol and head and neck cancer. While we know that heavy alcohol consumption is associated with a heightened risk of head and neck cancer, we also know that people who consume a lot of alcohol are also more likely to be smokers, and smoking in turn raises the risk of H&N CA. So, in this case smoking is a confounder of the relationship between alcohol consumption and the development of H&N CA. It is virtually impossible to get rid of all confounding completely in any study design, save for possibly in a well designed RCT, where randomization presumably assures equal distribution of all characteristics; and even there you need an element of luck. In observational studies our only hope to deal with confounding is through statistical manipulation we call "adjustment", as it is virtually impossible to chase it away any other way. And in the end we still sigh and admit to the possibility of residual confounding. Nevertheless, going through the exercise is still necessary in order to get closer to the true association of the main exposure and the outcome of interest.

There are multiple ways of dealing with the confounding conundrum. The techniques used are matching, stratification, regression modeling, propensity scoring and instrumental variables. By far the most commonly used method is regression modeling. This is a rather complex computation that requires much forethought (in other words, "Professional driver on a closed circuit; don't try this at home"). The frustrating part is that, just because the investigators did the regression, does not mean that they did it right. Yet word limits for journal articles often preclude authors from giving enough detail on what they did. At the very least they should tell you what kind of a regression they ran and how they chose the terms that went into it. Regression modeling relies on all kinds of assumptions about the data, and it is my personal belief, though I have no solid evidence to prove it, that these assumptions are not always met.

And here are the specific commonly encountered types of regressions and when each should be used:
1. Linear regression. This is a computation used for outcomes that are continuous variables (i.e., variables represented by a continuum of numbers, like age, for example). This technique's main assumption is that the exposure and outcome are related to each other in a linear fashion. The resulting beta coefficient is the slope of this relationship if it is graphed.
2. Logistic regression. This is done when the outcome variable is categorical (i.e., one of two or more categories, like gender, for example, or death). The result of a logistic regression is an adjusted odds ratio (OR). It is interpreted as an increase or a decrease in the odds of the outcome occurring due to the presence of the main exposure. Thus, a OR of 0.66 means that there is a 34% reduction in the odds (used interchangeably with risk, though this is not quite accurate) of the outcome due to the presence of the exposure. Conversely, a OR of 1.34 means the opposite, or a 34% increase in the odds of the outcome if the exposure is present.
3. Cox proportional hazards. This is a common type of a model developed for a time to event, also known as "survival analysis" (even if not done for survival per se as the outcome). The resulting value is a hazard ratio (HR). For example, if we are talking about a healthcare-associated infection's impact on the risk of remaining in the hospital longer, a HR of, say, 1.8 means that a HAI increases the risk of being in the hospital by 80% at any time during the hospitalization. To me this tends to be the most problematic technique in terms of assumptions, as it requires that the risk of an even stays constant throughout the time frame of the analysis, and how often does this hold true? For this reason the investigators should be explicit about whether or not they tested for the assumption of proportional hazards and whether this was met.

Let's now touch upon the other techniques that help us to unravel confounding. Matching is just that: it is a process of matching subjects with the primary exposure to those without in a cohort study or subjects with the outcome to those without in a case-control study, based on certain characteristics, such as age, gender, comorbidities, disease severity, etc.; you get the picture. By its nature, matching reduces the amount of analyzable data, and thus reduces the power of the study. So, is is most efficiently applied in a case-control setting, where it actually improves the efficiency of enrollment.

Stratification is the next technique. The word "stratum" means "layer", and stratification refers to describing what happens to the layers of the population of interest with and without the confounding characteristic. In the above example of smoking confounding the alcohol and H&N CA relationship, stratifying the analyses by smoking (comparing the H&N CA rates among drinkers and non-drinkers in the smoking group separately from the non-smoking group) can divorce the impact of the main exposure from that of the confounder on the outcome. This method has some distinct intuitive appeal, though its cognitive effectiveness and efficiency dwindle the more strata we need to examine.

Propensity scoring is gaining popularity as an adjustment method in the medical literature. A propensity score is essentially a number, usually derived from a regression analysis, giving the propensity of each subject for a particular exposure. So, in terms of smoking, we can create a propensity score based on other common characteristics that predict smoking. Interestingly, some of these characteristics will be present also in people who are not smokers, yielding a similar propensity score in the absence of this exposure. Matching smokers to non-smokers based on the propensity score and examining their respective outcomes allows us to understand the independent impact of smoking on, say, the development of coronary artery disease. As in regression modeling, the devil is in the details. Some studies have indicated that most papers that employ propensity scoring as the adjustment method do not do this correctly. So, again, questions need to be asked and details of the technique elicited. There is just no shortcut to statistics.

Finally, a couple of words about instrumental variables. This method comes to us from econometrics. An instrumental variable is one that is related to the exposure but not the outcome. One of the most famous uses of this method was published by a fellow you may have heard of, Mark McClellan, where he looked at the proximity to a cardiac intervention center as the instrumental variable in the outcomes of acute coronary events. Essentially, he argued, the randomness of whether or not you are close to a center randomizes you to the type of treatment you get. Incidentally, in this study he showed that invasive interventions were responsible for a very small fraction of the long-term outcomes of heart attacks. I have not seen this method used that much in the literature I read or review, but am intrigued by its potential.

And now, to finish out this post, let's talk about interaction. "Interaction" is a term mostly used by statisticians to describe what epidemiologists call "effect modification" or "effect heterogeneity". It is just what the name implies: there may be certain secondary exposures that either potentiate or diminish the impact of the main exposure of interest on the outcome. Take the triad of smoking, asbestos and lung cancer. We know that the risk of lung cancer among smokers who are also exposed to asbestos is far higher than among those who have not been exposed to asbestos. Thus, asbestos modifies the effect of smoking on lung cancer. So, to analyze those smokers exposed to asbestos together with those who were not will result in an inaccurate measure of the association of smoking with lung cancer. More importantly, it will fail to recognize this very important potentiator of tobacco's carcinogenic activity. To deal with this, we need to be aware of the potentially interacting exposures, and either stratify our analyses based on the effect modifier or work the interaction term (usually constructed as a product of the two exposures, in out case smoking and asbestos) into the regression modeling. In my experience as a peer reviewer, interactions are rarely explored adequately. In fact, I am not even sure that some investigators understand the importance of recognizing this phenomenon. Yet, the entire idea of heterogeneous treatment effect (HTE) and our pathetic lack of understanding of its impact on our current bleak therapeutic landscape, is the result of this very lack of awareness. The future of medicine truly hinges on understanding interaction. Literally. Seriously. OK, at least in part.

In the next installment(s) of the series we will start tackling study analyses. Thanks for sticking with me.        

Reviewing medical literature, part 2a: Study design

It is true that the study question should inform the study design. I am sure you are aware of the broadest categorization of study design -- observational vs. interventional. When I read a study, after identifying the research question I go through a simple 4-step exercise:
1. I look for what the authors say their study design is. This should be pretty easily accessible early in the Methods section of the paper, though that is not always the case. If it is available,
2. I mentally judge whether or not it is feasible to derive an answer to the posed question using the current study design. For example, I spend a lot of time thinking about issues of therapeutic effectiveness and cost-effectiveness, and a randomized controlled trial exploring efficacy of a therapy cannot adequately answer the effectiveness questions.
If the design of the study appears appropriate,
3. I structure my reading of the paper in such a way as to verify that the stated design is, in fact, the actual design. If it is, then I move on to evaluate other components of the paper. If it is not what the authors say,
4. I assign my own understanding to the actual design at hand an go through the same mental list as above with the current understanding in mind.

Here is a scheme that I often use to categorize study designs:

As already mentioned, the first broad division is between observational studies and interventional trials. An anecdote from my course this past semester illustrates that this is not always a straight-forward distinction to make. In my class we were looking at this sub-study of the Women's Health Initiative (WHI), that pesky undertaking that sank the post-menopausal hormone replacement enterprise. The data for the study were derived from the 3 randomized controlled trials (RCT) of HRT, diet and calcium and vitamin D, as well as from the observational component of the WHI. So, is it observational or interventional? The answer to this is confusing to the point of pulling the wool over even experienced clinicians' eyes, as became obvious in my class. To answer the question, we need to go back to definitions of "interventional" and "observational". To qualify as an interventional, a study needs to have the intervention be a deliberate part of the study design. A common example of this type of a study is the randomized controlled trial, the sine qua non of drug evaluation and approval process. Here the drug is administered as a part of the study, not as a background of regular treatment. In contradistinction, an observational study is just that: an opportunistic observation of what is happening to a group of people under ordinary circumstances. Here no specific treatment is predetermined by the study design. Given that the above study looked at multivitamin supplementation as the main exposure, despite its utilization of the data from RCTs, the study was observational. So, the moral of this tale is to be vigilant and examine the design carefully and thoroughly.

We often hear that observational designs are well suited to hypothesis generation only. Well, this is both true and false. Some studies actually can test hypotheses, while others are relegated to generation only. For example, cross-sectional and ecological studies are well suited to generating hypotheses to be tested by another design. To take a recent controversy as an example, the debunked link between vaccinations and autism initially gained steam from the observation that as the vaccination rates were rising, so was the incidence of autism. The type of a study that shows two events changing at the group/population level either in the same or in the opposite direction is called "ecologic". Similar types of studies gave rise to the vitamin D and cancer association hypothesis, showing geographic variation in cancer rates based on the availability of sun exposure. But, as demonstrated well by the vaccine-autism debacle, running with the links from ecological studies is dangerous, as they are prone to a so-called "ecological fallacy". It occurs when, despite the finding in groups of a linked change of the two factors under investigation, there is absolutely no connection between them at the individual level. So, don't let anyone tell you that they tested an hypothesis in an ecological study!

Similarly in cross-sectional studies an hypothesis cannot be tested, and, therefore, causation cannot be "proven". This is due to the fundamental property of "a snapshot in time" that defines a cross sectional study. Since all events (with few minor exceptions) happen at the same time, it is not possible to assign causation to the exposure-outcome couplet. These studies can merely help us think of further questions to test.

So, to connect the design back to the question, if a study purports to "explore a link between exposure X and outcome Y", either an ecologic or a cross-sectional design is OK. On the other hand, if you see one of these designs used to "test the hypothesis that exposure X causes outcome Y", run the other way screaming.

We will stop here for now, and in the next post will continue our discussion of study designs. Not sure yet if we can finish it in one more post, or if it will require multiple postings. Start praying to the goddess of conciseness now!

    

Reviewing medical literature, part 1: The study question

Let's start at the beginning. Why do we do research and write papers? No, not just to get famous, tenured or funded. The fundamental task of science is to answer questions. The big questions of all time get broken down into infinitesimally small chunks that can be answered with experimental or observational scientific methods. These answers integrated together provide the model for life as we understand it.

Clearly, the question is the most important part of the equation, and this is why in my semester-long graduate epidemiology course on the evaluative sciences we spend fully the first four to five weeks talking about how to develop a valid and answerable question. The cornerstone of this validity is its importance. Hence, the first question that we pose is: Is the study question important?

This is a bit of a loaded question, though. Important to whom? How is "important" defined? This is somewhat subjective, yet needs to be scrutinized nevertheless. In the context of an individual patient, the question may become: Is the study question important to me? So, importance is dependent on perspective. Nevertheless, there are questions upon whose importance we can all agree. For example, the importance of the question of whether our current fast-food life style promotes obesity and diabetes is hard to dispute.

Regardless of how we feel about the importance of the question, we must first identify the said research question. At least some of the time you will be able to find it in the primary paper, buried in the last paragraph of the Introduction section. Most of the questions we ask relate to etiologic relationships ("etiology" is medicalese for "causation"). Now, you have heard many times that an observational study cannot answer a causal question. Yet, why do we bother with the time, energy and money needed to run observational studies? Without getting too much into the weeds, philosophers of science tell us that no single study design can give us unequivocal evidence of causality. We can merely come close to it. What does this mean in practical terms? It means that, although most observational studies are still interested in causality rather than a mere association, we have to be more circumspect in how we interpret the results from such studies than from interventional ones. But I am jumping ahead.

Once we have identified and established the importance of the question, we need to evaluate its quality. A question of high quality is 1). clear, 2). specific, and 3). answerable. The question that I posed above regarding fast food and obesity possesses none of these characteristics. It is too broad and open to interpretation. If I were really posing a question in this vein, I would choose a single well defined exposure (consuming 3 cans of soda per day) influencing a single outcome (10% body weight gain) over a specific period of time (over 30 weeks). While this is a much narrower question that the one I proposed above, it is only by answering bundles of such narrow questions and putting the information together that we can arrive at the big picture.

A general principle that I like to teach to my student is the PICO or PECOT model (I did not come up with it, but am its avid user). In PICO, P=population, I=intervention or exposure, C=comparator, and O=outcome. The PECOT model is an adaptation of the PICO for observations over time, resulting in P=population, E=exposure, C=comparator, O=outcome, T=time. These models can help not only pose the question, but to unravel the often mysterious and far from transparent intent of the investigators.

Once you have identified the question and dealt with its importance, you are ready to move on to the next step: evaluating the study design as it relates to the question at hand. We will discuss this in the next post.

Series launch: Critical review of medical literature

Today I am launching a series of posts on how to read medical literature critically. The series should provide a solid foundation for this task and dove-tail nicely with some of the more dense methods themes that occur on this blog. Who should read the series? Everyone. Although the current model of dissemination of medical information relies on a layer of translators (journalists and clinicians), it is my belief that every educated patient must at the very least understand how these interpreters of medical knowledge (should) examine it to arrive at the information imparted to the public. At the same time, both journalists and clinicians may benefit from this refresher. Finally, my own pet project is to get to a better place with peer reviews -- you know how variable the quality of those can be from my previous posts. So, I particularly encourage new peer reviewers for clinical journals to read this series.  

First, a conflict of interest statement. What comes first -- the chicken or the egg? What comes first -- expertise in something or a company hiring you to develop a product? Well, in my case I would like to think that it was the expertise that came first and that Pfizer asked me to develop this content based on what I know, not on the fact that they funded the effort. At any rate, this is my disclaimer: I developed this presentation about three years ago with (modest) funding from Pfizer, and they had it on a web site intended for physician access. Does this mere fact invalidate what I have to say? I don't think so, but you be the judge.

Roughly, the series will examine how to evaluate the following components of any study:
1. Study question
2. Study design
3. Study analyses
4. Study results
5. Study reporting
6. Study conclusions
I am not trying to give you a comprehensive course on how all of this is done, but merely make the reader aware of what entails a critical review of a paper.

Look for the first installment of the series shortly.

The changing language of medicine

A very close friend of mine has breast cancer. It is a very small tumor, diagnosed on an annual mammogram, requiring confirmation with a breast MRI. She had a lumpectomy today, and I was with her at the hospital. This proved to be an enlightening experience.

To put things in perspective, when I was in training and in practice (yes, in the dark ages when we were expected to stay awake AND care for patients for over 48 hours at a time every 3 days), we had not heard of patient-centered medicine. I learned that my role was to diagnose, come up with a plan of action and convince the patient at any cost that my plan was the correct one. To be sure, I always tried to do this in a nice way, but would get a bit impatient when my judgment was questioned. This is the behavior modeled for me by my elders and others whom I respected.

Well, that was then. Having had quite a few years to reflect on the practice of medicine in the context of our healthcare system, I have learned just how misguided this attitude is. And, being a Sagittarius, I cannot fathom how this universal truth is escaping others. Yet escaping it is. This became obvious to me today.

My friend had to have a nuclear medicine test prior to her lumpectomy to define the extent of axillary nodal involvement. She had been told that this is an arduous and painful experience that cannot be mitigated with pre-medication. She was also informed that asking the radiologist to deliver the radionuclide slowly rather than as a rapid push might reduce the sensation. So, my friend, who is herself a physician, was prepared for a civilized and simple conversation with the practitioner. Yet, this is not what transpired. You would think that being asked to deliver the chemical slowly is not such a big and unreasonable request. Well, if you thought this, you were wrong: evidently this was such a big ego blow to the radiologist that she felt compelled to respond snidely, "Well, OK, I am not going to fight with you about it". Now, this is off-putting under the best of circumstances. Imagine being about to go to the OR to have a cancer removed from your breast, and having this snide come-back thrown at you. And why? What is the harm in going along with the patient's request if it makes no difference in the end-result of the test? Is it really necessary to diminish her in such a blatant way?

Well, this physician was of a similar vintage to me, and I can only imagine that she came into practice before patient-centered care became the standard. In her mind, as in mine in those distant days, my involvement with the patient's care was not really about the patient necessarily, unless they fell in line with my recommendation. The shameful fact is that my ego was much too fragile to allow a discussion or questions about my considered course of action. How could they go against my years of training, deep knowledge and their best interests? I cannot say for sure, but it is likely that my friend's radiologist was cut from similar cloth. And what is so obvious to me today has not yet been assimilated by so many of my colleagues, including this person.

As I have said before, the new direction for medicine cannot be what I am used to in real estate: "I do not have what you need, but I will show what I do have". The new direction in medicine must undoubtedly be one where the patient is the center of the encounter, and it is the patient's interest rather than the doctor's ego that must be protected assiduously.

Lest you think that the entire hospital experience was negative, let me be clear: of all the people taking care of my friend, the radiologist was the sole disappointing exception. Her surgeons, anesthesiologists, nurses and ancillary personnel went above and beyond my expectations. I was amazed by the level of civility, good humor, politeness and real involvement everyone exhibited -- it was truly different from my days on the wards and pleasantly eye-opening. It even gave me some hope for the future of medicine in the midst of my normally nihilistic ruminations.            

The great poet Rumi said that changing language can change our life. Well, when the recovery room nurse said to my friend "Let me know when you feel that you would rather rest at home than here", I was overcome with warmth and good will. The language of medicine does seem to be changing. And if it continues in this vein, perhaps it will change our lives.