The taxonomic laziness of "anti"

Sometimes I hear my kids arguing somewhere in the house, and I can almost predict what is coming next: my daughter will come to me and complain that her brother is being mean, and her brother will follow shortly and say that no, she is the one being mean. This is when I have to take a deep breath and ask what it is exactly they mean by "mean." What I am talking about here is an inference that they have made based on some behavior that they disliked in the other. Furthermore, this inference has morphed into a sledgehammer term "mean" that is now coloring their entire discussion. Deconstruction is always the order of the day.    

I observed a similar phenomenon, this time among my professional peers, when I published my paper a few years ago on how there is no evidence that VAP bundles do anything to prevent VAP: even some people whom I consider to possess equanimity decided that I must be "anti-quality." Are you serious? How can anyone be anti-quality? I am merely against spinning our wheels just so that we look like we are doing something. But this is not what this post is about.

Earlier today I came upon this remarkably intelligent post from Jack Stigloe, a blogger who is new to me. The post is aptly titled "Anti-anti-science," and takes the nuanced position that it is way too lazy and reductionist to call any dissenters with anything that remotely resembles a scientific idea "anti-science" (my words, not his). Indeed, Jack states,

My over-riding impression is that ‘anti-science’ is a term that is imaginary and unhelpful. It describes almost nobody and it gets us nowhere. Climate deniers are not anti-science, they are anti- a political view that considers environmental protection as important. Creationists, too, have moral objections to the implications of an evolutionary worldview (John Evans is very good on this). In both cases, these groups use science arguments as their vehicle because they are more sophisticated sociologists of science than the scientists themselves. Where scientists see their evidence as a solid stage on which the public drama of policy can take place, creationists, denialists, anti-vaccinationists and others see a precariously balanced house of cards. Yes, they are stupid and wrong, but calling them ‘anti-science’ doesn’t help. Hitting these people over the head with bigger and bigger science hammers will not win the argument, it will simply confirm their suspicions.

And I do completely agree -- denying the evidence for evolution is simply blind, but to lump this faction together with people who oppose GMO foods is just lazy and destructive to a civil discourse.

The author then goes on to make a poignant confession that

One reason the term ‘anti-science’ raises my hackles is that I think the big beasts of science who use it might be talking about a group that includes me. We social scientists and policy folk have been known to ask difficult questions of science that have been interpreted as attacks.

And then he says that the term "anti-science" represents a "privatization of the idea of progress that is dangerous for science and society." The implication is that those calling everyone else "anti-science" have some sort of a monopoly on scientific ideas and discourse. What is crystal clear is that scientific ideas belong to everyone and should not be subject to "us vs. them" schoolyard brawls. Science demands precision in its taxonomies. So to start calling people who argue out of ignorance anything other than ignorant, or those who argue out of political expediency anything other than politically expedient is imprecise and, well, unscientific.

But there is an even more important and subtle reason why such taxonomic laziness is pernicious. This reason is that anyone who disagrees with us can easily become "anti-science," an insulting waste basket connoting ignorance. Why is this so bad? At one level, an overt insult thrown at an individual or a group generally raises their hackles, and, for the most part, eliminates any chance for a civilized discussion of idea. In this way the groups become even more polarized and entrenched without any way to get to any common ground. At another level, such labels give rise to a deeply flawed impression that science is precise. Indeed, those who fling these labels tend to fall on the old refrain of "look, I am the first one to admit that scientific knowledge is fungible." However, in the instances of current confrontations, they somehow cannot imagine that our current knowledge is incomplete and that further questions are not only legitimate, but are indeed a scientific imperative.

So, what is the take-home? Simple: respect dissent and use correct terminology. In other words debate from a place of thirst for knowledge. I realize that it is tiresome to have to argue that creationism is not science, that evolution is more than just a theory, that vaccines have saved millions of lives. But to write off these arguments as beneath us and to throw insults at each other only works in Washington, and look how well that has been going. But even more importantly, if we are going to answer such deeply pragmatic scientific questions of our time as "what are the full implications of GM salmon," we cannot shroud ourselves in the "sacredness" of science. Science can only stay beautiful and true if it steers clear of being dogmatic. It is time to take this discourse out of the boxing ring of childish insults and back into the civic society where if belongs.   

Marie Curie, Geiger counters and mass hysteria: more in common than meets the eye

What do Marie Curie, a Geiger counter and mass hysteria have in common? Well, to answer this question we need to go Sir Arthur Eddington, who was a British astrophysicist and philosopher of science at the turn of the 20th century. He came up with what is frequently referred to as the Eddington parable, which has nothing to do with the stars specifically and everything to do with how we make scientific progress. Here it is for your reading enjoyment, as told in this editorial (available by subscriptionby Diamond and Kaul, two highly respected clinician-researchers:

Let us suppose that an ichthyologist is exploring the life of the ocean. He casts a net into the water and brings up a fishy assortment. Surveying his catch, he [concludes that no] sea-creature is less than two inches long. An onlooker may object that the generalization is wrong. "There are plenty of sea-creatures under two inches long, only your net is not adapted to catch them." The ichthyologist dismisses this objection contemptuously: "Anything uncatchable by my net is ipso facto outside the scope of ichthyological knowledge, and is not part of the kingdom of fishes which has been defined as the theme of ichthyological knowledge. In short, what my net can't catch isn't fish”.

Suppose that a more tactful onlooker makes a rather different suggestion: "I realize that you are right in refusing our friend's hypothesis of uncatchable fish, which cannot be verified by any tests you and I would consider valid. By keeping to your own method of study, you have reached a generalization of the highest importance—to fishmongers, who would not be interested in generalizations about uncatchable fish. Since these generalizations are so important, I would like to help you. You arrived at your generalization in the traditional way by examining the fish. May I point out that you could have arrived more easily at the same generalization by examining the net and the method of using it?"

So,you see my point? Tools determine knowledge. Period.

  

Tipping a sacred cow: QI under the microscope

So much media and journal space has been devoted to financial conflicts of interest, particularly within and related to pharma and device manufacturers, that to write any more about it may be redundant. On this site we have also intermittently addressed COI from other perspectives, such as financial interest of the members of the American College of Radiology in maintaining mammography screening status quo, thinly veiled in its own version of the pernicious "death panel" language. We have also spoken a bit about the non-financial COI. And even though we are so very much aware of COI's potential to lurk around every corner, there are still some surprises.

Take the sacred cow of "quality improvement" in healthcare. Even the name, much like the "pro life" moniker, suggests that it is untouchable in its purity and nobility of purpose. So necessary is it because of the epic magnitude of morbidity and mortality attributed to healthcare itself, that the billions of dollars spent on it seem unquestionably justified. Indeed, much like our public education system, the QI movement garners higher and higher allocations simply due to the sheer face validity of the assumption that more of it is better. And the most fascinating aspect is that, in our current zeal for sensible economic allocation through evidence, QI, much like education, appears immune to scrutiny. This is the very definition of politics driving policy.

I return to the case of ventilator-associated pneumonia, or VAP, as the poster child for this movement. I have already alluded to the fact that definitionally VAP is a slippery slope: its diagnosis varies based not only on the tools used to diagnose it, but also depending on who is doing the diagnosing. Yes, indeed, what one clinician calls VAP another may call absence of VAP. I have also dissected the weak evidence behind some of the strongest purportedly evidence-based recommendations aimed at VAP prevention. But what if VAP itself is the wrong endpoint? What if we are spending untold dollars and other resources on a futile pursuit?

Do you feel yourself bristling yet? If you said "yes", it is a normal response I get from my colleagues and people who read my scholarly papers. Because how can anyone be against QI? Well, I am not against QI. I am simply against sanctifying QI as a sacred cow and thus shielding it from a sensible and rational evaluation.

So, if you are over the initial shock, allow me to explain myself. I am sure you have heard of surrogate endpoints. Here is a definition from Wikipedia:

In clinical trials, a surrogate endpoint (or marker) is a measure of effect of a certain treatment that may correlate with a real clinical endpoint but doesn't necessarily have a guaranteed relationship. The National Institutes of Health (USA) defines surrogate endpoint as "a biomarker intended to substitute for a clinical endpoint".^[1][2]

Surrogate markers are used when the primary endpoint is undesired (e.g., death), or when the number of events is very small, thus making it impractical to conduct a clinical trial to gather a statistically significant number of endpoints. The FDA and other regulatory agencies will often accept evidence from clinical trials that show a direct clinical benefit to surrogate markers. ^[3]

This begs the question of what constitutes a "real" clinical endpoint. Well, in my simplemindedness I think of them as endpoints that matter to the patient or in the long run. So, death, disability, quality of life, functionality, these are the real endpoints. Something that alters one's life or threatens it is a real endpoint. Thus, blood pressure and cholesterol are surrogate endpoints, since they usually, but not always, correlate with the risk of a myocardial infarction or death. But what if such a correlation did not exist? Furthermore, what if a cholesterol level was measured with, say, tea leaves, and therefore was subject to a tremendous variation in detection? Would we then spend hundreds of billions of dollars on trying to alter this factor or would we calmly and rationally walk away and look for something that truly impacts the real outcome of a heart attack or death? I think I am making my point fairly clearly.


Let me explain why I think that VAP is but a surrogate outcome, and, given its diagnostic challenges, not a sensible one in the least. VAP by definition occurs in patients on mechanical ventilation (breathing machine), whose quality of life is fairly badly damaged in the short term. The literature would suggest that not all VAP impacts mortality adversely, but some forms of VAP indeed do, particularly VAP that develops late in the course of illness. So in this VAP does correlate with a real endpoint. Also, there is very little doubt that getting VAP prolongs one's dependence on mechanical ventilation, and increases the duration of the stay in the ICU and hospital overall. So, this can be considered not a very good, albeit real, outcome. An additional point to remember is that VAP engenders the use of additional, usually broad spectrum, antibiotics, putting both the individual and the society at risk for such unwanted consequences as the emergence of highly resistant microorganisms.


So, even though VAP is a surrogate endpoint, it certainly seems to fit the bill for something we would want to prevent. But here is the monkey wrench in this argument: what seem to be great surrogate endpoints do not always end up correlating with clinical reality. The association of VAP with morbidity and mortality has been detected in mostly retrospective observational studies. Trials of VAP prevention rarely, if ever, report any endpoint other than VAP. And, given how elusive VAP diagnosis is, there is plenty of room for gamesmanship so pervasive in the real world to make any data fit our preconceived hypotheses and political needs. 


So, what is my point? My point is that if QI wants to be a science, it needs to be subject to the same rules that all other science is guided by. Since we do not even know how much money we are spending on the ubiquitous QI efforts (likely hundreds of billions), and since we are not sure what they are accomplishing (see my many prior posts on the lack of validity of current claims in VAP prevention), we need to pause and ask ourselves whether the cheering alone justifies such an investment. I hate to say it, but can we really trust those with most to lose, financially and politically, if in reality QI does little more than lather the masses, to be the oracles of truth about the results of these efforts? The cognitive biases alone should disqualify them from being the arbiters of their own success. So, if we do not want to continue to indulge the principle of diminishing returns in QI, we need to take a sober look at what we have invested and what this investment has accomplished. Then and only then can we claim to practice evidence- rather than politics-or dogma-based policy.

The beautiful uncertainty of science

I am so tired of this all-or-nothing discussion about science! On the one hand there is a chorus singing praises to science and calling people who are skeptical of certain ideas unscientific idiots. On the other, with equal penchant for eminence-based thinking, are the masses convinced of conspiracies and nefarious motives of science and its perpetrators. And neither will stop and listen to the other side's objections, and neither will stop the name-calling. So, is it any wonder we are not getting any closer to the common ground? And if you are not a believer in the common ground, let me say that we are only getting farther away from the truth, if such a thing exists, by retreating further into our cognitive corners. These corners are comfortable places, with our comrades-in-arms sharing our, shall we say, passionate opinions. Yet this is not the way to get to a better understanding.

Because I spend so much time contemplating our larger understanding of science, the title "Are We Hard-Wired to Doubt Science" proved to be a really inflammatory way to suck me into thinking about everything I am interested in integrating: scientific method, science literacy and communication and brain science. The author, on the heels of doing a story on the opposition to smart meters in California, was led to try to understand why we are so quick to reject science:

But some very intelligent people I interviewed had little use for the existing (if sparse) science. How, in a rational society, does one understand those who reject science, a common touchstone of what is real and verifiable?

The absence of scientific evidence doesn’t dissuade those who believe childhood vaccines are linked to autism, or those who believe their headaches, dizziness and other symptoms are caused by cellphones and smart meters. And the presence of large amounts of scientific evidence doesn’t convince those who reject the idea that human activities are disrupting the climate.

She goes on to think about the different ways of perceiving risk, and how our brains play tricks on us by perpetuating our many cognitive biases. In essence, new data are unable to sway our opinion because of rescue bias, or our drive to preserve what we think we know to be true and to reject what our intuition tells us is false. If we follow this argument to its logical conclusion, it means that we just need to throw our hands up in the air and accept the status quo, whatever it is.

I happen to think that the author missed an opportunity to educate her readers about why we need to come to a better understanding and how to get there. The public (and even some of my fellow scientists) needs to understand what science is and, even more importantly, what it is not.

First, science is not dogma. Karl Popper had a very simple litmus test for scientific thinking: He asked how you would go about disproving a particular idea. If you think that the idea is above being disproved, then you are engaging in dogma and not science. The essence of scientific method is developing an hypothesis from either a systematically observed pattern or from a theoretical model. The hypothesis is necessarily formulated as the null, making the assumption of no association the departure point for proving the contrary. So, to "prove" that the association is present you need to rule out any other potential explanation for what may appear to be an association. For example, if thunder were always followed by rain, it might be easy to engage in the "post hoc ergo propter hoc" fallacy and conclude that thunder caused rain. But before this could become a scientific theory, you would have to show that there was no other explanation that would disprove this association.

So, the second point is that science is driven by postulating and then disproving the null hypotheses. By definition, an hypothesis can only be disproved if we 1). the association exists, and 2). the constellation of phenomena is not explained by something else. And here is the third and critical point, the point that produces equal parts frustration and inspiration to learn more: That "something else" as the explanation of a certain association is by definition informed only by what we know today. It is this very quality of knowledge production, the constancy of the pursuit, that lends the only certain property to science, the property of uncertainty. And our brains have a hard time holding and living with this uncertainty.

The tension between uncertainty and the need to make public policy has taken on a political life of its own. What started out as a modest storm of subversion of science by politics in the tobacco debate, has now escalated into a cyclone of everyday leveraging of the scientific uncertainties for political and economic gains. After all, how can we balance the accounting between the theoretical models predicting climate doom in the future and the robust current-day economic gains produced by the very pollution that feeds these models? How can we even conceive that our food production system, yielding more abundant and cheaper food than ever before, is driving the epidemic of obesity and the catastrophe of antimicrobial resistance? And because we are talking about science, and because, as that populist philosopher Yogi Berra famously quipped, "Predictions are hard, especially about the future," the uncertainty of our estimates overshadows the probability of their correctness. Yet by the time the future becomes present, we will be faced with potentially insurmountable challenges of a new world.

I have heard some scientists express reluctance about "coming clean" to the public about just how uncertain our knowledge is. Nonsense! What we need under the circumstances is greater transparency, public literacy and engagement. Science is not something that happens in the bastions of higher education or behind the thick walls of corporations. Science is all around and within us. And if you believe in God, you have to believe that God is a scientist, a tinkerer, always looking for a more elegant solution. The language of science that may seem daunting and obfuscatory. Yet do not be afraid -- patterns of a language are easy to decipher with some willingness and a dictionary. Our brains are attuned to the most beautiful explanations of the universe. Science is what provides them.

Self-determination is predicated upon knowledge and understanding. Abdicating our ability to understand the scientific method leaves us subject to political demagoguery. Don't be a puppet. We are all born scientists. Embrace your curiosity, tune out the noise of those at the margins who are not willing to engage in a sensible dialogue, leave them to their schoolyard brawling. And likewise, leave the politicians, corporate interests, and, alas, many a journalist, and start learning the basics of scientific philosophy and thought. Allow the uncertainty of knowledge excite and delight you. You will not be disappointed.                   
              

Reviewing medical literature, part 1: The study question

Let's start at the beginning. Why do we do research and write papers? No, not just to get famous, tenured or funded. The fundamental task of science is to answer questions. The big questions of all time get broken down into infinitesimally small chunks that can be answered with experimental or observational scientific methods. These answers integrated together provide the model for life as we understand it.

Clearly, the question is the most important part of the equation, and this is why in my semester-long graduate epidemiology course on the evaluative sciences we spend fully the first four to five weeks talking about how to develop a valid and answerable question. The cornerstone of this validity is its importance. Hence, the first question that we pose is: Is the study question important?

This is a bit of a loaded question, though. Important to whom? How is "important" defined? This is somewhat subjective, yet needs to be scrutinized nevertheless. In the context of an individual patient, the question may become: Is the study question important to me? So, importance is dependent on perspective. Nevertheless, there are questions upon whose importance we can all agree. For example, the importance of the question of whether our current fast-food life style promotes obesity and diabetes is hard to dispute.

Regardless of how we feel about the importance of the question, we must first identify the said research question. At least some of the time you will be able to find it in the primary paper, buried in the last paragraph of the Introduction section. Most of the questions we ask relate to etiologic relationships ("etiology" is medicalese for "causation"). Now, you have heard many times that an observational study cannot answer a causal question. Yet, why do we bother with the time, energy and money needed to run observational studies? Without getting too much into the weeds, philosophers of science tell us that no single study design can give us unequivocal evidence of causality. We can merely come close to it. What does this mean in practical terms? It means that, although most observational studies are still interested in causality rather than a mere association, we have to be more circumspect in how we interpret the results from such studies than from interventional ones. But I am jumping ahead.

Once we have identified and established the importance of the question, we need to evaluate its quality. A question of high quality is 1). clear, 2). specific, and 3). answerable. The question that I posed above regarding fast food and obesity possesses none of these characteristics. It is too broad and open to interpretation. If I were really posing a question in this vein, I would choose a single well defined exposure (consuming 3 cans of soda per day) influencing a single outcome (10% body weight gain) over a specific period of time (over 30 weeks). While this is a much narrower question that the one I proposed above, it is only by answering bundles of such narrow questions and putting the information together that we can arrive at the big picture.

A general principle that I like to teach to my student is the PICO or PECOT model (I did not come up with it, but am its avid user). In PICO, P=population, I=intervention or exposure, C=comparator, and O=outcome. The PECOT model is an adaptation of the PICO for observations over time, resulting in P=population, E=exposure, C=comparator, O=outcome, T=time. These models can help not only pose the question, but to unravel the often mysterious and far from transparent intent of the investigators.

Once you have identified the question and dealt with its importance, you are ready to move on to the next step: evaluating the study design as it relates to the question at hand. We will discuss this in the next post.

Guest post: How our brains are wired to advance science

We have a treat today. Today I am featuring a guest post from my brilliant 17-year-old niece Katherine Dana. She is currently applying to colleges, and this is one of her brief essays. Kathy is interested in animal communication specifically, but, as you can see, also spends a lot of time thinking about science in general. And oddly, she seems to be contemplating similar themes to the ones we address here. 
While it is hard for me to stop waxing poetic about how proud I am of her, I will now cut myself short, so that you can enjoy her lucid commentary.

By Katherine E. Dana

Marcel Proust once wrote, "The real voyage of discovery consists not in seeking new landscapes, but in having new eyes." Thus goes the song of science, humanity's great unifier. Science is not merely the means for collecting random information—it is the means through which we make sense of our world. It is messier than mathematics, less exact. And yet in some ways, it is this very inexactitude that gives science its potency, and allows it to cut to the very heart of nature's chaotic randomness. It works by taking the givens of nature and churning out elegant guesses, which predict as effectively as they describe.

One quality that distinguishes mind from machine is that leap of thought that psychologists term "heuristics"—mental shortcuts, expressly designed to help us connect the dots without having to consciously traverse the spaces between. This is our organic advantage.

While today's machines, no matter how complex, are restricted to lengthy algorithms, we may leap from branch to branch. Nowhere in human endeavors is this cognitive edge more apparent than in the combined efforts of humans seeking to find new truth. For before we can know, we must question; and this is where insight is most crucial. It is not enough to investigate the familiar. We must find the courage to ask uncomfortable questions, and be willing to uproot even our most cherished beliefs, all in the name of a deeper understanding.

Some implications of biologic plausibility

Ever since my... ahem... skirmish... with the folks over at the SBM, I have been contemplating the issue of biologic plausibility. They contend that our tax dollars are wasted by being allocated to the NCCAM to pursue research into CAM. Their reasoning is that there is no biological plausibility to any of it having any therapeutic effect. Now, this is a big bite to swallow. As I have said before there is CAM and then there is CAM. CAM seems to be a convenient wastebasket of modalities that we feel justified in bashing as "woo" since there is limited scientific evidence behind them. But really, I am more willing to give acupuncture and massage the benefit of the doubt than, say, healing crystals (even though I confess I really like rocks!).

So, what of this biologic plausibility, and who came up with it anyway? And is it truly fiscally irresponsible, and possibly even unethical, to test interventions that do not fit our biological plausibility criteria? As a corollary, is there a level of our understanding of biology that makes testing equally wasteful or even unethical? And finally, should plausibility of benefit and harm be required to reach the same evidentiary bar?

For the definition of biological plausibility we apparently thank the milestone 1964 Surgeon General's report linking smoking to cancer. This report was the first official US government document to state that there was enough evidence to implicate cigarette smoking in the rise in lung cancer and cancer deaths. Since the limitations of observational research were used by the critics for decades to derail this definitive statement, the report itself does a nice job laying out the methodologic considerations and the need to rely on the Bradford-Hill criteria. It was in the "coherence" criterion that biologic plausibility entered the picture.

A quick check of my favorite crowd-sourced information site, the Wikipedia, uncovers this treasure from Sir Bradford Hill himself:

It will be helpful if the causation we suspect is biologically plausible. But this is a feature I am convinced we cannot demand. What is biologically plausible depends upon the biological knowledge of the day. To quote again from my Alfred Watson Memorial Lecture [1962], there was

"…no biological knowledge to support (or to refute) Pott’s observation in the 18th century of the excess of cancer in chimney sweeps. It was lack of biological knowledge in the 19th that led to a prize essayist writing on the value and the fallacy of statistics to conclude, amongst other “absurd” associations, that 'it could be no more ridiculous for the strange who passed the night in the steerage of an emigrant ship to ascribe the typhus, which he there contracted, to the vermin with which bodies of the sick might be infected.' And coming to nearer times, in the 20th century there was no biological knowledge to support the evidence against rubella."

In short, the association we observe may be one new to science or medicine and we must not dismiss it too light-heartedly as just too odd. As Sherlock Holmes advised Dr. Watson, "when you have eliminated the impossible, whatever remains, however improbable, must be the truth."^[1]

Aha, so biologic plausibility is a function of the state of our current knowledge, today. By this litmus test, Marshall and Warren should have been laughed out of all funding agencies. Instead, they rewrote our understanding of what can live in the stomach, and how a microorganism can cause peptic ulcer disease and stomach cancer. And got themselves a cool Nobel to boot. So much for the ethics and finances of biologic plausibility informing meaningful research.

Now, on to the question of whether there exist relationships with such high biologic plausibility that they do not require irrefutable proof. Well, how about tobacco and its health effects? How about radiation exposure? Now, how about what we know today about the evolution of microbial resistance to antibiotics? Is it enough that the biologic plausibility for ill-effects of antibiotics in our food chain is strong? Can we now stop the madness? If my colleagues over at SBM are given to the same logic, they would say yes to this. However, extrapolating from this post about organic food production, I somehow think that they would not. So, I am guessing that, although they believe that lack of biologic plausibility should preclude attempts at study, they will nevertheless be reluctant to set a threshold for biologic plausibility that might obviate the need for further research. I am just guessing, and would love to hear what they really think.

And finally, what of the plausibility of benefit vs. that of harm? Should our bar for biologic plausibility for harm be lower than that for benefit? Well, the question really boils down to this: How many bodies do we need to see lying in the streets before we concede that there is a problem? My point is that we Americans have a hard time subscribing to the precautionary principle, applied generously in other parts of the world. If we were a tad less reckless with our need for irrefutable evidence, how many decades of equivocation about tobacco and cancer would we have avoided? How many lives might have been saved? Biologic plausibility for the connection was known even in the 1930s, yet it took another three decades for us to act. What are we obfuscating today that will come back to bite us (and our children) tomorrow? Could it be the cynical injection of doubt that our food production system is causing irreversible damage to us and life around us?

So, what I am saying is that biologic plausibility has several facets. We have to admit humbly that its assumption relies on our necessarily incomplete knowledge, and denying this may prevent us from awe-inspiring discoveries that will advance science in leaps. However, if we feel strongly about the need for it in order to justify our research allocation, some careful soul searching is in order for those thresholds of probability, especially of harm, where we may admit that science makes us sure enough, and, instead of awaiting perfect evidence, we must act promptly.      

    

Unidirectional skepticism

I thought I had coined that phrase. I have used it to refer to our tendency to examine ideas that are not aligned with our views of reality with less rigor than those that confirm it. It turns out that I am not the first to notice this phenomenon. In this Harper's article, Gary Greenberg uses the following quote from Mario Beauregard, a neuroscientist at the University of Montreal, writing about the seat of the "self":

The culture of popular science is one of unidirectional skepticism... It is skeptical of any idea that spirituality corresponds to something outside ourselves, but surprisingly gullible about any reductionist explanation for it.

Still, I think that my use of the term is somewhat broader, not simply alluding to the popular science, but also to professional science, and not only to the neurobiology of self. This idea goes hand in hand with the cognitive biases we are subject to.

This has got me thinking about a couple of things. Now, I am a big fan of neurobiology and the burgeoning field of research that it represents. We understand what goes on in the brain so much better now at the molecular level than we did, say, even 20 years ago. But what I am struck by is the comfort that this knowledge gives us about ideas of human behavior we had understood with lesser granularity in the prior era. For example, all parents have appreciated for millennia that teenage behavior is characterized by disinhibition of the higher self, if you will. Now we can say this in scientific lingo, that the area of the prefrontal cortex responsible for executive function lags in development behind other areas of the brain, as demonstrated in fMRI studies. Or how about the evidence from advanced neuroimaging that doing stuff that benefits our community lights up areas that correspond to happiness? We knew this before, only not in these words: we just had an idea that being a part of a community made us feel good. And those of us who subscribe to evolution can even come up with an explanation for why we should have developed this way.

Yet, aside from just gaining fascinating knowledge, we derive an odd sense of satisfaction from having our prior ideas, arrived through experience, observation and philosophical ruminations, confirmed by what we consider to be a more bona fide technique of science. The corollary to this is that, because we are so committed to the scientific method above all other possible ways of knowing, we may also be all too willing to discard rapidly closely-held notions that are not supported by science. But since science is ever-evolving and rather imprecise, in some areas more so than in others, are we always justified in such  prompt disposal of prior beliefs?

In the clinical world we often cite examples of when our intuition was wrong about some disease or another, and only after doing well controlled research studies were we able to arrive at the real understanding of the situation. But how often is it just our willingness to yield to what we think is scientifically sound at the moment? Is it possible that this unidirectional skepticism fueled by the cognitive bias in favor of new science, may not turn out to be the most efficient way to accumulate knowledge?

I do not know the answer to this question, as I am as guilty as my peers of falling in love with new scientific knowledge. But I also enjoy thinking about the nature of knowledge in general, and the many different ways of knowing. Are the materialism of science and other epistemologies mutually exclusive? I hope not.

Manipulating science

I picked up Devra Davis's The Secret History of the War on Cancer again, her sweeping indictment of the role politics has played in the science of epidemiology vis-a-vis health risks of smoking. The book is a great read, as it not only gives you the history of the fight, but also helps one understand the skepticism related to the methods we use to tie toxic exposures to adverse health outcomes.

One fascinating aspect of the story is the metastasis of public relations into the scientific discourse. On page 193, Davis quotes from Edward Bernays' 1928 book Propaganda:

"The conscious and intelligent manipulation of the organized habits and opinions of the masses is an important element of a Democratic society. Those who manipulate this unseen mechanism of society constitute an invisible government which is the true ruling power of our country.... We are governed, our minds are molded, our taste formed, our ideas suggested, largely by men we have never heard of. This is a logical result of the way in which our Democratic society is organized.... In almost every act of our daily lives, whether in this year of politics or business, in our social conduct or our ethical thinking, we are dominated by the relatively small number of persons... who understand the mental processes of the masses. It is they who pull the wires which control the public mind."

Note the split into the rulers and the sheep, the manipulation, the invisible governance, and again the manipulation. Sound familiar? For those of you not familiar with Bernays, he is generally considered to be the father of the public relations field. He was also the nephew of none other than Sigmund Freud, and, upon reading Freud's ruminations on the human psyche, adapted many of his ideas to manipulate consumers. In fact, it was Bernays who made it not only socially acceptable, but even desirable for women to smoke.

But I digress. What is the relevance of this quote from Bernays as used by Davis? Well, this was the strategy utilized to keep the mounting data on the dangers of tobacco from becoming credible. The popular tactic this strategy yielded was to have learned men with loud reputations refute the scientific basis of the reality of harm. Sound familiar?

I am not saying that the "learned men" of today are willfully obstructing the public from seeing the dirty underbelly of what medicine has become; I do not ascribe nefarious motives to them. However, given that we even have to ask the question "Should the public be told about the trouble with medical research?", it is an issue well worth discussing.

As you know, I am a great believer in transparency in both research and lay discourse. I believe that without a robust debate about topics that are far from clear, regardless of what the "learned men" will have us believe, we do harm to the society in several ways. First, lies damage relationships. Second, obfuscation creates distrust and sets up an adversarial atmosphere, which eventually leads to subversion of intentions, no matter how noble. Just look at the vaccine "debate" and how polarizing and destructive it is to both individual and public health. Third, it is manipulative and disrespectful, presuming lack of intellectual sophistication in our population. And while I am well aware that our literacy and numeracy are at an all time low, catering to the lowest common denominator is nothing but accepting and propagating the status quo. A better solution, though more time-consuming, is to start to fill these education gaps to improve our collective ability to engage in an edifying discussion.

To assert, as has been done here, that questioning any aspect of either the vaccine usefulness or the science of evolution for that matter equates to supporting the "anti-vaxers" or creationism, betrays a deep misunderstanding of how science is advanced. Indeed, it is by questioning certain aspects of science that do not seem to make sense that scientists both gain a better understanding of these ideas and at the same time demonstrate to the public how different our ways are from the entrenched dogma of organized religions.

I for one am proud of the process of science, and for this reason am working to bring what is under the hood out in the open. If we hide the messy uncertainties of our craft, we are doing a disservice to ourselves, our patients and the public at large. Worse yet, we are engaging in the manipulation, thus relegating scientific method to the wastebasket of PR and politics.        

 

Serendipity

I am always amazed how serendipitous life can be. Oddly I find that when I encounter a new word or concept, it all of a sudden begins to appear wherever I go. Or is it that when you open yourself to the Universe, it comes to your doorstep?  Cognitive bias, anyone? Imagine my surprise when after the exchanges of last week I came upon these three ideas in the current issue of Scientific American.

Consider this headline:

Climate Heretic: Judith Curry Turns on Her Colleagues

Why can't we have a civil conversation about climate?

And goes on to say in the fist paragraph:

For most of her career, Curry, who heads the School of Earth and Atmospheric Sciences at the Georgia Institute of Technology, has been known for her work on hurricanes, Arctic ice dynamics and other climate-related topics. But over the past year or so she has become better known for something that annoys, even infuriates, many of her scientific colleagues. Curry has been engaging actively with the climate change skeptic community, largely by participating on outsider blogs such as Climate Audit, the Air Vent and the Black­board. Along the way, she has come to question how climatologists react to those who question the science, no matter how well established it is. 

Now, I certainly do not claim to be the caliber of Judith Curry as a scientist, nor do I claim that the current conversation is nearly as crucial as climate change. Yet, some of the messages are instructive, not the least of which is the clarification of some of the subtle uncertainties in climate science.

I wish I had Scientific American's penchant for creating titillating headlines. How does this one grab you?

Hawking versus God: What Did the Physicist Really Say about the Deity?

The battle for eternity is fought on Larry King Live

And while I am profoundly interested in Hawking's ideas, here is what really spoke to me:

“The theories that Hawking and Mlodinow use to base their arguments on have as much empirical evidence as God,” wrote cosmologist Marcelo Gleiser on an NPR.org blog. Moreover, Gleiser added, “because we don’t have instruments capable of measuring all of nature, we cannot ever be certain that we have a final theory.”

This bears repeating: "Because we don't have instruments capable of measuring all of nature, we cannot ever be certain that we have a final theory." Isn't this beautiful? I love science!

And last, but not least, Christopher Hitchens provides this inimitably lucid observation:

“What can be asserted without evidence can also be dismissed without evidence.”

What a wonderful contradiction to hold, realizing that science is never complete, yet we must always go to the next level to gather the evidence. Science is about questions and discussion. Go forth, question and discuss.  

     

"The illusion of certainty"

This post was in part inspired by my recent exchange with e-Patient Dave. But really I have been thinking about this for quite some time now, and Dave provided the necessary nudge for me to write about it. Hence, his words, "the illusion of certainty".

As I like to tell my students, in science we are always on the way. We never really reach the end of the road because science is advanced through curiosity and constant questioning. For this reason, even when we think that we know something, think again. This reality is anathema not only to our model of medicine, but to our current model of society as well. What do I mean by that?

Well, let us reflect on our system of education. My son has a science workbook developed by what s considered to be a fairly progressive curriculum group. One of the early sections deals with the Linnaean classification of the animal kingdom (yes, the King Philip Came Over From Germany... well, not exactly sober). The worksheet asks the student to fill in blanks in a paragraph with the appropriate gradation of the classification. And believe it or not, the top of the page sports the full list of these strata. So, what is the message? The message, as I see it, is that this classification is so important and so solid that it requires rote memorization, as it is the only correct answer. Is this really true? Almost every day we hear about organisms requiring reclassification because we learn something new about their features that excludes them from their previous cubby. Bacteria have gained their own domain, which they did not have when I studies them in college. So, what is important here? What is important is why it is that we need these classifications, as well as how they are developed. My son is learning more from our explorations of what animals do not fit neatly into this system than from regurgitating the 7 vocabulary words. He is also learning that science is messy and the illusion of certainty that our educational systems foster are a house of cards. We could extend this argument to the way we test our children's knowledge in school as well, reducing what they learn to only one neatly packaged correct answer with all of the rest being wrong.

Now, let us wander into other parts of this American life. Take for instance the advertising industry. Cleverly they have devised market segmentation schemes that allow them to target products with very specific characteristics to a very specific population of consumers. (I wish we could be as good at doing this kind of a subgroup analysis in clinical sciences!) We are told with certainty that Charmin is better than Scotties, that Gif is more natural than Peter Pan, that buying Ford is more patriotic than buying a Toyota. And even though we are constantly faced with an absurd volume of choices, we are somehow under the impression that there is only one that is right for us.

My final example is in politics, which these days cannot be separated from religion in the US. Our politicians have realized that reducing every important issue to two ways of addressing it, and from those two ways choosing the one that can get them the most votes is indeed the way to get elected and re-elected, so that they can have a constant wedding banquet without ever getting bogged down in the hard work of marriage. This falsely reductionist and dichotomous approach has led us to a deeply divided nation, where, although we are more same than different, it is indeed these differences that are amplified to a deafening hysterical pitch. Yet, we are all secure in our certainty of being right.

I used these examples to illustrate our dependence on certainty in some major aspects of our every-day lives. This paternalism of certainty in education, politics and trusted brands likely extends to the sense of entitlement to certainty in healthcare as well. When I was in practice and on call for my group one Saturday morning, with a line out the door and the ancillary personnel about to go home for the day, I walked into one of the examining rooms to greet a woman in her 40s, well dressed and groomed, and not appearing in the least distressed. Immediately she let me know that she was being seen for a cough that was going on for a couple of days (no, she was not a smoker and did not have any chronic conditions), and wanted to nip it in the bud with a prescription for an antibiotic. After listening to her thoroughly and ascultating her lungs, I came to the conclusion that her ailment was overwhelmingly more likely to be viral than bacterial, and an antibiotic was not indicated. I shared my thinking and probability analysis with her, and following my thorough explanation of why she was going to leave my office empty-handed, she responded, "But my doctor always prescribes me an antibiotic!" I looked at her with incredulity, glanced at the growing line outside my door, and then at the clock, and, though I am not proud of it to this day, I pulled out my prescription pad. Her expectation of certainty did not leave any room for probability-based thinking, and her physician, and now I, were colluding in continuing this deception.

Today I look at medicine from a completely different perspective. As a clinical researcher, I am keenly aware of all the uncertainties inherent in what we do. The richness of research is in asking the question, and getting the answer, though it gets us published, is but a fleeting pleasure on the way to the next question. Science is the classic example of "there is no there there"; it is the road that is the gift. In the clinician's office, this attitude becomes difficult to hold together with the need for answers and certainty. Yet hold it we must. Yes, we have to give patients the best care possible based on what we know. But we cannot for a moment forget that all knowledge is evolving. This epiphany is an invitation to physicians actually to work with individual patients and their values and wishes to the extent possible. Dogmatic certainty, the alternative, is tantamount to worshipping at the altar of a false god.        

The reality of "science-based medicine"

Dr. Novella continues with his egregious oversimplification of the concept of science-based medicine here. I again felt compelled to respond. And given my previous difficulties with getting my responses accepted by the web site, I thought I'd post it here too. Tell me what you think.

Dear Dr. Novella,

Once again I have to agree with some of your premises, but disagree with your misguided leaps of illogic. I agree that if a modality has not been proven effective, the only way it should be left alone to be used by the public is if it has been shown to be safe. Alas, the risk-benefit equation is an individual choice, and we cannot impose our quantitative bottom line on it. Your assertion that scientific medicine is being eschewed because of acceptance of alternative modalities is as flawed as maintaining that a rain dance brings on rain. I know you said the relationship was complicated, but let's be honest: you think that CAM acceptance is killing allopathic medicine.

Now, let's get on to "proof" in science-based medicine. As you well know, while we do have evidence for efficacy and safety of some modalities, many are grandfathered without any science. Even those that are shown to have acceptable efficacy and safety profiles as mandated by the FDA, are arguably (and many do argue) not all that. There is an important concept in clinical science of heterogeneous response to treatment, HTE, which I have addressed extensively on my blog. I did not make it up, it is very real, and it is this phenomenon that makes it difficult to predict how an individual will respond to a particular intervention. This confounds much of what we think is God's own word on what is supposed to work in allopathic medicine.

Finally, do you really think that agents that are approved based on a 2-week prolongation of median survival in a desperately ill population of patients are used because of their supposed scientific merit? I would have to argue that there is a lot of subcortical emotional thinking that goes into these decisions. Can you really prove to me that a 2-week increase in median survival is not tantamount to placebo effect, aka type I error? Yet this is science-based. I think if I had a horrible disease, I might opt for acupuncture to make me feel better in the weeks I have left rather than rely on this kind of "science" to prolong my misery by 2 weeks.

Bottom line, we need to appreciate that none of the science is all that straightforward. Let us not dumb down the arguments and create false dichotomies. If we do, no one wins.

Does a flip of the switch cause the bulb to light?

"When men are most sure and arrogant they are commonly most mistaken, giving views to passion without that proper deliberation which alone can secure them from grossest absurdities"
                                                                                             --David Hume

The seemingly absurd question in the title of today's post becomes much less absurd when reading Rothman and Greenland's paper in the AJPH Supplement from 2005 found here. This paper, which is really an abridged version of a chapter in their textbook Modern Epidemiology, is creating a lot of angst among my MPH students this semester. As well it should: the authors make a compelling case for not trusting any assumptions whatsoever, and thus seem to imply that every time we think we know something, think again.

First, they talk about multicausality of any phenomenon in human biology. Pouring water on the dry ice of our cognitive abilities, they delight in vaporizing our most coveted ideas, that, for example, contributions of all causes to a condition cannot add up to more that 100%:

How can we attribute 75% of the cases to smoking and 67% to alcohol drinking among those who are exposed to both? We can because some cases are counted more than once. Smoking and alcohol interact in some cases of head and neck cancer, and these cases are attributable both to smoking and to alcohol drinking. One consequence of interaction is that we should not expect that the proportions of disease attributable to various component causes will sum to 100%.

Their discussion of causal inference is quotable for the context it provides for the various approaches of scientific inquiry. I feel that ideas in this entire section, with its apt name, are completely lost in our day-to-day scientific discourse:

Impossibility of Proof

Vigorous debate is a characteristic of modern scientific philosophy, no less in epidemiology than in other areas. Perhaps the most important common thread that emerges from the debated philosophies stems from 18th-century empiricist David Hume’s observation that proof is impossible in empirical science. This simple fact is especially important to epidemiologists, who often face the criticism that proof is impossible in epidemiology, with the implication that it is possible in other scientific disciplines. Such criticism may stem from a view that experiments are the definitive source of scientific knowledge. Such a view is mistaken on at least two counts. First, the nonexperimental nature of a science does not preclude impressive scientific discoveries; the myriad examples include plate tectonics, the evolution of species, planets orbiting other stars, and the effects of cigarette smoking on human health. Even when they are possible, experiments (including randomized trials) do not provide anything approaching proof, and in fact may be controversial, contradictory, or irreproducible. The cold-fusion debacle demonstrates well that neither physical nor experimental science is immune to such problems.

Some experimental scientists hold that epidemiologic relations are only suggestive, and believe that detailed laboratory study of mechanisms within single individuals can reveal cause–effect relations with certainty. This view overlooks the fact that all relations are suggestive in exactly the manner discussed by Hume: even the most careful and detailed mechanistic dissection of individual events cannot provide more than associations, albeit at a finer level. Laboratory studies often involve a degree of observer control that cannot be approached in epidemiology; it is only this control, not the level of observation, that can strengthen the inferences from laboratory studies. Furthermore, such control is no guarantee against error. All of the fruits of scientific work, in epidemiology or other disciplines, are at best only tentative formulations of a description of nature, even when the work itself is carried out without mistakes.

They further proceed to decimate Hill's criteria for confirming causality. In fact, in their usual scholarly fashion, having gone back to the original source, the authors convey Hill's own ambivalence:

As is evident, the standards of epidemiologic evidence offered by Hill are saddled with reservations and exceptions. Hill himself was ambivalent about the utility of these "viewpoints" (he did not use the word criteria in the paper). On the one hand, he asked, "In what circumstances can we pass from this observed association to a verdict of causation?" Yet despite speaking of verdicts on causation, he disagreed that any "hard-and-fast rules of evidence" existed by which to judge causation: This conclusion accords with the views of Hume, Popper, and others that causal inferences cannot attain the certainty of logical deductions.

Indeed, they even poke gaping holes in Bayesian thinking, which I myself am quite partial to. And incidentally, I am a big fan of Hill's "criteria", as well.

So, what is the point of assigning this paper to my students early in the course that addresses evidence generation and evaluation in policy? The paper serves as a reminder to address our assumptions systematically and regularly. By its nature, our science is imprecise. While most physical and biological processes are subject to mathematical rules, clinical sciences rely heavily on statistics, which is simply mathematics with uncertainty introduced into it. This uncertainty magnifies with every assumption that we make, and for this reason we need to be quite circumspect when insisting that we know something. In this respect, our science is more similar to civil prosecutions, where preponderance of evidence is enough for a conviction, rather that the standard of "beyond the shadow of a doubt" in criminal law.

Now, let's bring this back to the real world of clinical practice, where we do not have the luxury to ruminate on these uncertainties. This is the true intent of EBM: understand what the preponderance of evidence tells us about a phenomenon at the level of the pertinent population, but do not forget that these are mere approximations of what really happens, some closer to and some further from reality. So, while systematically observed measures of central tendency should give us comfort in the current gestalt at the bedside, the uncertainty around them is an invitation to tailor our approach to the individual before us.