A belated response to a challenge: Differentiating woo from non-woo in medicine

Over a week ago, fellow ScienceBlogger revere fired a bit of a pot shot across my bow regarding my bow regarding a study regarding, of all things, chicken soup. Initially, it was at a bad time, when I had other things to do, having just labored mightily to produce the latest Hitler Zombie extravaganza, after which I had to lay low blogging for a while because of obligations midweek. When those obligations were over, then blogging about the Tripoli Six took precedence, as did this week's Your Friday Dose of Woo (which, by the way, is still overrun by the tinfoil hat brigade). And then I just forgot about it until last night.

But never let it be said that I don't respond to such a tweak when I promise to (at least when it comes from one of my fellow ScienceBloggers, anyway). It's time to revisit this issue. Unfortunately, when I went back to revisit revere's article, I found that the first part of it was, I regret to have to say, a straw men that he happily tore down with respect to the dreaded medical "skeptic." Apparently he misinterprets my advocacy of evidence-based medicine as dogmatism, close-mindedness, and inflexibility. Well, you know what I always say: I'm for being open-minded as long as it's not so open-minded that my brain falls out (regardless of how much the Hitler Zombie might like to see that).

The study in question was published six years ago in the journal Chest by Dr.Stephen Rennard at the University of Nebraska Medical Center. In the study, Dr. Rennard wanted to see if there was anything in chicken soup that affected the activity of human neutrophils, a type of white blood cell whose activity is critical for an inflammatory response to many microorganisms. He found that, in cell culture at least, chicken soup inhibited the motility of neutrophils, suggesting a mechanism by which chicken soup might reduce symptoms in colds and the flu. Let's deal with revere's first volley first:

Chicken soup for flu and colds has to qualify as alternative medicine, I would think. The question I have is this. What would allow it to escape the slings and arrows of Orac's mighty pen? In perusing his posts on the subject he seems to have two complaints about alternative remedies, which he tends to conflate, often sliding from one to the other. The first is that the treatment has not been shown effective via a Randomized Clinical Trial or other acceptable study design to work. There are some variations on this theme when trials have actually been done. Sometimes critics (not Orac as far as I know) will mistakenly say that the lack of statistical significance in a trial means any effects seen are due to chance ("small numbers" are a variation of this). This, of course, is incorrect but here isn't the place to go into why. Another response is that the trial is flawed (which many are). So that's the first complaint: no convincing demonstration of efficacy. Unfortunately much the same can be said for a lot of what we conventional practitioners do. I won't single out surgeons, but they are hardly free of this millstone (lobotomy, anyone?). OK, I guess I did single out surgeons. Orac is a surgeon.

I guess my first question is why would revere consider chicken soup for flu or colds "alternative" medicine? I don't think that I do for the simple reason that warm, soothing liquids have been used for colds and the flu since time immemorial, for at least the purpose of soothing a cough-roughened throat. My second critique is that revere uses a strawman so dry that it threatens to spontaneously ignite. (Yes, I know I've used that one before, but I like it; so I used it again.) The strawman is that my position with regard to alternative medicine is that I dismiss them because "the treatment has not been shown effective via a Randomized Clinical Trial or other acceptable study design to work." Evidence-based medicine is all about using the best available evidence to guide treatment decisions, as I have explained before in my usual long-winded manner. When randomized double-blind clinical trials are available, their results should be considered the gold standard, unless there are methodological flaws severe enough to invalidate them or cast grave doubts on their findings. Indeed, most problems in "conventional" medicine do not have good RCTs to guide physicians, and many questions don't even have good data from an "acceptable study design" to guide treatment choices. This is certainly true in many areas of surgery, where in most cases it is ethically dubious to do sham surgery as a placebo and it is virtually impossible to blind surgeons involved in a trial as to what treatment group a patient is in, for obvious reasons. I think it's useful to revisit what I said about this very issue a few months ago:

Let's take an acutely hemorrhaging patient. It doesn't take an RCT to tell that taking the patient to the operating room to stop the hemorrhage will save the patient's life. Or consider appendicitis. It doesn't take an RCT to determine that removing the appendix in the case of acute appendicitis will result in far more people surviving than not operating. Indeed many surgical problems cannot easily be directly addressed by RCTs, at least not entirely ethically. For example, consider a test of doing a procedure versus not doing a procedure for a certain condition. It's impossible to blind the patients as to which group they are in, as one group would be getting an operation and the other wouldn't be. The only way you could truly blind the groups would be to do a sham operation on the control group--which leads to all sorts of ethical difficulties. Even then, the physicians would not be blinded to the treatment groups; they would know which patients got a real operation and which got a sham operation. (Even so, some groups have managed to do such trials.) Finally, there is the issue of operator skill; some surgeons are just better at some operations than others. If you're doing a randomized trial to compare one operation against another for the treatment of a disease, you can never be 100% sure that differences observed were due to differences in the operation or technical differences in the skills of the surgeons. That's why large numbers are often needed. The fact is, in surgery, much of the evidence upon which we base our clinical decisions does not come from classical double-blinded randomized studies.

And the same is true of a lot of questions in medicine. I'm at least glad revere didn't claim that I would use lack of statistical significance as an indication that the results of a negative trial were from chance alone. I might have been a bit insulted if he had. In any case, I have to suspect that revere in actuality doesn't read my blog regularly. Otherwise, he would know what my true position is in such matters. Fortunately, my apparently enormous surgeon's ego protects me from being hurt by this slight.

Next, revere appears to accuse me of "magical thinking":

And what if we have showings of efficacy without any scientific theory? You want an example? We still don't know how asbestos causes cancer, but on the basis of epidemiological studies we have no doubt that it does. A variation of this is that the proposed explanation is "magical thinking." That's perfectly fair as a criticism. Unfortunately, proposed counter-explanations are also vulnerable. If one takes "the placebo effect" to mean an interaction between the mind and the body that produces an effect, that's at the level of magical thinking in my book. If you provide a neurophysiological explanation that's fine (assuming you can give one). But then the remedy is actually having a physical effect so what's the problem? It's like a psychoactive agent. Counter-irritation as an assumed mechaism is another irritating wave of the hand that seems quite magical to me. What exactly is the mechanism of counter-irritation, say, for acupuncture anesthesia? If you give me one then aren't you giving me an explanation for how acupuncture really does work? If you can't, why isn't that a "magical explanation"? Or is the complaint about some alternative therapies that they might work but have the wrong reason? In which case much of conventional medicine is probably also in big trouble.

Not exactly.

Let me lay it on the line here: I am not so much disturbed by treatments that are efficacious for which our understanding of the mechanism is incomplete or lacking as I am about the attribution of mystical B.S. to such treatments or to making claims for such treatments when there really isn't compelling evidence that they "work.". As revere points out, there are a fair number of treatments in conventional medicine for which the physiological mechanism is poorly understood or unknown. However, unlike the case for alternative medicine, we actually do have compelling evidence to support the efficacy of such treatments, and it is not true that possible explanations for their efficacy fall outside the realm of scientific plausibility based on our current understanding of human biology. For the example of asbestos, there are any number of plausible biological mechanisms based on our current understanding of chemical carcinogenesis by which asbestos particles could cause lung cancer. We just don't yet know which one(s) to attribute its carcinogenic affect to. Revere mistakenly conflates such scientific uncertainty with my overall message about the implausibility or virtual impossibility of much of alternative medicine. What I've been talking about is alternative medicine therapies where the mechanisms by which they are said to work are highly implausible physically, biologically, chemically, based on what we know. Good examples of this include homeopathy, which, if true, would violate our present understanding of physics, chemistry, and pharmacology to the point that many well-accepted theories would have to be overthrown, and something like Reiki therapy, for which no "life force" (qi) has ever been detectable or changes in such a force noted in response to the ministrations of a Reiki master.

Even for twaddle like homeopathy or Reiki, I would still be willing to consider throwing out what we know and massively revising it in the light of new knowledge if compelling objective evidence for efficacy were presented for the real efficacy of either homeopathy, Reiki therapy, or any other alternative medicine to which an amazingly implausible mechanism of action is attributed. None ever has, as of yet. As Dr. Mark Crislip of QuackCast (someone who is even more--shall we say?--insolent than Orac is, but nowhere near as respectfully so, and that's saying something) put it in his podcasts on the evidence regarding homeopathy and acupuncture, as the quality and rigor of the clinical trial increases, the "effects" attributed to homeopathy and acupuncture tend to decrease until they are no better than placebo in the best clinical trials with the largest number of patients. Even with regards to herbal medicines, which, after all, could have efficacy that does not depend upon any sort of scientifically implausible mechanisms given that many of our presently used drugs were derived from plants, we still need compelling evidence and plausible potential mechanisms to base a study on. Abel summed it up well:

The basis for generating a hypothesis to test whether an alternative remedy has biological efficacy must have some scientific basis that is sound. I've heard herbalists tell me that "whole herbs" are superior to purified compounds because of synergy among components or because one herbal component blocks a side effect of an active component. These are both reasonable scenarios, but there is a great paucity of evidence in the literature to support such suppositions...

However, if an herbalist tells me that a whole herb works better than a pure compounds because the herb is "energized" or contains a "life force" that is extracted away by laboratory purification, I call bullshit unless there is a method to quantify these forces and demonstrate their causality with biological activity.

Indeed. My battle cry as far as these so-called "alternative" therapies is, "Show me compelling evidence that they work if you expect me also to radically alter my view of how human biology works!" My corrollary to this battle cry would also be that, if you want to get grant funding--taxpayer money-- to study your woo, you should be able to show at least one of two things: biological plausibility or compelling evidence that the woo in fact does what you claim that it does, preferably (but not necessarily) both.

As for placebos, remember that the placebo effect has never, to my knowledge, been demonstrated to have an objectively measurable therapeutic effect against the really nasty diseases. For example, no placebo will shrink a tumor or prolong life in a patient with cancer. Even in less life-threatening conditions, the placebo effect produces no objective alterations. For example, placebos do not alter blood pressure. Whatever "neurophysiological" effect that may be attributable to placebos only seems to work when it comes to subjective symptoms with a significant psychological component, like pain, or diseases with a significant psychological overlay. And, even though it can't be said that we fully understand the placebo effect, it does not require the invocation of biologically or physically implausible mechanisms to explain, as does homeopathy. Either way, the evidence for the existence of the placebo effect in conditions having to do with pain or subjective discomfort or suffering is strong, which is reason enough to accept its existence while studying its mechanism. In the case of revere's cited study, if it could be shown that whatever the active component or components in the chicken soup actually did to human neutrophils in vivo what they appear to have done in this study in vitro, I'd say, rock on, dudes! And then I'd want to know what the active component(s) are, if it could be figured out, a question whose answer also does not depend on any sort of scientifically implausible or near impossible bogus "mechanism."

Next, Janet dealt with revere's epistemological question, specifically the Demarcation Problem, or how to separate science from pseudoscience, very well by pointing out that it is the approach that largely helps to answer this question when we are out at the fringes. In other words, scientists seek ways by which their hypotheses can be falsified in order to test them, whereas pseudoscientists, like alties, look for ways to confirm their hypotheses or don't bother even to test them at all. This partially explains why quackery such as homeopathy, live cell analysis, and Reiki has persisted so long. (In my opinion, another part of the explanation is that people like their woo.) As far as I'm concerned, and as I have said over and over again since the very beginning and even before, there is no such thing as alternative medicine. In my book, the approach to evaluating the therapeutic value claimed for a treatment should be the same, regardless of whether the treatment is considered "alternative" or not. The gold standard would be an RCT, but if it's not possible to do an RCT then the best existing lesser evidence must be accepted and evaluated as the basis for using the treatment until better evidence is developed, lest we suffer "evidence-based paralysis" or "RCTomyopia." The patient's disease won't wait, in other words, and we as physicians have to do the best we can and use the best scientific evidence available to decide upon the best course of treatment for any given patient. Sometimes that means, in cases where evidence is very sketchy or conflicting, to use primarily anecdotal or personal experience as a guide, even though that's far from ideal.

None of this leaves out physician humility, revere's crack about surgeons notwithstanding. In fact, here's where real humility comes in and why this demarcation problem is somewhat different in medicine than it is in the rest of science. The difference is, boiled down to its essence, patients, and if anything distinguishes "conventional" medicine from "alternative" medicine, even when evidence may be scant for some "conventional" interventions, it is the humility in presenting his treatments to the patient. Yes, humility. However much people joke about the "God complex" of doctors, this attitude is nothing compared to the attitude among true woos. Despite their seeming otherwise, quacks and pseudoscientists are anything but humble and honest about their limitations. In non-medical areas, pseudoscientists often claim that scientists are wrong, that they know nothing, and that their woo will replace current science, all on minimal evidence. While such hubris usually doesn't do that much harm, other than to the pseudoscientist's reputation, in medicine it is disastrous to patients. Quacks claim to have the cure for cancer or even the cure for all diseases. They often disparage conventional medicine, claiming that their woo is better and can cure the patient without the risk of all those potentially nasty complications or side effects that we conventional doctors have to acknowledge and warn the patient about. Patients, afraid, desperate, and all too often distrustful of "conventional" medicine because of its known shortcomings, too often fall for the blandishment of quacks and suffer, often from a delay in receiving effective therapy until it is too late, accompanied by a draining of their bank accounts. Dr. RW described this difference well while asking When Woo Overlaps Mainstream Medicine, Can Patients Sort it Out?:

Emily correctly points out that conventional medical treatments aren't always evidence-based. She asks "Is that not, in effect, a form of woo, too?" The breach between evidence and practice to which she refers, let's call it the quality chasm, while every bit as serious as woo, is not in fact woo in most cases. The quality chasm is not a result of outlandish or implausible claims, (e.g. that water has memory) and that's what distinguishes it from woo. The quality chasm results from an extremely complex interplay of cognitive and system barriers to the consistent application of best evidence in practice. Mainstream medicine recognizes the need for widespread system change and promotion of evidence based medicine. Although there's no simple fix to the quality chasm, many in mainstream medicine are trying, which is in ironic contrast to the fact that mainstream medicine also increasingly promotes woo. For the mainstream, especially academic medicine, to promote woo is to engage in unethical scientific pretense and active deception of patients.

This is quite true, and this is one other huge area of "demarcation" between woo and non-woo in medicine. And Dr. RW is also very correct in pointing out another important aspect of differentiating woo from non-woo in medicine: ethics. Dr. Maurice Bernstein also points out that it is up to the physician to try explain to the patient what is and is not evidence-based about his recommendations because it is his fiduciary and ethical duty to the patient:

What has all of this to do with medical ethics? It has to do with the physician's duty to be trustworthy to the patient and to aim to do good and not produce harm to the patient. Informing the patient, unless some patient rejects being informed, is also duty of the physician and is part of the consent to treatment process. It should be a standard of medical practice that the patient understand what portions of the physician's decisions regarding diagnosis, prognosis or treatment is based on reliable evidence and which is based on hearsay or simply statistically not verified experience. But do most physicians have the time or skill to explain to the patient on what basis their decisions were made? Further, do most physicians even know which of the "facts" they are using have not been rigorously proven? Unfortunately, even with the best of physician attention to educate themselves on the evidence and to explain this to the patients, there is much in medical care that will remain "old tales" either because there is not sufficient interest to provide the time and expense to do scientific studies or realistically because some studies just can't be accomplished because of technical or ethical issues.

The real Demarcation Problem between quackery and evidence-based medicine, I would argue, is the willingness to be always striving to test our therapies against new therapies in the search for better ways to care for our patients versus stagnation, dogma, and unquestioning faith in a treatment on the basis of little or no evidence. Scientific medicine embraces the former; alternative medicine embraces the latter. Indeed, that is the very reason that so much of alternative medicine is claimed to be based on "ancient" knowledge or concepts that were considered scientifically state-of-the art hundreds of years ago. As I like to say, if you want to practice 18th century medicine, be prepared for 18th century results and mortality. Scientific medicine may have serious problems and shortcomings in too many of its treatments, but, whatever its faults, it is always striving to eliminate those shortcomings. Alternative medicine feels no such imperative to test what does and does not work and reject what does not work, and, indeed, in most cases, alties have little or no interest in testing their favorite woo scientifically. For this reason, arguably no "alternative" therapy is ever discarded as useless; indeed, laetrile, which was shown to be useless treating cancer in several large clinical trials 25 years ago, is still sold to desperate cancer patients. In fact, the authors of the very study that revere sites show quite well this demarcation between woo and non-woo quite well. Had Dr. Rennard been an altie, he would simply have accepted the ancient wisdom of Egyptian Jewish physician and philosopher, Moshe ben Maimonides (not to mention that of his wife and her mother) and believed that chicken soup was good for cold symptoms as a fact. Instead he wanted to know if chicken soup actually did anything that was objectively measurable to human cells. He formed a hypothesis based on that question and then went about trying to test it, ultimately submitting his results to peer review and publishing it for doctors and scientists to evaluate.

Thus, the difference between woo and non-woo in medicine could indeed be argued to be, at least in part, a matter of humility. For all the perceived arrogance of conventional physicians, think about what's more arrogant: to be willing to test one's ware's and discard what doesn't work (even if it may take longer than we would like to eliminate less effective therapies) or to keep doing the same thing decade after decade (or even century after century), never testing one's beliefs in one's treatments against cold, hard reality, and never really wanting to.

I know which one I'd pick as the more humble attitude.