A homeopath lectures scientists about anecdotal evidence

If there's one difference between so-called "complementary and alternative medicine" (CAM) and science-based medicine (SBM), it's the role of anecdotes in each. CAM and SBM each take a very different view of anecdotes. In SBM, anecdotes are relegated to a very low rung on the evidence ladder. They are a starting point in that, if well-documented enough and convincing enough, they can serve as the basis to suggest that clinical research or clinical trials might be indicated. Sometimes, in the case of rare diseases where numbers of patients are so small that randomized clinical trials are not feasible, they might even guide therapy. But, for the most part, they are not what we base therapeutic decisions on. In contrast, CAM practitioners view anecdotes very differently.

In fact, it can be argued that anecdotes, more than anything else, determine what CAM practitioners do. Indeed, what is the antivaccine movement, for example, based on other than anecdotes? What is homeopathy but anecdotal medicine. Think about it. Homeopathy is designed to treat symptoms, which, its practitioners say, can be relieved by ridiculously diluted solutions of substances or extracts that cause that symptom in healthy people. And how is it determined what symptoms are caused by what remedy? Homeopaths do something called "provings," which are basically anecdotes, in which healthy people are given the proposed remedy and then keep a detailed diary of what the the effects they feel (or think they feel) are, including dreams, thoughts, symptoms, and feelings. I've written about homeopathic provings before, and they can be a hoot. For instance, I've even seen an example of something claimed to be homeopathic antimatter.

No wonder it's no surprise that it was a homeopath named Heidi Stevenson, who wrote a piece called Anecdotal Evidence: The Basis of All Knowledge and decorated it with a rather bad bit of art by another homeopath named Gina Tyler. We've met Stevenson before a couple of years ago, when she launched a truly despicable tirade at Steve Barrett. Yes, in this piece, Stevenson argues that anecdotes trump science, so much so that doctors use anecdotes themselves. The post is a mixture of straw men and distortions, all mixed in with a profound misunderstanding of how doctors think and operate. She begins by lamenting how physicians will dismiss anecdotal evidence by pointing out that, yes, it is anecdotal evidence. Then she counters with what she considers to be examples of anecdotal evidence that we as physicians use all the time:

Doctors effectively and necessarily use anecdotal evidence every day. These bastions of evidence-based medicine actually base most of their practices on anecdotes. Sound crazy?

Consider:

• If you tell your doctor that a drug he's just given you is causing a terrible headache, the chances are that you'll be believed, and your treatment will be changed. He's basing that decision on the anecdotal evidence you've just given.
• Doctors tell each other stories of experiences during surgeries. If one doctor tries a new technique in surgery, it is almost never tested. Other doctors simply try it themselves if it sounds interesting. They're basing those decisions on nothing but anecdotal evidence.

These are basically straw men. First off, Stevenson confuses taking a good patient history with anecdotes. It is always a good idea to take a good patient history in order to identify what might be going on with the patient. SBM itself would point out that at least 75% of the process of coming to a diagnosis is history, and if the patient took something before experiencing the symptoms that brought him in to see the doctor, that's not anecdote. That's part of the patient history that guides later investigations and raises the suspicions of a drug reaction, every bit as much as a history of midepigastric pain that migrates to McBurney's point leads a surgeon to a suspicion that the diagnosis is acute appendicitis. Taking that bit of information, the doctor then assesses the likelihood that the drug the patient suspects is the culprit causing her symptoms based on--you guessed it!--science, epidemiology, and clinical studies. In other words, the patient's having taken a medication before having symptoms might or might not be related in a causative fashion. It will be science and further investigations that help the doctor figure that out, not that piece of the history alone.

As for Stevenson's mentioning how doctor's tell each other stories of experiences during surgeries, that is a very different thing. It's about craft, rather than science. In other words, it's not about which operation to do, but how to do the operation that has been chosen for the patient, hopefully based on science. In other words, surgeons are not making decisions on which operation to do based on anecdote. They're learning pointers from each other on how to do the operation. Surgery and procedure-based specialties are somewhat different from the rest of medicine in that the skill of the operator matters. A skillfully done procedure will in general produce usually better outcomes than a not-so-skillfully done procedure, particularly for complex operations. Experience matters, and SBM actually seeks to understand how much and in what circumstance experience matters the most.

Next, Stevenson completely misunderstands the role of anecdotes in medical evidence when she points out that anecdotes appear in medical journals all the time in the form of case reports:

In point of fact, anecdotal evidence is routinely provided in medical journals. They frequently produce articles of individual cases. If such anecdotal evidence weren't of value, then why are such stories printed? It's because they are evidence. Each case matters. Each case counts. The anecdotal evidence is of value.

No one ever said anecdotal evidence has no value. However, anecdotal evidence as published in medical journals is far different from the sorts of anecdotes that homeopaths mean. to be an anecdote in a medical journal, a case report must be well documented beginning to end, with all history, physical findings, laboratory and diagnostic tests, interventions, and responses to interventions, all recorded as objectively as possible. This is far different than a homeopath's anecdote that she tried this super-diluted remedy or that and the patient got better. Even then, case reports are considered among the lowest, least convincing forms of medical evidence. Case series are only marginally better. As we say in the biz, the plural of "anecdote" is not "data."

Not that that stops Stevenson from bloviating:

The blinded, placebo-controlled study attempts to eliminate anything the researchers deem to be irrelevant or likely to skew the results. That, though, narrows the focus to such a degree that it cannot account for all the variables that affect each individual person who might be subjected to the product, usually a drug, under investigation.

Population-based studies provide information about what's true across enormous groups of people. They provide averages and ranges. However, they tell us nothing whatsoever about each individual.

Ultimately, the only evidence that truly matters is anecdotal: what a treatment does to the individual. It matters not if some nonexistent average person might benefit from it. That has little bearing on the individual's reaction. It matters not if a large proportion of people tolerate a treatment well, if the individual is made ill by it.

The only medical evidence that truly counts for each person is anecdotal.

Well, yes and no. It's not as though SBM doesn't know the weaknesses of applying population-based data to individuals. That's why it's trying to identify predictive biomarkers that will allow physicians to predict better who will and will not respond to various treatments. Moreover, there is an unspoken assumption behind Stevenson broadside against using population-based data and average responses to guide the treatment of individuals. That assumption is that somehow, through intuition, woo, or outright quackery, homeopaths and other CAM practitioners can somehow do better at "individualizing" treatments than science-based practitioners. Whatever the flaws and difficulties there are in applying clinical trial data to individual patients, if I were going to put my money on who does better individualizing treatments, I'd put it on science-based practitioners, as the "individualization" of medical treatments in CAM is nothing more than "making it up as you go along."

So, when Stevenson writes:

The nature of life goes beyond chemistry and physics. Life exists because of something that doesn't exist in anything inanimate: the struggle to survive and reproduce. Life has volition, and that makes it complex beyond the ability of any science experiment to predict with surety what will happen when any treatment is given to any individual. Yes, science can predict fairly accurately what will happen, on average, in large groups--but not what happens with each person within that group. Yet, medicine must treat the individual. That is obvious.

And that is why so-called evidence-based medicine fails. It ignores the most salient point of all: No person is average.

Yes, but there is an average response that can guide therapy and allow estimates of the likelihood that a treatment should work.

I have a hard time not retorting that you can't just make it up as you go along, either. Whatever the difficulties involved in trying to take the results of a randomized clinical trial and apply it to patients, at least RCTs provide some guidance based on science. Homepaths provide guidance based on sympathetic magic.

Stevenson goes on and on about how, throughout history, most knowledge came through anecdotal evidence. That may or may not be true, but it utterly misses the point. Anecdotal evidence, as has been discussed here so many times, is capable of seriously misleading patient and practitioner. Regression to the mean, confirmation bias, placebo effects, and a large number of other potential confounders can easily mislead. Indeed, the very reason for a move away from relying on anecdotes and towards science was driven by one simple fact: Science is more reliable. It delivers the goods. It produces better results. whatever progress was made on the strength of anecdotes and inference, progress was so much faster using science.

So why, according to Stevenson, is science ascendant and anecdotal evidence relegated to the lowest level? I think you know why. Here it comes:

Why does this bull-headed blindness about science exist? Why is it so often used to bully people, especially those who espouse medical views that run counter to what's accepted by modern mainstream medicine? The reason is really quite simple and crass: money.

Science has become the tool of corporations to get what they want. The more money they have, the better they're able to control the outcomes. The better they control the outcomes, the more readily they can force their products on the masses of people. This alone should clarify that science is not the perfect tool so often presented.

You knew it was coming, didn't you? Certainly, I did.

Here's the issue I have, though. Acknowledging that science has its shortcomings as practiced in the real world, no one, least of all scientists, claims that it is the "perfect tool." It is, however, the best currently existing tool. Certainly, it's better than bogus "individualization" of treatments that all involved substances diluted to the point of not being present in the homeopathic solution anymore. So how is it that a small, poorly controlled "proving" or a homeopath's "making it up as she goes along"-style "individualization" of treatment is superior to large clinical trials? Only in a homeopath's mind, apparently.

Besides, it rather galls me to be lectured about what constitutes good science and evidence by someone who still believes in miasmas and, well, homeopathy.