When I first started blogging, I liked to refer to myself as a booster of evidence-based medicine (EBM). These days, I'm not nearly as likely to refer to myself this way. It's not because I've become a woo-meister of course. Even a cursory reading of this blog would show that that is most definitely not the case.
So what's changed? Basically, I've come to the realization that EBM is an imperfect tool. Don't get me wrong, EBM goes a long way towards systematizing how we approach clinical data, but there's one huge flaw in it. (I can just see a quack somewhere quote-mining that sentence: "Orac says EBM has a huge flaw!") That flaw is that it devalues basic science. In any hierarchy of evidence in the commonly used EBM systems, at the very top is, as they should be, are randomized, double-blind studies. Such studies control for the most potentially confounding variables and tries to rigorously isolate the difference between experimental groups to just the study drug or treatment. Thus, level 1a evidence is evidence from multiple randomized controlled trials with homogeneity of the trials. From there, the strength of studies falls by study type all the way down to the least powerful forms of evidence, such as case series. At the very bottom is the following:
Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles"
In other words, EBM devalues basic science.
So what, you ask? Why is that a problem? After all, why should basic science matter? Here's why. In the absence of a basic scientific basis to think that a treatment is at least plausible on a physiological basis, all sorts of mischief happens. That mischief takes the form of "complementary and alternative medicine," much of which has zero plausibility from a scientific viewpoint. I'm not just talking about mild improbability, either. I'm talking about relying on first principles that flout everything we know about physics and chemistry, that for them to be true would require the overthrow of major, well-established theories. Think homeopathy, for instance. For homeopathy to be true, not only would our entire understanding of the chemistry and physics of water have to be seriously in error, but so would our understanding of how cells respond to chemical compounds. Think reiki. For reiki to be true, not only would there have to be a life "energy" (which, by the way, no scientist has yet been able to detect or characterize), but "healers" would have to be able to manipulate it. And not just manipulate it either, they'd have to be able to make it do their bidding to heal. Or think acupuncture. For acupuncture's mechanism to be true, much of what we know about anatomy and physiology would have to be wrong. We know, for instance, that there are no such things as meridians, the mythical pathways along the body through which that magical, mystical life energy known as qi is claimed to flow and into which needles need to be inserted in order to "unblock" or "redirect" that flow in order to heal. True, it's marginally possible that acupuncture could "work" by another mechanism consistent with modern science (endorphin or opioid release, for example), but classical acupuncture is clearly pseudoscience. On the basis of enormous amounts of data gathered over literally centuries, we can say with confidence that all of the above CAM therapies are incredibly implausible, bordering on being downright impossible.
None of this matters in the EBM paradigm as formalized by the Cochrane Collaboration and others. First principles based on basic science, no matter how well supported that basic science is, fall under level 5 (the lowest level) evidence, and for purposes of EBM as prior probability doesn't matter. Even a poorly designed, badly carried out case series ranks higher on the scale of evidence than hundreds of years of science saying that homeopathy can't work. Worse, as John Ioannidis has shown us, clinical trials are so prone to producing "wrong" results more than the 5% of the time expected by random chance alone using the conventional 95% confidence interval, with the probability of such a result increasing with the implausibility of the treatment being tested. In other words, the less likely the prior probability of a positive result based on scientific principles, the less likely the positive result obtained is a "true positive" or that it can be explained by a real therapeutic effect. When the result is equivocal or only weakly "positive" (as virtually all of the few well designed studies of CAM modalities that report "positive" results are), it's even less likely that the positive result is due to a real effect.
What this blind spot in the EBM paradigm leads to is the appearance of enough "positive" trials of even incredibly improbable CAM modalities such as homeopathy when on the basis of prior probability alone it is not unreasonable from a scientific (not to mention ethical) viewpoint to reject these improbable remedies a priori while awaiting evidence that approaches being as compelling as the scientific evidence that says they can't work. That's a lot of evidence. It's also why I've started referring more to "science-based" medicine rather than EBM, or "science- and evidence-based" medicine (SEBM). To me SEBM takes into account both clinical trials, other forms of evidence, and, most of all, science in the form of estimating plausibility and prior probability. Rarely do I see anyone writing about such topics in literature directed at a lay audience. (Actually, come to think of it, I rarely see articles directed at professionals discussing such matters.)
That's why I was pleased to see an article in the New York Times yesterday that does about the best job I've seen at actually discussing the issue of how to rank the "believability" of clinical trials. Have you ever heard of the Avalon effect? The article uses it to demonstrate how three large, well-designed randomized trials that failed to find that beta carotene protected against cancer were trumped in the public consciousness by lots of lesser quality studies. As Frankie Avalon, while shilling for supplement manufacturers put it:
There were laboratory studies showing how beta carotene would work. There were animal studies confirming that it was protective against cancer. There were observational studies showing that the more fruit and vegetables people ate, the lower their cancer risk. So convinced were some scientists that they themselves were taking beta carotene supplements.
Then came three large, rigorous clinical trials that randomly assigned people to take beta carotene pills or a placebo. And the beta carotene hypothesis crumbled. The trials concluded that not only did beta carotene fail to protect against cancer and heart disease, but it might increase the risk of developing cancer.
It was "the biggest disappointment of my career," said one of the study researchers, Dr. Charles Hennekens, then at Brigham and Women's Hospital.
But Frankie Avalon, a '50s singer and actor turned supplement marketer, had another view. When the bad news was released, he appeared in an infomercial. On one side of him was a huge stack of papers. At his other side were a few lonely pages. What are you going to believe, he asked, all these studies saying beta carotene works or these saying it doesn't?
That, of course, is the question about medical evidence. What are you going to believe, and why? Why should a few clinical trials trump dozens of studies involving laboratory tests, animal studies and observations of human populations
Why indeed. Too bad the average lay person doesn't understand that quality matters far more than quantity when it comes to deciding which clinical trials to believe. It is true that two or three well-designed studies do trump hundreds of weaker studies.
What follows is a fairly standard and well-described discourse on the rationale for what makes a medical study convincing. The importance of comparing populations as similar as possible except for the study intervention, randomization, controlling for confounding variables, and as large a sample size as possible. What shocked me is that Bayes' theory was next described:
The third principle, Dr. Goodman says, "is often off the radar of even many scientists." But it can be a deciding factor in whether a result can be believed. It's a principle that comes from statistics, called Bayes' theorem. As Dr. Goodman explains it,
"What is the strength of all the supporting evidence separate from the study at hand?"
A clinical trial that randomly assigns groups to an intervention, like beta carotene or a placebo, Dr. Goodman notes, "is typically at the top of a pyramid of research." Large and definitive clinical trials can be hugely expensive and take years, so they usually are undertaken only after a large body of evidence indicates that a claim is plausible enough to be worth the investment. Supporting evidence can include laboratory studies indicating a biological reason for the effect, animal studies, observational studies of human populations and even other clinical trials.
That's science-based medicine he's talking about. Normally, the way in which treatments are developed and found to be effective in humans begins with a clinical observation or a scientific finding in the laboratory. It is then studied further using in vitro models, animal models, and all sorts of other forms of evidence. All these studies, known as "pre-clinical" studies, form the supporting basis that justifies small pilot studies in humans and ultimately larger randomized clinical trials. Dr. Goodman puts it very well, when he says that the guiding principle in interpreting clinical trials is "that "things that have a good reason to be true and that have good supporting evidence are likely to be true."
The article then does something I've never seen in such an article before in a major newspaper. It gives an example:
To teach students the power of that reasoning, Dr. Goodman shows them a paper on outcomes of patients in an intensive care unit, with every mention of the intervention blacked out. The study showed that the intervention helped, but that the result was barely statistically significant, just beyond the threshold of chance.
He asks the students to raise their hands if they believe the result. Most indicate that they do. Then Dr. Goodman reveals that the intervention was prayer for the patient by others. Most of the hands go down.
The reason for the skepticism, Dr. Goodman says, is not that the students are enemies of religion. It is that there is no plausible scientific explanation of why prayer should have that effect. When no such explanation or evidence exists, the bar is higher. It takes more clinical trial evidence to make a result credible.
And that, my friends, is what science-based medicine is: EBM with science taken into account. It takes into account all the other preclinical evidence and evidence from other sources, such as basic science, that bear on the believability and plausibility of a therapy under study. That's all. It really is that simple.
Nor is it being close-minded, either, and rejecting out of hand the possibility that a therapy might work. It is simply weighing all the evidence, rather than pretending that any therapy under study is as likely to work as any other. It is using what we already know to decide where to set the bar for evidence. For a therapy that is highly plausible, the bar is relatively low: A couple of convincing randomized trials might be enough. For something as improbable as homeopathy, whose principles go against so much of what is known about chemistry and physics, the bar would be much, much higher. There would have to be multiple well-designed randomized clinical trials with very clear, compelling, and undeniable results to make it reasonable to start to conclude that there may be something wrong with our understanding of chemistry and physics rather than something wrong with the clinical trials. Of course, in the case of homeopathy there are no such trials. The "positive" ones are almost invariably small and/or poorly designed, and even the occasional randomized trial that appears "positive" generally demonstrates an "effect" that is barely above statistical significance. Meanwhile, the better designed and more rigorous a clinical trial of homeopathy is, the less likely it is to show a "positive" result.
Of course, that's the elephant in the room in discussions like these: How CAM research utterly ignores the issue of prior probability. True, Gina Kolata, the writer of the Times article, discusses prior probability and even interviews Dr. Goodman. In that, she goes further than virtually any other science or medical writer I've seen before in understanding how EBM should be applied. However, there's still that damned elephant that can't be avoided in CAM studies. It's an obvious connection, but she doesn't make it, and rarely does anyone else, it seems. In fact, we tend to pretend that it isn't there. Sometimes we bump into it but pretend it's something else. But it won't go away, and it's the reason that the vast majority of CAM research done under the auspices of the National Center for Complementary and Alternative Medicine and promoted by wealthy private foundations such as the Bravewell Collaborative is so often a huge waste of resources and an abuse of science.
Too bad someone didn't tell another Times writer, William J. Broad, who clearly didn't get the idea. On the very same day Kolata's article appeared, he published an article called Applying Science to Alternative Medicine. Although there are some reasonable bits, there's a lot of the double-talk used to justify wasting resources studying highly implausible CAM treatments:
Dr. Briggs said such investments would be likely to pay off in the future by documenting real benefits from at least some of the unorthodox treatments. "I believe that as the sensitivities of our measures improve, we'll do a better job at detecting these modest but important effects" for disease prevention and healing, she said.
If the effects are so modest, why is it justified spending so much money and in the process twisting the very process of scientific medicine, I ask? On the other hand, tight funding may eventually bring some sense to the endeavor of studying CAM treatments by forcing a more rigorous form of triage:
An open question is how far the new wave will go. The high costs of good clinical trials, which can run to millions of dollars, means relatively few are done in the field of alternative therapies and relatively few of the extravagant claims are closely examined.
"In tight funding times, that's going to get worse," said Dr. Khalsa of Harvard, who is doing a clinical trial on whether yoga can fight insomnia. "It's a big problem. These grants are still very hard to get and the emphasis is still on conventional medicine, on the magic pill or procedure that's going to take away all these diseases."
I hate that "magic pill" straw man. If there is a "magic pill" in scientific medicine, its effects are not magic; they're documented by rigorous science and clinical trials. It's CAM that is looking for magic in the form of that "magic supplement" or extravagant magical thinking in the form of modalities like homeopathy, craniosacral therapy, much of chiropractic, "detoxification," and so many others. If there's a silver lining that might come out the current dire NIH funding situation, it's that it might force some rigor in our thinking about CAM and in how we decide what CAM modalities to study.
I guy can dream, can't he? Or, if you will, think magically.
I've got your back on the Broad article
http://scienceblogs.com/denialism/2008/09/the_times_doesnt_know_bayes.p…
Hmm. Not sure I agree with you here. The yoga study sounds potentially productive, and if it is effective it would be better for a lot of people than sleeping pills. Are you counting the yoga study in with the nonsense? Is there not a focus on pills rather than alternatives that are not potentially profit-generating for corporations in this area? I don't know much about it, but it seems like she has a point at some level.
Good post, anyway.
I read your blog entry with great interest. I share your belief that EBM is imperfect.
With regard to homeopathy - you make the following comment: For homeopathy to be true, not only would our entire understanding of the chemistry and physics of water have to be seriously in error, but so would our understanding of how cells respond to chemical compounds.
As a biologist, I too was a disbeliever. However, new research on water from top researchers at Penn State, The University of Washington, Stanford, and other institutions in Europe is dramatically changing our understanding of this substance and its potential to carry information into biological systems. Keep an open mind on this one.
If you mean Rustum Roy or Jacques Benveniste, please spare me. None of that supports homeopathy or the "memory of water."
Homeopaths will have to do a lot--a whole lot--better than that!
I was blown away by the actual mention and decent explanation of the Bayes Theorem in that article. The writer did an amazing job of making it accessible to my mother, who is my standard layperson test.
Sadly, today Tara Parker Pope covers the Vitamin C/cancer trial results from Sloan-Kettering. Her article is neutral, but comment number four, from "Ruth," says: "Hmm. Vitamin C is not patentable. Could this have something to do with the findings?"
Which just shows that evidence be damned, misguided "gut instinct" will win out. All my warm and fuzzy feelings from the Times article vanished.
EBM should have levels lower than level 5, expert opinion based on physiology, with correspondingly lower levels of weight and credibility.
I would put purely random interventions at having zero a priori probability. Also interventions which have zero basis in physiology would go here too. This would include prayer (no basis in physiology) and homeopathy. Interventions proposed with no basis in physiology would go here too.
Interventions that contradict expert opinion based on physiology would have negative a priori probability. I would put liver flushes here.
Interventions that have been shown to be harmful are lower still. I would put things like silver supplements below zero, they are known to be harmful.
Once an intervention is shown to be harmful, there is a negative expectation that it will help.
6. No basis in physiology (e.g. prayer, homeopathy, Riki, acupuncture) equivalent to zero a priori probability. Administration is probably useless.
7. Intervention expected to be harmful based on expert opinion based on physiology (coffee enemas, liver flushes, HBOT for seizure associated disorders, chelation) equivalent to negative a priori probability. Administration is probably harmful, people pushing it should be prosecuted for fraud.
8. Interventions shown to be harmful in humans (silver supplements, radioactive water). Known to be harmful people administering it should be prosecuted for criminal assault, poisoning and/or battery.
There are a number of interventions that are helpful at one level but which become harmful at another. Erythropoietin for example. At some level increasing hemoglobin in the anemia of kidney failure prolongs life, but at higher levels it shortens life.
EBM and SBM need to explicitly take such things into account.
Thank you for this analysis. I have been thinking something similar but not in such a clear-minded way: how can people make claims about alternative medicines that defy our understanding of physics, chemistry and biology. If they are genuine effects, then obviously we'd need to adjust our understandings of those fields but those claims have not been demonstrated. I particularly liked your point about 5% or more of studies showing false positives as applied to weakly positive results. Interesting stuff! What I'm not sure about is how much well-designed studies as funded through NCCAM will do. People who believe in CAM won't be convinced. A negative result won't prove anything, while a positive result (I expect there are some CAM practices that do something) may reinforce the view that scientists have been trying to quash practices that actually work.
Good post. I'm curious, what level you'd place "[e]xpert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles," if you were to recreate EBM as SEBM? Would you subdivide it?
It is a huge mistake to think alt med doesn't generate profits. It generates a huge truckload of profits, and that is why its practitioners will fight tooth and nail against any form of rigorous testing. And the best strategy they have found to defend themselves against negative results is a monetary argument based on a conspiracy theory.
The "magic pill" strawman is quite ironic in that you will see many alt med proponents, such as chiropractors, try and sell you those magic supplements, which according to them, are not "drugs", are "natural" and "have no side-effects". While the doctor will tell you that yes, you can take sleeping pills, but be careful because it can have such and such side effect. The simple, but inconvenient, truth is that no efficient drug, whether natural or synthetic, is devoid of side effect. Any substance that you take in the hopes of correcting a health problem is, by definition, a drug. Which one of the two drugs, based upon that description, seems more like "magic" ?
The other nasty strawman implied in what she said is that all doctors will resort to a pill with known side effects in all cases before suggesting that a patient tries relaxation methods, be they yoga or any other. Somehow that doesn't jive with the doctors I know of. I know someone whose longest night sleep are 4 hours long (that's when she's not slept for days) and somehow, she's not been prescribed sleeping pills yet, by all the doctors she's consulted. Good doctors are able to evaluate the risks and benefits of treatments, and will suggest trying alternatives (exercising, stop smoking, better food habits, relaxation) before prescribing drugs, especially if those drugs can have detrimental effects on the health of their patients. And ironically again, when they are refused a speedy prescription from their GP, that's when patients will go to an alt med peddler for the magical natural solution.
The health industry, be it pharmaceutical or "alternative", is driven by its clients' demands. And it is the client which demands a fast effortless solution to his/her problems. Thus, when the doctor honestly tells them there is no magical solution to their problems, patients will turn to alt med, which has no such compunction to be honest at all.
Yoga is probably a bad example to use when discussing the value of CAM (esp. w/ the sleep study above). True, it's Eastern in origin, and many followers of CAM practice yoga, so it gets thrown into the mix--but at the end of the day yoga is exercise, mixed w/ breath awareness, etc., and studies have already shown exercise is effective in treatment of insomnia. It would be more surprising if yoga *didn't* have a statistically significant effect.
The fundamental fallacy of trials-driven epistemology is that no trial or conceivable series of trials can prove that something doesn't have an effect. Oh, sure, they can put an upper bound on the amount of effect that it can plausibly have but they can't, ever, show that there is no effect at all.
This isn't really a problem with Real Medicine (tm) because Real Medicine is based on risk/benefit tradeoffs so the threshold isn't, "does it have an effect" but instead "does it have a benefit materially greater than the risk" (or sometimes conversely.) That's much easier to test empirically.
However, with woo the "risk" is often the same as the "benefit:" zero. A hundred and fifty years ago that was a serious advantage for homeopathy, since the conventional medicine of the time was often worse than no treatment at all. Maybe elsewise it might have achieved the lasting obscurity it so richly deserves.
That leaves us with self-moving goalposts in the realm of woo-evaluation.
You write:"much of what we know about anatomy and physiology would have to be wrong. We know, for instance, that there are no such things as meridians,"etc. I wonder how do we know that; meridiams, emergy points, etc. are not the elements of our western conception of man, they belong to a quite different system of knowledge, which seems to be hardly compatible with ours. You cannot disprove the claims of this eastern conception of man by using the criteria of our western science. But the fact that something is inconceivable within our western conceptual scheme does not necesserily mean that it is improbable or unreal. "The doctors- let me quote dr.D.J.Benor -have not yet learnt to accept that because something is different, it does not mean that the other is wrong."(A Clinician on Healing-An interview with dr.D.J.Benor in THE WORLD OF HEALERS by L. Dobrzanski)
ld said "You cannot disprove the claims of this eastern conception of man by using the criteria of our western science."
I hate the "east vs. west" canard. Science is science... and the laws of nature act the same way in Japan as they do in Germany, and also in Africa and Australia. The is no other mystical plane of existance that makes "energy points" measurable or useful that exists on this entire planet.
In the last few months I have benefitted from a couple of medical interventions that originated in Japan. Are they "western" or "eastern"?
Let us NOT confuse ancient Chinese conceptions with the REALITY of meridians and acupuncture points demonstrated by NMR studies. Go to the Past Issues archives and www.medicalacupuncture.org and see for yourselves.
Issues prior to Volume 18 are available for all to view.
Pay particular attention to Dr. Niemtzow's articles, they are quite good articles and people will CONTINUE to get acupuncture treatments while theorists pout that the merdians and points are .... "not real".
Let us NOT confuse the reality of research in Homeopathy
with the uneducated and unscientific shouts of "it's just water" or "it's nonsense". ESPECIALLY in this era, one's high school chemistry knowledge is INADEQUATE to draw any conclusions whatever.
The REALITY is that M Ennis' experiments, demonstrating clear biological activity stimulated by a highly dilute substance with NO atoms of the stimulating substance remaining, remains unrefuted and without scientific explanation. This experiment was repeated in several international labs, with positive results by several of them and is described in the journal Inflammation Research vol 53, pge 181.
Please do NOT mention the BBC Horizon documentary which some believe purported to repeat her experiment with negative results. Months after the documentary the producers finally begrudingly admitted to Ennis and others that they never intended to reproduce her experiment though they did nothing to counter the widely held belief that this was so.
So we have at least ONE major experiment demonstrating biological activity from the supposed "nothing". How interesting. There are, of course, MORE.
There is real scientific research and real scientists and then there are the quackspotters, who see a ruse, a quack and a charlatan in their closet, under their beds and in the pages of the Guardian and who would not recognize REAL scientific research if it fell on them, nor, apparently would they want to.
Perhaps we could as easily reduce chemotherapy with the expression, it's just poisons, or pharmacology by exclaiming that, "it's just chemicals".
The argument you give is exactly the argument that Cardinal Bellarmine gave to Galileo: I don't care what your telescope shows, Aristotle proved that the Earth is stationary. Bayes theorem, indeed.
@James Pannozzi
The REALITY is that M Ennis' experiments, demonstrating clear biological activity stimulated by a highly dilute substance...
I don't think "biological activity" means what you think it does.
@Prolix
Good analogy, except we're the ones with evidence, and you're the one deciding things by philisophical fiat.
Sorry, but I have a PhD in chemistry, I am working in medical research, and I'm afraid that's still "inadequate" to "understand" such nonsense.
The basic knowledge of chemistry one gets in high school is not nullyfied by what one learns at college and graduate levels. That you even speak like that betrays the fact that you don't understand science at all.
This is pseudoscientific gobbledygook which aims only at dishonest confusion by making it seem that you are so much more smarter than everybody else. Here's the news : you're not. You are talking nonsense.
If you knew anything of chemistry dude, you'd know everything is just chemicals. Here's the news, since you don't seem to have them yet : all the phenomenons of life consist of chemical interactions. All. of. them. And all chemical reactions obey the laws of physics. That is the true power of science. It all fits together ultimately.
I agree completely. This was not, however, a mistake I was making. I was speaking specifically about the yoga study (and similar studies) and the researcher who was quoted - not "alt med" in any vague, general way. That's why I asked for clarification. Please revisit my comment; also see Erika's.
From the BMJ (British Medical Journal):
Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials
Gordon C S Smith, professor, Jill P Pell, consultant
Abstract
Objectives To determine whether parachutes are effective in preventing major trauma related to gravitational challenge.
Design Systematic review of randomised controlled trials.
Data sources: Medline, Web of Science, Embase, and the Cochrane Library databases; appropriate internet sites and citation lists.
Study selection: Studies showing the effects of using a parachute during free fall.
Main outcome measure Death or major trauma, defined as an injury severity score > 15.
Results We were unable to identify any randomised controlled trials of parachute intervention.
Conclusions As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials. Advocates of evidence based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute
http://www.bmj.com/cgi/content/abstract/327/7429/1459
[ I saw this on a blog somewhere a while back, but can't remember where; sorry for the lack of credit. ]
I suggest you all read the following, it is the BEST characterization of this new DENIALIST FUNDAMENTALISM which has taken hold. The Abstract is posted here,
see this link:
http://www.hpathy.com/papersnew/milogram-homeopathy-fundamentalism.asp
for the full article. And KUDOS to Orac
for admitting the limitations of EBM!
Homeopathy and the New Fundamentalism:
A Critique of the Critics
- Lionel Milgrom, Ph.D., F.R.S.C., M.A.R.H.
ABSTRACT
"Though in use for over 200 years, and still benefiting millions of people worldwide today, homeopathy is currently under continuous attacks for being "unscientific." The reasons for this can be understood in terms of what might be called a "New Fundamentalism," emanating particularly but not exclusively from within biomedicine, and supported in some sections of the media. Possible reasons for this are discussed. New Fundamentalism's hallmarks include the denial of evidence for the efficacy of any therapeutic modality that cannot be consistently "proven" using double-blind, randomized controlled trials. It excludes explanations of homeopathy's efficacy; ignores, excoriates, or considers current research data supporting those explanations incomprehensible, particularly from outside biomedicine: it is also not averse to using experimental bias, hearsay, and innuendo in order to discredit homeopathy. Thus, New Fundamentalism is itself unscientific. This may have consequences in the future for how practitioners, researchers, and patients of homeopathy/complementary and alternative medicine engage and negotiate with primary health care systems. "
from:
http://www.hpathy.com/papersnew/milogram-homeopathy-fundamentalism.asp
Again, see the above link for the full text of Milgrom's article.
James, let me recapitulate Orac's post using fewer words, since you seem to have lost the jist:
Homeopathy is a crock of shit. You don't need randomized double-blinded placebo controlled crossover trial with a battery of multivariate statistical tests to tell you that shit stinks.
Patrick, let me recapitulate my post using fewer words, since YOU seem to have lost the jist:
Anti Homeopathy statements are a crock of shit.
You don't need anti-scientific innuendo about "woos" and "placebo" to tell you that the denialist fundamentalism at the heart of such un- scientific attacks to tell you that their shit stinks worse.
AGAIN, READ the comments of a scientist on this for the sordid details:
http://www.hpathy.com/papersnew/milogram-homeopathy-fundamentalism.asp
if you dare.
Let me recapitulate my perspective in brief:
Homeopathy is a crock of shit on a scientific basis. You don't need pseudoscientific hand-waving about "open-mindedness" and "memory of water" to realize that magical thinking are at the heart of such unscientific apologia that tell you that, whatever the deficiencies in "conventional" medicine, homeopathy's shit stinks far worse.
Please don't cite that guy, would you ? He's made a lot of theoretical chemists howl with pain at his abuse of quantum chemistry. He sounds like he's swallowed a quantum physics textbook backwards and proceeded to vomit it in the form of a crappy paper. Since quantum physics is not habitually part of a biochemist's or biologist's (or MD) curriculum, it is not easy for them to spot the glaring mistakes this crook makes.
For non-chemists and non-physicists : 99.999999% of the time when someone is applying quantum physics above nanoscale and below interstellar scale, you can safely assume he's talking nonsense.
Now do you want to discuss Rustum Roy's paper in which he demonstrates his inability to perform a calorimetric experiment properly ?
The Cochrane Reports have described as "promising" the effects of Oscillococcinum in the treatment (not prevention) of influenza and influenza-like syndrome. These results were from FOUR large (over 300 subjects each) double-blind trials.
Jennifer Jacobs (University of Washington) has conducted 3 clinical trials, one of which was published in PEDIATRICS and a meta-analysis was published in the PEDIATRICS INFECTIOUS DISEASE JOURNAL.
David Reilly (University of Glasgow) has conducted four high-quality randomized and double-blind trials on allergic disorders and found significant results in each study. An editorials in the Lancet acknowledged these studies as high quality and as unlikely to have been the result of false positive results.
Iris Bell and colleagues conducted a high quality trial on patients with fibromyalgia: Improved clinical status in fibromyalgia patients treated with individualized homeopathic remedies versus placebo, Rheumatology. 2004:1111-5. Participants on active treatment showed significantly greater improvements in tender point count and tender point pain, quality of life, global health and a trend toward less depression compared with those on placebo. People on homeopathic treatment also experienced changes in EEG readings. "Helpfulness from treatment" in homeopathic patients was very significant (P=.004). The clinical results AND the objective findings from the EEG readings combine to show that SOMETHING was going on (beyond and different from a placebo effect).
The experimentation by V. Elia, an Italian chemist, on the thermodynamics of homeopathic medicines have been published in the Annals of the New York Academy of Sciences and in respect chemistry journals.
There IS a body of evidence that differentiates results of homeopathic medicine as different from placebo, and it is inaccurate to suggest that there is "no" evidence.
Finally, Shang's Lancet review (2005) noted that he found 22 high quality homeopathic trials (out of the 110 studies) but only 9 high quality conventional medical trials (out of 110 studies evaluated). The results from these randomized and double-blind trials show that there IS a difference in homeopathic treatment. However, only when Shang chose to select the largest trials, 6 of 8 of which had no external validity to the homeopathic system, did he find a "negative" result.
Re Dana Ullman's post:
So the evidence is there for all to see.
Pretending that positive Homeopathy evidence is not there or that articles appearing in respected scientific journals written by fully credentialed scientists were composed by quacks and charlatans simply won't work any more.
Sorry "Orac". Time for a little innuendo distraction?
Dana,
There have been literally hundreds of studies of homeopathy. It's absolutely unsurprising that we get a few studies with a positive result. Indeed if there were not a few studies with a positive result (having done 100s) that would be evidence that homeopathy was harmful. When we look at bigger more rigorous studies, the effect goes away.
The publishing pattern is absolutely consistent with the notion that consuming a few drops of water albeit with an elaborate preparation does sod all either way as a medical intervention.
I should add if it's unclear, this is a simple property of using 95% confidence intervals. Periodically you will get "publishable" results coming up by chance alone.
Hmmm...people above are referring to a post that I sent you earlier in the day, but it seems that you've chosen to delete this post. Why?
My comment provided specific references to research that you considered too supportive of homeopathy.
Are you supportive of evidence based medicine or not?
The bottomline is that there are more than a "few" clinical trials showing positive results from homeopathic treatment...including the "high quality" studies.
Patrick, I think you make a good point with your comment on parachutes. I think that issue could be addressed if EBM added rules based on theoretical harm and demonstrated harm as I outlined. Potential for harm based on good theoretical reasoning trumps complete ignorance, but doesn't trump a well run clinical trial that shows positive results.
The essence of the decision to treat and to choose a specific treatment is the risk/benefit ratio of that treatment for that condition. There is no intervention that has only benefits, parachute use not withstanding. People can get tangled in parachutes and drown, they can get sucked into jet aircraft air intakes, they can present a bigger target for dragons. Since dragons are either non-existent or exceedingly rare, that "risk" is very small, but is in principle calculable. As someone pointed out, clinical trials can only put bounds on the likelihood of an outcome and those bounds depend on the sample size. Extremely small effects take extremely large (and hence expensive) trials to measure.
EBM should have explicit rules for calculating harm as well as explicit rules for calculating benefits. Then the two can be presented to the patient and the patient can evaluate them based on the patient's risk/benefit/cost analysis.
There was a similar series of letters regarding the evidence for use of specific techniques in cleaning skin and the final sterilization with iodine. Does one scrub back and forth, or does one do a final iodine application in a spiral, starting in the center and moving outward? A clinical trial to distinguish which is better would likely have to be gigantic. The very occasional failures are most likely not related to technique, but to other circumstances such as a speck of dust falling in the wrong place, or someone already having an infection beneath the skin where the iodine didn't penetrate, or the clinician not washing hands or changing gloves first.
The parachute argument doesn't hold much water for me; it's argument by analogy, and bad analogy at that.
It's possible to have a clinical trial of a parachute without making people jump out of airplanes. You don't have to intentionally expose a human being to anthrax after vaccination to test the efficacy of the vaccination.
How do you think they test new parachutes/ejection seat designs? How do you think they test new seat belt or airbag designs? Ever heard of a crash test dummy? Ever heard of animal trials? Ever heard of cadaver tests? I believe the first test of a modern parachute was done on a dog, not a human.
Arguments from physiology are considered acceptable for EBM, they are just the lowest level of evidence. There is a quite good theoretical analysis saying that parachutes are better than nothing, and no clinical trials saying that no parachute is better than having a parachute.
hum, no ?
Let's see what we have:
Benveniste: A fraudster uncovered when his experiment was repeated under James Randi's supervision
Lionel Milgrom: A guy who publishes pseudo quantum physics in a crappy journal where the mistakes in his reasoning cannot be spotted in the absence of qualified peer review
Rustum Roy : A guy who's not only unable to perform calorimetric experiments properly, but isn't even able to report results correctly
Dana Ullman : a homeopathy crank banned from wikipedia who publishes books
A few "memory of water" cranks à la Dr Emoto, and a couple good papers which have nothing to do with homeopathy, because homeopathy does not deal with "dilute" solutions but with solutions which typically do not even contain one molecule of the active principle per dose.
Qualified scientist defending homeopathy ? Where ? Did I miss something ?
Pardon me Orac, BUT
WHY DID YOU REMOVE DANA ULLMAN'S POST?
YOU CLAIM HOMEOPATHY IS NONSENSE BUT
WHEN SOMEONE POSTS SOME EVIDENTIARY
OR EXPERIMENTAL BASIS, YOU FIND IT
NECESSARY TO DELETE IT????
I delete nothing. I've only banned one person in the entire three and a half year history of this blog. I have no idea why it did that, but it was in the spam folder and I restored it. Stop typing in all caps. it makes you look like a crank.
Some people here like to call homeopaths and people who are pro-homeopathy "cranks." Because a "crank" is that mechanical device that was previously used to get a car moving, I have no problem being considered this catalyst.
By the way, Orac, I think that James Pannozzi was rightfully indignant because he and others saw my post at your site here, but then, all of a sudden, it disappear. He was trying to be a catalyst to encourage dialogue between points of view in the interest of information and scientific inquiry. Bless that type of crank.
Typing in all caps is the Internet equivalent of yelling and it is generally agreed that people who type in all caps come off looking like a crank.
Dana,
If I did 100 identical studies of homeopathy, and were looking for those with p-value < 0.05 (the commonly accepted standard for publication) it would be absolutely unremarkable to discover approximately 5 studies showing that homeopathy worked. Finding zero studies would be evidence of harm.
If we add to that a publication bias where people tend to publish studies which work more than studies which don't, what we get is absolutely consistent with the clinical trials.
What I've just described informally is what a meta-analysis does formally. Big meta-analyses say that it's bunkum.
But to repeat, we don't need high-falutin' statistics to get to this point, with thousands of people diligently quaffing down their micro-doses of nothing and recording it all at great expense. A great quantity of effort and energy which could be used on, for instance, healing the sick. All we need is the kind of chemistry we teach to 15 year olds.
Alternatively we could think to ourselves that near where I live, the big dam has a watercourse with a few cattle farms nearby, and the catchment is full of wild animals with all sorts of intestinal parasites. Most cities are the same. This water fills up the dams and gets massively diluted. We should all be completely homeopathically proofed against giardia and cryptosporidium, just from tap water.
Kemist, you are a valuble chap to have around in these discussions and perhaps with your help we can focus our perspective in this debate.
What have you missed?
Well, you might have missed pharmaceutical researcher M. Ennis, a skeptic of Homeopathy who set one day to repeat some experiments of a French researcher regarding the stimulation of basophil cells to produce histamine by a highly dilute substance which no longer contained any atoms of the stimulant. She was of course expecting negative results but instead got positive results. The experiment was repeated in several international labs, many of them getting positive results, some negative.
So.... what is your hypothesis regarding the mysterious production of the histamine demonstrated in this experiment. Cosmic rays? Enhanced nothingness of the high diliution substance? Ernst anomoly neutrino reaction?
Or do you admit that the experiment did show histamine release triggered by the "nothing" of the high dilution substance.
Ennis herself published the results in "Inflammation Research" and stated that the results were unexplainable by known science and that others should repeat the experiment to discover what's what.
Was she on to something or was it much ado about nothing?
Patrick, your logic is whack. By your logic, the atomic bomb was a placebo because there are atoms around us right now that are not "exploding." Duh. Please consider doing some homework on the mystery of water. Please also consider:
Vigorous shaking of the water in glass bottles can cause small amounts of silica (silicate) fragments or chips to fall into the water (this has been verified in physics journals). Homeopathic drug manufacturers use a double-distilled water in making their medicine, and whatever medicinal substance is placed in the water interact with the silicate fragments and can change the structure of the water in different ways.
It has been shown that micro-bubbles and nano-bubbles, caused by vigorous shaking, can "burst" to produce microenvironments of higher temperature and pressure. One group of material scientists have estimated that the vigorous shaking involved with making homeopathic medicines changes the pressure in the water, akin to water being at 10,000 feet in altitude. These scientists suggest that the homeopathic process of using double-distilled water and then diluting and shaking the medicine in a sequential fashion changes the structure of water.
Mr. Panozzi, I'd like to inform you of the way science is discussed among scientists. First, do not rant and insist that you're smarter than everybody. Second, post a reference for a paper, that we might actually have a valuable discussion about something that we can all examine for ourselves.
Several crappy science papers, for many reasons, go through peer review unchallenged. One recent example of this is a paper pushing hastily and obscurely deduced creationist ideas that made it into Proteomics, a well repected science journal. Another less recent example involves a certain individual called Sokal whom without even a degree in physics managed to publish crap in a respected physics journal.
Another more to the point example concerns the unamed french scientist that you happen to mention, which actually commited something that amounts to scientific fraud and published this, of all places, in Nature.
It is possible to clear some weeds by dismissing such crap heaps as Homeopathy, Journal of Medical Hypotheses or JPANDS, but, as you can see, that's not the only criteria for good science.
Anybody who's familiar with how biological activity is assayed knows that unexplained results are a frequent occurance. However, in the vast majority of cases they have less to do with weird new things that will make you win the Nobel prize than with uncontrolled variables and reagent contamination (of which any scientist, even the most rigorous and honest, can become a victim).
So until more is available, consider me very underwhelmed by somebody who repeated the fraudster Benveniste's experiment and found positive results, even if she calls herself a "skeptic about homeopathy". So, yes, unfortunately (because, like most scientists, I happen to like Nobel-worthy weird new ideas), much ado about nothing.
To Kemist:
Inflammation Research, vol 53,
Histamine dilutions modulate basophil activation
pp. 181-188(8)
Authors: Belon, P.; Cumps, J.; Ennis, M.; Mannaioni, P.; Roberfroid, M.; Sainte-Laudy, J.; Wiegant, F.
The unnamed French researcher to which I was referring, contrary to your mistaken supposition and rant about "fraudsters", was J. Saint-Laudy, the scientist whose experiments were repeated by Ennis.
Consider me underwhelmed by your response until you provide some explanation, theory, hypothesis, or hopefully, an admission that you don't know why the basophil cells' histamine release was stimulated by a high dilution substance having no atoms of the stimulant physically present, and admit that it is worthy of continued research.
Do not mistake my disgust at the easy talk of "fraudsters", "quacks" and "charlatans" applied to genunine scientists, for arrogance.
M'sieu P Belon, of course, works for French Homeopathic remedy manufacturer Boiron. Coincidentally (or not) Belon was also an author of the infamous / incompetent Jacques Benveniste Nature paper that someone already mentioned. Benveniste had people in the lab funded by Boiron. The person in Benveniste's lab who got positive results with homeopathic dilutions was - would you believe it - an enthusiastic supporter of homeopathy. And so on, and so on.
Or, if you prefer it in French:
Take the blinkers off, homeopaths. Oh, sorry - I forgot I was asking the impossible.
It is not even necessary for these people to "fix" the experiments deliberately. It is merely necessary for them to convince themselves (quite sincerely) not to exercise the customarily necessary critique of experimental design, or of their own fallibility. Or convince themselves they can edit out the non-positive results for homeopathic dilutions because "something is wrong there", or not do proper controls "because we don't need them" (both famously occurring in the Benveniste case).
As if it needed saying yet again, this is classic Cargo Cult Science - which can, of course, be found every day across the pages of numerous Journals of Alternative Medicine.
You started by accusing everyone here of being fundamentalists and denialists. That is not indignation, that is just plain arrogance, especially when you're the one who's bringing an opposing opinion to the table. And then, you brought up Milgrom. As a qualified scientist. It doesn't start a conversation on good terms, especially for one in my field.
But now, we have a proper paper to look at.
Two minor quibbles in this paper :
A potential conflict of interest which sort of negates what you said previously of Ennis. But, frankly that doesn't really matter, since we shouldn't juge science results according to opinion but rather quality (but hey, we humans all have prejudices of one sort or another).
Another thing which appears in this paper (one of the most annoying thing ever in discussions with homeopathy proponents), is the comparison of liquid solution phases with solid phases. It's... wrong. Both are very different and you just can't extrapolate significant results from solid phase to solution phase. But, ok, I'll let that pass, since it's just a weird sort of unrelated discussion of the results.
Now the more serious things:
The way the results are reported is a little strange from my POV. First we're told that the 4 labs performed the same experiments, with great emphasis upon the use of the same reagents, labware and training of technical staff... And then the experiments themselves become different. One lab reports only extra high dilutions, the other a wide range... There is a humongous variation on the error bars... (some consistently have very small, others huge, for what essentially seems to be the same experiment). It makes it difficult to get a feel of the noise on these experiments. I understood that there can be variation in the sensitivity of basophils from different donors but... Couldn't the samples be pooled together before the assay, so all labs would share the same source (It's been done with human lymphocytes in my lab, that's why I ask) ? Maybe it would make those error bars behave.
There is a sort of game going on also with statistical tests. That is also... strange. Why are the labs using different tests to find significant differences ? When they are supposed to be performing the same experiment (Maybe a statistician can answer me for that one)?
And in most instances lab #2's results are ditched, like they're not important or something. Indeed, lab #2's results are not discussed at all.
To be completely honest, it's a weird paper. Am I curious to see other people try things like this ? Mildly. Would I approve investing money in it ? heeshh... not really, just based on this particular paper.
Many thanks to Kemist for giving us some considerations regarding the Ennis paper and for giving us an overview of some questionable issues with it in the light of his experise.
And Kemist also has had the honesty to state:
"But, frankly that doesn't really matter, since we shouldn't judge science results according to opinion but rather quality (but hey, we humans all have prejudices of one sort or another)."
I agree.
And thank the maker, an entire post with no tirades against Benveniste.
Related papers you might find of interest follow:
Lorenz I, Schneider EM, Stolz P, Brack A, Strube J.
Influence of the diluent on the effect of highly diluted histamine on basophil activation.
Homeopathy. 2003 Jan;92(1):11-8.
PMID: 12587990 [PubMed - indexed for MEDLINE]
Sainte-Laudy J, Belon P.
Use of four different flow cytometric protocols for the analysis of human basophil activation. Application to the study of the biological activity of high dilutions of histamine.
Inflamm Res. 2006 Apr;55 Suppl 1:S23-4. No abstract available.
PMID: 16705375 [PubMed - indexed for MEDLINE]
Sainte-Laudy J, Boujenaini N, Belon P.
Confirmation of biological effects of high dilutions. Effects of submolecular concentrations of histamine and 1-, 3- and 4-methylhistamines on human basophil activation.
Inflamm Res. 2008;57 Suppl 1:S27-8. No abstract available.
PMID: 18345504 [PubMed - indexed for MEDLINE]
Sainte-Laudy J, Belon P.
Improvement of flow cytometric analysis of basophil activation inhibition by high histamine dilutions. A novel basophil specific marker: CD 203c.
Homeopathy. 2006 Jan;95(1):3-8.
PMID: 16399248 [PubMed - indexed for MEDLINE]
Guggisberg AG, Baumgartner SM, Tschopp CM, Heusser P.
Replication study concerning the effects of homeopathic dilutions of histamine on human basophil degranulation in vitro.
Complement Ther Med. 2005 Jun;13(2):91-100.
PMID: 16036166 [PubMed - indexed for MEDLINE]
Lorenz I, Schneider EM, Stolz P, Brack A, Strube J.
Sensitive flow cytometric method to test basophil activation influenced by homeopathic histamine dilutions.
Forsch Komplementarmed Klass Naturheilkd. 2003 Dec;10(6):316-24.
PMID: 14707480 [PubMed - indexed for MEDLINE]
Thanks
JP
Dana,
Understandably you completely missed the point of my posting, because the blog software apparently has a problem with displaying a "less than" sign, presumably because it mistakes it for HTML.
The first paragraph should have read something along the lines of:
If I did 100 identical studies of homeopathy, and were looking for those with p-value < 0.05 I would expect to find something in the order of 5 studies showing that homeopathy had an effect. Indeed if I found zero studies showing that homeopathy had an effect given the hundreds that have been done, that would be evidence that something odd was going on.
You have to do a lot more than show half a dozen studies showing that homeopathy works. You have to account for the overwhelming majority of studies that show that it's ineffective.
But to re-iterate, why bother? It's close to self-evidently bunkum from the basic chemistry. You first need to fundamentally transform basic chemistry before anybody with half a clue will even begin to listen to you; ill defined hogwash about "silica" doesn't cut it I'm afraid. Let's quit analyzing something so obviously stupid and spend our tax dollars/research dollars/grant money on something which at least has the potential to heal sick people.
Yup, he's a pretty good example of it. Have you seen his claim that double blind RCTs of homoeopathy fail because of "a kind of quantum superposition between the remedy and placebo"?
You forgot to mention the attempt by a team at Southampton to replicate Reilly's findings.
Excuse me Patrick but it not at all "close" to self evident bunkum based on your personal chemistry knowledge just as the existence of those "impossible" pyloric bacteria, who could not possibly be there because of acidic environment in the stomach, was not so self-evident after all because the little suckers were hiding in the stomach lining and one chap earned a Nobel proving it.
That's just the point - the denialist fundamentalist points to their chemistry knowledge, their sense of what is logical, their personal sense of what could not possibly be and then very unscientifically dismisses the research. Such an attitude presupposes godlike omniscience on the part of the denier, who somehow knows IN ADVANCE, that there exist know unknown factors to discover. Maybe that's the appeal.
Unfortuantely, back in the real world, the scientific world, there are many things which go completely counter to "common sense" and to one's personal intution and yet which are real.
And THAT is the entire point of research - to uncover the unknown, to explore, to map out uncharted terrirtoy. To short circuit this with your personal sense of outrage that anyone DARE suggest something contrary to your personal scientific world view, or the shared world view of a bunch of denialists, you want to tell me I should respect that??
The period of compulsive denialism regarding Acupncture is just about over. Neurogenic theories and other theories have been advanced and NMR studies clearly show visual cortex activites when a point on the foot, historically said to have effect or benefit on the eyes is stimulated and no visual cortex activity from the stimulation of nearby non acupuncture points.
But because homeopathy is lacking this basic theoretical basis, the opportunity still exists for the denialists to enlighten us with their outrage, chagrin, amusement, contempt - anything EXCEPT the willingness to wait until resarchers do their stuff. In fact, a common argument against Homeopathy is that there is not enough research to support it, espoused by the SAME PEOPLE who want to kill the research!! This coupled with insults against established scientists makes a mockery of the scientific process, particularly when it goes as far as the pages of the Lancet.
And this from the very people who claim to exalt science,
even to the point of creating a fictional "evidence' base
for their exalted standard medicine, even after the aforementioned
pyloric bacteria researcher found it necessary to ingest some of the bacteria and demonstrate that he now had (suprise!!) pyloric ulcer signs, to silence the "skeptics" once and for all....
Apparently there was a bit more too it than stress and diet.
The denialists were WRONG, WRONG AND WRONG.
He did. And so will the Homeopathic researchers.
Hey, James,
Why can't homeopathy cause negative effects?
But it can cause negative effects! Mostly by not be effective in real life:
http://whatstheharm.net/homeopathy.html
I find it interesting that homeopathy is still stuck in the early 19th century, without any change and is defended so vehimetly. While science has marched on and progressed, homeopathy is stuck.
Of course, it should be noted that many of its primary defenders have been told over and over and over again the flaws in their data, logic and even conclusions, yet they continue to spread the falsehoods. Case in point:
http://forums.randi.org/showthread.php?t=99751
For the serious-minded folks here who still think that "homeopathic water" is the same thing as placebo, they should take a look at the website of Dr. Martin Chaplin, a British professor whose website contains around 2,000 references to water research.
His section on "the memory of water" is of interest:
http://www.lsbu.ac.uk/water/memory.html
His writings on homeopathy are here:
http://www.lsbu.ac.uk/water/homeop.html
"Excuse me Patrick but it not at all "close" to self evident bunkum based on your personal chemistry knowledge"
You're right James, it *is* just self evidently bunkum, not just close to it. Apologies.
More fun with Mr. Ullman:
http://forums.randi.org/showthread.php?t=82393
Dana Ullman (MPH!!) your task is simple- provide good, clinical support for Homeopathy. Your mewling and puking is tiresome.
Hm, Martin Chaplin. Here's his paper in Homeopathy on memory of water. Here's my comment on it. And, for completeness, Chaplin's reply. The problem is that while Chaplin does show that water is pretty interesting stuff, he doesn't show anything that has any relevance to homeopathy. It's all arm-waving, and in any case the clinical evidence shows that there is no effect from homeopathy, so no explanation is required.
Shang's "review" of homeopathic research has now been totally blown out of the water but a re-analysis of this work in the October 2008 issue of the Journal of Clinical Epidemiology in an article by Ludtke and Rutten. (Sorry, this journal is not available online unless you go to a university library. It IS worth it!).
And wait until you see another article on this subject in an upcoming issue of HOMEOPATHY (published by Elsevier).
It is a tad ironic that skeptics insist (!) that homeopathic medicine has no foundation in science and that there is "no way" that these doses can have any effect. And yet, when references are made to the scientific literature that suggest that homeopathic medicine does not break any laws of nature but simply extends our understanding of principles in nature, the skeptics change the question and ask where is the clinical data. It too is there for those people who get off their high horse and look beyond their own prejudices. The irony is palpable.
Skeptics remind me of our American President who never admits to an error and whose worldview creates blinders so that he cannot even see what is around him.
Are you prepared to admit that you made errors about Charles Darwin and Oliver Wendell Holmes, Dana?
I agree with Mr. Ullman that the Ludtke and Rutten article is interesting reading. While the authors focus primarily on the effect of selection bias on meta-analyses, their graphs of the effectiveness of homeopathy vs placebo should be required reading for anyone thinking about using homeopathy.
Even with all 21 studies included, the odds ratio (OR) for homeopathy was only 0.76 with a confidence interval (CI) of 0.59 - 0.99 (p = 0.04).
For those not familiar with the terminology, the odds ratio is a way of comparing the probability of two different events. In this case, it is comparing the probability of "improvement" with homeopathy to the probability of "improvement" with placebo, with an OR less than one meaning that the people who received a homeopathic treatment had a higher probability of "improvement".
The magnitude of the "edge" that homeopathy has over placebo can be seen by inverting the OR - the homeopathic treatment was 1.27 times more likely to result in "improvement" than the placebo.
To put that number into perspective, a study of ondansetron against placebo for post-operative nausea (http://www.anesthesia-analgesia.org/cgi/content/full/101/2/356) showed an odds ratio of 0.33, meaning that the nausea was three times more likely with placebo than with the drug.
In another study, angiotensin converting enzyme inhibitors (ACE inhibitors) were found to protect against aortic "events" (rupture, dissection, etc.) with an odds ratio of 0.18 compared to placebo (5.6 times less likely to happen compared to placebo).
In both of these studies, real medications were used to prevent conditions that have proved resistant to treatment. The odds ratio of medications like antibiotics, antihypertensive drugs, and drugs used to treat diabetes are generally much better.
So, an odds ratio of 0.76 for homeopathy isn't anything to "crow" about.
The "p-value" gives the confidence with which you can say that the two groups (homeopathy and placebo) are different in their "effectiveness". The standard in science is a p-value of less than 0.05 is needed to claim a "statistically significant difference".
This meta-analysis had a p-value of 0.04, which indicates that there is a 4% chance that the difference seen is due simply to random events and not due to any effectiveness of the homeopathic treatment. For comparison, the p-value of the ondansetron study was less than 0.0001 (less than a 0.01% chance that the difference was due to random events).
If Mr. Ullman wants to claim that this article re-working the statistics of a meta-analysis somehow vindicates homeopathy, he is free to do so. Personally, I found that the article simply reinforced previous studies showing that the effect of homeopathy - even when studied by researchers sympathetic to homeopathy - is generally indistinguishable from placebo.
Prometheus
Mojo has asked me about Darwin and his homeopathic doctor. People who are interested will be amazed to see what I've written (http://www.homeopathic.com/articles/view,128), though my new book, "The Homeopathic Revolution" (www.HomeopathicRevolution.com) provides even more evidence.
The Ludtke/Rutten paper in the "Journal of Clinical Epidemiology" IS important, as is its companion paper in the October 2008 issue of "Homeopathy" (published by Elsevier) by Rutten/Stopler.
Both papers show that Shang's results (that the effects from homeopathic medicines are the same as that of a placebo) are "less definite" as they have been previously presented. The Lancet editors have even asserted this "study" now closes the case on homeopathy by publishing an editorial called "The End of Homeopathy". However, this new analysis shows how fatally flawed this study was and how embarrassingly biased its editors have been.
First, let's assume that the Shang paper was perfect, even though the above two articles prove that this is not true. Although Shang's paper asserts that the effect from homeopathic treatment is very small, even their skewed data show that the odds ratio (OR) from the 8 large and high quality homeopathic trials found an effect of OR = 0.88, which was the same as a meta-analysis of statin treatment and the occurrence of haemorrhagic stroke.
Both papers show that Shang's data was biased, and different data sets give different interpretations. Good statisticians would have acknowledged this, though these authors have a known bias against homeopathy...and they proved it.
Shang omitted certain high quality studies in homeopathy (was it a coincidence that the vast majority of these omitted studies had a positive result), how they defined what is "high quality" is open to question, their decision to never evaluate all of the "high quality" studies, and the size of the studies for the two groups were different (the lowest number of subjects in the homeopathic trials was 98, while the lowest for the conventional medical group was 146).
If Shang evaluated only those clinical trials that his own group defined as "high quality," there is a statistically significant difference between those patients given a homeopathic medicine and those given a placebo.
The new re-analysis notes that 2 studies by Reilly, one by de Lange-de-Klerk, and one by Hofmeyr were not defined by Shang as "high quality" but they were defined as such in a major meta-analysis of homeopathic clinical trials conducted by Linde, et al which was published in the Lancet (1997). Three of these four trials showed a positive effect towards homeopathic treatment.
It was interesting to note that Shang excluded Wiesenauer's chronic polyarthritis study (N=176) because no matching trial could be found (Linde, 1997, defined this study as "high quality"). And yet, none of the trials in the final evaluation matched each other in any way. Hmmm. How convenient, but please note that it was not "missing;" they purposefully omitted it and gave an illogical reason to do so (how convenient again).
Also, three of the eight large and high quality conventional medical trials tested drugs that were deemed to be "effective" and yet, these medical treatments have been withdrawn from medical use due to the serious side effects that later research confirmed.
Four (!) of the 21 high quality homeopathic trials sought to evaluate the prevention or treatment of muscle soreness. These three of the four trials had negative results, and if all of these trials were omitted from the analysis, there was a highly significant difference between homeopathic treatment and the placebo (P<0.007). All of the conventional medical studies that evaluated this condition found negative results, though none (!) of these studies were deemed by Shang to be "high quality," thus further skewing the results.
The new re-analysis of the Shang review did not question the outcome data that was extracted from the clinical trials. However, the authors did note that Shang reported on a study of traumatic brain injuries by Chapman, which found that reported on only one outcome measure as "negative" even though this study reported that 2 of the 3 outcome measures were "positive". Also, the Shang review surprisingly included a "weight-loss" study, and Shang extracted data from day 1, but day 2 had been defined as the main outcome parameter (whereas the day 1 results were "negative," the day 2 results were "positive" (in any case, this study should not have been included in the analysis because they had never undergone previous preliminary trial to deem their worthy of a larger clinical trial).
In addition to the information Dana has provided
I here make note that it took people MONTHS to
just to get information from the authors of the
Shang paper on the details of which studies
where omitted and the details of their evaluation
criteria.
This appears to happen quite a bit and one wonders
WHY, if Homeopathy were really the nonsense claimed by some, why it would be necessary for exagerations and ommissions about it in these kinds of articles.
A good example is the famous BBC Horizon documentary on Homeopathy which performed an experiment roughly LIKE the M. Ennis experiment which indicate biological activity stimulated by a high dilution substance in which all atoms of the stimulant had been diluted away. Her experiment WAS published in Inflammation Research (vol 53, p. 181) and of course the BBC Horizon pretend experiment was not published anywhere. Yet, it became widely believed after their widely seen documentary that they had repeated her experiment and gotten negative results.
BUT.... Ennis and other researchers questioned this and asked for details. They had to send emails and letters for MONTHS
before finally getting an answer from the BBC producers giving the name of the anonymous researcher who did the experiment,
and finally admitting that they had never intended to reproduce her experiment and, in fact, that the experiment that they had performed in the "documentary" utilized a different protocol than Ennis, including the introduction of a chemical know to kill basophil cells thereby either intentionally or through sheer stupidity ensuring failure.
How much of the attacks against Homeopathy is sheer innuendo initiated by the media and then parrotted by some of the posters here and by many unthinking others?
The defeated army, the screaming mob shouting to lynch an innocent without trial, the shouts of derision at the preposterous idea of stomach bacteria being involved in pyloric ulcers.... all follow, like a lightning bolt, the path of least resistance, an unthinking path of mass hysteria and mutually self assured certainty and omniscience while overlooking, or worse, ignoring, scientific details which Dana has noted both here and in several of his books.
One poster juggles some statistical numbers and draws sweeping conclusions without ever having seen a patient; another shouts "it's just water". Another one tells me that some Boiron researcher was involved in the Benveniste papers and was pro Homeopathy and apparently he thinks ALL Homeopathy research by ALL researchers is therefore instantly invalidated, forgetting that Ennis, a skeptic, honestly and openly had the courage to publish her unexpected postitive results and asked other to do so. They did. Most also got positive results.
We shall look forward to the next "End of Homeopathy" issue by Lancet, and the next and the next, as patients discover and use this system of medicine often finding it superiour to standard (it's just chemicals") remedies. "It's just water" and cries of "placebo effect"? Nature ignores such hysteria and remains true to itself. Those who can follow the way are rewarded with life.
And the arguments of the anti-Homeopaths....?
It's just words.
The Ludtke and Rutten paper is interesting, but I don't think it shows what Dana thinks it shows.
Would that be Darwin's "homeopathic doctor" who practised as a hydropath, who wrote a book, on which Darwin based his decision to consult him, called The Water Cure in Chronic Disease, and who
wrote, in 1861:
Are we to take it that the "Law of Similars" is no longer considered an essential part of homoeopathy?
I suppose people who are interested enough to follow up the references used to see what they actually say may well be "amazed" at what Dana has written. To find out why, see the JREF thread HCN linked to further up this thread, or the
Quackometer's discussions of his claims.
From James Panozzi "In addition to the information Dana has provided
I here make note that it took people MONTHS to
just to get information from the authors of the
Shang paper on the details of which studies
where omitted and the details of their evaluation
criteria."
Nice to see that it is now acknowledged that Shang et al did provide the information. Many homeopaths denied it happened. As to why it took so long, the Ludtke paper took months to be published from submission, that's what happens in science publishing, no conspiracy and nothing to hide so your capitalisation is OTT.
"it took people MONTHS to just to get information from the authors of the Shang paper on the details of which studies where omitted and the details of their evaluation criteria"
But the evaluation criteria were clearly stated in the original paper, despite what homeopaths will tell you.
It's probably escaped your notice (just as it appears to have escaped your notice that Shang et al stated their evaluation criteria in their original paper) that Barry Marshall and Robin Warren established that H. Pylori causes ulcers by producing conclusive evidence rather than by wittering on about defeated armies, screaming lynch mobs and mass hysteria.
Wilsontown has provided a properly careful rebuttal of DU's stubborn promotion of Ludtke & Rutten.
However, I think the following bears highlighting just for the insight it throws onto the form of thinking necessary to sustain a belief in homeopathy, and the need for care when interpreting pooled trial data.
"These three of the four trials had negative results, and if all of these trials were omitted from the analysis, there was a highly significant difference between homeopathic treatment and the placebo"
Well, spotted Dana. If you take data randomly scattered around a baseline and exclude enough negatives you will eventually show a statistically "positive" result.
I rolled a 6-sided die 100 times and excluded all the 1's 2's and 3's. By a miracle my average score was way over the expected 3.5 and this could be shown to be highly statistically significant. Spooky.
Dana, please construct a well-known phrase or saying from the words "pick" and "cherry".
As woobegone pointed out at Wilsontown's blog, homeopathy provides a perfect model system to explore the cultural phenomenon of an exactly worthless treatment. How can data from a zero-effect intervention be pulled, bent and selectively published to create the impression that it works? That is the service that homeopathy can provide us. It does nothing else.
Mojo, perhaps YOU failed to notice that Barry Marshall, one of the Pyloric bacteria researchers, ate some of the bacteria and soon demonstrated pyloric ulcer symptoms, thus silencing his critics once and for all - a perfect example of how unreasoning skepticism, such as is maintained against Homeopathy, can hinder research before it is complete.
Depsite the Nobel prize winning discovery, so firmly entrenched was the standard "stress causes ulcers" incorrect generalization, that it took years before new treatments reflecting the discovery became implemented universally.
And the denialists, in this case as well, were WRONG,
and this IRRESPECTIVE of their observations and knowledge about the stomach pH, ulcers and etiology. The problem was, as with Homeopathy, that NOT ENOUGH WAS KNOWN and what was "known" was partial, incorrect or misleading - and on the basis of THIS ALONE the two Austrailian Pyloric researcher encountered substantial opposition in even getting a hearing for their theory and discovery which eventually won a Nobel prize.
Denialist fundamentalism, it would appear, can hinder standard as well as alternative medicine research.
How can anyone think that Wilsontown adequately responded to the concerns expressed above about the Shang review? Garbage in, garbage out.
Next time,
1) Please respond to the exclusion of the polyarthritis study.
2) Please respond with an explanation for why several studies that were defined as "high quality" by other researchers would not be defined as such by Shang (be specific).
3) Please respond to why Shang selected the primary outcome measure with "negative" findings when the studies found primary outcome measures that were "positive."
4) Please explain why Shang purposefully ignored the "adverse events" problems from conventional and homeopathic trials (could it have been because the conventional medical trials had too many adverse events?).
Ultimately, Shang's review proves two things:
1) On average, homeopathic trials were of significantly higher quality than conventional trials (21 vs. 9!)
2) Conventional drugs seem to work more often than homeopathic medicines do, but 3 (!) of the 6 trials tested drugs that have been taken off the market due to their extremely dangerous side effects. So, conventional drugs work, but there's a 50/50 chance that they get rid of some symptoms but you get worse disease in the process (is that science?).
Once again, Shang proves GIGO!
James this is not ground worth fighting over, you cannot win.
1. The H. pylori data is supported by robust and repeatable data. The hypothesis may have come from left-field but the data that supported it were overwhelming. The data promoted in support of homeopathy are trivial in terms of any clinical significance and marginal in terms of any statistical significance, just as you would expect to be the case if you gave sick people sugar pills instead of medicine.
2. The H. pylori "denialists" as you caricature them were not "wrong" in the sense that you mean. The data that supported the "stress" hypothesis remained valid. The data supporting the efficacy of H2-blockers remained robust. Cimetidine did not stop effectively treating many ulcers the day that H. pylori was discovered. What happened was that a better explanation was found underlying the previous data so that narratives as to causation of gastric ulcers needed to be recast in terms of H. pylori's actions and treatments could be optimised based on that better understanding. This is how science progresses. Newton was not "wrong", but it turns out that what he discovered was a low-energy approximation which was a small part of a larger scheme discovered by Einstein. I'm afraid that idiotic caricatures of how science works do not help your cause. In contrast, homeopathy and its theories have not progressed. There are no robust data supporting its efficacy, here is no sign that a deeper narrative is needed. Homeopathy is sugar and water and its effects are illusory. That's all there is to it. The fact that many people are wedded to its fanciful ideas and generations of therapists have made a good living from those ideas has no bearing whatsoever on the validity of those ideas.
3. If homeopathy really worked, people like DU would not keep having to rake over the same handful of papers trying to build them up into some great scientific edifice. Have you not noticed that he and his ilk keep trotting out the same discredited papers, the only thing that changes is the venue? Today it's Orac's blog. Yesterday it was the JREF forum. His data do not change. He has a small handful of papers that he keeps citing. You seem to forget that this is the best that 200-years of homeopathic activity can produce. The paltriness of these paltry few papers merely reinforces the argument against homeopathy.
(That should have said "1. The H. pylori hypothesis is supported")
But not that many of them: homoeopathy has had over 200, and still no decent evidence.
What are you waiting for?
The singular of data is still not "anecdote".
And this has nothing to do with homeopathy. OK, your argument here seems to be along the lines of "they laughed at the Wright Brothers...": this doesn't mean that any passing crackpot theory that is derided is likely to be correct.
They also laughed at the Marx Brothers.
So after all the posturing of the last eight months, this is all Ullman has to offer. A study in Journal of Clinical Epidemiology which does not support his hypothesis and presumably a repeat with more extreme claims in Homeopathy, a journal with no repute whatsoever.
Remember his puffing of the Rustum Roy paper?
I think a pattern may be emerging.
Although it may be wandering off-topic, I'd really like to see DU respond to the published critiques of the Rustum Roy spectrometry work.
In case DU has forgotten, Roy's co-author came up with this feeble response;
http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6WXX-4RJXWNR-…
"Kerr et al's remarks concerning the probable
contamination in the sample of the original solvent
sent to us, although apparently not in any of the
dilution materials (and reported accurately by us, we
are inorganic materials scientists) illustrates one of the
values of our work to homeopathy producers and
users, and other health researchers. We have provided
them with potential quality control tools, and a
refining of the arguments away from incorrect generalizations."
In other words, she thanks the critics for allowing them the opportunity to acknowledge that spectrometry can detected contaminated solvents. To which a reasonable response would be "No sh*t, sherlock! Who'd have thunk it"
By the way, here's that elephant again;
Homeopaths: Give One, You Only Need One, Incontrovertible Example of Homeopathy Curing a Non-Self-Limiting Condition
If anyone thinks that Milgrom's 'quantum' mechanics has any worth, then they'd be well advised to see what a proper physicist makes of it. As for many of his other points in the 'new fundamentalists' paper: they don't hold water (this post is based on a presentation of his that covers much of the same ground). The Quackometer also picked up on some other problems with this rant.
BSM, strange that Dana hasn't had enough time to answer that one. Perhaps it's because he's spending too much time writing books, claiming poverty, selling homeopathic preparations and holidaying in Italy.
Here's a list of topics (some already mentioned on this thread) that I sent Ullman in August when he invited me to, "spell out what issues" I thought he had left hanging in the discussions that have been had with him in recent times:
"1. You never conceded that you were in error when you said, "... Perhaps, SOMEONE can finally tell us which were the 21 homeopathic trials and the 9 allopathic ones. Shang NEVER divulged, most likely because this review would show real benefits from homeopathic treatment. Isn't anyone suspicious of "black box" comparison studies like this? Why are only the homeopaths complaining here about junk science? Hmmmm. ..."
2. You have again asserted that two trails by Jacobs were excluded from Shang et al's analysis, when as we have seen, they were not.
3. You were invited some time ago over at JREF to, "GIVE ONE, YOU ONLY NEED ONE, INCONTROVERTIBLE EXAMPLE, WITH REFERENCES, OF HOMEOPATHY CURING A NON-SELF-LIMITING CONDITION"."
3. After talking up the Rao et al paper on possible evidence for the 'memory of water' you - much like the authors - never addressed yourself to the critique by Kerr et al published in Homeopathy.
5. We also had a brief exchange on the Oscillococcinum trials before. You never did get back to me on that one.
6. Last, but by no means least, you also left me hanging on the assertion that "silica has a tendency to store and broadcast information."
Wilsontown also suggested that he, "would like to see Ullman concede that it is not true that Shang et al. "didn't review ALL of the high calibre research but only a small part of it"."
Unless Ullman is prepared to admit his errors and answer questions that are asked of him, there is - to my mind - no point debating further with him. The internet will just be littered with more fragments of half-truths and failed arguments.
"Bravely taking to his feet
He beat a very brave retreat..."
I am inclined to agree with Mojo and others that the Milgrom Quantum Mechanics interpretation is rather stretched - but that has NO bearing on the ultimate theoretical validation of Homeopathy but merely represents, pardon me, a potential dead end, or, should Milgrom come up with more substantiation, the opening of intriguing new possibilities. Again, the issue is unresolved, requires more research and NO conclusion can be made at this time. The insult to your common sense, OR TO MINE, is totally insufficient to draw any conclusions,
even with regard to the likelihood or lack thereof of this possibility.
I will repeat the analogy with the Pyloric ulcer research - standard science said that there COULD NOT be those pesky bacteria in the acidic pH of the Stomach and that therefore the researchers were fundamentally wrong and wasting everyone's time.
Don't tell about how quickly their research confirmed the existence of the bacteria as compared to Homeopathy research results because nobody swills bacteria, out of sheer desperation, unless they were STILL being met with pretty serious skepticism.
I will admit one of my pet disagreements with Quantum Mechanics, an idea which I find utterly preposterous and nonsensical, namely the "many worlds" theory. NOTICE, however, that I do NOT run around shouting that all Quantum Mechanics physicists who espouse this idea are quacks and charlatans and that research should be cut off. MORE research is needed and in time the validity or falsehood of this theory will be made.
You all have been QUITE unreasonable with Dana, seeming to jump on him for not producing explicit proof of Homeopathy theory when all agree that such research is ongoing and has no definitive conclusion YET.
ALL he need do, and he has done so numerous times, is to produce evidence that some curative effect has occured and been validated in research, and that some research is underway to investigate those effects, which it is. He has in fact illustrated far more than that but closed little minds can't handle it, apparently.
I have a suggestion. Take the same attitude and apply it to Orac's field, oncology. Make him provide definitive double blinded randomized placebo controlled experiments for chemotherapy and cancer surgery (ah but there ARE NONE, are there? for that would mean placebo treatment for part of the test subjects). Let's look at the success rates, and the rates of cancer recurrence. Let's see the EVIDENCE for that and see if IT can measure up to the standards of Homeopathy which, when all is said and done, does NOT leave the patient's immune system so badly compromised that the common cold can kill them.
Once you have recognized the double standard at work here, then it will be time to admit, YES, there is reason to investigate Homeopathy and get at the bottom of what is occurring, even if ends up putting Orac's field of endeavor in a very bad perspective indeed.
It would seem that there is an elephant AND a rhinoceros in the room.
James, when you say, "You all have been QUITE unreasonable with Dana, seeming to jump on him for not producing explicit proof of Homeopathy theory when all agree that such research is ongoing and has no definitive conclusion YET" you miss an important point: Dana makes his living by claiming that homeopathy cures real diseases; on this basis he peddles books and potions. He supports these notions with misinterpreted and discredited studies. He's not saying, as you seem to be, suspend judgement until we have more facts. He's saying that the facts are there. It's little wonder that he gets challenged. Really, in the light of such claims, its only reasonable.
Neither will everyone agree that, "research is ongoing and has no definitive conclusion YET" Many people have reached the conclusion that it is clear that homeopathy is an elaborate placebo. For me, the consistant evidence of the recent meta-analyses that the better the study the more homeopathy tends to perform like a placebo suggest that more and better studies aren't warranted. Add to that the basic implausibility of the proposition and the answer is pretty clear: it doesn't work (beyond being a placebo) and there is no good reason that it should.
Finally James, wasn't it you who introduced Milgrom to the thread? If you don't think his ideas aren't the way forward why bother?
Ah, but he only introduced Milgrom so he could call us denialist fundamentalists (in BLOCK CAPS).
Ironic in view of the denialist response from homoeopaths to Shang etc. and their fundamentalist reliance on the writings of the prophet Hahnemann.
James wrote,
OK, and I'll repeat my point that this objection was overcome by the production of conclusive evidence of the existence of those pesky bacteria.
What happened to the "denialist fundamentalism"?
Pardon me apgaylard...
but each time Dana posts a riposte to the meta analysis claims,
as, for example, HERE:
Comments of Dana Ullman repeated here -
"How can anyone think that Wilsontown adequately responded to the concerns expressed above about the Shang review? Garbage in, garbage out. "
"Next time,
1) Please respond to the exclusion of the polyarthritis study.
2) Please respond with an explanation for why several studies that were defined as "high quality" by other researchers would not be defined as such by Shang (be specific).
3) Please respond to why Shang selected the primary outcome measure with "negative" findings when the studies found primary outcome measures that were "positive."
4) Please explain why Shang purposefully ignored the "adverse events" problems from conventional and homeopathic trials (could it have been because the conventional medical trials had too many adverse events?)."
End of Dana Ullman quote
there is no response.
Here is the problem with Orac, this blog and most of the posters here:
You folks all seem to be looking for the KNOCKOUT blow against Homeopathy. It keeps changing from one moment to the next - when the Homeopaths mention the double blinded tests, the anti-Homeopaths start up on the meta-analyses. When the holes in that are pointed out,
it's something else, then something else and again, and again and again.
Why don't you just admit that THERE IS NO KNOCKOUT BLOW AGAINST HOMEOPATHY and that research must be allowed to continue?
NOBODY responded about Ennis and her research.
Why not just admit that she discovered something which needs more investigation and for which no theory yet exists?
Note to Mojo - the objection against the pyloric researchers was indeed overcome by their production of conclusive evidence - are you willing to wait for Homeopathy researchers to do the same or would
you like to shut it all down right now? If the latter, how can you possibly know, in advance, that they won't make a key discovery.
The evidence at the moment points strongly towards it not working, and it would require a lot of what we already know to be wrong. Until there is good evidence that homoeopathy works, it is reasonable to conclude that it doesn't.
We've already waited for "the homeopathy researchers" to produce conclusive evidence (or for that matter some good quality research) for quite some time. Homoeopathy is over 200 years old, and meta-analyses (from Kleijnen 1991 onwards) that homoeopaths claim support homoeopathy go no further than to suggest that good quality research is needed. Good quality research supporting homoeopathy
I have never demanded that research into homoeopathy should be shut down. On the other hand, the current evidence would not appear to justify spending public money on it. If homoeopaths wish to persuade people otherwise, they should try to conduct some good quality research. the fact that they seem to prefer to produce stuff like this, to claim that the effects of homoeopathy cannot be detected in controlled trials, or to produce wild handwaving about quantum mechanics and the memory of water is, perhaps, rather revealing.
By the way, I suspect that the reason that people haven't bothered responding to Dana's points is that he doesn't appear to have bothered to read previous responses to the same points.
Sorry - the last sentence of the second paragraph of that post should read "Good quality research supporting homoeopathy has not been forthcoming: in fact it appears from recent research that good quality research fails to support homoeopathy."
James, perhaps you are not aware that these questions could all be answered by reading Shang et al. This may explain why these 'concerns' haven't been picked up in much detail on this thread; though wilsontown has provided some answers to Ullman in a debate on his blog, which is linked on this thread.
As you seem to have missed this, I'll help you out and add some comments of my own - some of which are taken from previous exchanges that I've had with Ullman.
"1) Please respond to the exclusion of the polyarthritis study."
Wilsontown put it nicely on his blog, "You know why the polyarthritis study was omitted. There was no matching study of conventional medicine. The point here was to get two comparable datasets to which to apply a meta-analysis. It doesn't matter that the higher quality and large studies are not matched: the point is that those trials were drawn from comparable populations. So, by the design of the study, excluding this trial was not just sensible but necessary."
2) "Please respond with an explanation for why several studies that were defined as "high quality" by other researchers would not be defined as such by Shang (be specific)."
On methodological questions like this it's always worth reading the paper. Now, it seems like Ullman has on his mind the comments made by The Lancet and BMJ editors about papers they published by David Reilly and co-workers that didn't make it into Shang et al's "higher methodological quality" sub-set. The point I would like you to think about is this: we don't actually know what criteria The Lancet and BMJ editors were using, but we do know Shang et al's criteria: they are in the paper (p 728):
Now, if anyone is concerned about how the criteria differed it would be appropriate for them to research the matter. I'm sure that they could get it published in Homeopathy. It may be that Shang et al's approach could be criticised, but this needs to be done on the basis of evidence. Who knows, it may be the editors of The Lancet and BMJ who were using less rigorous criteria.
"3) Please respond to why Shang selected the primary outcome measure with "negative" findings when the studies found primary outcome measures that were "positive." "
The answer to this one is actually in Shang et al. Under the heading 'procedures' on page 727 the authors wrote:
You may not like the methods, but they are clearly described and if critiqued need to be assessed on the basis of evidence rather than being upset by the outcome.
"4) Please explain why Shang purposefully ignored the "adverse events" problems from conventional and homeopathic trials (could it have been because the conventional medical trials had too many adverse events?)."
It's not a matter of, "purposefully ignoring" adverse events: it's that these were beyond the scope of the study. If you remember the paper is called, "Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy". Furthermore, the authors specifically address this issue in the paper:
Now, as you can see, Ullman's questions are not much of a "riposte". He could have answered them for himself by reading the paper, as could you. It does show, however, that it is always best if people actually take the time to read papers before launching into criticisms.
Now, the list of questions for Ullman that I posted is a compilation of the issues that he has raised in fora, like this one, and failed to follow through. As you can see, they are mostly cases where he makes clear mistakes and doesn't see the need to put them right.
Talking about not answering questions, I see that you have studiously ignored the points I put to you.
1. If, ""research is ongoing and has no definitive conclusion YET" then why do homeopaths, like Ullman, claim that there is good evidence that their treatments work and sell them to clients?
I'd also like to extend this quesion a little further: If sceptics should wait for a definitive conslusion, why shouldn't they wait for one before they start peddling their wares?
2. Why bring up Milgrom, when you don't accept his central thesis that some sort of quantum theory 'explains' homeopathy? What is it about his error-riddled diatribe that you think is a worthwhile contribution to this debate?
to apgaylard:
Many thanks for taking the trouble to explain the details, it is interesting but... read on.
Re your point # 2: My brining up of Milgrom though I do not agree with his Quantum explanation -> Because it is speculative, a shot in the dark, an attempt, just like someone saying "yes the pH of the stomach is inimical to bacteria but is there some way they could circumvent this?"
There are all sorts of interesting things he has brought up speculatively regarding water structure, quantum explanations etc..
I don't have to agree with all of it to find it interesting.
In other words, THINKING OUTSIDE THE BOX is what it's all about.
Re: your point #1 in which you state "if research is ongoing with no definitive conclusions why are the Homeopaths such as Ullman (perhaps you really want to say "how dare the Homeopaths such as Ullman" don't you!) claim that there is good evidence that their treatments work and sell them to clients?"
CHRIST in heaven apgaylard and others, are you all deaf dumb and blind - the CLINICAL EVIDENCE IN FAVOUR OF HOMEOPATHY IS OVERWHELMING. My goodness, there are books, first hand client reports, anecdotal histories -> most certainly NOT proof but at the clinical level powerful stuff - are YOU going to tell me that utilizing this is somehow IMPROPER for Homeopaths to do while it is done ALL THE TIME by standard physicians. Heart surgeries, chemotherapy, knee replacements - don't you think clinical evidence is what the MD's and surgeons use MOSTLY in their work? If not then you have been seriously duped by this EVIDENCE BASED MEDICINE NONSENSE. And please don't waste another moment of time telling me about placebo effect, I'm not buying into that explanation either. Read on....
Why, up until several decades ago, the exact mechanism of ASPIRIN was remained unknown yet was prescribed millions of times - how? BY CLINICAL EVIDENCE. Currently a large number of psychiatric drugs are routinely prescribed, for example for depression, while their mechanism of action remains unknown and under research.
So... sorry apgaylard and others, I'm not buying the double standard stuff for one moment, if the MD's and surgeons and oncologists are doing it then it is certainly just fine for the Homeopaths to do it too.
Last but not LEAST, I found blogger Laugingmysocksoff dealt with the Shang study, and its IMPERFECTIONS quite well, you can read it
and weep, HERE:
http://laughingmysocksoff.wordpress.com/2007/11/26/put-a-sock-in-it/
James,
I take it that you no longer hold with the criticisms of Shang et al made on this thread?
As for your intemperate rant, may I remind you that you said, "You all have been QUITE unreasonable with Dana, seeming to jump on him for not producing explicit proof of Homeopathy theory when all agree that such research is ongoing and has no definitive conclusion YET". This hardly fits with your protestations of "powerful stuff" at the clinical level.
Please note that Ullman cannot provide, "ONE, INCONTROVERTIBLE EXAMPLE, WITH REFERENCES, OF HOMEOPATHY CURING A NON-SELF-LIMITING CONDITION"." Can you? If the "CLINICAL EVIDENCE IN FAVOUR OF HOMEOPATHY IS OVERWHELMING" you'd think that someone could? It's a clinical quesion after all. Please try to be at least a little consistent.
James,
The laughingmysocksoff blog just re-hashes a tired mix of the irrelevant and untrue. Here are some examples:
"They didn't explain how they chose the particular cut off point that they used to select the final 14 trials" - Untrue, please read the paper.
"...neither did they identify which trials these were,..." - irrelevant. They provided this nearly three years ago, four months after publication, in the same Journal as the original paper and the same issue that the critical letters appeared in.
"...neither did they provide any information on which to assess whether those trails were still matched." - Irrelevant, the key thing is that the sub-sets were drawn from similar populations.
"...Initial requests by other researchers to identify these trials were refused repeatedly and data finally permitting their identification was not made available until several months after." - Irrelvent, as previously explained. This is just crying over spilt milk.
As for the comment on pooling the eight the upper respitory tract studies, the authors said:
This is the point: the approach that "laughingmysocksoff" wants is exactly the type of study (small-scale meta-analysis) that Shang et al show is vulnerable to particular bias and so, likely, "cannot be trusted". The analysis of the eight upper respiratory tract infection studies is an example of this problem.
After reading the criticisms I was laughing my socks off. Please, James, if you are not prepared to read this paper for yourself then it's pointless debating this with you. You are just regurgitating other people's misconceptions.
apgaylard:
Please enlighten me, what is a non self limiting condition?
Thanks
JP
James, I'd define it as a condition that does not resolve with the passage of time, rest and adequate nutrition. Its been discussed at length here. The post that I've linked contains quite a few suggestions.
One that won't get better when treated with homoeopathy.
apgaylard:
I see, so you exclude stuff like MS which has a history of remissions and exacerbations and for which Homeopathy is well known to benefit.
Likewise Autism - I refer to the book "Impossible Cure" which I thought quite good too, I note that the author, a computer scientist, became so grateful and intrigued by the one and only method that got her son back, Homeopathy, that she eventually gave up her career and entered the field of Homeopathy.
But, by the rules you are establishing, that does not count, eh?
Likewise with the Shang study, the debate seems to involve proscriptions on the parameters that are or are not to be allowed.
You will pardon me from witholding judgment on that one and NOT allowing a dispute over a single article on meta-anaylsis to DOMINATE MY thinking with regards to the entire field of Homeopathy.
With regards to more serious "non self limiting" diseases we must recall that at least here in the states, Homeopathy was more or less destroyed as a profession by about the World War l period.
I can provide examples of Smallpox or of cholera but they are from the 1800's and are described mainly in Homeopathy books which I am sure you will of course also discount immediately as biased.
Or I can ask you to email Dr. Vithoulkas who claims not only to have cured such diseases entirely with Homeopathy but, because of the understandable skepticism, has taken to VIDEOTAPING the results.
But... there IS one exception I can think of, described by laughingmysocks off, involving a cholera epidemic in London,
in which the Homeopathic doctors had dramtically superiour results than the allopaths or as opposed to those who did nothing. The reason THESE PARTICULAR statistics MIGHT be of more use is that they were initially suppressed by the Health Agency, populated largely by the MD's of the era, but so severe was the result of the epidemic and so great the popular clamor for action that a member of Parliament, who had heard of the success of the Homeopaths, ORDERED the health agency to provide the missing statistics. The results were quite dramatic.
Doing absolutely nothing gave a death rate of around 60%. Going to an allopath for treatment, even with the primitive and in some cases injurious methods of that era brought the mortality rate down to somewhere roughly at 30-40%. But going to a Homeopath for treatment gave an astonishing 15% or BETTER result. These statistics smash the idea that the Homeopathy remedies were the same as doing nothing.
HERE is the report of one of the medical investigators...
from laughingmysocksoff's excellent blog at:
http://laughingmysocksoff.wordpress.com/2007/11/27/sock-horror-in-cho...
At the time of the outbreak, the LHH (London Homeopathic Hospital)
was inspected by Dr Macloughlin, one of the
medical inspectors appointed by the General Board of Health, who was a
sceptic. His report said:
"You are aware that I went to your hospital prepossessed against the
homÅopathic system, that you had in me in your camp an enemy rather than a friend... and I need not tell you that I have taken some pains to make myself acquainted with the rise, progress and medical treatment of cholera, and that I claim for myself some right to be able to recognise the disease, and to know something of what the medical treatment ought to be, and that there may, therefore, be no misapprehension about the cases saw in your hospital,
I will add that, all I saw were true cases of cholera, in the various stages of the disease, and that I saw several cases
which did well under your treatment which I have no hesitation in
saying would have sunk under other.
In conclusion I must repeat to you what I have already told you, and
what I have told everyone whom I have conversed, that although an allopath by principle, education and practice yet were it the will of Providence to afflict me with cholera, and deprive me of the power of prescribing for myself, I would rather be in the hands of a homÅopathic than an allopathic adviser."
Read more at the blog link listed above.
Well there you have it apgaylard, I know you will find this quite inadequate from the point of view of your theoretical considerations
and I hope you will pardon me from deviating from your ancient and unenlightened prejudice.
There is one last point - since I find YOUR arguments against Homeopathy quite inadequate, I DO HOPE that you will not be so rash as to ever agree with legislation outlawing MY CHOICE of it should I so desire.
thanks
jp
You've convinced me, JP. Should I find myself in a 19th century cholera outbreak, homeopathy shall be the first thing I turn to.
In seriousness, did the importance of clean drinking water in treating cholera not cross your mind? Pure water's one thing homeopaths don't often struggle for. :D
James Pannozzi said "Likewise Autism - I refer to the book "Impossible Cure" which I thought quite good too, I note that the author, a computer scientist, became so grateful and intrigued by the one and only method that got her son back, Homeopathy, that she eventually gave up her career and entered the field of Homeopathy."
What makes that books suspect is that her child was never officially diagnosed with autism. It was a self-diagnosis, there was no third party verification of the diagnosis before or after.
I forgot to mention that the book is just an anecdote, and the plural of anecdote is not data.
Now there is a non-self-limiting disease that Hannemann wrote about extensively: syphilis. I would really like to know how much progress homeopathy has made in dealing with this disease in the past two centuries. How does it compare with the use of antibiotics, and which is the more current standard of care for this disease.
Also, during a debate ( http://www.theness.com/neurologicablog/?p=41 ), Dr. Andre Saine made the claim that homeopathy is more effective than modern medicine with rabies, from the link: He even claims that homeopathy can cure rabies with 100% success. Rabies is almost 100% fatal, even with modern treatment, so this is quite an astounding claim." Do you have any proof of this? What evidence is there that homeopathy is effective for rabies, and would you treat a child who has been bitten by a bat with homeopathy instead of the typical prophylactic treatment?
James, Just a small clarification: I don't want homeopathy banned, just not funded by my taxes or taught in universities. I think you should be free to spend your money as you wish and make whatever choices seem good to you.
I don't allow a single meta-analysis to dominate my thinking. I've just been answering quesions that you and Ullman have raised. I'm just happy that you don't seem to have any specific concerns about Shang et al now. I allow my thinking to be guided by the overall balance of the trial data as captured in a series of recent analyses plus an understanding of chemistry and physics.
I remain happy that vaccination has eradicated smallpox and that cholera is now very well treated by conventional medicine (in contrast to the historical period you cite). As the Encyclopædia Britannica points out, "with proper modern treatment, mortality can essentially be prevented, with rates kept to less than 1 percent of those requiring therapy". It's also good to know that the disease is caused by a bacterium not a Miasma, as Hahnemannian doctrine would have it.
James, you seem blissfully unaware that the reason for the development of scientific methods and values was to avoid investigators misleading themselves. In your reliance on poorly written up historical reports and anecdotes you make your assessments of effectiveness vulnerable to a whole range of biases and expectation effects.
The reason that the Shang study, "seems to involve proscriptions on the parameters that are or are not to be allowed" is simply that they defined a specific question and a set of methods, along with the approach that they would take to analysing the data before they collected it. This is to ensure that bias is minimised.
Finally, I think that it's a tad ironic for someone desparately propagandizing for something based on magical thinking, mysticism, the arachaic doctrine of vitalism, tales of cures in the 19th century and disavowal of the methods and values of science to accuse me of, "ancient and unenlightened prejudice."
Gentlmen (and Ladies):
I am well aware that the book I cited was essentailly anecdotal and of course does not provide any proof of the validity of Homeopathy at the theoretical level - but elsewhere I did comment on clinical evidence and that is but one more.
Apgaylard is still going on and on about his precious Shang study.
Never has so much thought been wasted on so little result from such few tests - hardly the basis for the sweeping conclusions that apgaylard attempts to make from it.
Again, many of you are looking for the knockout blow against Homeopathy and refuse to see the truth that there IS NONE
and refuse even to entertain the notion THAT IT WORKS.
Like wise, an attempt has been made to evaulate Homeopathy based on so called non self limiting conditions. But the reality of patients is that many of them have MS, diabetes, heart problems, thyroid problems, in other words, chronic conditions for some of which standard medicine has inadequate treatments, or worse, no treatment at all. Even if Homeopathy worked on just these types of conditions, it would be of enormous value.
Note to Sharpie re pure drinking water -> everyone seems to have failed to notice that a standard MD of the era, knowledgeable of cholera and its treatment, took a close look at the London Homeopathic hospital and its cholera patients and declared their treatments viable, even superiour to the known treatments. I gave his exact quote. The little nothing pills APPEAR TO HAVE BEEN DOING SOMETHING, eh Sharpie? The problem to discover is how and why, IF SO.
I really like the various contortions used to explain away the Homeopathic success in these epidemics (did I not mention cholera epidemics in the U.S. with exactly similar results? and in other countries such as in Europe? ah well the interested reader can easily find for themselves).
Again I will reiterate that these discussions have proved absolutely nothing - there is NO knockout blow against Homeopathy and I could not begin to tell you how it works. CONCLUSION: We must leave it to the researchers.
I go farther and submit to you that the attempt to prematurely kill the research based on the specious reasoning held up as gospel here, is ANTI-SCIENCE and constitutes a dangerous movment towards medical totalitarianism - in other words a situtation in which innuendo rather than science can influence the course of scientific resarch. This is most unhealthy, pun intended.
James, it's Ullman and yourself that brought up the Shang study. If, "thought been wasted on so little result" then I'd say it's the spurious criticisms that you have been peddling that's the key to the time wasting. I guess that you've never read the paper. And again, "from such few tests" please remember that the work is based on 110 matches pairs of trials: these enanbled the most reliable "few" to be identified. As for me drawing, "sweeping conclusions", I hardly think so. I don't go beyond the authors observation that, "we have shown that the effects seen in placebo controlled trials of homoeopathy are compatible with the placebo hypothesis. By contrast, with identical methods, we found that the benefits of conventional medicine are unlikely to be explained by unspecific effects."
It's the sweeping dismissal of this and other work by you and your ilk that's the problem here. Still, I'm done debating with you - you clearly don't have a clue.
apgaylard et al:
Here, read this, quoted from laughingmysocksoff blog at:
http://laughingmysocksoff.wordpress.com/2008/09/15/the-anatomy-of-obses…
"The facts of the matter are:
i) The lack of plausible mechanism for any purported homeopathic effect does absolutely nothing to detract from the existence of the effect itself. We are far from omniscient.
ii) The imputation that there's no evidence of a homeopathic effect after biases, placebo effects and poor studies are accounted for is not incontrovertible fact. This position relies on circular argument. For example, per Shang et al
(begin quote from Shang paper)
"For example, for the eight trials of homoeopathic remedies in acute infections of the upper respiratory tract that were included in our sample, the pooled effect indicated a substantial beneï¬cial effect (odds ratio 0·36 [95% CI 0·26-0·50]) and there was neither convincing evidence of funnel-plot asymmetry nor evidence that the effect differed between the trial classiï¬ed as of higher reported quality and the remaining trials. Such sensitivity analyses might suggest that there is robust evidence that the treatment under investigation works."
(end quote from the Shang paper)
laughingmysocksoff continues:
"Whereupon they of course concluded, according to their initial premise, that in the absence of methodological deficiency, such positive results must all be due to 'bias'. This argument is a self-maintaining construct fabricated out of the premise that your "fact" i) constitutes valid reason to deny the validity of homeopathy, and has no independent validity."
end quote from laughingmysocksoff
http://laughingmysocksoff.wordpress.com/2008/09/15/the-anatomy-of-obses…
James, I thought that I was the one who was meant to be obsessed with this paper? I am not going to debate this with you - as you clearly haven't read the paper youself: you just persist in quoting the 'laughingmysocksoff' blog.
Anyway, just to help you out, here are a couple of pointers:
(i) When the existance of any actual effect is in doubt, as it is in this case, the contention is demonstrably false. I'd suggest that prior probability is important.
(ii) I have already covered this. You need to think about the evidence that the authors provided in their discussion of this example. Read the paper.
apgaylord:
You really expect me to attach some credence to a paper which, under the full guise of scientific reasoning, utilizes as one of the critera of exclusion, that the Homeopathic remedy showed efficacy?
You saw lauhingmysocksoff's analysis and quote of this and you continue to persist in this charade that the paper has some relevant importance and that I should study it in your vain hope that I will somehow see the light.
Are you aware that the paper was roundly criticized not just by those favorable to Homeopathy but by many others regarding its shoddy statistics, cardboard science and pretensions of relevance.
Oh no, apgaylard, it is YOU who do not have the clue and it is I who cease discussions of this worthless tripe.
"utilizes as one of the critera of exclusion, that the Homeopathic remedy showed efficacy?"
I have the paper in front of me now, and I can tell you that that is a flat-out lie.
James,
I haven't a clue why you linked to the "laughing my socks off" blog. Fantastic. Instead of a rational rebuttal of Shang and other trials, we merely find an embarrasing, "consistent, repeatable and predictable" evasion from the homeopaths to simple questions they clearly stated they could answer. Laugh? I nearly died!
James Pannozzi said "But the reality of patients is that many of them have MS, diabetes, heart problems, thyroid problems, in other words, chronic conditions for some of which standard medicine has inadequate treatments, or worse, no treatment at all. Even if Homeopathy worked on just these types of conditions, it would be of enormous value."
There are treatments, but no real cures for many of those conditions in real medicine. You have no evidence that homeopathy could do anything for those conditions, and there is no evidence that homeopathy would have any real value (except the sugar pills would be detrimental to diabetics).
Though I have seen claims that homeopathy works for both syphilis (one of the original miasms put forth by Hahnemann), and for rabies (a claim by an Andre Saine). I have asked for evidence to those claims that homeopathy is a valid treatment for both of those non-self-limiting conditions especially since Mr. Saine claims that homeopathy is better than the present standard of care for rabies. Please, show us real scientific evidence verifying Mr. Saine's claims.
If you don't have that... then just show us that syphilis is still being effectively treated with homeopathy without the use of modern antibiotics. Again, I ask for real scientific evidence and not a list of anecdotes.
James, I have dealt with the very weak "laughing my socks off" blog claims, and the others that you made. I see that you have now moved on to an even more stupid claim, addressed by wilsontown. I keep asking you to read the paper, not in the vain hope that you'll see sense, but to enable you to see for yourself that the criticisms you keep regurgitating are not valid.
You have yet to come up with a single, valid, sustainable criticism. It doesn't mean that the paper is beyond criticism - just that you have shown yourself incapable of making any of any worth.
If you actually read and understood the paper then you just might be able to come up with something. Unfortunately, it seems that you have no intention of doing so - as your last comment so eloquently demonstrates.
James, please feel free to provide references to the many criticisms of the Shang paper by those not, "favorable to Homeopathy" which you mention. I may actually read them!
It does seem the DU has left the building, as is usual at about this stage in proceedings.
In his absence, I'd like to add my voice to the chorus asking James to produce some of that allegedly "overwhelming" clinical evidence to produce "One, You Only Need One, Incontrovertible Example of Homeopathy Curing a Non-Self-Limiting Condition"
I think he has been given a good list of examples that would raise eyebrows even while we would still agree that a single case-story cannot constitute proof.
When I first met the weird little world of homeopathy several years ago, I assumed that its track record was so littered with apparent miracle cures that it was no wonder its believers had a tough time accepting the objective evidence that refuted their belief. In other words, while I could be sure they were wrong, I could at least respect them for making a quite reasonable error. It has been one of the most striking features of my interactions with them that there turns out to be so little to provide foundation for their unshakeable convictions. The best they can manage is usually of the form, "I took a remedy and my cold got better in 7 days and it should have taken 10 days". To sustain that belief in the face of such thin evidence must require a considerable effort of will.
Subject and verb need to agree, instead of "such studies ... tries to," and "first principles ... doesn't matter."
And recast sentences for flow. See Strunk & White.
No one likes a grammar and style dictator, myself included. These posts are sometimes written on the fly and late at night; so if there's the occasional cut and paste error in which I forget to fix little problems like that, so be it.
Sorry to resurrect a zombie thread, but I've just noticed this (I was looking for another comment that I thought might be here):
James wrote (my emphasis):
Can I hear you say "regression to the mean"?