I used to be of the opinion that there might just be something to acupuncture. No, I never thought there was anything to the notion that acupuncture "works" by somehow rerouting the flow of a magical life force (qi) that no scientific instrument can detect and that no practitioner of acupuncture (or other practioners "healing arts" that invoke qi or something like it as the reason that they can heal) can detect either, even as they claim to "release blockages" of or somehow improve its flow. Rather, I wondered whether the simple act of sticking needles into the skin might release some hormone, endorphins, or other active physiological messnger that accounted for the rather modest effects attributed to acupuncture. Another possible explanation for its apparent efficacy was that it was a form of counterirritant that caused pain to be sensed as less intense. Either way, I didn't consider it totally implausible that acupuncture might do something therapeutic through one or more of the good, old-fashioned physiologic mechanisms that I was taught in medical school.
That was then.
Over the last several months, however, I've been becoming more and more skeptical of acupuncture. Perhaps it's because I had rarely looked at the actual peer-reviewed medical literature about acupuncture before, while over the last few months I've delved more and more into it. As I did so, the the realization dawned on me just how bad most of the studies hyped as "proof" that acupuncture "works" are. I've even blogged about a couple of studies that either failed to show any effect of acupuncture greater than placebo, had serious flaws that invalidated the claims of the investigators, or showed that perception among patients of efficacy was not reality. Meanwhile, über-skeptic Steve Novella also schooled me a bit in why most studies of acupuncture are poorly designed and hyped by the press as "proving" that acupuncture "works" when the study shows nothing of the sort and how many studies of "acupuncture" are actually a case of misdirection in which electrical current is passed through the needles, making them in essence clinical trials of the efficacy trancutaneous electrical nerve stimulators (TENS), a perfectly acceptable "conventional" therapy for pain for which there is abundant evidence for efficacy.
Recently, a study on the efficacy of acupuncture for treating hot flashes in breast cancer patients was published in a high profile journal by investigators at one of the premiere cancer centers in the U.S., if not the world. What? You didn't hear about it? Neither did I, until I actually happened to be perusing the stack of my most recent journals. I wonder why it wasn't publicized.
This new study1 now tells me that acupuncture appears to be useless for hot flashes in breast cancer patients undergoing hormonal or chemotherapy. The study comes out of the Integrative Medicine Service at cancer powerhouse Memorial Sloan-Kettering Cancer Center (another one to add to my Academic Woo Aggregator, perhaps?) and was recently published in the Journal of Clinical Oncology, and its senior author was Barry Cassileth, the author of The Alternative Medicine Handbook: The Complete Reference Guide to Alternative and Complementary Therapies. I looked around for breathless news stories hyping this study. Oddly enough, I didn't find any, not even a press release on Eurekalert!. The reason was, I'm sure, that this was a negative study.
Apparently the mainstream media suffers from the problem of publication bias too.
Before I discuss the study itself, I wondered why one would hypothesize that acupuncture would be useful for hot flashes. Certainly, this is a problem that interferes with the quality of life of premenopausal women who undergo cancer therapy, sometimes so seriously that the occasional woman even decides that she would rather take the increased risk of recurrence of their cancer than put up with the symptoms. For some, the chemotherapy itself can put them into temporary menopause; while for many others it is the treatment with estrogen-blocking drugs like tamoxifen that induces a state of menopause for as long as the patient is taking the drug, which is usually for five years after their initial breast cancer therapy. Because of the increased risk of breast cancer, particularly in a breast cancer survivor, using estrogen replacement to alleviate menopausal symptoms is not usually a good option. Worse, various non-estrogen treatments, including herbal remedies, have in general not produced convincing evidence of efficacy, although Vitamin E appears to be able to produce a mild decrease in the number of hot flashes.
This is where plausibility comes into the picture. Acupuncture is proclaimed as a treatment for many medical conditions that are unrelated, with no common physiological mechanism to explain why sticking needles in the skin in specified ways would alleviate the symptoms of such a diverse group of clinical entities. In other words, unlike the claims of "individualized treatments," that purveyors of alternative medicine so like to make, in the world of acupuncture, it's one size fits all for the most part, the only leeway being how many needles are put in and where--all without a plausible scientific basis to suggest why any needles would do any good. Such is the case with acupuncture and menopausal symptoms. None of this, of course, keeps true believers from trying to apply acupuncture to virtually every condition known to man.
Be that as it may, this study, however shaky the the scientific basis under which it was done (and apparently done pretty well) appears to have been funded by the National Center for Complementary and Alternative Medicine. Moreover, it was carried out at one of the top two cancer centers in the U.S., an old and respected institution. The study design was a randomized, controlled subject-blinded trial. The primary endpoint was the number of hot flashes reported by patients per day. The meridians chosen were du mai, gallbladder, bladder, pericardium, heart, kidney stomach, and spleen at specific acupuncture points. How the acupuncturists chose these points, don't ask me. It isn't explained in the paper. What is explained in the paper is that this set of acupuncture points was different from the ones used in a pilot study:
When the study was initially started, 27 participants were accrued and were treated with a set of points different than those shown in Figure 1. At this point, there was a change in staffing. The new group of acupuncturists believed that a different set of points (Table 1) would be more appropriate. We prespecified that we would accrue sufficient patients on the new point prescription to meet the original sample size requirements and that our primary analysis would include only patients on the new point prescription. Seventy-two participants were accrued to this new regimen. Data presented here derived solely from those 72 participants.
Again, who knows how this was determined? However, the sample size and power calculations appear to be adequate, and the investigators did appear to accrue a large enough number of patients. I also note right here that the way in which the subjects were blinded was almost, but not quite, the optimal way:
In the true acupuncture group, the needles used were stainless-steel filiform, needles sized 0.20 x 30 mm and manufactured by Seirin Corp (Shizuoka, Japan). After sterile swabbing of the skin, needles were inserted 0.25 to 0.5 inches into the skin at the designated acupuncture points and were manipulated manually to obtain De Qi.31 No electrical stimulation or other interventions were applied.
In the sham acupuncture group, Streitberger sham needles sized 0.30 x 30 mm and manufactuered by Asiamed (Pullach, Germany)37 were applied a few centimeters away from the points listed in Table 1. Rather than penetrating the skin, the needle retracted inside its handle after insertion through an adhesive tape placed on a plastic supporting ring. This type of sham needle has been shown to have high participant credibility and has been successfully implemented in randomized, controlled trials.38 The frequency and duration of the sham acupuncture intervention were identical to those of true acupuncture. To ensure consistency in technique, the therapists were coached by a single acupuncturist who also observed treatments periodically for integrity. Participants randomly assigned to the control group were offered eight sessions of true acupuncture starting at week 7. To aid the blinding process, all patients were asked to relax on the treatment table, gentle music was played, and an eye pillow was offered. In both groups, needles were retained for 20 minutes and then were removed. Participants with lymphedema were not administered needles in the affected arm.
Although the best "placebo acupuncture" needles were used, this design leaves something to be desired. For one thing, the practitioners were not blinded, as they are in the best designs and as is possible using these retractable needles. For another, I can't figure out why for the "sham" acupuncture the investigators both used the retractable needles to stick the sham acupuncture points. What should have been done was to use the "true" acupuncture needles at the "sham" acupuncture points and to add an additional experimental group with in which the "sham" needles were used for the "true" acupuncture points. That, and blind the acupuncturists to the treatment group, at least for the patients in the groups getting either the "true" acupuncture of the "sham" needles in the "true" acupuncture points. That's what needs to be done if one really wants to test properly whether acupuncture "works," or not. Still, the design of this study was better than most acupuncture studies:
Note the crossover design. Such a feature of a clinical trial is usually placed there to see if switching patients from the placebo arm to the treatment arm results in an effect. Finally, the study relied on self-reporting of hot flashes. As an accompanying editorial states, this was somewhat odd, given that there has been a reliable objective means of measuring hot flashes for a long time, and most investigators now believe that both patient reporting and objective measurement should be used in the best designed trials. Be that as it may, the results of this study were--shall we say?--underwhelming. The mean number of hot flashes per day was reduced from 8.7 to 6.2 in the "true" acupuncture group and from 10.0 to 7.6 in the sham group, both with wide variability. "True" acupuncture was associated with 0.8 fewer hot flashes per day than sham at 6 weeks, but the difference did not even come close to statistical significance. When participants in the sham acupuncture group were crossed over to true acupuncture, a further reduction in the frequency of hot flashes was supposedly seen, but again this was not even close to statistically significant. The reduction in hot flash frequency was reported to have persisted for up to 6 months after the completion of treatment. In all cases, the confidence intervals for the difference in the number of hot flashes in the sham and "true" acupuncture groups encompassed zero. Given that there was a reduction of hot flashes in both the sham and "true" acupuncture groups, the most likely explanation for the results of this study is the placebo effect. Maybe we should just treat women with soothing music, dark rooms, and eye pillows.
Of course, the authors try to salvage something out of a resoundingly negative trial using a most unconvincing argument:
There are several possible explanations of our findings in light of the above previous studies. First, it may be that the interventions used as sham controls were not entirely inactive....
Second, symptom improvement may result from the natural course of symptoms or from the psychological impact of treatment. In drug trials with a placebo control, the placebo effect on hot flashes frequency can vary from 13% to 22%. Improvement in our sham acupuncture group (24%) is close to that range. All trials of symptoms in which a certain baseline symptom severity is an eligibility criterion will be subject to regression to the mean. It is also widely believed that psychological impact of receiving treatment--the time and attention from a practitioner, and the patient's belief that they will be helped--is of therapeutic value. Both true and sham acupuncture also may create a relaxation effect, which can reduce hot flashes...
The third possible explanation for our findings is that the acupuncture intervention may not have been optimal. This may be because the point prescription was inadequate. Most investigators select acupuncture points based on classical theory, previous research reports, and/or expert opinion. This is an inherent limitation of all acupuncture research, because there is no reliable and consistent way to determine what would constitute the ideal prescription.
If there is no reliable and consistent way to determine the ideal "prescription" of acupuncture, how, then, do acupuncturists figure out how to treat a given patient? Doesn't this last line alone send up huge red flags that the acupuncture emperor has no clothes?
Finally, the authors can't resist stating that they "can't rule out" the possibility that a longer course of acupuncture would have resulted in a statistically significant difference in the number of hot flashes in the trial subjects receiving "true" acupuncture. I suppose that's possible, but the patients received four weeks of acupuncture therapy. There has to come a point when enough is enough. Longer treatment would also make regression to the mean more of a concern that has to be controlled for. Finally, there is the question of whether the effects of acupuncture would be clinically useful even if the study did show a statistically significant decrease in hot flashes. As pointed out in the accompanying editorial, the reduction in hot flash frequency was to only around 20% below baseline at the start of the study for both the sham and "true" acupuncture groups. This is far below the 50% or greater reduction in hot flashes desired by most women suffering from them. In other words, even if acupuncture "worked," it would probably be clinically irrelevant.
In the end, this is a negative study, no matter how much its authors may try to spin it as anything other or speculate that acupuncture might have worked if they had continued the treatment longer. It's also the reason why you probably haven't heard about it, as you can bet that a major study like this from a research powerhouse like Memorial Sloan-Kettering Cancer Center published in a very widely read, high impact journal like the Journal of Clinical Oncology would have been trumpeted to every news outlet in the land if it had shown a statistically significant decrease in hot flashes in women in the "true" acupuncture group. It didn't; so it wasn't.
In any case, the speculation about whether a longer course of acupuncture would have yielded statistically significant results will, I predict, provide the investigators with a rationale to try to get more money from NCCAM for a followup study. It's your tax dollars hard at work.
More like this
Isn't one of the important hallmarks of science that a theory shows predictive power? A single study like this one is devastating (to the non-idiot) since it demonstrates that accupuncture does not actually control anything.
Another way of looking at this problem is that, strangely, accupuncture appears to never show any negative effects. Isn't that odd? I suppose qi unblocking is a one-way street, huh? Otherwise, the positive refutation of the whole concept would be to take a healthy person and have an accupuncturist give them a specific problem through selective qi-poking.
What a crock.
"It's your tax dollars hard at work."
This is what I'm concerned about. Perusing through the NCCAM website I'm expectedly annoyed. Can't we dissolve NCCAM already and redistribute that NIH money elsewhere? Have you posted about this already? In a way, I like that some scientific method is applied to these alternative medicines in order to debunk them, but in reality, there is not enough rationale to justify the grants. If NIAID funded grants on such rationale, I can just propose that greeb tea cures HIV because of anecdotal evidence? That would be appalling.
I fail to see why this should cause one to doubt anything other than the ridiculous bases on which "acupuncture" rests. So far as I am aware, the only known results of needling are local pain relief, independent of whether any "meridians" or "points" are used. That acupuncture should be useless for hot flashes, digestive problems, anemia, or hepatitis is entirely expected, given the nebulously-termed, certainly imaginary "meridians" and folk (often conflicting) histories of combinations of points used for vague ailments.
If one stops thinking of the historical "acupuncture" and only of the observed, reproduceable results, needling as a helpful, generally harmless treatment for local pain-- a sort of physical therapy-- seems to genuinely offer some non-narcotic promise. Trying to figure out whether "acupuncture" will treat chickenpox does not, but is not even relevant to the efficacy of philosophy-less needling as a technique.
I like your post, but let me make a criticism.
You're unfair in one respect: you're asking why in a few areas the authors didn't do more, e.g. add another few arms of the trial one with sham needles with real points, and one with real needles but sham points, and why they didn't use objective measures of hot flushes rather than subjective measures.
The probable answer is resource constraints. I'm sure the researchers would be more than happy if you forwarded some of your grant money to them to solve these problems.
The question should not be "how does this trial fall short of the Platonic Form of clinical trials", but "is the study methodology strong enough to support the conclusions". From your precis above it seems that it is.
On the contrary, another group may not have been necessary, although three groups would have been optimal. This study could have been done with real needles in the "real" acupuncture points and compared to "sham" needles in the real acupuncture points. It would have been more valid, and they were using the sham needles anyway.
In any case, it's entirely legitimate and not at all unfair to criticize shortcomings in a study's methodology, regardless of what the reasons might have been for them.
Hey, I just thought of something!!
How do accupuncturists diagnose a problem, in the first place? Suppose I come to them reporting a cancer of the left appendix. How do they know whether I actually have such a cancer or not? Upon what do they base their Qi unblocking? Surely, since Qi isn't measured by an MRI, they aren't actually trusting those goofy modern technologies, are they? Since accupuncture is presented as a complete theory of medicine, wouldn't it be fairly easy to do a test measuring Qi-based diagnosis?
What would happen if I walked into an accupuncturist and said I had liver cancer? Would the accupuncturist adjust my Qi to help my (nonexistent, I hope!) cancer? Would that result in a mis-adjustment of my Qi? Would that perhaps make me light headed, or urinate from my ear, or develop cancer of the big toe?
Diagnosis represents a huge hole that accupuncturists don't appear to be talking about. Are they practicing "medicine" solely based on what the patient presents with? Isn't that dangerous?
Marcus,
They would probably treat your symptoms and send you to an oncologist. The Chinese acupuncture treatments are based specifically on observed physiological conditions and symptoms, not necessarily your spoken diagnosis.
Well, here's another one for your list, from New Scientist Acuncture Relieves Cancer Chemotherapy Fatigue refering to an article in 2006 in Complementary Therapies in Medicine. These articles keep popping up, but, as you indicate, they never seem to have any staying power.
I don't want to be rude about this, and I welcome correction, but it seems that this posting is seriously flawed: in at least one respect it is not nearly harsh enough on those investigaors.
They give some explanations of why the sham treatment seemed almost as good as the real thing. There are three of them, as follows.
Shorter:
1. We really don't know how to do acupuncture.
2. There may be placebo effects.
3. Nobody knows how to do acupuncture.
Longer:
1. Acupunture involves in its essence sticking needles down subtle channels known only to Chnese medicine.
So, maybe the sham treatments, which did not stick needles down anything, also produced an effect? We don't know whether that might be true? Then we don't know acupuncture. We don't know whether acupuncture exists as a real classification of anything. That's what follows from their claim, and it's good enough for me; but now I owe you 2 more points.
2. There may be placebos and relaxation effects (another sort of placebo)? Gee, ya think so?? Perhaps before they get their next research grant, they could be asked to take a course in medical research so they can find out what a control is and why it exists?
Would it be possible to teach them that if your results are not significantly better than controls, then there is no reason to believe (and reason to disbelieve, if the statistical power was great enough) that your expermental treatment is anything but a placebo?
3. They don't know for sure that they really poked the right place. The poor guys have, by their own account, nothing but 17 godzillion years of Chinese civilization to go on, and who knows whether the treatments that they're giving people are properly directed at all?
Maybe they might develop techniques that work so well that they can depend on them, and then get a grant for critical experiments to prove their method conclusively? Almost like, you know, science.
I'd like to give this stuff the scorn it deserves and aim some clever snark at it, but it's beyond my literary powers. I'd have hoped that the contempt module in Orac was adequate to the task; perhaps it's due for routine maintenance? Or am I missing something?
duh! Natural medicine does not work in the presence of drugs, they simply do not mix. Acupuncture is such a gentle medicine it can be cancelled by the drugs that person is taking. When a person goes in for acupuncture they are Tested to see what needs to be corrected, the test tells what points need to be done. ANY study mixing acupuncture and drugs together is not worth even giving the time of day to.