Yet again, acupuncture does not work for menopausal hot flashes

Arguably, one of the most popular forms of so-called "complementary and alternative medicine" (CAM) being "integrated" with real medicine by those who label their specialty "integrative medicine" is acupuncture. It's particularly popular in academic medical centers as a subject of what I like to refer to as "quackademic medicine"; that is, the study of pseudoscience and quackery as though it were real medicine. Consider this. It's very difficult to find academic medical centers that will proclaim that they offer, for example, The One Quackery To Rule Them All (homeopathy). True, a lot of integrative medicine programs at academic medical centers do offer homeopathy. They just don't do it directly or mention it on their websites. Instead, they offer naturopathy, and, as I've discussed several times, homeopathy is an integral—nay, required—part of naturopathy. (After graduation from naturopathy school, freshly minted naturopaths are even tested on homeopathy when they take the NPLEX, the naturopathic licensing examination.) Personally, I find this unwillingness of academic medical centers that offer naturopathy to admit to offering homeopathy somewhat promising, as it tells me that even at quackademic medical centers there are still CAM modalities too quacky for them to want to be openly associated with. That optimism rapidly fades when I contemplate what a hodge-podge of quackery naturopathy is and how many academic integrative medicine programs offer it.

If you believe acupuncturists, acupuncture can be used to treat almost anything. Anyone with a reasonable grasp of critical thinking should recognize that a claim that an intervention, whatever it is, can treat many unrelated disorders is a huge red flag that that intervention is almost certainly not science-based and is probably quackery. So it is with acupuncture; yet, that hasn't stopped the doyens of integrative medicine at the most respected medical schools from being seduced by the mysticism of acupuncture and studying it. I can't entirely blame them. I must admit, there was a time when even I thought that there might be something to acupuncture. After all, unlike so many other CAM interventions, acupuncture involved doing something physical, inserting actual needles into the body. However, as I critically examined more and more acupuncture studies, I eventually came to agree with David Colquhoun and Steve Novella that acupuncture is nothing more than a "theatrical placebo."

As I've said before, if you look at the evidence for acupuncture critically and in its totality, acupuncture has no specific effects and no effects distinguishable from placebo. Indeed, it doesn't matter where the needles are inserted, in "real" or "sham" acupuncture points, or even if the needles are actually inserted in the skin (a point that will become important later in this post). It's all the same. Basically, acupuncture is the gateway drug of "complementary and alternative medicine" (CAM) that is often a harbinger of the harder stuff, like naturopathy and homeopathy. But, if you believe its adherents, acupuncture can work for practically any symptom caused by pretty much any condition.

Acupuncture and menopause

One of the most popular uses for acupuncture is to treat menopausal symptoms. I'm familiar with this literature because the anti-estrogen drugs used to treat hormone-responsive breast cancer nearly always produce menopausal symptoms, and it is these symptoms that all too often hurt patient compliance with treatment and sometimes even lead patients to discontinue the drugs. Indeed, among practitioners of "integrative medicine" (the art of integrating quackery with real medicine), there's a great deal of enthusiasm for acupuncture for menopausal symptoms, particularly hot flashes, which can be very troublesome for both women undergoing natural menopause and even more so for women experiencing medication-induced menopause as part of their breast cancer treatment. Worse, in the case of cancer treatment, oncologists can't use supplemental estrogen to treat them, hence the search for other treatments. Never mind that clinical studies have been consistently unimpressive. None of this, however, prevents acupuncture advocates from continuing to do clinical trials. In fact, in just the first quarter of 2016, there have already been published two major studies examining acupuncture as an intervention to relieve menopausal symptoms. They are studies that support what I have been saying all along: The more rigorous the study, the more likely it is to find no specific effect greater than that of placebo.

Not long ago, I discussed a rigorous clinical trial. Like pretty much all rigorous clinical trials, it was a negative clinical trial. It failed to find an effect greater than placebo due to acupuncture in relieving menopausal hot flashes. Same as it ever was.

More recently, I saw another study. This one turns the normal rationale for using a good "sham" or placebo intervention in a clinical trial on its head in a way that would be funny if so many academic medical centers weren’t doing studies just like it..

There's already sufficient evidence out there (e.g., studies like this one) to show that acupuncture doesn't work for menopausal symptoms in breast cancer patients undergoing treatment with anti-estrogen drugs and/or chemotherapy (which can also induce menopausal symptoms); that is, other than studies that don't bother to blind anybody. This has not prevented, for example, the Society for Integrative Oncology from including acupuncture in its clinical guidelines for breast cancer patients. Nor did it stop what is considered to be one of the top journals for oncology, The Journal of Clinical Oncology (or, as we affectionately call it, JCO), from publishing a study like this one conducted in Italy, "Acupuncture As an Integrative Approach for the Treatment of Hot Flashes in Women With Breast Cancer: A Prospective Multicenter Randomized Controlled Trial (AcCliMaT)," by Lesi et al from the Unit of Medical Oncology Civil Hospital. The study was published as an Epub Ahead of Print late last month and even featured a a couple of weeks ago in the ASCO Connection, which is the news outlet of the American Society of Clinical Oncology, one of the largest oncology professional societies in the world.

AcCliMaT is what we in the biz like to call a "pragmatic" trial. Those who have been regular readers of SBM know what "pragmatic" means in this context. Basically, pragmatic trials are clinical trials designed to test how well an intervention that has already been validated in rigorous randomized controlled trials (RCTs) works in "real world" situations. The key assumptions behind a pragmatic study design are that we already know that a treatment works and that we are examining how well it works outside the tightly controlled environment of a clinical trial. Why do pragmatic trials? Simple. It's because often in the real world treatments don't work as well as they do in RCTs due to many reasons, including, for example, application of the treatment to patients who wouldn't have met the inclusion criteria of the RCT, less rigorous adherence to the treatment protocol, and patient noncompliance.

Of course, doing a pragmatic trial is putting the cart before the horse for treatments that have not been shown to work through rigorous RCTs, but that's exactly what AcCliMaT does. In fact, the rationale for AcCliMaT almost made me spew the iced tea I was drinking as I wrote this all over my MacBook Pro's screen. Check out the interview with one of the study's co-authors, Giorgia Razzini, in ASCO Connection:

According to study coauthor Giorgia Razzini, PhD, the strength of the study published in JCO is its rigorous and pragmatic design. Previous studies have compared acupuncture for hot flashes to "sham" or non-optimal acupuncture, with the goal of controlling for the therapeutic response that might arise from patients receiving increased attention and a laying on of hands (i.e., the placebo effect). However, this study compared acupuncture to the treatment women with breast cancer would actually receive in clinic, that is, self-care. To further strengthen the study's design, the acupuncturists paid great attention to minimizing behaviors that could potentially contribute to a placebo effect.

"The acupuncturists worked with a multidisciplinary team of oncologists and clinical trial managers to share their expertise and their knowledge and made an effort to standardize the acupuncture as much as possible to reduce the placebo factors present in many complementary medicines. In fact, acupuncturists delivered the intervention without talking to the patient, or at least keeping the talking to a minimum. The study was conducted with extreme attention to trying to minimize the placebo effect," said Dr. Razzini.

There's so much wrong here and only so much verbiage even I can devote to it. The key misconception at the root of this study is simple. Razzini seems to assume that placebo effects derive only from patient interaction, the "laying on of hands," and the like. No doubt these are important contributors to placebo effects, but it's nonsense to think that placebo effects can be minimized so much by not having the practitioners talk to the patients that a comparison of an intervention + self-care to self-care alone wouldn't still yield a positive result. It will, and, not surprisingly at all, in this study it did. Basically, this study had no good control group and failed to control for placebo effects, given that everyone involved was unblinded and knew which group each subject was in. It's also unclear what, exactly, was done to minimize interaction. All it says in the paper is this: "Conversation between acupuncturists and patients was kept to a minimum to limit nonspecific treatment effects."

So basically, this study was a randomized controlled trial of two interventions. The first is "enhanced self-care":

To standardize usual care recommendations, we enhanced self-care by providing all patients with a detailed information booklet about climacteric syndrome management. The booklet (Appendix, online only) was developed by the research team after a consensus process and included details about hot flashes and cancer and recommendations on diet, physical exercise, and eventual psychological support. The content of the booklet was explained to all patients before random assignment, and booklets were then distributed. Patients were asked to follow self-care recommendations for at least 12 weeks from random assignment.

So basically, the control group was given a booklet and told to follow its recommendations. I wonder how well that usually works out. (Hint: Not very.)

The experimental group received enhanced self-care plus acupuncture:

Within 2 weeks of random assignment, in addition to self-care recommendations, the acupuncture group was offered 10 traditional Chinese medicine (TCM) acupuncture sessions once per week for 12 weeks. The treatment protocol identifies six TCM menopausal syndromes according to Maciocia's recommendations (Table 1).²² At the beginning of each acupuncture session, a TCM evaluation of the tongue and radial pulses was performed to identify the prevailing syndrome and consequently choose appropriate acupoints in addition to three common acupoints (ie, SP 6, LI 11, CV 4). Patients received 20-minute acupuncture treatments based on the diagnosed TCM syndrome; moxibustion was provided as per TCM diagnosis. In some cases, supplementary points were punctured, but no more than 11 acupoints were used for each session. Sterile disposable 0.30 × 0.40 mm Huan Qiu needles (Suzhou Huanqiu Acupuncture Medical Appliance, Suzhou, People's Republic of China) were inserted bilaterally to a depth of 0.5 to 1 cm in most areas, except the hip, where the depth was 1 to 2 cm. They were manually manipulated to elicit the de qi sensation. No flicking or rotation of the needle took place once inserted. Participants with lymphoedema were not treated in the affected arm. No other complementary therapy was recommended during the course of acupuncture.

Here's a hint. In any study of nothing (in essence) versus nothing plus an intervention, the nothing plus intervention group will almost always come out on top, because it's not just the patient-practitioner interaction that is responsible for placebo effects. It's the intervention itself, the more invasive, the more powerful the placebo effects. It's not as though this isn't known and hasn't been studied, including how more costly sugar pills induce stronger placebo effects. It's even known that sham surgery is the most powerful placebo of all. This study was, whether intentional or inadvertently, designed to produce a positive response to acupuncture. It could hardly do otherwise.

And that's what happened.

A total of 190 women with breast cancer were randomly assigned. Inclusion criteria included, but were not limited to: diagnosis of breast cancer, age 18 to 65 years, intention to continue hormonal treatment at least through the length of the study, mean number of six or more hot flashes and/or daily mean score of 15 or greater on the Greene Climacteric Scale (GCS) during the week before enrollment, and vasomotor syndromes for at least 6 weeks. Some exclusion criteria included: treatment of climacteric syndromes such as systemic phytoestrogens, tibolone or analog, veralipride, or specific homoeopathic drugs and use of hormone replacement therapy and/or antidepressant drugs in the previous month and no intention to discontinue. The primary outcome measure was the hot flash score at week 12 (end of treatment), which was calculated as the frequency multiplied by the average severity of hot flashes. Secondary outcomes measured included climacteric symptoms and quality of life, measured by the Greene Climacteric and Menopause Quality of Life scales. Outcomes were measured for up to 6 months after treatment.

The results, which are utterly unsurprising given the trial design, are summarized in this graph:

The authors also found that women in the acupuncture plus enhanced self-care group reported a higher quality of life in terms of vasomotor, physical, and psychological symptoms, with most scores on the various measures being around 25%-30% improved compared to the control group.

Based on these utterly unsurprising results, the authors concluded:

In conclusion, AcCliMaT—a multicenter pragmatic trial with a standardized TCM acupuncture protocol—confirmed that acupuncture is an effective and safe intervention for severe menopausal symptoms in women with breast cancer. Further research could help to identify which variables predict treatment response and the optimal duration of acupuncture. Because these findings seem both statistically and clinically meaningful, we hope the practice of treating vasomotor symptoms in women with breast cancer will change.

No, what this trial found is that the acupuncture group did better than the control, but we have no way of knowing if this result was due to placebo effects or not because there was no sham acupuncture control. Of course, the authors know that their results don't show that acupuncture works better than sham acupuncture. They even admit it:

We acknowledge that this design does not allow estimation of the size of the effect resulting from needling itself or from other placebo-related factors, such as patient−provider interaction. Attempts were made to balance and control the potential impact of the latter by providing both groups with a booklet about climacteric syndrome management options and ensuring that therapists in the acupuncture group limited the communication and time spent with patients to the minimum needed for quality treatment. Notably, study participants did not differ in acupuncture response expectancy.

"Attempts were made…" Sure, utterly inadequate attempts. I could have saved the authors a whole lot of money and effort and predicted the results just by reading the study protocol. Sadly, no one asked me. I will say right now, though: Anyone thinking of designing a similar study, drop me an email or call. I'll predict the results. And JCO editors: The next time you get a manuscript describing a trial like this, consider me (or Steve Novella) as a reviewer. Your peer reviewers dropped the ball big time on this one, as did your publicity department for writing up a glowing article about this study.

It's not as though we didn't already know that acupuncture doesn't work for menopausal symptoms, either. For instance, a systematic review published in JCO in 2013 found no benefit for acupuncture for any cancer-related symptom other than chemotherapy-induced nausea and vomiting (CINV), all due to high risk of bias (ROB) in the studies included and evaluated. (Personally, I find the evidence that acupuncture helps with CINV unpersuasive, because it relied almost entirely on one trial with a low ROB and mixed electroacupuncture and acupuncture studies, but that's a topic for another post.) Similarly, a Cochrane review from 2013 (noted above in the discussion of the first paper) also found no benefit over placebo. Yet none of this stops acupuncture advocates from believing that acupuncture works to alleviate menopausal hot flashes, be they natural or breast cancer treatment-induced, or from doing more studies like the two I just discussed.

I’ve discussed on multiple occasions the ways that advocates try to make ineffective treatments seem effective and how they spin negative results as being positive, particularly how CAM advocates substitute low quality evidence for high quality evidence because high quality evidence shows that their interventions don't work. In these two studies, we have a perfect representation of this. The Italian study, whose design was decent for a pragmatic study but assumed what lawyers would call facts not in evidence (i.e., that acupuncture works for hot flashes), produced a seemingly "positive result." However, it did not adequately control for placebo effects. (An understatement!) When investigators do control for placebo effects adequately, as the Australian investigators did, the results show that acupuncture has no detectable specific effects greater than placebo. This is often much to the disappointment and consternation of the investigators, as the contortions the Australian group went through in its Discussion section illustrate rather amusingly.

Overall, there is no convincing evidence that acupuncture is anything but elaborate, theatrical placebo. The Australian study is quite consistent with this conclusion, and the Italian study provides no evidence to refute it. The Australian investigators, their attempts to explain their negative results notwithstanding, do deserve at least some praise for having followed the results of their study where they led. In contrast, the editors at JCO and ASCO itself should be ashamed for having published and promoted this study.