More Tooth Fairy Science: Acupuncture does not improve in vitro fertilization success rates, no matter what acupuncturists say

Here we go again.

Oh, well. These things come in waves, and sometimes I have theme weeks. Right now, this week appears to be developing into a week of quackademic medicine involving dubious acupuncture studies. Yesterday, it was acupuncture for lymphedema after breast cancer surgery, a study coming right about what is rapidly becoming the Barad-dûr of cancer quackademia, Memorial Sloan-Kettering Cancer Center. How a hospital that is so awesome in every other way can have such a blind spot bigger than the Eye of Sauron, I don't know, but it does, and the result is a steady stream of embarrassing forays into quackademic medicine like yesterday's.

Of course, there is one institution that far surpasses even MSKCC in the power of its quackademic woo, and that is the University of Maryland, home to Brian Berman, king of acupuncture quackademia, and my Google Alerts did there job and, well, alerted me to a new meta-analysis published online late last week in the Journal of Human Reproduction Update. Berman is the corresponding author (of course!), and a research associate by the name of Eric Manheimer is the lead author, and together with other colleagues, they have produced yet another fine analysis of tooth fairy medicine entitled, The effects of acupuncture on rates of clinical pregnancy among women undergoing in vitro fertilization: a systematic review and meta-analysis.

Now, remember all the things I said about the utter lack of prior plausibility for acupuncture for treating lymphedema? You remember, how there's no plausible biological mechanism that would lead one even to suspect that sticking needles into parts of the body completely unrelated to the physiological mechanisms that result in lymphedema after mehanical interruption of regional lymphatics by surgery? All of that goes double—nay, triple!—for using acupuncture for infertility and improving the pregnancy rates after in vitro fertilization (IVF). I mean, seriously. Think about it. How on earth would sticking needles into the skin improve the odds of conception after this procedure? It wouldn't, and it doesn't. That doesn't stop acupuncturists and acupuncture apologists from heavily selling acupuncture as somehow managing to do just that, against all physiology and reality.

So here's how the systematic review is being sold:

Acupuncture, when used as a complementary or adjuvant therapy for in vitro fertilization (IVF), may be beneficial depending on the baseline pregnancy rates of a fertility clinic, according to research from the University of Maryland School of Medicine. The analysis from the University of Maryland Center for Integrative Medicine is published in the June 27 online edition of the journal Human Reproduction Update.

“Our systematic review of current acupuncture/IVF research found that for IVF clinics with baseline pregnancy rates higher than average (32 percent or greater) adding acupuncture had no benefit,” says Eric Manheimer, lead author and research associate at the University of Maryland Center for Integrative Medicine. “However, at IVF clinics with baseline pregnancy rates lower than average (less than 32 percent) adding acupuncture seemed to increase IVF pregnancy success rates. We saw a direct association between the baseline pregnancy success rate and the effects of adding acupuncture: the lower the baseline pregnancy rate at the clinic, the more adjuvant acupuncture seemed to increase the pregnancy rate.”

It's hard not to be a bit snarky here and say that if your clinic is doing well with its pregnancy rate, then obviously you don't need mumbo-jumbo. However, if you're not doing so well, maybe some bread and circuses will help.

So let's look at the study itself or, as I like to say, go to the tape (or journal, or whatever). Basically, Berman and company examined sixteen trials with a total of 4,201 participants that compared needle acupuncture administered within one day of embryo transfer to shame acupuncture or no treatment. They left out studies that examined electroacupuncture (a famous bait-and-switch form of acupuncture frequently mixed in with regular acupuncture). Well, actually, not exactly. Teir rationale for not using electroacupuncture studies was not what you would think, namely because they didn't want to mix acupuncture studies with studies involving electricity, which didn't exist at the time when acupuncture was allegedly invented. Rather, the rational was that these studies involved studying electroacupuncture as an alternative to conventional anesthesia during oocyte retrieval and the points are therefore not chosen to improve fertility bur rather to reduce pain. Electroacupuncture was okay, as long as its intent was pregnancy. I kid you not. Or, as the authors say:

We included only trials in which acupuncture involved the insertion of needles into traditional meridian points. The needles could be inserted into tender points in addition to the traditional meridian points, and the needles could also be electrically stimulated.

Such rigor.

In any case, the methods used were fairly standard meta-analysis methodology, and when they were through they had 16 randomized trials. Now here's the annoying thing. This is a negative study. Oh, the authors, as you can see from the press release, jump, jive, and wail to try to extract something positive out of it, but the bottom line is this. When they looked at the pooled studies, there was no statistically significant difference in pregnancy rates between the acupuncture and control groups. None. Nada. Zero. Zip. And any other word you can think of for "no" or "zero." Or, as the authors put it, there was "no statistically significant difference between acupuncture and controls when combining all trials [risk ratio (RR) 1.12, 95% confidence interval (CI), 0.96–1.31; I2 = 68%; 16 trials; 4021 participants], or when restricting to sham-controlled (RR 1.02, 0.83–1.26; I2 = 66%; 7 trials; 2044 participants) or no adjuvant treatment-controlled trials (RR 1.22, 0.97–1.52; I2 = 67%; 9 trials; 1977 participants)."

In other words, that's a negative trial. Acupuncture does not improve pregnancy rates. Did I say that enough times? Heck, consistent with what one would expect for an intervention that doesn't have an effect, the asymmetric funnel plot showed a tendency for the intervention effects to be more beneficial in smaller trials. We see the same thing in virtually all clinical trials, but especially in clinical trials of CAM modalities. Homeopathy, in particular, is notorious for demonstrating this effect. So is acupuncture.

Of course, in so-called "complementary and alternative medicine" (CAM) or "integrative medicine" trials, there's never such a thing as a negative study. So we have subgroup analysis, the better to seek statistically significant results in smaller pools of patients, where there might be more variability. All too often it's not clear whether that subgroup analysis is prespecified or cooked up post hoc. To Berman's credit, in this case, the subgroups were specified before the meta-analysis:

We conducted subgroup analyses on five clinical characteristics that might influence the effect of adjuvant acupuncture on clinical pregnancy success rates: (i) two acupuncture sessions or more than two; (ii) selection of meridian acupuncture points the same as the points selected in the first published trial (Paulus et al., 2002) that evaluated acupuncture as an adjuvant to embryo transfer, and which showed a large effect, or a modified version of this trial's acupuncture point selection protocol; (iii) control group clinical pregnancy rate (as an estimate of the baseline clinical pregnancy rate) dichotomized as higher [32% or greater, which is the European average of pregnancy rate per embryo transfer (de Mouzon et al., 2012)] or lower; the control group clinical pregnancy rate was also analysed as a continuous variable to test whether the relation was linear and consistent with the findings of the categorical analysis; (iv) explanatory trials conducted to test the effects of adjuvant acupuncture under controlled conditions in which the acupuncture was administered onsite at the IVF clinic or pragmatic trials conducted to test the effects of adjuvant acupuncture delivered off-site, which might better approximate every day, ‘real life’ conditions since most IVF clinics do not have onsite acupuncturists (Arce et al., 2005); and (v) trials that involved a treating acupuncturist who was judged as adequately experienced or not adequately experienced, with such judgments made by acupuncturist assessors who were blinded to the identities and results of the trials.

One wonders what objective criteria these acupuncturist assessors use to judge other acupuncturists as "adequately experienced." They also prespecified six "risk of bias" domains to be examined: random allocation sequence generation; concealment of allocation of randomization sequence; blinding of patients (i.e. use of sham control); blinding of embryo transfer physicians; incomplete outcome data; and unequal co-intervention. And guess what? These were nearly all negative, too. There was only one exception, which of course was touted in the press release above. Oh, the authors dance around several variables that they describe as "almost" statistically significant, but "almost" only counts in horseshoes and hand grenades (and nuclear weapons). To be honest, these subgroups weren't even that close to being statistically significant. If I were reviewing this paper, I would have told the authors to cut out at least a couple paragraphs worth of verbiage dancing around these topics, although it is somewhat interesting to note that one of these "almost significant" subgroups was whether or not the physician doing the IVF embryo transfer was blinded to the acupuncture status of the subject.

In any case, there was an inverse correlation between the baseline pregnancy rate observed in a study and the effect of acupuncture. Studies reporting greater than average pregnancy rates (32% or greater) showed no effect of acupuncture. In fact, the risk ratio for such studies was 0.90 [95% confidence interval 0.80 to 1.01]. That's almost, but not quite, a statistically significant negative effect on pregnancy rates! In contrast, studies for which the baseline pregnancy rate was less than 32%, produced a risk ratio of 1.53 [95% confidence interval 1.28 to 1.84]. Why is this? Who knows? The authors speculate that additional interventions have little or no value when pregnancy rates are already high, but acupuncture can help when pregnancy rates are low, but there are so many factors that determine pregnancy rates, including embryo selection, prevailing practice, number of embryos transfered, and many others. There could very well be a confounder there that the authors didn't pick up.

The authors seem to recognize this, if grudgingly and/or subconciously, in their conclusion:

For future trials, if the objective is to investigate the mechanism of IVF adjuvant acupuncture, and specifically whether any effect is mediated through the point-specific needle placement, or alternatively through a non-needling-related psychobiological placebo mechanism, both a sham control arm and a no adjuvant treatment arm would be necessary. A sham control arm might also be necessary for recruitment, if potential trial participants would be unwilling to be randomized to a no adjuvant treatment control (Smith et al., 2012). However, because it is difficult to assess whether or not a sham control intervention has acupuncture-specific effects that may increase IVF success rates, sham-controlled trials in this area can also potentially complicate the interpretation of the overall evidence (Manheimer, 2011). For addressing the more clinically relevant question of the total effects of acupuncture (i.e. specific needling effects plus any non-needle-related placebo effects) in contributing to any increases in IVF pregnancy success rates, a sham control seems unnecessary (Manheimer, 2011). This question about the need for sham controls may also apply to other invasive, difficult to blind adjuvant procedures evaluated in IVF RCTs (Holt et al., 2009).

This is of course, veyr silly. There is no need for further stuies. Acupuncture does not work. It does not improve the pregnancy rate. There is no physiological mechanism to think that it should, and this meta-analysis confirms it. The questions of whether a sham control intervention is needed or not, whether multiple treatments are needed or not, whether sticking to the meridians matters or not are all the equivalent of what Harriet Hall likes to call Tooth Fairy science.