The objective of this trial, just published in the BMJ, was to assess the efficacy of manual acupuncture as prophylactic treatment for acupuncture naive patients with episodic migraine without aura. The study was designed as a multi-centre, randomised, controlled clinical trial with blinded participants, outcome assessment, and statistician. It was conducted in 7 hospitals in China with 150 acupuncture naive patients with episodic migraine without aura.
They were given the following treatments:
- 20 sessions of manual acupuncture at true acupuncture points plus usual care,
- 20 sessions of non-penetrating sham acupuncture at heterosegmental non-acupuncture points plus usual care,
- usual care alone over 8 weeks.
The main outcome measures were change in migraine days and migraine attacks per 4 weeks during weeks 1-20 after randomisation compared with baseline (4 weeks before randomisation).
A total of 147 were included in the final analyses. Compared with sham acupuncture, manual acupuncture resulted in a significantly greater reduction in migraine days at weeks 13 to 20 and a significantly greater reduction in migraine attacks at weeks 17 to 20. The reduction in mean number of migraine days was 3.5 (SD 2.5) for manual versus 2.4 (3.4) for sham at weeks 13 to 16 and 3.9 (3.0) for manual versus 2.2 (3.2) for sham at weeks 17 to 20. At weeks 17 to 20, the reduction in mean number of attacks was 2.3 (1.7) for manual versus 1.6 (2.5) for sham. No severe adverse events were reported. No significant difference was seen in the proportion of patients perceiving needle penetration between manual acupuncture and sham acupuncture (79% v 75%).
The authors concluded that twenty sessions of manual acupuncture was superior to sham acupuncture and usual care for the prophylaxis of episodic migraine without aura. These results support the use of manual acupuncture in patients who are reluctant to use prophylactic drugs or when prophylactic drugs are ineffective, and it should be considered in future guidelines.
Considering the many flaws in most acupuncture studies discussed ad nauseam on this blog, this is a relatively rigorous trial. Yet, before we accept the conclusions, we ought to evaluate it critically.
The first thing that struck me was the very last sentence of its abstract. I do not think that a single trial can ever be a sufficient reason for changing existing guidelines. The current Cochrance review concludes that the available evidence suggests that adding acupuncture to symptomatic treatment of attacks reduces the frequency of headaches. Thus, one could perhaps argue that, together with the existing data, this new study might strengthen its conclusion.
In the methods section, the authors state that at the end of the study, we determined the maintenance of blinding of patients by asking them whether they thought the needles had penetrated the skin. And in the results section, they report that they found no significant difference between the manual acupuncture and sham acupuncture groups for patients’ ability to correctly guess their allocation status.
I find this puzzling, since the authors also state that they tried to elicit acupuncture de-qi sensation by the manual manipulation of needles. They fail to report data on this but this attempt is usually successful in the majority of patients. In the control group, where non-penetrating needles were used, no de-qi could be generated. This means that the two groups must have been at least partly de-blinded. Yet, we learn from the paper that patients were not able to guess to which group they were randomised. Which statement is correct?
This may sound like a trivial matter, but I fear it is not.
Like this new study, acupuncture trials frequently originate from China. We and others have shown that Chinese trials of acupuncture hardly ever produce a negative finding. If that is so, one does not need to read the paper, one already knows that it is positive before one has even seen it. Neither do the researchers need to conduct the study, one already knows the result before the trial has started.
You don’t believe the findings of my research nor those of others?
Excellent! It’s always good to be sceptical!
But in this case, do you believe Chinese researchers?
In this systematic review, all RCTs of acupuncture published in Chinese journals were identified by a team of Chinese scientists. An impressive total of 840 trials were found. Among them, 838 studies (99.8%) reported positive results from primary outcomes and two trials (0.2%) reported negative results. The authors concluded that publication bias might be major issue in RCTs on acupuncture published in Chinese journals reported, which is related to high risk of bias. We suggest that all trials should be prospectively registered in international trial registry in future.
So, at least three independent reviews have found that Chinese acupuncture trials report virtually nothing but positive findings. Is that enough evidence to distrust Chinese TCM studies?
Perhaps not!
But there are even more compelling reasons for taking evidence from China with a pinch of salt:
A survey of clinical trials in China has revealed fraudulent practice on a massive scale. China’s food and drug regulator carried out a one-year review of clinical trials. They concluded that more than 80 percent of clinical data is “fabricated“. The review evaluated data from 1,622 clinical trial programs of new pharmaceutical drugs awaiting regulator approval for mass production. According to the report, much of the data gathered in clinical trials are incomplete, failed to meet analysis requirements or were untraceable. Some companies were suspected of deliberately hiding or deleting records of adverse effects, and tampering with data that did not meet expectations. “Clinical data fabrication was an open secret even before the inspection,” the paper quoted an unnamed hospital chief as saying. Chinese research organisations seem have become “accomplices in data fabrication due to cutthroat competition and economic motivation.”
So, am I claiming the new acupuncture study just published in the BMJ is a fake?
No!
Am I saying that it would be wise to be sceptical?
Yes.
Sadly, my scepticism is not shared by the BMJ’s editorial writer who concludes that the new study helps to move acupuncture from having an unproven status in complementary medicine to an acceptable evidence based treatment.
Call me a sceptic, but that statement is, in my view, hard to justify!
17 Responses to Does acupuncture prevent migraine attacks?
Leave a Reply to Richard Rawlins Cancel reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
only just saw this review of the same study
https://www.news-medical.net/news/20200325/Does-acupuncture-relieve-migraines.aspx?utm_content=buffere873b&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer
Very many thanks for this perspicacious review.
There are a lot of technical details to this paper which are beyond me!
And thank you for referencing Dr Liji Thomas’ paper which concludes:
“In the face of a small and limited study, such recommendations seem to underline the fact that the secret agenda of such research is to help change the perception of acupuncture as an unproven tool used by complementary medicine practitioners to one which is based on clinical evidence.”
The BMJ’s editorial writer, Dr Helen Angus-Leppan MD FRCP, should be castigated for having suggested that “Xu and colleague’s findings provide a solid evidence base for a non-pharmacological treatment often dismissed as an unproven complementary therapy…we now have good evidence that acupuncture is an effective treatment for episodic migraine”, and for suggesting that “acupuncture might be a first choice for people who want to avoid pharmacological treatment…”.
Clearly such an opinion is contentious.
Dr Angus-Leppan comes to those conclusions whilst acknowledging that “the effects of acupuncture are modest (2.1 fewer migraine days per month in the current study), and it is difficult for clinicians to know whether this level of benefit would be noticeable to patients.”
If not noticeable, it can hardly be said there is any benefit.
Dr Angus-Leppan knows perfectly well what the consensus on acupuncture is, and that ‘extraordinary claims require extraordinary evidence.’ After centuries, it is remarkable that here is a paper with the results as now published. It is more extraordinary that the BMJ published such a supportive editorial without more balance.
Dr Angus-Leppan’s comment that “Given that almost 90% of people with frequent migraine have no effective preventative treatment, acupuncture provides a useful additional tool in our therapeutic armoury” demonstrates the logical fallacy of non-sequitur:
The study confused ‘migraine prophylaxis’ (prevention of migraine) with therapy of patients experiencing migraine. Its conclusions were that “manual acupuncture was more effective than sham acupuncture and usual care in reducing migraine headaches.” No mention of preventing them ab initio.
Dr Angus-Lappan’s reasoning may appear to be plausible and valid, but it is a logical fallacy.
Sigh.
And a bigger sigh for the BMJ which allowed an editorial with such confirmation bias.
I note at the bottom of the page you referenced that it states: “Provenance and peer review: Commissioned; not peer reviewed.”
As you may recall, I am not a scientist so I would like to confirm what this means. On the surface, it looks to me like a “second set of expert eyes” has not reviewed the paper. Further, it appears that someone (company, organization, university?) paid to have this study done. If so and depending on who paid for it, is’t there the potential for a conflict of interest?
As always, thanks to you and the rest of those who comment for the education.
partly correct, I think.
it means the paper was invited [‘commissioned’] and was not checked by outside reviewers.
I was often invited by the BMJ to contribute an editorial but, as far as I remember, these were ALWAYS sent out to peer review. so, the footnote is remarkable, I think.
I have the BMJ 28th March 2020, page 488 in front of me.
The phrase ‘Provenance and peer review: Commissioned ; not peer reviewed’ does not appear in the printed BMJ.
Is it the paper which was commissioned, or the editorial (which surely all editorials are)?
There are many thousands of papers emmanating from China. On what basis was this one chosen?
Confirmation bias by the BMJ editor, or Dr Angus-Leppan? She of the minimally applied critical faculty.
“New trial moves acupuncture from complementary therapy to evidenced-based treatment.”
An opinion based on one paper, reporting treatment at ‘true acupuncture points’ – but with no indication of what or where they are.
This editorial cannot be taken seriously. More work needed!
“Is it the paper which was commissioned, or the editorial (which surely all editorials are)?”
editorials may be commissioned but they are usually peer-reviewed, in my experience.
It’s a pity they didn’t have another arm sticking needles in “non true” acupuncture points.
My understanding, though, is that the acupoint count is now up to around 2,000. Is there any spot on the human body that is not an acupoint? And they can’t agree on which one does what. These days, you get acupuncture for sleep problems and you might get your athlete’s foot cured by accident.
true, there is hardly a spot on the body’s surface that is NOT an acupuncture point. many people imagine an acupuncture point to be tiny, perhaps a square mm; in fact, it is said to be about the size of a 50p coin. this means that, even with some 300 points, most of the body surface is well covered.
At that rate, you could go in to get acupuncture for a sleep problem and accidentally cure your neighbour’s athlete’s foot.
only if he is not in self-isolation
Yes. The cognitive dissonance I experienced type “true acupuncture points” brought on a migraine. Now – where are those needles.
Xu et al. Three-armed RCT. 150 patients suffering migraine without aura. Found manual acupuncture to be significantly superior to sham and standard treatment. Successful blinding. Repeat: migraine without aura.
Heather Angus-Leppan cites says the only other acupuncture study to have demonstrated successful masking of sham acupuncture is Wang et al. Evidence Based Complementary Alternative Medicine 2015.
That 50 patient study was not limited to migraine without aura and found “real” acupuncture to be significantly superior to sham.
There is at least one other successful masking of sham acupuncture not limited to migraine without aura: Hans-Christoph Diener, GERAC Migraine Study Group, Lancet Neurology 2006.
This GERAC RCT enrolled 835 patients and found no significant difference between true and sham acupuncture or standard treatment.
This was a damning result for “true” acupuncture and standard treatment. A great result for fake treatment.
Xu’s results might suggest to acupuncture acolytes that a large scale trial for patients with migraine without aura might be worth considering.
The problem is that positive results of TCM trials in China today are worthless. Xi Jinping has spoken. TCM is a national treasure. Positive results are guaranteed.
It is a negative result. The “primary outcome,” is 10 sets of data – number of attacks and number of affected days in each of 5 intervals.
The manual acupuncture group was only “significantly better” than the sham group in 3 of those data sets. It was not significantly better during the treatments (20 treatments during weeks 1 to 7) or the first 5 weeks after treatment ended (weeks 8 to 12) and number of attacks wasn’t better during weeks 13-16. There is no reason to expect it to take 12 to 16 weeks before a 7 week course of acupuncture begins to have real effects, so the result is negative.
Revisiting this topic after checking out the latest Cochrane Review for acupuncture for migraine prophylaxis: Linde et al 2016.
The latest review amounts to a revision of its predecessor. The revised review found a small benefit compared to sham. The previous review did not. This difference is essentially due to the addition of one new study from China:
Acupuncture for migraine prophylaxis: a randomized controlled trial. Ying Li et al CMAJ 2012.
Li et al concluded that acupuncture “appeared to have a clinically minor effect on migraine prophylaxis compared with sham acupuncture” for a limited period after treatment. They found no benefit immediately following treatment, which was the primary outcome for the trial.
The trial has some problems.
One problem is that traditional Chinese acupuncture including manual stimulation of needles was tested together with subcutaneous electrical stimulation. These two treatments were tested against sham acupuncture (needling at non-acupuncture points with no manual stimulation) and electrical stimulation.
We see two distinct differences between the treatments: True acupuncture includes true acupoints and manual stimulation; fake acupuncture includes fake acupoints and no manual stimulation. In the event of differences being found in outcome we cannot be sure whether the cause is the incorrect acupoints or the lack of manual stimulation or both.
We also see that the true acupuncture group gets more treatment – the manual stimulation. In the event of greater benefit being found for the acupuncture group how do we know whether the manual stimulation is providing a specific benefit or whether it is turbo-charging the placebo effect?
The manual stimulation gives rise to another problem. A blinding problem.
The practitioners providing the treatments know whether their patient is receiving a key component of true acupuncture or not. So may those who receive or do not receive true acupuncture.
Manual stimulation is successfully confirmed by patients who experience “de qi” sensation. In China, what is the level of public awareness regarding de qi in acupuncture? If not negligible then some patients in Li et al may have known whether they were or were not receiving traditional Chinese acupuncture in addition to electrical stimulation.
Linde et al found no problem with blinding in Li et al. I find problems with them both.
The evidence in support of acupuncture for migraine in the Cochrane series of reviews since 2001 has progressed from dire to dodgy to is-that-it?
How long, oh Lord, how long?
And again, this time a more detailed look at Xabai Xu et al.
The authors deceived themselves by believing they successfully blinded treatment and control groups by asking patients if they thought the needles had penetrated their skin. That was the wrong question. They should have asked which treatment patients thought they had received, true or fake acupuncture?
It is important to know how many patients guess correctly which treatment they received. A similar number guessing correctly and wrongly in treatment and control groups is evidence of successful blinding as to treatment. Similar numbers in respect of perceived needle penetration is evidence of a successful blinding needle.
The blinding results showed that the Streitberger sham needle worked well. Well done sham needle.
What about evidence that patients were blind as to treatment? Go whistle.
All that was needed was to ask patients what treatment they thought they had received. They might have asked whether patients experienced di qi and whether they believed they had real acupuncture. Then compare how many guessed their treatment correctly, who reported di qi, with those reported no di qi. There is evidence that di qi is associated with unblinding, which is hardly surprising. Lene Vase et al 2015
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4352029/
But there’s more.
The acupuncture group lay on their backs and were thus able to see the practitioner at work. They also experienced the sensations associated with de qi. The sham group lay face down. The practitioner hovered over them, out of sight. No de qi sensations, just some blind prickings and twistings upon their backs.
Interaction between patient and provider was distinctly different in character in the two groups. No eye to eye contact – however fleeting – no face to face expressions to read for the sham group. A less personal experience for the sham group. Subjective and physiological experiences for the two groups were different.
One group was getting less of a placebo dose than the other. If acupuncture is itself nothing other than a placebo then one would expect the sham group in this trial to have a weaker response. As was the case.
The trial was naive in the extreme. Add it to the pile in the next Cochrane Review. Dross upon dross, tomorrow and tomorrow…
One more thing worth mentioning.
Shabai Xu et al had three groups:
1. Real acupuncture + usual care.
2. Fake acupuncture + usual care.
3. Usual care alone.
An attempt was made to blind the acupuncture groups. I have already mentioned the defects in blinding.
The fact that the usual care group was certainly unblinded and the real acupuncture group could have been unblinded compared with the fake group, means that adding extra treatment to usual care may have produced stronger placebo effects. Firstly by puncturing or creating the illusion of having punctured the skin; secondly by stimulating the needles to produce di qi sensations.
IMO that is by far the most likely result given the complete absence in this study of any scientific explanation as to how acupuncture might be able to help. The proposed scientific explanation of western “medical” acupuncture is hand-waving at best and unfalsifiable at worst, proposing as it does that one can put the needles any old where and a prick on the skin may be enough to do the trick.
This trial was far from producing good evidence to support the use of acupunture for migraine. This trial was a dud.