This new RCT by researchers from the National Institute of Complementary Medicine in Sydney, Australia was aimed at ‘examining the effect of changing treatment timing and the use of manual, electro acupuncture on the symptoms of primary dysmenorrhea’. It had four arms:
- low frequency manual acupuncture (LF-MA),
- high frequency manual acupuncture (HF-MA),
- low frequency electro acupuncture (LF-EA)
- and high frequency electro acupuncture (HF-EA).
A total of 74 women were given 12 treatments over three menstrual cycles, either once per week (LF groups) or three times in the week prior to menses (HF groups). All groups received a treatment in the first 48 hours of menses. The primary outcome was the reduction in peak menstrual pain at 12 months from trial entry.
During the treatment period and 9 month follow-up all groups showed statistically significant reductions in peak and average menstrual pain compared to baseline. However, there were no differences between groups. Health related quality of life increased significantly in 6 domains in groups having high frequency of treatment compared to two domains in low frequency groups. Manual acupuncture groups required less analgesic medication than electro-acupuncture groups. HF-MA was most effective in reducing secondary menstrual symptoms compared to both–EA groups.
The authors concluded that acupuncture treatment reduced menstrual pain intensity and duration after three months of treatment and this was sustained for up to one year after trial entry. The effect of changing mode of stimulation or frequency of treatment on menstrual pain was not significant. This may be due to a lack of power. The role of acupuncture stimulation on menstrual pain needs to be investigated in appropriately powered randomised controlled trials.
If I were not used to reading rubbish research of alternative medicine in general and acupuncture in particular, this RCT would amaze me – not so much because of its design, execution, or write-up, but primarily because of its conclusion (why, oh why, I ask myself, did PLOS ONE publish this paper?). They are, I think, utterly barmy.
Let me explain:
- “acupuncture treatment reduced menstrual pain intensity” – oh no, it didn’t; at least this is not what the study proves; the fact that pain was perceived as less could be due to a host of factors, for instance regression towards the mean, or social desirability; as there was no proper control group, nobody can tell;
- the lack of difference between treatments “may be due to a lack of power”. Yes, but more likely it is due to the fact that all versions of a placebo therapy generate similar outcomes.
- “acupuncture stimulation on menstrual pain needs to be investigated in appropriately powered randomised controlled trials”. Why? Because the authors have a quasi-religious belief in acupuncture? And if they have, why did they not design their study ‘appropriately’?
The best conclusion I can suggest for this daft trial is this: IN THIS STUDY, THE PRIMARY ENDPOINT SHOWED NO DIFFERENCE BETWEEN THE 4 TREATMENT GROUPS. THE RESULTS ARE THEREFORE FULLY COMPATIBLE WITH THE NOTION THAT ACUPUNCTURE IS A PLACEBO THERAPY.
Something along these lines would, in my view, have been honest and scientific. Sadly, in acupuncture research, we very rarely get such honest science and the ‘National Institute of Complementary Medicine in Sydney, Australia’ has no track record of being the laudable exception to this rule.
21 Responses to Acupuncture research: more wishful thinking than good science
Leave a Reply to Frank van der Kooy Cancel reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
‘Acupuncture works, period! Now, what clinical trial design can we invent to show exactly that?’ If you want to know, you can always attend their Symposium in Nov 2017; “Clinical Trials in Integrative Medicine: Challenges and Opportunities” where they will discuss “the challenges and opportunities of clinical trials by exploring novel clinical trial designs, ….” http://nicm.edu.au/news/events/symposium_integrative_medicine
I do wonder what the impact of diet/lifestyle advice (all four arms) was on the outcome of this study.
Acupuncture puzzles me and I strive to understand how someone could claim to understand it.
Placebo puzzles me, but it’s demonstrated repeatedly and without need to bend data or construct tests to “support” it.
The post’s finally helped me grasp the device by which it’s claimants and their customers fall for it: present a facade of scientific endeavour whose real function is to fuel marketing.
Thanks once again. I’m learning, slowly.
I would guess that PLOS ONE put this out to peer review, and the reviewers were acupuncturists. The last person you want to review a quack study is another quack.
These guys don’t really have any plans to do a large and well designed study, because they have already achieved their objective. They must fool the public into believing that acupuncture works for whatever. So, all they need to do is to put out a press release entitled “Period pain reduced by acupuncture treatment” based on this ‘peer reviewed scientific publication’. This idea will now spread like a fire on social media, and that is pretty much ‘job well done, so lets move on to the next study’ http://nicm.edu.au/news/period_pain_reduced_by_acupuncture_treatment
It makes me think of the acupuncture study conducted by this group regarding allergic rhinitis also discussed on this Blog in May 2016, where they went onto social media and simply stated that “So, in a nutshell, acupuncture is a safe, effective and cost-effective treatment for allergic rhinitis…” Did they change this advice based on the many comments made on this Blog? Nope, it is still there, even after 15 months – again, objective achieved for these guys.
https://southportacupuncture.org/2016/05/12/acupuncture-alleviates-symptoms-of-allergic-rhinitis-hay-fever-by-creating-lasting-modulation-of-the-immune-system/
Just a pity that the millions of dollars, coming from the Aus taxpayer, pumped into this group cannot be used for doing something useful e.g. tackling microbial drug resistance for example – but I guess acupuncture will solve this issue as well.
“Just a pity that the millions of dollars, coming from the Aus taxpayer, pumped into this group cannot be used for doing something useful e.g. tackling microbial drug resistance for example – but I guess acupuncture will solve this issue as well.”
+1
As far as pain therapy goes, I think the US will do anything at this point (even embrace no-factual pseudoscience) to combat the opioid epidemic.
Sure, that is a big problem, but ineffective methods will unfortunately not help. Hard to solve problems is of course where these quacks make a killing, by coming to the rescue with their ‘natural, safe, effective, holistic, cost effective etc’ treatments.
It is, however, quite refreshing to see that they have at least started to declare their conflicts of interest. Something that they never really did in the past (goo.gl/GmtPRK). There always seems to be a commercial acupuncture clinic involved, and this study appears to be no different.
“why, oh why, I ask myself, did PLOS ONE publish this paper?” PLOS ONE is a relatively soft place for scientific publication, in my experience. It describes its peer review process thus: PLOS ONE will rigorously peer-review your submissions and publish all papers that are judged to be technically sound. Judgments about the importance of any particular paper are then made after publication by the readership, who are the most qualified to determine what is of interest to them.”
So all you need is for a paper to be accepted is that it is judged ‘technically sound’. That’s a criterion wide open for a range of interpretations. Plus authors are given the opportunity to ask for particular individuals not to be invited as referees.
As Les Rose already suggested, the reviewers were probably fellow acupuncturists. The editorial board (that’s just the people to whom manuscripts are sent to be forwarded to peer reviewers) consists of more than 6,000 ‘experts’. You can search on this page for editorial board members who list ‘acupuncture’ as a speciality and you’ll find no fewer than five hits: Lisa Conboy, New England School of Acupuncture; Bing Hou, Institute of Automation Chinese Academy of Sciences; Fan Qu, Zhejiang University School of Medicine First Affiliated Hospital; Senthil Kumar Subramanian, Pondicherry Institute of Medical Sciences; and Qinhong Zhang, Stanford University School of Medicine. Les Rose is spot on, and I suggest that PLOS ONE is a scientifically weak journal that leans on the other, excellent members of the PLOS stable for its apparently good reputation.
Here is an extremely lengthy list of reviewers in 2016 http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0174259
Edzard, have you ever had a journal article that you authored rejected? If so, what was the reasoning behind the decision?
I have published ~ 1 000 articles in the peer reviewed literature; I estimate that about 30% of them were initially rejected by a journal. the reasons cover everything that you can think of [and some you cannot think of].
It is always interesting how the title evolve, depending on where it is published and who the target audience is. Here is the title of the manuscript “The role of treatment timing and mode of stimulation in the treatment of primary dysmenorrhea with acupuncture: An exploratory randomised controlled trial”.
Here is the title on the website of Western Sydney University “Study points to acupuncture to reduce period pain” (https://www.westernsydney.edu.au/newscentre/news_centre/more_news_stories/study_points_to_acupuncture_to_reduce_period_pain)
And here it is on the website of the National Institute of Complementary Medicine “Period pain reduced by Acupuncture treatment” (http://nicm.edu.au/news/period_pain_reduced_by_acupuncture_treatment)
So it goes from a very cautious “exploratory” study to a factual statement, and this is exactly what they want. And because the leading author, Mike Armour, happens to be a director of an acupuncture clinic they will post this title on their FB site: “Period pain: Acupuncture is a effective alternative to painkillers” (https://www.facebook.com/pg/TheBodyWorkshop/posts/?ref=page_internal)
Job done, I would say, no need to do any well designed study
Also interesting that the Cochrane review http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD007854.pub3/full with one of the authors being Mike Armour concluded “There is insufficient evidence to demonstrate whether or not acupuncture or acupressure are effective in treating primary dysmenorrhoea, and for most comparisons no data were available on adverse events. The quality of the evidence was low or very low for all comparisons. The main limitations were risk of bias, poor reporting, inconsistency and risk of publication bias.” and then proceeded to a non controlled trial
Can this then be called ‘scientific misconduct’? Especially when they then go on social media and sell at as a clear positive. And I am afraid this is not an isolated case, there are many more examples. Usually they employ the notorious A+B vs B design, but they are currently working quite hard to come up with ‘better’ (such as in this current study) trial designs to give them their positive results. Just have a look at their most recent publications:
https://scholar.google.co.za/citations?hl=en&user=SWpRXPQAAAAJ&view_op=list_works&sortby=pubdate
this is probably a question of interpretation; in my view, the publication of grossly misleading conclusions must be considered scientific misconduct.
The question is; is this paper ‘technically sound’?
Well, I’ve contacted the PLOS ONE editor and they are looking into it – will let you know what they decide.
Well, here is the response from PLOS ONE:
“Thank you very much for your message and for leaving a comment on the published article.
I have raised your concerns with our senior staff editors. We appreciate your comments regarding the lack of a control group in the study; this is a valid item to raise regarding the study design and the strength of the conclusions that can be drawn from the trial. The authors present the study as exploratory and discuss the limitations of the trial in the article, the conclusions acknowledge the strength of evidence that can be concluded and the need for further studies. The limitations in the study design were considered during the review process.
Thank you also for pointing out the competing interest held by the authors. As per the PLOS ONE’s Competing Interests Policy (http://journals.plos.org/plosone/s/competing-interests), the authors have declared links to an acupuncture clinic and The National Institute of Complementary Medicine in their declaration of competing interests. This declaration ensures transparency and allows readers to consider the work in the context of this professional affiliation.
You note concerns about how the results of the trial have been reported in other forums, however, we cannot take responsibility for how the work and its results are reported or represented in forums other than those associated with PLOS or PLOS ONE.
Again, we appreciate your posting a comment on the published article. We offer this mechanism as a means for other researchers, and the public, to engage with published research in a public forum. Please feel free to contact me if I can address anything further.”
This is an entire branch of medicine (veterinary medicine) that could potentially legitimize herbalism as a specialty and in doing so, wax the slippery slope of cam with the result of … ending up in the dumpster of pyramid schemes and charlatanism. What would be your best pearls of advice to confront this ? http://news.vin.com/VINNews.aspx?articleId=45631
This research group has just published a new interesting paper – its worth a look: “Reliability of the NICMAN Scale: An Instrument to Assess the Quality of Acupuncture Administered in Clinical Trials” https://www.hindawi.com/journals/ecam/2017/5694083/
Unsurprising reaction. See my previous comment on PLOS ONE. “the conclusions acknowledge the strength of evidence that can be concluded and the need for further studies.” I wish every journal could ban that final, anodyne cop-out. All scientific publications open a “need for further studies”: the shame is when the prevailing reason for the need is that the currently published study is so inadequate.
I fully agree, but I also fear that there is a bit more behind this paper than meets the eye. This is part of my reply to Plos One “…this research group is working very hard to design ‘new’ RCTs which will give them the positive results that they want. So, the “exploratory” in their title was to see where they can publish this ‘new’ trial design. For example ; they are hosting a symposium where they “….will discuss the challenges and opportunities of clinical trials by exploring novel clinical trial designs….”. https://www.westernsydney.edu.au/nicm/news/events/symposium_integrative_medicine
PLOS ONE has now obviously fell for it, so I suspect that this paper might be the star of the show at their symposium and that this might now open the floodgates for many more research papers using this trial design – simply, because they got away with it.
I would urge you to reconsider your decision.”
Lets see how they respond, if at all.