The aim of this three-armed, parallel, randomized exploratory study was to determine, if two types of acupuncture (auricular acupuncture [AA] and traditional Chinese acupuncture [TCA]) were feasible and more effective than usual care (UC) alone for TBI–related headache. The subjects were previously deployed Service members (18–69 years old) with mild-to-moderate TBI and headaches. The interventions explored were UC alone or with the addition of AA or TCA. The primary outcome was the Headache Impact Test (HIT). Secondary outcomes were the Numerical Rating Scale (NRS), Pittsburgh Sleep Quality Index, Post-Traumatic Stress Checklist, Symptom Checklist-90-R, Medical Outcome Study Quality of Life (QoL), Beck Depression Inventory, State-Trait Anxiety Inventory, the Automated Neuropsychological Assessment Metrics, and expectancy of outcome and acupuncture efficacy.
Mean HIT scores decreased in the AA and TCA groups but increased slightly in the UC-only group from baseline to week 6 [AA, −10.2% (−6.4 points); TCA, −4.6% (−2.9 points); UC, +0.8% (+0.6 points)]. Both acupuncture groups had sizable decreases in NRS (Pain Best), compared to UC (TCA versus UC: P = 0.0008, d = 1.70; AA versus UC: P = 0.0127, d = 1.6). No statistically significant results were found for any other secondary outcome measures.
The authors concluded that both AA and TCA improved headache-related QoL more than UC did in Service members with TBI.
The stated aim of this study (to determine whether AA or TCA both with UC are more effective than UC alone) does not make sense and should therefore never have passed ethics review, in my view. The RCT followed a design which essentially is the much-lamented ‘A+B versus B’ protocol (except that a further groups ‘C+B’ was added). The nature of such designs is that there is no control for placebo effects, the extra time and attention, etc. Therefore, such studies cannot fail but generate positive results, even if the tested intervention is a placebo. In such trials, it is impossible to attribute any outcome to the experimental treatment. This means that the positive results are known before the first patient has been enrolled; hence they are an unethical waste of resources which can only serve one purpose: to mislead us. It also means that the conclusions drawn above are not correct.
An alternative and in my view more accurate conclusion would be this one: both AA and TCA had probably no effect; the improved headache-related QoL was due to the additional attention and expectation in the two experimental groups and is unrelated to the interventions tested in this study.
In our new book, MORE HARM THAN GOOD, we discuss that such trials are deceptive to the point of being unethical. Considering the prominence and experience of Wayne Jonas, the 1st author of this paper, such obvious transgression is more than a little disappointing – I would argue that is amounts to overt scientific misconduct.
22 Responses to Acupuncture: poor research leading false-positive conclusions. In my view, this is scientific misconduct
Leave a Reply to Tom Kennedy Cancel reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
I came across a similar article on acupuncture before. Let’s call it the ‘exploratory’ trial design where they intentionally omit including a placebo or control group. http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0180177
As soon as they get it published somewhere they usually go onto social media and promote the results as a clear positive – job done for them.
I fear that this will be the new method of choice when it comes to acupuncture clinical trials. These researchers know that acupuncture is nothing more than a placebo and hence they have to come up with these ‘novel’ trial designs. I agree, it is scientific misconduct
Scientific misconduct for sure. Aside from the issue of researchers failing to behave ethically, it is depressing that ethics committees pass study proposals that are so deeply flawed.
It may well be a fair comment that the A+B v B design is a poor choice, but I don’t see how it follows that the interventions had ‘probably no effect’. From my understanding, the ‘placebo effect’ – whatever that mysterious thing really is – is impossible to pin down in terms of effect size, with estimates varying wildly. The acupuncture groups had a sizeable decrease in NRS. The study design may not allow for a conclusion about how much, if any, of this decrease was due to specific effects, but neither does it allow your conclusion. In fact, when taken in the context of the Cochrane migraine review, which shows statistical significance over sham, there’s more reason to question your conclusion.
that’s the difference between a critical thinker and an uncritical one, I guess.
the former always assumes that the null-hypothesis must be true until proven otherwise.
anyway, i did say ‘probably’ which is much fairer that the conclusion of the original authors.
I don’t mean to be facetious, this is a genuine question – do you consider physiotherapy to be a ‘proven’ intervention?
part of it
“Physiotherapy” is multi-factorial, broadly-inclusive BUT constantly evolving AND falsifiable science-based profession.
Like “psychotherapy” it is within reasonable societal-agreement (based on its critical and university based educational standards) that, though by no stretch perfect, complete or universally effective, IS the standardized bastion for real and falsifiable treatment and testing methods….at least as much as possible. It’s underpinned by the same scrutiny any “real scientific” method endeavors for. ‘Things’ neither acupuncture, Scientology or Chiropractic have ANY real interest in, and in fact are anathema to.
Odd that many physiotherapists use acupuncture then, isn’t it?
nothing odd about that; many conventional clinicians use a lot of quackery
The use of acupuncture by physiotherapists is not physiotherapy.
The choice of method was deliberate. Jonas eschews placebo controls. This was not a scientific trial. It was an article-of-faith trial. Acupuncture works, period. That is the unspoken premise of the study. The study was not designed to test the premise that acupuncture does not work – which would be science. The migraine review is irrelevant in the context of Jonas’ study. That review – which was science – found a debatable clinical effect for acupuncture.
Jonas’ study was not bad science, it wasn’t science at all – it was blind faith masquerading as science.
The lead author of the migraine review, Andrew Vickers, says Samueli Institute (Director and CEO = Jonas) should be judged on the quality of science it supports. I entirely agree with him.
Which part of Physiotherapy is proven?
did you say you don’t want to be fastidious?
i am not engaging in a discussion about physio in the comments section of a blog-post about acupuncture.
No, I said I didn’t want to be facetious! I’m quite happy to be fastidious, and it’s a shame you’re not. My question was entirely related to the topic, but it’s your prerogative not to explore things further.
Given the large number of critics of this and related lunacies, who decides, and why, that a legal case should be brought( I refer here of course to the Huber nonsense)?
“Acupuncture… has been shown to elicit a dynamic, multimechanism response and has the potential to influence multiple interacting components of trauma response and recovery. As such, acupuncture has the potential to address multiple rather than single symptoms in order to address whole-person recovery. Systematic reviews of acupuncture have shown its benefits for relieving headaches of various types… as well as sleep disturbance, anxiety, depression, PTSD, and pain. Consequently, it is a treatment approach that is well-suited for facilitating recovery after trauma.”
Secondary outcomes tested for included: NRS (pain best – headache in this case), sleep quality, PTSD, QoL, depression, anxiety, ANAM.
Headache-NRS scored better than usual care.
These results do not support the claim that acupuncture has potential to address whole-person recovery. I would have expected a greater placebo response. The fact that AA was as “effective” as TCM indicates that where neeedles are placed is irrelevant as far as eliciting the placebo effect is concerned. Which is to say as far as the null hypothesis is concerned. Which is the only scientific conclusion that can be made regarding the results of this silly, stupid study.
Just checked the references on the wikipedia link. All is explained. Disturbing.
Jonas is woo to his bones. The antithisis of scepticism. Open-minded is a euphemism. He’s devoid of brains.
The Samueli Institute pays Jonas a cool half-million bucks per annum to provide them with junk-science. Money supplied to the Institute by Congress cronies funded by the Institute. A lucrative circle of fraudulent science.
Quote from the Institute: “To the extent that the placebo effect can enhance healing, we should include it in our research, rather than control for it.”
Licenced to print money who wouldn’t?
Not so silly and stupid then.
Mary Ann Liebert published this study. They too are culpable.
@Bjorn @Michael Kenny describes Physiotherapy as ‘the standardised bastion for real and falsifiable treatment and testing methods’. And yet something like 6000 UK physios integrate acupuncture into their treatments. It seems strange to me that such a profession, which is such a bastion for the scientific method, would embrace an ineffective treatment. Perhaps they look at the available evidence, see enough promise to experiment with it, and are then impressed with the results they achieve? And presumably without the placebo-boosting woo woo explanations/pulse taking etc? I’ve met more than one physio who has begrudgingly admitted that acupuncture is the most effective tool they have available to them in many situations.
physiotherapy “a profession, which is such a bastion for the scientific method”
that is truly funny!
I have to credit Michael with the joke.
At least in the US getting a DPT degree involves real universities and rigorous standards. PT research outnumbers Chiropractic research about 100:1…and much research (and editorializing) conducted by PTs is negative, disruptive and iconoclastic.,,,nearly impossible on the DC or Alt-Med side. As far as adopting less-than science-based care….everyone needs to pay their bills. I’d still much rather have manipulation or acupuncture delivered by a US trained DPT than a religious adherent.
But I will stand corrected: the real bastion for science & evidenced-based care, research and evolution of such is in the halls-of-Chiropractic, Acupuncture and Homeopathic institutions. How did I miss that…?
Great post really nice blogs