Acupuncture is all over the news today. The reason is a study just out in BMJ-Open.
The aim of this new RCT was to investigate the efficacy of a standardised brief acupuncture approach for women with moderate-tosevere menopausal symptoms. Nine Danish primary care practices recruited 70 women with moderate-to-severe menopausal symptoms. Nine general practitioners with accredited education in acupuncture administered the treatments.
The acupuncture style was western medical with a standardised approach in the pre-defined acupuncture points CV-3, CV-4, LR-8, SP-6 and SP-9. The intervention group received one treatment for five consecutive weeks. The control group received no acupuncture but was offered treatment after 6 weeks. Outcomes were the differences between the two groups in changes to mean scores using the scales in the MenoScores Questionnaire, measured from baseline to week 6. The primary outcome was the hot flushes scale; the secondary outcomes were the other scales in the questionnaire. All analyses were based on intention-to-treat analysis.
Thirty-six patients received the intervention, and 34 were in the control group. Four participants dropped out before week 6. The acupuncture intervention significantly decreased hot flushes, day-and-night sweats, general sweating, menopausal-specific sleeping problems, emotional symptoms, physical symptoms and skin and hair symptoms compared with the control group at the 6-week follow-up. The pattern of decrease in hot flushes, emotional symptoms, skin and hair symptoms was already apparent three weeks into the study. Mild potential adverse effects were reported by four participants, but no severe adverse effects were reported.
The authors concluded that the standardised and brief acupuncture treatment produced a fast and clinically relevant reduction in moderate-to-severe menopausal symptoms during the six-week intervention.
The only thing that I find amazing here is the fact the a reputable journal published such a flawed trial arriving at such misleading conclusions.
- The authors call it a ‘pragmatic’ trial. Yet it excluded far too many patients to realistically qualify for this characterisation.
- The trial had no adequate control group, i.e. one that can account for placebo effects. Thus the observed outcomes are entirely in keeping with the powerful placebo effect that acupuncture undeniably has.
- The authors nevertheless conclude that ‘acupuncture treatment produced a fast and clinically relevant reduction’ of symptoms.
- They also state that they used this design because no validated sham acupuncture method exists. This is demonstrably wrong.
- In my view, such misleading statements might even amount to scientific misconduct.
So, what would be the result of a trial that is rigorous and does adequately control for placebo-effects? Luckily, we do not need to rely on speculation here; we have a study to demonstrate the result:
Background: Hot flashes (HFs) affect up to 75% of menopausal women and pose a considerable health and financial burden. Evidence of acupuncture efficacy as an HF treatment is conflicting.
Objective: To assess the efficacy of Chinese medicine acupuncture against sham acupuncture for menopausal HFs.
Design: Stratified, blind (participants, outcome assessors, and investigators, but not treating acupuncturists), parallel, randomized, sham-controlled trial with equal allocation. (Australia New Zealand Clinical Trials Registry: ACTRN12611000393954)
Setting: Community in Australia.
Participants: Women older than 40 years in the late menopausal transition or postmenopause with at least 7 moderate HFs daily, meeting criteria for Chinese medicine diagnosis of kidney yin deficiency.
Interventions:10 treatments over 8 weeks of either standardized Chinese medicine needle acupuncture designed to treat kidney yin deficiency or noninsertive sham acupuncture.
Measurements: The primary outcome was HF score at the end of treatment. Secondary outcomes included quality of life, anxiety, depression, and adverse events. Participants were assessed at 4 weeks, the end of treatment, and then 3 and 6 months after the end of treatment. Intention-to-treat analysis was conducted with linear mixed-effects models.
Results: 327 women were randomly assigned to acupuncture (n = 163) or sham acupuncture (n = 164). At the end of treatment, 16% of participants in the acupuncture group and 13% in the sham group were lost to follow-up. Mean HF scores at the end of treatment were 15.36 in the acupuncture group and 15.04 in the sham group (mean difference, 0.33 [95% CI, −1.87 to 2.52]; P = 0.77). No serious adverse events were reported.
Limitation: Participants were predominantly Caucasian and did not have breast cancer or surgical menopause.
Conclusion: Chinese medicine acupuncture was not superior to noninsertive sham acupuncture for women with moderately severe menopausal HFs.
My conclusion from all this is simple: acupuncture trials generate positive findings, provided the researchers fail to test it rigorously.
40 Responses to A new acupuncture trial published in the ‘BMJ-Open’ seems to smell of scientific misconduct
Leave a Reply to Leigh Jackson Cancel reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
What do you expect from belonetherapy?
(Greek: Belone, a needle.)
see also this:
https://www.thetimes.co.uk/article/acupuncture-menopause-claim-dismissed-by-edzard-ernst-mcbl2v69v?shareToken=9dc420de3e62d01820afe9eda5c04e4f
I read the Times piece – at least some balance – but what is the BMJ’s response?
I was not impressed by the authors’ response: “Responding to the criticism, the report’s authors said that giving the control group sham acupuncture rather than no treatment would have amounted to a trial of two types of acupuncture.”
Quite so, but so what?
My hypothesis would be that no difference could be discerned between two such groups (no effect being attributable to needling, irrespective of where the needles are put, or whether needles are used at all). If that were to be the outcome, we could say “It does not matter where needles are placed, or used at all” and so question theories that there are ‘meridians’ and ‘acupuncture points’ and could surmise beyond reasonable doubt that needles are not necessary at all to achieve an ‘effect’.
That ‘effect’ being due to placebo responses – which we know something about.
Much time, trouble and expense could be saved by not ‘training’ practitioners where and how to needle and from not purchasing the kit.
I guess trainers of ‘belonetherapy’ would not appreciate having nothing to do (or charge for), and needle manufacturers would have to diversify their product lines.
I charge that these conflicts of interest is why valid basic research is not carried out.
The very definition of quackery.
The BMJ should be ashamed.
it was the ‘BMJ-Open’
and the response of the authors is yet another error.
My comment on this paper is at the Science Media Centre, There is even more wrong with it than you say here.
http://www.sciencemediacentre.org/expert-reaction-to-study-on-acupuncture-for-symptoms-of-menopause/
interesting – thanks
The list of exclusion criteria is quite long, so it can certainly be argued, that this is not a real pragmatic trial.
But is a pragmatic trial even relevant in the first place? Pragmatic trials were designed to answer the question of how well a treatment works in a daily clinical setting, that has been shown to work in explanatory trials. Not to establish if it a treatment works at all.
yes, I agree
The reviewers comments on this paper are at https://bmjopen.bmj.com/content/bmjopen/9/1/e023637.reviewer-comments.pdf
Astonishingly, none of the reviewers even mentions that the study has no proper sham control group.
Peer review is truly broken. BMJ Open should be ashamed.
Interesting read. It appears that the idea of calling it a pragmatic study (it is not) was suggested by one of the reviewers (who should know better).
Trial participants were drawn from the acupuncture-doctors local patients. Please please, me me me, doctor!
Control patients had to wait 6 weeks for treatment compared with treatment patients. Blinding be damned! How surprising that those who received immediate treatment reported improvement of symptoms compared with those who had to wait! Gratitude may perhaps have had more immediate effect on reducing stress levels for those in the treatment group than those made to suffer for another six weeks.
The trial guaranteed the result which it obtained. No valid sham form of acupuncture means no falsifying the null hypothesis. The trial proves nothing except that gratefully treated patients report improvements compared with ungrateful, untreated patients.
Fake science.
One problem is that adding a sham arm (i.e. comparing one form of acupuncture with another) would require many more participants to draw meaningful conclusions, which would increase costs hugely. Perhaps that doesn’t excuse the shortcomings of this trial, but it’s an important point. How does physiotherapy get around these issues, for example? I’d be interested to see examples of physiotherapy trials that meet the requirements outlined by those above.
“One problem is that adding a sham arm (i.e. comparing one form of acupuncture with another) would require many more participants to draw meaningful conclusions, which would increase costs hugely.”
would you mind explaining this assumption?
Am I not right in thinking that when comparing two active treatments, trials need to be of much higher power to draw meaningful conclusions?
correct, but non-penetrating needle acupuncture is a placebo, not an active treatment.
https://www.ncbi.nlm.nih.gov/pubmed/12512790
‘non-penetrating needle acupuncture is a placebo, not an active treatment’
There’s debate over that, e.g. https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0140825
‘most studies noted that the placebo devices may not be a completely inert intervention’.
Also, presumably in trials using these devices, the ‘real’ arm use the device as well, but with a real need in the device? This would make it impossible to perform ‘real’ acupuncture, which involves careful needle manipulation, changes of angle etc.
thank you for mentioning this paper.it analysed RCTs using non-penetrating devices. there is no way this method can be used to test whether the devices are inert. if you disagree, please explain to me how this can be done [the authors are suspiciously silent on this point]. they also report that, of 20 evaluable studies 19 showed successful blinding. furthermore they stated that our device was better for blinding than the Streitberger needle.
my reading of this paper is that it needed to be published to open a back-door to acupuncture fans. hence the sentence you correctly quoted but which is not supported by the data.
You no doubt understand the data better than I do, and I don’t have the research expertise to pick apart the points you’ve just made. I’m just aware that there is debate over the validity of these sham devices. But even if they are 100% inert and indistinguishable from the ‘real’ version, as I mentioned before, there’s no way of doing ‘real’ acupuncture with a device like that.
To ask again, can anyone explain how physiotherapy research overcomes these problems? Presumably there must be lots of placebo-controlled trials validating physiotherapy techniques? How are they done?
” there is debate over the validity of these sham devices”
there is also a debate whether the earth is flat! it’s a pseudo-debate initiated and kept alive by acupuncture fans who feel they need an escape route when confronted with results they don’t like. hence the proliferation of studies like the one discussed in this post. and hence the misleading statement of its authors about sham devices.
” But even if they are 100% inert and indistinguishable from the ‘real’ version, as I mentioned before…”
I did not ‘mention’ this, we tested the hypothesis and have the data; I have shown it to you. in any case, even if not 100% inert, these devices are better than no patient-blinding and control over placebo-effects at all.
“there’s no way of doing ‘real’ acupuncture with a device like that. ”
I fail to understand this bit.
I think you’ve misread my sentence. I said ‘as *I* mentioned before’, referring to my own earlier mention of these sham devices being unsuitable for real acupuncture. I wasn’t suggesting that you merely ‘mentioned’ anything.
If I’m reading the abstract correctly, the studies you linked to used only the subjective experience of deqi sensation as a measure of activity in the sham device. This doesn’t allow for the potential physiological responses (mechanoreceptor activation) caused by pressure from the sham device (similar perhaps to acupressure, or the non-penetrating acupuncture techniques used in some traditions). See here for a fuller discussion: https://www.sciencedirect.com/science/article/pii/S1876382014000365#bbib0170
My other point was that using a needle encased in a tube with a perpendicular mount would render traditional acupuncture techniques impossible – varying angles/depths/manipulations depending on the patient and the situation, and engaging in dialogue about sensations experienced are standard practice. Presumably for trials using these devices to be considered valid, the ‘real’ group would need to be treated using the devices but with real needles in the mounting? That couldn’t be considered real traditional acupuncture.
“Presumably for trials using these devices to be considered valid, the ‘real’ group would need to be treated using the devices but with real needles in the mounting? That couldn’t be considered real traditional acupuncture.” the guide-tube allows variation of depth. As to angles, once you have penetrated the skin with a needle, variations of angle are limited with both real and guide-tube encased needles.
‘As to angles, once you have penetrated the skin with a needle, variations of angle are limited with both real and guide-tube encased needles.’
That’s something I do have a great deal of experience with, and I can’t say I agree. Using a real needle, there is really no restriction on changes of angle angle. It is quite possible (and commonly required) to stimulate a single point in multiple directions without withdrawing the needle.
Anatomy textbooks do not identify acupuncture points or meridians. Acupuncture textbooks do. They are either fictions or things beyond the power of science to comprehend. If there is no valid sham form of acupuncture then there is no way to test the null hypothesis.
This being so acupuncture is placed beyond the bounds of science completely.
Unicorn: a fiction or a creature beyond the bounds of science to prove or disprove?
Again, I’d ask what the valid sham forms of various physiotherapy techniques are?
Also, RCTs are not the only way of investigating acupuncture. An array of physiological responses have been documented, and scientific research continues into the various mechanisms at play.
“RCTs are not the only way of investigating acupuncture”
correct – but if you want to know about efficacy, they are the most conclusive option.
“an array of physiological responses have been documented, and scientific research continues into the various mechanisms at play”
correct – but after your recent comments you should be the 1st to point out THERE IS A LOT OF DEBATE ABOUT THAT.
in any case, physiological responses and mechanisms without proof of efficacy are close to meaningless: if I fall down the stairs, I will display a lot of physiological responses. that does not mean that falling down the stairs is therapeutic.
Yes, but acupuncture shows physiological responses as well as effectiveness in many pragmatic trials, unlike falling down the stairs.
If physiotherapy has the same failings as acupuncture that’s bad.
Research goes on as to what happens when needles are inserted into the body. All well and good.
There exists a huge pile of acupuncture RCTs. Mostly junk but there’s enough good ones to know that acupuncture is no better than placebo. The only way to avoid this conclusion is to claim that fake-needling at non-acupuncture points is acupuncture.
But it that’s true why not practise fake acupuncture for real?
Prick the skin at random points and tell patients that’s all there is to it. No special points and no puncturing needed. No meridians. No nothing. Just random pricks to the skin is all there is to it. Would be safer and cheaper. Toothpicks work just as well.
That doesn’t seem to happen though.
@Tom Kennedy
You’re right to pick up on the difficulties of placebo controls in physiotherapy. Maddocks et al. [Journal of Evaluation in Clinical Practice 22 (4):598-602 (2016)] concisely describe the problems and outline possible solutions. The paper sits behind a paywall but I was able to access it via my institution’s library.
You might benefit from reading it and comparing the situation for physiotherapy with your unshakeable faith in acupuncture. Placebos for acupuncture are more straightforward than placebos for manipulative procedures. But unsinkable acupuncturists always come up with reasons why the placebos are wrong, as you’ve been doing in this thread.
“Faith is belief in what you know ain’t so.” [Mark Twain]
@Frank, you’re right, I do keep coming up with reasons I don’t find these placebo trials convincing. You see them as ‘excuses’, but to anyone with traditional training they simply don’t represent what we do in our clinics. What is being investigated may be interesting and may lead to useful insights, but they are not trials of traditional acupuncture. When the ‘real’ arm isn’t real, and the ‘placebo’ arm has physiological effects, it doesn’t surprise me when the results don’t differ much.
I wish I had answers rather than simply complaints. Perhaps trials that compare for example usual care vs full traditional acupuncture (of various styles?!) vs Park sham device vs wait list, including biological markers (if and when valid markers are available) as outcome measures would be a good compromise. Of course, these would be expensive, and wouldn’t be seen as rigorous enough by some as the real acupuncture would still be uncontrolled.
But it does fascinate me that many people are happy to recommend physiotherapy when it essentially suffers the same problems as acupuncture when it comes to ‘proving’ itself. Thanks for the article suggestion, I’ll try to find it.
@Tom Kennedy
“..to anyone with traditional training they simply don’t represent what we do in our clinics”. Sadly, there seems to be no consistency between acupuncturists, however trained, just like chiropractors. I’m afraid you’re exemplifying the “no true Scotsman” fallacy.
RCTs don’t look at what goes on in the clinic. All kinds of stuff goes on in clinics.
RCTs dig down to the basics. Does x (where x = whatever) actually do what is believed it does? Does aspirin actually help with headache or help prevent blood clots? Or is it no more effective than a dud pill in a carefully controlled situation.
If acupuncture is so unspecifiable that nothing can ever qualify as acupuncture in any clinical trial, no matter how excellent it is in quality, then nobody can ever properly know whether acupuncture works or not. It can only ever be a matter of personal belief.
Would it be right for doctors to prescribe drugs if science was unable to determine whether they are safe and effective? For any reason at all? Including, for example, the reason that drugs as used in the clinical situation are never used quite the same way as they are used in clinical trials? Pragmatic trials are used to compare effectiveness of treatments in the clinical situation compared with their efficacy in a scientifically controlled situation. The acupuncture study here in question claims to be a pragmatic trial. The problem being that pragmatic trials must be worse than useless if carefully controlled trials are useless. And a waste of time if carefully controlled tials are not useless but acupuncture is.
You doubt that science can determine whether acupuncture is effective. If that is so it must be the case that science cannot determine whether accupuncture is safe. And if science cannot determine whether something is safe what can? Do we just have to take your word for it?
Leigh, before you get too attached to “beyond the power of science to comprehend”, you really should schedule some time with an acupuncturist and have them show you how to palpate points.
that’s bull
they cannot be palpated.
if you disagree, show me a blinded experiment that demonstrates it [very easy to conduct!]
Good grief, Edzard – I hope this is one of your many jokes. I’m assuming your “blinded experiment” was the jokey part, since students having problems palpating are instructed to close their eyes :).
How did you find points when you did your extensive acupuncture training? Of course you palpate points. Children in martial arts classes learn how to do that.
Having looked into the Park sham device a little more, not only does it seem impossible to use standard traditional needling manipulations at varying angles (insertion is vertical), depth of needling is limited to 15mm. This is not sufficient to replicate many standard situations. Also, I assume it would be very difficult/impossible to use this device on fingers and toes, and on the scalp, where several important points lie.
Tom,
Are you familiar with “western medical” acupuncture, used for the RCT Edzard posted about?
https://www.ncbi.nlm.nih.gov/pubmed/19369193
Angle and depth probably don’t really matter.
@jm I’m not sure that’s true actually – trigger point acupuncture involves needling at various angles and depths.
Fair enough. Might be fine for this one, though. Maybe there’s a “Park XL” for studying trigger points.
MedPage’s unfortunately credulous take on this study: https://www.medpagetoday.com/obgyn/menopause/78156.
MedPage could do with a crash course in critical thinking