Acupuncture is little more than a theatrical placebo! If we confront an acupuncture fan with this statement, he/she is bound to argue that there are some indications for which the evidence is soundly positive. One of these conditions, they would claim, is nausea and vomiting. But how strong are these data? A new study sheds some light on this question.
The objective of this RCT was to evaluate if consumption of antiemetics and eating capacity differed between patients receiving verum acupuncture, sham acupuncture, or standard care only during radiotherapy. Patients were randomized to verum (n = 100) or sham (n = 100) acupuncture (telescopic blunt sham needle) (12 sessions) and registered daily their consumption of antiemetics and eating capacity. A standard care group (n = 62) received standard care only.
The results show that more patients in the verum and the sham acupuncture group did not need any antiemetic medications, as compared to the standard care group after receiving 27 Gray dose of radiotherapy. More patients in the verum and the sham acupuncture group were capable of eating as usual, compared to the standard care group. Patients receiving acupuncture had lower consumption of antiemetics and better eating capacity than patients receiving standard antiemetic care, plausible by nonspecific effects of the extra care during acupuncture.
The authors concluded that patients receiving acupuncture had lower consumption of antiemetics and better eating capacity than patients receiving standard antiemetic care, plausible by nonspecific effects of the extra care during acupuncture.
I find these conclusions odd because they seem to state that acupuncture was more effective than standard care. Subsequently – almost as an afterthought – they mention that its effects are brought about by nonspecific effects. This is grossly misleading, in my view.
The study was designed as a comparison between real and sham acupuncture, and the standard care group was not a randomised comparison group. Therefore, the main result and conclusion has to focus on the comparison between verum and sham acupuncture. This comparison shows that the two did not produce different result. Therefore, the study shows that acupuncture was not effective.
A much more reasonable conclusion would have been: THIS STUDY FAILED TO FIND SIGNIFICANT EFFECTS OF ACUPUNCTURE BEYOND PLACEBO.
Drug and alcohol dependencies are notoriously difficult to treat effectively. Patients and their families are often desperate and willing to try anything. This seems like an ideal ground for acupuncturists who are, in my experience, experts in putting up smokescreens hiding the true value of their treatment.
The best way to determine the value of any intervention is probably conducting a systematic review of the evidence from rigorous clinical trials. Today we are in the fortunate position to have not just one of those articles; but do they really tell us the truth?
This brand-new systematic review investigated the effects of acupuncture on alcohol-related symptoms and behaviors in patients with this disorder. The PubMed database was searched until 23 August 2016, and reference lists from review studies were also reviewed. The inclusion criteria were the following: (1) being published in a peer-reviewed English-language journal, (2) use of randomized controlled trials (RCTs), (3) assessing the effects of acupuncture on psychological variables in individuals with a primary alcohol problem, and (4) reporting statistics that could be converted to effect sizes.
Seventeen studies were identified for a full-text inspection, and seven (243 patients) of these met our inclusion criteria. The outcomes assessed at the last post-treatment point and any available follow-up data were extracted from each of the studies. Five studies treated patients by inserting a needle into several acupoints in each ear. Two studies stimulated body points with or without ear stimulation. Four studies treated control patients with a placebo needle or under a completely different type of intervention, such as relaxation or transdermal stimulation, whereas the remaining studies inserted needles into nonspecific points. The patients were treated for 2 weeks to 3 months, and the treatment duration per session was 15–45 min. The results of the meta-analysis demonstrated that an acupuncture intervention had a stronger effect on reducing alcohol-related symptoms and behaviours than did the control intervention. A beneficial but weak effect of acupuncture treatment was also found in the follow-up data.
The authors concluded that although our analysis showed a significant difference between acupuncture and the control intervention in patients with alcohol use disorder, this meta-analysis is limited by the small number of studies included. Thus, a larger cohort study is required to provide a firm conclusion.
I am used to reading poor research papers, but this one is like a new dimension. Here are just the most obvious flaws:
- by searching just one database, the likelihood of missing studies is huge,
- by excluding non-English papers, the review automatically becomes non-systematic,
- the included studies differed vastly in many respects and can therefore not be pooled.
As it happens, a further meta-analysis has just been published. Here is its abstract:
Acupuncture has been widely used as a treatment for alcohol dependence. An updated and rigorously conducted systematic review is needed to establish the extent and quality of the evidence on the effectiveness of acupuncture as an intervention for reducing alcohol dependence. This review aimed to ascertain the effectiveness of acupuncture for reducing alcohol dependence as assessed by changes in either craving or withdrawal symptoms.
In this systematic review, a search strategy was designed to identify randomised controlled trials (RCTs) published in either the English or Chinese literature, with a priori eligibility criteria. The following English language databases were searched from inception until June 2015: AMED, Cochrane Library, EMBASE, MEDLINE, PsycINFO, and PubMed; and the following Chinese language databases were similarly searched: CNKI, Sino-med, VIP, and WanFang. Methodological quality of identified RCTs was assessed using the Jadad Scale and the Cochrane Risk of Bias tool.
Fifteen RCTs were included in this review, comprising 1378 participants. The majority of the RCTs were rated as having poor methodological rigour. A statistically significant effect was found in the two primary analyses: acupuncture reduced alcohol craving compared with all controls (SMD = −1.24, 95% CI = −1.96 to −0.51); and acupuncture reduced alcohol withdrawal symptoms compared with all controls (SMD = −0.50, 95% CI = −0.83 to −0.17). In secondary analyses: acupuncture reduced craving compared with sham acupuncture (SMD = −1.00, 95% CI = −1.79 to −0.21); acupuncture reduced craving compared with controls in RCTs conducted in Western countries (SMD = −1.15, 95% CI = −2.12 to −0.18); and acupuncture reduced craving compared with controls in RCTs with only male participants (SMD = −1.68, 95% CI = −2.62 to −0.75).
This study showed that acupuncture was potentially effective in reducing alcohol craving and withdrawal symptoms and could be considered as an additional treatment choice and/or referral option within national healthcare systems.
This Meta-analysis is only a little better than the first, I am afraid. What its conclusions do not sufficiently reflect, in my view, is the fact that the quality of the primary studies was mostly very poor – too poor to draw conclusions from (other than ‘acupuncture research is usually lousy’; see figure below). Therefore, I fail to see how the authors could draw the relatively firm and positive conclusions cited above. In my view, they should have stated something like this: DUE TO THE RISK OF BIAS IN MANY TRIALS, THE EFFECTIVENESS OF ACUPUNCTURE REMAINS UNPROVEN.
The authors of the first meta-analysis open the discussion by proudly declaring that “the present study is the first meta-analysis to examine the effect of acupuncture treatment on patients with alcohol use disorder and to provide data on the magnitude of this effect on alcohol-related clinical symptoms and behaviours.” They discretely overlook this meta-analysis from 2009 (and several others which even their rudimentary search would have identified):
Nineteen electronic databases, including English, Korean, Japanese, and Chinese databases, were systematically searched for RCTs of acupuncture for alcohol dependence up to June 2008 with no language restrictions. The methodological qualities of eligible studies were assessed using the criteria described in the Cochrane Handbook.
Eleven studies, which comprised a total of 1,110 individual cases, were systematically reviewed. Only 2 of 11 trials reported satisfactorily all quality criteria. Four trials comparing acupuncture treatment and sham treatments reported data for alcohol craving. Three studies reported that there were no significant differences. Among 4 trials comparing acupuncture and no acupuncture with conventional therapies, 3 reported significant reductions. No differences between acupuncture and sham treatments were found for completion rates (Risk Ratio = 1.07, 95% confidence interval, CI = 0.91 to 1.25) or acupuncture and no acupuncture (Risk Ratio = 1.15, 95% CI = 0.79 to 1.67). Only 3 RCTs reported acupuncture-related adverse events, which were mostly minimal.
The results of the included studies were equivocal, and the poor methodological quality and the limited number of the trials do not allow any conclusion about the efficacy of acupuncture for treatment of alcohol dependence. More research and well-designed, rigorous, and large clinical trials are necessary to address these issues.
One does not need to be an expert in interpreting meta-analyses, I think, to see that this paper is more rigorous than the new ones (which incidentally were published in the very dubious journals). And this is why I trust the conclusions of this last-named meta-analysis more than those of the new one: the efficacy of acupuncture remains unproven. And this means that we should not employ or promote it for routine care.
The aim of this paper was to systematically review effectiveness, safety, and robustness of evidence for complementary and alternative medicine in managing premature ejaculation (PE). Nine databases were searched through September 2015. Randomized controlled trials (RCTs) evaluating complementary and alternative medicine for PE were included. Studies were included if they reported on intravaginal ejaculatory latency time (IELT) and/or another validated PE measurement. Adverse effects were summarized.
Ten RCTs were included. Two assessed acupuncture, five assessed Chinese herbal medicine, one assessed Ayurvedic herbal medicine, and two assessed topical “severance secret” cream. Risk of bias was unclear in all studies because of unclear allocation concealment or blinding, and only five studies reported stopwatch-measured IELT. Acupuncture slightly increased IELT over placebo in one study (mean difference [MD] = 0.55 minute, P = .001). In another study, Ayurvedic herbal medicine slightly increased IELT over placebo (MD = 0.80 minute, P = .001). Topical severance secret cream increased IELT over placebo in two studies (MD = 8.60 minutes, P < .001), although inclusion criteria were broad (IELT < 3 minutes). Three studies comparing Chinese herbal medicine with selective serotonin reuptake inhibitors (SSRIs) favored SSRIs (MD = 1.01 minutes, P = .02). However, combination treatment with Chinese medicine plus SSRIs improved IELT over SSRIs alone (two studies; MD = 1.92 minutes, P < .00001) and over Chinese medicine alone (two studies; MD = 2.52 minutes, P < .00001). Adverse effects were not consistently assessed but where reported were generally mild.
The authors concluded that there is preliminary evidence for the effectiveness of acupuncture, Chinese herbal medicine, Ayurvedic herbal medicine, and topical severance secret cream in improving IELT and other outcomes. However, results are based on clinically heterogeneous studies of unclear quality. There are sparse data on adverse effects or potential for drug interactions. Further well-conducted randomized controlled trials would be valuable.
One has to be an optimist to agree that this constitutes ‘preliminary evidence for the effectiveness of acupuncture, Chinese herbal medicine, Ayurvedic herbal medicine, and topical severance secret cream in improving IELT and other outcomes.’ In the discussion section, the authors stress that “…all 10 studies were classed as having an overall unclear risk of bias because of unclear reporting of allocation concealment (all 10 studies) and unclear blinding of participants and personnel (five studies).” This hardly allows even a preliminary conclusion, in my view.
So, what DOES this review show? I think it demonstrates that
- alternative therapies are being touted and occasionally tested for even the most unlikely conditions,
- the quality of the studies is generally too poor to justify the research (particularly in an area as intrusive as PE),
- clinical trials often seem to be used not for finding answers but for promotion,
- in alternative medicine, trialists regularly violate research ethics by failing to report adverse effects.
George Lewith has died on 17 March, aged 67. He was one of the most productive researchers of alternative medicine in the UK; specifically he was interested in acupuncture. If you search this blog, you find several posts that mention him or are entirely dedicated to his work. Undeniably, my own views and research were often very much at odds with those of Lewith.
Wikipedia informs us that Lewith graduated from Trinity College, Cambridge in medicine and biochemistry. He then went on to Westminster Medical School to complete his clinical studies and began working clinically in 1974. In 1977 Lewith became a member of the Royal College of Physicians. Then, in 1980, he became a member of the Royal College of General Practitioners and, later in 1999, was elected a fellow of the Royal College of Physicians.
He was a Professor of Health Research in the Department of Primary Care at the University of Southampton and a director of the International Society for Complementary Medicine Research. Lewith has obtained a significant number of institutional peer reviewed fellowships at doctoral and post-doctoral level and has been principal investigator or collaborator in research grants totally over £5 million during the last decade.
Between 1980 and 2010, Lewith was a partner at the Centre for Complementary and Integrated Medicine, a private practice providing complementary treatments with clinics in London and Southampton.
A tribute by the British Acupuncture Council is poignant, in my view: “George was a friend not only to all of the acupuncture profession, be it traditional, medical or physiotherapist – he was a member of all three professional bodies – but to the whole of complementary medicine. As well as being a research leader he was also politically savvy, working tirelessly up front and behind the scenes to try to bring acupuncture and CAM further into the mainstream. Nobody did more.”
I find it poignant because it hints at the many differences I (and many others) had with Lewith during the last 25 years that I knew him. George was foremost a proponent of acupuncture. His 1985 book – the first of many – advocated treating many internal diseases with acupuncture!
George did not strike me as someone who had the ability or even the ambition to use science for finding the truth and for falsifying hypotheses; in my view, he employed it to confirm his almost evangelic belief. In the pursuit of this all-important aim he did indeed spend a lot of time and energy pulling strings, including on the political level. George was undoubtedly successful but the question has to be asked to what extent this was due to his tireless work ‘behind the scenes’.
In my view, George was a typical example of someone who first and foremost was an advocate and a researcher second. During my time in Exeter, I have met numerous co-workers who had the same problem. Almost without exception, I found that it is impossible to turn such a person into a decent scientist. The advocacy of alternative medicine always got in the way of objectivity and rationality, qualities that are, of course, essential for good science.
George’s very first publication on acupuncture was a ‘letter to the editor’ published in the BMJ. In it, he announced that he is planning to conduct a trial of acupuncture and stated that “acupuncture will be compared to an equally magical placebo”. Yes, George always had the ability to make me laugh; and this is why I will miss him.
One phenomenon that can be noted more frequently than any other in alternative medicine research is that studies arrive at wrong or misleading conclusions. This is more than a little disappointing, not least because it is the conclusion of a trial that is often picked up by health writers and others who in turn mislead the public. On this blog, we must have seen hundreds of examples of this irritating phenomenon. Here is yet another one. This study, a randomized, parallel, open-label exploratory trial, evaluated and compared the effects of systemic manual acupuncture, periauricular electroacupuncture and distal electroacupuncture for treating patients with tinnitus. It included patients who suffered from idiopathic tinnitus for more than two weeks were recruited. They were divided into three groups:
- systemic manual acupuncture group (MA),
- periauricular electroacupuncture group (PE),
- distal electroacupuncture group (DE).
Nine acupoints (TE 17, TE21, SI19, GB2, GB8, ST36, ST37, TE3 and TE9), two periauricular acupoints (TE17 and TE21), and four distal acupoints (TE3, TE9, ST36, and ST37) were selected. The treatment sessions were performed twice weekly for a total of 8 sessions over 4 weeks. Outcome measures were the tinnitus handicap inventory (THI) score and the loud and uncomfortable visual analogue scales (VAS). Demographic and clinical characteristics of all participants were compared between the groups upon admission using one-way analysis of variance (ANOVA). One-way ANOVA was used to evaluate the THI, VAS loud, and VAS uncomfortable scores. The least significant difference test was used as a post-hoc test. In total, 39 subjects were eligible for analysis. No differences in THI and VAS loudness scores were observed between groups. The VAS uncomfortable scores decreased significantly in MA and DE compared with those in PE. Within the group, all three treatments showed some effect on THI, VAS loudness scores and VAS uncomfortable scores after treatment except DE in THI. The authors concluded that there was no statistically significant difference between systemic manual acupuncture, periauricular electroacupuncture and distal electroacupuncture in tinnitus. However, all three treatments had some effect on tinnitus within the group before and after treatment. Systemic manual acupuncture and distal electroacupuncture have some effect on VAS. Neither of the three treatments tested in this study have been previously proven to work. Therefore, it is quite simply nonsensical to compare them. Comparative studies are indicated only with therapies that have a solid evidence-base. They are called ‘superiority trials’ and require a different statistical approach as well as much larger sample sizes. In other words, this study was an unethical waste of resources from the outset. With this in mind, there is only one conclusion that fits the data: there was no statistically significant difference between the three types of acupuncture. The data are therefore in keeping with the notion that all three are placebos. Alternatively one might conclude more clearly for those who are otherwise resistant to learning a lesson: POORLY DESIGNED CLINICAL TRIALS ARE UNETHICAL AND NEVER LEND THEMSELVES TO MEANINGFUL CONCLUSIONS.
A new study published in JAMA investigated the long-term effects of acupuncture compared with sham acupuncture and being placed in a waiting-list control group for migraine prophylaxis. The trial was a 24-week randomized clinical trial (4 weeks of treatment followed by 20 weeks of follow-up). Participants were randomly assigned to 1) true acupuncture, 2) sham acupuncture, or 3) a waiting-list control group. The trial was conducted from October 2012 to September 2014 in outpatient settings at three clinical sites in China. Participants 18 to 65 years old were enrolled with migraine without aura based on the criteria of the International Headache Society, with migraine occurring 2 to 8 times per month.
Participants in the true acupuncture and sham acupuncture groups received treatment 5 days per week for 4 weeks for a total of 20 sessions. Participants in the waiting-list group did not receive acupuncture but were informed that 20 sessions of acupuncture would be provided free of charge at the end of the trial. Participants used diaries to record migraine attacks. The primary outcome was the change in the frequency of migraine attacks from baseline to week 16. Secondary outcome measures included the migraine days, average headache severity, and medication intake every 4 weeks within 24 weeks.
A total of 249 participants 18 to 65 years old were enrolled, and 245 were included in the intention-to-treat analyses. Baseline characteristics were comparable across the 3 groups. The mean (SD) change in frequency of migraine attacks differed significantly among the 3 groups at 16 weeks after randomization; the mean (SD) frequency of attacks decreased in the true acupuncture group by 3.2 (2.1), in the sham acupuncture group by 2.1 (2.5), and the waiting-list group by 1.4 (2.5); a greater reduction was observed in the true acupuncture than in the sham acupuncture group (difference of 1.1 attacks; 95% CI, 0.4-1.9; P = .002) and in the true acupuncture vs waiting-list group (difference of 1.8 attacks; 95% CI, 1.1-2.5; P < .001). Sham acupuncture was not statistically different from the waiting-list group (difference of 0.7 attacks; 95% CI, −0.1 to 1.4; P = .07).
The authors concluded that among patients with migraine without aura, true acupuncture may be associated with long-term reduction in migraine recurrence compared with sham acupuncture or assigned to a waiting list.
Note the cautious phraseology: “… acupuncture may be associated with long-term reduction …”
The authors were, of course, well advised to be so atypically cautious:
- Comparisons to the waiting list group are meaningless for informing us about the specific effects of acupuncture, as they fail to control for placebo-effects.
- Comparisons between real and sham acupuncture must be taken with a sizable pinch of salt, as the study was not therapist-blind and the acupuncturists may easily have influenced their patients in various ways to report the desired result (the success of patient-blinding was not reported but would have gone some way to solving this problem).
- The effect size of the benefit is tiny and of doubtful clinical relevance.
My biggest concern, however, is the fact that the study originates from China, a country where virtually 100% of all acupuncture studies produce positive (or should that be ‘false-positive’?) findings and data fabrication has been reported to be rife. These facts do not inspire trustworthiness, in my view.
So, does acupuncture work for migraine? The current Cochrane review included 22 studies and its authors concluded that the available evidence suggests that adding acupuncture to symptomatic treatment of attacks reduces the frequency of headaches. Contrary to the previous findings, the updated evidence also suggests that there is an effect over sham, but this effect is small. The available trials also suggest that acupuncture may be at least similarly effective as treatment with prophylactic drugs. Acupuncture can be considered a treatment option for patients willing to undergo this treatment. As for other migraine treatments, long-term studies, more than one year in duration, are lacking.
So, maybe acupuncture is effective. Personally, I am not convinced and certainly do not think that the new JAMA study significantly strengthened the evidence.
You may recall, we have dealt with the JCAM many times before; for instance here, here, here and here. Now they have come out with another remarkable paper. This study – no, the authors called it a ‘pilot study’ – was to compare the efficacy of Emotional Freedom Techniques (EFT) with that of Cognitive-Behavioral Therapy (CBT) in reducing adolescent anxiety. Sixty-three American high-ability students in grades 6–12, ages 10–18 years, who scored in the moderate to high ranges for anxiety on the Revised Children’s Manifest Anxiety Scale-2 (RCMAS-2) were randomly assigned to one of three groups:
- CBT (n = 21),
- EFT (n = 21),
- or waitlist control (n = 21).
EFT is an alternative therapy that incorporates acupoint stimulation. Students assigned to the CBT or EFT treatment groups received three individual sessions of the identified protocols from trained graduate counseling, psychology, or social work students enrolled at a large northeastern research university. The RCMAS-2 was used to assess preintervention and postintervention anxiety levels in participants.
EFT participants showed significant reduction in anxiety levels compared with the waitlist control group with a moderate to large effect size. CBT participants did not differ significantly from the EFT or control.
The authors concluded that EFT is an efficacious intervention to significantly reduce anxiety for high-ability adolescents.
They also state in their abstract that EFT is an evidence-based treatment for anxiety…
Are you happy with these conclusions?
Are you convinced that this trial lends itself to establish efficacy of anything?
Are you impressed with the trial design, the sample size, etc?
Are you sure that EFT is plausible, credible or evidence-based in any way?
If you look up EFT, you will find that there is a surprising amount of papers on it. Most of them have one thing in common: they were published in highly dubious journals. The field does not inspire trust or competence. The authors of the study state that EFT is an easily implemented strategy that uses such techniques as awareness building, exposure, reframing of interpretation, and systematic desensitization, while teaching the participant to self-stimulate protocol-identified acupoints (i.e., acupuncture points) by tapping. The effectiveness of acupuncture for treating anxiety has been well documented. Rather than using acupuncture needles, EFT relies on the manual stimulation of the acupoints. A recent meta-analysis indicated that interventions using acupoint stimulation had a moderate effect size (Hedge’s g = −0.66 95% CI [−0.99, −0.33]) in reducing symptoms. In EFT, the client stimulates the protocol-identified acupoints by tapping on them. Preliminary studies have suggested that tapping and other alternative ways of stimulating acupuncture points to be as effective as acupuncture needling. The EFT protocol and identified acupoints that were used in this study are the ones recommended for research purposes by the Association for Comprehensive Energy Psychology…
Wikipedia tells us that “Emotional Freedom Techniques (EFT) is a form of counseling intervention that draws on various theories of alternative medicine including acupuncture, neuro-linguistic programming, energy medicine, and Thought Field Therapy (TFT). It is best known through Gary Craig’s EFT Handbook, published in the late 1990s, and related books and workshops by a variety of teachers. EFT and similar techniques are often discussed under the umbrella term “energy psychology”. Advocates claim that the technique may be used to treat a wide variety of physical and psychological disorders, and as a simple form of self-administered therapy. The Skeptical Inquirer describes the foundations of EFT as “a hodgepodge of concepts derived from a variety of sources, [primarily] the ancient Chinese philosophy of chi, which is thought to be the ‘life force’ that flows throughout the body.” The existence of this life force is “not empirically supported”. EFT has no benefit as a therapy beyond the placebo effect or any known-effective psychological techniques that may be provided in addition to the purported “energy” technique. It is generally characterized as pseudoscience and it has not garnered significant support in clinical psychology.”
A recent systematic review of EFT concluded that “there were too few data available comparing EFT to standard-of-care treatments such as cognitive behavioral therapy, and further research is needed to establish the relative efficacy of EFT to established protocols.”
Notwithstanding these and many other verdicts on EFT, we now are asked to agree with the new study that EFT IS EFFICACIOUS.
Is this a joke?
They want us to believe this on the basis of a PILOT STUDY? Such studies are not even supposed to test efficacy! (Yet the authors of the trial state that this study was designed to meet the American Psychological Association (APA) Division 12 quality control criteria and the Consolidated Standards for Reporting Trials (CONSORT) criteria. I have to admit, they could have fooled me!)
No, it is not a joke, it is yet another nonsense from the ‘The Journal of Complementary and Alternative Medicine’ which, in my view, should henceforth be called THE JOURNAL OF ALTERNATIVE FACTS (JAF).
Mike Cummings recently stated on this blog “I’m not into blog banter.” Is he perhaps referring to some ‘alternative facts’? The truth seems to be that he blogs happily, regularly and – I am sad to say – disgracefully. This is a quote from his new post about the discussions regarding an acupuncture trial which was in the press a few days ago, (and also has been discussed on this blog):
START OF QUOTE
So there has been a big response to this paper press released by BMJ on behalf of the journal Acupuncture in Medicine. The response has been influenced by the usual characters – retired professors who are professional bloggers and vocal critics of anything in the realm of complementary medicine. They thrive on oiling up and flexing their EBM muscles for a baying mob of fellow sceptics (see my ‘stereotypical mental image’ here). Their target in this instant is a relatively small trial on acupuncture for infantile colic. Deserving of being press released by virtue of being the largest to date in the field, but by no means because it gave a definitive answer to the question of the efficacy of acupuncture in the condition. We need to wait for an SR where the data from the 4 trials to date can be combined. On this occasion I had the pleasure of joining a short segment on the Today programme on BBC Radio 4 led by John Humphreys. My protagonist was the ever-amusing David Colquhoun (DC), who spent his short air-time complaining that the journal was even allowed to be published in the first place. You can learn all about DC care of Wikipedia – he seems to have a surprisingly long write up for someone whose profession career was devoted to single ion channels, perhaps because a significant section of the page is devoted to his activities as a quack-busting blogger. So why would BBC Radio 4 invite a retired basic scientist and professional sceptic blogger to be interviewed alongside one of the journal editors – a clinician with expertise in acupuncture (WMA)? At no point was it made manifest that only one of the two had ever been in a position to try to help parents with a baby that they think cries excessively. Of course there are a lot of potential causes of excessive crying, but I am sure DC would agree that it is unlikely to be attributable to a single ion channel…
END OF QUOTE
I encourage everyone to read Cummings post in full; it’s full of surprises. Here I just want to comment very briefly why I find his post disgraceful (the Cummings quotes are in bold followed by my comments):
….usual characters….. Disrespectful to the point of being derogatory, in my view
….retired professors….. Not true, non-retired professionals commented as well
….professional bloggers…. Meaning people who earn their income by blogging? Certainly not true!
….vocal critics of anything in the realm of complementary medicine…. Critic not of ‘anything’ but merely of things that are false or misleading like the trial in question
….a baying mob of fellow sceptics…. Unquestionably meant to be insulting and arguably libelous
….Deserving of being press released by virtue of being the largest to date in the field…. Large is not necessarily a virtue that merits a press-release, particularly, if it is not matched with quality
….We need to wait for an SR where the data from the 4 trials to date can be combined…. More than doubtful that we ‘need to wait’. The 4 trials in question are all very weak and therefore cannot provide a firm answer via a systematic review
….the ever-amusing David Colquhoun…. Derogatory to the extreme
….why would BBC Radio 4 invite a retired basic scientist and professional sceptic blogger…. The answer could be because he understands science and has vast experience exposing false claims
….only one of the two had ever been in a position to try to help parents with a baby that they think cries excessively…. This does not necessarily mean that such a person understands science, and Cummings might even be an example for this
….is unlikely to be attributable to a single ion channel…. Even Cummings’ attempts at humour are quite appalling.
The comments of Dr Mike Cummings MB ChB Dip Med Ac, I am afraid, befit an ill-educated acupuncturist who feels personally hurt because his views have been challenged and who is not quite bright enough to have a rational discussion about his favourite subject, particularly with someone who has a superior grasp of the issues at hand (which are clearly not ‘how to stick a needle in a baby’). However, Cummings is not a simple acupuncturist; he happens to be a member of the medical profession, a medical director of the British Medical Acupuncture Society and (as he seems keen to point out) a journal editor. With these credentials, he should, in my view, be able to argue a bit more intelligently, truthfully and a lot more gracefully.
One could almost think he wants to give acupuncture a bad name!!!
Acupuncture is often recommended as a treatment for shoulder pain, but its effectiveness is far from proven. A new study has just been published; but does it change this uncertainty?
A total of 227 patients with subacromial pain syndrome were recruited to this RCT. The patients were allocated to three groups who received either A) group exercise, B) group exercise plus acupuncture or C) group exercise plus electro-acupuncture. The primary outcome measure was the Oxford Shoulder Score. Follow-up was post treatment, and at 6 and 12 months. Data were analysed on intention-to-treat principles with imputation of missing values.
Treatment groups were similar at baseline. All treatment groups demonstrated improvements over time. Between-group estimates were, however, small and non-significant.
The authors concluded that neither acupuncture nor electro-acupuncture were found to be more beneficial than exercise alone in the treatment of subacromial pain syndrome.
Well, that was to be expected!… I hear the rationalists amongst us exclaim.
Actually, I am not so sure.
One could easily have expected that the acupuncture groups (B and C) show a significant advantage over group A.
Because acupuncture is a ‘theatrical placebo’, a ritual that impresses patients and thus impacts on results, particularly on subjective outcomes like pain. If the results had shown a benefit for acupuncture + exercise (groups B and C) versus exercise alone (group A), what would we have made of it? Acupuncture fans would surely have claimed that it is evidence confirming acupuncture’s effectiveness. Sceptics, on the other hand, would have rightly insisted that it demonstrates nothing of the sort – it merely confirms that placebo effects can affect clinical outcomes such as pain.
As it turned out, however, this trial results happened to indicate that these placebo-effects can be so small that they fail to reach the level of statistical significance.
I think there is one noteworthy message here: RCTs with such a design (no adequate control for placebo effects) can easily generate false-positive results (in this case, this did not happen, but it was nevertheless a possible outcome). Such studies are popular but utterly useless: they don’t advance our knowledge one single iota. If that is so, we should not waste our resources on them because, in the final analysis, this is not ethical. In other words, we must stop funding research that has little or no chance of advancing our knowledge.
This new RCT was embargoed until today; so, I had to wait until I was able to publish my comments. Here are the essentials of the study:
The Swedish investigators compared the effect of two types of acupuncture versus no acupuncture in infants with colic in public child health centres (CHCs). The study was designed as a multicentre, randomised controlled, single-blind, three-armed trial (ACU-COL) comparing two styles of acupuncture with no acupuncture, as an adjunct to standard care. Among 426 infants whose parents sought help for colic and registered their child’s fussing/crying in a diary, 157 fulfilled the criteria for colic and 147 started the intervention.
Parallel to usual care, study participants visited the study CHC twice a week for 2 weeks. Thus, all infants received usual care plus 4 extra visits to a CHC, during which parents met a nurse for 20–30 min and were able to discuss their infant’s symptoms. Together these were considered to represent gold standard care. The nurse listened, and gave evidence-based advice and calming reassurance. Breastfeeding mothers were encouraged to continue breastfeeding. At each visit, the study nurse carried the infant to a separate treatment room where they were left alone with the acupuncturist for 5 min.
The acupuncturist treated the baby according to group allocation and recorded the treatment procedures and any adverse events. Disposable stainless steel 0.20×13 mm Vinco needles (Helio, Jiangsu Province, China) were used. Infants allocated to group A received standardised MA at LI4. One needle was inserted to a depth of approximately 3 mm unilaterally for 2–5 s and then withdrawn without stimulation. Infants allocated to group B received semi-standardised individualised acupuncture, mimicking clinical TCM practice. Following a manual, the acupuncturists were able to choose one point, or any combination of Sifeng, LI4 and ST36, depending on the infant’s symptoms, as reported in the diary. A maximum of five insertions were allowed per treatment. Needling at Sifeng consisted of 4 insertions, each to a depth of approximately 1 mm for 1 s. At LI4 and ST36, needles were inserted to a depth of approximately 3 mm, uni- or bilaterally. Needles could be retained for 30 seconds. De qi was not sought, therefore stimulation was similarly minimal in groups A and B. Infants in group C spent 5 min alone with the acupuncturist without receiving acupuncture.
The effect of the two types of acupuncture was similar and both were superior to gold standard care alone. Relative to baseline, there was a greater relative reduction in time spent crying and colicky crying by the second intervention week (p=0.050) and follow-up period (p=0.031), respectively, in infants receiving either type of acupuncture. More infants receiving acupuncture cried <3 hours/day, and thereby no longer fulfilled criteria for colic, in the first (p=0.040) and second (p=0.006) intervention weeks. No serious adverse events were reported.
The authors concluded that acupuncture appears to reduce crying in infants with colic safely.
Notice that the investigators are cautious and state in the abstract that “acupuncture appears to reduce crying…” Their conclusions from the actual article are, however, quite different; here they state the following:
Among those initially experiencing excessive infant crying, the majority of parents reported normal values once the infant’s crying had been evaluated in a diary and a diet free of cow’s milk had been introduced. Therefore, objective measurement of crying and exclusion of cow’s milk protein are recommended as first steps, to avoid unnecessary treatment. For those infants that continue to cry >3 hours/day, acupuncture may be an effective treatment option. The two styles of MA tested in ACU-COL had similar effects; both reduced crying in infants with colic and had no serious side effects. However, there is a need for further research to find the optimal needling locations, stimulation and treatment intervals.
Such phraseology is much more assertive and seems to assume acupuncture caused specific therapeutic effects. Yet, I think, this assumption is not warranted.
In fact, I believe, the study shows almost the opposite of what the authors conclude. Both minimal and TCM acupuncture seemed to reduce the symptoms of colic compared to no acupuncture at all. I think, this confirms previous research showing that acupuncture is a ‘theatrical placebo’. The study was designed without an adequate placebo group. It would have been easy to use some form of sham acupuncture in the control group. Why did the authors not do that? Heaven knows, but one might speculate that they were aiming for a positive result – and what better way to ensure it than with a ‘no treatment’ control group?
There are, of course, numerous other flaws. For instance, Prof David Colquhoun FRS, Professor of Pharmacology at University College London, criticised the study because of its lousy statistics:
START OF QUOTE
“It is truly astonishing that, in the 21st century, the BMJ still publishes a journal devoted to a form of pre-scientific medicine which after more than 3000 trials has still not been able to produce convincing evidence of efficacy1. Like most forms of alternative medicine, acupuncture has been advocated for a vast range of problems, and there is little evidence that it works for any of them. Colic has not been prominent in these claims. What parent would think that sticking needles into their baby would stop it crying? The idea sounds bizarre. It is. This paper certainly doesn’t show that it works.
“The statistical analysis in the paper is incompetent. This should have been detected by the referees, but wasn’t. For a start, the opening statement, ‘A two-sided P value ≤0.05 was considered statistically significant’ is simply unacceptable in the light of all recent work about reproducibility. Still worse, Table 1 uses the description ‘statistical tendency towards significance (p=0.051–0.1)’.
“Worst of all, Table 1 reports 24 different P values, of which three are (just) below 0.05. Yet no correction has been used for multiple comparisons. This is very bad practice. It’s highly unlikely that, if the proper correction had been done, any of the results would have given a type 1 error rate below 5%.
“Even were it not for this, most of the ‘significant’ P values are marginal (only slightly less than 0.05). It is now well known that the type 1 error rate gives an optimistic view. What matters is the false positive rate – the chance that a ‘significant’ result is a false positive. A p-value close to 0.05 implies that there is at least a 30% chance that they are false positives. If one thought, a priori, that the chance of colic being cured by sticking needles into a baby was less than 50%, the false positive rate could easily be greater than 80%2. It is now recognised that this misinterpretation of p-values is a major contributor to the crisis of reproducibility.
“Other problems concern the power calculation. A priori calculations of power are well-known to be overoptimistic, because small trials usually overestimate the effect size. In this case the initial estimated sample size was not attained, and a rather mysterious recalculation of power was used.
“Another small problem: the discussion points out that ‘the majority of infants in this cohort did not have colic’.
“The nature of the control group is not very clear. An appropriate control might have been to cuddle the baby – this was used in a study in which another implausible treatment, chiropractic, was shown not to work. This appears not to have been done.
“Lastly, p-values are reported in the text without mention of effect sizes. This is contrary to all statistical advice.
“In conclusion, the design of the trial is reasonable (apart from the control group) but the statistical analysis is appalling. It’s very likely that there aren’t any real effects of acupuncture at all. This paper serves more to muddy the waters than to add useful information. It’s a model for the sort of mistakes that have led to the crisis in reproducibility. The BMJ should not be publishing this sort of stuff, and the referees seem to have no understanding of statistics.”
END OF QUOTE
Despite these rather obvious – some would say fatal – flaws, the editor of ACUPUNCTURE IN MEDICINE (AIM) thought this trial to be so impressively rigorous that he issued a press-release about it. This, I think, is particularly telling, perhaps even humorous: it shows what kind of a journal AIM is, and also provides an insight into the state of acupuncture research in general.
The long and short of it is that conclusions about specific therapeutic effects of acupuncture are not permissible. We know that colicky babies respond even to minimal attention, and this trial confirms that even a little additional TLC in the form of acupuncture will generate an effect. The observed outcome is most likely unrelated to acupuncture.