Clinical trials of acupuncture can be quite challenging. In particular, it is often difficult to make sure that any observed outcome is truly due to the treatment and not caused by some other factor(s). How tricky this can be, shows a recently published study.
A new RCT has all (well, almost all) the features of a rigorous study. It tested the effects of acupuncture in patients suffering from hay fever. The German investigators recruited 46 specialized physicians in 6 hospital clinics and 32 private outpatient clinics. In total, 422 patients with IgE sensitization to birch and grass pollen were randomized into three groups: 1) acupuncture plus rescue medication (RM) (n= 212), 2) sham acupuncture plus RM (n= 102), or 3) RM alone (n= 108). Twelve acupuncture sessions were provided in groups 1 and 2 over 8 weeks. The outcome measures included changes in the Rhinitis Quality of Life Questionnaire (RQLQ) overall score and the RM score (RMs) from baseline to weeks 7, 8 and 16 in the first year as well as week 8 in the second year after randomization.
Compared with sham acupuncture and with RM, acupuncture was associated with improvement in RQLQ score and RMS. There were no differences after 16 weeks in the first year. After the 8-week follow-up phase in the second year, small improvements favoring real acupuncture over sham were noted.
Based on these results, the authors concluded that “acupuncture led to statistically significant improvements in disease-specific quality of life and antihistamine use measures after 8 weeks of treatment compared with sham acupuncture and with RM alone, but the improvements may not be clinically significant.”
The popular media were full of claims that this study proves the efficacy of acupuncture. However, I am not at all convinced that this conclusion is not hopelessly over-optimistic.
It might not have been the acupuncture itself that led to the observed improvements; they could well have been caused by several factors unrelated to the treatment itself. To understand my concern, we need to look closer at the actual interventions employed by the investigators.
The real acupuncture was done on acupuncture points thought to be indicated for hay fever. The needling was performed as one would normally do it, and the acupuncturists were asked to treat the patients in group 1 in such a way that they were likely to experience the famous ‘de-qi’ feeling.
The sham acupuncture, by contrast, was performed on non-acupuncture points; acupuncturists were asked to use shallow needling only and they were instructed to try not to produce ‘de-qi’.
This means that the following factors in combination or alone could have caused [and in my view probably did cause] the observed differences in outcomes between the acupuncture and the sham group:
1) verbal or non-verbal communication between the acupuncturists and the patient [previous trials have shown this factor to be of crucial importance]
2) the visibly less deep needling in the sham-group
3) the lack of ‘de-qi’ experience in the sham-group.
Sham-treatments in clinical trials serve the purpose of a placebo. They are thus meant to be indistinguishable from the verum. If that is not the case [as in the present study], the trial cannot be accepted as being patient-blind. If a trial is not patient-blind, the expectations of patients will most certainly influence the results.
Therefore I believe that the marginal differences noted in this study were not due to the effects of acupuncture per se, but were an artifact caused through de-blinding of the patients. De facto, neither the patients nor the acupuncturists were blinded in this study.
If that is true, the effects were not just not clinically relevant, as noted by the authors, they also had nothing to do with acupuncture. In other words, acupuncture is not of proven efficacy for this condition – a verdict which is also supported by our systematic review of the subject which concluded that “the evidence for the effectiveness of acupuncture for the symptomatic treatment or prevention of allergic rhinitis is mixed. The results for seasonal allergic rhinitis failed to show specific effects of acupuncture…”
Once again, we have before us a study which looks impressive at first glance. At closer scrutiny, we find, however, that it had important design flaws which led to false positive results and conclusions. In my view, it would have been the responsibility of the authors to discuss these limitations in full detail and to draw conclusions that take them into account. Moreover, it would have been the duty of the peer-reviewers and journal editors to pick up on these points. Instead the editors even commissioned an accompanying editorial which displays an exemplary lack of critical thinking.
Having failed to do any of this, they are in my opinion all guilty of misleading the world media who reported extensively and often uncritically on this new study thus misleading us all. Sadly, the losers in this bonanza of incompetence are the many hay fever sufferers who will now be trying (and paying for) useless treatments.
This week is acupuncture awareness week, and I will use this occasion to continue focusing on this therapy. This first time ever event is supported by the British Acupuncture Council who state that it aims to “help better inform people about the ancient practice of traditional acupuncture. With 2.3 million acupuncture treatments carried out each year, acupuncture is one of the most popular complementary therapies practised in the UK today.“
Right, let’s inform people about acupuncture then! Let’s show them that there is often more to acupuncture research than meets the eye.
My team and I have done lots of research into acupuncture and probably published more papers on this than any other subject. We had prominent acupuncturists on board from the UK, Korea, China and Japan, we ran conferences, published books and are proud to have been innovative and productive in our multidisciplinary research. But here I do not intend to dwell on our own achievements, rather I will highlight several important new papers in this area.
Korean authors just published a meta-analysis to assess the effectiveness of acupuncture as therapy for gouty arthritis. Ten RCTs involving 852 gouty arthritis patients were included. Six studies of 512 patients reported a significant decrease in uric acid in the treatment group compared with a control group, while two studies of 120 patients reported no such effect. The remaining four studies of 380 patients reported a significant decrease in pain in the treatment group.
The authors conclude “that acupuncture is efficacious as complementary therapy for gouty arthritis patients”.
We should be delighted with such a positive and neat result! Why then do I hesitate and have doubts?
I believe that this paper reveals several important issues in relation to systematic reviews of Chinese acupuncture trials and studies of other TCM interventions. In fact, this is my main reason for discussing the new meta-analysis here. The following three points are crucial, in my view:
1) All the primary studies were from China, and 8 of the 10 were only available in Chinese.
2) All of them had major methodological flaws.
3) It has been shown repeatedly that all acupuncture-trials from China are positive.
Given this situation, the conclusions of any review for which there are only Chinese acupuncture studies might as well be written before the actual research has started. If the authors are pro-acupuncture, as the ones of the present article clearly are, they will conclude that “acupuncture is efficacious“. If the research team has some critical thinkers on board, the same evidence will lead to an entirely different conclusion, such as “due to the lack of rigorous trials, the evidence is less than compelling.“
Systematic reviews are supposed to be the best type of evidence we currently have; they are supposed to guide therapeutic decisions. I find it unacceptable that one and the same set of data could be systematically analysed to generate such dramatically different outcomes. This is confusing and counter-productive!
So what is there to do? How can we prevent being misled by such articles? I think that medical journals should refuse to publish systematic reviews which so clearly lack sufficient critical input. I also believe that reviewers of predominantly Chinese studies should provide English translations of these texts so that they can be independently assessed by those who are not able to read Chinese – and for the sake of transparency, journal editors should insist on this point.
And what about the value of acupuncture for gouty arthritis? I think I let the readers draw their own conclusion.
Everyone knows, I think, that smoking is bad for our health. Why then do so many of us still smoke? Because smoking is addictive – and addictions are, by definition, far from easy to get rid of. Many smokers try acupuncture, and acupuncturists are making a ‘pretty penny’ on the assumption that their treatment is an effective way to stop the habit. But what does the best evidence tell us?
A new randomized, double-blind, placebo-controlled clinical trial with 125 smokers was conducted to determine whether ear acupuncture with electrical stimulation (auriculotherapy) once a week for 5 consecutive weeks is more effective than sham treatment.
The results showed that there was no difference in the rate of smoking cessation between the two groups. After 6 weeks, the auriculotherapy group achieved a rate of 20.9% abstinence which was not significantly different from the 17.9% in the sham group.
The authors of this study concluded that “the results … do not support the use of auriculotherapy to assist with smoking cessation. It is possible that a longer treatment duration, more frequent sessions, or other modifications of the intervention protocol used in this study may result in a different outcome. However, based on the results of this study, there is no evidence that auriculotherapy is superior to placebo when offered once a week for 5 weeks, as described in previous uncontrolled studies.”
Of course, they are correct to state that, theoretically, a different treatment regimen might have generated different outcomes. But how likely is that in reality?
To answer this question, we might consult the Cochrane review on the subject (which incidentally is close to my heart: I initiated it many years ago and was its senior author until it was plagiarised by my former co-worker and my name was replaced by that of his new boss [never a dull day in alternative medicine research!]).
The latest version of this article concludes that “there is no consistent, bias-free evidence that acupuncture, acupressure, laser therapy or electrostimulation are effective for smoking cessation, but lack of evidence and methodological problems mean that no firm conclusions can be drawn. Further, well designed research into acupuncture, acupressure and laser stimulation is justified since these are popular interventions and safe when correctly applied, though these interventions alone are likely to be less effective than evidence-based interventions”
This is a very, very (yes, I meant very, very) odd conclusion, I think. If I had still been an author of this plagiarised paper, I would have suggested something a little more straightforward: 33 studies of various types of acupuncture for smoking cessation are currently available (if we include the new trial, the number is 34). The totality of this evidence fails to show that acupuncture is effective. Therefore acupuncture should NOT be considered a valid option for this indication.
Since weeks I have been searching for new (2013) studies which actually report POSITIVE results. I like good news as much as the next man but, in my line of business, it seems awfully hard to come by. Therefore I am all the more delighted to present these two new articles to my readers.
The first study is a randomized trial with patients suffering from metastatic cancer who received one of three interventions: massage therapy, no-touch intervention or usual care. Primary outcomes were pain, anxiety, and alertness; secondary outcomes were quality of life and sleep. The mean number of massage therapy sessions per patient was 2.8.
The results show significant improvement in the quality of life of the patients who received massage therapy after 1-week follow-up which was not observed in either of the other groups. Unfortunately, the difference was not sustained at 1 month. There were also trends towards improvement in pain and sleep of the patients after massage. No serious adverse events were noted.
The authors conclude that “providing therapeutic massage improves the quality of life at the end of life for patients and may be associated with further beneficial effects, such as improvement in pain and sleep quality. Larger randomized controlled trials are needed to substantiate these findings“.
The second study examined the effectiveness of a back massage for improving sleep quality in 60 postpartum women suffering from poor sleep. They were randomized to either the intervention or the control group. Participants in both groups received the same care except for the back massages. The intervention group received one 20-minutes back massage at the same time each evening for 5 consecutive days by a certified massage therapist. The outcome measure was the Pittsburgh Sleep Quality Index (PSQI). The results showed that the changes in mean PSQI were significantly lower in the intervention group than in controls indicating a positive effect of massage on sleep quality.
The authors’ conclusions were clear: “an intervention involving back massage in the postnatal period significantly improved the quality of sleep.”
Where I was trained (Germany), massage is not deemed to be an alternative but an entirely mainstream treatment. Despite this fact, there is precious little evidence to demonstrate that it is effective. Our own research has found encouraging evidence for a range of conditions, including autism, cancer palliation, constipation, DOMS and back pain. In addition, we have shown that massage is not entirely free of risks but that its potential for harm is very low (some might say that this was never in question but it is good to have a bit more solid evidence).
The new studies are, of course, not without flaws; this can hardly be expected in an area where logistical, financial and methodological problems abound. The fact that there are many different approaches to massage does not make things easier either. The new evidence is nevertheless encouraging and seems to suggest that massage has relaxing effects which are clinically relevant. In my view, massage is a therapy worth considering for more rigorous research.
Acupuncture remains a highly controversial treatment: its mechanism of action is less than clear and the clinical results are equally unconvincing. Of course, one ought to differentiate between different conditions; the notion that acupuncture is a panacea is most certainly nonsense.
In many countries, acupuncture is being employed mostly in the management of pain, and it is in this area where the evidence is perhaps most encouraging. Yet, even here the evidence from the most rigorous clinical trials seems to suggest that much, if not all of the effects of acupuncture might be due to placebo.
Moreover, we ought to be careful with generalisations and ask what type of pain? One very specific pain is that caused by aromatase inhibitors (AI), a medication frequently prescribed to women suffering from breast cancer. Around 50 % of these patients complain of AI-associated musculoskeletal symptoms (AIMSS) and 15 % discontinue treatment because of these complaints. So, can acupuncture help these women?
A recent randomised, sham-controlled trial tested whether acupuncture improves AIMSS. Postmenopausal women with early stage breast cancer, experiencing AIMSS were randomized to eight weekly real or sham acupuncture sessions. The investigators evaluated changes in the Health Assessment Questionnaire Disability Index (HAQ-DI) and pain visual analog scale (VAS). Serum estradiol, β-endorphin, and proinflammatory cytokine concentrations were also measured pre and post-intervention. In total, 51 women were enrolled of whom 47 were evaluable (23 randomized to real and 24 to sham acupuncture).
Baseline characteristics turned out to be balanced between groups with the exception of a higher HAQ-DI score in the real acupuncture group. The results failed to show a statistically significant difference in reduction of HAQ-DI or VAS between the two groups. Following eight weekly treatments, a significant reduction of IL-17 was noted in both groups. No significant modulation was seen in estradiol, β-endorphin, or other proinflammatory cytokine concentrations in either group. No difference in AIMSS changes between real and sham acupuncture was seen.
Even though this study was not large, it was rigorously executed and well-reported. As many acupuncturists claim that their treatment alleviates pain and as many women suffering from AM-induced pain experience benefit, acupuncture advocates will nevertheless claim that the findings of this study are wrong, misleading or irrelevant. The often remarkable discrepancy between experience and evidence will again be the subject of intense discussions. How can a tiny trial overturn the experience of so many?
The answer is: VERY EASILY! In fact, the simplest explanation is that both are correct. The trial was well-done and its findings are thus likely to be true. The experience of patients is equally true – yet it relies not on the effects of acupuncture per se, but on the context in which it is given. In simple language, the effects patients experience after acupuncture are due to a placebo-response.
This is the only simple explanation which tallies with both the evidence and the experience. Once we think about it carefully, we realise that acupuncture is highly placebo-genic:
It is exotic.
It is invasive.
It is slightly painful.
It involves time with a therapist.
It involves touch.
If anyone had the task to develop a treatment that maximises placebo-effects, he could not come up with a better intervention!
Ahhh, will acupuncture-fans say, this means that acupuncture is a helpful therapy. I don’t care how it works, as long as it does help. Did we not just cover this issues in some detail? Indeed we did – and I do not feel like re-visiting the three fallacies which underpin this sentence again.
During the last decade, Professor Claudia Witt and co-workers from the Charite in Berlin have published more studies of homeopathy than any other research group. Much of their conclusions are over-optimistic and worringly uncritical, in my view. Their latest article is on homeopathy as a treatment of eczema. As it happens, I have recently published a systematic review of this subject; it concluded that “the evidence from controlled clinical trials… fails to show that homeopathy is an efficacious treatment for eczema“. The question therefore arises whether the latest publication of the Berlin team changes my conclusion in any way.
Their new article describes a prospective multi-centre study which included 135 children with mild to moderate atopic eczema. The parents of the kids enrolled in this trial were able to choose either homeopathic or conventional doctors for their children who treated them as they saw fit. The article gives only scant details about the actual treatments administered. The main outcome of the study was a validated symptom score at 36 months. Further endpoints included quality of life, conventional medicine consumption, safety and disease related costs at six, 12 and 36 months.
The results showed no significant differences between the groups at 36 months. However, the children treated conventionally seemed to improve quicker than those in the homeopathy group. The total costs were about twice higher in the homoeopathic compared to the conventional group. The authors conclude as follows: “Taking patient preferences into account, while being unable to rule out residual confounding, in this long-term observational study, the effects of homoeopathic treatment were not superior to conventional treatment for children with mild to moderate atopic eczema, but involved higher costs“.
At least one previous report of this study has been available for some time and had thus been included in my systematic review. It is therefore unlikely that this new analysis might change my conclusion, particularly as the trial by Witt et al has many flaws. Here are just some of the most obvious ones:
Patients were selected according to parents’ preferences.
This means expectations could have played an important role.
It also means that the groups were not comparable in various, potentially important prognostic variables.
Even though much of the article reads as though the homeopaths exclusively employed homeopathic remedies, the truth is that both groups received similar amounts of conventional care and treatments. In other words, the study followed a ‘A+B versus B’ design (here is the sentence that best gives the game away “At 36 months the frequency of daily basic skin care was… comparable in both groups, as was the number of different medications (including corticosteroids and antihistamines)…”). I have previously stated that this type of study-design can never produce a negative result because A+B is always more than B.
Yet, at first glance, this new study seems to prove my thesis wrong: even though the parents chose their preferred options, and even though all patients were treated conventionally, the addition of homeopathy to conventional care failed to produce a better clinical outcome. On the contrary, the homeopathically treated kids had to wait longer for their symptoms to ease. The only significant difference was that the addition of homeopathy to conventional eczema treatments was much more expensive than conventional therapy alone (this finding is less than remarkable: even the most useless additional intervention costs money).
So, is my theory about ‘A+B versusB’ study-designs wrong? I don’t think so. If B equals zero, one would expect exactly the finding Witt et al produced: A+0=A. In turn, this is not a compliment for the homeopaths of this study: they seem to have been incapable of even generating a placebo-response. And this might indicate that homeopathy was not even usefull as a means to generate a placebo-response. Whatever interpretation one adopts, this study tells us very little of value (as children often grow out of eczema, we cannot even be sure whether the results are not simply a reflection of the natural history of the disease); in my view, it merely demonstrates that weak study designs can only create weak findings which, in this particular case, are next to useless.
The study was sponsored by the Robert Bosch Stiftung, an organisation which claims to be dedicated to excellence in research and which has, in the past, spent millions on researching homeopathy. It seems doubtful that trials of this caliber can live up to any claim of excellence. In any case, the new analysis is certainly no reason to change the conclusion of my systematic review.
To their credit, Witt et al are well aware of the many weaknesses of their study. Perhaps in an attempt to make them appear less glaring, they stress that “the aim of this study was to reflect the real world situation“.Usually I do not accept the argument that pragmatic trials cannot be rigorous – but I think Witt et al do have a point here: the real word tells us that homeopathic remedies are pure placebos!