Clinical trials of acupuncture can be quite challenging. In particular, it is often difficult to make sure that any observed outcome is truly due to the treatment and not caused by some other factor(s). How tricky this can be, shows a recently published study.
A new RCT has all (well, almost all) the features of a rigorous study. It tested the effects of acupuncture in patients suffering from hay fever. The German investigators recruited 46 specialized physicians in 6 hospital clinics and 32 private outpatient clinics. In total, 422 patients with IgE sensitization to birch and grass pollen were randomized into three groups: 1) acupuncture plus rescue medication (RM) (n= 212), 2) sham acupuncture plus RM (n= 102), or 3) RM alone (n= 108). Twelve acupuncture sessions were provided in groups 1 and 2 over 8 weeks. The outcome measures included changes in the Rhinitis Quality of Life Questionnaire (RQLQ) overall score and the RM score (RMs) from baseline to weeks 7, 8 and 16 in the first year as well as week 8 in the second year after randomization.
Compared with sham acupuncture and with RM, acupuncture was associated with improvement in RQLQ score and RMS. There were no differences after 16 weeks in the first year. After the 8-week follow-up phase in the second year, small improvements favoring real acupuncture over sham were noted.
Based on these results, the authors concluded that “acupuncture led to statistically significant improvements in disease-specific quality of life and antihistamine use measures after 8 weeks of treatment compared with sham acupuncture and with RM alone, but the improvements may not be clinically significant.”
The popular media were full of claims that this study proves the efficacy of acupuncture. However, I am not at all convinced that this conclusion is not hopelessly over-optimistic.
It might not have been the acupuncture itself that led to the observed improvements; they could well have been caused by several factors unrelated to the treatment itself. To understand my concern, we need to look closer at the actual interventions employed by the investigators.
The real acupuncture was done on acupuncture points thought to be indicated for hay fever. The needling was performed as one would normally do it, and the acupuncturists were asked to treat the patients in group 1 in such a way that they were likely to experience the famous ‘de-qi’ feeling.
The sham acupuncture, by contrast, was performed on non-acupuncture points; acupuncturists were asked to use shallow needling only and they were instructed to try not to produce ‘de-qi’.
This means that the following factors in combination or alone could have caused [and in my view probably did cause] the observed differences in outcomes between the acupuncture and the sham group:
1) verbal or non-verbal communication between the acupuncturists and the patient [previous trials have shown this factor to be of crucial importance]
2) the visibly less deep needling in the sham-group
3) the lack of ‘de-qi’ experience in the sham-group.
Sham-treatments in clinical trials serve the purpose of a placebo. They are thus meant to be indistinguishable from the verum. If that is not the case [as in the present study], the trial cannot be accepted as being patient-blind. If a trial is not patient-blind, the expectations of patients will most certainly influence the results.
Therefore I believe that the marginal differences noted in this study were not due to the effects of acupuncture per se, but were an artifact caused through de-blinding of the patients. De facto, neither the patients nor the acupuncturists were blinded in this study.
If that is true, the effects were not just not clinically relevant, as noted by the authors, they also had nothing to do with acupuncture. In other words, acupuncture is not of proven efficacy for this condition – a verdict which is also supported by our systematic review of the subject which concluded that “the evidence for the effectiveness of acupuncture for the symptomatic treatment or prevention of allergic rhinitis is mixed. The results for seasonal allergic rhinitis failed to show specific effects of acupuncture…”
Once again, we have before us a study which looks impressive at first glance. At closer scrutiny, we find, however, that it had important design flaws which led to false positive results and conclusions. In my view, it would have been the responsibility of the authors to discuss these limitations in full detail and to draw conclusions that take them into account. Moreover, it would have been the duty of the peer-reviewers and journal editors to pick up on these points. Instead the editors even commissioned an accompanying editorial which displays an exemplary lack of critical thinking.
Having failed to do any of this, they are in my opinion all guilty of misleading the world media who reported extensively and often uncritically on this new study thus misleading us all. Sadly, the losers in this bonanza of incompetence are the many hay fever sufferers who will now be trying (and paying for) useless treatments.
13 Responses to Acupuncture for hay fever: breakthrough or bonanza of incompetence?
Leave a Reply to Karina Sutton Cancel reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
as if to support my post, Chinese researchers just published another trial http://www.ncbi.nlm.nih.gov/pubmed/23439629 for which more or less the came criticism applies.
here is the abstract in full:
BACKGROUND:The traditional Chinese theory of acupuncture emphasizes that the intensity of acupuncture must reach a threshold to generate de qi, which is necessary to acheive the best therapeutic effect. De qi is an internal compound sensation of soreness, tingling, fullness, aching, cool, warmth and heaviness, and a radiating sensation at and around the acupoints. However, the notion that de qi must be acheived for maximum benefit has not been confirmed by modern scientific evidence. METHODS:We performed a prospective multicentre randomized controlled trial involving patients with Bell palsy. Patients were randomly assigned to the de qi (n = 167) or control (n = 171) group. Both groups received acupuncture: in the de qi group, the needles were manipulated manually until de qi was reached, whereas in the control group, the needles were inserted without any manipulation. All patients recevied prednisone as a basic treatment. The primary outcome was facial nerve function at month 6. We also assessed disability and quality of life 6 months after randomization. RESULTS:After 6 months, patients in the de qi group had better facial function (adjusted odds ratio [OR] 4.16, 95% confidence interval [CI] 2.23-7.78), better disability assessment (differences of least squares means 9.80, 95% CI 6.29-13.30) and better quality of life (differences of least squares means 29.86, 95% CI 22.33-37.38). Logistic regression analysis showed a positive effect of the de qi score on facial-nerve function (adjusted OR 1.07, 95% CI 1.04-1.09). INTERPRETATION:Among patients with Bell palsy, acupuncture with strong stimulation that elicited de qi had a greater therapeutic effect, and stronger intensity of de qi was associated with the better therapeutic effects
I previously thought that the new sham needle design with the adhesive base and retracting dull needle body was allowing truly double-blind treatments for the first time, but it appears that the slightly painful sensation of De Qi is a confounding factor which is hard to replicate with a non-penetrating placebo needle. That German study which used toothpicks to stimulate the skin may be closer, as they would product some slightly painful sensation, too (but not in a double-blind way).
A study design of traditional acupuncture point functions which may reveal more truth would be to design one treatment to improve the symptoms and one which (in TCM theory) would make them worse. Instead of a placebo arm, this would be a counter-therapy arm. 4 groups would be ideal, like the Placebo Balanced Study Design talked about by Benedetti: 1 told points were being done to help the situation and the TCM positive points used, 1 told points would make it worse and the same TCM positive points used, 1 told the points would help with counter-to-theory points done, and lastly 1 told the points would help with the counter-theory points done. This could fairly easily be done with a few common conditions such as headache, high blood pressure, perhaps indigestion…
I understand that spending limited resources testing things which are implausible are already disproven is a low priority for medical research. However, I’m still interested in how a study should be designed to shed light on placebo effects, traditional claims, and reality. _Trick or Treatment_ was a huge step forward for me as an acupuncturist in thinking more accurately about issues in research. _Snake Oil Science_ was the next big step, especially where acupuncture research is concerned. Now when I read about a new study, I am very suspicious of bias and look for how the placebo/control arm was designed.
Thanks for the thought-provoking writings, Edzard!
Would it not be sufficient to use normal acupuncture needles, but simply to place them in the “wrong” places, in order to prove at least any “theory” behind acupuncture is bogus? It does not allow for complete blinded treatment to sticking in needles, but it blinds to the “correct” insertion points “indicated” for the conditions. Isn’t that enough to blow away most of the claptrap surrounding acupuncture?
The basic problem – it seems to me – is that a true placebo that is indistinguishable from real acupuncture is not possible by definition. A placebo should be inert – placing needles anywhere in the body has been shown to release endorphins, therefore needling ‘non-acupuncture points’ is no good, and neither is shallow needling. With this in mind, if the ‘real’ treatment needs to be indistinguishable to the patient from the ‘sham’, and deqi is elicited in the ‘real’ group, how can this be done in an inert fashion?
Also, an important aspect of acupuncture theory is the ‘ashi point’. These are points that are painful on palpation, and can occur almost anywhere. They can bring about radical changes in a patient’s condition, and it is quite possible one or more of these could be inadvertently needled.
As for the practitioner being blind to whether the treatment is real or not, that is simply impossible if proper acupuncture is being investigated. A skilled acupuncturist will manipulate the needle in various ways, depending on the patient and the condition being treated. To use acupuncture naive practitioners in a trial – which they would have to be in order to maintain ignorance about the validity of the treatment – would be akin to asking a plumber to perform heart surgery – you shouldn’t expect good results!
These are the kinds of problems facing acupuncture research. Comparative studies seem a far more useful test of acupuncture’s efficacy, but sadly these are not taken very seriously.
I received Acupuncture, When I Had Finished My 1st Session, As Soon As I Came Out My Knee, started swelling Badly.
This Has given me problems ever since, My Dr Gives Me Anti Inflammatorys,It Help For short time.l actually went for
Actually Went For Back Problems, And Ended Up Coming Out With Severe Knee Problems.
This Has given me problems ever since, My Dr Gives Me Anti Inflammatorys,It Help For short time.l actually went for back problems, It did nothing but give me more problems, which was as l Say my knee ( right knee ). There was this Chinese Man who could not speck English who She Had Visiting and was there the whole time as Well. My Dr gave me a letter to take with me showing Where my problems in my back were, When I tried to give these to her, She just pushed these back at me across her desk, without even reading them. So how did she know what she was even treating me for.
Korea is believed to be the second country that acupuncture spread to outside of China. Within Korea there is a legend that acupuncture was developed by the legendary emperor Dangun though it is more likely to have been brought into Korea from a Chinese colonial prefecture.
The Lady I Went to was at Reedy Creek on the Gold Coast.When l was called in, She Was Very Nasty In Nature.
I can now hardly walk at times and it is becoming more Frequently.
Is there any way of having this reversed, but not with Acupuncture???