Cochrane reviews have the reputation to be the most reliable evidence available anywhere. They are supposed to be independent, rigorous, transparent and up-to-date. Usually, this reputation is justified, in my view. But do the 54 Cochrane reviews of acupuncture quoted in my previous post live up to it?
If one had to put the entire body of evidence in a nutshell, it would probably look something like this:
TOTAL NUMBER OF SYSTEMATIC REVIEWS = 54
POSITIVE CONCLUSIONS BASED ON MORE THAN ONE HIGH QUALITY STUDY = 2
FAILURE TO REACH CONVINCINGLY POSITIVE CONCLUSIONS = 52
The two positive reviews are on:
1) prevention of migraine
2) prevention of tension-type headache
Both of the positive reviews are by Linde et al.
Allow me to raise just a few further critical points:
- If I counted correctly, 19 of the 54 reviews are authored entirely by Chinese authors. Why could this be a problem? One reason could be that many Chinese authors seem to be biased in favour of acupuncture. Another reason could be that data fabrication is rife in China.
- Many if not most of the primary studies are published in Chinese. This means that it is impossible for most non-Chinese co-authors of the review as well as for the referees of the paper to check the accuracy of the data extraction.
- I counted a total of 15 reviews which were by authors who one could categorise as outspoken enthusiasts of acupuncture. In these cases, one might be concerned about the trustworthiness of the review’s conclusion.
- Many (some would say most) of the reviews cover subject areas which are frankly bizarre. Who would, for instance, consider acupuncture a plausible treatment for Glaucoma, Mumps or chronic hepatitis B?
- Despite almost all of the reviews demonstrating that there is no good reason to recommend acupuncture for the condition in question, hardly any of them draw a transparent, helpful and clear conclusion. One example might suffice: the review of acupuncture for hordeolum concluded that “Low‐certainty evidence suggests that acupuncture with or without conventional treatments may provide short‐term benefits for treating acute hordeolum…” Its Chinese authors reached this conclusion on the basis of 6 primary studies (all from China) which were all of lousy quality. In such a case, the only justified conclusion would be, in my view, something like this: THERE IS NO RELIABLE EVIDENCE …
Despite these serious limitations and avoidable confusions, the totality of the evidence from these 54 Cochrane reviews does send an important message: there is hardly a single condition for which acupuncture is clearly, convincingly and indisputably effective. What I find most regrettable, however, is that the Cochrane Collaboration allowed the often biased review authors to obscure this crucial message so thoroughly. One needs a healthy portion of critical thinking to get through to the truth here – and how many fans of acupuncture possess such a thing?
Moxibustion is an ancient variation of acupuncture using moxa made from dried mugwort (Artemisia argyi). It has long played an important role in the traditional heath care systems of China and other Asian countries. More recently, it has become popular also in the West. Practitioners use moxa sticks indirectly to warm acupuncture needles, or burn it close to the patient’s skin. Essentially, moxibustion is a treatment where acupuncture points are stimulated mainly or exclusively by the heat of burning moxa.
Because of moxibustion’s long history of usage and the fact that it is employed in many countries for a very wide range of conditions, some might argue that it has stood the ‘test of time’ and should be considered to be a well-established therapy. More critical thinkers would, however, point out that this is not an argument but a classical fallacy.
My team at Exeter regularly had research fellows from Korea and other Asian countries, and we managed to develop a truly productive cooperation. It enabled us to conduct systematic reviews including the Asian literature – and this is how we got involved in an unusual amount of research into moxibustion which, after all, is a fairly exotic alternative therapy. In 2010, we began a series of systematic reviews of moxibustion.
One of the first such articles included 9 RCTs testing the effectiveness of this treatment for stroke rehabilitation. Three RCTs reported favorable effects of moxibustion plus standard care on motor function versus standard care alone Three randomized clinical trials compared the effects of moxibustion on activities of daily living alone but failed to show favorable effects of moxibustion.
Also in 2010, our systematic review of RCTs of moxibustion as a treatment of ulcerative colitis (UC) concluded that current evidence is insufficient to show that moxibustion is an effective treatment of UC. Most of included trials had high risk of bias. More rigorous studies seem warranted.
Our (2010) systematic review od RCTs of moxibustion as a therapy in cancer care found that the evidence was limited to suggest moxibustion is an effective supportive cancer care in nausea and vomiting. However, all studies had a high risk of bias so effectively there was not enough evidence to draw any conclusion.
Our (2010) systematic review of RCTs of moxibustion for treating hypertension concluded that there was insufficient evidence to suggest that moxibustion is an effective treatment for hypertension.
Our (2010) systematic review of RCTs of moxibustion for constipation concluded as follows: Given that the methodological quality of all RCTs was poor, the results from the present review are insufficient to suggest that moxibustion is an effective treatment for constipation. More rigorous studies are warranted.
Our (2010) systematic review found few RCTs were available that test the effectiveness of moxibustion in the management of pain, and most of the existing trials had a high risk of bias. Therefore, more rigorous studies are required before the effectiveness of moxibustion for the treatment of pain can be determined.
Our (2011) systematic review of 14 RCTs of moxibustion for rheumatic conditions failed to provide conclusive evidence for the effectiveness of moxibustion compared with drug therapy in rheumatic conditions.
The, so far, last article in this series has only just been published. The purpose of this systematic review was to assess the efficacy of moxibustion as a treatment of chemotherapy-induced leukopenia. Twelve databases were searched from their inception through June 2014, without a language restriction. Randomized clinical trials (RCTs) were included, if moxibustion was used as the sole treatment or as a part of a combination therapy with conventional drugs for leukopenia induced by chemotherapy. Cochrane criteria were used to assess the risk of bias.
Six RCTs with a total of 681 patients met our inclusion criteria. All of the included RCTs were associated with a high risk of bias. The trials included patients with various types of cancer receiving ongoing chemotherapy or after chemotherapy. The results of two RCTs suggested the effectiveness of moxibustion combined with chemotherapy vs. chemotherapy alone. In four RCTs, moxibustion was more effective than conventional drug therapy. Six RCTs showed that moxibustion was more effective than various types of control interventions in increasing white blood cell counts.
Our conclusion: there is low level of evidence based on these six trials that demonstrates the superiority of moxibustion over drug therapies in the treatment of chemotherapy-induced leukopenia. However, the number of trials, the total sample size, and the methodological quality are too low to draw firm conclusions. Future RCTs appear to be warranted.
Was all this research for nothing?
I know many people who would think so. However, I disagree. If nothing else, these articles demonstrated several facts quite clearly:
- There is quite a bit of research even on the most exotic alternative therapy; sometimes one needs to look hard and include languages other than English.
- Studies from China and other Asian counties very rarely report negative results; this fact casts a dark shadow on the credibility of such data.
- The poor quality of trials in most areas of alternative medicine is lamentable and must be stimulus for researchers in this field to improve their act.
- Authors of systematic reviews must resist the temptation to draw positive conclusions based on flawed primary data.
- Moxibustion is a perfect example for demonstrating that the ‘test of time’ is no substitute for evidence.
- As for moxibustion, it cannot currently be considered an evidence-based treatment for any condition.
Alternative medicine thrives in the realm of common chronic conditions which conventional medicine cannot cure and which respond well to treatment with placebos. Irritable bowel syndrome (IBS) is such a condition, and IBS-sufferers who are often frustrated with the symptomatic relief conventional medicine has to offer are only too keen to try any therapy that promises help. There is hardly an alternative therapy which does not claim to be the solution to IBS-symptoms: herbal medicine, mind-body interventions, homeopathy (the subject of my next post), acupuncture, even ‘MOXIBUSTION‘.
Moxibustion is a derivative of acupuncture; instead of needles, this method employs heat to stimulate acupuncture points. Proponents believe that the effects of moxibustion are roughly equivalent to those of acupuncture but many acupuncturists feel that they are less powerful. One website explains: Moxibustion is a traditional Chinese medicine technique that involves the burning of mugwort, a small, spongy herb, to facilitate healing. Moxibustion has been used throughout Asia for thousands of years; in fact, the actual Chinese character for acupuncture, translated literally, means “acupuncture-moxibustion.” The purpose of moxibustion, as with most forms of traditional Chinese medicine, is to strengthen the blood, stimulate the flow of qi, and maintain general health.
Many proponents of moxibustion claim that their treatment works for IBS. The evidence is, however, far less clear. Two recent meta-analyses might tell us more.
The first systematic review and meta-analysis was published by Korean researchers and aimed at critically evaluating the current evidence on moxibustion for improving global symptoms of IBS. The authors conducted extensive searches and found a total of 20 RCTs to be included in their analyses. The risk of bias in these studies was generally high. Compared with pharmacological medications, moxibustion significantly alleviated overall IBS symptoms but there was a moderate inconsistency among the 7 RCTs. Moxibustion combined with acupuncture was more effective than pharmacological therapy but a moderate inconsistency among the 4 studies was found. When moxibustion was added to pharmacological medications or herbal medicine, no additive benefit of moxibustion was shown compared with pharmacological medications or herbal medicine alone. One small sham-controlled trial found no difference between moxibustion and sham control in symptom severity. Moxibustion appeared to be associated with few adverse events but the evidence is limited due to poor reporting.
The authors concluded that moxibustion may provide benefit to IBS patients although the risk of bias in the included studies is relatively high. Future studies are necessary to confirm whether this finding is reproducible in carefully-designed and conducted trials and to firmly establish the place of moxibustion in current practice.
The way I see it, these conclusions are far too optimistic. There was only one RCT that controlled for placebo-effects, and the results of that study were negative. Thus I would conclude that some studies report effectiveness of moxibustion for IBS, yet the effects seem not to be caused by the treatment per se but are most likely due to a placebo-effect.
The second systematic review and meta-analysis was published by Chinese researchers and aimed at evaluating the clinical efficacy and safety of moxibustion and acupuncture in treatment of IBS. The authors included randomized and quasi-randomized clinical trials in their analyses and were able to include 11 trials. Their meta analysis suggests that the effectiveness of the combined methods of acupuncture and moxibustion is superior to conventional western medication treatment. The authors concluded that acupuncture-moxibustion for IBS is better than the conventional western medication treatment.
While the first meta-analysis was at least technically sound, the second seems to have too many flaws to mention: the search methodology was flimsy, many available studies were not included, their risk of bias was not assessed critically, the conclusions are based more on wishful thinking than on the available data, etc.
If we consider that moxibustion is a method of stimulating acupoints, we have to assume that it can at best be as effective as acupuncture, quite possibly slightly less. Thus it is relevant to see what the evidence tells us about acupuncture for IBS. The current Cochrane review of acupuncture for IBS shows that sham-controlled RCTs have found no benefits of acupuncture relative to a credible sham acupuncture control for IBS symptom severity or IBS-related quality of life.
I think I rest my case.