I have warned you before to be sceptical about Chinese studies. This is what I posted on this blog more than 2 years ago, for instance:
Imagine an area of therapeutics where 100% of all findings of hypothesis-testing research are positive, i.e. come to the conclusion that the treatment in question is effective. Theoretically, this could mean that the therapy is a miracle cure which is useful for every single condition in every single setting. But sadly, there are no miracle cures. Therefore something must be badly and worryingly amiss with the research in an area that generates 100% positive results.
Acupuncture is such an area; we and others have shown that Chinese trials of acupuncture hardly ever produce a negative finding. In other words, one does not need to read the paper, one already knows that it is positive – even more extreme: one does not need to conduct the study, one already knows the result before the research has started. But you might not believe my research nor that of others. We might be chauvinist bastards who want to discredit Chinese science. In this case, you might perhaps believe Chinese researchers.
In this systematic review, all randomized controlled trials (RCTs) of acupuncture published in Chinese journals were identified by a team of Chinese scientists. A total of 840 RCTs were found, including 727 RCTs comparing acupuncture with conventional treatment, 51 RCTs with no treatment controls, and 62 RCTs with sham-acupuncture controls. Among theses 840 RCTs, 838 studies (99.8%) reported positive results from primary outcomes and two trials (0.2%) reported negative results. The percentages of RCTs concealment of the information on withdraws or sample size calculations were 43.7%, 5.9%, 4.9%, 9.9%, and 1.7% respectively.
The authors concluded that publication bias might be major issue in RCTs on acupuncture published in Chinese journals reported, which is related to high risk of bias. We suggest that all trials should be prospectively registered in international trial registry in future.
END OF QUOTE
Now an even more compelling reason emerged for taking evidence from China with a pinch of salt:
A recent survey of clinical trials in China has revealed fraudulent practice on a massive scale. China’s food and drug regulator carried out a one-year review of clinical trials. They concluded that more than 80 percent of clinical data is “fabricated“. The review evaluated data from 1,622 clinical trial programs of new pharmaceutical drugs awaiting regulator approval for mass production. Officials are now warning that further evidence malpractice could still emerge in the scandal.
According to the report, much of the data gathered in clinical trials are incomplete, failed to meet analysis requirements or were untraceable. Some companies were suspected of deliberately hiding or deleting records of adverse effects, and tampering with data that did not meet expectations.
“Clinical data fabrication was an open secret even before the inspection,” the paper quoted an unnamed hospital chief as saying. Contract research organizations seem have become “accomplices in data fabrication due to cutthroat competition and economic motivation.”
A doctor at a top hospital in the northern city of Xian said the problem doesn’t lie with insufficient regulations governing clinical trials data, but with the failure to implement them. “There are national standards for clinical trials in the development of Western pharmaceuticals,” he said. “Clinical trials must be carried out in three phases, and they must be assessed at the very least for safety,” he said. “But I don’t know what happened here.”
Public safety problems in China aren’t limited to the pharmaceutical industry and the figure of 80 percent is unlikely to surprise many in a country where citizens routinely engage in the bulk-buying of overseas-made goods like infant formula powder. Guangdong-based rights activist Mai Ke said there is an all-pervasive culture of fakery across all products made in the country. “It’s not just the medicines,” Mai said. “In China, everything is fake, and if there’s a profit in pharmaceuticals, then someone’s going to fake them too.” He said the problem also extends to traditional Chinese medicines, which are widely used in conjunction with Western pharmaceuticals across the healthcare system.
“It’s just harder to regulate the fakes with traditional medicines than it is with Western pharmaceuticals, which have strict manufacturing guidelines,” he said.
According to Luo, academic ethics is an underdeveloped field in China, leading to an academic culture that is accepting of manipulation of data. “I don’t think that the 80 percent figure is overstated,” Luo said.
And what should we conclude from all this?
I find it very difficult to reach a verdict that does not sound hopelessly chauvinistic but feel that we have little choice but to distrust the evidence that originates from China. At the very minimum, I think, we must scrutinise it thoroughly; whenever it looks too good to be true, we ought to discard it as unreliable and await independent replications.
A new Cochrane review evaluated the effectiveness and safety of Chinese herbal medicines (CHM) in the treatment of menopausal symptoms. Its authors conducted a thorough search for randomised controlled trials (RCTs) comparing the effectiveness of CHM with placebo, hormone therapy (HT), pharmaceutical drugs, acupuncture, or another CHM formula in women suffering from menopausal symptoms.
Two review authors independently assessed 864 studies for eligibility. Data extractions were performed by them with disagreements resolved through group discussion and clarification of data or direct contact with the study authors. Data analyses were performed in accordance with Cochrane Collaboration guidelines.
In total, 22 RCTs (2902 women) could be included. When CHM was compared with placebo (8 RCTs), there was little or no evidence of a difference between the groups for the following outcomes: hot flushes per day (MD 0.00, 95% CI -0.88 to 0.89; 2 trials, 199 women; moderate quality evidence); hot flushes per day assessed by an overall hot flush score in which a difference of one point equates to one mild hot flush per day (MD -0.81 points, 95% CI -2.08 to 0.45; 3 RCTs, 263 women; low quality evidence); and overall vasomotor symptoms per month measured by the Menopause-Specific Quality of Life questionnaire (MENQOL, scale 0 to 6) (MD -0.42 points; 95% CI -1.52 to 0.68; 3 RCTs, 256 women; low quality evidence). In addition, results from individual studies suggested there was no evidence of a difference between the groups for daily hot flushes assessed by severity (MD -0.70 points, 95% CI -1.00, -0.40; 1 RCT, 108 women; moderate quality evidence); or overall monthly hot flushes scores (MD -2.80 points, 95% CI -8.93 to 3.33; 1 RCT, 84 women; very low quality evidence); or overall daily night sweats scores (MD 0.07 points, 95% CI -0.19 to 0.33, 1 RCT, 64 women; low quality evidence); or overall monthly night sweats scores (MD 1.30 points, 95% CI -1.76 to 4.36, 1 RCT, 84 women; very low quality evidence). However, one study reported that overall monthly vasomotor symptom scores were lower in the CHM group (MD -4.79 points, 95% CI -5.52 to -4.06; 1 RCT, 69 women; low quality evidence).
When CHM was compared with HT (10 RCTs), only two RCTs reported monthly vasomotor symptoms using MENQOL. It was uncertain whether CHM reduces vasomotor symptoms (MD 0.47 points, 95% CI -0.50 to 1.44; 2 RCTs, 127 women; very low quality evidence).
Adverse effects were not fully reported in the included studies. Adverse events reported by women taking CHM included mild diarrhoea, breast tenderness, gastric discomfort and an unpleasant taste. Effects were inconclusive because of imprecise estimates of effects: CHM versus placebo (RR 1.51; 95% CI 0.69 to 3.33; 7 trials, 705 women; I² = 40%); CHM versus HT (RR 0.96; 95% CI 0.66 to 1.39; 2 RCTs, 864 women; I² = 0%); and CHM versus specific conventional medications (such as Fluoxetine and Estazolam) (RR 0.20; 95% CI 0.03 to 1.17; 2 RCTs, 139 women; I² = 61%).
The authors concluded: We found insufficient evidence that Chinese herbal medicines were any more or less effective than placebo or HT for the relief of vasomotor symptoms. Effects on safety were inconclusive. The quality of the evidence ranged from very low to moderate; there is a need for well-designed randomised controlled studies.
This review seems well done and reports clear findings. The fact that there was insufficient evidence for CHM is probably no surprise to most readers of this blog. However, I would like to draw your attention to a finding that could easily be missed: most of the primary studies failed to mention adverse effects; to be perfectly clear: they did not state “there were no adverse effects”, but they simply did not mention the subject of adverse effects at all.
In my view, this is a breach of research ethics. I have been banging on about this phenomenon for some time now, because I think it is important. Many if not most clinical trials in this area neglect reporting adverse effects. This means that we get an entirely misleading impression about the safety of the treatments in question. Reviewers of such studies are bound to conclude that they seem to be safe, while, in fact, researchers have only been withholding crucial information from us.
The solution to this fast-growing problem would be simple: trialists must be forced to fully report adverse effects. This is less complicated that it might seem: journal editors must insist that all authors fully report adverse effects of alternative treatments. Even if there were none at all – a very unlikely proposition if you think about it – they must disclose this fact.
No, I don’t want to put you off your breakfast… but you probably have seen so many pictures of attractive athletes with cupping marks and read articles about the virtues of this ancient therapy, that I feel I have to put this into perspective:
I am sure you agree that this is slightly less attractive. But, undeniably, these are also cupping marks. So, if you read somewhere that this treatment is entirely harmless, take it with a pinch of salt.
Cupping has existed for centuries in most cultures, and there are several variations of the theme. We differentiate between wet and dry cupping. The above picture is of wet cupping gone wrong. What the US Olympic athletes currently seem to be so fond of is dry cupping.
The principles of both forms are similar. In dry cupping, a vacuum cup is placed over the skin which provides enough suction to create a circular bruise. Eventually the vacuum diminishes, and the cup falls off; what is left is the mark. In wet cupping, the procedure is much the same, except that the skin is injured before the cup is placed. The suction then pulls out a small amount of blood. Obviously the superficial injury can get infected, and that is what we see on the above picture.
In the homeopathic hospital where I worked ~40 years ago, we did a lot of both types of cupping. We used it mostly for musculoskeletal pain. Our patients responded well.
But why? How does cupping work?
The answer is probably more complex than you expect. It clearly has a significant placebo effect. Athletes are obviously very focussed on their body, and they are therefore the ideal placebo-responders. Evidently, my patients 40 years ago also responded to all types of placebos, even to the homeopathic placebos which they received ‘en masse’.
But there might be other mechanisms as well. A TCM practitioner will probably tell you that cupping unblocks the energy flow in our body. This might sound very attractive to athletes or consumers, and therefore could even enhance the placebo response, but it is nevertheless nonsense.
The most plausible mode of action is ‘counter-irritation’: if you have a pain somewhere, a second pain elsewhere in your body can erase the original pain. You might have a headache, for instance, and if you accidentally hit your thumb with a hammer, the headache is gone, at least for a while. Cupping too would cause mild to moderate pain, and this is a distraction from the muscular pain the athletes aim to alleviate.
When I employed cupping 40 years ago, there was no scientific evidence testing its effects. Since a few years, however, clinical trials have started appearing. Many are from China, and I should mention that TCM studies from China almost never report a negative result. According to the Chinese, TCM (including cupping) works for everything. More recently,also some trials from other parts of the world have emerged. They have in common with the Chinese studies that they tend to report positive findings and that they are of very poor quality. (One such trial has been discussed previously on this blog.) In essence, this means that we should not rely on their conclusions.
A further problem with clinical trails of cupping is that it is difficult, if not impossible, to control for the significant placebo effects that this treatment undoubtedly generates. There is no placebo that could mimic all the features of real cupping in clinical trials; and there is no easy way to blind either the patient or the therapist.
So, we are left with an ancient treatment backed by a host of recent but flimsy studies and a growing craze for cupping fuelled by the Olympic games. What can one conclude in such a situation?
Personally, I would, whenever possible, recommend treatments that work beyond a placebo effect, because the placebo response tends to be unreliable and is usually of short duration – and I am not at all sure that cupping belongs into this category. I would also avoid wet cupping, because it can cause substantial harm. Finally, I would try to keep healthcare costs down; cupping itself is cheap but the therapist’s time might be expensive.
In a nutshell: would I recommend cupping? No, not any more than using a hammer for counter irritation! Will the Olympic athletes care a hoot about my recommendations? No, probably not!
Guest post by Frank Van der Kooy
Something happened in 2008. Something, or a number of things, triggered an exponential rise in the number of rhinos being killed in South Africa. Poaching numbers remained quite low and was stable for a decade with only 13 being killed in 2007. But then suddenly it jumped to 83 in 2008 and it reached a total of 1 175 in 2015. To explain this will be difficult and it will be due to a number of factors or events coinciding in 2008. One possible contributing factor, which I will discuss here, is the growing acceptance of TCM in western countries! For example: Phynova recently advertised a new product as being the first traditional Chinese medicine (TCM) being registered in the UK. By directing customers to a separate site for more information regarding their product they ‘accidently’ linked to a site which ‘advertised’ rhino horn (this link has since been removed). Another example is a University in Australia who published a thesis in 2008, in which they described the current use of Rhino horn as a highly effective medicine, just like you would describe any other real medicine. Surely this will have an impact!
But first a bit of background, so please bear with me. There are two ‘opposing’ aspects regarding TCM that most members of the public do not seem to understand well. Not their fault, because the TCM lobby groups are spending a huge amount of effort to keep the lines between these two aspects as blurred as possible. The first aspect is the underlying pseudoscientific TCM principles; the yin and yang and the vitalistic “energy” flow through “meridians” and much more. Science has relegated this to the pseudosciences, just like bloodletting, which was seen as a cure-all hundreds of years ago. Unfortunately, the pseudoscientific TCM principles are still with us and based on these principles almost every single TCM modality works! From acupuncture to herbs to animal matter (including rhino horn) – everything is efficacious, safe and cost effective. Evidence for this is that close to a 100% of clinical trials done on TCM in China give positive results. Strange isn’t it! People in China should thus no die of any disease – they have ‘effective’ medicine for everything! This is the world of TCM in a nutshell.
The second aspect of TCM is the application of the modern scientific method to test which of the thousands of TCM modalities are really active, which ones are useless and which ones are dangerous. Decades of investigation have come up empty-handed with one or two exceptions. One notable exception is Artemisia annua which contain a single compound that is highly effective for the treatment of malaria, and once identified and intensely studied, it was taken up into conventional medicine – not the herb, but the compound. If you investigate all the plants in the world you are bound to find some compounds that can be used as medicine – it has nothing to do with TCM principles and it can most definitely not be used as evidence that the TCM principles are correct or that it based on science.
These two aspects are therefore quite different.
In the TCM world just about everything works, but it is not backed up by science. It is huge market ($170 billion) and it creates employment for many – something that make politicians smile. In the modern scientific world, almost nothing in TCM works, but it is based on science. It is however not profitable at all – you have to investigate thousands of plants in order to find one useful compound.
Many TCM practitioners and researchers are avidly trying to combine the positives of these two worlds. They focus mainly on the money and employment aspect of the TCM world and try and combine this with the modern scientific approach. They tend to focus on the one example where modern science discovered a useful compound (artemisinin) in the medicinal plant Artemisia annua, which was also coincidently used as an herb in TCM – as evidence that TCM works! Here are some examples:
“To stigmatise all traditional medicine would be unfair. After all, a Chinese medicine practitioner last year won a Nobel prize.” No, a Chinese scientist using the modern scientific method identified artemisinin after testing hundreds or even thousands of different plants.
“This year, Chinese medicine practitioners will be registered in Australia. ….. Chinese herbal medicine is administered routinely in hospitals for many chronic diseases. …… This has led to recognising herbs such as Artemisia as a proven anti-malarial ……” No, the compound artemisinin is a proven anti-malarial!
“There has been enormous progress in the last 20 years or so. I am sure you are familiar with the use of one of the Chinese herbs in managing resistant malaria.” No, very little progress and no, the compound artemisinin!
So this is a game that is being played with the simple intention to blur the lines between these two aspects regarding TCM – but the real reason might simply be “A new research-led Chinese medicine clinic in Sydney, better patient outcomes and the potential for Australia to tap into the $170 billion global traditional Chinese medicine market”
Prof Alan Bensoussan the director of the National Institute of Complementary Medicine (NICM) and registered in Australia as a TCM and acupuncture practitioner is a champion in blurring this line. Alan has been instrumental in lobbying the Australian regulatory agency that a long tradition of use is all you need to be able to register new products. He was also influential in establishing the Chinese medicine practitioner registry in Australia, in 2012, and thereby legitimising TCM in Australia. He has been actively chipping away at the resistance that the Australian public have against these pseudoscientific healthcare systems such as TCM – one can argue that he has done so quite successfully because they are expanding their operations into the Westmead precinct of Sydney with a new TCM clinic/hospital.
Enough background; so what does all of this have to do with Rhino horn? (and for that matter other endangered species). We have to remember that in the TCM world just about everything works and that includes rhino horn! Searching Western Sydney University’s theses portal for Xijiao (Chinese for Rhino horn) I found a thesis published in 2008 from the NICM and co-supervised by Alan; “Development of an evidence-based Chinese herbal medicine for the management of vascular dementia”
On page 45-46: “Recently, with fast developing science and technologies being applied in the pharmaceutical manufacturing area, more and more herbs or herbal mixtures have been extracted or made into medicinal injections. These have not only largely facilitated improved application to patients, but also increased the therapeutic effectiveness and accordingly reduced the therapeutic courses …… lists the most common Chinese herbal medicine injections used for the treatment of VaD. “
“Xing Nao Jing Injection (for clearing heat toxin and opening brain, removing phlegm) contains ….. Rhinoceros unicornis (Xijiao), …… Moschus berezovskii (Shexiang), …..”
“…. Xing Nao Jing injection has been widely applied in China for stroke and vascular dementia. …. After 1-month treatment intervention, they found the scores in the treatment group increased remarkably, as compared with the control group …… “
They list two endangered species; the Rhino and the Chinese forest musk deer (Moschus berezovskii). But what is truly worrying is that they don’t even mention the endangered status or at least recommend that the non-endangered substitutes, which do exist in the TCM world, should be used instead – or maybe use fingernails as a substitute? It is not discussed at all. Clearly they are stating that using these endangered animals are way more effective than western medicine (the control group) for the treatment of vascular dementia! This is deplorable to say the least. Statements like this fuels the decimation of this species. But this shows that they truly believe and support the underlying pseudoscientific principles of TCM – they have to, their ability to tap into the TCM market depends on it!
As a scientist you are entitled to discuss historic healthcare treatments such as bloodletting. But make sure to also state that this practice has been shown to be ineffective, and quite dangerous, and that modern science has since come up with many other effective treatments. If it is stated that bloodletting is currently being used and it is effective – then you will simply be promoting bloodletting! The same goes for Rhino horn and this is exactly what they have done here. But then again they live in a world where all TCM modalities are active!
How to solve this problem of growing acceptance of TCM in western countries? A simple step could be that people like Alan publicly denounce the underlying pseudoscientific TCM principles and make the ‘difficult’ switch to real science! Admittedly, he will have to part with lots of money from the CM industry and his Chinese partners, and maybe not built his new TCM hospital! But for some reason I strongly doubt that this will happen. The NICM have successfully applied a very thin, but beautiful, veneer of political correctness and modernity over the surface of complementary medicine. Anyone who cares to look underneath this veneer will find a rotten ancient pseudoscientific TCM world – in this case the promotion and the use of endangered animals.
After reading chapter two of this thesis one cannot believe that this is from an Australian University and paid for by the Australian taxpayer! The main question though: Can I directly link this thesis with the increase in rhino poaching? This will be very difficult if not impossible to do. But that is not the problem. Promoting the pseudoscientific principles of TCM in Australia expands the export market for TCM, and hence will lead to an increased need for raw materials, including the banned Rhino horn. That Rhino horn has been a banned substance since the 1980’s clearly does not seem to have any impact looking at the poaching statistics. In an unrelated paper published in 2010 the ingredients in the Xingnaojing injection is listed as “…. consisting of Chinese herbs such as Moschus, Borneol, Radix Curcumae, Fructus Gardeniae, ….” No full list is given in the paper – dare I say because it contains Rhino horn as well? The drug Ice is also banned, but if you are going to promote it at a ‘trusted’ university, then you shouldn’t be surprised that Ice production increases and more of it flows into Australia – even if it is illegal. The same goes for Rhino horn!
On this blog, I have discussed the lamentable quality of TCM products before (e. g. here, here and here). In a nutshell, far too high percentages of them are contaminated with toxic substances or adulterated with prescription drugs. It is no question: these deficits put many consumers at risk. Equally, there is no question that the problem has been known for decades.
For the Chinese exporters, such issues are a great embarrassment, not least because TCM-products are amongst the most profitable of all the Chinese exports. In the past, Chinese officials have tried to ignore or suppress the subject as much as possible. I presume they fear that their profits might be endangered by being open about the dubious quality of their TCM-exports.
Recently, however, I came across a website where unusually frank and honest statements of Chinese officials appeared about TCM-products. Here is the quote:
China is to unroll the fourth national survey of traditional Chinese medicine (TCM) resources to ensure a better development of the industry, said a senior health official…
With the public need for TCM therapies growing, the number of medicine resources has decreased and people have turned to the cultivated ones. However, due to a lack of standards, the cultivated TCM resources are sometimes less effective or even unsafe for human use, said Wang Guoqiang, director of the State Administration of TCM, at a TCM seminar held in Kunming, Yunnan Province in southwest China.
There is a pressing need to protect TCM resources, Wang said. “I’ve heard people saying that medicine quality will spell doom for the TCM industry, which I must admit, is no exaggeration,” he said.
The survey has been piloted in 922 counties in 31 provinces in China since 2011. According to its official website, it will draw a clear picture of the variety, distribution, storage and growth trends of TCM resources, including herbs, animals, minerals and synthetic materials.
TCM includes a range of traditional medical practices originating in China. It includes such treatments as herbal medicine, acupuncture, massage (tuina), exercise (qigong) and dietary therapy.
Although well accepted in the mainstream of medical care throughout East Asia, TCM is considered an alternative medical system in much of the western world and has been a source of controversy. A milestone in the recognition of TCM came when Chinese pharmaceutical chemist Tu Youyou won a Nobel Prize in 2015 for her discovery of Artemisinin, a medicinal herb, to help treat malaria.
END OF QUOTE
Surely, these are remarkable, perhaps even unprecedented statements by Chinese officials:
…cultivated TCM resources are sometimes less effective or even unsafe for human use…
…medicine quality will spell doom for the TCM industry…
Let’s hope that, after such words, there will be appropriate actions… finally.
Don’t get me wrong, I have nothing against systematic reviews. Quite to the contrary, I am sure they are an important source of information for patients, doctors, scientists, policy makers and others – after all, I have published more than 300 of such papers!
Having said that, I do dislike a certain type of systematic review, namely systematic reviews by Chinese authors evaluating TCM therapies and arriving at misleading conclusions. Such papers are currently swamping the marked.
At first glance, they look fine. On closer scrutiny, however, most turn out to be stereotypically useless, boring and promotional. The type of article I mean starts by stating its objective which usually is to evaluate the evidence for a traditional Chinese therapy as a treatment of a condition which few people in their right mind would treat with any form of TCM. It continues with details about the methodologies employed and then, in the results section, informs the reader that x studies were included in the review which mostly reported encouraging results but were wide open to bias. And then comes the crucial bit: THE CONCLUSIONS.
They are as predictable as they are misleading. let me give you two examples only published in the last few days.
The first review drew the following conclusions: This systematic review suggests that Chinese Herbal Medicine as an adjunctive therapy can improve cognitive impairment and enhance immediate response and quality of life in Senile Vascular Dementia patients. However, because of limitations of methodological quality in the included studies, further research of rigorous design is needed.
The second review concluded that the evidence that external application of traditional Chinese medicine is an effective treatment for venous ulcers is encouraging, but not conclusive due to the low methodological quality of the RCTs. Therefore, more high-quality RCTs with larger sample sizes are required.
Why does that sort of thing frustrate me so much? Because it is utterly meaningless and potentially harmful:
- I don’t know what treatments the authors are talking about.
- Even if I managed to dig deeper, I cannot get the information because practically all the primary studies are published in obscure journals in Chinese language.
- Even if I did read Chinese, I do not feel motivated to assess the primary studies because we know they are all of very poor quality – too flimsy to bother.
- Even if they were formally of good quality, I would have my doubts about their reliability; remember: 100% of these trials report positive findings!
- Most crucially, I am frustrated because conclusions of this nature are deeply misleading and potentially harmful. They give the impression that there might be ‘something in it’, and that it (whatever ‘it’ might be) could be well worth trying. This may give false hope to patients and can send the rest of us on a wild goose chase.
So, to ease the task of future authors of such papers, I decided give them a text for a proper EVIDENCE-BASED conclusion which they can adapt to fit every review. This will save them time and, more importantly perhaps, it will save everyone who might be tempted to read such futile articles the effort to study them in detail. Here is my suggestion for a conclusion soundly based on the evidence, not matter what TCM subject the review is about:
OUR SYSTEMATIC REVIEW HAS SHOWN THAT THERAPY ‘X’ AS A TREATMENT OF CONDITION ‘Y’ IS CURRENTLY NOT SUPPORTED BY SOUND EVIDENCE.
On this blog, we have repeatedly discussed the risks of acupuncture. Contrary to what we often hear, there clearly is potential for harm. Acupuncture is, of course most popular in China where it has been used for thousands of years. Therefore the Chinese literature, which is not easy to access for non-Chinese speakers and therefore often disregarded by Western researchers, might hold treasures of valuable information on the subject. It follows that a thorough review of this information might be helpful. A recent paper by Chinese scientists has tackled this issue.
The objective of this review was to determine the frequency and severity of adverse complications and events in acupuncture treatment reported from 1980 to 2013 in China. All first-hand case reports of acupuncture-related complications and adverse events that could be identified in the scientific literature were reviewed and classified according to the type of complication and adverse event, circumstance of the event, and long-term patient outcome. The selected case reports were published between 1980 and 2013 in 3 databases. Relevant papers were collected and analyzed by 2 reviewers.
Over the 33 years, 182 incidents were identified in 133 relevant papers. Internal organ, tissue, or nerve injury is the main complications of acupuncture especially for pneumothorax and central nervous system injury. Adverse effects also included syncope, infections, hemorrhage, allergy, burn, aphonia, hysteria, cough, thirst, fever, somnolence, and broken needles.
The authors of this review concluded that qualifying training of acupuncturists should be systemized and the clinical acupuncture operations should be standardized in order to effectively prevent the occurrence of acupuncture accidents, enhance the influence of acupuncture, and further popularize acupuncture to the rest of the world.
This is a bizarrely disappointing article followed by a most strange conclusion. The authors totally fail to discuss the most important issue and they arrive at conclusions which, I think, make little sense.
The issue to discuss here is, of course, under-reporting. We know that under-reporting in the Western literature is already huge. For every complication reported there could easily be 10 or even 100 that go unreported. There is no monitoring system for adverse effects, and acupuncturists have no incentive to publish their mistakes. Accurate and realistic prevalence data are therefore anybody’s guess.
In China, under-reporting is likely to be one or two orders of magnitude bigger. I say this because close to zero % of all Chinese papers on acupuncture report negative findings. For China, TCM is a huge export business, and the interest in reporting adverse effects is close to zero.
Seen from this perspective, it seems at first glance laudable that the Chinese authors dared to address this thorny issue. In the text of the article, they even mention that the included complications resulted in a total of 25 fatalities! This seems courageous. But one only needs to read the full article to get a strong suspicion that the authors are either in denial about the real figures, or their paper is a deliberate attempt to ‘white-wash’ acupuncture from its potential to do harm.
In 2010, we published a very similar review of the Chinese literature (unsurprisingly, it was not cited by the authors of the new paper). At the time, we found almost 500 published cases of serious adverse events and stated that we suspect that underreporting of such events in the Chinese-language literature is much higher than in the English-language literature.
The truth is that nobody knows how frequent adverse events of acupuncture truly are in China – or most other countries, for that matter. I believe that, before we “further popularize acupuncture to the rest of the world”, it would be ethical and necessary to 1) state this fact openly and 2) do something about it.
One could define alternative medicine by the fact that it is used almost exclusively for conditions for which conventional medicine does not have an effective and reasonably safe cure. Once such a treatment has been found, few patients would look for an alternative.
Alzheimer’s disease (AD) is certainly one such condition. Despite intensive research, we are still far from being able to cure it. It is thus not really surprising that AD patients and their carers are bombarded with the promotion of all sorts of alternative treatments. They must feel bewildered by the choice and all too often they fall victim to irresponsible quacks.
Acupuncture is certainly an alternative therapy that is frequently claimed to help AD patients. One of the first websites that I came across, for instance, stated boldly: acupuncture improves memory and prevents degradation of brain tissue.
But is there good evidence to support such claims? To answer this question, we need a systematic review of the trial data. Fortunately, such a paper has just been published.
The objective of this review was to assess the effectiveness and safety of acupuncture for treating AD. Eight electronic databases were searched from their inception to June 2014. Randomized clinical trials (RCTs) with AD treated by acupuncture or by acupuncture combined with drugs were included. Two authors extracted data independently.
Ten RCTs with a total of 585 participants were included in a meta-analysis. The combined results of 6 trials showed that acupuncture was better than drugs at improving scores on the Mini Mental State Examination (MMSE) scale. Evidence from the pooled results of 3 trials showed that acupuncture plus donepezil was more effective than donepezil alone at improving the MMSE scale score. Only 2 trials reported the incidence of adverse reactions related to acupuncture. Seven patients had adverse reactions related to acupuncture during or after treatment; the reactions were described as tolerable and not severe.
The Chinese authors of this review concluded that acupuncture may be more effective than drugs and may enhance the effect of drugs for treating AD in terms of improving cognitive function. Acupuncture may also be more effective than drugs at improving AD patients’ ability to carry out their daily lives. Moreover, acupuncture is safe for treating people with AD.
Anyone reading this and having a friend or family member who is affected by AD will think that acupuncture is the solution and warmly recommend trying this highly promising option. I would, however, caution to remain realistic. Like so very many systematic reviews of acupuncture or other forms of TCM that are currently flooding the medical literature, this assessment of the evidence has to be taken with more than just a pinch of salt:
- As far as I can see, there is no biological plausibility or mechanism for the assumption that acupuncture can do anything for AD patients.
- The abstract fails to mention that the trials were of poor methodological quality and that such studies tend to generate false-positive findings.
- The trials had small sample sizes.
- They were mostly not blinded.
- They were mostly conducted in China, and we know that almost 100% of all acupuncture studies from that country draw positive conclusions.
- Only two trials reported about adverse effects which is, in my view, a sign of violation of research ethics.
As I already mentioned, we are currently being flooded with such dangerously misleading reviews of Chinese primary studies which are of such dubious quality that one could do probably nothing better than to ignore them completely.
Isn’t that a bit harsh? Perhaps, but I am seriously worried that such papers cause real harm:
- They might motivate some to try acupuncture and give up conventional treatments which can be helpful symptomatically.
- They might prompt some families to spend sizable amounts of money for no real benefit.
- They might initiate further research into this area, thus drawing money away from research into much more promising avenues.
IT IS HIGH TIME THAT RESEARCHERS START THINKING CRITICALLY, PEER-REVIEWERS DO THEIR JOB PROPERLY, AND JOURNAL EDITORS STOP PUBLISHING SUCH MISLEADING ARTICLES.
Moxibustion is an ancient variation of acupuncture using moxa made from dried mugwort (Artemisia argyi). It has long played an important role in the traditional heath care systems of China and other Asian countries. More recently, it has become popular also in the West. Practitioners use moxa sticks indirectly to warm acupuncture needles, or burn it close to the patient’s skin. Essentially, moxibustion is a treatment where acupuncture points are stimulated mainly or exclusively by the heat of burning moxa.
Because of moxibustion’s long history of usage and the fact that it is employed in many countries for a very wide range of conditions, some might argue that it has stood the ‘test of time’ and should be considered to be a well-established therapy. More critical thinkers would, however, point out that this is not an argument but a classical fallacy.
My team at Exeter regularly had research fellows from Korea and other Asian countries, and we managed to develop a truly productive cooperation. It enabled us to conduct systematic reviews including the Asian literature – and this is how we got involved in an unusual amount of research into moxibustion which, after all, is a fairly exotic alternative therapy. In 2010, we began a series of systematic reviews of moxibustion.
One of the first such articles included 9 RCTs testing the effectiveness of this treatment for stroke rehabilitation. Three RCTs reported favorable effects of moxibustion plus standard care on motor function versus standard care alone Three randomized clinical trials compared the effects of moxibustion on activities of daily living alone but failed to show favorable effects of moxibustion.
Also in 2010, our systematic review of RCTs of moxibustion as a treatment of ulcerative colitis (UC) concluded that current evidence is insufficient to show that moxibustion is an effective treatment of UC. Most of included trials had high risk of bias. More rigorous studies seem warranted.
Our (2010) systematic review od RCTs of moxibustion as a therapy in cancer care found that the evidence was limited to suggest moxibustion is an effective supportive cancer care in nausea and vomiting. However, all studies had a high risk of bias so effectively there was not enough evidence to draw any conclusion.
Our (2010) systematic review of RCTs of moxibustion for treating hypertension concluded that there was insufficient evidence to suggest that moxibustion is an effective treatment for hypertension.
Our (2010) systematic review of RCTs of moxibustion for constipation concluded as follows: Given that the methodological quality of all RCTs was poor, the results from the present review are insufficient to suggest that moxibustion is an effective treatment for constipation. More rigorous studies are warranted.
Our (2010) systematic review found few RCTs were available that test the effectiveness of moxibustion in the management of pain, and most of the existing trials had a high risk of bias. Therefore, more rigorous studies are required before the effectiveness of moxibustion for the treatment of pain can be determined.
Our (2011) systematic review of 14 RCTs of moxibustion for rheumatic conditions failed to provide conclusive evidence for the effectiveness of moxibustion compared with drug therapy in rheumatic conditions.
The, so far, last article in this series has only just been published. The purpose of this systematic review was to assess the efficacy of moxibustion as a treatment of chemotherapy-induced leukopenia. Twelve databases were searched from their inception through June 2014, without a language restriction. Randomized clinical trials (RCTs) were included, if moxibustion was used as the sole treatment or as a part of a combination therapy with conventional drugs for leukopenia induced by chemotherapy. Cochrane criteria were used to assess the risk of bias.
Six RCTs with a total of 681 patients met our inclusion criteria. All of the included RCTs were associated with a high risk of bias. The trials included patients with various types of cancer receiving ongoing chemotherapy or after chemotherapy. The results of two RCTs suggested the effectiveness of moxibustion combined with chemotherapy vs. chemotherapy alone. In four RCTs, moxibustion was more effective than conventional drug therapy. Six RCTs showed that moxibustion was more effective than various types of control interventions in increasing white blood cell counts.
Our conclusion: there is low level of evidence based on these six trials that demonstrates the superiority of moxibustion over drug therapies in the treatment of chemotherapy-induced leukopenia. However, the number of trials, the total sample size, and the methodological quality are too low to draw firm conclusions. Future RCTs appear to be warranted.
Was all this research for nothing?
I know many people who would think so. However, I disagree. If nothing else, these articles demonstrated several facts quite clearly:
- There is quite a bit of research even on the most exotic alternative therapy; sometimes one needs to look hard and include languages other than English.
- Studies from China and other Asian counties very rarely report negative results; this fact casts a dark shadow on the credibility of such data.
- The poor quality of trials in most areas of alternative medicine is lamentable and must be stimulus for researchers in this field to improve their act.
- Authors of systematic reviews must resist the temptation to draw positive conclusions based on flawed primary data.
- Moxibustion is a perfect example for demonstrating that the ‘test of time’ is no substitute for evidence.
- As for moxibustion, it cannot currently be considered an evidence-based treatment for any condition.
Acupuncture seems to be as popular as never before – many conventional pain clinics now employ acupuncturists, for instance. It is probably true to say that acupuncture is one of the best-known types of all alternative therapies. Yet, experts are still divided in their views about this treatment – some proclaim that acupuncture is the best thing since sliced bread, while others insist that it is no more than a theatrical placebo. Consumers, I imagine, are often left helpless in the middle of these debates. Here are 7 important bits of factual information that might help you make up your mind, in case you are tempted to try acupuncture.
- Acupuncture is ancient; some enthusiast thus claim that it has ‘stood the test of time’, i. e. that its long history proves its efficacy and safety beyond reasonable doubt and certainly more conclusively than any scientific test. Whenever you hear such arguments, remind yourself that the ‘argumentum ad traditionem’ is nothing but a classic fallacy. A long history of usage proves very little – think of how long blood letting was used, even though it killed millions.
- We often think of acupuncture as being one single treatment, but there are many different forms of this therapy. According to believers in acupuncture, acupuncture points can be stimulated not just by inserting needles (the most common way) but also with heat, electrical currents, ultrasound, pressure, etc. Then there is body acupuncture, ear acupuncture and even tongue acupuncture. Finally, some clinicians employ the traditional Chinese approach based on the assumption that two life forces are out of balance and need to be re-balanced, while so-called ‘Western’ acupuncturists adhere to the concepts of conventional medicine and claim that acupuncture works via scientifically explainable mechanisms that are unrelated to ancient Chinese philosophies.
- Traditional Chinese acupuncturists have not normally studied medicine and base their practice on the Taoist philosophy of the balance between yin and yang which has no basis in science. This explains why acupuncture is seen by traditional acupuncturists as a ‘cure all’ . In contrast, medical acupuncturists tend to cite neurophysiological explanations as to how acupuncture might work. However, it is important to note that, even though they may appear plausible, these explanations are currently just theories and constitute no proof for the validity of acupuncture as a medical intervention.
- The therapeutic claims made for acupuncture are legion. According to the traditional view, acupuncture is useful for virtually every condition affecting mankind; according to the more modern view, it is effective for a relatively small range of conditions only. On closer examination, the vast majority of these claims can be disclosed to be based on either no or very flimsy evidence. Once we examine the data from reliable clinical trials (today several thousand studies of acupuncture are available – see below), we realise that acupuncture is associated with a powerful placebo effect, and that it works better than a placebo only for very few (some say for no) conditions.
- The interpretation of the trial evidence is far from straight forward: most of the clinical trials of acupuncture originate from China, and several investigations have shown that very close to 100% of them are positive. This means that the results of these studies have to be taken with more than a small pinch of salt. In order to control for patient-expectations, clinical trials can be done with sham needles which do not penetrate the skin but collapse like miniature stage-daggers. This method does, however, not control for acupuncturists’ expectations; blinding of the therapists is difficult and therefore truly double (patient and therapist)-blind trials of acupuncture do hardly exist. This means that even the most rigorous studies of acupuncture are usually burdened with residual bias.
- Few acupuncturists warn their patients of possible adverse effects; this may be because the side-effects of acupuncture (they occur in about 10% of all patients) are mostly mild. However, it is important to know that very serious complications of acupuncture are on record as well: acupuncture needles can injure vital organs like the lungs or the heart, and they can introduce infections into the body, e. g. hepatitis. About 100 fatalities after acupuncture have been reported in the medical literature – a figure which, due to lack of a monitoring system, may disclose just the tip of an iceberg.
- Given that, for the vast majority of conditions, there is no good evidence that acupuncture works beyond a placebo response, and that acupuncture is associated with finite risks, it seems to follow that, in most situations, the risk/benefit balance for acupuncture fails to be convincingly positive.