I have repeatedly cautioned about the often poor quality of research into alternative medicine. This seems particularly necessary with studies of acupuncture, and especially true for such research carried out in China. I have also frequently noted that certain ‘CAM journals’ are notoriously prone to publishing rubbish. So, what can we expect from a paper that:
- is on alternative medicine,
- focusses on acupuncture,
- is authored by Chinese researchers,
- was published in the Journal of Alternative and Complementary Medicine (JACM)?
The answer is PROBABLY NOT A LOT!
As if for confirming my prediction, The JACM just published this systematic review. It reports pairwise and network meta-analyses to determine the effectiveness of acupuncture and acupuncture-related techniques for the treatment of psoriasis. A total of 13 RCTs were included. The methodological quality of these studies was ‘not rigorous’ according to the authors – in fact, it was lousy. Acupoint stimulation seemed to be more effective than non-acupoint stimulation. The short-term treatment effect was superior to the long-term effect (as one would expect with placebo). Network meta-analysis suggested that acupressure or acupoint catgut embedding generate superior effects compared to medications. It was noted that acupressure was the most effective treatment of all the acupuncture-like therapies.
The authors concluded that acupuncture-related techniques could be considered as an alternative or adjuvant therapy for psoriasis in short term, especially of acupressure and acupoint catgut embedding. This study recommends further well-designed, methodologically rigorous, and more head-to-head randomized trials to explore the effects of acupuncture-related techniques for treating psoriasis.
And what is wrong with that?
- The review is of very poor quality.
- The primary studies are even worse.
- The English language is defective to the point of being not understandable.
- The conclusions are misleading.
Correct conclusions should read something like this: Due to the paucity and the poor quality of the clinical trials, this review could not determine whether acupuncture and similar therapies are effective for psoriasis.
And then there is, of course, the question about plausibility. How plausible is the assumption that acupuncture might affect a genetic autoimmune disease like psoriasis. The answer, I think, is that the assumption is highly unlikely.
In the above review, most of the 13 primary RCTs were from China. One of the few studies not conducted in China is this one:
56 patients suffering from long-standing plaque psoriasis were randomized to receive either active treatment (electrostimulation by needles placed intramuscularly, plus ear-acupuncture) or placebo (sham, ‘minimal acupuncture‘) twice weekly for 10 weeks. The severity of the skin lesions was scored (PASI) before, during, and 3 months after therapy. After 10 weeks of treatment the PASI mean value had decreased from 9.6 to 8.3 in the ‘active’ group and from 9.2 to 6.9 in the placebo group (p < 0.05 for both groups). These effects are less than the usual placebo effect of about 30%. There were no statistically significant differences between the outcomes in the two groups during or 3 months after therapy. The patient’s own opinion about the results showed no preference for ‘active’ therapy. It was also clear from the answers that the blinded nature of the study had not been discovered by the patients. In conclusion, classical acupuncture is not superior to sham (placebo) ‘minimal acupuncture‘ in the treatment of psoriasis.
Somehow, I trust these conclusions more than the ones from the review!
And somehow, I get very tired of journal editors failing to do their job of rejecting papers that evidently are embarrassing, unethical rubbish.
The ‘Daily Mail’ is not a paper famed for its objective reporting. In politics, this can influence elections; in medicine, it can endanger public health.
A recent article is a case in point, I think.
START OF QUOTE
Traditional Chinese medicines could help prevent heart disease and the progression of pre-diabetes, according to research. Some herbal treatments proved as effective in lowering blood pressure as Western drugs and improved heart health by lowering cholesterol, scientists found. Certain alternative medicines could lower blood sugar and insulin levels, too.
Chinese medicines could be used alongside conventional treatments, say researchers from Shandong University Qilu Hospital in China. Or they can be beneficial as an alternative for patients intolerant of Western drugs, they said in their review of medical studies over a ten-year period. Senior review author from the university’s department of traditional Chinese medicine said: ‘The pharmacological effects and the underlying mechanisms of some active ingredients of traditional Chinese medications have been elucidated. Thus, some medications might be used as a complementary and alternative approach for primary and secondary prevention of cardiovascular disease.”
It’s potentially good news for people living with diabetes, which is now a global epidemic and has proved a tricky condition to manage for many people. High blood pressure is very common too, affecting more than one in four adults in the UK, although many won’t show symptoms and realise it. If untreated, it increases your risk of serious problems including heart disease, the number one killer globally.
The Chinese have used herbs for treating diseases for thousands of years and have become increasingly popular in Europe and North America, mainly as complement to Western medicine. But the researchers also warn that much of the research conducted have limitations and so their long-term effects are not proven.
Herbs for high blood pressure
The blood pressure-lowering effect of herb zhongfujiangya was found to be similar to that of oral anti-hypertension medication benazeprilm, which goes by the brand name Lotensin. Similarly, patients treated for eight weeks with herbal tiankuijiangya had a lower reading than those given a placebo. Herbal Jiangya tablets were found to ‘significantly lower’ systolic blood pressure, that is the amount of pressure in your arteries during contraction of your heart muscle compared to a fake treatment. The herb Jiangyabao also had a significant effect compared to a placebo, but just at night. But overall, compared to the drug Nimodipine, a calcium channel blocker, it worked just as well. Qiqilian capsules also proved more effective compared to a placebo.
Herbs for diabetes
The team report some Chinese medicines medications – such as xiaoke, tangminling, jinlida, and jianyutangkang – have a ‘potent’ effect on lowering blood sugar levels and b-cell function, which controls the release of insulin. Some remedies – such as tangzhiping and tianqi – might prevent the progression of pre-diabetes to diabetes, they note.
Herbs for cholesterol
The researchers looked at research on dyslipidemia, the term for unbalanced or unhealthy cholesterol levels. They found that jiangzhitongluo, salviamiltiorrhiza and pueraria lobata, and zhibitai capsule all have a ‘potent lipid-lowing effect’.
Herbs for heart disease
Some traditional Chinese medicines such as qiliqiangxin, nuanxin, shencaotongmai, and yangxinkang, might be effective in improving function in patients with chronic heart failure, they wrote.
Limitations with trials
But Western scientists often reject Chinese medicine for specific reasons, warned Dr Zhao’s team. Chinese medicines are frowned upon because they do not go through the same exhaustive approval process as trials conducted domestically, they pointed out. Plus, one treatment can be made of many different ingredients with various chemical compounds, making it hard to pinpoint how their benefits work. ‘One should bear in mind that traditional Chinese medicine medications are usually prescribed as complex formulae, which are often further manipulated by the practitioner on a personalized basis,’ said Dr Zhao.
END OF QUOTE
Apart from the fact that this article is badly written, it is also misleading to the point of being outright dangerous. Regular readers of my blog will be aware that Chinese research is everything but reliable; there are practically no Chinese TCM-trials that report negative results. Furthermore, the safety of Chinese herbal preparations is as good as unknown and they are often contaminated with toxic substances as well as adulterated with synthetic drugs. Most of these preparations are also unavailable outside China. Moreover, Chinese herbal treatments are usually individualised (mixtures are tailor-made for each individual patient), and there is no good evidence that this approach is effective. Crucially, the trial evidence is often of such poor quality that it would be a dangerous mistake to trust these findings.
None of these important caveats, it seems, are important enough to get a mention in the Daily Mail.
Don’t let the truth get in the way of a sensational story!
Let’s just for a moment imagine what would happen if people took the Mail article seriously (is there anyone out there who does take the Mail seriously?). In a best case scenario, they would take Chinese herbs in addition to their prescribed medication. This might case plenty of unwanted side-effects and herb-drug interactions. In addition, people would lose a lot of their hard-earned cash. In a worst case scenario, they would abandon their prescribed medication for dubious Chinese herbal mixtures. This could cause thousands of premature deaths.
With just a little research, I managed to find the original article on which the Mail’s report was based. Here is its abstract:
Traditional Chinese medicine (TCM) has more than 2,000 years of history and has gained widespread clinical applications. However, the explicit role of TCM in preventing and treating cardiovascular disease remains unclear due to a lack of sound scientific evidence. Currently available randomized controlled trials on TCM are flawed, with small sample sizes and diverse outcomes, making it difficult to draw definite conclusions about the actual benefits and harms of TCM. Here, we systematically assessed the efficacy and safety of TCM for cardiovascular disease, as well as the pharmacological effects of active TCM ingredients on the cardiovascular system and potential mechanisms. Results indicate that TCM might be used as a complementary and alternative approach to the primary and secondary prevention of cardiovascular disease. However, further rigorously designed randomized controlled trials are warranted to assess the effect of TCM on long-term hard endpoints in patients with cardiovascular disease.
In my view, the authors of this review are grossly over-optimistic in their conclusions (but nowhere near as bad as the Mail journalist). If the trials are of poor quality, as the review-authors admit, no firm conclusions should be permissible about the usefulness of the therapies in question.
As the Mail article is obviously based on a press release (several other papers worldwide reported about the review as well), it seems interesting to note what the editor of the Journal of the American College of Cardiology (the journal that published the review) recently had to say about the responsibility of journalists and researchers:
START OF QUOTE
…I would like to suggest that journalists and researchers must share equally in shouldering the burden of responsibility to improve appropriate communication about basic and clinical research.
First, there is an obligation on the part of the researchers not to inflate the importance of their findings. This has been widely recognized as damaging, especially if bias is introduced in the paper…
Second, researchers should take some responsibility for the creation of the press release about their research, which is written by the media or press relations department at their hospital or society. Press releases are often how members of the media get introduced to a particular study, and these releases can often introduce errors or exaggerations. In fact, British researchers evaluated 462 press releases on biomedical and health-related science issued by 20 leading U.K. universities in 2011, alongside their associated peer-reviewed research papers and the news stories that followed (n = 668). They found that 40% of the press releases contained exaggerated advice, 33% contained exaggerated causal claims, and 36% contained exaggerated inference to humans from animal research. When press releases contained such exaggeration, 58%, 81%, and 86% of news stories, respectively, contained further exaggeration, compared with rates of 17%, 18%, and 10% in the news when the press releases were not exaggerated. Researchers should not be excused from being part of the press release process, as the author(s) should at least review the release before it gets disseminated to the media. I would even encourage researchers to engage in the process at the writing stage and to not allow their hospital’s or society’s public relations department to extrapolate their study’s results. Ultimately, the authors and the journals in which the studies are published will be held accountable for the information that trickles into the headlines, not the public relations departments, so we must make sure that the information is accurate and representative of the study’s actual findings.
END OF QUOTE
Sound advice indeed.
Now we only need to ALL follow it!!!
Tui Na is a massage technique that is based on the Taoist principles of TCM. It involves a range of manipulations usually performed by an operator’s finger, hand, elbow, knee, or foot applied to muscle or soft tissue at specific parts of the body. According to one website of TCM-proponents “Tui Na makes use of various hand techniques in combination with acupuncture and other manipulation techniques. To enhance the healing process, the practitioner may recommend the use of Chinese herbs. Many of the techniques used in this massage resemble that of a western massage like gliding, kneading, vibration, tapping, friction, pulling, rolling, pressing and shaking. In Tui Na massage, the muscles and tendons are massaged with the help of hands, and an acupressure technique is applied to directly affect the flow of Qi at different acupressure points of the body, thus facilitating the healing process. It removes the blockages and keeps the energy moving through the meridians as well as the muscles. A typical session of Tui Na massage may vary from thirty minutes to an hour. The session timings may vary depending on the patient’s needs and condition. The best part of the therapy is that it relaxes as well as energizes the person. The main benefit of Tui Na massage is that it focuses on the specific problem, whether it is an acute or a chronic pain associated with the joints, muscles or a skeletal system. This technique is very beneficial in reducing the pain of neck, shoulders, hips, back, arms, highs, legs and ankle disorders. It is a very effective therapy for arthritis, pain, sciatica and muscle spasms. Other benefits of this massage therapy include alleviation of the stress related disorders like insomnia, constipation, headaches and other disorders related to digestive, respiratory and reproductive systems. The greatest advantage of Tui Na is that it focuses on maintaining overall balance with both physical and mental health. Any one who wants to avoid the side effects of drugs or a chemical based treatment can adopt this effective massage technique to alleviate their pain. Tui Na massage therapy is now becoming a more common therapy method due to its focus on specific problems rather than providing a general treatment.”
This clearly begs the question IS IT EFFECTIVE?
This systematic review assessed the evidence of Tui Na for cervical radiculopathy. Seven databases were searched. Randomised controlled trials (RCTs) incorporating Tui Na alone or Tui Na combined with conventional treatment were included. Five studies involving 448 patients were found. The pooled analysis from the 3 trials indicated that Tui Na alone showed a significant lowering immediate effects on pain score with moderate heterogeneity compared to cervical traction. The meta-analysis from 2 trials revealed significant immediate effects of Tui Na plus cervical traction in improving pain score with no heterogeneity compared to cervical traction alone. None of the RCTs mentioned adverse effects. There was very low quality or low quality evidence to support the results.
The authors concluded that “Tui Na alone or Tui Na plus cervical traction may be helpful to cervical radiculopathy patients, but supportive evidence seems generally weak. Future clinical studies with low risk of bias and adequate follow-up design are recommended.”
In my view, this is a misleading conclusion. A correct one would have been: THE CURRENT EVIDENCE IS INSUFFICIENT TO DRAW ANY CONCLUSIONS ABOUT THE EFFECTIVENESS OF TUI NA.
Here are some of the most obvious reasons:
- there are far too few studies for a firm conclusion,
- the included RCTs lack scientific rigour,
- all trials originate from China where reliability seems to be a serious problem,
- traction is not a useful therapy for radiculopathy,
- the primary studies violate research ethics by not reporting adverse effects.
Personally, I am getting very tired of conclusions stating ‘…XY MAY BE EFFECTIVE/HELPFUL/USEFUL/WORTH A TRY…’ It is obvious that the therapy in question MAY be effective, otherwise one would surely not conduct a systematic review. If a review fails to produce good evidence, it is the authors’ ethical, moral and scientific obligation to state this clearly. If they don’t, they simply misuse science for promotion and mislead the public. Strictly speaking, this amounts to scientific misconduct.
Therapeutic Touch is a therapy mostly popular with nurses. We have discussed it before, for instance here, here, here and here. To call it implausible would be an understatement. But what does the clinical evidence tell us? Does it work?
This literature review by Iranian authors was aimed at critically evaluating the data from clinical trials examining the clinical efficacy of therapeutic touch as a supportive care modality in adult patients with cancer.
Four electronic databases were searched from the year 1990 to 2015 to locate potentially relevant peer-reviewed articles using the key words therapeutic touch, touch therapy, neoplasm, cancer, and CAM. Additionally, relevant journals and references of all the located articles were manually searched for other potentially relevant studies.
The number of 334 articles was found on the basis of the key words, of which 17 articles related to the clinical trial were examined in accordance with the objectives of the study. A total of 6 articles were in the final dataset in which several examples of the positive effects of healing touch on pain, nausea, anxiety and fatigue, and life quality and also on biochemical parameters were observed.
The authors concluded that, based on the results of this study, an affirmation can be made regarding the use of TT, as a non-invasive intervention for improving the health status in patients with cancer. Moreover, therapeutic touch was proved to be a useful strategy for adult patients with cancer.
This review is badly designed and poorly reported. Crucially, its conclusions are not credible. Contrary to what the authors stated when formulating their aims, the methods lack any attempt of critically evaluating the primary data.
A systematic review is more than a process of ‘pea counting’. It requires a rigorous assessment of the risk of bias of the included studies. If that crucial step is absent, the article is next to worthless and the review degenerates into a promotional excercise. Sadly, this is the case with the present review.
You may think that this is relatively trivial (“Who cares what a few feeble-minded nurses do?”), but I would disagree: if the medical literature continues to be polluted by such irresponsible trash, many people (nurses, journalists, healthcare decision makers, researchers) who may not be in a position to see the fatal flaws of such pseudo-reviews will arrive at the wrong conclusions and make wrong decisions. This will inevitably contribute to a hindrance of progress and, in certain circumstances, must endanger the well-being or even the life of vulnerable patients.
A new study published in JAMA investigated the long-term effects of acupuncture compared with sham acupuncture and being placed in a waiting-list control group for migraine prophylaxis. The trial was a 24-week randomized clinical trial (4 weeks of treatment followed by 20 weeks of follow-up). Participants were randomly assigned to 1) true acupuncture, 2) sham acupuncture, or 3) a waiting-list control group. The trial was conducted from October 2012 to September 2014 in outpatient settings at three clinical sites in China. Participants 18 to 65 years old were enrolled with migraine without aura based on the criteria of the International Headache Society, with migraine occurring 2 to 8 times per month.
Participants in the true acupuncture and sham acupuncture groups received treatment 5 days per week for 4 weeks for a total of 20 sessions. Participants in the waiting-list group did not receive acupuncture but were informed that 20 sessions of acupuncture would be provided free of charge at the end of the trial. Participants used diaries to record migraine attacks. The primary outcome was the change in the frequency of migraine attacks from baseline to week 16. Secondary outcome measures included the migraine days, average headache severity, and medication intake every 4 weeks within 24 weeks.
A total of 249 participants 18 to 65 years old were enrolled, and 245 were included in the intention-to-treat analyses. Baseline characteristics were comparable across the 3 groups. The mean (SD) change in frequency of migraine attacks differed significantly among the 3 groups at 16 weeks after randomization; the mean (SD) frequency of attacks decreased in the true acupuncture group by 3.2 (2.1), in the sham acupuncture group by 2.1 (2.5), and the waiting-list group by 1.4 (2.5); a greater reduction was observed in the true acupuncture than in the sham acupuncture group (difference of 1.1 attacks; 95% CI, 0.4-1.9; P = .002) and in the true acupuncture vs waiting-list group (difference of 1.8 attacks; 95% CI, 1.1-2.5; P < .001). Sham acupuncture was not statistically different from the waiting-list group (difference of 0.7 attacks; 95% CI, −0.1 to 1.4; P = .07).
The authors concluded that among patients with migraine without aura, true acupuncture may be associated with long-term reduction in migraine recurrence compared with sham acupuncture or assigned to a waiting list.
Note the cautious phraseology: “… acupuncture may be associated with long-term reduction …”
The authors were, of course, well advised to be so atypically cautious:
- Comparisons to the waiting list group are meaningless for informing us about the specific effects of acupuncture, as they fail to control for placebo-effects.
- Comparisons between real and sham acupuncture must be taken with a sizable pinch of salt, as the study was not therapist-blind and the acupuncturists may easily have influenced their patients in various ways to report the desired result (the success of patient-blinding was not reported but would have gone some way to solving this problem).
- The effect size of the benefit is tiny and of doubtful clinical relevance.
My biggest concern, however, is the fact that the study originates from China, a country where virtually 100% of all acupuncture studies produce positive (or should that be ‘false-positive’?) findings and data fabrication has been reported to be rife. These facts do not inspire trustworthiness, in my view.
So, does acupuncture work for migraine? The current Cochrane review included 22 studies and its authors concluded that the available evidence suggests that adding acupuncture to symptomatic treatment of attacks reduces the frequency of headaches. Contrary to the previous findings, the updated evidence also suggests that there is an effect over sham, but this effect is small. The available trials also suggest that acupuncture may be at least similarly effective as treatment with prophylactic drugs. Acupuncture can be considered a treatment option for patients willing to undergo this treatment. As for other migraine treatments, long-term studies, more than one year in duration, are lacking.
So, maybe acupuncture is effective. Personally, I am not convinced and certainly do not think that the new JAMA study significantly strengthened the evidence.
Stable angina is a symptom of coronary heart disease which, in turn, is amongst the most frequent causes of death in developed countries. It is an alarm bell to any responsible clinician and requires causal, often life-saving treatments of which we today have several options. The last thing a patient needs in this condition is ACUPUNCTURE, I would say.
Yet acupuncture is precisely the therapy such patients might be tempted to employ.
Because irresponsible or criminally naïve acupuncturists advertise it!
Take this website, for instance; it informs us that a meta-analysis of eight clinical trials conducted between 2000 and 2014 demonstrates the efficacy of acupuncture for the treatment of stable angina. In all eight clinical trials, patients treated with acupuncture experienced a greater rate of angina relief than those in the control group treated with conventional drug therapies (90.1% vs 75.7%)….
I imagine that this sounds very convincing to patients and I fear that many might opt for acupuncture instead of potentially invasive/unpleasant but life-saving intervention. The original meta-analysis to which the above promotion referred to is equally optimistic. Here is its abstract:
Angina pectoris is a common symptom imperiling patients’ life quality. The aim of this study is to evaluate the efficacy and safety of acupuncture for stable angina pectoris. Clinical randomized-controlled trials (RCTs) comparing the efficacy of acupuncture to conventional drugs in patients with stable angina pectoris were searched using the following database of PubMed, Medline, Wanfang and CNKI. Overall odds ratio (ORs) and weighted mean difference (MD) with their 95% confidence intervals (CI) were calculated by using fixed- or random-effect models depending on the heterogeneity of the included trials. Total 8 RCTs, including 640 angina pectoris cases with 372 patients received acupuncture therapy and 268 patients received conventional drugs, were included. Overall, our result showed that acupuncture significantly increased the clinical curative effects in the relief of angina symptoms (OR=2.89, 95% CI=1.87-4.47, P<0.00001) and improved the electrocardiography (OR=1.83, 95% CI=1.23-2.71, P=0.003), indicating that acupuncture therapy was superior to conventional drugs. Although there was no significant difference in overall effective rate relating reduction of nitroglycerin between two groups (OR=2.13, 95% CI=0.90-5.07, P=0.09), a significant reduction on nitroglycerin consumption in acupuncture group was found (MD=-0.44, 95% CI=-0.64, -0.24, P<0.0001). Furthermore, the time to onset of angina relief was longer for acupuncture therapy than for traditional medicines (MD=2.44, 95% CI=1.64-3.24, P<0.00001, min). No adverse effects associated with acupuncture therapy were found. Acupuncture may be an effective therapy for stable angina pectoris. More clinical trials are needed to systematically assess the role of acupuncture in angina pectoris.
In the discussion section of the full paper, the authors explain that their analysis has several weaknesses:
Several limitations were presented in this meta-analysis. Firstly, conventional drugs in control group were different, this may bring some deviation. Secondly, for outcome of the time to onset of angina relief with acupuncture, only one trial included. Thirdly, the result of some outcomes presented in different expression method such as nitroglycerin consumption. Fourthly, acupuncture combined with traditional medicines or other factors may play a role in angina pectoris.
However, this does not deter them to conclude on a positive note:
In conclusion, we found that acupuncture therapy was superior to the conventional drugs in increasing the clinical curative effects of angina relief, improving the electrocardiography, and reducing the nitroglycerin consumption, indicating that acupuncture therapy may be effective and safe for treating stable angina pectoris. However, further clinical trials are needed to systematically and comprehensively evaluate acupuncture therapy in angina pectoris.
So, why do I find this irresponsibly and dangerously misleading?
Here a just a few reasons why this meta-analysis should not be trusted:
- There was no systematic attempt to evaluate the methodological rigor of the primary studies; any meta-analysis MUST include such an assessment, or else it is not worth the paper it was printed on.
- The primary studies all look extremely weak; this means they are likely to be false-positive.
- They often assessed not acupuncture alone but in combination with other treatments; consequently the findings cannot be attributed to acupuncture.
- All the primary studies originate from China; we have seen previously (see here and here) that Chinese acupuncture trials deliver nothing but positive results which means that their results cannot be trusted: they are false-positive.
My conclusion: the authors, editors and reviewers responsible for this article should be ashamed; they committed or allowed scientific misconduct, mislead the public and endangered patients’ lives.
I have warned you before to be sceptical about Chinese studies. This is what I posted on this blog more than 2 years ago, for instance:
Imagine an area of therapeutics where 100% of all findings of hypothesis-testing research are positive, i.e. come to the conclusion that the treatment in question is effective. Theoretically, this could mean that the therapy is a miracle cure which is useful for every single condition in every single setting. But sadly, there are no miracle cures. Therefore something must be badly and worryingly amiss with the research in an area that generates 100% positive results.
Acupuncture is such an area; we and others have shown that Chinese trials of acupuncture hardly ever produce a negative finding. In other words, one does not need to read the paper, one already knows that it is positive – even more extreme: one does not need to conduct the study, one already knows the result before the research has started. But you might not believe my research nor that of others. We might be chauvinist bastards who want to discredit Chinese science. In this case, you might perhaps believe Chinese researchers.
In this systematic review, all randomized controlled trials (RCTs) of acupuncture published in Chinese journals were identified by a team of Chinese scientists. A total of 840 RCTs were found, including 727 RCTs comparing acupuncture with conventional treatment, 51 RCTs with no treatment controls, and 62 RCTs with sham-acupuncture controls. Among theses 840 RCTs, 838 studies (99.8%) reported positive results from primary outcomes and two trials (0.2%) reported negative results. The percentages of RCTs concealment of the information on withdraws or sample size calculations were 43.7%, 5.9%, 4.9%, 9.9%, and 1.7% respectively.
The authors concluded that publication bias might be major issue in RCTs on acupuncture published in Chinese journals reported, which is related to high risk of bias. We suggest that all trials should be prospectively registered in international trial registry in future.
END OF QUOTE
Now an even more compelling reason emerged for taking evidence from China with a pinch of salt:
A recent survey of clinical trials in China has revealed fraudulent practice on a massive scale. China’s food and drug regulator carried out a one-year review of clinical trials. They concluded that more than 80 percent of clinical data is “fabricated“. The review evaluated data from 1,622 clinical trial programs of new pharmaceutical drugs awaiting regulator approval for mass production. Officials are now warning that further evidence malpractice could still emerge in the scandal.
According to the report, much of the data gathered in clinical trials are incomplete, failed to meet analysis requirements or were untraceable. Some companies were suspected of deliberately hiding or deleting records of adverse effects, and tampering with data that did not meet expectations.
“Clinical data fabrication was an open secret even before the inspection,” the paper quoted an unnamed hospital chief as saying. Contract research organizations seem have become “accomplices in data fabrication due to cutthroat competition and economic motivation.”
A doctor at a top hospital in the northern city of Xian said the problem doesn’t lie with insufficient regulations governing clinical trials data, but with the failure to implement them. “There are national standards for clinical trials in the development of Western pharmaceuticals,” he said. “Clinical trials must be carried out in three phases, and they must be assessed at the very least for safety,” he said. “But I don’t know what happened here.”
Public safety problems in China aren’t limited to the pharmaceutical industry and the figure of 80 percent is unlikely to surprise many in a country where citizens routinely engage in the bulk-buying of overseas-made goods like infant formula powder. Guangdong-based rights activist Mai Ke said there is an all-pervasive culture of fakery across all products made in the country. “It’s not just the medicines,” Mai said. “In China, everything is fake, and if there’s a profit in pharmaceuticals, then someone’s going to fake them too.” He said the problem also extends to traditional Chinese medicines, which are widely used in conjunction with Western pharmaceuticals across the healthcare system.
“It’s just harder to regulate the fakes with traditional medicines than it is with Western pharmaceuticals, which have strict manufacturing guidelines,” he said.
According to Luo, academic ethics is an underdeveloped field in China, leading to an academic culture that is accepting of manipulation of data. “I don’t think that the 80 percent figure is overstated,” Luo said.
And what should we conclude from all this?
I find it very difficult to reach a verdict that does not sound hopelessly chauvinistic but feel that we have little choice but to distrust the evidence that originates from China. At the very minimum, I think, we must scrutinise it thoroughly; whenever it looks too good to be true, we ought to discard it as unreliable and await independent replications.
A new Cochrane review evaluated the effectiveness and safety of Chinese herbal medicines (CHM) in the treatment of menopausal symptoms. Its authors conducted a thorough search for randomised controlled trials (RCTs) comparing the effectiveness of CHM with placebo, hormone therapy (HT), pharmaceutical drugs, acupuncture, or another CHM formula in women suffering from menopausal symptoms.
Two review authors independently assessed 864 studies for eligibility. Data extractions were performed by them with disagreements resolved through group discussion and clarification of data or direct contact with the study authors. Data analyses were performed in accordance with Cochrane Collaboration guidelines.
In total, 22 RCTs (2902 women) could be included. When CHM was compared with placebo (8 RCTs), there was little or no evidence of a difference between the groups for the following outcomes: hot flushes per day (MD 0.00, 95% CI -0.88 to 0.89; 2 trials, 199 women; moderate quality evidence); hot flushes per day assessed by an overall hot flush score in which a difference of one point equates to one mild hot flush per day (MD -0.81 points, 95% CI -2.08 to 0.45; 3 RCTs, 263 women; low quality evidence); and overall vasomotor symptoms per month measured by the Menopause-Specific Quality of Life questionnaire (MENQOL, scale 0 to 6) (MD -0.42 points; 95% CI -1.52 to 0.68; 3 RCTs, 256 women; low quality evidence). In addition, results from individual studies suggested there was no evidence of a difference between the groups for daily hot flushes assessed by severity (MD -0.70 points, 95% CI -1.00, -0.40; 1 RCT, 108 women; moderate quality evidence); or overall monthly hot flushes scores (MD -2.80 points, 95% CI -8.93 to 3.33; 1 RCT, 84 women; very low quality evidence); or overall daily night sweats scores (MD 0.07 points, 95% CI -0.19 to 0.33, 1 RCT, 64 women; low quality evidence); or overall monthly night sweats scores (MD 1.30 points, 95% CI -1.76 to 4.36, 1 RCT, 84 women; very low quality evidence). However, one study reported that overall monthly vasomotor symptom scores were lower in the CHM group (MD -4.79 points, 95% CI -5.52 to -4.06; 1 RCT, 69 women; low quality evidence).
When CHM was compared with HT (10 RCTs), only two RCTs reported monthly vasomotor symptoms using MENQOL. It was uncertain whether CHM reduces vasomotor symptoms (MD 0.47 points, 95% CI -0.50 to 1.44; 2 RCTs, 127 women; very low quality evidence).
Adverse effects were not fully reported in the included studies. Adverse events reported by women taking CHM included mild diarrhoea, breast tenderness, gastric discomfort and an unpleasant taste. Effects were inconclusive because of imprecise estimates of effects: CHM versus placebo (RR 1.51; 95% CI 0.69 to 3.33; 7 trials, 705 women; I² = 40%); CHM versus HT (RR 0.96; 95% CI 0.66 to 1.39; 2 RCTs, 864 women; I² = 0%); and CHM versus specific conventional medications (such as Fluoxetine and Estazolam) (RR 0.20; 95% CI 0.03 to 1.17; 2 RCTs, 139 women; I² = 61%).
The authors concluded: We found insufficient evidence that Chinese herbal medicines were any more or less effective than placebo or HT for the relief of vasomotor symptoms. Effects on safety were inconclusive. The quality of the evidence ranged from very low to moderate; there is a need for well-designed randomised controlled studies.
This review seems well done and reports clear findings. The fact that there was insufficient evidence for CHM is probably no surprise to most readers of this blog. However, I would like to draw your attention to a finding that could easily be missed: most of the primary studies failed to mention adverse effects; to be perfectly clear: they did not state “there were no adverse effects”, but they simply did not mention the subject of adverse effects at all.
In my view, this is a breach of research ethics. I have been banging on about this phenomenon for some time now, because I think it is important. Many if not most clinical trials in this area neglect reporting adverse effects. This means that we get an entirely misleading impression about the safety of the treatments in question. Reviewers of such studies are bound to conclude that they seem to be safe, while, in fact, researchers have only been withholding crucial information from us.
The solution to this fast-growing problem would be simple: trialists must be forced to fully report adverse effects. This is less complicated that it might seem: journal editors must insist that all authors fully report adverse effects of alternative treatments. Even if there were none at all – a very unlikely proposition if you think about it – they must disclose this fact.
No, I don’t want to put you off your breakfast… but you probably have seen so many pictures of attractive athletes with cupping marks and read articles about the virtues of this ancient therapy, that I feel I have to put this into perspective:
I am sure you agree that this is slightly less attractive. But, undeniably, these are also cupping marks. So, if you read somewhere that this treatment is entirely harmless, take it with a pinch of salt.
Cupping has existed for centuries in most cultures, and there are several variations of the theme. We differentiate between wet and dry cupping. The above picture is of wet cupping gone wrong. What the US Olympic athletes currently seem to be so fond of is dry cupping.
The principles of both forms are similar. In dry cupping, a vacuum cup is placed over the skin which provides enough suction to create a circular bruise. Eventually the vacuum diminishes, and the cup falls off; what is left is the mark. In wet cupping, the procedure is much the same, except that the skin is injured before the cup is placed. The suction then pulls out a small amount of blood. Obviously the superficial injury can get infected, and that is what we see on the above picture.
In the homeopathic hospital where I worked ~40 years ago, we did a lot of both types of cupping. We used it mostly for musculoskeletal pain. Our patients responded well.
But why? How does cupping work?
The answer is probably more complex than you expect. It clearly has a significant placebo effect. Athletes are obviously very focussed on their body, and they are therefore the ideal placebo-responders. Evidently, my patients 40 years ago also responded to all types of placebos, even to the homeopathic placebos which they received ‘en masse’.
But there might be other mechanisms as well. A TCM practitioner will probably tell you that cupping unblocks the energy flow in our body. This might sound very attractive to athletes or consumers, and therefore could even enhance the placebo response, but it is nevertheless nonsense.
The most plausible mode of action is ‘counter-irritation’: if you have a pain somewhere, a second pain elsewhere in your body can erase the original pain. You might have a headache, for instance, and if you accidentally hit your thumb with a hammer, the headache is gone, at least for a while. Cupping too would cause mild to moderate pain, and this is a distraction from the muscular pain the athletes aim to alleviate.
When I employed cupping 40 years ago, there was no scientific evidence testing its effects. Since a few years, however, clinical trials have started appearing. Many are from China, and I should mention that TCM studies from China almost never report a negative result. According to the Chinese, TCM (including cupping) works for everything. More recently,also some trials from other parts of the world have emerged. They have in common with the Chinese studies that they tend to report positive findings and that they are of very poor quality. (One such trial has been discussed previously on this blog.) In essence, this means that we should not rely on their conclusions.
A further problem with clinical trails of cupping is that it is difficult, if not impossible, to control for the significant placebo effects that this treatment undoubtedly generates. There is no placebo that could mimic all the features of real cupping in clinical trials; and there is no easy way to blind either the patient or the therapist.
So, we are left with an ancient treatment backed by a host of recent but flimsy studies and a growing craze for cupping fuelled by the Olympic games. What can one conclude in such a situation?
Personally, I would, whenever possible, recommend treatments that work beyond a placebo effect, because the placebo response tends to be unreliable and is usually of short duration – and I am not at all sure that cupping belongs into this category. I would also avoid wet cupping, because it can cause substantial harm. Finally, I would try to keep healthcare costs down; cupping itself is cheap but the therapist’s time might be expensive.
In a nutshell: would I recommend cupping? No, not any more than using a hammer for counter irritation! Will the Olympic athletes care a hoot about my recommendations? No, probably not!
Guest post by Frank Van der Kooy
Something happened in 2008. Something, or a number of things, triggered an exponential rise in the number of rhinos being killed in South Africa. Poaching numbers remained quite low and was stable for a decade with only 13 being killed in 2007. But then suddenly it jumped to 83 in 2008 and it reached a total of 1 175 in 2015. To explain this will be difficult and it will be due to a number of factors or events coinciding in 2008. One possible contributing factor, which I will discuss here, is the growing acceptance of TCM in western countries! For example: Phynova recently advertised a new product as being the first traditional Chinese medicine (TCM) being registered in the UK. By directing customers to a separate site for more information regarding their product they ‘accidently’ linked to a site which ‘advertised’ rhino horn (this link has since been removed). Another example is a University in Australia who published a thesis in 2008, in which they described the current use of Rhino horn as a highly effective medicine, just like you would describe any other real medicine. Surely this will have an impact!
But first a bit of background, so please bear with me. There are two ‘opposing’ aspects regarding TCM that most members of the public do not seem to understand well. Not their fault, because the TCM lobby groups are spending a huge amount of effort to keep the lines between these two aspects as blurred as possible. The first aspect is the underlying pseudoscientific TCM principles; the yin and yang and the vitalistic “energy” flow through “meridians” and much more. Science has relegated this to the pseudosciences, just like bloodletting, which was seen as a cure-all hundreds of years ago. Unfortunately, the pseudoscientific TCM principles are still with us and based on these principles almost every single TCM modality works! From acupuncture to herbs to animal matter (including rhino horn) – everything is efficacious, safe and cost effective. Evidence for this is that close to a 100% of clinical trials done on TCM in China give positive results. Strange isn’t it! People in China should thus no die of any disease – they have ‘effective’ medicine for everything! This is the world of TCM in a nutshell.
The second aspect of TCM is the application of the modern scientific method to test which of the thousands of TCM modalities are really active, which ones are useless and which ones are dangerous. Decades of investigation have come up empty-handed with one or two exceptions. One notable exception is Artemisia annua which contain a single compound that is highly effective for the treatment of malaria, and once identified and intensely studied, it was taken up into conventional medicine – not the herb, but the compound. If you investigate all the plants in the world you are bound to find some compounds that can be used as medicine – it has nothing to do with TCM principles and it can most definitely not be used as evidence that the TCM principles are correct or that it based on science.
These two aspects are therefore quite different.
In the TCM world just about everything works, but it is not backed up by science. It is huge market ($170 billion) and it creates employment for many – something that make politicians smile. In the modern scientific world, almost nothing in TCM works, but it is based on science. It is however not profitable at all – you have to investigate thousands of plants in order to find one useful compound.
Many TCM practitioners and researchers are avidly trying to combine the positives of these two worlds. They focus mainly on the money and employment aspect of the TCM world and try and combine this with the modern scientific approach. They tend to focus on the one example where modern science discovered a useful compound (artemisinin) in the medicinal plant Artemisia annua, which was also coincidently used as an herb in TCM – as evidence that TCM works! Here are some examples:
“To stigmatise all traditional medicine would be unfair. After all, a Chinese medicine practitioner last year won a Nobel prize.” No, a Chinese scientist using the modern scientific method identified artemisinin after testing hundreds or even thousands of different plants.
“This year, Chinese medicine practitioners will be registered in Australia. ….. Chinese herbal medicine is administered routinely in hospitals for many chronic diseases. …… This has led to recognising herbs such as Artemisia as a proven anti-malarial ……” No, the compound artemisinin is a proven anti-malarial!
“There has been enormous progress in the last 20 years or so. I am sure you are familiar with the use of one of the Chinese herbs in managing resistant malaria.” No, very little progress and no, the compound artemisinin!
So this is a game that is being played with the simple intention to blur the lines between these two aspects regarding TCM – but the real reason might simply be “A new research-led Chinese medicine clinic in Sydney, better patient outcomes and the potential for Australia to tap into the $170 billion global traditional Chinese medicine market”
Prof Alan Bensoussan the director of the National Institute of Complementary Medicine (NICM) and registered in Australia as a TCM and acupuncture practitioner is a champion in blurring this line. Alan has been instrumental in lobbying the Australian regulatory agency that a long tradition of use is all you need to be able to register new products. He was also influential in establishing the Chinese medicine practitioner registry in Australia, in 2012, and thereby legitimising TCM in Australia. He has been actively chipping away at the resistance that the Australian public have against these pseudoscientific healthcare systems such as TCM – one can argue that he has done so quite successfully because they are expanding their operations into the Westmead precinct of Sydney with a new TCM clinic/hospital.
Enough background; so what does all of this have to do with Rhino horn? (and for that matter other endangered species). We have to remember that in the TCM world just about everything works and that includes rhino horn! Searching Western Sydney University’s theses portal for Xijiao (Chinese for Rhino horn) I found a thesis published in 2008 from the NICM and co-supervised by Alan; “Development of an evidence-based Chinese herbal medicine for the management of vascular dementia”
On page 45-46: “Recently, with fast developing science and technologies being applied in the pharmaceutical manufacturing area, more and more herbs or herbal mixtures have been extracted or made into medicinal injections. These have not only largely facilitated improved application to patients, but also increased the therapeutic effectiveness and accordingly reduced the therapeutic courses …… lists the most common Chinese herbal medicine injections used for the treatment of VaD. “
“Xing Nao Jing Injection (for clearing heat toxin and opening brain, removing phlegm) contains ….. Rhinoceros unicornis (Xijiao), …… Moschus berezovskii (Shexiang), …..”
“…. Xing Nao Jing injection has been widely applied in China for stroke and vascular dementia. …. After 1-month treatment intervention, they found the scores in the treatment group increased remarkably, as compared with the control group …… “
They list two endangered species; the Rhino and the Chinese forest musk deer (Moschus berezovskii). But what is truly worrying is that they don’t even mention the endangered status or at least recommend that the non-endangered substitutes, which do exist in the TCM world, should be used instead – or maybe use fingernails as a substitute? It is not discussed at all. Clearly they are stating that using these endangered animals are way more effective than western medicine (the control group) for the treatment of vascular dementia! This is deplorable to say the least. Statements like this fuels the decimation of this species. But this shows that they truly believe and support the underlying pseudoscientific principles of TCM – they have to, their ability to tap into the TCM market depends on it!
As a scientist you are entitled to discuss historic healthcare treatments such as bloodletting. But make sure to also state that this practice has been shown to be ineffective, and quite dangerous, and that modern science has since come up with many other effective treatments. If it is stated that bloodletting is currently being used and it is effective – then you will simply be promoting bloodletting! The same goes for Rhino horn and this is exactly what they have done here. But then again they live in a world where all TCM modalities are active!
How to solve this problem of growing acceptance of TCM in western countries? A simple step could be that people like Alan publicly denounce the underlying pseudoscientific TCM principles and make the ‘difficult’ switch to real science! Admittedly, he will have to part with lots of money from the CM industry and his Chinese partners, and maybe not built his new TCM hospital! But for some reason I strongly doubt that this will happen. The NICM have successfully applied a very thin, but beautiful, veneer of political correctness and modernity over the surface of complementary medicine. Anyone who cares to look underneath this veneer will find a rotten ancient pseudoscientific TCM world – in this case the promotion and the use of endangered animals.
After reading chapter two of this thesis one cannot believe that this is from an Australian University and paid for by the Australian taxpayer! The main question though: Can I directly link this thesis with the increase in rhino poaching? This will be very difficult if not impossible to do. But that is not the problem. Promoting the pseudoscientific principles of TCM in Australia expands the export market for TCM, and hence will lead to an increased need for raw materials, including the banned Rhino horn. That Rhino horn has been a banned substance since the 1980’s clearly does not seem to have any impact looking at the poaching statistics. In an unrelated paper published in 2010 the ingredients in the Xingnaojing injection is listed as “…. consisting of Chinese herbs such as Moschus, Borneol, Radix Curcumae, Fructus Gardeniae, ….” No full list is given in the paper – dare I say because it contains Rhino horn as well? The drug Ice is also banned, but if you are going to promote it at a ‘trusted’ university, then you shouldn’t be surprised that Ice production increases and more of it flows into Australia – even if it is illegal. The same goes for Rhino horn!