MD, PhD, FMedSci, FRSB, FRCP, FRCPEd.

Chinese studies

1 2 3 7

Thread embedding acupuncture therapy (TEAT) involves the insertion of thread at specific points on the body surface. The claim is that TEAT provides a sustained stimulation of acupoints and is therefore superior to needle acupuncture. Initially, TEAT was used in China to treat obesity, today it is employed to treat many conditions, including musculoskeletal conditions such as ankle sprain, shoulder pain, lumbar intervertebral disc herniation, and plantar fasciitis. Its effectiveness is, however, doubtful and so is its safety.

This review evaluated the safety of thread embedding acupuncture therapy (TEAT) and discuss the prevention and treatment of some adverse events (AEs).

Databases, including China National Knowledge Infrastructure (CNKI), CBMdisc, Wanfang, VIP databases and PubMed, MEDLINE, EMBASE, and Web of Science, were searched from their inception to January 2020. Included were randomized controlled trials (RCTs) and case reports in which AEs with TEAT were reported. Cochrane Collaboration’s tool and RevMan V.5.3.3 software were used to evaluate the quality of the studies.

A total of 61 articles (45 RCTs and 16 case reports) with a total of 620 cases of AEs were included in this review. These studies were published in two countries: China and South Korea. Twenty-eight kinds of AEs were noted. The most common AEs were induration, bleeding and ecchymosis, redness and swelling, fever, and pain. They accounted for 75.35% of all AEs.  Most AEs were mild.; The rarest AEs were epilepsy, irregular menstruation, skin ulcer, thread malabsorption, and fat liquefaction, with 1 case each. Not all of them had a clear causal relationship with TEAT. Most of the AEs were local reactions and systemic reactions accounted for only 1.27%. Although the included studies showed that AEs were very commonly encountered (11.09%), only 5 cases of severe AEs reported from 2013 to 2017 (0.1%) by using catgut thread, which is rarely employed nowadays with new absorbable surgical suture being more popular. All of the patients with severe AEs were recovered after symptomatic treatment with no sequelae.

The authors concluded that the evidence showed that TEAT is a relatively safe and convenient therapy especially since application of new absorbable surgical suture. Improving practitioner skills, regulating operations, and paying attention to the patients’ conditions may reduce the incidence of AEs and improve safety of TEAT.

TEAT was initially used in China only but recently it has become popular elsewhere as well. Therefore the question about its risks has become relevant. The present paper is interesting in that it demonstrates that AEs do occur with some regularity. The authors’ conclusion that TEAT is “relatively safe” is, however, not justified because:

  1. the total sample size was not large enough for a generalizable conclusion;
  2. only RCTs and case reports were included, whereas case series and case-control studies (which would provide more relevant data) were excluded or might not even exist;
  3. RCTs of acupuncture often fail to mention or under-report AEs;
  4. acupuncture papers from China are notoriously unreliable.

So, all we can conclude from the evidence presented here is that AEs after TEAT do occur and do not seem to be all that rare. As the efficacy of TEAT has not been shown beyond doubt, this must inevitably lead to the conclusion that the risk-benefit balance of TEAT is not positive. In turn, that means that TEAT cannot be recommended as a treatment for any condition.

 

The Chinese have made several attempts to persuade us that their traditional remedies are effective for COVID-19 infections. Here is yet another one. This review summarised the evidence of the therapeutic effects and safety of Chinese herbal medicine (CHM) used with or without conventional western therapy for COVID-19. All clinical studies of the therapeutic effects and safety of CHM for COVID-19 were included. The authors

  • summarized the general characteristics of included studies,
  • evaluated the methodological quality of the randomized controlled trials (RCTs) using the Cochrane risk of bias tool,
  • analyzed the use of CHM,
  • used Revman 5.4 software to present the risk ratio (RR) or mean difference (MD) and their 95% confidence interval (CI) to estimate the therapeutic effects and safety of CHM.

A total of 58 clinical studies were identified including;

  • 10 RCTs,
  • 1 non-randomized controlled trials,
  • 11 retrospective studies with a control group,
  • 12 case-series,
  • 24 case-reports.

All of the studies had been performed in China. No RCTs of high methodological quality were identified. The most frequently tested oral Chinese patent medicine, Chinese herbal medicine injection, or prescribed herbal decoction were:

  • Lianhua Qingwen granule/capsule,
  • Xuebijing injection,
  • Maxing Shigan Tang.

The pooled analyses showed that there were statistical differences between the intervention group and the comparator group (RR 0.42, 95% CI 0.21 to 0.82, six RCTs; RR 0.38, 95% CI 0.23 to 0.64, five retrospective studies with a control group), indicating that CHM plus conventional western therapy appeared to be better than conventional western therapy alone in reducing aggravation rate.

In addition, compared with conventional western therapy, CHM plus conventional western therapy had the potential advantages in increasing the recovery rate and shortening the duration of fever, cough, and fatigue, improving the negative conversion rate of nucleic acid test, and increasing the improvement rate of chest CT manifestations and shortening the time from receiving the treatment to the beginning of chest CT manifestations improvement.

For adverse events, the pooled data showed that there were no statistical differences between the CHM and the control groups.

The authors concluded that current low certainty evidence suggests that there maybe a tendency that CHM plus conventional western therapy is superior to conventional western therapy alone. The use of CHM did not increase the risk of adverse events.

One of the principles to remember here is this: RUBBISH IN, RUBBISH OUT. If you meta-analyze primary data that are rubbish, your findings can only be rubbish as well.

All one needs to know about the primary data entered into the present analysis is that there were no rigorous RCTs… not one! That means the evidence is, as the authors rightly but modestly conclude of LOW CERTAINTY. My conclusions would have been a little different:

  1. In terms of safety, the dataset is too small and unreliable to make any judgment.
  2. In terms of efficacy, there is no sound data that CHM has a positive effect.

There are skeptics who keep claiming that there is no research in so-called alternative medicine (SCAM). And there are plenty of SCAM enthusiasts who claim that there is an abundance of good research in SCAM.

Who is right and who is wrong?

I submit that both camps are incorrect.

To demonstrate the volume of SCAM research I looked into Medline to find the number of papers published in 2020 for the SCAMs listed below:

  • acupuncture 2 752
  • anthroposophic medicine 29
  • aromatherapy 173
  • Ayurvedic medicine 183
  • chiropractic 426
  • dietary supplement 5 739
  • essential oil 2 439
  • herbal medicine 5 081
  • homeopathy 154
  • iridology 0
  • Kampo medicine 132
  • massage 824
  • meditation 780
  • mind-body therapies 968
  • music therapy 539
  • naturopathy 68
  • osteopathic manipulation 71
  • Pilates 97
  • qigong 97
  • reiki 133
  • tai chi 397
  • Traditional Chinese Medicine 15 277
  • yoga 698

I think the list proves anyone wrong who claims there is no (or very little) research into SCAM.

As to the enthusiasts who claim that there is plenty of good evidence, I am afraid, I disagree with them too. The above-quoted numbers are perhaps impressive to some SCAM proponents, but they are not large. To make my point more clearly, let me show you the 2020 volumes for a few topics in conventional medicine:

  • psychiatry 668,492
  • biologicals 300,679
  • chemotherapy 109,869
  • radiotherapy 17,964
  • rehabilitation 21,751
  • rehabilitation medicine 21,751
  • surgery 256,958

I think we can agree that these figures make the SCAM numbers look pitifully small.

But the more important point is, I think, not the quantity but the quality of the SCAM research. As this whole blog is about the often dismal rigor of SCAM research, I do surely not need to produce further evidence to convince you that it is poor, often even very poor.

So, both camps tend to be incorrect when they speak about SCAM research. The truth is that there is quite a lot, but sadly reliable studies are like gold dust.

But actually, when I started writing this post and doing all these Medline searches to produce the above-listed volumes of SCAM research, I was thinking of a different subject entirely. I wanted to see which areas of SCAM were research-active and which are not. This is why I chose terms for my list that do not overlap with others (yet we need to realize that the figures are not precise due to misclassification and other factors). And in this respect, the list is interesting too, I find.

It identifies the SCAMs that are remarkably research-inactive:

  • anthroposophic medicine
  • iridology
  • naturopathy
  • osteopathy
  • Pilates
  • qigong

Perhaps more interesting are the areas that show a relatively high research activity:

  • acupuncture
  • dietary supplements
  • essential oils
  • herbal medicine
  • massage
  • meditation
  • mind-body therapies
  • TCM
  • yoga

This, in turn, suggests two things:

  1. It is not true that only commercial interests drive research activity.
  2. The Chinese (TCM and acupuncture) are pushing the ferociously hard to conquer SCAM research.

The last point is worrying, in my view, because we know from several independent studies that Chinese studies are often the flimsiest and least reliable of all the SCAM literature. As I have suggested recently, the unreliability of SCAM research might one day be its undoing: This self-destructive course of SCAM might be applauded by some skeptics. However, if you believe (as I do) that there are a few good things to be found in SCAM, this development can only be regrettable. I fear that the growing dominance of Chinese research will help to speed up this process.

Tuina is a massage therapy that originates from Traditional Chinese Medicine. Many of the techniques used in tuina resemble those of a western massage like gliding, kneading, vibration, tapping, friction, pulling, rolling, pressing, and shaking. Tuina involves a range of manipulations usually performed by the therapist’s finger, hand, elbow, knee, or foot. They are applied to muscle or soft tissue at specific locations of the body.

The aim of Tuina is to enhance the flow of the ‘vital energy’ or ‘chi’, that is alleged to control our health. Proponents of the therapy recommend Tuina for a range of conditions, including paediatric ones. Paediatric Tuina has been widely used in children with acute diarrhea in China. However, due to a lack of high-quality clinical evidence, the benefit of Tuina is not clear.

This study aimed to assess the effect of paediatric Tuina compared with sham Tuina as add-on therapy in addition to usual care for 0-6-year-old children with acute diarrhea.

Eighty-six participants aged 0-6 years with acute diarrhea were randomized to receive Tuina plus usual care (n = 43) or sham Tuina plus usual care (n = 43). The primary outcomes were days of diarrhea from baseline and times of diarrhea on day 3. Secondary outcomes included a global change rating (GCR) and the number of days when the stool characteristics returned to normal. Adverse events were assessed.

Tuina treatment in the intervention group was performed on the surface of the children’s body using moderate pressure (Fig. 1a). Tuina treatment in the control group was different: the therapist used one hand to hold the child’s hand or put one hand on the child’s body, while the other hand performed manipulations on the therapist’s own hand instead of the child’s hand or body (Fig. ​(Fig.11b).

Tuina was associated with a reduction in times of diarrhea on day 3 compared with sham Tuina in both ITT and per-protocol analyses. However, the results were not significant when adjusted for social-demographic and clinical characteristics. No significant difference was found between groups in days of diarrhea, global change rating, or number of days when the stool characteristics returned to normal.

The authors concluded that in children aged 0-6 years with acute diarrhea, pediatric Tuina showed significant effects in terms of reducing times of diarrhea compared with sham Tuina. Studies with larger sample sizes and adjusted trial designs are warranted to further evaluate the effect of pediatric Tuina therapy.

This study was well-reported and has interesting features, such as the attempt to use a placebo control and blinding (whether blinding was successful is a different matter and was not tested in the trial). It is, therefore, all the more surprising that the essentially negative result is turned into a positive one. After adjustment, the differences disappear (a fact which the authors hardly mention in the paper), which means they are not due to the treatment but to group differences and confounding. This, in turn, means that the study shows not the effectiveness but the ineffectiveness of Tuina.

The state of acupuncture research has long puzzled me. The first thing that would strike who looks at it is its phenomenal increase:

  • Until around the year 2000, Medline listed about 200 papers per year on the subject.
  • From 2005, there was a steep, near-linear increase.
  • It peaked in 2020 when we had a record-breaking 20515 acupuncture papers currently listed in Medline.

Which this amount of research, one would expect to get somewhere. In particular, one would hope to slowly know whether acupuncture works and, if so, for which conditions. But this is not the case.

On the contrary, the acupuncture literature is a complete mess in which it gets more and more difficult to differentiate the reliable from the unreliable, the useful from the redundant, and the truth from the lies. Because of this profound confusion, acupuncture fans are able to claim that their pet-therapy is demonstrably effective for a wide range of conditions, while skeptics insist it is a theatrical placebo. The consumer might listen in bewilderment.

Yesterday (18/1/2021), I had a quick (actually, it was not that quick after all) look into what Medline currently lists in terms of new acupuncture research published in 2021 and found a few other things that are remarkable:

  1. There were already 100 papers dated 2021 (today, there were even 118); that corresponds to about 5 new articles per day and makes acupuncture one of the most research-active areas of so-called alternative medicine (SCAM).
  2. Of these 100 papers, only 7 were clinical trials (CTs). In my view, clinical trials would be more important than any other type of research on acupuncture. To see that they amount to just 7% of the total is therefore disappointing.
  3. Twelve papers were systematic reviews (SRs). It is odd, I find, to see almost twice the amount of SRs than CTs.
  4. Eighteen papers referred to protocols of studies of SRs. In particular protocols of SRs are useless in my view. It seems to me that the explanation for this plethora of published protocols might be the fact that Chinese researchers are extremely keen to get papers into Western journals; it is an essential boost to their careers.
  5. Seven papers were surveys. This multitude of survey research is typical for all types of SCAM.
  6. Twenty-four articles were on basic research. I find basic research into an ancient therapy of questionable clinical use more than a bit strange.
  7. The rest of the articles were other types of publications and a few were misclassified.
  8. The vast majority (n = 81) of the 100 papers were authored exclusively by Chinese researchers (and a few Korean). In view of the fact that it has been shown repeatedly that practically all acupuncture studies from China report positive results and that data fabrication seems rife in China, this dominance of China could be concerning indeed.

Yes, I find all this quite concerning. I feel that we are swamped with plenty of pseudo-research on acupuncture that is of doubtful (in many cases very doubtful) reliability. Eventually, this will create an overall picture for the public that is misleading to the extreme (to check the validity of the original research is a monster task and way beyond what even an interested layperson can do).

And what might be the solution? I am not sure I have one. But for starters, I think, that journal editors should get a lot more discerning when it comes to article submissions from (Chinese) acupuncture researchers. My advice to them and everyone else:

if it sounds too good to be true, it probably is!

Bloodletting has been used for centuries in many cultures. Its principle, it was assumed, consisted in re-balancing the body’s four humours. Bloodletting had a detrimental effect on most diseases and must have killed millions. It is a good historical example of the harm that ensues, if healthcare adheres to dogma. Today, we know that bloodletting is useful only in rare conditions such as polycythaemia vera or haemochromatosis (and some 30 years ago, a variation of bloodletting, isovolaemic haemodilution, was being discussed as a treatment for circulatory diseases such as intermittent claudication or stroke).

Yet, in so-called alternative medicine (SCAM), there are some practitioners who seem to find it hard to concede that ancient treatments might be not as good as they think. Thus, bloodletting has survived in this realm as a therapy for a wide range of conditions. This study assessed the efficacy of bloodletting therapy (acupoint pricking and cupping) in patients with chronic idiopathic urticaria (CIU) in a randomized, control, parallel-group trial.

A total of 174 patients with CIU were randomized into three groups:

  1. group A was treated with bloodletting therapy and ebastine (an anti-histamine),
  2. group B was treated with placebo treatment (acupoint pseudopricking and cupping) and ebastine,
  3. group C was treated with ebastine only.

The treatment period lasted 4 weeks. An intention-to-treat analysis was conducted, and the primary outcome was the effective rate of UAS7 score being reduced to 7 or below after treatment phase.

The effective rates at the end of treatment phase were different among the three groups, which were

  • 73.7% in group A,
  • 45.6% in group B,
  • and 42.9% in group C.

Multiple analysis indicated differences between groups A and B (P < 0.0125) and groups A and C (P < 0.0125) and no difference between groups B and C (P > 0.0125). No severe bloodletting therapy-related adverse events were observed.

The authors concluded that one month of bloodletting therapy combined with ebastine is clinically beneficial compared with placebo treatment combined with ebastine and treatment with ebastine only. Thus, bloodletting therapy can be an effective complementary treatment in CIU.

Amazed?

Me too!

How on earth might bloodletting help for CIU? Luckily, the authors have an answer to this question:

The clinical feature of urticaria with wheals and pruritus coming and going quickly is the manifestation of wind-evil that lurks in and circulates with blood. Hence, in the treatment of urticaria, dispersing wind is the one of the principle methods, and treating blood before wind is an important procedure because when blood flows fluently , wind-evil will resolve spontaneously. Bloodletting therapy is a direct and effective way of regulating blood.

You see, it’s all perfectly clear!

In this case, the results must be true. And the argument that patients might have known in which treatment group they had ended up (and were thus not blinded) can be discarded.

Acupuncture-moxibustion therapy (AMT) is a so-called alternative medicine (SCAM) that has been used for centuries in treatment of numerous diseases. Some enthusiasts even seem to advocate it for chemotherapy-induced leukopenia (CIL)  The purpose of this review was to evaluate the efficacy and safety of acupuncture-moxibustion therapy in treating CIL.

Relevant studies were searched in 9 databases up to September 19, 2020. Two reviewers independently screened the studies for eligibility, extracted data, and assessed the methodological quality of selected studies. Meta-analysis of the pooled mean difference (MD) and risk ratio (RR) with their respective 95% confidence intervals (CI) were calculated.

Seventeen studies (1206 patients) were included, and the overall quality of the included studies was moderate. In comparison with medical therapy, AMT has a better clinical efficacy for CIL (RR, 1.24; 95% CI, 1.17-1.32; P < 0.00001) and presents advantages in increasing leukocyte count (MD, 1.10; 95% CI, 0.67-1.53; P < 0.00001). Also, the statistical results show that AMT performs better in improving the CIL patients’ Karnofsky performance score (MD, 5.92; 95% CI, 3.03-8.81; P < 0.00001).

The authors concluded that this systematic review and meta-analysis provides updated evidence that AMT is a safe and effective alternative for the patients who suffered from CIL.

A CIL is a serious complication. If I ever were afflicted by it, I would swiftly send any acupuncturist approaching my sickbed packing.

But this is not an evidence-based attitude!!!, I hear some TCM-fans mutter. What more do you want that a systematic review showing it works?

I beg to differ. Why? Because the ‘evidence’ is hardly what critical thinkers can accept as evidence. Have a look at the list of the primary studies included in this review:

  1. Lin Z. T., Wang Q., Yu Y. N., Lu J. S. Clinical observation of post-chemotherapy-leukopenia treated with ShenMai injectionon ST36. World Journal of Integrated Traditional and Western Medicine2010;5(10):873–876. []
  2. Wang H. Clinical Observation of Acupoint Moxibustion on Leukopenia Caused by Chemotherapy. Beijing, China: Beijing University of Chinese Medicine; 2011. []
  3. Fan J. Y. Coupling of Yin and Yang between Ginger Moxibustion Improve the Clinical Effect of the Treatment of Chemotherapy Adverse Reaction. Henan, China: Henan University of Chinese Medicine; 2013. []
  4. Lu D. R., Lu D. X., Wei M., et al. Acupoint injection with addie injection for patients of nausea and vomiting with cisplatin induced by chemotherapy. Journal of Clinical Acupuncture and Moxibustion2013;29(10):33–38. []
  5. Yang J. E. The Clinical Observation on Treatment of Leukopenia after Chemotherapy with Needle Warming Moxibustion. Hubei, China: Hubei University of Chinese Medicine; 2013. []
  6. Fu Y. H., Chi C. Y., Zhang C. Y. Clinical effect of acupuncture and moxibustion on leukopenia after chemotherapy of malignant tumor. Guide of China Medicine2014;12(12) []
  7. Wang J. N., Zhang W. X., Gu Q. H., Jiao J. P., Liu L., Wei P. K. Protection of herb-partitioned moxibustion on bone marrow suppression of gastric cancer patients in chemotherapy period. Chinese Archives of Traditional Chinese Medicine2014;32(12):110–113. []
  8. Zhang J. The Clinical Research on Myelosuppression and Quality of Life after Chemotherapy Treated by Grain-Sized Moxibustion. Nanjing, China: Nanjing University of Chinese Medicine; 2014. []
  9. Tian H., Lin H., Zhang L., Fan Z. N., Zhang Z. L. Effective research on treating leukopenia following chemotherapy by moxibustion. Clinical Journal of Chinese Medicine2015;7(10):35–38. []
  10. Hu G. W., Wang J. D., Zhao C. Y. Effect of acupuncture on the first WBC reduction after chemotherapy for breast cancer. Beijing Journal of Traditional Chinese Medicine2016;35(8):777–779. []
  11. Zhu D. L., Lu H. Y., Lu Y. Y., Wu L. J. Clinical observation of Qi-blood-supplementing needling for leukopenia after chemotherapy for breast cancer. Shanghai Journal of Acupuncture and Moxibustion2016;35(8):964–966. []
  12. Chen L, Xu G. Y. Observation on the prevention and treatment of chemotherapy-induced leukopenia by moxibustion therapy. Zhejiang Journal of Traditional Chinese Medicine2016;51(8):p. 600. []
  13. Mo T., Tian H., Yue S. B., Fan Z. N., Zhang Z. L. Clinical observation of acupoint moxibustion on leukocytopenia caused by tumor chemotherapy. World Chinese Medicine2016;11(10):2120–2122. []
  14. Nie C. M. Nursing observation of acupoint moxibustion in the treatment of leucopenia after chemotherapy. Today Nurse2017;4:93–95. []
  15. Wang D. Y. Clinical Research on Post-chemotherapy-leukopenia with Spleen-Kidney Yang Deficiency in Colorectal Cancer Treated with Point-Injection. Yunnan, China: Yunnan University of Chinese Medicine; 2017. []
  16. Gong Y. Q, Zhang M. Q, Zhang B. C. Prevention and treatment of leucocytopenia after chemotherapy in patients with malignant tumor with ginger partitioned moxibustion. Chinese Medicine Modern Distance Education of China2018;16(21):135–137. []
  17. Li Z. C., Lian M. J., Miao F. G. Clinical observation of fuzheng moxibustion combined with wenyang shengbai decoction in the treatment of 80 cases of leukopenia after chemotherapy. Hunan Journal of Traditional Chinese Medicine2019;35(3):64–66. []

Notice anything peculiar?

  • The studies are all from China where data fabrication was reported to be rife.
  • They are mostly unavailable for checking (why the published adds links that go nowhere is beyond me).
  • Many do not look at all like randomised clinical trials (which, according to the authors, was an inclusion criterion).
  • Many do not look as though their primary endpoint was the leukocyte count (which, according to the authors, was another inclusion criterion).

Intriguingly, the authors conclude that AMT is not just effective but also ‘safe’. How do they know? According to their own data extraction table, most studies failed to mention adverse effects. And how exactly is acupuncture supposed to increase my leukocyte count? Here is what the authors offer as a mode of action:

Based on the theory of traditional Chinese medicine (TCM), CIL belongs to the category of consumptive disease, owing to the exhaustion of genuine qi in the zang-fu viscera and the insufficiency of kidney essence and qi-blood. Researchers believe that there is an intimate association between the occurrence of malignant tumors and the deficiency of genuine qi. During attacking the cancer cells, chemotherapeutics also damaged the function of zang-fu viscera and qi-blood, leading to CIL. According to the theory of TCM and meridian, acupuncture-moxibustion is an ancient therapeutic modality that may be traced back more than 3500 years in China. Through meridian conduction, acupuncture-moxibustion therapy stimulates acupoints to strengthen the condition of zang-fu viscera and immune function, supporting genuine qi to improve symptoms of consumption.

I think it is high time that we stop tolerating that the medical literature gets polluted with such nonsense (helped, of course, by journals that are beyond the pale) – someone might actually believe it, in which case it would surely hasten the death of vulnerable patients.

Some consumers believe that research is by definition reliable, and patients are even more prone to this error. When they read or hear that ‘RESEARCH HAS SHOWN…’ or that ‘A RECENT STUDY HAS DEMONSTRATED…’, they usually trust the statements that follow. But is this trust in research and researchers justified? During 25 years that I have been involved in so-called alternative medicine (SCAM), I have encountered numerous instances which make me doubt. In this post, I will briefly discuss some the many ways in which consumers can be mislead by apparently sound evidence (for an explanation as to what is and what isn’t evidence, see here).

ABSENCE OF EVIDENCE

I have just finished reading a book by a German Heilpraktiker that is entirely dedicated to SCAM. In it, the author makes hundreds of statements and presents them as evidence-based facts. To many lay people or consumers, this will look convincing, I am sure. Yet, it has one fatal defect: the author fails to offer any real evidence that would back up his statements. The only references provided were those of other books which are equally evidence-free. This popular technique of making unsupported claims allows the author to make assertions without any checks and balances. A lay person is usually unable or unwilling to differentiate such fabulations from evidence, and this technique is thus easy and poular for misleading us about SCAM.

FAKE-EVIDENCE

On this blog, we have encountered this phenomenon ad nauseam: a commentator makes a claim and supports it with some seemingly sound evidence, often from well-respected sources. The few of us who bother to read the referenced articles quickly discover that they do not say what the commentator claimed. This method relies on the reader beeing easily bowled over by some pretend-evidence. As many consumers cannot be bothered to look beyond the smokescreen supplied by such pretenders, the method usually works surprisingly well.

An example: Vidatox is a homeopathic cancer ‘cure’ from Cuba. The Vidatox website clains that it is effective for many cancers. Considering how sensational this claim is, one would expect to find plenty of published articles on Vidatox. However, a Medline search resulted in one paper on the subject. Its authors drew the following conclusion: Our results suggest that the concentration of Vidatox used in the present study has not anti-neoplastic effects and care must be taken in hiring Vidatox in patients with HCC. 

The question one often has to ask is this: where is the line between misleading research and fraud?

SURVEYS

There is no area in healthcare that produces more surveys than SCAM. About 500 surveys are published every year! This ‘survey-mania’ has a purpose: it promotes a positive message about SCAM which hypothesis-testing research rarely does.

For a typical SCAM survey, a team of enthusiastic researchers might put together a few questions and design a questionnaire to find out what percentage of a group of individuals have tried SCAM in the past. Subsequently, the investigators might get one or two hundred responses. They then calculate simple descriptive statistics and demonstrate that xy % use SCAM. This finding eventually gets published in one of the many third-rate SCAM journals. The implication then is that, if SCAM is so popular, it must be good, and if it’s good, the public purse should pay for it. Few consumers would realise that this conclusion is little more that a fallacious appeal to popularity.

AVOIDING THE QUESTION

Another popular way of SCAM researchers to mislead the public is to avoid the research questions that matter. For instance, few experts would deny that one of the most urgent issues in chiropractic relates to the risk of spinal manipulations. One would therefore expect that a sizable proportion of the currently published chiropractic research is dedicated to it. Yet, the opposite is the case. Medline currently lists more than 3 000 papers on ‘chiropractic’, but only 17 on ‘chiropractic, harm’.

PILOT-STUDIES

A pilot study is a small scale preliminary study conducted in order to evaluate feasibility, time, cost, adverse events, and improve upon the study design prior to performance of a full-scale research project. Yet, the elementary preconditions are not fulfilled by the plethora of SCAM pilot studies that are currently being published. True pilot studies of SCAM are, in fact, very rare. The reason for the abundance of pseudo-pilots is obvious: they can easily be interpreted as showing encouragingly positive results for whatever SCAM is being tested. Subsequently, SCAM proponents can mislead the public by claiming that there are plenty of positive studies and therefore their SCAM is supported by sound evidence.

‘SAFE‘ STUDY-DESIGNS

As regularly mentioned on this blog, there are several ways to design a study such that the risk of producing a negative result is minimal. The most popular one in SCAM research is the ‘A+B versus B’ design. In this study, for instance, cancer patients who were suffering from fatigue were randomised to receive usual care or usual care plus regular acupuncture. The researchers then monitored the patients’ experience of fatigue and found that the acupuncture group did better than the control group. The effect was statistically significant, and an editorial in the journal where it was published called this evidence “compelling”. Due to a cleverly over-stated press-release, news spread fast, and the study was celebrated worldwide as a major breakthrough in cancer-care.

Imagine you have an amount of money A and your friend owns the same sum plus another amount B. Who has more money? Simple, it is, of course your friend: A+B will always be more than A [unless B is a negative amount]. For the same reason, such “pragmatic” trials will always generate positive results [unless the treatment in question does actual harm]. Treatment as usual plus acupuncture is more than treatment as usual alone, and the former is therefore more than likely to produce a better result. This will be true, even if acupuncture is a pure placebo – after all, a placebo is more than nothing, and the placebo effect will impact on the outcome, particularly if we are dealing with a highly subjective symptom such as fatigue.

A more obvious method for generating false positive results is to omit blinding. The purpose of blinding the patient, the therapist and the evaluator of the group allocation in clinical trials is to make sure that expectation is not a contributor to the result. Expectation might not move mountains, but it can certainly influence the result of a clinical trial. Patients who hope for a cure regularly do get better, even if the therapy they receive is useless, and therapists as well as evaluators of the outcomes tend to view the results through rose-tinted spectacles, if they have preconceived ideas about the experimental treatment.

Failure to randomise is another source of bias which can mislead us. If we allow patients or trialists to select or chose which patients receive the experimental and which get the control-treatment, it is likely that the two groups differ in a number of variables. Some of these variables might, in turn, impact on the outcome. If, for instance, doctors allocate their patients to the experimental and control groups, they might select those who will respond to the former and those who don’t to the latter. This may not happen with intent but through intuition or instinct: responsible health care professionals want those patients who, in their experience, have the best chances to benefit from a given treatment to receive that treatment. Only randomisation can, when done properly, make sure we are comparing comparable groups of patients. Non-randomisation can easily generate false-positive findings.

It is also possible to mislead people with studies which do not test whether an experimental treatment is superior to another one (often called superiority trials), but which assess whether it is equivalent to a therapy that is generally accepted to be effective. The idea is that, if both treatments produce similarly positive results, both must be effective.  Such trials are called non-superiority or equivalence trials, and they offer a wide range of possibilities for misleading us. If, for example, such a trial has not enough patients, it might show no difference where, in fact, there is one. Let’s consider a simple, hypothetical example: someone comes up with the idea to compare antibiotics to acupuncture as treatments of bacterial pneumonia in elderly patients. The researchers recruit 10 patients for each group, and the results reveal that, in one group, 2 patients died, while, in the other, the number was 3. The statistical tests show that the difference of just one patient is not statistically significant, and the authors therefore conclude that acupuncture is just as good for bacterial infections as antibiotics.

Even trickier is the option to under-dose the treatment given to the control group in an equivalence trial. In the above example, the investigators might subsequently recruit hundreds of patients in an attempt to overcome the criticism of their first study; they then decide to administer a sub-therapeutic dose of the antibiotic in the control group. The results would then seemingly confirm the researchers’ initial finding, namely that acupuncture is as good as the antibiotic for pneumonia. Acupuncturists might then claim that their treatment has been proven in a very large randomised clinical trial to be effective for treating this condition. People who do not happen to know the correct dose of the antibiotic could easily be fooled into believing them.

Obviously, the results would be more impressive, if the control group in an equivalence trial received a therapy which is not just ineffective but actually harmful. In such a scenario, even the most useless SCAM would appear to be effective simply because it is less harmful than the comparator.

A variation of this theme is the plethora of controlled clinical trials in SCAM which compare one unproven therapy to another unproven treatment. Perdicatbly, the results would often indicate that there is no difference in the clinical outcome experienced by the patients in the two groups. Enthusiastic SCAM researchers then tend to conclude that this proves both treatments to be equally effective. The more likely conclusion, however, is that both are equally useless.

Another technique for misleading the public is to draw conclusions which are not supported by the data. Imagine you have generated squarely negative data with a trial of homeopathy. As an enthusiast of homeopathy, you are far from happy with your own findings; in addition you might have a sponsor who puts pressure on you. What can you do? The solution is simple: you only need to highlight at least one positive message in the published article. In the case of homeopathy, you could, for instance, make a major issue about the fact that the treatment was remarkably safe and cheap: not a single patient died, most were very pleased with the treatment which was not even very expensive.

OMISSION

A further popular method for misleading the public is the outright omission findings that SCAM researchers do not like. If the aim is that the public believe the myth that all SCAM is free of side-effects, SCAM researchers only need to omit reporting them in clinical trials. On this blog, I have alerted my readers time and time again to this common phenomenon. We even assessed it in a systematic review. Sixty RCTs of chiropractic were included. Twenty-nine RCTs did not mention adverse effects at all. Sixteen RCTs reported that no adverse effects had occurred. Complete information on incidence, severity, duration, frequency and method of reporting of adverse effects was included in only one RCT.

Most trails have many outcome measures; for instance, a study of acupuncture for pain-control might quantify pain in half a dozen different ways, it might also measure the length of the treatment until pain has subsided, the amount of medication the patients took in addition to receiving acupuncture, the days off work because of pain, the partner’s impression of the patient’s health status, the quality of life of the patient, the frequency of sleep being disrupted by pain etc. If the researchers then evaluate all the results, they are likely to find that one or two of them have changed in the direction they wanted (especially, if they also include half a dozen different time points at which these variables are quatified). This can well be a chance finding: with the typical statistical tests, one in 20 outcome measures would produce a significant result purely by chance. In order to mislead us, the researchers only need to “forget” about all the negative results and focus their publication on the ones which by chance have come out as they had hoped.

FRAUD

When it come to fraud, there is more to chose from than one would have ever wished for. We and others have, for example, shown that Chinese trials of acupuncture hardly ever produce a negative finding. In other words, one does not need to read the paper, one already knows that it is positive – even more extreme: one does not need to conduct the study, one already knows the result before the research has started. This strange phenomenon indicates that something is amiss with Chinese acupuncture research. This suspicion was even confirmed by a team of Chinese scientists. In this systematic review, all randomized controlled trials (RCTs) of acupuncture published in Chinese journals were identified by a team of Chinese scientists. A total of 840 RCTs were found, including 727 RCTs comparing acupuncture with conventional treatment, 51 RCTs with no treatment controls, and 62 RCTs with sham-acupuncture controls. Among theses 840 RCTs, 838 studies (99.8%) reported positive results from primary outcomes and two trials (0.2%) reported negative results. The percentages of RCTs concealment of the information on withdraws or sample size calculations were 43.7%, 5.9%, 4.9%, 9.9%, and 1.7% respectively. The authors concluded that publication bias might be major issue in RCTs on acupuncture published in Chinese journals reported, which is related to high risk of bias. We suggest that all trials should be prospectively registered in international trial registry in future.

A survey of clinical trials in China has revealed fraudulent practice on a massive scale. China’s food and drug regulator carried out a one-year review of clinical trials. They concluded that more than 80 percent of clinical data is “fabricated“. The review evaluated data from 1,622 clinical trial programs of new pharmaceutical drugs awaiting regulator approval for mass production. Officials are now warning that further evidence malpractice could still emerge in the scandal.

I hasten to add that fraud in SCAM research is certainly not confined to China. On this blog, you will find plenty of evidence for this statement, I am sure.

CONCLUSION

Research is obviously necessary, if we want to answer the many open questions in SCAM. But sadly, not all research is reliable and much of SCAM research is misleading. Therefore, it is always necessary to be on the alert and apply all the skills of critical evaluation we can muster.

 

At present, we see a wave of promotion of Traditional Chinese Medicine (TCM) as a treatment of corona-virus infections. In this context, we should perhaps bear in mind that much of the Chinese data seem to be less than reliable. Moreover, I find it important to alert people to a stern warning recently published by two Australian experts. Here is the crucial passage from their paper:

We wish to highlight significant concerns regarding the association between traditional herbal medicines and severe, non-infective interstitial pneumonitis and other aggressive pulmonary syndromes, such as diffuse alveolar haemorrhage and ARDS which have emerged from Chinese and Japanese studies particularly during the period 2017−2019. Initially the association between traditional herbal therapies and pneumonitis was based on isolated case reports. These included hypersensitivity pneumonitis associated with the use of traditional Chinese or Japanese medicines such as Sai-rei-to, Oren-gedoku-to, Seisin-renshi-in and Otsu-ji-to (9 references in supplemental file). Larger cohorts and greater numbers now support this crucial relationship. In a Japanese cohort of 73 patients, pneumonitis development occurred within 3 months of commencing traditional medicine in the majority of patients [], while a large report from the Japanese Ministry of Health, Labor and Welfare, described more than 1000 cases of lung injury secondary to traditional medications, the overwhelming majority of which (852 reports) were described as ‘interstitial lung disease [].

Currently the constituent of traditional herbal medicines which is considered most likely to underlie causation of lung disease is Scutellariae Radix also known as Skullcap or ou-gon, which has been implicated through immunological evidence of hypersensitivity as well as circumstantial evidence, being present in all of those medicines outlined above []. Notably, skullcap is a constituent of QPD as used and described in the paper by Ren et al. relating to COVID-19 []. Scutellariae Radix-induced ARDS and COVID-19 disease share the same characteristic chest CT changes such as ground-glass opacities and airspace consolidation, therefore distinguishing between lung injury due to SARS-CoV-2 and that secondary to TCM may be very challenging. The potential for iatrogenic lung injury with TCM needs to be acknowledged []…

Morbidity and mortality from COVID-19 are almost entirely related to lung pathology []. Factors which impose a burden on lung function such as chronic lung disease and smoking are associated with increased risk for a poor outcome. Severe COVID-19 may be associated with a hypersensitivity pneumonitis component responsive to corticosteroid therapy []. Against this background the use of agents with little or no evidence of clinical efficacy and which have been significantly implicated in causing interstitial pneumonitis that could complicate SARS-CoV-2 infection, should be considered with extreme caution.

In conclusion, the benefits of TCM in the treatment of COVID-19 remain unproven and may be potentially deleterious. We recognise that there is currently insufficient evidence to prove the role of TCM in the causation of interstitial pneumonitis, however the circumstantial data is powerful and it would seem prudent to avoid these therapies in patients with known or suspected SARS-CoV-2 infection, until the evidence supports their use.

Declaration of Competing Interest: There are no conflicts to declare.

Yesterday, I received a tweet from a guy called Bart Huisman (“teacher beekeeping, nature, biology, classical homeopathy, agriculture, health science, social science”). I don’t know him and cannot remember whether I had previous contact with him. His tweet read as follows:

“Why should anyone believe what Professor Edzard Ernst says, after he put his name to a BBC programme, he now describes as “deception”.”

This refers to a story that I had almost forgotten. It’s a nice one with a ‘happy ending’, so let me recount it here briefly.

In 2005, the BBC had hired me as an advisor for their 4-part TV series on alternative medicine.

The first part of the series was on acupuncture, and Prof Kathy Sykes presented the opening scene taking place in a Chinese operation theatre. In it a Chinese women was having open heart surgery with the aid of acupuncture. Kathy’s was visibly impressed and said on camera that the patient was having the surgery “with only needles to control the pain.”  However, the images clearly revealed that the patient was receiving all sorts of other treatments given through intra-venous lines. So, Prof Sykes was telling the UK public a bunch of porkies. This was bound to confuse many viewers.

One of them was Simon Singh. At the time, I did not know Simon (to be honest, I did not even know of him) and was surprised to receive a phone call from him. He politely asked me to confirm that I had been the adviser of the BBC on this production. I was happy to confirm this fact. Then he asked why I had missed such a grave error. I replied that I could not possibly have spotted it, because all I had been asked to do was to review and correct the text of the programme which the BBC had sent to me by email. Before it was broadcast, I had not seen a single passage of the film.

Correcting the text had already led to several problems (not so much regarding the acupuncture part but mostly the other sections), because the BBC was reluctant to change several of the mistakes I had identified. When I told them that, in this case, I would quit, they finally found a way to alter them. But the cooperation had been far from easy. I explained all this to Simon and eventually he asked me whether I would be willing to support the official complaint he was about to file with the BBC. I agreed. This is probably where I used the term ‘deception’ that Mr Huisman mentioned in his tweet.

So, Simon submitted his complaint and eventually won the case.

But this is not the happy ending I was referring to.

During the course of the complaint, Simon and I realised that we were thinking alike and were getting on well. A few months later, he suggested that the two of us write a book together about alternative medicine. At first, I was hesitant. Simon said, “let’s try just one chapter, and see how it works out.” So we did. It turned out to be fun and instructive for both of us. So we did the other chapters as well. The book was published in 2008 and is called TRICK OR TREATMENT. It was published in about 20 different languages and the German version became ‘science book of the year in 2011 (I think).

And that’s not the happy ending either (in fact, it caused a lot of hardship for Simon who was sued by the BCA; luckily, he won that case too).

The real happy ending is the fact that Simon and I became friends for life.

Thank you Bart Huisman for reminding me of this rather lovely story.

 

1 2 3 7
Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted but you must tick the box: “Save my name, email, and website in this browser for the next time I comment.”

The most recent comments from all posts can be seen here.

Archives
Categories