MD, PhD, FMedSci, FRSB, FRCP, FRCPEd.

systematic review

1 2 3 33

Several previously published clinical trials have suggested that both acupuncture and sham acupuncture exert significant, non-specific effects on treatment outcomes when compared to no-treatment controls. A recently developed framework (mechanisms in orthodox and complementary and alternative medicine-MOCAM) suggests that the non-specific effects of acupuncture originate from multiple domains (e.g. patient characteristics, acupuncturist skill/technique, the patient-acupuncturist relationship, and the acupuncture environment). However, it remains to be determined precisely how these domains influence the non-specific effects of treatment among patients receiving acupuncture and sham acupuncture in clinical trials.

To address this issue, researchers conducted a systematic review to synthesize existing qualitative evidence on how trial participants randomized to acupuncture and sham acupuncture groups experience non-specific effects, regardless of the types of medical conditions investigated.

This systematic review included primary qualitative studies embedded in randomized controlled trials designed to investigate acupuncture or sham acupuncture interventions. Eligible studies published in English were derived from a search of five international databases. The methodological quality of included studies was evaluated using the Critical Appraisal Skills Programme (CASP) tool. Using a framework synthesis approach, the identified MOCAM framework was adapted based on the synthesis of the available qualitative evidence.

A total of 20 studies of high methodological quality were included. The proposed model indicated that the effects of acupuncture may be increased by:

  • maintaining a professional status,
  • applying a holistic treatment approach,
  • practicing empathy,
  • providing patients with an appropriate explanation of the theory behind acupuncture and sham acupuncture.

From the patient’s perspective, the efficacy of treatment can be increased by:

  • following the lifestyle modification advice provided by acupuncturists,
  • maintaining a positive attitude toward treatment efficacy,
  • actively engaging with acupuncturists during the consultation,
  • making behavioral changes based on experience gained during the trial.

The authors concluded that the results of this study may provide a basis for improving and standardizing key components of non-specific effects in acupuncture treatment, and for improving the isolation of specific effects in future clinical trials involving acupuncture and sham acupuncture.

The authors also state that having a positive attitude and high expectations regarding treatment efficacy can lead to positive health outcomes, along with a sense of curiosity and altruistic desire to join clinical trials. Indeed, previous clinical trials have reported that higher expectations regarding treatment effects may help to reduce fatigue and alleviate osteoarthritis in both acupuncture and sham acupuncture groups. Similar benefits of positive expectations have also been observed among patients with irritable bowel syndrome in sham acupuncture trials. 

SO CLOSE AND YET SO FAR!

So close to admitting that these findings indicate quite strongly that acupuncture is but a theatrical placebo.

Yesterday, someone posted a disparaging comment about Indian research into homeopathy; he claimed that it was unreliable. I agreed, but later I thought ‘HOW ARROGANT OF ME!’. So, I decided to do a little research – actually, it turned out to be a little more than just ‘a little’.

I searched Medline for ‘homeopathy, study, India’. This resulted in 101 hits. Of these 101 articles, 31 contained data published by Indian authors providing evidence at least vaguely related to the effectiveness of homeopathy. I decided to include these in my analysis. Below I quote first the title of each paper followed by (in brackets) the sentence from the 31 abstracts that best describes the direction of the results.

  1. Multimorbidity After Surgical Menopause Treated with Individualized Classical Homeopathy: A Case Report (She was treated with individualized classical homeopathy and followed up for 31 months. She was relieved of the vasomotor symptoms and psychological disturbances of climacteric syndrome, her weight reduced, the ultrasound scan showed absence of lipomatosis/gall bladder disease/hepatic steatosis. Blood tests showed reduction of thyroid stimulating hormone and a balance in the lipid status. Individualized classical homeopathy may have a role in the climacteric syndrome and comorbidities after surgical menopause.)
  2. Therapeutic evaluation of homeopathic treatment for canine oral papillomatosis (The current study proves that the combination of homeopathy drugs aids in fastening the regression of canine oral papilloma and proved to be safe and cost-effective.)
  3. Deep vein thrombosis cured by homeopathy: A case report (The present case report intends to record yet another case of DVT in an old patient totally cured exclusively by the non-invasive method of treatment with micro doses of potentized homeopathic drugs selected on the basis of the totality of symptoms and individualization of the case.)
  4. Diabetic retinopathy screening uptake after health education with or without retinal imaging within the facility in two AYUSH hospitals in Hyderabad, India: A nonrandomized pilot study (AYUSH hospitals could provide a feasible and acceptable location for providing DR screening services.)
  5. Individualised Homeopathic Therapy in ANCA Negative Rapidly Progressive Necrotising Crescentic Glomerulonephritis with Severe Renal Insufficiency – A Case Report (A 60-year-old Indian woman was treated with classical homeopathy for ANCA-negative RPGN, and after one year of treatment, serum creatinine and other parameters indicating renal injury dropped steadily despite the withdrawal of immunosuppressive drugs; renal dialysis, which was conducted twice a week initially, was made rarer and stopped after one year. Classical homeopathy may be considered a potential therapeutic modality in severe pathologies.)
  6. Improvements in long standing cardiac pathologies by individualized homeopathic remedies: A case series (… individualized homeopathic therapy was instituted along with the conventional medicines and the results were encouraging. The changes in the laboratory diagnostic parameters (single-photon emission computed tomography, electrocardiograph, echocardiography and ejection fraction as the case may be) are demonstrated over time. The key result seen in all three cases was the preservation of general well-being while the haemodynamic states also improved.)
  7. Could Homeopathy Become An Alternative Therapy In Dengue Fever? An example Of 10 Case Studies (We present a retrospective case series of 10 Indian patients who were diagnosed with dengue fever and treated exclusively with homeopathic remedies at Bangalore, India. This case series demonstrates with evidence of laboratory reports that even when the platelets dropped considerably there was good result without resorting to any other means.)
  8. Homeopathic Treatment of Vitiligo: A Report of Fourteen Cases (In 14 patients with vitiligo treated with individualized homeopathy, the best results were achieved in the patients who were treated in the early stages of the disease. We believe that homeopathy may be effective in the early stages of vitiligo, but large controlled clinical studies are needed in this area.)
  9. An Exploratory Study of Autonomic Function Investigations in Hemophiliacs on Homoeopathy Medications Using Impedance Plethysmography (Homoeopathic medicines used as an adjunct was associated with decrease in parasympathetic modulations.)
  10. Embryonic Zebrafish Model – A Well-Established Method for Rapidly Assessing the Toxicity of Homeopathic Drugs: – Toxicity Evaluation of Homeopathic Drugs Using Zebrafish Embryo Model (Our findings clearly demonstrate that no toxic effects were observed for these three homeopathic drugs at the potencies and exposure times used in this study. The embryonic zebrafish model is recommended as a well-established method for rapidly assessing the toxicity of homeopathic drugs.)
  11. Treatment of hemorrhoids with individualized homeopathy: An open observational pilot study (Under classical homeopathic treatment, hemorrhoids patients improved considerably in symptoms severity and anoscopic scores. However, being observational trial, our study cannot provide efficacy data. Controlled studies are required.)
  12. Patients’ preference for integrating homeopathy (PPIH) within the standard therapy settings in West Bengal, India: The part 1 (PPIH-1) study (A favorable attitude toward integrating homeopathy into conventional healthcare settings was obtained among the patients attending the homeopathic hospitals in West Bengal, India.)
  13. Patients’ Preference for Integrating Homoeopathy Services within the Secondary Health Care Settings in India: The Part 3 (PPIH-3) Study (A total of 82.40% (95% confidence interval = 79.23, 85.19) of the participants were in favor of integrating homoeopathy services.)
  14. Obstetrics and gynecology outpatient scenario of an Indian homeopathic hospital: A prospective, research-targeted study (The most frequently treated conditions were leucorrhea (20.5%), irregular menses (13.3%), dysmenorrhea (10%), menorrhagia (7.5%), and hypomenorrhea (6.3%). Strongly positive outcomes (+3/+2) were mostly recorded in oligomenorrhea (41.7%), leucorrhea (34.1%), polycystic ovary (33.3%), dysmenorrhea (28%), and irregular menses (22.2%). Individualized prescriptions predominated (95.6%).)
  15. Relative Apoptosis-inducing Potential of Homeopa-thic Condurango 6C and 30C in H460 Lung Cancer Cells In vitro: -Apoptosis-induction by homeopathic Condurango in H460 cells (Condurango 30C had greater apoptotic effect than Condurango 6C as claimed in the homeopathic doctrine.)
  16. Beliefs, attitudes and self-use of Ayurveda, Yoga and Naturopathy, Unani, Siddha, and Homeopathy medicines among senior pharmacy students: An exploratory insight from Andhra Pradesh, India (Pharmacy students held favorable attitude and beliefs about AYUSH use.)
  17. Integrative nanomedicine: treating cancer with nanoscale natural products (Taken together, the nanoparticulate research data and the Banerji Protocols for homeopathic remedies in cancer suggest a way forward for generating advances in cancer treatment with natural product-derived nanomedicines.)
  18. Evidence of an Epigenetic Modification in Cell-cycle Arrest Caused by the Use of Ultra-highly-diluted Gonolobus Condurango Extract (Condurango 30C appeared to trigger key epigenetic events of gene modulation in effectively combating cancer cells, which the placebo was unable to do.)
  19. Calcarea carbonica induces apoptosis in cancer cells in p53-dependent manner via an immuno-modulatory circuit (Our results indicated a “two-step” mechanism of the induction of apoptosis in tumor cells by calcarea carbonica)
  20. Post-cancer Treatment with Condurango 30C Shows Amelioration of Benzo[a]pyrene-induced Lung Cancer in Rats Through the Molecular Pathway of Caspa- se-3-mediated Apoptosis Induction: -Anti-lung cancer potential of Condurango 30C in rats (The overall result validated a positive effect of Condurango 30C in ameliorating lung cancer through caspase-3-mediated apoptosis induction and EGFR down-regulation.)
  21. The potentized homeopathic drug, Lycopodium clavatum (5C and 15C) has anti-cancer effect on hela cells in vitro (Thus, the highly-diluted, dynamized homeopathic remedies LC-5C and LC-15C demonstrated their capabilities to induce apoptosis in cancer cells, signifying their possible use as supportive medicines in cancer therapy.)
  22. Ameliorating effect of mother tincture of Syzygium jambolanum on carbohydrate and lipid metabolic disorders in streptozotocin-induced diabetic rat: Homeopathic remedy (The result of the present study indicated that the homeopathic drug S jambolanum (mother tincture) has a protective effect on diabetic induced carbohydrate and lipid metabolic disorders in STZ-induced diabetic animal.)
  23. SEM studies on blood cells of Plasmodium berghei infected Balb/c mice treated with artesunate and homeopathic medicine China (The combination of artesunate and China was found to be very effective and did not cause any alteration on the surface of blood cells as observed in SEM.)
  24. Induction of apoptosis of tumor cells by some potentiated homeopathic drugs: implications on mechanism of action (These data indicate that apoptosis is one of the mechanisms of tumor reduction of homeopathic drugs.)
  25. TDZ-induced high frequency shoot regeneration in Cassia sophera Linn. via cotyledonary node explants (Regenerated plantlets were successfully acclimatized and hardened off inside the culture room and then transferred to green house with 100 % survival rate.)
  26. Modulation of Signal Proteins: A Plausible Mechanism to Explain How a Potentized Drug Secale Cor 30C Diluted beyond Avogadro’s Limit Combats Skin Papilloma in Mice (We tested the hypothesis if suitable modulations of signal proteins could be one of the possible pathways of action of a highly diluted homeopathic drug, Secale cornutum 30C (diluted 10(60) times; Sec cor 30). It could successfully combat DMBA + croton oil-induced skin papilloma in mice as evidenced by histological, cytogenetical, immunofluorescence, ELISA and immunoblot findings.)
  27. Can homeopathy bring additional benefits to thalassemic patients on hydroxyurea therapy? Encouraging results of a preliminary study (The homeopathic remedies being inexpensive and without any known side-effects seem to have great potentials in bringing additional benefits to thalassemic patients; particularly in the developing world where blood transfusions suffer from inadequate screening and fall short of the stringent safety standards followed in the developed countries.)
  28. Effect of homeopathic medicines on transplanted tumors in mice (These findings support that homeopathic preparations of Ruta and Hydrastis have significant antitumour activity. The mechanism of action of these medicines is not known at present.)
  29. Inhibition of chemically induced carcinogenesis by drugs used in homeopathic medicine (These studies demonstrate that homeopathic drugs, at ultra low doses, may be able to decrease tumor induction by carcinogen administration.)
  30. Can homeopathic treatment slow prostate cancer growth? (The findings indicate that selected homeopathic remedies for the present study have no direct cellular anticancer effects but appear to significantly slow the progression of cancer and reduce cancer incidence and mortality in Copenhagen rats injected with MAT-LyLu prostate cancer cells.)
  31. Ameliorating effect of microdoses of a potentized homeopathic drug, Arsenicum Album, on arsenic-induced toxicity in mice (The results lend further support to our earlier views that microdoses of potentized Arsenicum Album are capable of combating arsenic intoxication in mice, and thus are strong candidates for possible use in human subjects in arsenic contaminated areas under medical supervision.)

So, 31 of 31 yield positive results and conclusions – 100%!

When I suggested that Indian research into homeopathy is suspect, I was merely speculating on the basis of reading such papers for many years. I had not seen a systematic analysis to justify my harsh judgment; in fact, I don’t think that such a review is currently available (which would make this post the first of its kind). I had no idea how true my seemingly disrespectful remark would turn out to be. There is not one paper from India that does not suggest positive findings for homeopathy. I find this truly remarkable!

You can, of course, interpret my findings in two very different ways:

  • Either you assume that homeopathy is hugely effective and works always and for everything under every experimental condition.
  • Or you conclude that Indian research into homeopathy is suspect and far from trustworthy.

If you believe the first option to be true, I fear that you must be as deluded as homeopathic remedies are diluted.

The state of acupuncture research has long puzzled me. The first thing that would strike who looks at it is its phenomenal increase:

  • Until around the year 2000, Medline listed about 200 papers per year on the subject.
  • From 2005, there was a steep, near-linear increase.
  • It peaked in 2020 when we had a record-breaking 20515 acupuncture papers currently listed in Medline.

Which this amount of research, one would expect to get somewhere. In particular, one would hope to slowly know whether acupuncture works and, if so, for which conditions. But this is not the case.

On the contrary, the acupuncture literature is a complete mess in which it gets more and more difficult to differentiate the reliable from the unreliable, the useful from the redundant, and the truth from the lies. Because of this profound confusion, acupuncture fans are able to claim that their pet-therapy is demonstrably effective for a wide range of conditions, while skeptics insist it is a theatrical placebo. The consumer might listen in bewilderment.

Yesterday (18/1/2021), I had a quick (actually, it was not that quick after all) look into what Medline currently lists in terms of new acupuncture research published in 2021 and found a few other things that are remarkable:

  1. There were already 100 papers dated 2021 (today, there were even 118); that corresponds to about 5 new articles per day and makes acupuncture one of the most research-active areas of so-called alternative medicine (SCAM).
  2. Of these 100 papers, only 7 were clinical trials (CTs). In my view, clinical trials would be more important than any other type of research on acupuncture. To see that they amount to just 7% of the total is therefore disappointing.
  3. Twelve papers were systematic reviews (SRs). It is odd, I find, to see almost twice the amount of SRs than CTs.
  4. Eighteen papers referred to protocols of studies of SRs. In particular protocols of SRs are useless in my view. It seems to me that the explanation for this plethora of published protocols might be the fact that Chinese researchers are extremely keen to get papers into Western journals; it is an essential boost to their careers.
  5. Seven papers were surveys. This multitude of survey research is typical for all types of SCAM.
  6. Twenty-four articles were on basic research. I find basic research into an ancient therapy of questionable clinical use more than a bit strange.
  7. The rest of the articles were other types of publications and a few were misclassified.
  8. The vast majority (n = 81) of the 100 papers were authored exclusively by Chinese researchers (and a few Korean). In view of the fact that it has been shown repeatedly that practically all acupuncture studies from China report positive results and that data fabrication seems rife in China, this dominance of China could be concerning indeed.

Yes, I find all this quite concerning. I feel that we are swamped with plenty of pseudo-research on acupuncture that is of doubtful (in many cases very doubtful) reliability. Eventually, this will create an overall picture for the public that is misleading to the extreme (to check the validity of the original research is a monster task and way beyond what even an interested layperson can do).

And what might be the solution? I am not sure I have one. But for starters, I think, that journal editors should get a lot more discerning when it comes to article submissions from (Chinese) acupuncture researchers. My advice to them and everyone else:

if it sounds too good to be true, it probably is!

I was criticised for not referencing this article in a recent post on adverse effects of spinal manipulation. In fact the commentator wrote: Shame on you Prof. Ernst. You get an “E” for effort and I hope you can do better next time. The paper was published in a third-class journal, but I will nevertheless quote the ‘key messages’ from this paper, because they are in many ways remarkable.

  • Adverse events from manual therapy are few, mild, and transient. Common AEs include local tenderness, tiredness, and headache. Other moderate and severe adverse events (AEs) are rare, while serious AEs are very rare.
  • Serious AEs can include spinal cord injuries with severe neurological consequences and cervical artery dissection (CAD), but the rarity of such events makes the provision of epidemiological evidence challenging.
  • Sports-related practice is often time sensitive; thus, the manual therapist needs to be aware of common and rare AEs specifically associated with spinal manipulative therapy (SMT) to fully evaluate the risk-benefit ratio.

The author of this paper is Aleksander Chaibi, PT, DC, PhD who holds several positions in the Norwegian Chiropractors’ Association, and currently holds a position as an expert advisor in the field of biomedical brain research for the Brain Foundation of the Netherlands. I feel that he might benefit from reading some more critical texts on the subject. In fact, I recommend my own 2020 book. Here are a few passages dealing with the safety of SMT:

Relatively minor AEs after SMT are extremely common. Our own systematic review of 2002 found that they occur in approximately half of all patients receiving SMT. A more recent study of 771 Finish patients having chiropractic SMT showed an even higher rate; AEs were reported in 81% of women and 66% of men, and a total of 178 AEs were rated as moderate to severe. Two further studies reported that such AEs occur in 61% and 30% of patients. Local or radiating pain, headache, and tiredness are the most frequent adverse effects…

A 2017 systematic review identified the characteristics of AEs occurring after cervical spinal manipulation or cervical mobilization. A total of 227 cases were found; 66% of them had been treated by chiropractors. Manipulation was reported in 95% of the cases, and neck pain was the most frequent indication for the treatment. Cervical arterial dissection (CAD) was reported in 57%, and 46% had immediate onset symptoms. The authors of this review concluded that there seems to be under-reporting of cases. Further research should focus on a more uniform and complete registration of AEs using standardized terminology…

In 2005, I published a systematic review of ophthalmic AEs after SMT. At the time, there were 14 published case reports. Clinical symptoms and signs included:

  • central retinal artery occlusion,
  • nystagmus,
  • Wallenberg syndrome,
  • ptosis,
  • loss of vision,
  • ophthalmoplegia,
  • diplopia,
  • Horner’s syndrome…

Vascular accidents are the most frequent serious AEs after chiropractic SMT, but they are certainly not the only complications that have been reported. Other AEs include:

  • atlantoaxial dislocation,
  • cauda equina syndrome,
  • cervical radiculopathy,
  • diaphragmatic paralysis,
  • disrupted fracture healing,
  • dural sleeve injury,
  • haematoma,
  • haematothorax,
  • haemorrhagic cysts,
  • muscle abscess,
  • muscle abscess,
  • myelopathy,
  • neurologic compromise,
  • oesophageal rupture
  • pneumothorax,
  • pseudoaneurysm,
  • soft tissue trauma,
  • spinal cord injury,
  • vertebral disc herniation,
  • vertebral fracture…

In 2010, I reviewed all the reports of deaths after chiropractic treatments published in the medical literature. My article covered 26 fatalities but it is important to stress that many more might have remained unpublished. The cause usually was a vascular accident involving the dissection of a vertebral artery (see above). The review also makes the following important points:

  • … numerous deaths have been associated with chiropractic. Usually high-velocity, short-lever thrusts of the upper spine with rotation are implicated. They are believed to cause vertebral arterial dissection in predisposed individuals which, in turn, can lead to a chain of events including stroke and death. Many chiropractors claim that, because arterial dissection can also occur spontaneously, causality between the chiropractic intervention and arterial dissection is not proven. However, when carefully evaluating the known facts, one does arrive at the conclusion that causality is at least likely. Even if it were merely a remote possibility, the precautionary principle in healthcare would mean that neck manipulations should be considered unsafe until proven otherwise. Moreover, there is no good evidence for assuming that neck manipulation is an effective therapy for any medical condition. Thus, the risk-benefit balance for chiropractic neck manipulation fails to be positive.
  • Reliable estimates of the frequency of vascular accidents are prevented by the fact that underreporting is known to be substantial. In a survey of UK neurologists, for instance, under-reporting of serious complications was 100%. Those cases which are published often turn out to be incomplete. Of 40 case reports of serious adverse effects associated with spinal manipulation, nine failed to provide any information about the clinical outcome. Incomplete reporting of outcomes might therefore further increase the true number of fatalities.
  • This review is focussed on deaths after chiropractic, yet neck manipulations are, of course, used by other healthcare professionals as well. The reason for this focus is simple: chiropractors are more frequently associated with serious manipulation-related adverse effects than osteopaths, physiotherapists, doctors or other professionals. Of the 40 cases of serious adverse effects mentioned above, 28 can be traced back to a chiropractor and none to a osteopath. A review of complications after spinal manipulations by any type of healthcare professional included three deaths related to osteopaths, nine to medical practitioners, none to a physiotherapist, one to a naturopath and 17 to chiropractors. This article also summarised a total of 265 vascular accidents of which 142 were linked to chiropractors. Another review of complications after neck manipulations published by 1997 included 177 vascular accidents, 32 of which were fatal. The vast majority of these cases were associated with chiropractic and none with physiotherapy. The most obvious explanation for the dominance of chiropractic is that chiropractors routinely employ high-velocity, short-lever thrusts on the upper spine with a rotational element, while the other healthcare professionals use them much more sparingly.

Another review summarised published cases of injuries associated with cervical manipulation in China. A total of 156 cases were found. They included the following problems:

  • syncope (45 cases),
  • mild spinal cord injury or compression (34 cases),
  • nerve root injury (24 cases),
  • ineffective treatment/symptom increased (11 cases),
  • cervical spine fracture (11 cases),
  • dislocation or semi-luxation (6 cases),
  • soft tissue injury (3 cases),
  • serious accident (22 cases) including paralysis, deaths and cerebrovascular accidents.

Manipulation including rotation was involved in 42% of all cases. In total, 5 patients died…

To sum up … chiropractic SMT can cause a wide range of very serious complications which occasionally can even be fatal. As there is no AE reporting system of such events, we nobody can be sure how frequently they occur.

[references from my text can be found in the book]

This is an analysis that I have long hesitated to conduct. The reason for my hesitation is simple: some people might think it is vindictive, revengeful or ad hominem. After reflecting about it for years, I have now decided to go ahead with it (sorry, it’s a bit lengthy). This case study is not meant to be vindictive, but offers an important insight into the power of conflicts of interest in SCAM that are not financial but ideological. I think it is crucial that people are aware of and consider such conflicts carefully, and I can’t see how else I might demonstrate my point so plainly.

Dr Adrian White was a co-worker of mine for about 10 years. He became a trusted colleague, my ‘right hand’ man and even my deputy at my Exeter department. When I discovered that my trust had been misplaced, I did not prolong his contract (I will not dwell on this episode, those who are interested find it in my memoir). Adrian then got a senior research fellowship with Prof John Campbell (not my favourite colleague at Exeter) at the department of general practice where he continued his research on acupuncture for about 10 more years largely unsupervised.

Adrian had been an acupuncturist body and soul (in fact, I had never before met anyone so utterly convinced of the value of this therapy). When he joined my team, he was scientifically naive, and we spent many month trying to teach him how to think like a scientist. Initially, he found it very difficult to think critically about acupuncture. Later, I thought the problem was under control. Yet, most of his research in my department was guided by me and tightly supervised (i.e. I made sure that out studies were testing rather than promoting SCAM, and that our reviews were critical assessments of the existing evidence).

Thus there exist two separate and well-documented periods of a pro-acupuncture researcher:

  • 10 years guided by me and members of my team;
  • 10 years largely unsupervised.

What could be more tempting than to compare Adrian’s output during these two periods?

To do this, I looked up all of Adrian’s 120 publications on acupuncture and selected those 52 articles that generated factual new data (mostly clinical trials or systematic reviews). As it happens, they are numerically distributed almost equally within the two periods. The endpoints for my analysis were the directions of the conclusions of his papers. I therefore extracted, dated, and rated the 52 articles as follows:

  • P = positive from the point of view of an acupuncture advocate,
  • N = negative from the point of view of an acupuncture advocate.
  • P/N = not clearly pointing in either direction.

To render this exercise transparent (occasionally, I was not entirely sure about my ratings), I copied all the 52 conclusions and provided links to the original papers so that anyone inferested is able to check easily.

Here are my findings. Articles 1 – 27 were published AFTER Adrian had left my department; articles 28 – 52 are his papers from the time while he worked with me.

  1. A definitive three-arm trial is feasible. Further follow-up reminders, minimum data collection and incentives should be considered to improve participant retention in the follow-up processes in the standardised advice and exercise booklet arm. (2016) P/N
  2. The available evidence suggests that adding acupuncture to symptomatic treatment of attacks reduces the frequency of headaches. Contrary to the previous findings, the updated evidence also suggests that there is an effect over sham, but this effect is small. The available trials also suggest that acupuncture may be at least similarly effective as treatment with prophylactic drugs. Acupuncture can be considered a treatment option for patients willing to undergo this treatment. As for other migraine treatments, long-term studies, more than one year in duration, are lacking. (2016) P
  3. The available results suggest that acupuncture is effective for treating frequent episodic or chronic tension-type headaches, but further trials – particularly comparing acupuncture with other treatment options – are needed. (2016) P
  4. Acupuncture during pregnancy appears to be associated with few AEs when correctly applied. (2014) P
  5. Although pooled estimates suggest possible short-term effects there is no consistent, bias-free evidence that acupuncture, acupressure, or laser therapy have a sustained benefit on smoking cessation for six months or more. However, lack of evidence and methodological problems mean that no firm conclusions can be drawn. Electrostimulation is not effective for smoking cessation. Well-designed research into acupuncture, acupressure and laser stimulation is justified since these are popular interventions and safe when correctly applied, though these interventions alone are likely to be less effective than evidence-based interventions. (2014) P
  6. The current evidence suggests that acupuncture may have some effects on drug dependence that have been missed because of choice of outcome in many previous studies, and future studies should use outcomes suggested by clinical experience. Body points and electroacupuncture, used in the original clinical observation, justify further research. (2013) P
  7. Acceptability is very high and may be maximised by taking a number of factors into account: full information should be provided before treatment begins; flexibility should be maintained in the appointment system and different levels of contact between fellow patients should be fostered; sufficient space and staffing should be provided and single-sex groups used wherever possible. (2012) P
  8. This is the first evaluation of nurse-led group (multibed) acupuncture clinics for patients with knee osteoarthritis to include a 2 year follow-up. It shows the practicability of offering a low-cost acupuncture service as an alternative to knee surgery and the service’s success in providing long-term symptom relief in about a third of patients. Using realistic assumptions, the cost consequences for the local commissioning group are an estimated saving of £100 000 a year. Sensitivity analyses are presented using different assumptions. (2012) P
  9. There is no consistent, bias-free evidence that acupuncture, acupressure, laser therapy or electrostimulation are effective for smoking cessation, but lack of evidence and methodological problems mean that no firm conclusions can be drawn. Further, well designed research into acupuncture, acupressure and laser stimulation is justified since these are popular interventions and safe when correctly applied, though these interventions alone are likely to be less effective than evidence-based interventions. (2011) P/N
  10. Eight (8) of 10 international acupuncture experts were able to reach consensus on the syndromes, symptoms, and treatment of postmenopausal women with hot flashes. The syndromes were similar to those used by practitioners in the ACUFLASH clinical trial, but there were considerable differences between the acupuncture points. This difference is likely to be the result of differences in approach of training schools, and whether it is relevant for clinical outcomes is not well understood. (2011) P
  11. 70% of those patients eligible to participate volunteered to do so; all participants had clinically identified MTrPs; a 100% completion rate was achieved for recorded self-assessment data; no serious adverse events were reported as a result of either intervention; and the end of treatment attrition rate was 17%. A phase III study is both feasible and clinically relevant. This study is currently being planned. (2010) P
  12. In conclusion, the results from all studies are in agreement with the hypothesis that acupuncture needling relieves hot flushes. There are few data however supporting the hypothesis that the effect of acupuncture is point specific. Future research should investigate whether there is a biological effect of needling on hot flushes or not, whether tailored treatment is superior to standardised treatment, and ways of delivering treatment that causes least discomfort and least cost. (2010) P
  13. Acupuncture can contribute to a more rapid reduction in vasomotor symptoms and increase in health-related quality of life in postmenopausal women but probably has no long-term effects. (2010) P
  14. within the context of this pilot study, the sham acupuncture intervention was found to be a credible control for acupuncture. This supports its use in a planned, definitive, randomised controlled trial on a similar whiplash injured population. (2009) N/P
  15. factors other than the TCM syndrome diagnoses and the point selection may be of importance regarding the outcome of the treatment. (2009) N/P
  16. Acupuncture plus self-care can contribute to a clinically relevant reduction in hot flashes and increased health-related quality of life in postmenopausal women. (2009) P
  17. the authors conclude that acupuncture could be a valuable non-pharmacological tool in patients with frequent episodic or chronic tension-type headaches. (2009) P
  18. there is consistent evidence that acupuncture provides additional benefit to treatment of acute migraine attacks only or to routine care. There is no evidence for an effect of ‘true’ acupuncture over sham interventions, though this is difficult to interpret, as exact point location could be of limited importance. Available studies suggest that acupuncture is at least as effective as, or possibly more effective than, prophylactic drug treatment, and has fewer adverse effects. Acupuncture should be considered a treatment option for patients willing to undergo this treatment. (2009) P
  19. We have conducted the first survey of the effects of provision of acupuncture in UK general practice, using data provided by the NHS, and uncovered a wide variation in the availability of the service in different areas. We have been unable to demonstrate any consistent differences in the prescribing or referral rates that could be due to the use of acupuncture in these practices. The wide variation in the data means that if such a trend exists, a very large survey would be needed to identify it. However, we discovered inaccuracies and variations in presentation of data by the PCTs which have made the numerical input, and hence our results, unreliable. Thus the practicalities of access to data and the problems with data accuracy would preclude a nationwide survey. (2008) P
  20. In conclusion, there is limited evidence deriving from one study that deep needling directly into myofascial trigger points has an overall treatment effect when compared with standardised care. Whilst the result of the meta-analysis of needling compared with placebo controls does not attain statistically significant, the overall direction could be compatible with a treatment effect of dry needling on myofascial trigger point pain. However, the limited sample size and poor quality of these studies highlights and supports the need for large scale, good quality placebo controlled trials in this area. (2009) P
  21. We conclude that limited evidence supports acupuncture use in treating pregnancy-related pelvic and back pain. Additional high-quality trials are needed to test the existing promising evidence for this relatively safe and popular complementary therapy. (2008) P
  22. Acupuncture appears to offer symptomatic improvement to some patients with fibromyalgia in a tertiary clinic who have failed to respond to other treatments. In view of its safety, further acupuncture research is justified in this population. (2007) P
  23. It is speculated that optimal results from acupuncture treatment for osteoarthritis of the knee may involve: climatic factors, particularly high temperature; high expectations of patients; minimum of four needles; electroacupuncture rather than manual acupuncture, and particularly, strong electrical stimulation to needles placed in muscle; and a course of at least 10 treatments. These factors offer some support to criteria for adequate acupuncture used in the recent review. In addition, ethnic and cultural factors may influence patients’ reporting of their symptoms, and different versions of an outcome measure are likely to differ in their sensitivity – both factors which may lead to apparent rather than real differences between studies. The many variables in a study are likely to be more tightly controlled in a single centre study than in multicentre studies.  (2007) P
  24. Any effects of acupressure on smoking withdrawal, as an adjunct to the use of NRT and behavioural intervention, are unlikely to be detectable by the methods used here and further preliminary studies are required before the hypothesis can be tested. (2007) P
  25. Auricular acupuncture appears to be effective for smoking cessation, but the effect may not depend on point location. This calls into question the somatotopic model underlying auricular acupuncture and suggests a need to re-evaluate sham controlled studies which have used ‘incorrect’ points. Further experiments are necessary to confirm or refute these observational conclusions. (2006) P
  26. Acupuncture that meets criteria for adequate treatment is significantly superior to sham acupuncture and to no additional intervention in improving pain and function in patients with chronic knee pain. Due to the heterogeneity in the results, however, further research is required to confirm these findings and provide more information on long-term effects. (2007) P
  27. There is no consistent evidence that acupuncture, acupressure, laser therapy or electrostimulation are effective for smoking cessation, but methodological problems mean that no firm conclusions can be drawn. Further research using frequent or continuous stimulation is justified. (2006) N/P
  28. Acupuncture is not superior to sham treatment for recovery in activities of daily living and health-related quality of life after stroke, although there may be a limited effect on leg function in more severely affected patients.  (2005) N
  29. The evidence from controlled trials is insufficient to conclude whether acupuncture is an effective treatment for depression, but justifies further trials of electroacupuncture. (2005) N
  30. Acupuncture effectively relieves chronic low back pain. No evidence suggests that acupuncture is more effective than other active therapies. (2005) N/P
  31. In view of the small number of studies and their variable quality, doubt remains about the effectiveness of acupuncture for gynaecological conditions. Acupuncture and acupressure appear promising for dysmenorrhoea, and acupuncture for infertility, and further studies are justified. (2003) N
  32.  In conclusion, the results suggest that the procedure using the new device is indistinguishable from the same procedure using real needles in acupuncture naïve subjects, and is inactive, where the specific needle sensation (de qi) is taken as a surrogate measure of activity. It is therefore a valid control for acupuncture trials. The findings also lend support to the existence of de qi, a major concept underlying traditional Chinese acupuncture. (2002) N/P
  33. There is no clear evidence that acupuncture, acupressure, laser therapy or electrostimulation are effective for smoking cessation. (2002) N
  34. Collectively, these data imply that acupuncture is superior to various control interventions, although there is insufficient evidence to state whether it is superior to placebo. (2002) N/P
  35. In conclusion, the incidence of adverse events following acupuncture performed by doctors and physiotherapists can be classified as minimal; some avoidable events do occur. Acupuncture seems, in skilled hands, one of the safer forms of medical intervention. (2001) N/P
  36. Based on the evidence of rigorous randomised controlled trials, there is no compelling evidence to show that acupuncture is effective in stroke rehabilitation. Further, better-designed studies are warranted. (2001) N
  37. Although it has already been demonstrated that severe adverse events seem to be uncommon in standard practice, many serious cases of negligence have been found in the present review, suggesting that training system for acupuncturists (including medical doctors) should be improved and that unsupervised self-treatment should be discouraged. (2001) N
  38. Direct needling of myofascial trigger points appears to be an effective treatment, but the hypothesis that needling therapies have efficacy beyond placebo is neither supported nor refuted by the evidence from clinical trials. Any effect of these therapies is likely because of the needle or placebo rather than the injection of either saline or active drug. Controlled trials are needed to investigate whether needling has an effect beyond placebo on myofascial trigger point pain. (2001) N/P
  39. Although the incidence of minor adverse events associated with acupuncture may be considerable, serious adverse events are rare. Those responsible for establishing competence in acupuncture should consider how to reduce these risks. (2001) N
  40. In conclusion, this study does not provide evidence that this form of acupuncture is effective in the prevention of episodic tension-type headache. (2000) N
  41. The present study provides no strong evidence to support the hypothesis that the acupuncture point SP6 is more tender in women and in men. Recommendations for further investigations are discussed.  (2000) N
  42. Acupuncture has not been demonstrated to be efficacious as a treatment for tinnitus on the evidence of rigorous randomized controlled trials. (2000) N
  43. We conclude that acupuncture continues to be associated with occasional, serious adverse events and fatalities. These events have no geographical limits. Most of these events are due to negligence. Everyone concerned with setting standards, delivering training, and maintaining competence in acupuncture should familiarise themselves with the lessons to be learnt from these untoward events. (2000) N
  44. Overall, the existing evidence suggests that acupuncture has a role in the treatment of recurrent headaches. However, the quality and amount of evidence is not fully convincing. There is urgent need for well-planned, large-scale studies to assess effectiveness and efficiency of acupuncture under real life conditions. (1999) N/P
  45. While the frequency of adverse effects of acupuncture is unknown and they may be rare, knowledge of normal anatomy and anatomical variations is essential for safe practice and should be reviewed by regulatory bodies and those responsible for training courses. (1999) N
  46.  In conclusion, the hypothesis that acupuncture is efficacious in the treatment of neck pain is not based on the available evidence from sound clinical trials. Further studies are justified. (1999) N
  47. Even though all studies are in accordance with the notion that acupuncture is effective for temporomandibular joint dysfunction, this hypothesis requires confirmation through more rigorous investigations. (1999) N
  48. Acupuncture is not free of risks. All adverse events reported in 1997 would have been avoidable. The absolute number of cases is small, but the degree of underreporting remains unknown. (1999) N
  49. This form of electroacupuncture is no more effective than placebo in reducing nicotine withdrawal symptoms. (1998) N
  50. Acupuncture was shown to be superior to various control interventions, although there is insufficient evidence to state whether it is superior to placebo. (1998) N/P
  51. Considerable variation was observed in the scores awarded by the acupuncture experts. (1998) N
  52. It is therefore concluded that, according to the data published to date, the evidence that acupuncture is a useful adjunct for stroke rehabilitation is encouraging but not compelling. More and better trials are required to clarify this highly relevant issue. (1996) N

The results are remarkable (particularly considering that one would not expect unbiased studies or reviews of acupuncture to generate plenty of positive conclusions):

0 times N, 5 times N/P, 22 times P – after Adrian had left my department,

17 times N, 7 times N/P, 0 times P – while Adrian worked in my department.

From these figures, it is tempting to calculate the ratios for both periods of negative : positive conclusions:

zero versus infinite

If that is not impressive, I don’t know what is!

Looking just at the positive and the negative papers over the years:

One could discuss these papers in more detail, but I think this is hardly necessary. Just a few highlights perhaps: look at articles No 5, 20 and 27 for examples of turning an essentially negative finding into a positive conclusion. Notice that Adrian conducted a clinical trial of acupuncture for smoking cessation (No 49) while working with me and later published uncritical positive reviews on the subject. Does this not indicate that he distrusted his own study because it had not generated the result he had hoped for?

Of course, my analysis is merely a case study and therefore my findings are not generalisable. However, in my personal experience, the described phenomenon is by no means an exception in SCAM research. I have observed similar phenomena over and over again. Just look at the ALTERNATIVE MEDICINE HALL OF FAME that I created for this blog:

But Adrian’s case might be unique because it allows us to make a longitudinal observation over two decades. And it suggests to me that an ideological bias can (and often is) so strong and indistructable that is re-emerges as soon as it is no longer kept under strict control.

I have long suspected that ideological conflicts of interest have a much more powerful influence in SCAM research than financial ones. Such an overpowering influence might even be characteristic to much of SCAM research. And because it can be so dominant, it seems important to know about. People reading research need to be aware that it originates from a biased source, and funders who finance research would be wise to think twice about supporting researchers who are likely to generate findings that are biased and therefore false-positive. In the final analysis, such research is worse than no research at all.

In 2012, we published a systematic review of adverse effects of homeopathy. Here is its abstract:

Aim: The aim of this systematic review was to critically evaluate the evidence regarding the adverse effects (AEs) of homeopathy.

Method: Five electronic databases were searched to identify all relevant case reports and case series.

Results: In total, 38 primary reports met our inclusion criteria. Of those, 30 pertained to direct AEs of homeopathic remedies; and eight were related to AEs caused by the substitution of conventional medicine with homeopathy. The total number of patients who experienced AEs of homeopathy amounted to 1159. Overall, AEs ranged from mild-to-severe and included four fatalities. The most common AEs were allergic reactions and intoxications. Rhus toxidendron was the most frequently implicated homeopathic remedy.

Conclusion: Homeopathy has the potential to harm patients and consumers in both direct and indirect ways. Clinicians should be aware of its risks and advise their patients accordingly.

The paper prompted a number of angry reactions from proponents of homeopathy who claimed, for instance, that homeopathic remedies are highly diluted and thus safe. We responded that homeopaths can nevertheless be dangerous to patients through neglect and bad advice by homeopaths, and that not all homeopathic remedies are highly diluted, and that some might be toxic because of poor quality control of the manufacturing process.

Now, a different group of researchers have looked at the problem from a slightly different angle and with different methodologies. This systematic review and meta-analysis by researchers from NAFKAM focused on observational studies, as a substantial amount of the research base for homeopathy are observational.

Eight electronic databases, central webpages and journals were searched for eligible studies, and a total of 1,169 studies were identified, 41 were included in this review. Eighteen studies were included in a meta-analysis that made an overall comparison between homeopathy and control (conventional medicine and herbs).

Eighty-seven percent (n = 35) of the studies reported adverse effects. They were graded as CTCAE 1, 2 or 3 and equally distributed between the intervention and control groups. Homeopathic aggravations (homeopaths believe that, when the optimal remedy is given, patients will experience an aggravation of their presenting symptoms) were reported in 22,5% (n = 9) of the studies and graded as CTCAE 1 or 2. The frequency of adverse effects for control versus homeopathy was statistically significant (P < 0.0001). Analysis of sub-groups indicated that, compared to homeopathy, the number of adverse effects was significantly higher for conventional medicine (P = 0.0001), as well as other complementary therapies (P = 0.05).

The authors concluded that adverse effects of homeopathic remedies are consistently reported in observational studies, while homeopathic aggravations are less documented. This meta-analysis revealed that the proportion of patients experiencing adverse effects was significantly higher when receiving conventional medicine and herbs, compared to patients receiving homeopathy. Nonetheless, the development and implementation of a standardized reporting system of adverse effects in homeopathic studies is warranted in order to facilitate future risk assessments.

While these results are interesting, they have to be taken with a pinch of salt and beg a number of questions:

  • Is there proof that aggravations exist at all?
  • How can one differentiate them from adverse effects?
  • As even placebos are known to cause adverse effects (nocebo effects), how can one be sure that the adverse effects of homeopathy are not nocebo effects?
  • Is it a good reason to focus on largely inconclusive observational studies, because a substantial amount of the research base for homeopathy are observational?
  • Can one produce conclusive results by meta-analysing inconclusive studies?

For me, the most impressive findings of this review is that in total 86 studies had to be excluded by the authors because they reported no adverse effects or aggravations. I think this renders the interpretation of the evidence from the 41 studies they did include even more flimsy. In fact, I don’t see how any meaningful conclusion can be drawn at all – except of course that many researchers of homeopathy violate the rules of research ethics by not reporting adverse effects in their studies.

As to aggravations, we clearly need to rely on placebo controlled studies, if we want to find out whether they exist at all. This we have done in our 2003 paper:

Homeopathic aggravations have often been described anecdotally. However, few attempts have been made to scientifically verify their existence. This systematic review aimed at comparing the frequency of homeopathic aggravations in the placebo and verum groups of double-blind, randomised clinical trials. Eight independent literature searches were carried out to identify all such trials mentioning either adverse effects or aggravations. All studies thus found were validated and data were extracted by both authors. Twenty-four trials could be included. The average number of aggravations was low. In total, 50 aggravations were attributed to patients treated with placebo and 63 to patients treated with homoeopathically diluted remedies. We conclude that this systematic review does not provide clear evidence that the phenomenon of homeopathic aggravations exists.

What is interesting, from my perspective, is the fact that the NAFKAM authors chose to ignore our 2012 paper completely (even though it is highly relevant to their paper and was not published in an obscure journal) and elected to completely misinterpret the findings of our 2003 paper (stating this about it: Grabia and Ernst reported a total of 103 cases of homeopathic aggravations in 3437 participants (3%), while, in fact, our paper demonstrated that aggravations are a homeopathic figment of imagination).

I wonder why.

In the past, NAFKAM did not have the reputation of doing research that was overtly biased towards homeopathy. Recently, the head of the team retired and was replaced by Miek C. Jong who is a co-author of the present review (plus head of CAMcancer, an organisation of which I am a founding member and which did, I think, some good work in the past). She happens to have a long history as a homeopath or homeopathic researcher and is co-author of many papers in this area. Here are three of her conclusions:

Could it be that, within NAFKAM, the attitude towards homeopathy has changed?

Acupuncture-moxibustion therapy (AMT) is a so-called alternative medicine (SCAM) that has been used for centuries in treatment of numerous diseases. Some enthusiasts even seem to advocate it for chemotherapy-induced leukopenia (CIL)  The purpose of this review was to evaluate the efficacy and safety of acupuncture-moxibustion therapy in treating CIL.

Relevant studies were searched in 9 databases up to September 19, 2020. Two reviewers independently screened the studies for eligibility, extracted data, and assessed the methodological quality of selected studies. Meta-analysis of the pooled mean difference (MD) and risk ratio (RR) with their respective 95% confidence intervals (CI) were calculated.

Seventeen studies (1206 patients) were included, and the overall quality of the included studies was moderate. In comparison with medical therapy, AMT has a better clinical efficacy for CIL (RR, 1.24; 95% CI, 1.17-1.32; P < 0.00001) and presents advantages in increasing leukocyte count (MD, 1.10; 95% CI, 0.67-1.53; P < 0.00001). Also, the statistical results show that AMT performs better in improving the CIL patients’ Karnofsky performance score (MD, 5.92; 95% CI, 3.03-8.81; P < 0.00001).

The authors concluded that this systematic review and meta-analysis provides updated evidence that AMT is a safe and effective alternative for the patients who suffered from CIL.

A CIL is a serious complication. If I ever were afflicted by it, I would swiftly send any acupuncturist approaching my sickbed packing.

But this is not an evidence-based attitude!!!, I hear some TCM-fans mutter. What more do you want that a systematic review showing it works?

I beg to differ. Why? Because the ‘evidence’ is hardly what critical thinkers can accept as evidence. Have a look at the list of the primary studies included in this review:

  1. Lin Z. T., Wang Q., Yu Y. N., Lu J. S. Clinical observation of post-chemotherapy-leukopenia treated with ShenMai injectionon ST36. World Journal of Integrated Traditional and Western Medicine2010;5(10):873–876. []
  2. Wang H. Clinical Observation of Acupoint Moxibustion on Leukopenia Caused by Chemotherapy. Beijing, China: Beijing University of Chinese Medicine; 2011. []
  3. Fan J. Y. Coupling of Yin and Yang between Ginger Moxibustion Improve the Clinical Effect of the Treatment of Chemotherapy Adverse Reaction. Henan, China: Henan University of Chinese Medicine; 2013. []
  4. Lu D. R., Lu D. X., Wei M., et al. Acupoint injection with addie injection for patients of nausea and vomiting with cisplatin induced by chemotherapy. Journal of Clinical Acupuncture and Moxibustion2013;29(10):33–38. []
  5. Yang J. E. The Clinical Observation on Treatment of Leukopenia after Chemotherapy with Needle Warming Moxibustion. Hubei, China: Hubei University of Chinese Medicine; 2013. []
  6. Fu Y. H., Chi C. Y., Zhang C. Y. Clinical effect of acupuncture and moxibustion on leukopenia after chemotherapy of malignant tumor. Guide of China Medicine2014;12(12) []
  7. Wang J. N., Zhang W. X., Gu Q. H., Jiao J. P., Liu L., Wei P. K. Protection of herb-partitioned moxibustion on bone marrow suppression of gastric cancer patients in chemotherapy period. Chinese Archives of Traditional Chinese Medicine2014;32(12):110–113. []
  8. Zhang J. The Clinical Research on Myelosuppression and Quality of Life after Chemotherapy Treated by Grain-Sized Moxibustion. Nanjing, China: Nanjing University of Chinese Medicine; 2014. []
  9. Tian H., Lin H., Zhang L., Fan Z. N., Zhang Z. L. Effective research on treating leukopenia following chemotherapy by moxibustion. Clinical Journal of Chinese Medicine2015;7(10):35–38. []
  10. Hu G. W., Wang J. D., Zhao C. Y. Effect of acupuncture on the first WBC reduction after chemotherapy for breast cancer. Beijing Journal of Traditional Chinese Medicine2016;35(8):777–779. []
  11. Zhu D. L., Lu H. Y., Lu Y. Y., Wu L. J. Clinical observation of Qi-blood-supplementing needling for leukopenia after chemotherapy for breast cancer. Shanghai Journal of Acupuncture and Moxibustion2016;35(8):964–966. []
  12. Chen L, Xu G. Y. Observation on the prevention and treatment of chemotherapy-induced leukopenia by moxibustion therapy. Zhejiang Journal of Traditional Chinese Medicine2016;51(8):p. 600. []
  13. Mo T., Tian H., Yue S. B., Fan Z. N., Zhang Z. L. Clinical observation of acupoint moxibustion on leukocytopenia caused by tumor chemotherapy. World Chinese Medicine2016;11(10):2120–2122. []
  14. Nie C. M. Nursing observation of acupoint moxibustion in the treatment of leucopenia after chemotherapy. Today Nurse2017;4:93–95. []
  15. Wang D. Y. Clinical Research on Post-chemotherapy-leukopenia with Spleen-Kidney Yang Deficiency in Colorectal Cancer Treated with Point-Injection. Yunnan, China: Yunnan University of Chinese Medicine; 2017. []
  16. Gong Y. Q, Zhang M. Q, Zhang B. C. Prevention and treatment of leucocytopenia after chemotherapy in patients with malignant tumor with ginger partitioned moxibustion. Chinese Medicine Modern Distance Education of China2018;16(21):135–137. []
  17. Li Z. C., Lian M. J., Miao F. G. Clinical observation of fuzheng moxibustion combined with wenyang shengbai decoction in the treatment of 80 cases of leukopenia after chemotherapy. Hunan Journal of Traditional Chinese Medicine2019;35(3):64–66. []

Notice anything peculiar?

  • The studies are all from China where data fabrication was reported to be rife.
  • They are mostly unavailable for checking (why the published adds links that go nowhere is beyond me).
  • Many do not look at all like randomised clinical trials (which, according to the authors, was an inclusion criterion).
  • Many do not look as though their primary endpoint was the leukocyte count (which, according to the authors, was another inclusion criterion).

Intriguingly, the authors conclude that AMT is not just effective but also ‘safe’. How do they know? According to their own data extraction table, most studies failed to mention adverse effects. And how exactly is acupuncture supposed to increase my leukocyte count? Here is what the authors offer as a mode of action:

Based on the theory of traditional Chinese medicine (TCM), CIL belongs to the category of consumptive disease, owing to the exhaustion of genuine qi in the zang-fu viscera and the insufficiency of kidney essence and qi-blood. Researchers believe that there is an intimate association between the occurrence of malignant tumors and the deficiency of genuine qi. During attacking the cancer cells, chemotherapeutics also damaged the function of zang-fu viscera and qi-blood, leading to CIL. According to the theory of TCM and meridian, acupuncture-moxibustion is an ancient therapeutic modality that may be traced back more than 3500 years in China. Through meridian conduction, acupuncture-moxibustion therapy stimulates acupoints to strengthen the condition of zang-fu viscera and immune function, supporting genuine qi to improve symptoms of consumption.

I think it is high time that we stop tolerating that the medical literature gets polluted with such nonsense (helped, of course, by journals that are beyond the pale) – someone might actually believe it, in which case it would surely hasten the death of vulnerable patients.

Some consumers believe that research is by definition reliable, and patients are even more prone to this error. When they read or hear that ‘RESEARCH HAS SHOWN…’ or that ‘A RECENT STUDY HAS DEMONSTRATED…’, they usually trust the statements that follow. But is this trust in research and researchers justified? During 25 years that I have been involved in so-called alternative medicine (SCAM), I have encountered numerous instances which make me doubt. In this post, I will briefly discuss some the many ways in which consumers can be mislead by apparently sound evidence (for an explanation as to what is and what isn’t evidence, see here).

ABSENCE OF EVIDENCE

I have just finished reading a book by a German Heilpraktiker that is entirely dedicated to SCAM. In it, the author makes hundreds of statements and presents them as evidence-based facts. To many lay people or consumers, this will look convincing, I am sure. Yet, it has one fatal defect: the author fails to offer any real evidence that would back up his statements. The only references provided were those of other books which are equally evidence-free. This popular technique of making unsupported claims allows the author to make assertions without any checks and balances. A lay person is usually unable or unwilling to differentiate such fabulations from evidence, and this technique is thus easy and poular for misleading us about SCAM.

FAKE-EVIDENCE

On this blog, we have encountered this phenomenon ad nauseam: a commentator makes a claim and supports it with some seemingly sound evidence, often from well-respected sources. The few of us who bother to read the referenced articles quickly discover that they do not say what the commentator claimed. This method relies on the reader beeing easily bowled over by some pretend-evidence. As many consumers cannot be bothered to look beyond the smokescreen supplied by such pretenders, the method usually works surprisingly well.

An example: Vidatox is a homeopathic cancer ‘cure’ from Cuba. The Vidatox website clains that it is effective for many cancers. Considering how sensational this claim is, one would expect to find plenty of published articles on Vidatox. However, a Medline search resulted in one paper on the subject. Its authors drew the following conclusion: Our results suggest that the concentration of Vidatox used in the present study has not anti-neoplastic effects and care must be taken in hiring Vidatox in patients with HCC. 

The question one often has to ask is this: where is the line between misleading research and fraud?

SURVEYS

There is no area in healthcare that produces more surveys than SCAM. About 500 surveys are published every year! This ‘survey-mania’ has a purpose: it promotes a positive message about SCAM which hypothesis-testing research rarely does.

For a typical SCAM survey, a team of enthusiastic researchers might put together a few questions and design a questionnaire to find out what percentage of a group of individuals have tried SCAM in the past. Subsequently, the investigators might get one or two hundred responses. They then calculate simple descriptive statistics and demonstrate that xy % use SCAM. This finding eventually gets published in one of the many third-rate SCAM journals. The implication then is that, if SCAM is so popular, it must be good, and if it’s good, the public purse should pay for it. Few consumers would realise that this conclusion is little more that a fallacious appeal to popularity.

AVOIDING THE QUESTION

Another popular way of SCAM researchers to mislead the public is to avoid the research questions that matter. For instance, few experts would deny that one of the most urgent issues in chiropractic relates to the risk of spinal manipulations. One would therefore expect that a sizable proportion of the currently published chiropractic research is dedicated to it. Yet, the opposite is the case. Medline currently lists more than 3 000 papers on ‘chiropractic’, but only 17 on ‘chiropractic, harm’.

PILOT-STUDIES

A pilot study is a small scale preliminary study conducted in order to evaluate feasibility, time, cost, adverse events, and improve upon the study design prior to performance of a full-scale research project. Yet, the elementary preconditions are not fulfilled by the plethora of SCAM pilot studies that are currently being published. True pilot studies of SCAM are, in fact, very rare. The reason for the abundance of pseudo-pilots is obvious: they can easily be interpreted as showing encouragingly positive results for whatever SCAM is being tested. Subsequently, SCAM proponents can mislead the public by claiming that there are plenty of positive studies and therefore their SCAM is supported by sound evidence.

‘SAFE‘ STUDY-DESIGNS

As regularly mentioned on this blog, there are several ways to design a study such that the risk of producing a negative result is minimal. The most popular one in SCAM research is the ‘A+B versus B’ design. In this study, for instance, cancer patients who were suffering from fatigue were randomised to receive usual care or usual care plus regular acupuncture. The researchers then monitored the patients’ experience of fatigue and found that the acupuncture group did better than the control group. The effect was statistically significant, and an editorial in the journal where it was published called this evidence “compelling”. Due to a cleverly over-stated press-release, news spread fast, and the study was celebrated worldwide as a major breakthrough in cancer-care.

Imagine you have an amount of money A and your friend owns the same sum plus another amount B. Who has more money? Simple, it is, of course your friend: A+B will always be more than A [unless B is a negative amount]. For the same reason, such “pragmatic” trials will always generate positive results [unless the treatment in question does actual harm]. Treatment as usual plus acupuncture is more than treatment as usual alone, and the former is therefore more than likely to produce a better result. This will be true, even if acupuncture is a pure placebo – after all, a placebo is more than nothing, and the placebo effect will impact on the outcome, particularly if we are dealing with a highly subjective symptom such as fatigue.

A more obvious method for generating false positive results is to omit blinding. The purpose of blinding the patient, the therapist and the evaluator of the group allocation in clinical trials is to make sure that expectation is not a contributor to the result. Expectation might not move mountains, but it can certainly influence the result of a clinical trial. Patients who hope for a cure regularly do get better, even if the therapy they receive is useless, and therapists as well as evaluators of the outcomes tend to view the results through rose-tinted spectacles, if they have preconceived ideas about the experimental treatment.

Failure to randomise is another source of bias which can mislead us. If we allow patients or trialists to select or chose which patients receive the experimental and which get the control-treatment, it is likely that the two groups differ in a number of variables. Some of these variables might, in turn, impact on the outcome. If, for instance, doctors allocate their patients to the experimental and control groups, they might select those who will respond to the former and those who don’t to the latter. This may not happen with intent but through intuition or instinct: responsible health care professionals want those patients who, in their experience, have the best chances to benefit from a given treatment to receive that treatment. Only randomisation can, when done properly, make sure we are comparing comparable groups of patients. Non-randomisation can easily generate false-positive findings.

It is also possible to mislead people with studies which do not test whether an experimental treatment is superior to another one (often called superiority trials), but which assess whether it is equivalent to a therapy that is generally accepted to be effective. The idea is that, if both treatments produce similarly positive results, both must be effective.  Such trials are called non-superiority or equivalence trials, and they offer a wide range of possibilities for misleading us. If, for example, such a trial has not enough patients, it might show no difference where, in fact, there is one. Let’s consider a simple, hypothetical example: someone comes up with the idea to compare antibiotics to acupuncture as treatments of bacterial pneumonia in elderly patients. The researchers recruit 10 patients for each group, and the results reveal that, in one group, 2 patients died, while, in the other, the number was 3. The statistical tests show that the difference of just one patient is not statistically significant, and the authors therefore conclude that acupuncture is just as good for bacterial infections as antibiotics.

Even trickier is the option to under-dose the treatment given to the control group in an equivalence trial. In the above example, the investigators might subsequently recruit hundreds of patients in an attempt to overcome the criticism of their first study; they then decide to administer a sub-therapeutic dose of the antibiotic in the control group. The results would then seemingly confirm the researchers’ initial finding, namely that acupuncture is as good as the antibiotic for pneumonia. Acupuncturists might then claim that their treatment has been proven in a very large randomised clinical trial to be effective for treating this condition. People who do not happen to know the correct dose of the antibiotic could easily be fooled into believing them.

Obviously, the results would be more impressive, if the control group in an equivalence trial received a therapy which is not just ineffective but actually harmful. In such a scenario, even the most useless SCAM would appear to be effective simply because it is less harmful than the comparator.

A variation of this theme is the plethora of controlled clinical trials in SCAM which compare one unproven therapy to another unproven treatment. Perdicatbly, the results would often indicate that there is no difference in the clinical outcome experienced by the patients in the two groups. Enthusiastic SCAM researchers then tend to conclude that this proves both treatments to be equally effective. The more likely conclusion, however, is that both are equally useless.

Another technique for misleading the public is to draw conclusions which are not supported by the data. Imagine you have generated squarely negative data with a trial of homeopathy. As an enthusiast of homeopathy, you are far from happy with your own findings; in addition you might have a sponsor who puts pressure on you. What can you do? The solution is simple: you only need to highlight at least one positive message in the published article. In the case of homeopathy, you could, for instance, make a major issue about the fact that the treatment was remarkably safe and cheap: not a single patient died, most were very pleased with the treatment which was not even very expensive.

OMISSION

A further popular method for misleading the public is the outright omission findings that SCAM researchers do not like. If the aim is that the public believe the myth that all SCAM is free of side-effects, SCAM researchers only need to omit reporting them in clinical trials. On this blog, I have alerted my readers time and time again to this common phenomenon. We even assessed it in a systematic review. Sixty RCTs of chiropractic were included. Twenty-nine RCTs did not mention adverse effects at all. Sixteen RCTs reported that no adverse effects had occurred. Complete information on incidence, severity, duration, frequency and method of reporting of adverse effects was included in only one RCT.

Most trails have many outcome measures; for instance, a study of acupuncture for pain-control might quantify pain in half a dozen different ways, it might also measure the length of the treatment until pain has subsided, the amount of medication the patients took in addition to receiving acupuncture, the days off work because of pain, the partner’s impression of the patient’s health status, the quality of life of the patient, the frequency of sleep being disrupted by pain etc. If the researchers then evaluate all the results, they are likely to find that one or two of them have changed in the direction they wanted (especially, if they also include half a dozen different time points at which these variables are quatified). This can well be a chance finding: with the typical statistical tests, one in 20 outcome measures would produce a significant result purely by chance. In order to mislead us, the researchers only need to “forget” about all the negative results and focus their publication on the ones which by chance have come out as they had hoped.

FRAUD

When it come to fraud, there is more to chose from than one would have ever wished for. We and others have, for example, shown that Chinese trials of acupuncture hardly ever produce a negative finding. In other words, one does not need to read the paper, one already knows that it is positive – even more extreme: one does not need to conduct the study, one already knows the result before the research has started. This strange phenomenon indicates that something is amiss with Chinese acupuncture research. This suspicion was even confirmed by a team of Chinese scientists. In this systematic review, all randomized controlled trials (RCTs) of acupuncture published in Chinese journals were identified by a team of Chinese scientists. A total of 840 RCTs were found, including 727 RCTs comparing acupuncture with conventional treatment, 51 RCTs with no treatment controls, and 62 RCTs with sham-acupuncture controls. Among theses 840 RCTs, 838 studies (99.8%) reported positive results from primary outcomes and two trials (0.2%) reported negative results. The percentages of RCTs concealment of the information on withdraws or sample size calculations were 43.7%, 5.9%, 4.9%, 9.9%, and 1.7% respectively. The authors concluded that publication bias might be major issue in RCTs on acupuncture published in Chinese journals reported, which is related to high risk of bias. We suggest that all trials should be prospectively registered in international trial registry in future.

A survey of clinical trials in China has revealed fraudulent practice on a massive scale. China’s food and drug regulator carried out a one-year review of clinical trials. They concluded that more than 80 percent of clinical data is “fabricated“. The review evaluated data from 1,622 clinical trial programs of new pharmaceutical drugs awaiting regulator approval for mass production. Officials are now warning that further evidence malpractice could still emerge in the scandal.

I hasten to add that fraud in SCAM research is certainly not confined to China. On this blog, you will find plenty of evidence for this statement, I am sure.

CONCLUSION

Research is obviously necessary, if we want to answer the many open questions in SCAM. But sadly, not all research is reliable and much of SCAM research is misleading. Therefore, it is always necessary to be on the alert and apply all the skills of critical evaluation we can muster.

 

There is some encouraging evidence regarding the positive influence of vitamin D on COVID-19. But is it convincing? Is it causal? As always, it is worth looking at the totality of the reliable evidence.

In this systematic review and meta-analysis, the researchers analyze the association between vitamin D deficiency and COVID-19 severity. They conducted an analysis of the prevalence of vitamin D deficiency and insufficiency in people with the disease. Five online databases—Embase, PubMed, Scopus, Web of Science, ScienceDirect and pre-print Medrevix were searched. The inclusion criteria were observational studies measuring serum vitamin D in adult and elderly subjects with COVID-19. The main outcome was the prevalence of vitamin D deficiency in severe cases of COVID-19.

The researchers identified 1542 articles and 27 met their inclusion criteria. The results show that

  • vitamin D deficiency was not associated with a higher chance of infection by COVID-19,
  • severe cases of COVID-19 present 64% more vitamin D deficiency compared with mild cases,
  • vitamin D concentration insufficiency increased hospitalization and mortality rates,
  • There was a positive association between vitamin D deficiency and the severity of the disease.

The authors concluded that the results of the meta-analysis confirm the high prevalence of vitamin D deficiency in people with COVID-19, especially the elderly. We should add that vitamin D deficiency was not associated with COVID-19 infection. However, we observed a positive association between vitamin D deficiency and the severity of the disease. From this perspective, evaluating blood vitamin D levels could be considered in the clinical practice of health professionals. Moreover, vitamin D supplementation could be considered in patients with vitamin D deficiency and insufficiency, if they have COVID-19. However, there is no support for supplementation among groups with normal blood vitamin D values with the aim of prevention, prophylaxis or reducing the severity of the disease.

These are interesting findings, no doubt. They relate to associations, as the authors repeatedly stress in the text of the paper. They do not, however, signify cause and effect relationships. The principal outcome of this research should be a hypothesis that subsequently needs testing in clinical trials.

So, why on earth did the authors chose that seriously misleading title of their paper? It clearly implies a causal effect; and this can only be verified by conducting clinical trials. One such study has been published (as discussed here) and it concluded that administration of calcifediol may improve the clinical outcome of subjects requiring hospitalization for COVID-19.

My conclusion: it seems well worth conducting more and more rigorous clinical trials.

Many experts doubt that acupuncture generates the many positive health effects that are being claimed by enthusiasts. Yet, few consider that acupuncture might not be merely useless but could even make things worse. Here is a trial that seems to suggest exactly that.

This study evaluated whether combining two so-called alternative medicines (SCAMs), acupuncture and massage, reduce postoperative stress, pain, anxiety, muscle tension, and fatigue more than massage alone.

Patients undergoing autologous tissue breast reconstruction were randomly assigned to one of two postoperative SCAMs for three consecutive days. All participants were observed for up to 3 months. Forty-two participants were recruited from January 29, 2016 to July 11, 2018. Twenty-one participants were randomly assigned to massage alone and 21 to massage and acupuncture. Stress, anxiety, relaxation, nausea, fatigue, pain, and mood (score 0-10) were measured at enrollment before surgery and postoperative days 1, 2, and 3 before and after the intervention. Patient satisfaction was evaluated.

Stress decreased from baseline for both Massage-Only Group and Massage+Acupuncture Group after each treatment intervention. Change in stress score from baseline decreased significantly more in the Massage-Only Group at pretreatment and posttreatment. After adjustment for baseline values, change in fatigue, anxiety, relaxation, nausea, pain, and mood scores did not differ between groups. When patients were asked whether they would recommend the study, 100% (19/19) of Massage-Only Group and 94% (17/18) of Massage+Acupuncture Group responded yes.

The authors concluded tha no additive beneficial effects were observed with addition of acupuncture to massage for pain, anxiety, relaxation, nausea, fatigue, and mood. Combined massage and acupuncture was not as effective in reducing stress as massage alone, although both groups had significant stress reduction. These findings indicate a need for larger studies to explore these therapies further.

I recently went to the supermarket to find out whether combining two bank notes (£10 + £5) can buy more goods than one £10 note alone. What I found was interesting: the former did indeed purchase more than the latter. Because I am a scientist, I did not stop there; I went to a total of 10 shops and my initial finding was confirmed each time: A+B results in more than A alone.

It stands to reason that the same thing happens with clinical trials. We even tested this hypothesis in a systematic review entitled ‘A trial design that generates only ”positive” results‘. Here is our abstract:

In this article, we test the hypothesis that randomized clinical trials of acupuncture for pain with certain design features (A + B versus B) are likely to generate false positive results. Based on electronic searches in six databases, 13 studies were found that met our inclusion criteria. They all suggested that acupuncture is effective (one only showing a positive trend, all others had significant results). We conclude that the ‘A + B versus B’ design is prone to false positive results and discuss the design features that might prevent or exacerbate this problem.

But why is this not so with the above-mentioned study?

Why is, in this instance, A even more that A+B?

There are, of course, several possible answers. To use my supermarket example again, the most obvious one is that B is not a £5 note but a negative amount, a dept note, in other words: A + B can only be less than A alone, if B is a minus number. In the context of the clinical trail, this means acupuncture must have caused a negative effect.

But is that possible? Evidently yes! Many patients don’t like needles and experience stress at the idea of a therapist sticking one into their body. Thus acupuncture would cause stress, and stress would have a negative effect on all the other parameters quantified in the study (pain, anxiety, muscle tension, and fatigue).

My conclusion: in certain situations, acupuncture is more than just useless; it makes things worse.

1 2 3 33
Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted but you must tick the box: “Save my name, email, and website in this browser for the next time I comment.”

The most recent comments from all posts can be seen here.

Archives
Categories