MD, PhD, FMedSci, FSB, FRCP, FRCPEd

bias

Readers of this blog will know that few alternative treatments are more controversial and less plausible than homeopathy. Therefore they might be interested to read about the latest attempt of homeopathy-enthusiasts to convince the public that, despite all the clinical evidence to the contrary, homeopathy does work.

The new article was published in German by Swiss urologist and is a case-report describing a patient suffering from paralytic ileus. This condition is a typical complication of ileocystoplasty of the bladder, the operation the patient had undergone. The patient had also been suffering from a spinal cord injury which, due to a pre-existing neurogenic bowel dysfunction, increases the risk of paralytic ileus.

The paraplegic patient developed a massive paralytic ileus after ileocystoplasty and surgical revision. Conventional stimulation of bowel function was unsuccessful. But after adjunctive homeopathic treatment normalization of bowel function was achieved.

The authors conclude that adjunctive homeopathic therapy is a promising treatment option in patients with complex bowel dysfunction after abdominal surgery who do not adequately respond to conventional treatment.

YES, you did read correctly: homeopathic therapy is a promising treatment

In case anyone doubts that this is more than a trifle too optimistic, let me suggest three much more plausible reasons why the patient’s bowel function finally normalised:

  • It could have been a spontaneous recovery (in most cases, even severe ones, this is what happens).
  • It could have been all the conventional treatments aimed at stimulating bowel function.
  • It could have been a mixture of the two.

The article made me curious, and I checked whether the authors had previously published other material on homeopathy. Thus I found two further articles in a very similar vein:

Article No 2 (dated 2014):

We present the clinical course of a patient with an epididymal abscess caused by multiresistant bacteria. As the patient declined surgical intervention, a conservative approach was induced with intravenous antibiotic treatment. As the clinical findings did not ameliorate, adjunctive homeopathic treatment was used. Under combined treatment, laboratory parameters returned to normal, and the epididymal abscess was rapidly shrinking. After 1 week, merely a subcutaneous liquid structure was detected. Fine-needle aspiration revealed sterile purulent liquid, which was confirmed by microbiological testing when the subcutaneous abscess was drained. Postoperative course was uneventful.

As the risk for recurrent epididymitis is high in persons with spinal cord injury, an organ-preserving approach is justified even in severe cases. Homeopathic treatment was a valuable adjunctive treatment in the above-mentioned case. Therefore, prospective studies are needed to further elucidate the future opportunities and limitations of classical homeopathy in the treatment of urinary tract infections.

Article No 3 (dated 2012):

Recurrent urinary tract infections (UTI) in patients with spinal cord injury are a frequent clinical problem. Often, preventive measures are not successful. We present the case reports of five patients with recurrent UTI who received additional homeopathic treatment. Of these patients, three remained free of UTI, whereas UTI frequency was reduced in two patients. Our initial experience with homeopathic prevention of UTI is encouraging. For an evidence-based evaluation of this concept, prospective studies are required.

It seems clear that all of the three more plausible explanations for the patients’ recovery listed above also apply to these two cases.

One might not be far off speculating that J Pannek, the first author of all these three articles, is a fan of homeopathy (this suspicion is confirmed by a link between him and the HOMEOPATHY RESEARCH INSTITUE: Prof Jürgen Pannek on the use of homeopathy for prophylaxis of UTI’s in patients with neurogenic bladder dysfunction). If that is so, I wonder why he does not conduct a controlled trial, rather than publishing case-report after case-report of apparently successful homeopathic treatments. Does he perhaps fear that his effects might dissolve into thin air under controlled conditions?

Case-reports of this nature can, of course, be interesting and some might even deserve to be published. But it would be imperative to draw the correct conclusions. Looking at the three articles above, I get the impression that, as time goes by, the conclusions of Prof Pannek et al (no, I know nobody from this group of authors personally) are growing more and more firm on less and less safe ground.

In my view, responsible authors should have concluded much more cautiously and reasonably. In the case of the paralytic ileus, for instance, they should not have gone further than stating something like this: adjunctive homeopathic therapy might turn out to be a promising treatment option for such patients. Despite the implausibility of homeopathy, this case-report might deserve to be followed up with a controlled clinical trial. Without such evidence, firm conclusions are clearly not possible.

Blinding patients in clinical trials is a key methodological procedure for minimizing bias and thus making sure that the results are reliable. In alternative medicine, blinding is not always straight forward, and many studies are therefore not patient-blinded. We all know that this can introduce bias into a trial, but how large is its effect on study outcomes?

This was the research question addressed by a recent systematic review of randomized clinical trials with one sub-study (i.e. experimental vs control) involving blinded patients and another, otherwise identical, sub-study involving non-blinded patients. Within each trial, the researchers compared the difference in effect sizes (i.e. standardized mean differences) between the two sub-studies. A difference <0 indicates that non-blinded patients generated a more optimistic effect estimate. The researchers then pooled the differences with random-effects inverse variance meta-analysis, and explored reasons for heterogeneity.

The main analysis included 12 trials with a total of 3869 patients. Ten of these RCTs were studies of acupuncture. The average difference in effect size for patient-reported outcomes was -0.56 (95% confidence interval -0.71 to -0.41), (I(2 )= 60%, P = 0.004), indicating that non-blinded patients exaggerated the effect size by an average of 0.56 standard deviation, but with considerable variation. Two of the 12 trials also used observer-reported outcomes, showing no indication of exaggerated effects due lack of patient blinding.

There was an even larger effect size difference in the 10 acupuncture trials [-0.63 (-0.77 to -0.49)], than in the two non-acupuncture trials [-0.17 (-0.41 to 0.07)]. Lack of patient blinding was also associated with increased attrition rates and the use of co-interventions: ratio of control group attrition risk 1.79 (1.18 to 2.70), and ratio of control group co-intervention risk 1.55 (0.99 to 2.43).

The authors conclude that this study provides empirical evidence of pronounced bias due to lack of patient blinding in complementary/alternative randomized clinical trials with patient-reported outcomes.

This is a timely, rigorous and important analysis. In alternative medicine, we currently see a proliferation of trials that are not patient-blinded. We always suspected that they are at a high risk of generating false-positive results – now we know that this is, in fact, the case.

What should we do with this insight? In my view, the following steps would be wise:

  1. Take the findings from the existing trials that are devoid of patient-blinding with more than just a pinch of salt.
  2. Discourage the funding of future studies that fail to include patient-blinding.
  3. If patient-blinding is truly and demonstrably impossible – which is not often the case – make sure that the trialists at least include blinding of the assessors of the primary outcome measures.

There must be well over 10 000 clinical trials of acupuncture; Medline lists ~5 000, and many more are hidden in the non-Medline listed literature. That should be good news! Sadly, it isn’t.

It should mean that we now have a pretty good idea for what conditions acupuncture is effective and for which illnesses it does not work. But we don’t! Sceptics say it works for nothing, while acupuncturists claim it is a panacea. The main reason for this continued controversy is that the quality of the vast majority of these 10 000 studies is not just poor, it is lousy.

“Where is the evidence for this outraging statement???” – I hear the acupuncture-enthusiasts shout. Well, how about my own experience as editor-in-chief of FACT? No? Far too anecdotal?

How about looking at Cochrane reviews then; they are considered to be the most independent and reliable evidence in existence? There are many such reviews (most, if not all [co-]authored by acupuncturists) and they all agree that the scientific rigor of the primary studies is fairly awful. Here are the crucial bits of just the last three; feel free to look for more:

All of the studies had a high risk of bias

All included trials had a high risk of bias…

The studies were not judged to be free from bias…

Or how about providing an example? Good idea! Here is a new trial which could stand for numerous others:

This study was performed to compare the efficacy of acupuncture versus corticosteroid injection for the treatment of Quervain’s tendosynovitis (no, you do not need to look up what condition this is for understanding this post). Thirty patients were treated in two groups. The acupuncture group received 5 acupuncture sessions of 30 minutes duration. The injection group received one methylprednisolone acetate injection in the first dorsal compartment of the wrist. The degree of disability and pain was evaluated by using the Quick Disabilities of the Arm, Shoulder, and Hand (Q-DASH) scale and the Visual Analogue Scale (VAS) at baseline and at 2 weeks and 6 weeks after the start of treatment. The baseline means of the Q-DASH and the VAS scores were 62.8 and 6.9, respectively. At the last follow-up, the mean Q-DASH scores were 9.8 versus 6.2 in the acupuncture and injection groups, respectively, and the mean VAS scores were 2 versus 1.2. Thus there were short-term improvements of pain and function in both groups.

The authors drew the following conclusions: Although the success rate was somewhat higher with corticosteroid injection, acupuncture can be considered as an alternative option for treatment of De Quervain’s tenosynovitis.

The flaws of this study are exemplary and numerous:

  • This should have been a study that compares two treatments – the technical term is ‘equivalence trial – and such studies need to be much larger to produce a meaningful result. Small sample sizes in equivalent trials will always make the two treatments look similarly effective, even if one is a pure placebo.
  • There is no gold standard treatment for this condition. This means that a comparative trial makes no sense at all. In such a situation, one ought to conduct a placebo-controlled trial.
  • There was no blinding of patients; therefore their expectation might have distorted the results.
  • The acupuncture group received more treatments than the injection group; therefore the additional attention might have distorted the findings.
  • Even if the results were entirely correct, one cannot conclude from them that acupuncture was effective; the notion that it was similarly ineffective as the injections is just as warranted.

These are just some of the most fatal flaws of this study. The sad thing is that similar criticisms can be made for most of the 10 000 trials of acupuncture. But the point here is not to nit-pick nor to quack-bust. My point is a different and more serious one: fatally flawed research is not just a ‘poor show’, it is unethical because it is a waste of scarce resources and, even more importantly, an abuse of patients for meaningless pseudo-science. All it does is it misleads the public into believing that acupuncture might be good for this or that condition and consequently make wrong therapeutic decisions.

In acupuncture (and indeed in most alternative medicine) research, the problem is so extremely wide-spread that it is high time to do something about it. Journal editors, peer-reviewers, ethics committees, universities, funding agencies and all others concerned with such research have to work together so that such flagrant abuse is stopped once and for all.

When someone has completed a scientific project, it is customary to publish it ['unpublished science is no science', someone once told me many years ago]. To do so, he needs to write it up and submit it to a scientific journal. The editor of this journal will then submit it to a process called ‘peer review’.

What does ‘peer review’ entail? Well, it means that 2-3 experts are asked to critically assess the paper in question, make suggestions as to how it can be improved and submit a recommendation as to whether or not the article deserves to be published.

Peer review has many pitfalls but, so far, nobody has come up with a solution that is convincingly better. Many scientists are under pressure to publish ['publish or perish'], and therefore some people resort to cheating. A most spectacular case of fraudulent peer review has been reported recently in this press release:

SAGE statement on Journal of Vibration and Control

London, UK (08 July 2014) – SAGE announces the retraction of 60 articles implicated in a peer review and citation ring at the Journal of Vibration and Control (JVC). The full extent of the peer review ring has been uncovered following a 14 month SAGE-led investigation, and centres on the strongly suspected misconduct of Peter Chen, formerly of National Pingtung University of Education, Taiwan (NPUE) and possibly other authors at this institution.

In 2013 the then Editor-in-Chief of JVC, Professor Ali H. Nayfeh,and SAGE became aware of a potential peer review ring involving assumed and fabricated identities used to manipulate the online submission system SAGE Track powered by ScholarOne Manuscripts™. Immediate action was taken to prevent JVC from being exploited further, and a complex investigation throughout 2013 and 2014 was undertaken with the full cooperation of Professor Nayfeh and subsequently NPUE.

In total 60 articles have been retracted from JVC after evidence led to at least one author or reviewer being implicated in the peer review ring. Now that the investigation is complete, and the authors have been notified of the findings, we are in a position to make this statement.

While investigating the JVC papers submitted and reviewed by Peter Chen, it was discovered that the author had created various aliases on SAGE Track, providing different email addresses to set up more than one account. Consequently, SAGE scrutinised further the co-authors of and reviewers selected for Peter Chen’s papers, these names appeared to form part of a peer review ring. The investigation also revealed that on at least one occasion, the author Peter Chen reviewed his own paper under one of the aliases he had created.

Unbelievable? Perhaps, but sadly it is true; some scientists seem to be criminally ingenious when it comes to getting their dodgy articles into peer-reviewed journals.

And what does this have to do with ALTERNATIVE MEDICINE, you may well ask. The Journal of Vibration and Control is not even medical and certainly would never consider publishing articles on alternative medicine. Such papers go to one of the many [I estimate more that 1000] journals that cover either alternative medicine in general or any of the modalities that fall under this wide umbrella. Most of these journals, of course, pride themselves with being peer-reviewed – and, at least nominally, that is correct.

I have been on the editorial board of most of the more important journals in alternative medicine, and I cannot help thinking that their peer review process is not all that dissimilar from the fraudulent scheme set up by Peter Chen and disclosed above. What happens in alternative medicine is roughly as follows:

  • a researcher submits a paper for publication,
  • the editor sends it out for peer review,
  • the peer reviewers are either those suggested by the original author or members of the editorial board of the journal,
  • in either case, the reviewers are more than likely to be uncritical and recommend publication,
  • in the end, peer review turns out to be a farcical window dressing exercise with no consequence,
  • thus even very poor research and pseudo-research are being published abundantly.

The editorial boards of journals of alternative medicine tend to be devoid of experts who are critical about the subject at hand. If you think that I am exaggerating, have a look at the editorial board members of ‘HOMEOPATHY’ (or any other journal of alternative medicine) and tell me who might qualify as a critic of homeopathy. When the editor, Peter Fisher, recently fired me from his board because he felt I had tarnished the image of homeopathy, this panel lost the only person who understood the subject matter and, at the same time, was critical about it (the fact that the website still lists me as an editorial board member is merely a reflection of how slow things are in the world of homeopathy: Fisher fired me more than a year ago).

The point I am trying to make is simple: peer review is never a perfect method but when it is set up to be deliberately uncritical, it cannot possibly fulfil its function to prevent the publication of dodgy research. In this case, the quality of the science will be inadequate and generate false-positive messages that mislead the public.

Reiki is a Japanese technique which, according to a proponent, … is administered by “laying on hands” and is based on the idea that an unseen “life force energy” flows through us and is what causes us to be alive. If one’s “life force energy” is low, then we are more likely to get sick or feel stress, and if it is high, we are more capable of being happy and healthy…

A treatment feels like a wonderful glowing radiance that flows through and around you. Reiki treats the whole person including body, emotions, mind and spirit creating many beneficial effects that include relaxation and feelings of peace, security and wellbeing. Many have reported miraculous results.

Reiki is a simple, natural and safe method of spiritual healing and self-improvement that everyone can use. It has been effective in helping virtually every known illness and malady and always creates a beneficial effect. It also works in conjunction with all other medical or therapeutic techniques to relieve side effects and promote recovery [my emphasis].

Many websites give much more specific information about the health effects of Reiki:

Some Of The Reiki Healing Health Benefits 

  • Creates deep relaxation and aids the body to release stress and tension,
  • It accelerates the body’s self-healing abilities,
  • Aids better sleep,
  • Reduces blood pressure
  • Can help with acute (injuries) and chronic problems (asthma, eczema, headaches, etc.) and aides the breaking of addictions,
  • Helps relieve pain,
  • Removes energy blockages, adjusts the energy flow of the endocrine system bringing the body into balance and harmony,
  • Assists the body in cleaning itself from toxins,
  • Reduces some of the side effects of drugs and helps the body to recover from drug therapy after surgery and chemotherapy,
  • Supports the immune system,
  • Increases vitality and postpones the aging process,
  • Raises the vibrational frequency of the body,
  • Helps spiritual growth and emotional clearing.

With such remarkable claims being made, I had to look into this extraordinary treatment.

In 2008, I had a co-worker in my team who was (still is, I think) a Reiki healer. He also happened to be a decent scientist, and we thus decided to conduct a systematic review summarising the evidence for the effectiveness of Reiki. We searched the literature using 23 databases from their respective inceptions through to November 2007 (search again 23 January 2008) without language restrictions. Methodological quality was assessed using the Jadad score. The searches identified 205 potentially relevant studies. Nine randomised clinical trials (RCTs) met our inclusion criteria. Two RCTs suggested beneficial effects of Reiki compared with sham control on depression, while one RCT did not report intergroup differences. For pain and anxiety, one RCT showed intergroup differences compared with sham control. For stress and hopelessness, a further RCT reported effects of Reiki and distant Reiki compared with distant sham control. For functional recovery after ischaemic stroke there were no intergroup differences compared with sham. There was also no difference for anxiety between groups of pregnant women undergoing amniocentesis. For diabetic neuropathy there were no effects of reiki on pain. A further RCT failed to show the effects of Reiki for anxiety and depression in women undergoing breast biopsy compared with conventional care.

Overall, the trial data for any one condition were scarce and independent replications were not available for any condition. Most trials suffered from methodological flaws such as small sample size, inadequate study design and poor reporting. We therefore concluded that the evidence is insufficient to suggest that Reiki is an effective treatment for any condition. Therefore the value of Reiki remains unproven.

But this was in 2008! In the meantime, the evidence might have changed. Here are two recent publications which, I think, are worth having a look at:

The first article is a case-report of a nine-year-old female patient with a history of perinatal stroke, seizures, and type-I diabetes was treated for six weeks with Reiki. At the end of this treatment period, there was a decrease in stress in both the child and the mother, as measured by a modified Perceived Stress Scale and a Perceived Stress Scale, respectively. No change was noted in the child’s overall sense of well-being, as measured by a global questionnaire. However, there was a positive change in sleep patterns on 33.3% of the nights as reported on a sleep log kept by the mother. The child and the Reiki Master (a Reiki practitioner who has completed all three levels of Reiki certification training, trains and certifies individuals in the practice of Reiki, and provides Reiki to individuals) experienced warmth and tingling sensations on the same area of the child during the Reiki 7 minutes of each session. There were no reports of seizures during the study period.

The author concluded that Reiki is a useful adjunct for children with increased stress levels and sleep disturbances secondary to their medical condition. Further research is warranted to evaluate the use of Reiki in children, particularly with a large sample size, and to evaluate the long-term use of Reiki and its effects on adequate sleep.

In my view, this article is relevant because it typifies the type of research that is being done in this area and the conclusions that are being drawn from it. It should be clear to anyone who has the slightest ability of critical thinking that a case report of this nature tells us as good as nothing about the effectiveness of a therapy. Considering that Reiki is just about the least plausible intervention anyone can think of, the child’s condition in all likelihood improved not because of the Reiki healing but because of a myriad of unrelated factors; just think of placebo-effects, regression towards the mean, natural history of the condition, concomitant treatments, etc.

The plausibility of energy/biofield/spiritual healing such as Reiki is also the focus of the second remarkable article that was just published. It reports a systematic review of studies designed to examine whether bio-field therapists undergo physiological changes as they enter the healing state (remember: the Reiki healer in the above study experienced ‘warmth and tingling sensations’ during therapies). If reproducible changes could be identified, the authors argue, they might serve as markers to reveal events that correlate with the healing process.

Databases were searched for controlled or non-controlled studies of bio-field therapies in which physiological measurements were made on practitioners in a healing state. Design and reporting criteria, developed in part to reflect the pilot nature of the included studies, were applied using a yes (1.0), partial (0.5), or no (0) scoring system.

Of 67 identified studies, the inclusion criteria were met by 22, 10 of which involved human patients. Overall, the studies were of moderate to poor quality and many omitted information about the training and experience of the healer. The most frequently measured biomarkers were electroencephalography (EEG) and heart rate variability (HRV). EEG changes were inconsistent and not specific to bio-field therapies. HRV results suggest an aroused physiology for Reconnective Healing, Bruyere healing, and Hawaiian healing, but no changes were detected for Reiki or Therapeutic Touch.

The authors of this paper concluded that despite a decades-long research interest in identifying healing-related biomarkers in bio-field healers, little robust evidence of unique physiological changes has emerged to define the healers׳ state.

Now, let me guess why this is so. One does not need to be a rocket scientist to come up with the suggestion that no robust evidence for Reiki and all the other nonsensical forms of healing can be found for one disarmingly simple reason: NO SUCH EFFECTS EXIST.

Yesterday, BBC NEWS published the following interesting text about a BBC4 broadcast entitled ‘THE ROYAL ACTIVIST’ aired on the same day:

Prince Charles has been a well-known supporter of complementary medicine. According to a… former Labour cabinet minister, Peter Hain, it was a topic they shared an interest in.

“He had been constantly frustrated at his inability to persuade any health ministers anywhere that that was a good idea, and so he, as he once described it to me, found me unique from this point of view, in being somebody that actually agreed with him on this, and might want to deliver it.”
Mr Hain added: “When I was Secretary of State for Northern Ireland in 2005-7, he was delighted when I told him that since I was running the place I could more or less do what I wanted to do.***
“I was able to introduce a trial for complementary medicine on the NHS, and it had spectacularly good results, that people’s well-being and health was vastly improved.

“And when he learnt about this he was really enthusiastic and tried to persuade the Welsh government to do the same thing and the government in Whitehall to do the same thing for England, but not successfully,” added Mr Hain.

*** obviously there is no homeopathic remedy for megalomania (but that’s a different story)

Oh really?

A TRIAL?

SPECTACULARLY GOOD RESULTS?

NO KIDDING?

Let’s have a look at the ‘trial’ and its results. An easily accessible report provides the following details about it:

From February 2007 to February 2008, Get Well UK ran the UK’s first government-backed complementary therapy pilot. Sixteen practitioners provided treatments including acupuncture, osteopathy and aromatherapy, to more than 700 patients at two GP practices in Belfast and Derry.   

The BBC made an hour long documentary following our trials and tribulations, which was broadcast on BBC1 NI on 5 May 2008.

Following the successful completion of the pilot, the results were analysed by Social and Market Research and recommendations were made to the Health Minister

Aims and Objectives 

The aim of the project was to pilot services integrating complementary medicine into existing primary care services in Northern Ireland. Get Well UK provided this pilot project for the Department for Health, Social Services and Public Safety (DHSSPS) during 2007.

The objectives were:

  • To measure the health outcomes of the service and monitor health improvements.
  • To redress inequalities in access to complementary medicine by providing therapies through the NHS, allowing access regardless of income.
  • To contribute to best practise in the field of delivering complementary therapies through primary care.
  • To provide work for suitably skilled and qualified practitioners.
  • To increase patient satisfaction with quick access to expert care.
  • To help patients learn skills to improve and retain their health.
  • To free up GP time to work with other patients.
  • To deliver the programme for 700 patients.

Results 

The results of the pilot were analysed by Social and Market Research, who produced this report.

The findings can be summarised as follows: 

Following the pilot, 80% of patients reported an improvement in their symptoms, 64% took less time off work and 55% reduced their use of painkillers.

In the pilot, 713 patients with a range of ages and demographic backgrounds and either physical or mental health conditions were referred to various complementary and alternative medicine (CAM) therapies via nine GP practices in Belfast and Londonderry. Patients assessed their own health and wellbeing pre and post therapy and GPs and CAM practitioners also rated patients’ responses to treatment and the overall effectiveness of the scheme.

Health improvement
• 81% of patients reported an improvement in their physical health
• 79% reported an improvement in their mental health
• 84% of patients linked an improvement in their health and wellbeing directly to their CAM treatment
• In 65% of patient cases, GPs documented a health improvement, correlating closely to patient-reported improvements
• 94% of patients said they would recommend CAM to another patient with their condition
• 87% of patient indicated a desire to continue with their CAM treatment

Painkillers and medication
• Half of GPs reported prescribing less medication and all reported that patients had indicated to them that they needed less
• 62% of patients reported suffering from less pain
• 55% reported using less painkillers following treatment
• Patients using medication reduced from 75% before treatment to 61% after treatment
• 44% of those taking medication before treatment had reduced their use afterwards

Health service and social benefits
• 24% of patients who used health services prior to treatment (i.e. primary and secondary care, accident and emergency) reported using the services less after treatment
• 65% of GPs reported seeing the patient less following the CAM referral
• Half of GPs said the scheme had reduced their workload and 17% reported a financial saving for their practice
• Half of GPs said their patients were using secondary care services less.

Impressed? Well, in case you are, please consider this:

  • there was no control group
  • therefore it is not possible to attribute any of the outcomes to the alternative therapies offered
  • they could have been due to placebo-effects
  • or to the natural history of the disease
  • or to regression towards the mean
  • or to social desirability
  • or to many other factors which are unrelated to the alternative treatments provided
  • most outcome measures were not objectively verified
  • the patients were self-selected
  • they would all have had conventional treatments in parallel
  • this ‘trial’ was of such poor quality that its findings were never published in a peer-reviewed journal
  • this was not a ‘trial’ but a ‘pilot study’
  • pilot studies are not normally for measuring outcomes but for testing the feasibility of a proper trial
  • the research expertise of the investigators was close to zero
  • the scientific community merely had pitiful smiles for this ‘trial’ when it was published
  • neither Northern Ireland nor any other region implemented the programme despite its “spectacularly good results”.

So, is the whole ‘trial’ story an utterly irrelevant old hat?

Certainly not! Its true significance does not lie in the fact that a few amateurs are trying to push bogus treatments into the NHS via the flimsiest pseudo-research of the century. The true significance, I think, is that it shows how Prince Charles, once again, oversteps the boundaries of his constitutional role.

If you are pregnant, a ‘breech presentation’ is not good news. It occurs when the fetus presents ‘bottom-down’ in the uterus. There are three types:

  • Breech with extended legs (frank) – 85% of cases
  • Breech with fully flexed legs (complete)
  • Footling (incomplete) with one or both thighs extended

The significance of breech presentation is its association with higher perinatal mortality and morbidity when compared to cephalic presentations. This is due both to pre-existing congenital malformation, increased incidence of breech in premature deliveries and increased risk of intrapartum trauma or asphyxia. Caesarean section has been adopted as the ‘normal’ mode of delivery for term breech presentations in Europe and the USA, as the consensus is that this reduces the risk of birth-related complications.

But Caesarian section is also not a desirable procedure. Something far less invasive would be much more preferable, of course. This is where the TCM-practitioners come in. They claim they have the solution: moxibustion, i.e. the stimulation of acupuncture points by heat. But does it really work? Can it turn the fetus into the correct position?

This new study aimed to assess the efficacy of moxibustion (heating of the acupuncture needle with an igniting charcoal moxa stick) with acupuncture for version of breech presentations to reduce their rate at 37 weeks of gestation and at delivery. It was a randomized, placebo-controlled, single-blind trial including 328 pregnant women recruited in a university hospital center between 33 4/7 and 35 4/7 weeks of gestation. Moxibustion with acupuncture or inactivated laser (placebo) treatment was applied to point BL 67 for 6 sessions. The principal endpoint was the percentage of fetuses in breech presentation at 37 2/7 weeks of gestation.

The results show that the percentage of fetuses in breech presentation at 37 2/7 weeks of gestation was not significantly different in both groups (72.0 in the moxibustion with acupuncture group compared with 63.4% in the placebo group).

The authors concluded that treatment by moxibustion with acupuncture was not effective in correcting breech presentation in the third trimester of pregnancy.

You might well ask why on earth anyone expected that stimulating an acupuncture point would turn a fetus in the mother’s uterus into the optimal position that carries the least risk during the process of giving birth. This is what proponents of this technique say about this approach:

During a TCM consultation to turn a breech baby the practitioner will take a comprehensive case history, make a diagnosis and apply the appropriate acupuncture treatment.  They will assess if moxibustion might be helpful. Practitioners will then instruct women on how to locate the appropriate acupuncture points and demonstrate how to safely apply moxa at home. The acupuncture point UB 67 is the primary point selected for use because it is the most dynamic point to activate the uterus.  Its forte is in turning malpositioned babies.  It is located on the outer, lower edge of both little toenails. According to TCM theory, moxa has a tonifying and warming effect which promotes movement and activity.  The nature of heat is also rising.  This warming and raising effect is utilised to encourage the baby to become more active and lift its bottom up in order to gain adequate momentum to summersault into the head down position. This technique can also be used to reposition transverse presentation, a situation where the baby’s has its shoulder or back pointing down, or is lying sideways across the abdomen.

Not convinced? I can’t say I blame you!

Clearly, we need to know what the totality of the most reliable evidence shows; and what better than a Cochrane review to inform us about it? Here is what it tells us:

Moxibustion was not found to reduce the number of non-cephalic presentations at birth compared with no treatment (P = 0.45). Moxibustion resulted in decreased use of oxytocin before or during labour for women who had vaginal deliveries compared with no treatment (risk ratio (RR) 0.28, 95% confidence interval (CI) 0.13 to 0.60). Moxibustion was found to result in fewer non-cephalic presentations at birth compared with acupuncture (RR 0.25, 95% CI 0.09 to 0.72). When combined with acupuncture, moxibustion resulted in fewer non-cephalic presentations at birth (RR 0.73, 95% CI 0.57 to 0.94), and fewer births by caesarean section (RR 0.79, 95% CI 0.64 to 0.98) compared with no treatment. When combined with a postural technique, moxibustion was found to result in fewer non-cephalic presentations at birth compared with the postural technique alone (RR 0.26, 95% CI 0.12 to 0.56).

In other words, there is indeed some encouraging albeit not convincing evidence! How can this be? There is no plausible explanation why this treatment should work!

But there is a highly plausible explanation why the results of many of the relevant trials are false-positive thus rendering a meta-analysis false-positive as well. I have repeatedly pointed out on this blog that practically all Chinese TCM-studies report (false) positive results; and many of the studies included in this review were done in China. The Cochrane review provides a strong hint about the lack of rigor in its ‘plain language summary’:

The included trials were of moderate methodological quality, sample sizes in some of the studies were small, how the treatment was applied differed and reporting was limited. While the results were combined they should be interpreted with caution due to the differences in the included studies. More evidence is needed concerning the benefits and safety of moxibustion.

So, would I recommend moxibustion for breech conversion? I don’t think so!

Traditional Chinese medicine (TCM) is often promoted as an effective therapy for cancer, and are numerous controlled clinical studies published in Chinese literature, yet no systematic analysis has been done of this body of evidence. This systematic review summarizes the evidence from controlled clinical studies published in Chinese on this subject.

The researchers looked for controlled clinical studies of TCM therapies for all kinds of cancers published in Chinese in four main Chinese electronic databases and found 2964 reports including 2385 randomized clinical trials and 579 non-randomized controlled studies.

The top seven cancer types treated were lung cancer, liver cancer, stomach cancer, breast cancer, esophagus cancer, colorectal cancer and nasopharyngeal cancer. The majority of studies (72%) applied TCM therapy combined with conventional treatments, whilst fewer (28%) applied only TCM therapy in the experimental groups. Herbal medicine was the most frequently tested TCM therapy (2677 studies, 90.32%).

The most frequently reported outcome was clinical symptom improvement (1667 studies, 56.24%) followed by biomarker indices (1270 studies, 42.85%), quality of life (1129 studies, 38.09%), chemo/radiotherapy induced side effects (1094 studies, 36.91%), tumor size (869 studies, 29.32%) and safety (547 studies, 18.45%). Completeness and adequacy of reporting appeared to improve with time.

The authors of this paper drew the following conclusion: data from controlled clinical studies of TCM therapies in cancer treatment is substantial, and different therapies are applied either as monotherapy or in combination with conventional medicine. Reporting of controlled clinical studies should be improved based on the CONSORT and TREND Statements in future. Further studies should address the most frequently used TCM therapy for common cancers and outcome measures should address survival, relapse/metastasis and quality of life.

Almost 3000 controlled clinical trials! This number is likely to impress many people – unless, of course, one knows that the quality of these studies is dismal. Interestingly, no formal assessment of study quality was included in this analysis. But it was mentioned that only 63 of these trials reported patient-blinding, and only 5 were deemed to be “relatively well designed” by the authors of this paper (who, incidentally, are strong proponents of TCM).

What I find the most interesting aspect of this article is the fact that the authors fail to mention how many of the studies reported a positive result – in a way, they don’t need to: there is plenty of evidence to show that virtually all of the Chinese studies of TCM are positive. In my view, this invalidates this body of evidence completely.

Analysis like the present one tend to lead us up the garden path. They suggest that there is a realistic hope for effective new treatments hidden in this difficult to access, large amount of data. This might lead other researchers to try to replicate some of the original studies. I fear that they would be wasting their time. From all I know, they are irreproducible.

If we search on ‘Medline’ for ‘complementary alternative medicine’ (CAM), we currently get about 13000 hits. A little graph on the side of the page demonstrates that, during the last 4 years, the number of articles on this subject has grown exponentially.

Surely, this must be very good news: such intense research activity will soon tell us exactly which alternative treatments work for which conditions and which don’t.

I beg to differ. Let me explain why.

The same ‘Medline’ search informs us that the majority of the recent articles were published in an open access journal called ‘Evidence-Based Complementary and Alternative Medicine’ (eCAM). For example, of the 80 most recent articles listed in Medline (on 26/5/2014), 53 came from that journal. The publication frequency of eCAM and its increase in recent years beggars belief: in 2011, they published just over 500 articles which is already a high number, but, in 2012, the figure had risen to >800, and in 2013 it was >1300 (the equivalent 2013 figure for the BMJ/BMJ Open by comparison is 4, and that for another alt med journal, e.g. Forsch Komplement, is 10)

How do they do it? How can eCAM be so dominant in publishing alt med research? The trick seems to be fairly simple.

Let’s assume you are an alt med researcher and you have an article that you would like to see published. Once you submit it to eCAM, your paper is sent to one of the ~150 members of the editorial board. These people are almost all strong proponents of alternative medicine; critics are a true rarity in this group. At this stage, you are able to suggest the peer reviewers for your submission (all who ever accepted this task are listed on the website; they amount to several thousand!), and it seems that, with the vast majority of submissions, the authors’ suggestions are being followed.

It goes without saying that most researchers suggest colleagues for peer reviewing who are not going to reject their work (the motto seems to be “if you pass my paper, I will pass yours). Therefore even faily flimsy bits of research pass this peer review process and get quickly published online in eCAM.

This process explains a lot, I think: 1) the extraordinarily high number of articles published 2) why currently more than 50% of all alt med research originate from eCAM 3) why so much of it is utter rubbish.

Even the mere titles of some of the articles might demonstrate my point. A few examples have to suffice:

  • Color distribution differences in the tongue in sleep disorder
  • Wen-dan decoction improves negative emotions in sleep-deprived rats by regulating orexin-a and leptin expression.
  • Yiqi Huoxue Recipe Improves Heart Function through Inhibiting Apoptosis Related to Endoplasmic Reticulum Stress in Myocardial Infarction Model of Rats.
  • Protective Effects of Bu-Shen-Huo-Xue Formula against 5/6 Nephrectomy-Induced Chronic Renal Failure in Rats
  • Effects and Mechanisms of Complementary and Alternative Medicine during the Reproductive Process
  • Evidence-based medicinal plants for modern chronic diseases
  • Transforming Pain into Beauty: On Art, Healing, and Care for the Spirit

This system of uncritical peer review and fast online publication seems to suit many of the people involved in this process: the journal’s owners are laughing all the way to the bank; there is a publication charge of US$ 2000 per article, and, in 2013, the income of eCAM must therefore have been well over US$2 000 000. The researchers are equally delighted; they get even their flimsiest papers published (remember: ‘publish or perish’!). And the evangelic believers in alternative medicine are pleased because they can now claim that their field is highly research-active and that there is plenty of evidence to support the use of this or that therapy.

But there are others who are not served well by eCAM habit of publishing irrelevant, low quality articles:

  • professionals who would like to advance health care and want to see reliable evidence as to which treatments work and which don’t,
  • the public who, in one way or another, pay for all this and might assume that published research tends to be relevant and reliable,
  • the patients who have given their time to researchers in the hope that their gift will improve health care,
  • ill individuals who hope that alternative treatments might relieve their suffering,
  • politicians who rely on research to be reliable in order to arrive at the right decisions.

Come to think of it, the vast majority of people should be less than enchanted with eCAM and similar journals.

Boiron is the world’s biggest producer of homeopathic remedies. It also is a firm that is relatively active in research into homeopathy. Here is one of their investigations which I find most remarkable.

This study was designed to describe the sociodemographic and clinical characteristics of patients recommended allopathic and/or homeopathic medicines for influenza-like illness (ILI) or ear nose and throat ENT disorders by pharmacists in France and to investigate the effectiveness of these treatments.

The introduction of the article includes interesting information; it informs us that, although homeopathy is more popular in Europe than in the Unites States, sales of homeopathic medicines in the United States grew by more than 1,000% in the late 1970s and early 1980s and continue to grow. In parallel, the number of physicians specializing in homeopathy doubled between 1980 and 1982. In 2003, sales of homeopathic medicines in the United States were estimated to be between $300 and $450 million, with an average growth rate of approximately 8% per year. Homeopathic drugs are among the top 10 nonprescription products sold in the category of analgesics to treat coughs, colds, and flu. The sale of homeopathic medicines in the United States is controlled by the Food, Drug, and Cosmetic Act and regulations issued by the U.S Food and Drug Administration (FDA). Homeopathic medicines in the United States are subject to well-controlled regulatory processes that closely resemble those used for allopathic medicines. FDA regulations for the sale of homeopathic medicines in the United States state that they can only be sold without prescription if they are for self-limiting conditions such as the common cold…

Am I mistaken, or does that paragraph read a bit like a text written by the marketing team of Boiron wanting to establish their products in the US?

Anyway, the methodology and results of the study are described in the abstract as follows:

A prospective, observational, multicenter study was carried out in randomly selected pharmacies across the 8 IDREM medical regions of France. Pharmacies that agreed to participate recruited male or female patients who responded to the following inclusion criteria: age ≥ 12 years presenting with the first symptoms of an ILI or ENT disorder that were 
present for less than 36 hours prior to the pharmacy visit. All medicines recorded in the study were recommended by the pharmacists. The following data were recorded at inclusion and after 3 days of treatment: the intensity of 13 symptoms, global symptom score, and disease impact on daily activities and sleep. Two groups of patients were compared: those recommended allopathic medicine only (AT group) and those recommended homeopathic medicine with or without allopathic medicine (HAT group). The number and severity of symptoms, change in global symptom score, and disease impact on daily activities and sleep were compared in the 2 treatment groups after 3 days of treatment. Independent predictors of recommendations for homeopathic medicine were identified by multi-
factorial logistic regression analysis.

A total of 242 pharmacies out of 4,809 (5.0%) contacted agreed to participate in the study, and 133 (2.8%) included at least 1 patient; 573 patients were analyzed (mean age: 42.5 ± 16.2 years; 61.9% female). Of these, 428 received allopathic medicines only (74.7%; AT group), and 145 (25.3%) received homeopathic medicines (HAT group) alone (9/145, 1.6%) or associated with allopathy (136/145, 23.7%). At inclusion, HAT patients were significantly younger (39.6 ± 14.8 vs. 43.4 ± 16.1 years; P  less than  0.05), had a higher mean number of symptoms (5.2 ± 2.5 vs. 4.4 ± 2.5; P  less than  0.01), and more severe symptoms (mean global symptom score: 24.3 ± 5.5 vs. 22.3 ± 5.8; P = 0.0019) than AT patients. After 3 days, the improvement in symptoms and disease impact on daily activities and sleep was comparable in both groups of patients.

From these findings, the authors draw the following conclusions: Patients recommended homeopathic medicine by pharmacists were younger and had more severe symptoms than those recommended allopathic medicine. After 3 days of treatment, clinical improvement was comparable in both treatment groups. Pharmacists have an important role to play in the effective management of ILI and ENT disorders.

And, to make perfectly clear what all this is about, the first sentence of the ‘discussion’ puts it to the point by stating that homeopathic medicine, with or without allopathic medicine, appears to be effective at alleviating the symptoms of ILI or ENT disorders.

Oh really?

As I have heard it said that Boiron seems to have the nasty habit of threatening their critics with legal action, I ought to be quite cautious in my assessment of this ‘masterpiece of promotion’. Yet a few comments must surely be permitted.

‘To describe the sociodemographic and clinical characteristics of patients recommended allopathic and/or homeopathic medicines’ is not what I personally find an interesting subject of research, nor is it anything that will affect health care meaningfully, I think. Yet ‘to investigate the effectiveness of these treatments’ is certainly interesting and important. I will therefore focus on this second aim of the study.

Hold on, was this really a ‘study’? On closer inspection, it seemed much more like a survey. People who felt that they were suffering from ILI and ENT disorders and thus went to a pharmacy to buy something for their problem were offered either homeopathic or conventional medicines. Those who accepted either of the recommendations were asked to fill out some self-assessment forms and received a phone call three days later to check their symptoms. 94% of all patients in the homeopathy group took homeopathic medicine in combination with ‘allopathic’ medicine (it is interesting, perhaps even telling, that this term used by the authors was invented by Hahnemann as an insult to conventional medicine!). There was no examination by a doctor to verify what condition the survey-participants were truly suffering from, and there was no verification that the information provided during the follow-up telephone call was in any way real. The most frequently recommended homeopathic medicine was Anas barbariae 200C (Oscillococcinum) which is Boiron’s famous homeopathically diluted (about one molecule per universe, I guess) duck-liver heavily promoted in France against colds and similar conditions.

As it turns out, those survey-participants who accepted the homeopathic recommendation were significantly younger than those who accepted the recommendation for a conventional treatment (many surveys confirm that younger people are more prone to trying alternative medicine than older ones). It stands to reason, that the younger (and therefore fitter) patients were in better general health and therefore might recover quicker than the older ones. But, in fact, they did not!

Could this be due to the homeopathic remedies actually delaying recovery? Of course not! Who would be silly enough to claim that homeopathy could have this (or any other) effect? According to the authors, it is due to the fact that this group ‘had more severe symptoms than those recommended allopathic medicine’. But, as I said, we have to take their word for it; there is no independent verification of this. It would, of course, be quite ridiculous to postulate that those survey-participants accepting homeopathy were also a little more introspective or concerned about their own health (perhaps even more gullible) and thus claimed more severe symptoms!

And what about the authors’ conclusion that clinical improvement was comparable in both treatment groups? Well, this is more than a little problematic, in my view: first, we have no independent verification of the ‘improvement’ in either group. Second, we don’t know that the conventional treatments actually worked, and it could well be that both approaches were similarly ineffective, and that the observed outcomes are merely a reflection of the natural history of the condition. And third, one might expect the homeopathic (younger) group to do not similarly well but slightly better, simply because the natural history of the illness would tend to be more benign in younger people.

Before I finish,I should make a brief comment about the authors’ courageous statement that  homeopathic medicine, with or without allopathic medicine, appears to be effective at alleviating the symptoms of ILI or ENT disorders. I think, for the reasons I already provided, this is extremely doubtful. In my view, more critical scientists would have phrased the conclusions differently:

THIS SURVEY SHOWS THAT EVALUATING THE EFFECTIVENESS OF MEDICAL INTERVENTIONS REQUIRES A MORE RIGOROUS METHODOLOGY THAN THAT OF A SURVEY.

But perhaps this would be asking a little too much of the authors; after all, at the end of the article, we find this telling footnote: Laboratoires Boiron provided financial support for the study. Cognet-Dementhon, Thevenard, Duru, and Allaert received consulting fees from Laboratoires Boiron for this study. Danno and Bordet are employees of Laboratoires Boiron.

Recent Comments
Click here for a comprehensive list of recent comments.
Categories