There must be well over 10 000 clinical trials of acupuncture; Medline lists ~5 000, and many more are hidden in the non-Medline listed literature. That should be good news! Sadly, it isn’t.
It should mean that we now have a pretty good idea for what conditions acupuncture is effective and for which illnesses it does not work. But we don’t! Sceptics say it works for nothing, while acupuncturists claim it is a panacea. The main reason for this continued controversy is that the quality of the vast majority of these 10 000 studies is not just poor, it is lousy.
“Where is the evidence for this outraging statement???” – I hear the acupuncture-enthusiasts shout. Well, how about my own experience as editor-in-chief of FACT? No? Far too anecdotal?
How about looking at Cochrane reviews then; they are considered to be the most independent and reliable evidence in existence? There are many such reviews (most, if not all [co-]authored by acupuncturists) and they all agree that the scientific rigor of the primary studies is fairly awful. Here are the crucial bits of just the last three; feel free to look for more:
Or how about providing an example? Good idea! Here is a new trial which could stand for numerous others:
This study was performed to compare the efficacy of acupuncture versus corticosteroid injection for the treatment of Quervain’s tendosynovitis (no, you do not need to look up what condition this is for understanding this post). Thirty patients were treated in two groups. The acupuncture group received 5 acupuncture sessions of 30 minutes duration. The injection group received one methylprednisolone acetate injection in the first dorsal compartment of the wrist. The degree of disability and pain was evaluated by using the Quick Disabilities of the Arm, Shoulder, and Hand (Q-DASH) scale and the Visual Analogue Scale (VAS) at baseline and at 2 weeks and 6 weeks after the start of treatment. The baseline means of the Q-DASH and the VAS scores were 62.8 and 6.9, respectively. At the last follow-up, the mean Q-DASH scores were 9.8 versus 6.2 in the acupuncture and injection groups, respectively, and the mean VAS scores were 2 versus 1.2. Thus there were short-term improvements of pain and function in both groups.
The authors drew the following conclusions: Although the success rate was somewhat higher with corticosteroid injection, acupuncture can be considered as an alternative option for treatment of De Quervain’s tenosynovitis.
The flaws of this study are exemplary and numerous:
- This should have been a study that compares two treatments – the technical term is ‘equivalence trial – and such studies need to be much larger to produce a meaningful result. Small sample sizes in equivalent trials will always make the two treatments look similarly effective, even if one is a pure placebo.
- There is no gold standard treatment for this condition. This means that a comparative trial makes no sense at all. In such a situation, one ought to conduct a placebo-controlled trial.
- There was no blinding of patients; therefore their expectation might have distorted the results.
- The acupuncture group received more treatments than the injection group; therefore the additional attention might have distorted the findings.
- Even if the results were entirely correct, one cannot conclude from them that acupuncture was effective; the notion that it was similarly ineffective as the injections is just as warranted.
These are just some of the most fatal flaws of this study. The sad thing is that similar criticisms can be made for most of the 10 000 trials of acupuncture. But the point here is not to nit-pick nor to quack-bust. My point is a different and more serious one: fatally flawed research is not just a ‘poor show’, it is unethical because it is a waste of scarce resources and, even more importantly, an abuse of patients for meaningless pseudo-science. All it does is it misleads the public into believing that acupuncture might be good for this or that condition and consequently make wrong therapeutic decisions.
In acupuncture (and indeed in most alternative medicine) research, the problem is so extremely wide-spread that it is high time to do something about it. Journal editors, peer-reviewers, ethics committees, universities, funding agencies and all others concerned with such research have to work together so that such flagrant abuse is stopped once and for all.
Reiki is a Japanese technique which, according to a proponent, … is administered by “laying on hands” and is based on the idea that an unseen “life force energy” flows through us and is what causes us to be alive. If one’s “life force energy” is low, then we are more likely to get sick or feel stress, and if it is high, we are more capable of being happy and healthy…
A treatment feels like a wonderful glowing radiance that flows through and around you. Reiki treats the whole person including body, emotions, mind and spirit creating many beneficial effects that include relaxation and feelings of peace, security and wellbeing. Many have reported miraculous results.
Reiki is a simple, natural and safe method of spiritual healing and self-improvement that everyone can use. It has been effective in helping virtually every known illness and malady and always creates a beneficial effect. It also works in conjunction with all other medical or therapeutic techniques to relieve side effects and promote recovery [my emphasis].
Many websites give much more specific information about the health effects of Reiki:
- Creates deep relaxation and aids the body to release stress and tension,
- It accelerates the body’s self-healing abilities,
- Aids better sleep,
- Reduces blood pressure
- Can help with acute (injuries) and chronic problems (asthma, eczema, headaches, etc.) and aides the breaking of addictions,
- Helps relieve pain,
- Removes energy blockages, adjusts the energy flow of the endocrine system bringing the body into balance and harmony,
- Assists the body in cleaning itself from toxins,
- Reduces some of the side effects of drugs and helps the body to recover from drug therapy after surgery and chemotherapy,
- Supports the immune system,
- Increases vitality and postpones the aging process,
- Raises the vibrational frequency of the body,
- Helps spiritual growth and emotional clearing.
With such remarkable claims being made, I had to look into this extraordinary treatment.
In 2008, I had a co-worker in my team who was (still is, I think) a Reiki healer. He also happened to be a decent scientist, and we thus decided to conduct a systematic review summarising the evidence for the effectiveness of Reiki. We searched the literature using 23 databases from their respective inceptions through to November 2007 (search again 23 January 2008) without language restrictions. Methodological quality was assessed using the Jadad score. The searches identified 205 potentially relevant studies. Nine randomised clinical trials (RCTs) met our inclusion criteria. Two RCTs suggested beneficial effects of Reiki compared with sham control on depression, while one RCT did not report intergroup differences. For pain and anxiety, one RCT showed intergroup differences compared with sham control. For stress and hopelessness, a further RCT reported effects of Reiki and distant Reiki compared with distant sham control. For functional recovery after ischaemic stroke there were no intergroup differences compared with sham. There was also no difference for anxiety between groups of pregnant women undergoing amniocentesis. For diabetic neuropathy there were no effects of reiki on pain. A further RCT failed to show the effects of Reiki for anxiety and depression in women undergoing breast biopsy compared with conventional care.
Overall, the trial data for any one condition were scarce and independent replications were not available for any condition. Most trials suffered from methodological flaws such as small sample size, inadequate study design and poor reporting. We therefore concluded that the evidence is insufficient to suggest that Reiki is an effective treatment for any condition. Therefore the value of Reiki remains unproven.
But this was in 2008! In the meantime, the evidence might have changed. Here are two recent publications which, I think, are worth having a look at:
The first article is a case-report of a nine-year-old female patient with a history of perinatal stroke, seizures, and type-I diabetes was treated for six weeks with Reiki. At the end of this treatment period, there was a decrease in stress in both the child and the mother, as measured by a modified Perceived Stress Scale and a Perceived Stress Scale, respectively. No change was noted in the child’s overall sense of well-being, as measured by a global questionnaire. However, there was a positive change in sleep patterns on 33.3% of the nights as reported on a sleep log kept by the mother. The child and the Reiki Master (a Reiki practitioner who has completed all three levels of Reiki certification training, trains and certifies individuals in the practice of Reiki, and provides Reiki to individuals) experienced warmth and tingling sensations on the same area of the child during the Reiki 7 minutes of each session. There were no reports of seizures during the study period.
The author concluded that Reiki is a useful adjunct for children with increased stress levels and sleep disturbances secondary to their medical condition. Further research is warranted to evaluate the use of Reiki in children, particularly with a large sample size, and to evaluate the long-term use of Reiki and its effects on adequate sleep.
In my view, this article is relevant because it typifies the type of research that is being done in this area and the conclusions that are being drawn from it. It should be clear to anyone who has the slightest ability of critical thinking that a case report of this nature tells us as good as nothing about the effectiveness of a therapy. Considering that Reiki is just about the least plausible intervention anyone can think of, the child’s condition in all likelihood improved not because of the Reiki healing but because of a myriad of unrelated factors; just think of placebo-effects, regression towards the mean, natural history of the condition, concomitant treatments, etc.
The plausibility of energy/biofield/spiritual healing such as Reiki is also the focus of the second remarkable article that was just published. It reports a systematic review of studies designed to examine whether bio-field therapists undergo physiological changes as they enter the healing state (remember: the Reiki healer in the above study experienced ‘warmth and tingling sensations’ during therapies). If reproducible changes could be identified, the authors argue, they might serve as markers to reveal events that correlate with the healing process.
Databases were searched for controlled or non-controlled studies of bio-field therapies in which physiological measurements were made on practitioners in a healing state. Design and reporting criteria, developed in part to reflect the pilot nature of the included studies, were applied using a yes (1.0), partial (0.5), or no (0) scoring system.
Of 67 identified studies, the inclusion criteria were met by 22, 10 of which involved human patients. Overall, the studies were of moderate to poor quality and many omitted information about the training and experience of the healer. The most frequently measured biomarkers were electroencephalography (EEG) and heart rate variability (HRV). EEG changes were inconsistent and not specific to bio-field therapies. HRV results suggest an aroused physiology for Reconnective Healing, Bruyere healing, and Hawaiian healing, but no changes were detected for Reiki or Therapeutic Touch.
The authors of this paper concluded that despite a decades-long research interest in identifying healing-related biomarkers in bio-field healers, little robust evidence of unique physiological changes has emerged to define the healers׳ state.
Now, let me guess why this is so. One does not need to be a rocket scientist to come up with the suggestion that no robust evidence for Reiki and all the other nonsensical forms of healing can be found for one disarmingly simple reason: NO SUCH EFFECTS EXIST.
There is some (albeit not compelling) evidence to suggest that chiropractic spinal manipulation might be effective for treating non-specific back pain. But what about specific back pain, such as the one caused by a herniated disc? Some experts believe that, in patients suffering from such a condition, manipulations are contra-indicated (because the latter can cause the former), while others think that manipulation might be an effective treatment option (although the evidence is far from compelling). Who is correct? The issue can only be resolved with data from well-designed clinical investigations. A new trial might therefore enlighten us.
The stated purposes of this study were:
- to evaluate patients with low-back pain (LBP) and leg pain due to magnetic resonance imaging-confirmed disc herniation treated with high-velocity, low-amplitude spinal manipulation in terms of their short-, medium-, and long-term outcomes of self-reported global impression of change and pain levels
- to determine if outcomes differ between acute and chronic patients using.
The researchers conducted a ‘prospective cohort outcomes study‘ with 148 patients with LBP, leg pain, and physical examination abnormalities with concordant lumbar disc herniations. Baseline numerical rating scale (NRS) data for LBP, leg pain, and the Oswestry questionnaire were obtained. The specific lumbar spinal manipulation was dependent upon whether the disc herniation was intraforaminal or paramedian as seen on the magnetic resonance images and was performed by a chiropractor. Outcomes included the patient’s global impression of change scale for overall improvement, the NRS for LBP, leg pain, and the Oswestry questionnaire at 2 weeks, 1, 3, and 6 months, and 1 year. The proportion of patients reporting “improvement” on the patient’s global impression of change scale was calculated for all patients and for acute vs chronic patients. Pre-treatment and post-treatment NRS scores were compared using the paired t test. Baseline and follow-up Oswestry scores were compared using the Wilcoxon test. Numerical rating scale and Oswestry scores for acute vs chronic patients were compared using the unpaired t test for NRS scores and the Mann-Whitney U test for Oswestry scores.
Significant improvements for all outcomes at all time points were reported. At 3 months, 91% of patients were “improved”, and 88% were “improved” after 1 year. Acute patients improved faster by 3 months than did chronic patients. 81.8% of chronic patients 89.2% felt “improved” at 1 year. No adverse events were reported.
The researchers concluded that a large percentage of acute and importantly chronic lumbar disc herniation patients treated with chiropractic spinal manipulation reported clinically relevant improvement.
Does this new study meaningfully contribute to our knowledge about the effectiveness of chiropractic manipulation for back pain caused by herniated discs? The short answer to this question is NO.
A longer answer might be that the report does tell us something relevant about the quality of this research project. We know from countless studies that ~50% of patients experience adverse effects after spinal manipulations by a chiropractor. This means that any report claiming that NO ADVERSE EFFECTS WERE REPORTED is puzzling to a degree that we have to seriously question its quality or even honesty. In this context, it is relevant to mention that a recent review of the evidence concluded that a cause-effect relationship exists between the manipulative treatment and the development of disc herniation.
The positive outcomes reported in this new study could, of course, be due to a range of factors which are unrelated to the manipulations administered by the chiropractors:
- natural history of disc herniation
- regression towards the mean
- other treatments employed by the patients
- social desirability
To be able to say with any degree of certainty that the manipulations had anything to do with the observed positive outcomes would require an entirely different study-design. Should we assume that this is not known in the world of chiropractic? Or should we consider that chiropractors shy away from rigorous research because they fear its results?
The term prospective cohort outcomes study, seems to be a chiropractic invention (cohort studies are by definition prospective, and observational studies are usually prospective). It seems that, behind this long and impressive word, one can easily hide the fact that this study design fails to make the slightest attempt of controlling for non-specific effects; the term sounds scientific – but when we analyse what it means, we discover that this methodology is little more than a self-serving consumer survey. Most scientists would call such an investigation quite simply an OBSERVATIONAL STUDY.
I think it is time that chiropractors start doing proper research which actually does answer some of the many open questions regarding spinal manipulation.
A remarkable article about homeopathy and immunisation entitled THE IMMUNISATION DILEMMA came to my attention recently. Its abstract promised: “evidence quantifying the effectiveness of vaccination and HP (homeoprophylaxis) will be examined. New international research describing and analysing HP interventions will be reported. An evidence-based conclusion will be reached.”
Sounds interesting? Let’s see what the article really offers. Here is the relevant text:
…evidence does exist to support claims regarding the effectiveness of homeopathic immunisation is undeniable.
I was first invited to visit Cuba in December 2008 to present at an international conference hosted by the Finlay Institute, which is a W. H. O.-accredited vaccine manufacturer. The Cubans described their use of HP to control an outbreak of leptospirosis (Weilʼs syndrome – a potentially fatal, water-born bacterial disease) in 2007 among the residents of the three eastern provinces which were most severely damaged by a severe hurricane – over 2.2 million people . 2008 was an even worse year involving three hurricanes, and the countryʼs food production was only just recovering at the time of the conference. The HP program had been repeated in 2008, but data was not available at the conference regarding that intervention.
I revisited Cuba in 2010 and 2012, each time to work with the leader of the HP interventions, Dr. Bracho, to analyse the data available. Dr. Bracho is not a homeopath; he is a published and internationally recognised expert in the manufacture of vaccine adjuvants. He worked in Australia at Flinders University during 2004 with a team trying to develop an antimalarial vaccine.
In 2012 we accessed the raw leptospirosis surveillance data, comprising weekly reports from 15 provinces over 9 years (2000 to 2008) reporting 21 variables. This yielded a matrix with 147 420 possible entries. This included data concerning possible confounders, such as vaccination and chemoprophylaxis, which allowed a careful examination of possible distorting effects. With the permission of the Cubans, I brought this data back to Australia and it is being examined by mathematicians at an Australian university to see what other information can be extracted. Clearly, there is objective data supporting claims regarding the effectiveness of HP.
The 2008 result was remarkable, and could only be explained by the effectiveness of the HP intervention. Whilst the three hurricanes caused immense damage throughout the country it was again worse in the east, yet the three homeopathically immunised provinces experienced a negligible increase in cases whilst the rest of the country showed significant increases until the dry season in January 2009 .
This is but one example – there are many more. It is cited to show that there is significant data available, and that orthodox scientists and doctors have driven the HP interventions, in the Cuban case. Many people internationally now know this, so once again claims by orthodox authorities that there is no evidence merely serves to show that either the authorities are making uninformed/unscientific statements, or that they are aware but are intentionally withholding information. Either way, confidence is destroyed and leads to groups of people questioning what they are told…
The attacks against homeopathy in general and HP in particular will almost certainly continue. If we can achieve a significant level of agreement then we would be able to answer challenges to HP with a single, cohesive, evidence-based, and generally united response. This would be a significant improvement to the existing situation.
Reference 7 is the following article: Bracho G, Varela E, Fernández R et al. Large-scale application of highly-diluted bacteria for Leptospirosis epidemic control. Homeopathy 2010; 99: 156-166. The crucial bit if this paper are as follows:
A homeoprophylactic formulation was prepared from dilutions of four circulating strains of Leptospirosis. This formulation was administered orally to 2.3 million persons at high risk in an epidemic in a region affected by natural disasters. The data from surveillance were used to measure the impact of the intervention by comparing with historical trends and non-intervention regions.
After the homeoprophylactic intervention a significant decrease of the disease incidence was observed in the intervention regions. No such modifications were observed in non-intervention regions. In the intervention region the incidence of Leptospirosis fell below the historic median. This observation was independent of rainfall.
The homeoprophylactic approach was associated with a large reduction of disease incidence and control of the epidemic. The results suggest the use of HP as a feasible tool for epidemic control, further research is warranted.
The paper thus describes little more than an observational study. It shows that one region was less affected than another. I think it is quite clear that this could have many reasons which are unrelated to the homeopathic immunisation. Even the authors are cautious and speak in their conclusions not of a causal effect but of an “association”.
The 2012 data cited in the text remains unpublished; until it is available for public scrutiny, it is impossible to confirm that it is sound and meaningful.
Reference 8 refers to this article: Golden I, Bracho G. Adaptability of homœoprophylaxis in endemic, epidemic and stable background conditions. Homœopathic Links 2009; 22: 211-213. I have no access to this paper (if someone does, please fill us in) but, judging from both its title and the way it is described in the text, it does not seem to show reliable data about the efficacy of homeopathic immunisation.
So, is it true that “evidence does exist to support claims regarding the effectiveness of homeopathic immunisation”?
I do not think so!
Immunisation is by no means a trivial matter; wrong decisions in this area have the potential to cost the lives of millions. Therefore proofs of efficacy need to be published in peer-reviewed journals of high standing. These findings need then be criticised, replicated and re-criticised and re-replicated. Only when there is a wide consensus about the efficacy/safety or lack of efficacy/safety of a new form of immunisation, can it be generally accepted and implemented into clinical practice.
The current consensus about homeopathic immunisation is that it is nothing less than dangerous phantasy. Those who promote this quackery should be publicly exposed as charlatans of the worst kind.
Yesterday, BBC NEWS published the following interesting text about a BBC4 broadcast entitled ‘THE ROYAL ACTIVIST’ aired on the same day:
Prince Charles has been a well-known supporter of complementary medicine. According to a… former Labour cabinet minister, Peter Hain, it was a topic they shared an interest in.
“He had been constantly frustrated at his inability to persuade any health ministers anywhere that that was a good idea, and so he, as he once described it to me, found me unique from this point of view, in being somebody that actually agreed with him on this, and might want to deliver it.”
Mr Hain added: “When I was Secretary of State for Northern Ireland in 2005-7, he was delighted when I told him that since I was running the place I could more or less do what I wanted to do.***
“I was able to introduce a trial for complementary medicine on the NHS, and it had spectacularly good results, that people’s well-being and health was vastly improved.
“And when he learnt about this he was really enthusiastic and tried to persuade the Welsh government to do the same thing and the government in Whitehall to do the same thing for England, but not successfully,” added Mr Hain.
*** obviously there is no homeopathic remedy for megalomania (but that’s a different story)
SPECTACULARLY GOOD RESULTS?
Let’s have a look at the ‘trial’ and its results. An easily accessible report provides the following details about it:
From February 2007 to February 2008, Get Well UK ran the UK’s first government-backed complementary therapy pilot. Sixteen practitioners provided treatments including acupuncture, osteopathy and aromatherapy, to more than 700 patients at two GP practices in Belfast and Derry.
The BBC made an hour long documentary following our trials and tribulations, which was broadcast on BBC1 NI on 5 May 2008.
Aims and Objectives
The aim of the project was to pilot services integrating complementary medicine into existing primary care services in Northern Ireland. Get Well UK provided this pilot project for the Department for Health, Social Services and Public Safety (DHSSPS) during 2007.
The objectives were:
- To measure the health outcomes of the service and monitor health improvements.
- To redress inequalities in access to complementary medicine by providing therapies through the NHS, allowing access regardless of income.
- To contribute to best practise in the field of delivering complementary therapies through primary care.
- To provide work for suitably skilled and qualified practitioners.
- To increase patient satisfaction with quick access to expert care.
- To help patients learn skills to improve and retain their health.
- To free up GP time to work with other patients.
- To deliver the programme for 700 patients.
The results of the pilot were analysed by Social and Market Research, who produced this report.
The findings can be summarised as follows:
Following the pilot, 80% of patients reported an improvement in their symptoms, 64% took less time off work and 55% reduced their use of painkillers.
In the pilot, 713 patients with a range of ages and demographic backgrounds and either physical or mental health conditions were referred to various complementary and alternative medicine (CAM) therapies via nine GP practices in Belfast and Londonderry. Patients assessed their own health and wellbeing pre and post therapy and GPs and CAM practitioners also rated patients’ responses to treatment and the overall effectiveness of the scheme.
• 81% of patients reported an improvement in their physical health
• 79% reported an improvement in their mental health
• 84% of patients linked an improvement in their health and wellbeing directly to their CAM treatment
• In 65% of patient cases, GPs documented a health improvement, correlating closely to patient-reported improvements
• 94% of patients said they would recommend CAM to another patient with their condition
• 87% of patient indicated a desire to continue with their CAM treatment
Painkillers and medication
• Half of GPs reported prescribing less medication and all reported that patients had indicated to them that they needed less
• 62% of patients reported suffering from less pain
• 55% reported using less painkillers following treatment
• Patients using medication reduced from 75% before treatment to 61% after treatment
• 44% of those taking medication before treatment had reduced their use afterwards
Health service and social benefits
• 24% of patients who used health services prior to treatment (i.e. primary and secondary care, accident and emergency) reported using the services less after treatment
• 65% of GPs reported seeing the patient less following the CAM referral
• Half of GPs said the scheme had reduced their workload and 17% reported a financial saving for their practice
• Half of GPs said their patients were using secondary care services less.
Impressed? Well, in case you are, please consider this:
- there was no control group
- therefore it is not possible to attribute any of the outcomes to the alternative therapies offered
- they could have been due to placebo-effects
- or to the natural history of the disease
- or to regression towards the mean
- or to social desirability
- or to many other factors which are unrelated to the alternative treatments provided
- most outcome measures were not objectively verified
- the patients were self-selected
- they would all have had conventional treatments in parallel
- this ‘trial’ was of such poor quality that its findings were never published in a peer-reviewed journal
- this was not a ‘trial’ but a ‘pilot study’
- pilot studies are not normally for measuring outcomes but for testing the feasibility of a proper trial
- the research expertise of the investigators was close to zero
- the scientific community merely had pitiful smiles for this ‘trial’ when it was published
- neither Northern Ireland nor any other region implemented the programme despite its “spectacularly good results”.
So, is the whole ‘trial’ story an utterly irrelevant old hat?
Certainly not! Its true significance does not lie in the fact that a few amateurs are trying to push bogus treatments into the NHS via the flimsiest pseudo-research of the century. The true significance, I think, is that it shows how Prince Charles, once again, oversteps the boundaries of his constitutional role.
A recent meta-analysis evaluated the efficacy of acupuncture for treatment of irritable bowel syndrome (IBS) and arrived at bizarrely positive conclusions.
The authors state that they searched 4 electronic databases for double-blind, placebo-controlled trials investigating the efficacy of acupuncture in the management of IBS. Studies were screened for inclusion based on randomization, controls, and measurable outcomes reported.
Six RCTs were included in the meta-analysis, and 5 articles were of high quality. The pooled relative risk for clinical improvement with acupuncture was 1.75 (95%CI: 1.24-2.46, P = 0.001). Using two different statistical approaches, the authors confirmed the efficacy of acupuncture for treating IBS and concluded that acupuncture exhibits clinically and statistically significant control of IBS symptoms.
As IBS is a common and often difficult to treat condition, this would be great news! But is it true? We do not need to look far to find the embarrassing mistakes and – dare I say it? – lies on which this result was constructed.
The largest RCT included in this meta-analysis was neither placebo-controlled nor double blind; it was a pragmatic trial with the infamous ‘A+B versus B’ design. Here is the key part of its methods section: 116 patients were offered 10 weekly individualised acupuncture sessions plus usual care, 117 patients continued with usual care alone. Intriguingly, this was the ONLY one of the 6 RCTs with a significantly positive result!
The second largest study (as well as all the other trials) showed that acupuncture was no better than sham treatments. Here is the key quote from this trial: there was no statistically significant difference between acupuncture and sham acupuncture.
So, let me re-write the conclusions of this meta-analysis without spin, lies or hype: These results of this meta-analysis seem to indicate that:
- currently there are several RCTs testing whether acupuncture is an effective therapy for IBS,
- all the RCTs that adequately control for placebo-effects show no effectiveness of acupuncture,
- the only RCT that yields a positive result does not make any attempt to control for placebo-effects,
- this suggests that acupuncture is a placebo,
- it also demonstrates how misleading studies with the infamous ‘A+B versus B’ design can be,
- finally, this meta-analysis seems to be a prime example of scientific misconduct with the aim of creating a positive result out of data which are, in fact, negative.
Do you think that chiropractic is effective for asthma? I don’t – in fact, I know it isn’t because, in 2009, I have published a systematic review of the available RCTs which showed quite clearly that the best evidence suggested chiropractic was ineffective for that condition.
But this is clearly not true, might some enthusiasts reply. What is more, they can even refer to a 2010 systematic review which indicates that chiropractic is effective; its conclusions speak a very clear language: …the eight retrieved studies indicated that chiropractic care showed improvements in subjective measures and, to a lesser degree objective measures… How on earth can this be?
I would not be surprised, if chiropractors claimed the discrepancy is due to the fact that Prof Ernst is biased. Others might point out that the more recent review includes more studies and thus ought to be more reliable. The newer review does, in fact, have about twice the number of studies than mine.
How come? Were plenty of new RCTs published during the 12 months that lay between the two publications? The answer is NO. But why then the discrepant conclusions?
The answer is much less puzzling than you might think. The ‘alchemists of alternative medicine’ regularly succeed in smuggling non-evidence into such reviews in order to beautify the overall picture and confirm their wishful thinking. The case of chiropractic for asthma does by no means stand alone, but it is a classic example of how we are being misled by charlatans.
Anyone who reads the full text of the two reviews mentioned above will find that they do, in fact, include exactly the same amount of RCTs. The reason why they arrive at different conclusions is simple: the enthusiasts’ review added NON-EVIDENCE to the existing RCTs. To be precise, the authors included one case series, one case study, one survey, two randomized controlled trials (RCTs), one randomized patient and observer blinded cross-over trial, one single blind cross study design, and one self-reported impairment questionnaire.
Now, there is nothing wrong with case reports, case series, or surveys – except THEY TELL US NOTHING ABOUT EFFECTIVENESS. I would bet my last shirt that the authors know all of that; yet they make fairly firm and positive conclusions about effectiveness. As the RCT-results collectively happen to be negative, they even pretend that case reports etc. outweigh the findings of RCTs.
And why do they do that? Because they are interested in the truth, or because they don’t mind using alchemy in order to mislead us? Your guess is as good as mine.
Whenever a new trial of an alternative intervention emerges which fails to confirm the wishful thinking of the proponents of that therapy, the world of alternative medicine is in turmoil. What can be done about yet another piece of unfavourable evidence? The easiest solution would be to ignore it, of course – and this is precisely what is often tried. But this tactic usually proves to be unsatisfactory; it does not neutralise the new evidence, and each time someone brings it up, one has to stick one’s head back into the sand. Rather than denying its existence, it would be preferable to have a tool which invalidates the study in question once and for all.
The ‘fatal flaw’ solution is simpler than anticipated! Alternative treatments are ‘very special’, and this notion must be emphasised, blown up beyond all proportions and used cleverly to discredit studies with unfavourable outcomes: the trick is simply to claim that studies with unfavourable results have a ‘fatal flaw’ in the way the alternative treatment was applied. As only the experts in the ‘very special’ treatment in question are able to judge the adequacy of their therapy, nobody is allowed to doubt their verdict.
Take acupuncture, for instance; it is an ancient ‘art’ which only the very best will ever master – at least that is what we are being told. So, all the proponents need to do in order to invalidate a trial, is read the methods section of the paper in full detail and state ‘ex cathedra’ that the way acupuncture was done in this particular study is completely ridiculous. The wrong points were stimulated, or the right points were stimulated but not long enough [or too long], or the needling was too deep [or too shallow], or the type of stimulus employed was not as recommended by TCM experts, or the contra-indications were not observed etc. etc.
As nobody can tell a correct acupuncture from an incorrect one, this ‘fatal flaw’ method is fairly fool-proof. It is also ever so simple: acupuncture-fans do not necessarily study hard to find the ‘fatal flaw’, they only have to look at the result of a study – if it was favourable, the treatment was obviously done perfectly by highly experienced experts; if it was unfavourable, the therapists clearly must have been morons who picked up their acupuncture skills in a single weekend course. The reasons for this judgement can always be found or, if all else fails, invented.
And the end-result of the ‘fatal flaw’ method is most satisfactory; what is more, it can be applied to all alternative therapies – homeopathy, herbal medicine, reflexology, Reiki healing, colonic irrigation…the method works for all of them! What is even more, the ‘fatal flaw’ method is adaptable to other aspects of scientific investigations such that it fits every conceivable circumstance.
An article documenting the ‘fatal flaw’ has to be published, of course – but this is no problem! There are dozens of dodgy alternative medicine journals which are only too keen to print even the most far-fetched nonsense as long as it promotes alternative medicine in some way. Once this paper is published, the proponents of the therapy in question have a comfortable default position to rely on each time someone cites the unfavourable study: “WHAT NOT THAT STUDY AGAIN! THE TREATMENT HAS BEEN SHOWN TO BE ALL WRONG. NOBODY CAN EXPECT GOOD RESULTS FROM A THERAPY THAT WAS NOT CORRECTLY ADMINISTERED. IF YOU DON’T HAVE BETTER STUDIES TO SUPPORT YOUR ARGUMENTS, YOU BETTER SHUT UP.”
There might, in fact, be better studies – but chances are that the ‘other side’ has already documented a ‘fatal flaw’ in them too.
Yes, it is unlikely but true! I once was the hero of the world of energy healing, albeit for a short period only. An amusing story, I hope you agree.
Back in the late 1990s, we had decided to run two trials in this area. One of them was to test the efficacy of distant healing for the removal of ordinary warts, common viral infections of the skin which are quite harmless and usually disappear spontaneously. We had designed a rigorous study, obtained ethics approval and were in the midst of recruiting patients, when I suggested I could be the trial’s first participant, as I had noticed a tiny wart on my left foot. As patient-recruitment was sluggish at that stage, my co-workers consulted the protocol to check whether it might prevent me from taking part in my own trial. They came back with the good news that, as I was not involved in the running of the study, there was no reason for me to be excluded.
The next day, they ‘processed’ me like all the other wart sufferers of our investigation. My wart was measured, photographed and documented. A sealed envelope with my trial number was opened (in my absence, of course) by one of the trialists to see whether I would be in the experimental or the placebo group. The former patients were to receive ‘distant healing’ from a group of 10 experienced healers who had volunteered and felt confident to be able to cure warts. All they needed was a few details about each patients, they had confirmed. The placebo group received no such intervention. ‘Blinding’ the patient was easy in this trial; since they were not themselves involved in any healing-action, they could not know whether they were in the placebo or the verum group.
The treatment period lasted for several weeks during which time my wart was re-evaluated in regular intervals. When I had completed the study, final measurements were done, and I was told that I had been the recipient of ‘healing energy’ from the 10 healers during the past weeks. Not that I had felt any of it, and not that my wart had noticed it either: it was still there, completely unchanged.
I remember not being all that surprised…until, the next morning, when I noticed that my wart had disappeared! Gone without a trace!
Of course, I told my co-workers who were quite excited, re-photographed the spot where the wart had been and consulted the study protocol to determine what had to be done next. It turned out that we had made no provisions for events that might occur after the treatment period.
But somehow, this did not feel right, we all thought. So we decided to make a post-hoc addendum to our protocol which stipulated that all participants of our trial would be asked a few days after the end of the treatment whether any changes to their warts had been noted.
Meanwhile the healers had got wind of the professorial wart’s disappearance. They were delighted and quickly told other colleagues. In no time at all, the world of ‘distant healing’ had agreed that warts often reacted to their intervention with a slight delay – and they were pleased to hear that we had duly amended our protocol to adequately capture this important phenomenon. My ‘honest’ and ‘courageous’ action of acknowledging and documenting the disappearance of my wart was praised, and it was assumed that I was about to prove the efficacy of distant healing.
And that’s how I became their ‘hero’ – the sceptical professor who had now seen the light with his own eyes and experienced on his own body the incredible power of their ‘healing energy’.
Incredible it remained though: I was the only trial participant who lost his wart in this way. When we published this study, we concluded: Distant healing from experienced healers had no effect on the number or size of patients’ warts.
AND THAT’S WHEN I STOPPED BEING THEIR ‘HERO’.
Irritable bowel syndrome (IBS) is common and often difficult to treat – unless, of course, you consult a homeopath. Here is just one of virtually thousands of quotes from homeopaths available on the Internet: Homeopathic medicine can reduce Irritable Bowel Syndrome (IBS) symptoms by lowering food sensitivities and allergies. Homeopathy treats the patient as a whole and does not simply focus on the disease. Careful attention is given to the minute details about the presenting complaints, including the severity of diarrhea, constipation, pain, cramps, mucus in the stools, nausea, heartburn, emotional triggers and conventional laboratory findings. In addition, the patient’s eating habits, food preferences, thermal attributes and sleep patterns are noted. The patient’s family history and diseases, along with the patient’s emotions are discussed. Then the homeopathic practitioner will select the remedy that most closely matches the symptoms.
Such optimism might be refreshing, but is there any reason for it? Is homeopathy really an effective treatment for IBS? To answer this question, we now have a brand-new Cochrane review. The aim of this review was to assess the effectiveness and safety of homeopathic treatment for treating irritable bowel syndrome (IBS). (This type of statement always makes me a little suspicious; how on earth can anyone truly assess the safety of a treatment by looking at a few studies? This is NOT how one evaluates safety!) The authors conducted extensive literature searches to identify all RCTs, cohort and case-control studies that compared homeopathic treatment with placebo, other control treatments, or usual care in adults with IBS. The primary outcome was global improvement in IBS.
Three RCTs with a total of 213 participants were included. No cohort or case-control studies were identified. Two studies compared homeopathic remedies to placebos for constipation-predominant IBS. One study compared individualised homeopathic treatment to usual care defined as high doses of dicyclomine hydrochloride, faecal bulking agents and a high fibre diet. Due to the low quality of reporting, the risk of bias in all three studies was unclear on most criteria and high for some criteria.
A meta-analysis of two studies with a total of 129 participants with constipation-predominant IBS found a statistically significant difference in global improvement between the homeopathic ‘asafoetida’ and placebo at a short-term follow-up of two weeks. Seventy-three per cent of patients in the homeopathy group improved compared to 45% of placebo patients. There was no statistically significant difference in global improvement between the homeopathic asafoetida plus nux vomica compared to placebo. Sixty-eight per cent of patients in the homeopathy group improved compared to 52% of placebo patients.
The overall quality of the evidence was very low. There was no statistically significant difference between individualised homeopathic treatment and usual care for the outcome “feeling unwell”. None of the studies reported on adverse events (which, by the way, should be seen as a breech in research ethics on the part of the authors of the three primary studies).
The authors concluded that a pooled analysis of two small studies suggests a possible benefit for clinical homeopathy, using the remedy asafoetida, over placebo for people with constipation-predominant IBS. These results should be interpreted with caution due to the low quality of reporting in these trials, high or unknown risk of bias, short-term follow-up, and sparse data. One small study found no statistically difference between individualised homeopathy and usual care (defined as high doses of dicyclomine hydrochloride, faecal bulking agents and diet sheets advising a high fibre diet). No conclusions can be drawn from this study due to the low number of participants and the high risk of bias in this trial. In addition, it is likely that usual care has changed since this trial was conducted. Further high quality, adequately powered RCTs are required to assess the efficacy and safety of clinical and individualised homeopathy compared to placebo or usual care.
THIS REVIEW REQUIRES A FEW FURTHER COMMENTS, I THINK
Asafoetida, the remedy used in two of the studies, is a plant native to Pakistan, Iran and Afghanistan. It is used in Ayurvedic herbal medicine to treat colic, intestinal parasites and irritable bowel syndrome. In the ‘homeopathic’ trials, asafoetida was used in relatively low dilutions, one that still contains molecules. It is therefore debatable whether this was really homeopathy or whether it is more akin to herbal medicine – it was certainly not homeopathy with its typical ultra-high dilutions.
Regardless of this detail, the Cochrane review does hardly provide sound evidence for homeopathy’s efficacy. On the contrary, my reading of its findings is that the ‘possible benefit’ is NOT real but a false positive result caused by the serious limitations of the original studies. The authors stress that the apparently positive result ‘should be interpreted with caution’; that is certainly correct.
So, if you are a proponent of homeopathy, as the authors of the review seem to be, you will claim that homeopathy offers ‘possible benefits’ for IBS-sufferers. But if you are not convinced of the merits of homeopathy, you might suggest that the evidence is insufficient to recommend homeopathy. I imagine that IBS-sufferers might get as frustrated with such confusion as most scientists will be. Yet there is hope; the answer could be imminent: apparently, a new trial is to report its results within this year.
IS THIS NEW TRIAL GOING TO CONTRIBUTE MEANINGFULLY TO OUR KNOWLEDGE?
It is a three-armed study (same 1st author as in the Cochrane review) which, according to its authors, seeks to explore the effectiveness of individualised homeopathic treatment plus usual care compared to both an attention control plus usual care and usual care alone, for patients with IBS. (Why “explore” and not “determine”, I ask myself.) Patients are randomly selected to be offered, 5 sessions of homeopathic treatment plus usual care, 5 sessions of supportive listening plus usual care or usual care alone. (“To be offered” looks odd to me; does that mean patients are not blinded to the interventions? Yes, indeed it does.) The primary clinical outcome is the IBS Symptom Severity at 26 weeks. Analysis will be by intention to treat and will compare homeopathic treatment with usual care at 26 weeks as the primary analysis, and homeopathic treatment with supportive listening as an additional analysis.
Hold on…the primary analysis “will compare homeopathic treatment with usual care“. Are they pulling my leg? They just told me that patients will be “offered, 5 sessions of homeopathic treatment plus usual care… or usual care alone“.
Oh, I see! We are again dealing with an A+B versus B design, on top of it without patient- or therapist-blinding. This type of analysis cannot ever produce a negative result, even if the experimental treatment is a pure placebo: placebo + usual care is always more than usual care alone. IBS-patients will certainly experience benefit from having the homeopaths’ time, empathy and compassion – never mind the remedies they get from them. And for the secondary analyses, things do not seem to be much more rigorous either.
Do we really need more trials of this nature? The Cochrane review shows that we currently have three studies which are too flimsy to be interpretable. What difference will a further flimsy trial make in this situation? When will we stop wasting time and money on such useless ‘research’? All it can possibly achieve is that apologists of homeopathy will misinterpret the results and suggest that they demonstrate efficacy.
Obviously, I have not seen the data (they have not yet been published) but I think I can nevertheless predict the conclusions of the primary analysis of this trial; they will read something like this: HOMEOPATHY PROVED TO BE SIGNIFICANTLY MORE EFFECTIVE THAN USUAL CARE. I have asked the question before and I do it again: when does this sort of ‘research’ cross the line into the realm of scientific misconduct?