The aim of a new meta-analysis was to estimate the clinical effectiveness and safety of acupuncture for amnestic mild cognitive impairment (AMCI), the transitional stage between the normal memory loss of aging and dementia. Randomised controlled trials (RCTs) of acupuncture versus medical treatment for AMCI were identified using six electronic databases.
Five RCTs involving a total of 568 subjects were included. The methodological quality of the RCTs was generally poor. Participants receiving acupuncture had better outcomes than those receiving nimodipine with greater clinical efficacy rates (odds ratio (OR) 1.78, 95% CI 1.19 to 2.65; p<0.01), mini-mental state examination (MMSE) scores (mean difference (MD) 0.99, 95% CI 0.71 to 1.28; p<0.01), and picture recognition score (MD 2.12, 95% CI 1.48 to 2.75; p<0.01). Acupuncture used in conjunction with nimodipine significantly improved MMSE scores (MD 1.09, 95% CI 0.29 to 1.89; p<0.01) compared to nimodipine alone. Three trials reported adverse events.
The authors concluded that acupuncture appears effective for AMCI when used as an alternative or adjunctive treatment; however, caution must be exercised given the low methodological quality of included trials. Further, more rigorously designed studies are needed.
Meta-analyses like this one are, in my view, perfect examples for the ‘rubbish in, rubbish out’ principle of systematic reviews. This may seem like an unfair statement, so let me justify it by explaining the shortfalls of this specific paper.
The authors try to tell us that their aim was “to estimate the clinical effectiveness and safety of acupuncture…” While it might be possible to estimate the effectiveness of a therapy by pooling the data of a few RCTs, it is never possible to estimate its safety on such a basis. To conduct an assessment of therapeutic safety, one would need sample sizes that go two or three dimensions beyond those of RCTs. Thus safety assessments are best done by evaluating the evidence from all the available evidence, including case-reports, epidemiological investigations and observational studies.
The authors tell us that “two studies did not report whether any adverse events or side effects had occurred in the experimental or control groups.” This is a common and serious flaw of many acupuncture trials, and another important reason why RCTs cannot be used for evaluating the risks of acupuncture. Too many such studies simply don’t mention adverse effects at all. If they are then submitted to systematic reviews, they must generate a false positive picture about the safety of acupuncture. The absence of adverse effects reporting is a serious breach of research ethics. In the realm of acupuncture, it is so common, that many reviewers do not even bother to discuss this violation of medical ethics as a major issue.
The authors conclude that acupuncture is more effective than nimodipine. This sounds impressive – unless you happen to know that nimodipine is not supported by good evidence either. A Cochrane review provided no convincing evidence that nimodipine is a useful treatment for the symptoms of dementia, either unclassified or according to the major subtypes – Alzheimer’s disease, vascular, or mixed Alzheimer’s and vascular dementia.
The authors also conclude that acupuncture used in conjunction with nimodipine is better than nimodipine alone. This too might sound impressive – unless you realise that all the RCTs in question failed to control for the effects of placebo and the added attention given to the patients. This means that the findings reported here are consistent with acupuncture itself being totally devoid of therapeutic effects.
The authors are quite open about the paucity of RCTs and their mostly dismal methodological quality. Yet they arrive at fairly definitive conclusions regarding the therapeutic value of acupuncture. This is, in my view, a serious mistake: on the basis of a few poorly designed and poorly reported RCTs, one should never arrive at even tentatively positive conclusion. Any decent journal would not have published such misleading phraseology, and it is noteworthy that the paper in question appeared in a journal that has a long history of being hopelessly biased in favour of acupuncture.
Any of the above-mentioned flaws could already be fatal, but I have kept the most serious one for last. All the 5 RCTs that were included in the analyses were conducted in China by Chinese researchers and published in Chinese journals. It has been shown repeatedly that such studies hardly ever report anything other than positive results; no matter what conditions is being investigated, acupuncture turns out to be effective in the hands of Chinese trialists. This means that the result of such a study is clear even before the first patient has been recruited. Little wonder then that virtually all reviews of such trials – and there are dozens of then – arrive at conclusions similar to those formulated in the paper before us.
As I already said: rubbish in, rubbish out!
The German Association of Homeopaths (Deutscher Zentralverein Homoeopathischer Aerzte) just issued a press-release explaining that they have recently determined that homeopathy works.
Well, aren’t we relieved!
Otherwise, we would have had to assume they are all quacks.
Their statement is based on what they consider a thorough analysis of the published evidence. As the whole document is about 60 pages long, I will not bother you with all the details. Instead, I will focus on what they say about systematic reviews/meta-analyses in the press-release:
Eine Betrachtung der Meta-Analysen zur Homöopathie zeigt überwiegend statistisch signifikante Ergebnisse gegenüber Placebo, die auf eine spezifische Wirksamkeit potenzierter Arzneien hinweisen. Je nach den verwendeten Selektionskriterien werden hierbei unterschiedliche Studien in die Auswertung eingeschlossen. Diese Befunde werden von den Autoren der jeweiligen Meta-Analysen zum Teil stark relativiert. Die angeführten Vorbehalte entsprechen hierbei nicht immer den üblichen wissenschaftlichen Standards.
Let me translate this for you: An assessment of the meta-analyses of homeopathy shows mostly significant results compared to placebo which indicates a specific effectiveness of potentised remedies. Depending on the selection criteria, various studies are included in the evaluation. These results are relativized by the authors of the respective meta-analyses. The listed caveats do not always reflect the usual scientific standards.
You think my English has deteriorated or my brain gone soft? No, it’s their German! It makes almost no sense at all.
Therefore, I am afraid, we need to briefly go into the hefty document after all. Their chapter on meta-analyses concludes as follows: Insgesamt ergibt sich hinsichtlich der bis dato publizierten maßgeblichen Meta-Analysen zur Homöopathie, dass in vier von fünf Fällen tendenziell eine spezifische Wirksamkeit potenzierter Arzneimittel über Placebo hinaus erkennbar ist. That makes (linguistically) a little more sense: Overall, it emerges that the currently published decisive meta-analyses show, in 4 of 5 cases, that a specific effectiveness of potentised remedies is noticeable.
In other words, it is now proven, homeopathic remedies work beyond placebo!!!
But how can this be?
Did the NHMRC not just do a similar analysis concluding that “the evidence from research in humans does not show that homeopathy is effective for treating the range of health conditions considered… homeopathy should not be used to treat health conditions that are chronic, serious, or could become serious. People who choose homeopathy may put their health at risk if they reject or delay treatments for which there is good evidence for safety and effectiveness. People who are considering whether to use homeopathy should first get advice from a registered health practitioner. Those who use homeopathy should tell their health practitioner and should keep taking any prescribed treatments.
Obviously ‘down under’ they don’t know how to evaluate published data!
Or could it be that the Germans are mistaken? Or are they perhaps joking?
Let’s have a look!
The Germans selected (cherry-picked) 5 meta-analyses which they believed to be ‘decisive’, while the Australian panel of independent experts (funded by government) assessed 57 meta-analyses and systematic reviews (all they found via extensive literature searches).
But the German evaluation was done by homeopaths (and financed by a homeopathic lobby group)! And they understand homeopathy best and would not have a bias or conflict of interest, would they?[FOR A MORE COMPLETE ANALYSIS, SEE HERE (in German)]
Polycystic ovarian syndrome (PCOS) is a common condition characterised by oligo-amenorrhoea, infertility and hirsutism. Conventional treatment of PCOS includes a range of oral pharmacological agents, lifestyle changes and surgical modalities. Some studies have suggested that acupuncture might be helpful but the evidence is often flawed and the results are mixed. What is needed in such a situation is, of course, a systematic review.
The aim of this new Cochrane review was to assess the effectiveness and safety of acupuncture treatment of oligo/anovulatory women with polycystic ovarian syndrome (PCOS). The authors identified relevant studies from databases including the Cochrane Central Register of Controlled Trials (CENTRAL), Ovid MEDLINE, EMBASE, PsycINFO, CNKI and trial registries. The data are current to 19 October 2015.
They included randomised controlled trials (RCTs) that studied the efficacy of acupuncture treatment for oligo/anovulatory women with PCOS. We excluded quasi- or pseudo-RCTs. Primary outcomes were live birth and ovulation (primary outcomes), and secondary outcomes were clinical pregnancy, restoration of menstruation, multiple pregnancy, miscarriage and adverse events. We assessed the quality of the evidence using GRADE methods.
Two review authors independently selected the studies, extracted data and assessed risk of bias. They calculated Mantel-Haenszel odds ratios (ORs) and mean difference (MD) and 95% confidence intervals (CIs).
Five RCTs with 413 women were included. They compared true acupuncture versus sham acupuncture (two RCTs), true acupuncture versus relaxation (one RCT), true acupuncture versus clomiphene (one RCT) and electroacupuncture versus physical exercise (one RCT). Four of the studies were at high risk of bias in at least one domain. No study reported live birth rate. Two studies reported clinical pregnancy and found no evidence of a difference between true acupuncture and sham acupuncture (OR 2.72, 95% CI 0.69 to 10.77, two RCTs, 191 women, very low quality evidence). Three studies reported ovulation. One RCT reported number of women who had three ovulations during three months of treatment but not ovulation rate. One RCT found no evidence of a difference in mean ovulation rate between true and sham acupuncture (MD -0.03, 95% CI -0.14 to 0.08, one RCT, 84 women, very low quality evidence). However, one other RCT reported very low quality evidence to suggest that true acupuncture might be associated with higher ovulation frequency than relaxation (MD 0.35, 95% CI 0.14 to 0.56, one RCT, 28 women). Two studies reported menstrual frequency. One RCT reported true acupuncture reduced days between menstruation more than sham acupuncture (MD 220.35, 95% CI 252.85 to 187.85, 146 women). One RCT reported electroacupuncture increased menstrual frequency more than no intervention (0.37, 95% CI 0.21 to 0.53, 31 women). There was no evidence of a difference between the groups in adverse events. Evidence was very low quality with very wide CIs and very low event rates. Overall evidence was low or very low quality. The main limitations were failure to report important clinical outcomes and very serious imprecision.
The authors concluded that, thus far, only a limited number of RCTs have been reported. At present, there is insufficient evidence to support the use of acupuncture for treatment of ovulation disorders in women with PCOS.
This is, in my view, a rigorous assessment of the evidence leading to a clear conclusion. Foremost, I applaud the authors from the Faculty of Science, University of Technology Sydney for using such clear language. Such clarity seems to be getting a rare event in reviews of alternative medicine. To demonstrated this point, here are the most recent 5 systematic reviews which came up on my screen when I searched today Medline for ‘complementary alternative medicine, systematic review’.
The combination of TGP and LEF in treatment of RA presented the characteristics of notably decreasing the levels of laboratory indexes and higher safety in terms of liver function. However, this conclusion should be further investigated based on a larger sample size.
CHM as an adjunctive therapy is associated with a decreased risk of in-hospital mortality compared with WT in patients with AKI. Further studies with high quality and large sample size are needed to verify our conclusions.
As an important supplementary treatment, TCM may provide benefits in repair of injured spinal cord. With a general consensus that future clinical approaches will be diversified and a combination of multiple strategies, TCM is likely to attract greater attention in SCI treatment.
I think the phenomenon is fairly obvious: authors of such papers are far too often not able or willing to express the bottom line of their work openly. As systematic reviews are supposed to be the ultimate type of evidence, this trend is very worrying, I think. In my view, such conclusions merely display the bias of the authors. If the evidence is not convincingly positive (which it very rarely is), authors have an ethical obligation to clearly say so.
If they don’t do it, journal editors have the duty to correct the error. If neither of these actions happen, funding agencies should make sure that such teams get no further research money until they can demonstrate that they have learnt the lesson.
This may sound a bit drastic but I think such steps would be both necessary and urgent. The problem is now extremely common, and if we do not quickly implement some effective preventative measures, our scientific literature will become contaminated to the point of becoming useless. This surely would be a disaster that affects us all.
There can, of course, be several reasons for the evidence being not positive:
- there can be a paucity of data
- the results might be contradictory
- the trials might be open to bias
- some of the primary data might look suspicious
In all of these cases, the evidence would be not convincingly positive, and it would be wrong and unhelpful not to be frank about it. Beating about the bush, like so many authors nowadays do, is misleading, unhelpful, unethical and borderline fraudulent. Therefore it constitutes a disservice to everyone concerned.
This seems to be the question that occupies the minds of several homeopaths.
So was I!
Let me explain.
In 1997, Linde et al published their now famous meta-analysis of clinical trials of homeopathy which concluded that “The results of our meta-analysis are not compatible with the hypothesis that the clinical effects of homeopathy are completely due to placebo. However, we found insufficient evidence from these studies that homeopathy is clearly efficacious for any single clinical condition. Further research on homeopathy is warranted provided it is rigorous and systematic.”
This paper had several limitations which Linde was only too happy to admit. The authors therefore conducted a re-analysis which, even though published in an excellent journal, is rarely cited by homeopaths. Linde et al stated in their re-analysis of 2000: “there was clear evidence that studies with better methodological quality tended to yield less positive results.” It was this phenomenon that prompted me and my colleague Max Pittler to publish a ‘letter to the editor’ which now – 15 years later – seems the stone of homeopathic contention.
A blog-post by a believer in homeopathy even asks the interesting question: Did Professor Ernst Sell His Soul to Big Pharma? It continues as follows:
Edzard Ernst is an anti-homeopath who spent his career attacking traditional medicine. In 1993 he became Professor of Complementary Medicine at the University of Exeter. He is often described as the first professor of complementary medicine, but the title he assumed should have fooled no-one. His aim was to discredit medical therapies, notably homeopathy, and he then published some 700 papers in ‘scientific’ journals to do so.
Now, Professor Robert Hahn, in his blog, has made an assessment of the quality of his work… In the interests of the honesty and integrity in science, it is an important assessment. It shows, in his view, how science has been taken over by ideology (or as I would suggest, more accurately, the financial interests of Big Corporations, in this case, Big Pharma). The blog indicates that in order to demonstrate that homeopathy is ineffective, over 95% of scientific research into homeopathy has to be discarded or removed!
So for those people who, like myself, cannot read the original German, here is an English translation of the blog…
“I have never seen a science writer so blatantly biased as Edzard Ernst: his work should not be considered of any worth at all, and discarded” finds Sweden’s Professor Robert Hahn, a leading medical scientist, physician, and Professor of Anaesthesia and Intensive Care at the University of Linköping, Sweden.
Hahn determined therefore to analyze for himself the ‘research’ which supposedly demonstrated homeopathy to be ineffective, and reached the shocking conclusion that:
“only by discarding 98% of homeopathy trials and carrying out a statistical meta-analysis on the remaining 2% negative studies, can one ‘prove’ that homeopathy is ineffective”.
In other words, all supposedly negative homeopathic meta-analyses which opponents of homeopathy have relied on, are scientifically bogus…
Who can you trust? We can begin by disregarding Edzard Ernst. I have read several other studies that he has published, and they are all untrustworthy. His work should be discarded…
In the case of homeopathy, one should stick with what the evidence reveals. And the evidence is that only by removing 95-98% of all studies is the effectiveness of homeopathy not demonstrable…
So, now you are wondering, I am sure: HOW MUCH DID HE GET FOR SELLING HIS SOUL TO BIG PHARMA?
No? You are wondering 1) who this brilliant Swedish scientist, Prof Hahn, is and 2) what article of mine he is criticising? Alright, I will try to enlighten you.
Here I can rely on a comment posted on my blog some time ago by someone who can read Swedish (thank you Bjorn). He commented about Hahn as follows:
A renowned director of medical research with well over 300 publications on anesthesia and intensive care and 16 graduated PhD students under his mentorship, who has been leading a life on the side, blogging and writing about spiritualism, and alternative medicine and now ventures on a public crusade for resurrecting the failing realm of homeopathy!?! Unbelievable!
I was unaware of this person before, even if I have lived and worked in Sweden for decades.
I have spent the evening looking up his net-track and at his blog at roberthahn.nu (in Swedish).
I will try to summarise some first impressions:
Hahn is evidently deeply religious and there is the usual, unmistakably narcissistic aura over his writings and sayings. He is religiously confident that there is more to this world than what can be measured and sensed. In effect, he seems to believe that homeopathy (as well as alternative medical methods in general) must work because there are people who say they have experienced it and denying the possibility is akin to heresy (not his wording but the essence of his writing).
He has, along with his wife, authored at least three books on spiritual matters with titles such as (my translations) “Clear replies from the spiritual world” and “Connections of souls”.
He has a serious issue with skeptics and goes on at length about how they are dishonest bluffers[sic] who willfully cherry-pick and misinterpret evidence to fit their preconceived beliefs.
He feels that desperate patients should generally be allowed the chance that alternative methods may offer.
He believes firmly in former-life memories, including his own, which he claims he has found verification for in an ancient Italian parchment.
His main arguments for homeopathy are Claus Linde’s meta analyses and the sheer number of homeopathic research that he firmly believes shows it being superior to placebo, a fact that (in his opinion) shows it has a biological effect. Shang’s work from 2005 he dismisses as seriously flawed.
He also points to individual research like this as credible proof of the biologic effect of remedies.
He somewhat surprisingly denies recommending homeopathy despite being convinced of its effect and maintains that he wants better, more problem oriented and disease specific studies to clarify its applicability. (my interpretation)
If it weren’t for his track record of genuine, acknowledged medical research and him being a renowned authority in a genuine, scientific medical field, this man would be an ordinary, religiously devout quack.
What strikes me as perhaps telling of a consequence of his “exoscientific” activity, is that Hahn, who holds the position of research director at a large city trauma and emergency hospital is an “adjungerad professor”, which is (usually) a part time, time limited, externally financed professorial position, while any Swedish medical doctor with his very extensive formal merits would very likely hold a full professorship at an academic institution.
END OF QUOTE
MY 2000 PAPER THAT SEEMS TO IRRITATE HAHN
This was a short ‘letter to the editor’ by Ernst and Pittler published in the J Clin Epidemiol commenting on the above-mentioned re-analysis by Linde et al which was published in the same journal. As its text is not available on-line, I re-type parts of it here:
In an interesting re-analysis of their meta-analysis of clinical trials of homeopathy, Linde et al conclude that there is no linear relationship between quality scores and study outcome. We have simply re-plotted their data and arrive at a different conclusion. There is an almost perfect correlation between the odds ratio and the Jadad score between the range of 1-4… [some technical explanations follow which I omit]…Linde et al can be seen as the ultimate epidemiological proof that homeopathy is, in fact, a placebo.
And that is, as far as I can see, the whole mysterious story. I cannot even draw a conclusion – all I can do is to ask a question:
DOES ANYONE UNDERSTAND WHAT THEY ARE GOING ON ABOUT?
Being constantly on the look-out for new, good quality articles on alternative therapy which suggest that a treatment might actually work, I was excited to find not just one or two but four recent publications on an old favourite of mine: massage therapy.
The first paper described a study aimed to investigate the effect of whole body massage on the vital signs, Glasgow Coma Scale (GCS) scores and arterial blood gases (ABG) in trauma ICU patients.
In a randomized, double-blind trial, 108 trauma ICU patients received whole body massage or routine care only. The patients vital signs; systolic blood pressure (SBP), diastolic blood pressure (DBP), respiratory rate (RR), pulse rate (PR), Temperature (T), GCS score and ABG parameters were measured in both groups before the intervention and 1 hour and 3 hours after the intervention. The patient in experimental group received full body massage in 45 minute by a family member.
Significant differences were observed between experimental and control groups in SBP 1 hour and 3 hours after intervention, DBP, RR and PR 1 hour after intervention, and GCS 1 hour and 3 hours after intervention. Significant differences were also observed between experimental and control groups in O2 saturation, PH and pO2. No significant differences between experimental and control groups were noted in Temperature, pCO2 and HCO3.
The authors concluded that massage therapy is a safe and effective treatment in intensive care units to reduce patient’s physical and psychological problems. Therefore the use of massage therapy is recommended to clinical practice as a routine method.
The second paper reported a clinical trial on 66 male and female nurses working in intensive care units of Isfahan University of Medical Sciences, Iran.
Patients were randomly divided into experimental and control groups. The Occupational Stress Inventory (OSI) (Osipow and Spokane, 1987) was completed by participants of the two groups before, immediately after, and 2 weeks after the intervention. Swedish massage was performed on participants of the experimental group for 25 min in each session, twice a week for 4 weeks.
Results showed a significant difference in favour of the massage therapy in overall mean occupation stress scores between experimental and control groups two weeks after the intervention.
The authors concluded that it is recommended that massage, as a valuable noninvasive method, be used for nurses in intensive care units to reduce their stress, promote mental health, and prevent the decrease in quality of nursing work life.
The third paper described a randomized controlled trial evaluating the effects of post-operative massage in patients undergoing abdominal colorectal surgery.
One hundred twenty-seven patients were randomized to receive a 20-min massage or social visit and relaxation session on postoperative days 2 and 3. Vital signs and psychological well-being (pain, tension, anxiety, satisfaction with care, relaxation) were assessed before and after each intervention.
Post-operative massage significantly improved the patients’ perception of pain, tension, and anxiety, but overall satisfaction was unchanged.
The authors concluded that massage may be beneficial during postoperative recovery for patients undergoing abdominal colorectal surgery. Further studies are warranted to optimize timing and duration and to determine other benefits in this clinical setting.
The fourth paper reported a systematic review was to evaluate the effectiveness of massage on the short- and long-term outcomes of pre-term infants.
Literature searches were conducted using the PRISMA framework. Validity of included studies was assessed using criteria defined by the Cochrane Collaboration. Assessments were carried out independently by two reviewers with a third reviewer to resolve differences.
Thirty-four studies met the inclusion criteria, 3 were quasi-experimental, 1 was a pilot study, and the remaining 30 were RCTs. The outcomes that could be used in the meta-analysis and found in more than three studies suggested that massage improved daily weight gain by 0.53 g, and resulted in a significant improvement in mental scores by 7.89 points. There were no significant effects on length of hospital stay, caloric intake, or weight at discharge. Other outcomes were not analyzed either because the units of measurement varied between studies, or because means and standard deviations were not provided by the authors. The quality of the studies was variable with methods of randomization and blinding of assessment unclear in 18 of the 34 trials.
The authors concluded that massage therapy could be a comforting measure for infants in the NICU to improve weight gain and enhance mental development. However, the high heterogeneity, the weak quality in some studies, and the lack of a scientific association between massage and developmental outcomes preclude making definite recommendations and highlight the need for further RCTs to contribute to the existing body of knowledge.
I am not saying that these articles are flawless, nor that I agree with all of their conclusion. What I am trying to indicate is that we finally have here an alternative therapy that is promising.
When I worked in Germany and later in Austria, massage was considered to be entirely mainstream. It was only after I had moved to the UK when I realised that, in English-speaking countries, it is mostly considered to be alternative. Perhaps this classification is wrong?
Perhaps we should differentiate according to what type of massage we are talking about. In the realm of alternative medicine – and not just there, I suppose – this seems good advice indeed.
The above papers are about classical massage therapy, but there are some types pf massage which are less than conventional: aura-massage, Marma massage, Indian head massage, shiatsu etc. etc. the list seems endless. These are alternative in more than one sense, and they have one thing in common: there is, as far as I can see, no good evidence to show that they do anything to human health.
My conclusion therefore is that, even with something as common as massage therapy, we need to be careful not to be roped in by the charlatans.
Of all alternative treatments, aromatherapy (i.e. the application of essential oils to the body, usually by gentle massage or simply inhalation) seems to be the most popular. This is perhaps understandable because it certainly is an agreeable form of ‘pampering’ for someone in need of come TLC. But is aromatherapy more than that? Is it truly a ‘THERAPY’?
A recent systematic review was aimed at evaluating the existing data on aromatherapy interventions as a means of improving the quality of sleep. Electronic literature searches were performed to identify relevant studies published between 2000 and August 2013. Randomized controlled and quasi-experimental trials that included aromatherapy for the improvement of sleep quality were considered for inclusion. Of the 245 publications identified, 13 studies met the inclusion criteria, and 12 studies could be used for a meta-analysis.
The meta-analysis of the 12 studies revealed that the use of aromatherapy was effective in improving sleep quality. Subgroup analysis showed that inhalation aromatherapy was more effective than aromatherapy applied via massage.
The authors concluded that readily available aromatherapy treatments appear to be effective and promote sleep. Thus, it is essential to develop specific guidelines for the efficient use of aromatherapy.
Perfect! Let’s all rush out and get some essential oils for inhalation to improve our sleep (remarkably, the results imply that aroma therapists are redundant!).
Not so fast! As I see it, there are several important caveats we might want to consider before spending our money this way:
- Why did this review focus on such a small time-frame? (Systematic reviews should include all the available evidence of a pre-defined quality.)
- The quality of the included studies was often very poor, and therefore the overall conclusion cannot be definitive.
- The effect size of armoatherapy is small. In 2000, we published a similar review and concluded that aromatherapy has a mild, transient anxiolytic effect. Based on a critical assessment of the six studies relating to relaxation, the effects of aromatherapy are probably not strong enough for it to be considered for the treatment of anxiety. The hypothesis that it is effective for any other indication is not supported by the findings of rigorous clinical trials.
- It seems uncertain which essential oil is best suited for this indication.
- Aromatherapy is not always entirely free of risks. Another of our reviews showed that aromatherapy has the potential to cause adverse effects some of which are serious. Their frequency remains unknown. Lack of sufficiently convincing evidence regarding the effectiveness of aromatherapy combined with its potential to cause adverse effects questions the usefulness of this modality in any condition.
- There are several effective ways for improving sleep when needed; we need to know how aromatherapy compares to established treatments for that indication.
All in all, I think stronger evidence is required that aromatherapy is more that pampering.
One could define alternative medicine by the fact that it is used almost exclusively for conditions for which conventional medicine does not have an effective and reasonably safe cure. Once such a treatment has been found, few patients would look for an alternative.
Alzheimer’s disease (AD) is certainly one such condition. Despite intensive research, we are still far from being able to cure it. It is thus not really surprising that AD patients and their carers are bombarded with the promotion of all sorts of alternative treatments. They must feel bewildered by the choice and all too often they fall victim to irresponsible quacks.
Acupuncture is certainly an alternative therapy that is frequently claimed to help AD patients. One of the first websites that I came across, for instance, stated boldly: acupuncture improves memory and prevents degradation of brain tissue.
But is there good evidence to support such claims? To answer this question, we need a systematic review of the trial data. Fortunately, such a paper has just been published.
The objective of this review was to assess the effectiveness and safety of acupuncture for treating AD. Eight electronic databases were searched from their inception to June 2014. Randomized clinical trials (RCTs) with AD treated by acupuncture or by acupuncture combined with drugs were included. Two authors extracted data independently.
Ten RCTs with a total of 585 participants were included in a meta-analysis. The combined results of 6 trials showed that acupuncture was better than drugs at improving scores on the Mini Mental State Examination (MMSE) scale. Evidence from the pooled results of 3 trials showed that acupuncture plus donepezil was more effective than donepezil alone at improving the MMSE scale score. Only 2 trials reported the incidence of adverse reactions related to acupuncture. Seven patients had adverse reactions related to acupuncture during or after treatment; the reactions were described as tolerable and not severe.
The Chinese authors of this review concluded that acupuncture may be more effective than drugs and may enhance the effect of drugs for treating AD in terms of improving cognitive function. Acupuncture may also be more effective than drugs at improving AD patients’ ability to carry out their daily lives. Moreover, acupuncture is safe for treating people with AD.
Anyone reading this and having a friend or family member who is affected by AD will think that acupuncture is the solution and warmly recommend trying this highly promising option. I would, however, caution to remain realistic. Like so very many systematic reviews of acupuncture or other forms of TCM that are currently flooding the medical literature, this assessment of the evidence has to be taken with more than just a pinch of salt:
- As far as I can see, there is no biological plausibility or mechanism for the assumption that acupuncture can do anything for AD patients.
- The abstract fails to mention that the trials were of poor methodological quality and that such studies tend to generate false-positive findings.
- The trials had small sample sizes.
- They were mostly not blinded.
- They were mostly conducted in China, and we know that almost 100% of all acupuncture studies from that country draw positive conclusions.
- Only two trials reported about adverse effects which is, in my view, a sign of violation of research ethics.
As I already mentioned, we are currently being flooded with such dangerously misleading reviews of Chinese primary studies which are of such dubious quality that one could do probably nothing better than to ignore them completely.
Isn’t that a bit harsh? Perhaps, but I am seriously worried that such papers cause real harm:
- They might motivate some to try acupuncture and give up conventional treatments which can be helpful symptomatically.
- They might prompt some families to spend sizable amounts of money for no real benefit.
- They might initiate further research into this area, thus drawing money away from research into much more promising avenues.
IT IS HIGH TIME THAT RESEARCHERS START THINKING CRITICALLY, PEER-REVIEWERS DO THEIR JOB PROPERLY, AND JOURNAL EDITORS STOP PUBLISHING SUCH MISLEADING ARTICLES.
Nonspecific neck pain is extremely common, often disabling, and very costly for us all. If we believe those who earn their money with them, effective treatments for the condition abound. One of these therapies is osteopathy. But does osteopathic manipulation/mobilisation really work?
The objective of a recent review (the link I originally put in here does not work, I will supply a new one as soon as the article becomes available on Medline) was to find out. Specifically, the authors wanted to assess the effectiveness of osteopathic manipulative treatment (OMT) in the management of chronic nonspecific neck pain regarding pain, functional status, and adverse events.
Electronic literature searches unrestricted by language were performed in March 2014. A manual search of reference lists and personal communication with experts identified additional studies. Only randomized clinical trials (RCTs) were included, and studies of specific neck pain or single treatment techniques were excluded. Primary outcomes were pain and functional status, and secondary outcome was adverse events.
Studies were independently reviewed using a standardized data extraction form. Mean difference (MD) or standard mean difference (SMD) with 95% confidence intervals (CIs) and overall effect size were calculated for primary outcomes. GRADE was used to assess quality of the evidence.
Of 299 identified articles, 18 were evaluated and 15 excluded. The three included RCTs had low risk of bias. The results show that moderate-quality evidence suggested OMT had a significant and clinically relevant effect on pain relief (MD: -13.04, 95% CI: -20.64 to -5.44) in chronic nonspecific neck pain, and moderate-quality evidence suggested a non-significant difference in favour of OMT for functional status (SMD: -0.38, 95% CI: -0.88 to -0.11). No serious adverse events were reported.
The authors concluded that, based on the three included studies, the review suggested clinically relevant effects of OMT for reducing pain in patients with chronic nonspecific neck pain. Given the small sample sizes, different comparison groups, and lack of long-term measurements in the few available studies, larger, high-quality randomized controlled trials with robust comparison groups are recommended.
Yet again I am taken aback by several things simultaneously:
- the extreme paucity of RCTs, particularly considering that neck pain is one of the main indication for osteopaths,
- the rather uncritical text by the authors,
- the nonsensical conclusions.
Let me offer my own conclusions which are, I hope, a little more realistic:
GIVEN THE PAUCITY OF THE RCTs AND THEIR SMALL SAMPLE SIZES, IT IS NOT POSSIBLE TO CLAIM THAT OMT FOR NONSPECIFIC NECK PAIN IS AN EVIDENCE-BASED APPROACH
Distant healing is one of the most bizarre yet popular forms of alternative medicine. Healers claim they can transmit ‘healing energy’ towards patients to enable them to heal themselves. There have been many trials testing the effectiveness of the method, and the general consensus amongst critical thinkers is that all variations of ‘energy healing’ rely entirely on a placebo response. A recent and widely publicised paper seems to challenge this view.
This article has, according to its authors, two aims. Firstly it reviews healing studies that involved biological systems other than ‘whole’ humans (e.g., studies of plants or cell cultures) that were less susceptible to placebo-like effects. Secondly, it presents a systematic review of clinical trials on human patients receiving distant healing.
All the included studies examined the effects upon a biological system of the explicit intention to improve the wellbeing of that target; 49 non-whole human studies and 57 whole human studies were included.
The combined weighted effect size for non-whole human studies yielded a highly significant (r = 0.258) result in favour of distant healing. However, outcomes were heterogeneous and correlated with blind ratings of study quality; 22 studies that met minimum quality thresholds gave a reduced but still significant weighted r of 0.115.
Whole human studies yielded a small but significant effect size of r = .203. Outcomes were again heterogeneous, and correlated with methodological quality ratings; 27 studies that met threshold quality levels gave an r = .224.
From these findings, the authors drew the following conclusions: Results suggest that subjects in the active condition exhibit a significant improvement in wellbeing relative to control subjects under circumstances that do not seem to be susceptible to placebo and expectancy effects. Findings with the whole human database suggests that the effect is not dependent upon the previous inclusion of suspect studies and is robust enough to accommodate some high profile failures to replicate. Both databases show problems with heterogeneity and with study quality and recommendations are made for necessary standards for future replication attempts.
In a press release, the authors warned: the data need to be treated with some caution in view of the poor quality of many studies and the negative publishing bias; however, our results do show a significant effect of healing intention on both human and non-human living systems (where expectation and placebo effects cannot be the cause), indicating that healing intention can be of value.
My thoughts on this article are not very complimentary, I am afraid. The problems are, it seems to me, too numerous to discuss in detail:
- The article is written such that it is exceedingly difficult to make sense of it.
- It was published in a journal which is not exactly known for its cutting edge science; this may seem a petty point but I think it is nevertheless important: if distant healing works, we are confronted with a revolution in the understanding of nature – and surely such a finding should not be buried in a journal that hardly anyone reads.
- The authors seem embarrassingly inexperienced in conducting and publishing systematic reviews.
- There is very little (self-) critical input in the write-up.
- A critical attitude is necessary, as the primary studies tend to be by evangelic believers in and amateur enthusiasts of healing.
- The article has no data table where the reader might learn the details about the primary studies included in the review.
- It also has no table to inform us in sufficient detail about the quality assessment of the included trials.
- It seems to me that some published studies of distant healing are missing.
- The authors ignored all studies that were not published in English.
- The method section lacks detail, and it would therefore be impossible to conduct an independent replication.
- Even if one ignored all the above problems, the effect sizes are small and would not be clinically important.
- The research was sponsored by the ‘Confederation of Healing Organisations’ and some of the comments look as though the sponsor had a strong influence on the phraseology of the article.
Given these reservations, my conclusion from an analysis of the primary studies of distant healing would be dramatically different from the one published by the authors: DESPITE A SIZABLE AMOUNT OF PRIMARY STUDIES ON THE SUBJECT, THE EFFECTIVENESS OF DISTANT HEALING REMAINS UNPROVEN. AS THIS THERAPY IS BAR OF ANY BIOLOGICAL PLAUSIBILITY, FURTHER RESEARCH IN THIS AREA SEEMS NOT WARRANTED.
The discussion whether acupuncture is more than a placebo is as long as it is heated. Crucially, it is also quite tedious, tiresome and unproductive, not least because no resolution seems to be in sight. Whenever researchers develop an apparently credible placebo and the results of clinical trials are not what acupuncturists had hoped for, the therapists claim that the placebo is, after all, not inert and the negative findings must be due to the fact that both placebo and real acupuncture are effective.
Laser acupuncture (acupoint stimulation not with needle-insertion but with laser light) offers a possible way out of this dilemma. It is relatively easy to make a placebo laser that looks convincing to all parties concerned but is a pure and inert placebo. Many trials have been conducted following this concept, and it is therefore highly relevant to ask what the totality of this evidence suggests.
A recent systematic review did just that; specifically, it aimed to evaluate the effects of laser acupuncture on pain and functional outcomes when it is used to treat musculoskeletal disorders.
Extensive literature searches were used to identify all RCTs employing laser acupuncture. A meta-analysis was performed by calculating the standardized mean differences and 95% confidence intervals, to evaluate the effect of laser acupuncture on pain and functional outcomes. Included studies were assessed in terms of their methodological quality and appropriateness of laser parameters.
Forty-nine RCTs met the inclusion criteria. Two-thirds (31/49) of these studies reported positive effects. All of them were rated as being of high methodological quality and all of them included sufficient details about the lasers used. Negative or inconclusive studies mostly failed to demonstrate these features. For all diagnostic subgroups, positive effects for both pain and functional outcomes were more consistently seen at long-term follow-up rather than immediately after treatment.
The authors concluded that moderate-quality evidence supports the effectiveness of laser acupuncture in managing musculoskeletal pain when applied in an appropriate treatment dosage; however, the positive effects are seen only at long-term follow-up and not immediately after the cessation of treatment.
Surprised? Well, I am!
This is a meta-analysis I always wanted to conduct and never came round to doing. Using the ‘trick’ of laser acupuncture, it is possible to fully blind patients, clinicians and data evaluators. This eliminates the most obvious sources of bias in such studies. Those who are convinced that acupuncture is a pure placebo would therefore expect a negative overall result.
But the result is quite clearly positive! How can this be? I can see three options:
- The meta-analysis could be biased and the result might therefore be false-positive. I looked hard but could not find any significant flaws.
- The primary studies might be wrong, fraudulent etc. I did not see any obvious signs for this to be so.
- Acupuncture might be more than a placebo after all. This notion might be unacceptable to sceptics.
I invite anyone who sufficiently understands clinical trial methodology to scrutinise the data closely and tell us which of the three possibilities is the correct one.