bias
A German homeopathic journal, Zeitschrift Homoeopathie, has just published the following interesting article entitled HOMEOPATHIC DOCTORS HELP IN LIBERIA. It provides details about the international team of homeopaths that travelled to Liberia to cure Ebola. Here I take the liberty of translating it from German into English. As most of it is fairly self-explanatory, I abstain from any comments of my own – however, I am sure that my readers will want to add their views.
In mid-October, an international team of 4 doctors travelled to the West African country for three weeks. The mission in a hospital in Ganta, a town with about 40 000 inhabitants on the border to Guinea, ended as planned on 7 November. The exercise was organised by the World Association of Homeopathic Doctors, the Liga Medicorum Homoeopathica Internationalis (LMHI), with support of by the German Central Association of Homeopathic Doctors. The aim was to support the local doctors in the care of the population and, if possible, also to help in the fight against the Ebola epidemic. The costs for the three weeks’ stay were financed mostly through donations from homeopathic doctors.
“We know that we were invited mainly as well-trained doctors to Liberia, and that or experience in homeopathy was asked for only as a secondary issue”, stresses Cornelia Bajic, first chairperson of the DZVhA (German Central Association of Homeopathic Doctors). The doctors from India, USA, Switzerland and Germany were able to employ their expertise in several wards of the hospital, to help patients, and to support their Liberian colleagues. It was planned to use and document the homeopathic treatment of Ebola-patients as an adjunct to the WHO prescribed standard treatment. “Our experience from the treatment of other epidemics in the history of medicine allows the conclusion that a homeopathic treatment might significantly reduce the mortality of Ebola patients”, judges Bajic. The successful use of homeopathic remedies has been documented for example in Cholera, Diphtheria or Yellow Fever.
Overview of the studies related to the homeopathic treatment of epidemics
In Ganta, the doctors of the LMHI team treated patients with “at times most serious diseases, particularly inflammatory conditions, children with Typhus, meningitis, pneumonias, and unclear fevers – each time only under the supervision of the local doctor in charge”, reports Dr Ortrud Lindemann, who also worked obstetrically in Ganta. The medical specialist reports after her return: “When we had been 10 days in the hospital, the successes had become known, and the patients stood in queues to get treated by us.” The homeopathic doctors received thanks from the Ganta hospital for their work, it was said that it had been helpful for the patients and a blessing for the employees of the hospital.
POLITICAL CONSIDERATIONS MORE IMPORTANT THAN MEDICAL TREATMENT?
This first LMHI team of doctors was forbidden to care for patients from the “Ebola Treatment Unit”. The decision was based on an order of the WHO. A team of Cuban doctors was also waiting in vain for being allowed to work. “We are dealing here with a dangerous epidemic and a large number of seriously ill patients. And despite a striking lack of doctors in West Africa political considerations are more important than the treatment of these patients”, criticises the DZVhA chairperson Bajic. Now a second team is to travel to Ganta to support the local doctors.
The volume of medical research, as listed on Medline, is huge and increases steadily each year. This phenomenon can easily be observed with simple Medline searches. If we use search terms related to conventional medicine, we find near linear increases in the number of articles (here I do not make a distinction between types of articles) published in each area over time, invariably with a peak in 2013, the last year for which Medline listing is currently complete. Three examples will suffice:
PHARMACOTHERAPY 117 414 articles in 2013
PHARMACOLOGY 210 228 articles in 2013
ADVERSE EFFECTS 86 067 articles in 2013
Some of the above subjects are obviously heavily industry-dependent and thus perhaps not typical of the volume of research in health care generally. Let’s therefore look up three fields where there is no such powerful industry to support research:
PSYCHOTHERAPY 7 208 articles in 2013
PHYSIOTHERAPY 7 713 articles in 2013
SURGERY 154 417 articles in 2013
Now, if we conduct similar searches for topics related to alternative medicine, the picture changes in at least three remarkable ways: 1) there is no linear increase of the volume per year; instead the curves look flat and shapeless (the only exception is ‘herbal medicine’ where the increase even looks exponential). 2) The absolute volume does not necessarily peak in 2013 (exceptions are ‘acupuncture’ and ‘herbal medicine’). 3) The number of articles in the year with the most articles (as listed below) is small or even tiny:
ACUPUNCTURE 1 491 articles in 2013
CHIROPRACTIC 283 articles in 2011
HERBAL MEDICINE 2 503 articles in 2013
HOMEOPATHY 233 articles in 2005
NATUROPATHY 69 articles in 2010
You may think: so what? But I find these figures intriguing. They demonstrate that the research output in alternative medicine is minimal compared to that in conventional medicine. Moreover, they imply that this output is not only not increasing steadily, as it is in conventional medicine, but in the case of chiropractic, homeopathy and naturopathy, it has recently been decreasing.
To put this into context, we need to know that:
- there is a plethora of journals dedicated to alternative medicine which are keen to publish all sorts of articles,
- the peer-review process of most of these journals seems farcically poor,
- as a result, the quality of the research into alternative medicine is often dismal, as regularly disclosed on this blog,
- enthusiasts of alternative medicine often see rigorous research into their subject as a dangerous threat: it might disprove their prior beliefs.
In their defence, proponents of alternative medicine would probably claim that the low volume of research is due to a severe and unfair lack of funding. However, I fail to see how this can be the sole or even the main explanation: areas of conventional medicine that do not have industry support seem to manage a much higher output than alternative medicine (and I should stress that I have chosen 5 sections within alternative medicine that are associated with the highest number of articles per year). Research in these areas is usually sponsored by charitable and government sources, and it needs to be stressed that these are open to any researcher who submits good science.
What follows, I think, is simple: in general, alternative medicine advocates have little interest in research and even less expertise to conduct it.
Reiki is a form of energy healing that evidently has been getting so popular that, according to the ‘Shropshire Star’, even stressed hedgehogs are now being treated with this therapy. In case you argue that this publication is not cutting edge when it comes to reporting of scientific advances, you may have a point. So, let us see what evidence we find on this amazing intervention.
A recent systematic review of the therapeutic effects of Reiki concludes that the serious methodological and reporting limitations of limited existing Reiki studies preclude a definitive conclusion on its effectiveness. High-quality randomized controlled trials are needed to address the effectiveness of Reiki over placebo. Considering that this article was published in the JOURNAL OF ALTERNATIVE AND COMPLEMENTARY MEDICINE, this is a fairly damming verdict. The notion that Reiki is but a theatrical placebo recently received more support from a new clinical trial.
This pilot study examined the effects of Reiki therapy and companionship on improvements in quality of life, mood, and symptom distress during chemotherapy. Thirty-six breast cancer patients received usual care, Reiki, or a companion during chemotherapy. Data were collected from patients while they were receiving usual care. Subsequently, patients were randomized to either receive Reiki or a companion during chemotherapy. Questionnaires assessing quality of life, mood, symptom distress, and Reiki acceptability were completed at baseline and chemotherapy sessions 1, 2, and 4. Reiki was rated relaxing and caused no side effects. Both Reiki and companion groups reported improvements in quality of life and mood that were greater than those seen in the usual care group.
The authors of this study conclude that interventions during chemotherapy, such as Reiki or companionship, are feasible, acceptable, and may reduce side effects.
This is an odd conclusion, if there ever was one. Clearly the ‘companionship’ group was included to see whether Reiki has effects beyond simply providing sympathetic attention. The results show that this is not the case. It follows, I think, that Reiki is a placebo; its perceived relaxing effects are the result of non-specific phenomena which have nothing to do with Reiki per se. The fact that the authors fail to spell this out more clearly makes me wonder whether they are researchers or promoters of Reiki.
Some people will feel that it does not matter how Reiki works, the main thing is that it does work. I beg to differ!
If its effects are due to nothing else than attention and companionship, we do not need ‘trained’ Reiki masters to do the treatment; anyone who has time, compassion and sympathy can do it. More importantly, if Reiki is a placebo, we should not mislead people that some super-natural energy is at work. This only promotes irrationality – and, as Voltaire once said: those who make you believe in absurdities can make you commit atrocities.
After yesterday’s post mentioning ‘biopuncture’, I am sure you are all dying to know what this mysterious treatment might be. A website promoting biopuncture tells us (almost) all we need to know:
Biopuncture is a therapy whereby specific locations are injected with biological products. The majority of the products are derived from plants. Most of these injections are given into the skin or into muscles. Products commonly used in Biopuncture are, for example, arnica, echinacea, nux vomica and chamomile. Arnica is used for muscle pain, nux vomica is injected for digestive problems, echinacea is used to increase the natural defense system of the body. Biopuncturists always inject cocktails of natural products. Lymphomyosot is used for lymphatic drainage, Traumeel for inflammations and sports injuries, Spascupreel for muscular cramps. Injections with antiflogistics, hyaluronic acid, blood platelets, blood, procaine, ozon, cortisone or vitamin B are not considered as Biopuncture…
How can such a small dose influence your body and stimulate healing? Scientists don’t have the final proof yet, but they postulate that these injections are working through the stimulation of the immune system (which is in fact your defense system). Let’s compare it with a vaccination. When you receive a tetanus vaccination, only small amounts of a particular product are necessary to stimulate the immune system against lockjaw. In other words, just a few injections can protect your body for years…
An important issue in Biopuncture is the detoxification of the body. It literally means “cleaning the body” from all the toxins that have accumulated: for example from the environment (air pollution, smoking), from bad nutrition, or from medication (e.g., antibiotics and steroids you’ve taken). These toxins can block your defense system. Some injections work specifically on the liver and others on the kidneys. Cleaning up the lymphatic system with Lymphomyosot is considered very important in Biopuncture. It is like taking the leaves out of the gutter. The down side of such an approach is that old symptoms (which have been suppressed earlier on) may come to the surface again. But that is sometimes part of the healing strategy of the body…
That sounds strange, to say the least. But remember: strange treatments might still work! The question is therefore: IS BIOPUNCTURE AN EFFECTIVE THERAPY? If you ask it to Dr Oz, the answer would be a resounding YES – but let’s not ask Oz, let’s try to find some reliable evidence instead. In my quest to locate such evidence, I came across claims like these.
Examples of some acute conditions we treat with biopuncture:
- Knee and ankle sprains
- Muscle sprains- quadriceps, hamstring, adductors, rotator cuff
- Whiplash
Examples of some chronic conditions we treat with biopuncture:
- Headaches
- Achilles tendinitis
- Tennis elbow
- Chronic arthritis of the knee, hip, shoulder
- Back pain
- Myofascial pains
- Irritable bowel syndrome
- TMJ syndrome
Somehow I had the feeling that this was more than a little too optimistic, and I decided to conduct a rudimentary Medline search. The results were sobering indeed: not a single clinical trial seems to be available that supports any of the claims that are being made for biopuncture.
So, what should we conclude? I don’t know about you, but to me it seems that biopuncture is quackery at its purest.
Acute tonsillitis (AT) is an upper respiratory tract infection which is prevalent, particularly in children. The cause is usually a viral or, less commonly, a bacterial infection. Treatment is symptomatic and usually consists of ample fluid intake and pain-killers; antibiotics are rarely indicated, even if the infection is bacterial by nature. The condition is self-limiting and symptoms subside normally after one week.
Homeopaths believe that their remedies are effective for AT – but is there any evidence? A recent trial seems to suggest there is.
It aimed, according to its authors, to determine the efficacy of a homeopathic complex on the symptoms of acute viral tonsillitis in African children in South Africa.
The double-blind, placebo-controlled RCT was a 6-day “pilot study” and included 30 children aged 6 to 12 years, with acute viral tonsillitis. Participants took two tablets 4 times per day. The treatment group received lactose tablets medicated with the homeopathic complex (Atropa belladonna D4, Calcarea phosphoricum D4, Hepar sulphuris D4, Kalium bichromat D4, Kalium muriaticum D4, Mercurius protoiodid D10, and Mercurius biniodid D10). The placebo consisted of the unmedicated vehicle only. The Wong-Baker FACES Pain Rating Scale was used for measuring pain intensity, and a Symptom Grading Scale assessed changes in tonsillitis signs and symptoms.
The results showed that the treatment group had a statistically significant improvement in the following symptoms compared with the placebo group: pain associated with tonsillitis, pain on swallowing, erythema and inflammation of the pharynx, and tonsil size.
The authors drew the following conclusions: the homeopathic complex used in this study exhibited significant anti-inflammatory and pain-relieving qualities in children with acute viral tonsillitis. No patients reported any adverse effects. These preliminary findings are promising; however, the sample size was small and therefore a definitive conclusion cannot be reached. A larger, more inclusive research study should be undertaken to verify the findings of this study.
Personally, I agree only with the latter part of the conclusion and very much doubt that this study was able to “determine the efficacy” of the homeopathic product used. The authors themselves call their trial a “pilot study”. Such projects are not meant to determine efficacy but are usually designed to determine the feasibility of a trial design in order to subsequently mount a definitive efficacy study.
Moreover, I have considerable doubts about the impartiality of the authors. Their affiliation is “Department of Homoeopathy, University of Johannesburg, Johannesburg, South Africa”, and their article was published in a journal known to be biased in favour of homeopathy. These circumstances in itself might not be all that important, but what makes me more than a little suspicious is this sentence from the introduction of their abstract:
“Homeopathic remedies are a useful alternative to conventional medications in acute uncomplicated upper respiratory tract infections in children, offering earlier symptom resolution, cost-effectiveness, and fewer adverse effects.”
A useful alternative to conventional medications (there are no conventional drugs) for earlier symptom resolution?
If it is true that the usefulness of homeopathic remedies has been established, why conduct the study?
If the authors were so convinced of this notion (for which there is, of course, no good evidence) how can we assume they were not biased in conducting this study?
I think that, in order to agree that a homeopathic remedy generates effects that differ from those of placebo, we need a proper (not a pilot) study, published in a journal of high standing by unbiased scientists.
Rigorous research into the effectiveness of a therapy should tell us the truth about the ability of this therapy to treat patients suffering from a given condition — perhaps not one single study, but the totality of the evidence (as evaluated in systematic reviews) should achieve this aim. Yet, in the realm of alternative medicine (and probably not just in this field), such reviews are often highly contradictory.
A concrete example might explain what I mean.
There are numerous systematic reviews assessing the effectiveness of acupuncture for fibromyalgia syndrome (FMS). It is safe to assume that the authors of these reviews have all conducted comprehensive searches of the literature in order to locate all the published studies on this subject. Subsequently, they have evaluated the scientific rigor of these trials and summarised their findings. Finally they have condensed all of this into an article which arrives at a certain conclusion about the value of the therapy in question. Understanding this process (outlined here only very briefly), one would expect that all the numerous reviews draw conclusions which are, if not identical, at least very similar.
However, the disturbing fact is that they are not remotely similar. Here are two which, in fact, are so different that one could assume they have evaluated a set of totally different primary studies (which, of course, they have not).
One recent (2014) review concluded that acupuncture for FMS has a positive effect, and acupuncture combined with western medicine can strengthen the curative effect.
Another recent review concluded that a small analgesic effect of acupuncture was present, which, however, was not clearly distinguishable from bias. Thus, acupuncture cannot be recommended for the management of FMS.
How can this be?
By contrast to most systematic reviews of conventional medicine, systematic reviews of alternative therapies are almost invariably based on a small number of primary studies (in the above case, the total number was only 7 !). The quality of these trials is often low (all reviews therefore end with the somewhat meaningless conclusion that more and better studies are needed).
So, the situation with primary studies of alternative therapies for inclusion into systematic reviews usually is as follows:
- the number of trials is low
- the quality of trials is even lower
- the results are not uniform
- the majority of the poor quality trials show a positive result (bias tends to generate false positive findings)
- the few rigorous trials yield a negative result
Unfortunately this means that the authors of systematic reviews summarising such confusing evidence often seem to feel at liberty to project their own pre-conceived ideas into their overall conclusion about the effectiveness of the treatment. Often the researchers are in favour of the therapy in question – in fact, this usually is precisely the attitude that motivated them to conduct a review in the first place. In other words, the frequently murky state of the evidence (as outlined above) can serve as a welcome invitation for personal bias to do its effect in skewing the overall conclusion. The final result is that the readers of such systematic reviews are being misled.
Authors who are biased in favour of the treatment will tend to stress that the majority of the trials are positive. Therefore the overall verdict has to be positive as well, in their view. The fact that most trials are flawed does not usually bother them all that much (I suspect that many fail to comprehend the effects of bias on the study results); they merely add to their conclusions that “more and better trials are needed” and believe that this meek little remark is sufficient evidence for their ability to critically analyse the data.
Authors who are not biased and have the necessary skills for critical assessment, on the other hand, will insist that most trials are flawed and therefore their results must be categorised as unreliable. They will also emphasise the fact that there are a few reliable studies and clearly point out that these are negative. Thus their overall conclusion must be negative as well.
In the end, enthusiasts will conclude that the treatment in question is at least promising, if not recommendable, while real scientists will rightly state that the available data are too flimsy to demonstrate the effectiveness of the therapy; as it is wrong to recommend unproven treatments, they will not recommend the treatment for routine use.
The difference between the two might just seem marginal – but, in fact, it is huge: IT IS THE DIFFERENCE BETWEEN MISLEADING PEOPLE AND GIVING RESPONSIBLE ADVICE; THE DIFFERENCE BETWEEN VIOLATING AND ADHERING TO ETHICAL STANDARDS.
“Dr” Brian Moravec is a chiropractor from the US; he has a website where he describes himself and his skills as follows: I attended Chiropractic College and I am a graduate of Palmer College of Chiropractic in Davenport Iowa. I earned a Bachelor of Science degree as well as my Doctor of Chiropractic degree from Palmer College, which is the first chiropractic college in the world and the origin of our profession. I also attend continuing education seminars designed to keep doctors current with regard to clinical chiropractic, technique and nutrition.
The key to overall health and wellness is to have a healthy nervous system and that is what I do as a chiropractor – I make sure that your spine is functioning at its best so that your nervous system functions at its best. When the nervous system is functioning at 100%, you are a healthier individual that experiences a higher quality of life and health. I know this to be true in myself, my family and my patients.
I go to great lengths to provide my patients with the best chiropractic care I can give. I work with my patients to design a treatment plan that will be effective for their particular condition and specific to their needs. We utilize manual and low force techniques (safe and effective for newborns to seniors), to correct sublaxations in the spine. Chiropractic adjustments remove nerve interference, which allows the body to perform at its best again. I also am available for consultations on nutrition and diet, dietary supplementation and how to minimize the wear and tear on your spine.
[Emphases are mine]What he does not state is the fact that he also is a nifty e-mail writer!
To my great surprise, I received an e-mail from him which is far too good to be kept for myself. So I decided to share it with my readers; here it is in its full and unabbreviated beauty:
its interesting to see someone with your education, and is a self proclaimed “expert” on alternative medicine, promote so much misinformation with regard to chiropractic care. fortunately you look old. and soon will be gone. I always refer to the few of you anti chiropractic fools left here as “dinosaurs”. the proof is in the pudding my “friend”. chiropractic works and will continue to be here for centuries more. you and others with much much more power than you (the AMA for example) have tried to perpetuate lies and squash chiropractic. you fail, and they failed, because whatever better serves mankind will stand the test of time. you’re a dying breed edzard. thank God.
yours in health,
brian moravec d.c.
I am encouraged to see that he recognises my education but do wonder why his upbringing obviously failed so dismally teach him even a minimum of politeness, tact, or critical thinking. It is disappointing, I think, that he does not even mention what he perceives as my lies about his beloved chiropractic. So sad, I am sure it would have been fun to debate with him.
Whenever I give a public lecture about homeopathy, I explain what it is, briefly go in to its history, explain what its assumptions are, and what the evidence tells us about its efficacy and safety. When I am finished, there usually is a discussion with the audience. This is the part I like best; in fact, it is the main reason why I made the effort to do the lecture in the first place.
The questions vary, of course, but you can bet your last shirt that someone asks: “We know it works for animals; animals cannot experience a placebo-response, and therefore your claim that homeopathy relies on nothing but the placebo-effect must be wrong!” At this stage I often despair a little, I must admit. Not because the question is too daft, but because I did address it during my lecture. Thus I feel that I have failed to get the right message across – I despair with my obviously poor skills of giving an informative lecture!
Yet I need to answer the above question, of course. So I reiterate that the perceived effectiveness of homeopathy relies not just on the placebo-effect but also on phenomena such as regression towards the mean, natural history of the condition etc. I also usually mention that it is erroneous to assume that animals cannot benefit from placebo-effects; they can be conditioned, and pets can react to the expectations of their owners.
Finally, I need to mention the veterinary clinical evidence which – just like in the case of human patients – fails to show that homeopathic remedies are better than placebos for treating animals. Until recently, this was not an easy task because no systematic review of randomised placebo-controlled trials (RCTs) of veterinary homeopathy was available. Now, I am happy to announce, this situation has changed.
Using Cochrane methods, a brand-new review aimed to assess risk of bias and to quantify the effect size of homeopathic interventions compared with placebo for each eligible peer-reviewed trial. Judgement in 7 assessment domains enabled a trial’s risk of bias to be designated as low, unclear or high. A trial was judged to comprise reliable evidence, if its risk of bias was low or was unclear in specified domains. A trial was considered to be free of vested interest, if it was not funded by a homeopathic pharmacy.
The 18 RCTs found by the researchers were disparate in nature, representing 4 species and 11 different medical conditions. Reliable evidence, free from vested interest, was identified in only two trials:
- homeopathic Coli had a prophylactic effect on porcine diarrhoea (odds ratio 3.89, 95 per cent confidence interval [CI], 1.19 to 12.68, P=0.02);
- individualised homeopathic treatment did not have a more beneficial effect on bovine mastitis than placebo intervention (standardised mean difference -0.31, 95 per cent CI, -0.97 to 0.34, P=0.35).
The authors conclusions are clear: Mixed findings from the only two placebo-controlled RCTs that had suitably reliable evidence precluded generalisable conclusions about the efficacy of any particular homeopathic medicine or the impact of individualised homeopathic intervention on any given medical condition in animals.
My task when lecturing about homeopathy has thus become a great deal easier. But homeopathy-fans are not best pleased with this new article, I guess. They will try to claim that it was a biased piece of research conducted, most likely, by notorious anti-homeopaths who cannot be trusted. So who are the authors of this new publication?
They are RT Mathie from the British Homeopathic Association and J Clausen from one of Germany’s most pro-homeopathic institution, the ‘Karl und Veronica Carstens-Stiftung’.
DOES ANYONE BELIEVE THAT THIS ARTICLE IS BIASED AGAINST HOMEOPATHY?
‘Healing, hype or harm? A critical analysis of complementary or alternative medicine’ is the title of a book that I edited and that was published in 2008. Its publication date coincided with that of ‘Trick or Treatment?’ and therefore the former was almost completely over-shadowed by the latter. Consequently few people know about it. This is a shame, I think, and this post is dedicated to encouraging my readers to have a look at ‘Healing, hype or harm?’
One reviewer commented on Amazon about this book as follows: Vital and informative text that should be read by everyone alongside Ben Goldacre’s ‘Bad Science’ and Singh and Ernt’s ‘Trick or Treatment’. Everyone should be able to made informed choices about the treatments that are peddled to the desperate and gullible. As Tim Minchin famously said ‘What do you call Alternative Medicine that has been proved to work? . . . Medicine!’
This is high praise indeed! But I should not omit the fact that others have commented that they were appalled by our book and found it “disappointing and unsettling”. This does not surprise me in the least; after all, alternative medicine has always been a divisive subject.
The book was written by a total of 17 authors and covers many important aspects of alternative medicine. Some of its most famous contributors are Michael Baum, Gustav Born, David Colquhoun, James Randi and Nick Ross. Some of the most important subjects include:
- Compassion
- Quackademia
- Impartiality
- Ethics
- Politics
- Holism
- Vitalism
- Placebo
As already mentioned, our book is already 6 years old; however, this does not mean that it is now out-dated. The subject areas were chosen such that it will be timely for a long time to come. Nor does this book reflect one single point of view; as it was written by over a dozen different experts with vastly different backgrounds, it offers an entire spectrum of views and attitudes. It is, in a word, a book that stimulates critical thinking and thoughtful analysis.
I sincerely think you should have a look at it… and, in case you think I am hoping to maximise my income by telling you all this: all the revenues from this book go to charity.
One of the most commonly ‘accepted’ indications for acupuncture is anxiety. Many trials have suggested that it is effective for that condition. But is this really true? To find out, we need someone to conduct a systematic review or meta-analysis.
Korean researchers have just published such a paper; they wanted to assess the preoperative anxiolytic efficacy of acupuncture therapy and therefore conducted a meta-analysis of all RCTs on the subject. Four electronic databases were searched up to February 2014. Data were included in the meta-analysis from RCTs in which groups receiving preoperative acupuncture treatment were compared with control groups receiving a placebo for anxiety.
Fourteen publications with a total of 1,034 patients were included. Six RCTs, using the State-Trait Anxiety Inventory-State (STAI-S), reported that acupuncture interventions led to greater reductions in preoperative anxiety relative to sham acupuncture. A further eight publications, employing visual analogue scales, also indicated significant differences in preoperative anxiety amelioration between acupuncture and sham acupuncture.
The authors concluded that aacupuncture therapy aiming at reducing preoperative anxiety has a statistically significant effect relative to placebo or nontreatment conditions. Well-designed and rigorous studies that employ large sample sizes are necessary to corroborate this finding.
From these conclusions most casual readers might get the impression that acupuncture is indeed effective. One has to dig a bit deeper to realise that is perhaps not so.
Why? Because the quality of the primary studies was often dismally poor. Most did not even mention adverse effects which, in my view, is a clear breach of publication ethics. What is more, all the studies were wide open to bias. The authors of the meta-analysis include in their results section the following short paragraph:
The 14 included studies exhibited various degrees of bias susceptibility (Figure 2 and Figure 3). The agreement rate, as measured using Cohen’s kappa, was 0.8 [32]. Only six studies reported concealed allocation; the other six described a method of adequate randomization, although the word “randomization” appeared in all of the articles. Thirteen studies prevented blinding of the participants. Participants in these studies had no previous experience of acupuncture. According to STRICTA, two studies enquired after patients’ beliefs as a group: there were no significant differences [20, 24].
There is a saying amongst experts about such meta-analyses: RUBBISH IN, RUBBISH OUT. It describes the fact that several poor studies, pooled meta-analytically, can never give a reliable result.
This does, however, not mean that such meta-analyses are necessarily useless. If the authors prominently (in the abstract) stress that the quality of the primary studies was wanting and that therefore the overall result is unreliable, they might inspire future researchers to conduct more rigorous trials and thus generate progress. Most importantly, by insisting on pointing out these limitations and by not drawing positive conclusions from flawed data, they would avoid misleading those health care professionals – and let’s face it, they are the majority – who merely read the abstract or even just the conclusions of such articles.
The authors of this review have failed to do any of this; they and the journal EBCAM have thus done a disservice to us all by contributing to the constant drip of misleading and false-positive information about the value of acupuncture.
