MD, PhD, FMedSci, FSB, FRCP, FRCPEd

systematic review

1 2 3 8

Whenever I give a public lecture about homeopathy, I explain what it is, briefly go in to its history, explain what its assumptions are, and what the evidence tells us about its efficacy and safety. When I am finished, there usually is a discussion with the audience. This is the part I like best; in fact, it is the main reason why I made the effort to do the lecture in the first place.

The questions vary, of course, but you can bet your last shirt that someone asks: “We know it works for animals; animals cannot experience a placebo-response, and therefore your claim that homeopathy relies on nothing but the placebo-effect must be wrong!” At this stage I often despair a little, I must admit. Not because the question is too daft, but because I did address it during my lecture. Thus I feel that I have failed to get the right message across – I despair with my obviously poor skills of giving an informative lecture!

Yet I need to answer the above question, of course. So I reiterate that the perceived effectiveness of homeopathy relies not just on the placebo-effect but also on phenomena such as regression towards the mean, natural history of the condition etc. I also usually mention that it is erroneous to assume that animals cannot benefit from placebo-effects; they can be conditioned, and pets can react to the expectations of their owners.

Finally, I need to mention the veterinary clinical evidence which – just like in the case of human patients – fails to show that homeopathic remedies are better than placebos for treating animals. Until recently, this was not an easy task because no systematic review of randomised placebo-controlled trials (RCTs) of veterinary homeopathy was available. Now, I am happy to announce, this situation has changed.

Using Cochrane methods, a brand-new review aimed to assess risk of bias and to quantify the effect size of homeopathic interventions compared with placebo for each eligible peer-reviewed trial. Judgement in 7 assessment domains enabled a trial’s risk of bias to be designated as low, unclear or high. A trial was judged to comprise reliable evidence, if its risk of bias was low or was unclear in specified domains. A trial was considered to be free of vested interest, if it was not funded by a homeopathic pharmacy.

The 18 RCTs found by the researchers were disparate in nature, representing 4 species and 11 different medical conditions. Reliable evidence, free from vested interest, was identified in only two trials:

  1. homeopathic Coli had a prophylactic effect on porcine diarrhoea (odds ratio 3.89, 95 per cent confidence interval [CI], 1.19 to 12.68, P=0.02);
  2. individualised homeopathic treatment did not have a more beneficial effect on bovine mastitis than placebo intervention (standardised mean difference -0.31, 95 per cent CI, -0.97 to 0.34, P=0.35).

The authors conclusions are clear: Mixed findings from the only two placebo-controlled RCTs that had suitably reliable evidence precluded generalisable conclusions about the efficacy of any particular homeopathic medicine or the impact of individualised homeopathic intervention on any given medical condition in animals.

My task when lecturing about homeopathy has thus become a great deal easier. But homeopathy-fans are not best pleased with this new article, I guess. They will try to claim that it was a biased piece of research conducted, most likely, by notorious anti-homeopaths who cannot be trusted. So who are the authors of this new publication?

They are RT Mathie from the British Homeopathic Association and J Clausen from one of Germany’s most pro-homeopathic institution, the ‘Karl und Veronica Carstens-Stiftung’.

DOES ANYONE BELIEVE THAT THIS ARTICLE IS BIASED AGAINST HOMEOPATHY?

One of the most commonly ‘accepted’ indications for acupuncture is anxiety. Many trials have suggested that it is effective for that condition. But is this really true? To find out, we need someone to conduct a systematic review or meta-analysis.

Korean researchers have just published such a paper; they wanted to assess the preoperative anxiolytic efficacy of acupuncture therapy and therefore conducted a meta-analysis of all RCTs on the subject. Four electronic databases were searched up to February 2014. Data were included in the meta-analysis from RCTs in which groups receiving preoperative acupuncture treatment were compared with control groups receiving a placebo for anxiety.

Fourteen publications with a total of 1,034 patients were included. Six RCTs, using the State-Trait Anxiety Inventory-State (STAI-S), reported that acupuncture interventions led to greater reductions in preoperative anxiety relative to sham acupuncture. A further eight publications, employing visual analogue scales, also indicated significant differences in preoperative anxiety amelioration between acupuncture and sham acupuncture.

The authors concluded that aacupuncture therapy aiming at reducing preoperative anxiety has a statistically significant effect relative to placebo or nontreatment conditions. Well-designed and rigorous studies that employ large sample sizes are necessary to corroborate this finding.

From these conclusions most casual readers might get the impression that acupuncture is indeed effective. One has to dig a bit deeper to realise that is perhaps not so.

Why? Because the quality of the primary studies was often dismally poor. Most did not even mention adverse effects which, in my view, is a clear breach of publication ethics. What is more, all the studies were wide open to bias. The authors of the meta-analysis include in their results section the following short paragraph:

The 14 included studies exhibited various degrees of bias susceptibility (Figure 2 and Figure 3). The agreement rate, as measured using Cohen’s kappa, was 0.8 [32]. Only six studies reported concealed allocation; the other six described a method of adequate randomization, although the word “randomization” appeared in all of the articles. Thirteen studies prevented blinding of the participants. Participants in these studies had no previous experience of acupuncture. According to STRICTA, two studies enquired after patients’ beliefs as a group: there were no significant differences [20, 24].

There is a saying amongst experts about such meta-analyses: RUBBISH IN, RUBBISH OUT. It describes the fact that several poor studies, pooled meta-analytically, can never give a reliable result.

This does, however, not mean that such meta-analyses are necessarily useless. If the authors prominently (in the abstract) stress that the quality of the primary studies was wanting and that therefore the overall result is unreliable, they might inspire future researchers to conduct more rigorous trials and thus generate progress. Most importantly, by insisting on pointing out these limitations and by not drawing positive conclusions from flawed data, they would avoid misleading those health care professionals – and let’s face it, they are the majority – who merely read the abstract or even just the conclusions of such articles.

The authors of this review have failed to do any of this; they and the journal EBCAM have thus done a disservice to us all by contributing to the constant drip of misleading and false-positive information about the value of acupuncture.

Reflexology? Isn’t that an alternative therapy? And as such, a physiotherapist would not normally use it, most of us might think.

Well, think again! Here is what the UK Chartered Society of Physiotherapists writes about reflexology:

Developed centuries ago in countries such as China, Egypt and India, reflexology is often referred to as a ‘gentle’ and ‘holistic’ therapy that benefits both mind and body. It centres on the feet because these are said by practitioners to be a mirror, or topographical map, for the rest of the body. Manipulation of certain pressure, or reflex, points is claimed to have an effect on corresponding zones in the body. The impact, say reflexologists, extends throughout – to bones, muscles, organs, glands, circulatory and neural pathways. The head and hands can also be massaged in some cases. The treatment is perhaps best known for use in connection with relaxation and relief from stress, anxiety, pain, sleep disorders, headaches, migraine, menstrual and digestive problems. But advocates say it can be used to great effect far more widely, often in conjunction with other treatments.

Reflexology, or Reflex Therapy (RT) as some physiotherapists prefer to call it, clearly is approved by the UK Chartered Society of Physiotherapists. And what evidence do they have for it?

One hundred members of the Association of Chartered Physiotherapists in Reflex Therapy (ACPIRT) participated in an audit to establish a baseline of practice. Findings indicate that experienced therapists use RT in conjunction with their professional skills to induce relaxation (95%) and reduce pain (86%) for patients with conditions including whiplash injury and chronic pain. According to 68% of respondents, RT is “very good,” “good” or “as good as” orthodox physiotherapy practices. Requiring minimal equipment, RT may be as cost effective as orthodox physiotherapy with regards to duration and frequency of treatment.

But that’s not evidence!!! I hear you grumble. No, it isn’t, I agree.

Is there good evidence to show that RT is effective?

I am afraid not!

My own systematic review concluded that the best evidence available to date does not demonstrate convincingly that reflexology is an effective treatment for any medical condition.

Does that mean that the Chartered Society of Physiotherapists promotes quackery?

I let my readers answer that question.

Kinesiology tape is all the rage. Its proponents claim that it increases cutaneous stimulation, which facilitates motor unit firing, and consequently improves functional performance. But is this just clever marketing, wishful thinking or is it true? To find out, we need reliable data.

The current trial results are sparse, confusing and contradictory. A recent systematic review indicated that kinesiology tape may have limited potential to reduce pain in individuals with musculoskeletal injury; however, depending on the conditions, the reduction in pain may not be clinically meaningful. Kinesiology tape application did not reduce specific pain measures related to musculoskeletal injury above and beyond other modalities compared in the context of included articles. 

The authors concluded that kinesiology tape may be used in conjunction with or in place of more traditional therapies, and further research that employs controlled measures compared with kinesiology tape is needed to evaluate efficacy.

This need for further research has just been met by Korean investigators who conducted a study testing the true effects of KinTape by a deceptive, randomized, clinical trial.

Thirty healthy participants performed isokinetic testing of three taping conditions: true facilitative KinTape, sham KinTape, and no KinTape. The participants were blindfolded during the evaluation. Under the pretense of applying adhesive muscle sensors, KinTape was applied to their quadriceps in the first two conditions. Normalized peak torque, normalized total work, and time to peak torque were measured at two angular speeds (60°/s and 180°/s) and analyzed with one-way repeated measures ANOVA.

Participants were successfully deceived and they were ignorant about KinTape. No significant differences were found between normalized peak torque, normalized total work, and time to peak torque at 60°/s or 180°/s (p = 0.31-0.99) between three taping conditions. The results showed that KinTape did not facilitate muscle performance in generating higher peak torque, yielding a greater total work, or inducing an earlier onset of peak torque.

The authors concluded that previously reported muscle facilitatory effects using KinTape may be attributed to placebo effects.

The claims that are being made for kinesiology taping are truly extraordinary; just consider what this website is trying to tell us:

Kinesiology tape is a breakthrough new method for treating athletic sprains, strains and sports injuries. You may have seen Olympic and celebrity athletes wearing multicolored tape on their arms, legs, shoulders and back. This type of athletic tape is a revolutionary therapeutic elastic style of support that works in multiple ways to improve health and circulation in ways that traditional athletic tapes can’t compare. Not only does this new type of athletic tape help support and heal muscles, but it also provides faster, more thorough healing by aiding with blood circulation throughout the body.

Many athletes who have switched to using this new type of athletic tape report a wide variety of benefits including improved neuromuscular movement and circulation, pain relief and more. In addition to its many medical uses, Kinesiology tape is also used to help prevent injuries and manage pain and swelling, such as from edema. Unlike regular athletic taping, using elastic tape allows you the freedom of motion without restricting muscles or blood flow. By allowing the muscles a larger degree of movement, the body is able to heal itself more quickly and fully than before.

Whenever I read such over-enthusiastic promotion that is not based on evidence but on keen salesmanship, my alarm-bells start ringing and I see parallels to the worst type of alternative medicine hype. In fact, kinesiology tapes have all the hallmarks of alternative medicine and its promoters have, as far as I can see, all the characteristics of quacks. The motto seems to be: LET’S EARN SOME MONEY FAST AND IGNORE THE SCIENCE WHILE WE CAN.

Chiropractors, like other alternative practitioners, use their own unique diagnostic tools for identifying the health problems of their patients. One such test is the Kemp’s test, a manual test used by most chiropractors to diagnose problems with lumbar facet joints. The chiropractor rotates the torso of the patient, while her pelvis is fixed; if manual counter-rotative resistance on one side of the pelvis by the chiropractor causes lumbar pain for the patient, it is interpreted as a sign of lumbar facet joint dysfunction which, in turn would be treated with spinal manipulation.

All diagnostic tests have to fulfil certain criteria in order to be useful. It is therefore interesting to ask whether the Kemp’s test meets these criteria. This is precisely the question addressed in a recent paper. Its objective was to evaluate the existing literature regarding the accuracy of the Kemp’s test in the diagnosis of facet joint pain compared to a reference standard.

All diagnostic accuracy studies comparing the Kemp’s test with an acceptable reference standard were located and included in the review. Subsequently, all studies were scored for quality and internal validity.

Five articles met the inclusion criteria. Only two studies had a low risk of bias, and three had a low concern regarding applicability. Pooling of data from studies using similar methods revealed that the test’s negative predictive value was the only diagnostic accuracy measure above 50% (56.8%, 59.9%).

The authors concluded that currently, the literature supporting the use of the Kemp’s test is limited and indicates that it has poor diagnostic accuracy. It is debatable whether clinicians should continue to use this test to diagnose facet joint pain.

The problem with chiropractic diagnostic methods is not confined to the Kemp’s test, but extends to most tests employed by chiropractors. Why should this matter?

If diagnostic methods are not reliable, they produce either false-positive or false-negative findings. When a false-negative diagnosis is made, the chiropractor might not treat a condition that needs attention. Much more common in chiropractic routine, I guess, are false-positive diagnoses. This means chiropractors frequently treat conditions which the patient does not have. This, in turn, is not just a waste of money and time but also, if the ensuing treatment is associated with risks, an unnecessary exposure of patients to getting harmed.

The authors of this review, chiropractors from Canada, should be praised for tackling this subject. However, their conclusion that “it is debatable whether clinicians should continue to use this test to diagnose facet joint pain” is in itself highly debatable: the use of nonsensical diagnostic tools can only result in nonsense and should therefore be disallowed.

Bach Flower Remedies are the brain child of Dr Edward Bach who, as an ex-homeopath, invented his very own highly diluted remedies. Like homeopathic medicines, they are devoid of active molecules and are claimed to work via some non-defined ‘energy’. Consequently, the evidence for these treatments is squarely negative: my systematic review analysed the data of all 7 RCTs of human patients or volunteers that were available in 2010. All but one were placebo-controlled. All placebo-controlled trials failed to demonstrate efficacy. I concluded that the most reliable clinical trials do not show any differences between flower remedies and placebos.

But now, a new investigation has become available. The aim of this study was to evaluate the effect of Bach flower Rescue Remedy on the control of risk factors for cardiovascular disease in rats.

A randomized longitudinal experimental study was conducted on 18 Wistar rats which were randomly divided into three groups of six animals each and orogastrically dosed with either 200μl of water (group A, control), or 100μl of water and 100μl of Bach flower remedy (group B), or 200μl of Bach flower remedy (group C) every 2 days, for 20 days. All animals were fed standard rat chow and water ad libitum.

Urine volume, body weight, feces weight, and food intake were measured every 2 days. On day 20, tests of glycemia, hyperuricemia, triglycerides, high-density lipoprotein (HDL) cholesterol, and total cholesterol were performed, and the anatomy and histopathology of the heart, liver and kidneys were evaluated. Data were analyzed using Tukey’s test at a significance level of 5%.

No significant differences were found in food intake, feces weight, urine volume and uric acid levels between groups. Group C had a significantly lower body weight gain than group A and lower glycemia compared with groups A and B. Groups B and C had significantly higher HDL-cholesterol and lower triglycerides than controls. Animals had mild hepatic steatosis, but no cardiac or renal damage was observed in the three groups.

From these results, the authors conclude that Bach flower Rescue Remedy was effective in controlling glycemia, triglycerides, and HDL-cholesterol and may serve as a strategy for reducing risk factors for cardiovascular disease in rats. This study provides some preliminary “proof of concept” data that Bach Rescue Remedy may exert some biological effects.

If ever there was a bizarre study, it must be this one:

  • As far as I know, nobody has ever claimed that Rescue Remedy modified cardiovascular risk factors.
  • It seems debatable whether the observed changes are all positive as far as the cardiovascular risk is concerned.
  • It seems odd that a remedy that does not contain active molecules is associated with some sort of dose-effect response.
  • The modification of cardiovascular risk factors in rats might be of little relevance for humans.
  • A strategy for reducing cardiovascular risk factors in rats seems a strange idea.
  • Even the authors cannot offer a mechanism of action [other than pure magic].

Does this study tell us anything of value? The authors are keen to point out that it provides a preliminary proof of concept for Rescue Remedy having biological effects. Somehow, I doubt that this conclusion will convince many of my readers.

“If ever there was a permanent cure for migraine, homeopathic medicines are the only one that can do this miracle. It may sound like an overstatement and quite quackerish, but it’s true. Long term treatment with homeopathy has an excellent cure for migraine headaches.” Statements like this can be found by the thousands on the internet, not just in relation to migraine but also about osteoarthritis. Both migraine and osteoarthritis are important domains for homeopathy, and most homeopaths would not doubt for a second that they can treat these conditions effectively. This is why it is so important to highlight the few sources which are not misleading consumers into making the wrong therapeutic decisions.

‘Healthcare Improvement Scotland’ (HCIS) have just published advice for patients suffering from migraine and osteoarthritis (the full document with all the evidence can be found here). I think it is worth having a close look and I therefore cite it in full:

Homeopathic remedies are prepared by repeated dilution and vigorous shaking of substances in water. Remedies are prepared from substances that in healthy people cause the signs and symptoms of the condition being treated. The more dilute the remedy is the more potent it becomes so that the most potent remedies are unlikely to contain any of the original substance.

People in Scotland have access to homeopathy through some GPs or a referral to homeopaths in the private sector, regional NHS clinics or the Centre for Integrative Care (CIC) (formerly Glasgow Homeopathic Hospital). Not all NHSScotland health boards provide funding for homeopathy; investment varies widely among those that do, and individual boards have begun to review funding for homeopathy services.

Clinical effectiveness

  • Evidence of clinical effectiveness was reviewed from systematic reviews of four placebo controlled randomised trials of homeopathy for migraine published between 1991 and 1997; and systematic reviews of four active treatment controlled randomised trials of homeopathy for osteoarthritis published between 1983 and 2000. The quality of the evidence was low to moderate.
  • Homeopathy for migraine has not been compared with active treatment in randomised controlled trials (RCTs). Of four RCTs comparing homeopathy with placebo, only one found homeopathy to be superior.
  • Three RCTs in osteoarthritis comparing homeopathy with medicines for pain relief found either no difference between the interventions, or that analgesic treatment had a better effect than homeopathy. A further RCT comparing intra-articular injection of a homeopathic remedy with hyaluronic acid injections showed similar pain reduction in both groups.

Safety

  • Published systematic reviews of homeopathy for migraine and osteoarthritis contain insufficient information to inform conclusions about safety.

Cost effectiveness

  • No evidence on the cost effectiveness of homeopathy for migraine was identified; and the evidence from a single cost-minimisation analysis of one homeopathic preparation for osteoarthritis is not generalisable to the UK.

Context/conclusion

  • Homeopathy for migraine has not been compared with standard care in RCTs and no evidence of cost effectiveness has been identified..
  • There is insufficient evidence to determine whether or not homeopathic treatment for osteoarthritis is clinically effective compared with standard care, and no relevant evidence of cost effectiveness has been identified.
  • The evidence does not support treating migraine or osteoarthritis with homeopathy.

Before the fans of homeopathy start shouting “THIS IS ALL RUBBISH AND DISREGARDS IMPORTANT EVIDENCE!!!”, I should mention that the top experts in homeopathy were asked to contribute their evidence and were unable to find any convincing data that would have changed this negative verdict. And it is important to point out that HCIS is a respected, independent organisation that issues statements based on thorough, unbiased reviews of the evidence.

As I reported a while ago, the Australian ‘NATIONAL HEALTH AND MEDICAL RESEARCH COUNCIL’ has assessed the effectiveness of homeopathy. The evaluation looks like the most comprehensive and most independent in the history of homeopathy. Its draft report  concluded that “the evidence from research in humans does not show that homeopathy is effective for treating the range of health conditions considered.”

So, the HCIS is in excellent company and I have no doubt whatsoever that this new statement is correct – but I also have little doubt that homeopaths will dispute it.

 

For every condition which is not curable by conventional medicine there are dozens of alternative treatments that offer a cure or at least symptomatic relief. Multiple sclerosis (MS) is such a disease. It is hard to find an alternative therapy that is not being promoted for MS.

Acupuncture is, of course, no exception. It is widely promoted for treating MS symptoms and many MS patients spend lots of money hoping that it does. The US ‘National MS Society’, For instance claim that acupuncture may provide relief for some MS-related symptoms, including pain, spasticity, numbness and tingling, bladder problems, and depression. There is no evidence, however, that acupuncture can reduce the frequency of MS exacerbations or slow the progression of disability. And the ‘British Acupuncture Council’ state that acupuncture may provide relief for some MS-related symptoms, including pain, spasticity, numbness and tingling, bladder problems, and depression.

Such claims seem a little over-optimistic; let’s have a look what the evidence really tells us.

The purpose of this brand-new review was to assess the literature on the effectiveness of acupuncture for treating MS. A literature search resulted in 12 peer-reviewed articles on the subject that examined the use of acupuncture to treat MS related quality of life, fatigue, spasticity, and pain. The majority of the studies were poorly designed-without control, randomization, or blinding. Description of the subjects, interventions, and outcome measures as well as statistical analysis were often lacking or minimal.

The authors concluded that although many of the studies suggested that acupuncture was successful in improving MS related symptoms, lack of statistical rigor and poor study design make it difficult to draw any conclusions about the true effectiveness of this intervention in the MS population. Further studies with more rigorous designs and analysis are needed before accurate claims can be made as to the effectiveness of acupuncture in this population.

And what about other alternative therapies? Our own systematic review of the subject included 12 randomized controlled trials: nutritional therapy (4), massage (1), Feldenkrais bodywork (1), reflexology (1), magnetic field therapy (2), neural therapy (1) and psychological counselling (2). But the evidence was not compelling for any of these therapies, with many trials suffering from significant methodological flaws. There is evidence to suggest some benefit of nutritional therapy for the physical symptoms of MS. Magnetic field therapy and neural therapy appear to have a short-term beneficial effect on the physical symptoms of MS. Massage/bodywork and psychological counselling seem to improve depression, anxiety and self-esteem.

That was some time ago,  and it is therefore reasonable to ask: has the evidence changed? Thankfully, the ‘American Academy of Neurology’ has just published the following guidelines entitles complementary and alternative medicine in multiple sclerosis:

Clinicians might offer oral cannabis extract for spasticity symptoms and pain (excluding central neuropathic pain) (Level A). Clinicians might offer tetrahydrocannabinol for spasticity symptoms and pain (excluding central neuropathic pain) (Level B). Clinicians should counsel patients that these agents are probably ineffective for objective spasticity (short-term)/tremor (Level B) and possibly effective for spasticity and pain (long-term) (Level C). Clinicians might offer Sativex oromucosal cannabinoid spray (nabiximols) for spasticity symptoms, pain, and urinary frequency (Level B). Clinicians should counsel patients that these agents are probably ineffective for objective spasticity/urinary incontinence (Level B). Clinicians might choose not to offer these agents for tremor (Level C). Clinicians might counsel patients that magnetic therapy is probably effective for fatigue and probably ineffective for depression (Level B); fish oil is probably ineffective for relapses, disability, fatigue, MRI lesions, and quality of life (QOL) (Level B); ginkgo biloba is ineffective for cognition (Level A) and possibly effective for fatigue (Level C); reflexology is possibly effective for paresthesia (Level C); Cari Loder regimen is possibly ineffective for disability, symptoms, depression, and fatigue (Level C); and bee sting therapy is possibly ineffective for relapses, disability, fatigue, lesion burden/volume, and health-related QOL (Level C). Cannabinoids may cause adverse effects. Clinicians should exercise caution regarding standardized vs nonstandardized cannabis extracts and overall CAM quality control/nonregulation. Safety/efficacy of other CAM/CAM interaction with MS disease-modifying therapies is unknown.

Interestingly, on yesterday it was announced that the NHS in Wales has just made available a cannabis-based spray for MS-sufferers (I should mention that most cannabis-based preparations are not full plant extracts and thus by definition not herbal but conventional medicines).

It would be wonderful, if other alternative therapies were of proven benefit to MS-sufferers. But sadly, this does not seem to be the case. I think it is better to be truthful about this than to raise false hopes of desperate patients.

Blinding patients in clinical trials is a key methodological procedure for minimizing bias and thus making sure that the results are reliable. In alternative medicine, blinding is not always straight forward, and many studies are therefore not patient-blinded. We all know that this can introduce bias into a trial, but how large is its effect on study outcomes?

This was the research question addressed by a recent systematic review of randomized clinical trials with one sub-study (i.e. experimental vs control) involving blinded patients and another, otherwise identical, sub-study involving non-blinded patients. Within each trial, the researchers compared the difference in effect sizes (i.e. standardized mean differences) between the two sub-studies. A difference <0 indicates that non-blinded patients generated a more optimistic effect estimate. The researchers then pooled the differences with random-effects inverse variance meta-analysis, and explored reasons for heterogeneity.

The main analysis included 12 trials with a total of 3869 patients. Ten of these RCTs were studies of acupuncture. The average difference in effect size for patient-reported outcomes was -0.56 (95% confidence interval -0.71 to -0.41), (I(2 )= 60%, P = 0.004), indicating that non-blinded patients exaggerated the effect size by an average of 0.56 standard deviation, but with considerable variation. Two of the 12 trials also used observer-reported outcomes, showing no indication of exaggerated effects due lack of patient blinding.

There was an even larger effect size difference in the 10 acupuncture trials [-0.63 (-0.77 to -0.49)], than in the two non-acupuncture trials [-0.17 (-0.41 to 0.07)]. Lack of patient blinding was also associated with increased attrition rates and the use of co-interventions: ratio of control group attrition risk 1.79 (1.18 to 2.70), and ratio of control group co-intervention risk 1.55 (0.99 to 2.43).

The authors conclude that this study provides empirical evidence of pronounced bias due to lack of patient blinding in complementary/alternative randomized clinical trials with patient-reported outcomes.

This is a timely, rigorous and important analysis. In alternative medicine, we currently see a proliferation of trials that are not patient-blinded. We always suspected that they are at a high risk of generating false-positive results – now we know that this is, in fact, the case.

What should we do with this insight? In my view, the following steps would be wise:

  1. Take the findings from the existing trials that are devoid of patient-blinding with more than just a pinch of salt.
  2. Discourage the funding of future studies that fail to include patient-blinding.
  3. If patient-blinding is truly and demonstrably impossible – which is not often the case – make sure that the trialists at least include blinding of the assessors of the primary outcome measures.

There must be well over 10 000 clinical trials of acupuncture; Medline lists ~5 000, and many more are hidden in the non-Medline listed literature. That should be good news! Sadly, it isn’t.

It should mean that we now have a pretty good idea for what conditions acupuncture is effective and for which illnesses it does not work. But we don’t! Sceptics say it works for nothing, while acupuncturists claim it is a panacea. The main reason for this continued controversy is that the quality of the vast majority of these 10 000 studies is not just poor, it is lousy.

“Where is the evidence for this outraging statement???” – I hear the acupuncture-enthusiasts shout. Well, how about my own experience as editor-in-chief of FACT? No? Far too anecdotal?

How about looking at Cochrane reviews then; they are considered to be the most independent and reliable evidence in existence? There are many such reviews (most, if not all [co-]authored by acupuncturists) and they all agree that the scientific rigor of the primary studies is fairly awful. Here are the crucial bits of just the last three; feel free to look for more:

All of the studies had a high risk of bias

All included trials had a high risk of bias…

The studies were not judged to be free from bias…

Or how about providing an example? Good idea! Here is a new trial which could stand for numerous others:

This study was performed to compare the efficacy of acupuncture versus corticosteroid injection for the treatment of Quervain’s tendosynovitis (no, you do not need to look up what condition this is for understanding this post). Thirty patients were treated in two groups. The acupuncture group received 5 acupuncture sessions of 30 minutes duration. The injection group received one methylprednisolone acetate injection in the first dorsal compartment of the wrist. The degree of disability and pain was evaluated by using the Quick Disabilities of the Arm, Shoulder, and Hand (Q-DASH) scale and the Visual Analogue Scale (VAS) at baseline and at 2 weeks and 6 weeks after the start of treatment. The baseline means of the Q-DASH and the VAS scores were 62.8 and 6.9, respectively. At the last follow-up, the mean Q-DASH scores were 9.8 versus 6.2 in the acupuncture and injection groups, respectively, and the mean VAS scores were 2 versus 1.2. Thus there were short-term improvements of pain and function in both groups.

The authors drew the following conclusions: Although the success rate was somewhat higher with corticosteroid injection, acupuncture can be considered as an alternative option for treatment of De Quervain’s tenosynovitis.

The flaws of this study are exemplary and numerous:

  • This should have been a study that compares two treatments – the technical term is ‘equivalence trial – and such studies need to be much larger to produce a meaningful result. Small sample sizes in equivalent trials will always make the two treatments look similarly effective, even if one is a pure placebo.
  • There is no gold standard treatment for this condition. This means that a comparative trial makes no sense at all. In such a situation, one ought to conduct a placebo-controlled trial.
  • There was no blinding of patients; therefore their expectation might have distorted the results.
  • The acupuncture group received more treatments than the injection group; therefore the additional attention might have distorted the findings.
  • Even if the results were entirely correct, one cannot conclude from them that acupuncture was effective; the notion that it was similarly ineffective as the injections is just as warranted.

These are just some of the most fatal flaws of this study. The sad thing is that similar criticisms can be made for most of the 10 000 trials of acupuncture. But the point here is not to nit-pick nor to quack-bust. My point is a different and more serious one: fatally flawed research is not just a ‘poor show’, it is unethical because it is a waste of scarce resources and, even more importantly, an abuse of patients for meaningless pseudo-science. All it does is it misleads the public into believing that acupuncture might be good for this or that condition and consequently make wrong therapeutic decisions.

In acupuncture (and indeed in most alternative medicine) research, the problem is so extremely wide-spread that it is high time to do something about it. Journal editors, peer-reviewers, ethics committees, universities, funding agencies and all others concerned with such research have to work together so that such flagrant abuse is stopped once and for all.

1 2 3 8
Recent Comments
Click here for a comprehensive list of recent comments.
Categories