The aim of this pragmatic study was “to investigate the effectiveness of acupuncture in addition to routine care in patients with allergic asthma compared to treatment with routine care alone.”
Patients with allergic asthma were included in a controlled trial and randomized to receive up to 15 acupuncture sessions over 3 months plus routine care, or to a control group receiving routine care alone. Patients who did not consent to randomization received acupuncture treatment for the first 3 months and were followed as a cohort. All trial patients were allowed to receive routine care in addition to study treatment. The primary endpoint was the asthma quality of life questionnaire (AQLQ, range: 1–7) at 3 months. Secondary endpoints included general health related to quality of life (Short-Form-36, SF-36, range 0–100). Outcome parameters were assessed at baseline and at 3 and 6 months.
A total of 1,445 patients were randomized and included in the analysis (184 patients randomized to acupuncture plus routine care and 173 to routine care alone, and 1,088 in the nonrandomized acupuncture plus routine care group). In the randomized part, acupuncture was associated with an improvement in the AQLQ score compared to the control group (difference acupuncture vs. control group 0.7 [95% confidence interval (CI) 0.5–1.0]) as well as in the physical component scale and the mental component scale of the SF-36 (physical: 2.5 [1.0–4.0]; mental 4.0 [2.1–6.0]) after 3 months. Treatment success was maintained throughout 6 months. Patients not consenting to randomization showed similar improvements as the randomized acupuncture group.
The authors concluded that in patients with allergic asthma, additional acupuncture treatment to routine care was associated with increased disease-specific and health-related quality of life compared to treatment with routine care alone.
We have been over this so many times (see for instance here, here and here) that I am almost a little embarrassed to explain it again: it is fairly easy to design an RCT such that it can only produce a positive result. The currently most popular way to achieve this aim in alternative medicine research is to do a ‘A+B versus B’ study, where A = the experimental treatment, and B = routine care. As A always amounts to more than nothing – in the above trial acupuncture would have placebo effects and the extra attention would also amount to something – A+B must always be more than B alone. The easiest way of thinking of this is to imagine that A and B are both finite amounts of money; everyone can understand that A+B must always be more than B!
Why then do acupuncture researchers not get the point? Are they that stupid? I happen to know some of the authors of the above paper personally, and I can assure you, they are not stupid!
I am afraid there is only one reason I can think of: they know perfectly well that such an RCT can only produce a positive finding, and precisely that is their reason for conducting such a study. In other words, they are not using science to test a hypothesis, they deliberately abuse it to promote their pet therapy or hypothesis.
As I stated above, it is fairly easy to design an RCT such that it can only produce a positive result. Yet, it is arguably also unethical, perhaps even fraudulent, to do this. In my view, such RCTs amount to pseudoscience and scientific misconduct.
CBC news (Canada) reported yesterday that, more than a decade ago, the Manitoba Chiropractic Health Care Commission had been tasked to review the cost effectiveness of chiropractic services. It therefore prepared a report in 2004 for the Manitoba province and the Manitoba Chiropractors Association. Since then, this report has been kept secret. The report makes 37 recommendations, including:
- Manitoba Health should limit its funding to “chiropractic treatment of acute lower back pain.”
- Manitoba Health should provide “limited coverage of the treatment of neck pain.” The report called the literature around the efficacy of chiropractic care for neck pain “ambiguous or at best weakly supportive” and noted such treatment carried a “not insignificant safety risk.”
- Manitoba Health should not fund chiropractic treatment anyone under 18 “as the literature does not unequivocally justify” the “efficacy or safety” of such treatment.
A Manitoba Ombudsman’s Office report from 2012 might shed some light on why the Manitoba Chiropractic Health Care Commission’s report was never made public. Someone had attempted to get a copy of the report, but large parts of it were redacted. “Access to this record was refused on the basis that disclosure would be harmful to a third party’s business interest,” the ombudsman report notes, “and harm the economic or financial interests or negotiating position of a public body.”
The report also challenged claims that chiropractic treatments can be address a wide variety of medical conditions. It stated that there was not enough evidence to conclude chiropractic treatments are effective in treating muscle tension, migraines, HIV, carpal tunnel syndrome, gastrointestinal problems, infertility or cancer, or as a preventive care treatment. It also said there was not enough evidence to conclude chiropractic treatments are effective for children.
The report urged Manitoba Health to establish a monitoring system to keep a closer eye on “the advertising practices of the Manitoba Chiropractors Association and its members to ensure claims regarding treatments are restricted to those for which proof of efficacy and safety exist.” It suggested the government should have regulatory powers over chiropractic ads.
A recent CBC I-Team investigation found Manitoba chiropractors advertising treatment for a wide range of conditions including Alzheimer’s, autism and pediatric services. The commission report contained sharp criticisms of previous reports that suggested funding chiropractic care could save the health-care system money. Dr. Pranlal Manga authored two widely cited reports which claim that by offering publicly funded chiropractic care, provinces can cut health-care costs. “The Manga study on Manitoba must be rejected as a guide to public policy,” the commission report states, “because its assumptions, methodology and costing of recommendations are all deeply flawed.” The reports states, “What limited evidence the Commission has suggests he [Manga] grossly exaggerates possible medical savings.” Dr. Manga did not respond to CBC’s repeated attempts to contact him.
The commission report also made recommendations around the use of X-ray machines by chiropractors. It suggested chiropractors not own and operate X-ray machines “Given the restrictive conditions under which X-rays are advisable, their poor correlation with low-back problems, their apparent limitation as a guide to appropriate treatment …[and] the apparent complete lack of monitoring [of] the use of X-ray by chiropractors.” Instead, it recommended consulting with radiologists when imaging is deemed necessary. “The Commission is of the view that the public interest, and even chiropractic itself, would be better served if chiropractors had access to radiologists for this service, rather than perform it themselves,” the report said.
All three report authors declined comment. Calls to Dave Chomiak, who was health minister at the time the report was prepared, were not returned. In an email to CBC, Manitoba Chiropractors Association president Perry Taylor said, “I personally have never seen this 13-year-old document and [it] pre-dates my time as President. As such I have no comment on this.” The CBC I-Team offered to go through the report with Taylor but he did not respond.
This report seems to confirm much of what we have discussed repeatedly on this blog: Chiropractic is not nearly as effective and safe as chiropractors try to make us believe. To hide this fact is certainly dishonest and unethical, but it is in some ways understandable: this knowledge would directly threaten the income of most chiropractors.
Yesterday I commented on another post: “the conflict of interest seems obvious: if homeopaths speak the truth, they are out of business. therefore, they are taught untruths from the first day of their training and eventually end up believing them. there is only one solution, as far as I can see: regulators must prevent them from making false claims. if not, this will go on for another 200 years and damage many patients’ health”. In the light of the above report, I will now re-phrase this: the conflict of interest seems obvious: if chiropractors allowed the truth to be known, they would soon be out of business. Therefore, they are taught untruths from the first day of their training and many end up believing them. There is only one solution, as far as I can see: regulators must prevent chiropractors from making false claims. If not, this abuse will go on for another 120 years and damage many patients’ health.
Charlotte Leboeuf-Yde, DC,MPH,PhD, is professor in Clinical Biomechanics at the University of Southern Denmark and works at the French-European Institute of Chiropractic in Paris. She is a chiropractor with extensive research experience, for example, she was one of the first chiropractors to have studied adverse reactions of spinal manipulation.
Charlotte certainly knows a thing or two about adverse effects of spinal manipulation, and I have always found her work interesting. Therefore, I was delighted to find a recent blog post where she discussed the Cassidy study of 2008 and two opposed views on the validity of this much-discussed paper.
One team (Paulus &Thaler) argued, Charlotte explained, that the Cassidy case-control study is faulty, because vertebro-basilar stroke in general was not separated from stroke specifically caused by vertebral artery dissections, the presumed culprit in cervical spinal manipulation. According to Paulus & Thaler, this would potentially result in a dilution of ‘real’ manipulative-related strokes among all other causes of stroke that are much more common. They argue that the Cassidy-analyses therefore were polluted by this misclassification, whereas the other team (Murphy et al) vehemently disagrees.
The final word is clearly not yet pronounced on this issue, Charlotte concluded, and both teams agree that research has to address various methodological challenges to obtain a trustable answer. Nevertheless, without an international collaboration involving prospective cases this seems an almost impossible task, particularly in view of the rarity of the condition; problems in capturing all cases (going from the reversible to the permanent injuries); the likely large anatomical and physiological variations between individuals; and the daunting task of obtaining relevant and precise descriptions of treatments from a multitude of practitioners.
In the meantime, Charlotte concluded, “practitioners and patients have to make a decision, similarly to judging risk in other walks of life, such as, should I take the plane or stay at home?”
I have always thought highly of Charlotte’s work, however, her conclusion made me doubt whether my high opinion of her reasoning was justified.
Should I take the plane or stay at home?
This question is not remotely similar to the question “should I have chiropractic upper neck manipulation or not?”
Here are a the two main reasons why:
- Taking the plane of demonstrably effective in transporting you from A to B, while neck manipulation is not demonstrably effective for anything.
- If you want to go from A to B [assuming B is far way], you need to fly. If you have neck pain or other symptoms, you can employ plenty of therapies other than neck manipulations.
Charlotte Leboeuf-Yde, DC,MPH,PhD, may be a professor in Clinical Biomechanics etc., etc., however, logical and critical thinking do not seem to be her forte.
So, how should we deal with the risks of chiropractic neck manipulations? I think, we should deal with them as responsible healthcare professionals deal with any other suspected therapeutic risks: we must ask whether the known risks of the treatment outweigh the known benefits (as they do with spinal manipulation). If that is so, we have an ethical, legal and moral duty not to employ the therapy in question in routine care. At the same time, we must focus or research efforts on producing full clarity about the open questions. It’s called the precautionary principle!
The ‘SOCIETY OF HOMEOPATHS’ (SoH) have published an official complaint they recently filed with the BBC. As it gives an intriguing insight into their mind-set, I could not resist reproducing it here (warts and all):
“Prompted by the interview with Simon Stevens of NHS England on the Today Programme, on 31st March, the Society of Homeopaths deplores the lack of balance in the BBC’s coverage of Homeopathy and urges you to review your approach to coverage of the subject.
During the Today interview, following wide-ranging discussion of issues around the future of the NHS, Sarah Montague suddenly threw in a question about the amount spent on Homeopathy within the NHS, evidently catching Mr Stevens unawares.
The annual budget of the NHS is approximately £110billion. Of this, £4million per year (0.0036 of the NHS budget) is spent on Homeopathy. This hardly justifies the unbalanced and hectoring approach from Sarah Montague.
We acknowledge that it is not always possible or necessary to achieve balance on a particular topic within a single programme but the BBC seems to have a consistent line across all of its platforms of opposition to, and disparagement of, Homeopathy. A recent example is a piece on the Health section of the BBC website in October 2106 by Nick Tiggle which displayed no balance at all and denigrated Homeopathy and Homeopaths with little or no space given to alternative views.
From these and other instances, it seems clear that the BBC has a biased attitude towards Homeopathy, which may be the result of relying too heavily on a small number of ‘experts’, who openly and persistently campaign against complementary and alternative medicine. These ‘experts’ operate in a similar way to climate change deniers, referring to a limited range of research, often of poor quality, to support their claims that there is ‘no evidence for homeopathy’.
We look forward to BBC programmes which fulfill its mission to explain and provide balance and coverage of the positive effects of Homeopathy.
Mark Taylor Chief Executive Society of Homeopaths”
END OF QUOTE
This hardly needs a comment – perhaps just 6 short points:
- To the best of my knowledge, the BBC has a policy of not being seen to be biased. The discussion referred to above was about the NHS stopping to pay for treatments that are either not effective (e. g. cough syrups) or cheaper to buy OTC than on prescription (e. g. paracetamol). Homeopathy is both. Therefore it would have even been biased NOT to bring homeopathy into the discussion.
- To claim the BBC-interviewer caught Stevens off guard is just silly: when you go on the radio to discuss such issues, homeopathy MUST be on your mind.
- To claim that the BBC is generally biased against homeopathy (on the basis of two anecdotes) is equally silly. The SoH should have done some systematic research on this – perhaps they did and found it failed to support their point? – this would have shown that there is plenty of (far too much) pro-homeopathy stuff on the BBC.
- To say or imply that homeopathy is of debatable or even no value to the NHS does not disclose bias; on the contrary, it is a reflection of the scientific truth which the BBC has an obligation to report.
- With their complaint, the SoH disclose an embarrassing degree of naivety and an alarming detachment from reality.
- Whichever way a rational observer might look at this, the BBC should in future become a much more outspoken defender of the scientific truth – on homeopathy and everything else!!!
The recent meta-analysis by Mathie et al for non-individualised homeopathy (recently discussed here) identified just 3 RCTs that were rated as ‘reliable evidence’. But just how rigorous are these ‘best’ studies? Let’s find out!
THE FIRST STUDY
The objective of the first trial was “to evaluate the efficacy of the non-hormonal treatment BRN-01 in reducing hot flashes in menopausal women.” Its design was that of a multicentre (35 centres in France), randomized, double-blind, placebo-controlled. One hundred and eight menopausal women, ≥50 years of age, were enrolled in the study. The eligibility criteria included menopause for <24 months and ≥5 hot flashes per day with a significant negative effect on the women’s professional and/or personal life. Treatment was either BRN-01 tablets, a registered homeopathic medicine [not registered in the UK] containing Actaea racemosa (4 centesimal dilutions [4CH]), Arnica montana (4CH), Glonoinum (4CH), Lachesis mutus (5CH), and Sanguinaria canadensis (4CH), or placebo tablets, prepared by Laboratoires Boiron according to European Pharmacopoeia standards [available OTC in France]. Oral treatment (2 to 4 tablets per day) was started on day 3 after study enrolment and was continued for 12 weeks. The main outcome measure was the hot flash score (HFS) compared before, during, and after treatment. Secondary outcome criteria were the quality of life (QoL) [measured using the Hot Flash Related Daily Interference Scale (HFRDIS)], severity of symptoms (measured using the Menopause Rating Scale), evolution of the mean dosage, and compliance. All adverse events (AEs) were recorded. One hundred and one women were included in the final analysis (intent-to-treat population: BRN-01, n = 50; placebo, n = 51). The global HFS over the 12 weeks, assessed as the area under the curve (AUC) adjusted for baseline values, was significantly lower in the BRN-01 group than in the placebo group (mean ± SD 88.2 ± 6.5 versus 107.2 ± 6.4; p = 0.0411). BRN-01 was well tolerated; the frequency of AEs was similar in the two treatment groups, and no serious AEs were attributable to BRN-01. The authors concluded that BRN-01 seemed to have a significant effect on the HFS, compared with placebo. According to the results of this clinical trial, BRN-01 may be considered a new therapeutic option with a safe profile for hot flashes in menopausal women who do not want or are not able to take hormone replacement therapy or other recognized treatments for this indication.
Laboratoires Boiron provided BRN-01, its matching placebo, and financial support for the study. Randomization and allocation were carried out centrally by Laboratoires Boiron. I would argue that the treatment time in this study was way too short for generating a therapeutic response. The evolution of the HFS in the two groups was assessed by analysis of the area under the curve (AUC) of the mean scores recorded weekly from each patient in each group over the duration of the study, including those at enrollment (before any treatment). I wonder whether this method was chosen only when the researchers noted that the HFS at the pre-defined time points did not yield a significant result or whether it was pre-determined (elsewhere in the methods section we are told that “The primary evaluation criterion was the effect of BRN-01 on the HFS, compared with placebo. The HFS was defined as the product of the daily frequency and intensity of all hot flashes experienced by the patient, graded by the women from 1 to 4 (1 = mild; 2 = moderate; 3 = strong; 4 = very strong). These data were recorded by the women on a self-administered questionnaire, assisted by a telephone call from a clinical research associate. Data were collected (i) during the first 2 days after enrolment and before any medication had been taken; (ii) then every Tuesday and Wednesday of each week until the 11th week of treatment, inclusive; and (iii) finally, every day of the 12th week of treatment.”). Two of the authors of this paper are employees of Boiron.
THE SECOND STUDY
The second trial was aimed at finding out “whether a well-known and frequently prescribed homeopathic preparation could mitigate post-operative pain.” It was a randomized, double-blind, placebo-controlled trial to evaluate the efficacy of the homeopathic preparation Traumeel S® in minimizing post-operative pain and analgesic consumption following surgical correction of hallux valgus. Eighty consecutive patients were randomized to receive either Traumeel tablets or an indistinguishable placebo, and took primary and rescue oral analgesics as needed. Maximum numerical pain scores at rest and consumption of oral analgesics were recorded on day of surgery and for 13 days following surgery. Traumeel was not found superior to placebo in minimizing pain or analgesic consumption over the 14 days of the trial, however a transient reduction in the daily maximum post-operative pain score favoring the Traumeel arm was observed on the day of surgery, a finding supported by a treatment-time interaction test (p = 0.04). The authors concluded that Traumeel was not superior to placebo in minimizing pain or analgesic consumption over the 14 days of the trial. A transient reduction in the daily maximum post-operative pain score on the day of surgery is of questionable clinical importance.
Traumeel is a mixture of 6 ingredients, 4 of which are in the D2 potency. Thus it neither is administered as a homeopathic remedy (no ‘like cures like’) nor is it highly diluted. In fact, it is not homeopathy at all but belongs to a weird offspring of homeopathy called ‘homotoxicology’ [this is an explanation from my book: Homotoxicology is a method inspired by homeopathy which was developed by Hans Heinrich Reckeweg (1905 – 1985). He believed that all or most illness is caused by an overload of toxins in the body. The toxins originate, according to Reckeweg, both from the environment and from the malfunction of physiological processes within the body. His treatment consists mainly in applying homeopathic remedies which usually consist of combinations of single remedies, because health cannot be achieved without ridding the body of toxins. The largest manufacturer and promoter of remedies used in homotoxicology is the German firm Heel.] The HEEL Company (Baden-Baden, Germany) provided funding for the performance and monitoring of this project, supplied the study medication and placebo, and prepared the randomization list. The positive outcome mentioned in the authors’ conclusion refers to a secondary endpoint. I would argue that the authors should not have noted it there and should have made it clear that the trial generated a negative result.
THE THIRD STUDY
Finally, the third of the 3 ‘rigorous’ studies “evaluated the effectiveness of the homeopathic preparation Plumbum Metallicum (PM) in reducing the blood lead levels of workers exposed to this metal.” The Brazilian researchers recruited 131 workers to this RCT who took PM in the CH15 potency or placebo for 35 days (10 drops twice daily). Thereafter, the percentage of workers whose lead level had fallen by at least 25% did not differ between the groups, both on intention to treat and per protocol analyses. The authors concluded that PM “had no effect in this study in terms of reducing serum lead in workers exposed to lead.”
This study lacks a power calculation, and arguably the period might have been too short to show an effect. The trial was published in the journal HOMEOPATHY which, some might argue, has not the most rigorous of peer-review procedures.
The third study seems the most rigorous by far, in my view. The other two trials are seriously under-whelming in several respects, primarily because we cannot be sure how much influence the commercial interests of the sponsor had on their findings. I am sure others will spot weaknesses in all three trials that I failed to see.
Mathie et al partly disagree with my assessment when they write in their paper: “We report separately our model validity assessments of these trials, evaluating consequently their overall quality based on a GRADE-like principle of ‘downgrading’ : two trials [23, 25] rated here as reliable evidence were downgraded to ‘low quality’ overall due to the inadequacy of their model validity; the remaining trial with reliable evidence  was judged to have adequate model validity. The latter study  thus comprises the sole RCT that can be designated ‘high quality’ overall by our approach, a stark finding that reveals further important aspects of the preponderantly low quality of the current body of evidence in non-individualised homeopathy.”
What Mathie et al seem to forget entirely is that none of the 3 RCTs is a trial of homeopathy as defined by treatment according to the ‘like cures like’ principle. The authors of the second study acknowledge this fact by stating: “Homeopathic purists may find fault in the administration of a standardized combination homeopathic formula to all patients, based upon clinical diagnosis – as opposed to the individualized manner dictated by standard homeopathic practice.”
So, which ever way we look upon this evidence, we cannot possibly deny that the evidence for non-individualised homeopathy is rubbish.
D D Palmer was born on March 7, 1845; so, why do chiros celebrate the ‘CHIROPRACTIC AWARENESS WEEK’ from 10 – 16 of April? Perhaps out of sympathy with the homeopaths (many US chiros also use homeopathy) who had their ‘big week’ during the same period? Please tell me, I want to know!
Anyway, the HAW almost ‘drowned’ the CAW – but only almost.
The British Chiropractic Association did its best to make sure we don’t forget the CAW. On their website, we find an article that alerts us to their newest bit of research. Here are some excerpts:
The consumer survey by the British Chiropractic Association (BCA) of more than 2,000 UK adults who currently suffer from back or neck pain, or have done so in the past, found that almost three in five (56%) people experienced pain after using some form of technological device. Despite this, only 27% of people surveyed had limited or stopped using their devices due to concerns for their back or neck health and posture. The research showed people were most likely to experience back or neck pain after using the following technological devices:
• Laptop computer (35%)
• Desktop computer (35%)
• Smart phone (22%)
• Tablet (20%)
• Games console (17%)
The age group most likely to experience back or neck pain when using their smart phone were 16-24 year olds, while nearly half (45%) of young adults 25-34 year olds) admitted to experiencing back or neck pain after using a laptop. One in seven (14%) 16-24 year olds attributed their back or neck pain to virtual reality headsets.
As part of Chiropractic Awareness Week (10-16 April) the BCA is calling for technology companies to design devices with posture in mind, to help tech proof our back health. BCA chiropractor Rishi Loatey comments: “We all know how easy it is to remain glued to our smart phone or tablet, messaging friends or scrolling through social media. However, this addiction to technology could be causing changes to posture, which can lead to increased pressure on the muscles, joints and discs in the spine. Technology companies are now starting to issue older phone models which hark back to a time before smart phones enabled people to do everything from check emails and take pictures, to internet banking. Returning to a time of basic functionality, which may see people look to limit the time spent on their phone, can only be good news for our backs. Yet, in an age where people can now track their health and wellbeing using their phone, technology companies should also start looking at ways to make their devices posture friendly from the outset, encouraging us to take time away from our desks and breaks from our scrolling, gaming and messaging.”
END OF QUOTE
So, here we have it: another piece of compelling, cutting edge research by the BCA. They have made us giggle before but rarely have I laughed so heartily about a ‘professional’ organisation confusing so unprofessionally correlation with causation.
Considering the amount of highly public blunders they managed to inflict on the profession in recent years, I have come to the conclusion that the BCA is a cover organisation of BIG PHARMA with the aim of giving chiropractic a bad name!
‘What Doctors Don’t Tell You’ (WDDTY) have been shown to be strangely economical with the truth many times before (for instance here, here and here). Now they have published an article entitled ‘Ombudsman investigates ‘flawed’ homeopathic study that claimed it doesn’t work’ It attacks in no uncertain terms the ‘NHMRC Statement on Homeopathy and NHMRC Information Paper – Evidence on the effectiveness of homeopathy for treating health conditions’ which I believe to be a sound evaluation of homeopathy and therefore have mentioned repeatedly on this blog. Here is what WDDTY stated:
START OF QUOTE
A major and influential review of homeopathy concluded that the controversial therapy doesn’t work—but it was so riddled with error and bad science that it’s sparked an official ombudsman investigation.
The world’s media announced that homeopathy was a scam after the Australia government’s National Health and Medical Research Council (NHMRC) published its findings in 2015 that “there are no health conditions for which there is reliable evidence that homeopathy is effective.”
But now the Commonwealth Ombudsman is investigating the review’s procedures after receiving reports of inaccuracies, mishandling of evidence and conflicts of interest.
The review has been triggered by the Australian Homeopathic Association (AHA), supported by the Homeopathic Research Institute (HRI), which began questioning the review’s processes after several solid studies that demonstrated homeopathy’s benefits had been overlooked.
The NHMRC review team set arbitrary parameters that only studies that involved more than 150 people—and which met standards that even drug trials rarely achieve—would be considered. Those requirements reduced the number of qualifying studies to just five—from an initial pool of more than 1,800 trials—and none of these showed that homeopathy was effective.
One of the NHMRC’s own reviewers produced a mysterious first report that has never been published, and hasn’t been released despite Freedom of Information requests.
And the AHA has discovered that Prof Peter Brooks, chair of the NHMRC committee that carried out the homeopathy review, never declared that he was a member of the anti-homeopathy lobby group, Friends of Science in Medicine.
There are solid studies that demonstrate homeopathy is effective against childhood diarrhea, sinusitis and hay fever—but they all involve fewer than 150 people, said HRI chief executive Rachel Roberts. “The public has a right to know that there are high quality studies showing homeopathy works for some medical conditions—information that was lost only due to NHMRC’s mishandling of the evidence.”
The homeopaths aren’t alone in challenging the NHMRC review: Australia’s independent Cochrane Centre said its conclusions are not an accurate reflection of the evidence, and a second expert also said he felt “uncertain of the definitive nature of the report’s conclusions.”
END OF QUOTE
As it happens, I am in contact with the lead author of this report, Paul Glasziou, not least because he very kindly wrote the foreword for my book HOMEOPATHY, THE UNDILUTED FACTS. So, we corresponded and discussed the latest WDDTY diatribe. Thus I am now in a position to put a few things straight (I hope Paul does not mind).
ISSUE 1. – The NHMRC review team set arbitrary parameters that only studies involving more than 150 people—and which met standards that even drug trials rarely achieve—would be considered.
The truth is that report focused on systematic reviews of trials, not individual trials. The 57 included systematic reviews found 176 individual trials which covered 61 conditions: an average of about 3 trials per condition. But some conditions only had 1 trial, and one small trial would, of course, not be considered a reasonable basis for reliable conclusions. GRADE – the international standard for assessing evidence – downgrades reviews for “imprecision” – the GRADE Handbook suggests “whenever there are sample sizes that are less than 400, review authors and guideline developers should certainly consider rating down for imprecision.” Hence the criterion of 150 which the Australians decided to use is considerably more lenient than the current GRADE guideline.
ISSUE 2 – Those requirements reduced the number of qualifying studies to just five—from an initial pool of more than 1,800 trials—and none of these showed that homeopathy was effective.
This is simply not correct. The report found 57 systematic reviews that contained 176 individual trials, not 5. These 176 trials, which covered 61 conditions, formed the body of evidence for the NHMRC report’s conclusions.
ISSUE 3 – There are solid studies that demonstrate homeopathy is effective against childhood diarrhoea, sinusitis and hay fever—but they all involve fewer than 150 people, said HRI chief executive Rachel Roberts.
The NHMRC report focused on systematic reviews that covered all trials for individual conditions. Given the conventional p-value of 0.05, one would expect 1 in 20 single trials to be “false positives”. So with 176 trials, we expect about 9 “false positive” trials. But using systematic reviews that combine all trials for individual conditions, reduces this risk of false positives. Most national evidence review bodies require more than 1 trial, e.g, the FDA requires 2 positive trials, whereas many others require a systematic review which has at least 2 trials. Replication of findings is obviously a cornerstone of science.
ISSUE 4 The homeopaths aren’t alone in challenging the NHMRC review: Australia’s independent Cochrane Centre said its conclusions are not an accurate reflection of the evidence, and a second expert also said he felt “uncertain of the definitive nature of the report’s conclusions.”
The truth is that the Cochrane Centre, which provided an independent check during the processes of the NHMRC review, concluded that “Overall, the conclusions arising from the review appear justified based on the evidence presented.”
I REST MY CASE.
This new systematic review by proponents of homeopathy (and supported by a grant from the Manchester Homeopathic Clinic) tested the null hypothesis that “the main outcome of treatment using a non-individualised (standardised) homeopathic medicine is indistinguishable from that of placebo“. An additional aim was to quantify any condition-specific effects of non-individualised homeopathic treatment. In reporting this paper, I will stay very close to the published text hoping that this avoids both misunderstandings and accusations of bias on my side:
Literature search strategy, data extraction and statistical analysis followed the methods described in a pre-published protocol. A trial comprised ‘reliable evidence’ if its risk of bias was low or it was unclear in one specified domain of assessment. ‘Effect size’ was reported as standardised mean difference (SMD), with arithmetic transformation for dichotomous data carried out as required; a negative SMD indicated an effect favouring homeopathy.
The authors excluded the following types of trials: studies of crossover design; of radionically prepared homeopathic medicines; of homeopathic prophylaxis; of homeopathy combined with other (complementary or conventional) intervention; for other specified reasons. The final explicit exclusion criterion was that there was obviously no blinding of participants and practitioners to the assigned intervention.
Forty-eight different clinical conditions were represented in 75 eligible RCTs; 49 were classed as ‘high risk of bias’ and 23 as ‘uncertain risk of bias’; the remaining three trials displayed sufficiently low risk of bias to be designated reliable evidence. Fifty-four trials had extractable data: pooled SMD was -0.33 (95% confidence interval (CI) -0.44, -0.21), which was attenuated to -0.16 (95% CI -0.31, -0.02) after adjustment for publication bias. The three trials with reliable evidence yielded a non-significant pooled SMD: -0.18 (95% CI -0.46, 0.09). There was no single clinical condition for which meta-analysis produced reliable evidence.
A meta-regression was performed to test specifically for within-group differences for each sub-group. The results showed that there were no significant differences between studies that were and were not:
- included in previous meta-analyses (p = 0.447);
- pilot studies (p = 0.316);
- greater than the median sample (p = 0.298);
- potency ≥ 12C (p = 0.221);
- imputed for meta-analysis (p = 0.384);
- free from vested interest (p = 0.391);
- acute/chronic (p = 0.796);
- different types of homeopathy (p = 0.217).
After removal of ‘C’-rated trials, the pooled SMD still favoured homeopathy for all sub-groups, but was statistically non-significant for 10 of the 18 (included in previous meta-analysis; pilot study; sample size > median; potency ≥12C; data imputed; free of vested interest; not free of vested interest; combination medicine; single medicine; chronic condition). There remained no significant differences between sub-groups—with the exception of the analysis for sample size > median (p = 0.028).
Meta-analyses were possible for eight clinical conditions, each analysis comprising two to 5 trials. A statistically significant pooled SMD, favouring homeopathy, was observed for influenza (N = 2), irritable bowel syndrome (N = 2), and seasonal allergic rhinitis (N = 5). Each of the other five clinical conditions (allergic asthma, arsenic toxicity, infertility due to amenorrhoea, muscle soreness, post-operative pain) showed non-significant findings. Removal of ‘C’-rated trials negated the statistically significant effect for seasonal allergic rhinitis and left the non-significant effect for post-operative pain unchanged; no higher-rated trials were available for additional analysis of arsenic toxicity, infertility due to amenorrhoea or irritable bowel syndrome. There were no ‘C’-rated trials to remove for allergic asthma, influenza, or muscle soreness. Thus, influenza was the only clinical condition for which higher-rated trials indicated a statistically significant effect; neither of its contributing trials, however, comprised reliable evidence.
The authors concluded that the quality of the body of evidence is low. A meta-analysis of all extractable data leads to rejection of our null hypothesis, but analysis of a small sub-group of reliable evidence does not support that rejection. Reliable evidence is lacking in condition-specific meta-analyses, precluding relevant conclusions. Better designed and more rigorous RCTs are needed in order to develop an evidence base that can decisively provide reliable effect estimates of non-individualised homeopathic treatment.
I am sure that this paper will lead to lively discussions in the comments section of this blog. I will therefore restrict my comments to a bare minimum.
In my view, this new meta-analysis essentially yield a negative result and confirms most previous, similar reviews.
- It confirms Linde’s conclusion that “insufficient evidence from these studies that homeopathy is clearly efficacious for any single clinical condition”.
- It confirms Linde’s conclusion that “there was clear evidence that studies with better methodological quality tended to yield less positive results”.
- It confirms Kleinjen’s conclusion that “most trials are of low methodological quality”.
- It also confirms the results of the meta-analysis by Shang et al (much-maligned by homeopaths) than “finding is compatible with the notion that the clinical effects of homoeopathy are placebo effects.”
- Finally, it confirms the conclusion of the analysis of the Australian National Health and Medical Research Council: “Homeopathy should not be used to treat health conditions that are chronic, serious, or could become serious. People who choose homeopathy may put their health at risk if they reject or delay treatments for which there is good evidence for safety and effectiveness. People who are considering whether to use homeopathy should first get advice from a registered health practitioner. Those who use homeopathy should tell their health practitioner and should keep taking any prescribed treatments.”
Another not entirely unimportant point that often gets missed in these discussions is this: even if we believe (which I do not) the most optimistic interpretation of these (and similar data) by homeopaths, we ought to point out that there is no evidence whatsoever that homeopathy cures anything. At the very best it provides marginal symptomatic relief. Yet, the claim of homeopaths that we hear constantly is that homeopathy is a causal and curative therapy.
The first author of the new meta-analysis is an employee of the Homeopathy Research Institute. We might therefore forgive him that he he repeatedly insists on dwelling on largely irrelevant (i. e. based on unreliable primary studies) findings. It seems obvious that firm conclusions can only be based on reliable data. I therefore disregard those analyses and conclusions that include such studies.
In the discussion, the authors of the new meta-analysis confirm my interpretation this by stating that they “reject the null hypothesis (non-individualised homeopathy is indistinguishable from placebo) on the basis of pooling all studies, but fail to reject the null hypothesis on the basis of the reliable evidence only.” And, in the long version of their conclusions, we find this remarkable statement: “Our meta-analysis of the current reliable evidence base therefore fails to reject the null hypothesis that the outcome of treatment using a non-individualised homeopathic medicine is not distinguishable from that using placebo.” A most torturous way of stating the obvious: the more reliable data show no difference between homeopathy and placebo.
“Millions of people have adverse drug reactions to prescribed medicine; it is ranked as the third leading causes of death. In the US, health-care spending reached $1.6 trillion in 2003. Considering this enormous expenditure, we should have the best medicine in the world. But we don’t. Bottom line, people are suffering. The public is calling out for a reform in mainstream medicine.” These seem to be the conclusions of a new film about homeopathy entitled JUST ONE DROP. It was shown recently for the first time in London, and we already have a fascinating comment about it.
“This professional, eight-year effort attempted to be quite even-handed, while featuring many compelling and documented success stories”, states “The World’s No. 1 Authority on Intention, Spirituality and the New Science”, Lynne Mc Taggart. An ‘even-handed’ effort is worth pursuing, I thought, and so I read on.
When I reached the point where Lynne writes “The greatest revelation had to do with the dirty pool employed by the Australian government’s National Health and Medical Research Council (NHMRC), when it decided to assess the effectiveness of homeopathy by reviewing all research that had been done to date”, I got a little suspicious. We discussed the NHMRC report here and here and I had an entirely different impression of it.
Lynne also elaborated at length on the infamous ‘Swiss report’: “The Swiss team had comprehensively reviewed all the major evidence for homeopathy, everything from preclinical research to double-blind, placebo-controlled studies and meta-analyses. After assessing all the available data, the Swiss team concluded that the high-quality investigations of preclinical basic research proved that homeopathic high-potency remedies induce “regulative and specific changes in cells or living organisms”. Of the systematic reviews of human research, said the report, 20 out of 22 detected “at least a trend in favor of homeopathy”, and five showed “clear evidence for homeopathic therapy”. The report found particularly strong evidence for the use of homeopathy for upper respiratory tract infections and allergic reactions. Perhaps most significantly, the report concluded that the effectiveness of homeopathy “can be supported by clinical evidence” and “regarded as safe”.(Forsch Komple-mentmed, 2006; 13 Suppl 2: 19–29).”
This report has been evaluated by many experts (see for instance here). One expert even called it ‘research misconduct’ and concluded that “…the authors of this report adopted a very unusual strategy in what should have been an impartial evidence appraisal. It appears that their goal was not to provide an independent assessment but to choose criteria that would lead to their chosen conclusion that homeopathy is effective. To this end, they chose to adopt a highly questionable criterion of “real-world” effectiveness, ignore negative findings concerning homeopathy in favour of implausible reinterpretation of results, and attack RCTs. This use of a unique and suspect methodology in an appraisal designed to assess healthcare objectively gives cause for particular concern; one imagines that the Swiss government wanted homeopathy to be judged against existing standards rather than new ones created specially for the evaluation. In doing so the authors have distorted the evidence and misled the public; these actions, combined with their conflicts of interest, strongly suggest that they are guilty of research misconduct. It is extremely unfortunate that the Swiss government lent legitimacy to this report by attaching its name to it, and also unfair that the English-language text is not available free of charge to the public when it is being widely misrepresented all over the world as proof of the efficacy of homeopathy. It remains possible that homeopathy is effective, but the authors of this report do the practice a grave disservice.”
Could it be that Lynne does not know all this?
Or is she not interested in an ‘even-handed’ approach?
For me, the last drop arrived when Lynne started writing about my friend “Simon Singh, the self-appointed attack dog on all things alternative”, as she calls him. This is where I began to feel nauseous, so much so that I had to reach for my Nux Vomica C30. Alas, it did not help – Lynne’s writing was too overpoweringly sickening, particularly when she tried to motivate her readers to defend charities that endanger public health by promoting bogus treatments for life-threatening conditions: “If you value alternative medicine, here’s what to write the CC before the deadline: Tell them to read the Swiss report on homeopathy, the most contentious of alternative therapies, which shows very good evidence for it. Demand a level playing field. If they are going to challenge charities for alternative medicine based on scientific evidence, then they need to evaluate Cancer Research UK, Arthritis Research UK and every other charity partly or wholly funded by pharmaceutical companies, an estimated 75 percent of whose research is massaged, manipulated or fabricated.”
My only hope now is that the film JUST ONE DROP is less bonkers than Lynne’s comment about it!
Tui Na is a massage technique that is based on the Taoist principles of TCM. It involves a range of manipulations usually performed by an operator’s finger, hand, elbow, knee, or foot applied to muscle or soft tissue at specific parts of the body. According to one website of TCM-proponents “Tui Na makes use of various hand techniques in combination with acupuncture and other manipulation techniques. To enhance the healing process, the practitioner may recommend the use of Chinese herbs. Many of the techniques used in this massage resemble that of a western massage like gliding, kneading, vibration, tapping, friction, pulling, rolling, pressing and shaking. In Tui Na massage, the muscles and tendons are massaged with the help of hands, and an acupressure technique is applied to directly affect the flow of Qi at different acupressure points of the body, thus facilitating the healing process. It removes the blockages and keeps the energy moving through the meridians as well as the muscles. A typical session of Tui Na massage may vary from thirty minutes to an hour. The session timings may vary depending on the patient’s needs and condition. The best part of the therapy is that it relaxes as well as energizes the person. The main benefit of Tui Na massage is that it focuses on the specific problem, whether it is an acute or a chronic pain associated with the joints, muscles or a skeletal system. This technique is very beneficial in reducing the pain of neck, shoulders, hips, back, arms, highs, legs and ankle disorders. It is a very effective therapy for arthritis, pain, sciatica and muscle spasms. Other benefits of this massage therapy include alleviation of the stress related disorders like insomnia, constipation, headaches and other disorders related to digestive, respiratory and reproductive systems. The greatest advantage of Tui Na is that it focuses on maintaining overall balance with both physical and mental health. Any one who wants to avoid the side effects of drugs or a chemical based treatment can adopt this effective massage technique to alleviate their pain. Tui Na massage therapy is now becoming a more common therapy method due to its focus on specific problems rather than providing a general treatment.”
This clearly begs the question IS IT EFFECTIVE?
This systematic review assessed the evidence of Tui Na for cervical radiculopathy. Seven databases were searched. Randomised controlled trials (RCTs) incorporating Tui Na alone or Tui Na combined with conventional treatment were included. Five studies involving 448 patients were found. The pooled analysis from the 3 trials indicated that Tui Na alone showed a significant lowering immediate effects on pain score with moderate heterogeneity compared to cervical traction. The meta-analysis from 2 trials revealed significant immediate effects of Tui Na plus cervical traction in improving pain score with no heterogeneity compared to cervical traction alone. None of the RCTs mentioned adverse effects. There was very low quality or low quality evidence to support the results.
The authors concluded that “Tui Na alone or Tui Na plus cervical traction may be helpful to cervical radiculopathy patients, but supportive evidence seems generally weak. Future clinical studies with low risk of bias and adequate follow-up design are recommended.”
In my view, this is a misleading conclusion. A correct one would have been: THE CURRENT EVIDENCE IS INSUFFICIENT TO DRAW ANY CONCLUSIONS ABOUT THE EFFECTIVENESS OF TUI NA.
Here are some of the most obvious reasons:
- there are far too few studies for a firm conclusion,
- the included RCTs lack scientific rigour,
- all trials originate from China where reliability seems to be a serious problem,
- traction is not a useful therapy for radiculopathy,
- the primary studies violate research ethics by not reporting adverse effects.
Personally, I am getting very tired of conclusions stating ‘…XY MAY BE EFFECTIVE/HELPFUL/USEFUL/WORTH A TRY…’ It is obvious that the therapy in question MAY be effective, otherwise one would surely not conduct a systematic review. If a review fails to produce good evidence, it is the authors’ ethical, moral and scientific obligation to state this clearly. If they don’t, they simply misuse science for promotion and mislead the public. Strictly speaking, this amounts to scientific misconduct.