Chiropractors are back pain specialists, they say. They do not pretend to treat non-spinal conditions, they claim.
If such notions were true, why are so many of them still misleading the public? Why do many chiropractors pretend to be primary care physicians who can take care of most illnesses regardless of any connection with the spine? Why do they continue to happily promote bogus treatments? Why do chiropractors, for instance, claim they can treat gastrointestinal diseases?
This recent narrative review of the literature, for example, was aimed at summarising studies describing the management of disorders of the gastrointestinal (GI) tract using ‘chiropractic therapy’ broadly defined here as spinal manipulation therapy, mobilizations, soft tissue therapy, modalities and stretches.
Twenty-one articles were found through searching the published literature to meet the authors’ inclusion criteria. The retrieved articles included case reports to clinical trials to review articles. The majority of articles chronicling patient experiences under chiropractic care reported that they experienced mild to moderate improvements in GI symptoms. No adverse effects were reported.
From this, the authors concluded that chiropractic care can be considered as an adjunctive therapy for patients with various GI conditions providing there are no co-morbidities.
I think, we would need to look for a long time to find an article with conclusions that are more ridiculous, false and unethical than these.
The old adage applies: rubbish in, rubbish out. If we include unreliable reports such as anecdotes, our finding will be unreliable as well. If we do not make this mistake and conduct a proper systematic review, we will arrive at very different conclusions. My own systematic review, for instance, of controlled clinical trials drew the following conclusion: There is no supportive evidence that chiropractic is an effective treatment for gastrointestinal disorders.
That probably says it all. I only want to add a short question: SHOULD THIS LATEST CHIROPRACTIC ATTEMPT TO MISLEAD THE PUBLIC BE CONSIDERED ‘SCIENTIFIC MISCONDUCT’ OR ‘FRAUD’?
I thought I had seen everything that is lamentable about homeopathy. When I came across this article, I had to change my opinion. It is a more despicable, unethical and dangerous promotion of falsehoods than I could have imagined.
Strong words? Read for yourself:
There are treatments that can heal vaccine damage, but few physicians in the conventional medical care system know about them, since vaccine injuries are usually denied as the cause of any illness. Some parents with autistic children report that homeopathy has completely reversed their children’s autism and healed other serious health conditions caused by vaccines. This article explains how homeopathic remedies can bring about healing for many types of vaccine injuries.
Homeopathy is not the only treatment that has helped children and adults recover from vaccine damage, but it is the one that is the focus of this article. I will describe how homeopathy can bring about a true cure for the harm that vaccines have caused to children and adults…
It is a tragedy when a normal young child suddenly starts losing the ability to speak sentences or even to speak words after receiving vaccines. The ability to have positive social interactions with other children or adults can disappear in a matter of days after vaccines have been given to children. Intellectual development can be lost and even successful potty training skills can disappear. The ability to sit quietly, listen to a story being read, and the ability to learn can suddenly be replaced with hand flapping, body spinning, head banging, food allergies, asthma, agitation, hyperactivity, learning disabilities, chronic colds and fevers, constant stomach pain, constipation, and a general failure to grow and thrive. There are also serious consequences for adults who use vaccines. Formerly productive adults can lose their independence and become paralyzed, infertile, chronically ill, and even die, because of vaccine damage. It happens every day, yet few people make the connection between their illnesses and vaccine use…
By the time parents fully awaken to the harm that has occurred to their children, many have already resigned themselves to a lifetime of caretaking their disabled children. Some parents will even receive counsel from their physicians to give up their children to the care of the state, because they have no treatments to offer and can offer no hope of recovery. Some physicians will try to convince parents that this is a genetic problem that might be cured someday, but not in the near future. The conventional medical care system leaves parents feeling like helpless victims without any good options. The truth is there are good options for restoring health after vaccine damage, and homeopathy is one of them!…
Homeopathy does not wage war on disease and seek to destroy the symptoms of disease through brute force. It does not bring substances into the body as is done with allopathic drugs, for the purpose of doing hand to hand combat against disease. Instead, homeopathy and its remedies are intended to gently stimulate and strengthen the body so that it can overcome illness through its own vital force and strength. Homeopathic remedies restore the natural ability of the body to defend itself against illness and to heal itself. When this happens, a person is truly cured of what ails him…
Allopathic drugs and treatments do not have a positive effect upon the vital force in the body. They do not improve the strength of a person, and they do not provide for physical, emotional, or mental renewal. Rather, they just suppress symptoms, and add side effects…
You may also wish to ask for a referral from your chiropractor, osteopath, or acupuncturist. Such practitioners are often aware of good homeopaths in the area. Sometimes the person who is responsible for managing supplements and remedies sold at health food stores will be aware of experienced homeopaths as well…
I know, apologists will claim that such extreme idiocy is always the work of a few ‘rotten apples’, even most homeopaths would object to such dangerous and amoral lunacy. But the fact is, they don’t! If you disagree, please show me the protests from homeopaths or other alternative practitioners.
When Wakefield was shown to be a fraud endangering public health with his bogus claims about vaccine damage, there were protests in abundance, and he was ousted by the medical and scientific communities. Where are the protests by the alternative medicine fraternity against this article and the many, many others like it?
NOBODY SHOULD BE ALLOWED TO ENDANGER PUBLIC HEALTH IN THIS WAY.
In case you wonder who wrote the above article, it is John P. Thomas. He is a health writer for Health Impact News. He holds a B.A. in Psychology from the University of Michigan, and a Master of Science in Public Health (M.S.P.H.) from the School of Public Health, Department of Health Administration, at the University of North Carolina at Chapel Hill. John specializes in environmental health, but writes on a variety of issues.
I have repeatedly stressed that herbal remedies can cause harm in a range of ways. Indian rheumatologists recently enforced this point by publishing a case-report of adrenal suppression caused by herbal remedies.
A 49-year-old male presented with polyarthritis from which he had suffered for more than 10 years. His serum cortisol levels were extremely low, he had vitamin D deficiency, and his rheumatoid factor was negative. He revealed symptoms of adrenal suppression, mainly muscle weakness and suicidal tendency, and few other psychiatric disturbances.
The patient eventually discontinued his herbal medicine. Then, he was put on deflazacort for 12 weeks at 12 mg twice daily and later the dose was tapered to 6 mg/day. Deflazocort, an intermediate-acting corticosteroid, was prescribed to minimize the probable withdrawal symptoms due to the probable presence of dexamethasone or betamethasone (long-acting steroids) presumably from the herbal medication.
The herbal samples of used by the patient was analysed by mass spectrometry. It showed the presence of steroidal compounds by the mass 393.81, which may be dexamethasone or betamethasone.
The authors of this paper believe that the symptoms of adrenal suppression could have precipitated or exacerbated the neuropsychiatric disturbances due to Hypothalamus-Pituitary-Adrenal (HPA) suppression. In their view, adrenal suppression following ingestion of herbal remedies is of major concern. Abrupt withdrawal of such products could precipitate adrenal failure which can be fatal.
It should be added, I think, that such illegal adulterations of herbal remedies have been reported with some regularity, particularly in Indian (and Chinese) preparations. Our systematic review showed that this problem has caused serious harm. The most severe documented adverse effects include agranulocytosis, meningitis, multi-organ failure, perinatal stroke, arsenic, lead or mercury poisoning, malignancies or carcinomas, hepatic encephalopathy, hepatorenal syndrome, nephrotoxicity, rhabdomyolysis, metabolic acidosis, renal or liver failure, cerebral edema, coma, intracerebral haemorrhage, and death.
As under-reporting can be suspected to be huge, we do currently not know how frequent these events are.
A new study of homeopathic arnica suggests efficacy. How come?
Subjects scheduled for rhinoplasty surgery with nasal bone osteotomies by a single surgeon were prospectively randomized to receive either oral perioperative arnica or placebo in a double-blinded fashion. A commercially available preparation was used which contained 12 capsules: one 500 mg capsule with arnica 1M is given preoperatively on the morning of surgery and two more later that day after surgery. Thereafter, arnica was administered in the 12C potency three times daily for the next 3 days (“C” indicates a 100-fold serial dilution; and M, a 1000-fold dilution)
Ecchymosis was measured in digital “three-quarter”-view photographs at three postoperative time points. Each bruise was outlined with Adobe Photoshop and the extent was scaled to a standardized reference card. Cyan, magenta, yellow, black, and luminosity were analyzed in the bruised and control areas to calculate change in intensity.
Compared with 13 subjects receiving placebo, 9 taking arnica had 16.2%, 32.9%, and 20.4% less extent on postoperative days 2/3, 7, and 9/10, a statistically significant difference on day 7. Color change initially showed 13.1% increase in intensity with arnica, but 10.9% and 36.3% decreases on days 7 and 9/10, a statistically significant difference on day 9/10. One subject experienced mild itching and rash with the study drug that resolved during the study period.
The authors concluded that Arnica montana seems to accelerate postoperative healing, with quicker resolution of the extent and the intensity of ecchymosis after osteotomies in rhinoplasty surgery, which may dramatically affect patient satisfaction.
Why are the results positive? Pervious systematic reviews confirm that homeopathic arnica is a pure placebo. First, I thought the answer lies in the 1M potency. It could well still contain active molecules. But then I realised that the answer is much more simple: if we apply the conventional level of statistical significance, there are no statistically significant differences to placebo at all! I had not noticed the little sentence by the authors: a P value of 0.1 was set as a meaningful difference with statistical significance. In fact, none of the effects called significant by the authors pass the conventionally used probability level of 5%.
So, what so the results of this new study truly mean? In my view, they show what was known all along: HOMEOPATHIC REMEDIES ARE PLACEBOS.
Lyme disease, also known as Lyme borreliosis, is an infectious disease caused by Borrelia infection transmitted by ticks. The most common early sign is an expanding area of redness beginning at the site of a bite about a week after a tick-bite. Fever, tiredness and headaches often follow. Later stages are characterised by more severe and remarkably variable illness.
Patients with medically unexplained or vague symptoms are sometimes told that they suffer from Lyme disease. These patients are commonly targeted by providers of alternative therapies who promise hope by claiming that their particular brand of quackery is effective for this chronic condition.
A recent review was designed to identify and characterize the range of unorthodox alternative therapies advertised to patients with a diagnosis of Lyme disease.
Internet searches using the Google search engine were performed to identify the websites of clinics and services that marketed non-antimicrobial therapies for Lyme disease. Subsequently the PubMed search engine was employed to identify any scientific studies evaluating such treatments for Lyme disease. Websites were included in this review, if they advertised a commercial, non-antimicrobial product or service that specifically mentioned utility for Lyme disease. Websites with patient testimonials (such as discussion groups) were excluded unless the testimonial appeared as marketing on a commercial site.
More than 30 different alternative treatments were identified. They fell into several broad categories: these included oxygen and reactive oxygen therapy; energy and radiation-based therapies; nutritional therapies; chelation and heavy metal therapies; and biological and pharmacological therapies ranging from certain medications without recognized therapeutic effects on Borrelia burgdorgeri to stem cell transplantation. The review of the medical literature did not substantiate efficacy or, in most cases, any rationale for the advertised treatments.
The authors concluded that providers of alternative therapies commonly target patients who believe they have Lyme disease. The efficacy of these unconventional treatments for Lyme disease is not supported by scientific evidence, and in many cases they are potentially harmful.
Being a bacterial infection, Lyme disease can be successfully treated with oral or intra-venous antibiotics. But, of course, patients need to have the infection in order to benefit from antibiotic treatment. Those patients who don’t are easy targets for charlatans promising help from bogus treatments. It seems that an entire, profitable industry has developed around this situation.
Chinese proprietary herbal medicines (CPHMs) are a well-established and a hugely profitable part of Traditional Chinese Medicine (TCM) with a long history in China and elsewhere; they are used for all sorts of conditions, not least for the treatment of common cold. Many CPHMs have been listed in the ‘China national essential drug list’ (CNEDL), the official reference published by the Chinese Ministry of Health. One would hope that such a document to be based on reliable evidence – but is it?
The aim of a recent review was to provide an assessment on the potential benefits and harms of CPHMs for common cold listed in the CNEDL.
The authors of this assessment were experts from the Chinese ‘Centre for Evidence-Based Medicine’ and one well-known researcher of alternative medicine from the UK. They searched CENTRAL, MEDLINE, EMBASE, SinoMed, CNKI, VIP, China Important Conference Papers Database, China Dissertation Database, and online clinical trial registry websites from their inception to 31 March 2013 for clinical studies of CPHMs listed in the CNEDL for common cold.
Of the 33 CPHMs listed in the 2012 CNEDL for the treatment of common cold, only 7 had any type of clinical trial evidence at all. A total of 6 randomised controlled trials (RCTs) and 7 case series (CSs) could be included in the assessments.
All these studies had been conducted in China and published in Chinese. All of them were burdened with poor study design and low methodological quality, and all had to be graded as being associated with a very high risk of bias.
The authors concluded that the use of CPHMs for common cold is not supported by robust evidence. Further rigorous well designed placebo-controlled, randomized trials are needed to substantiate the clinical claims made for CPHMs.
I should state that it is, in my view, most laudable that the authors draw such a relatively clear, negative conclusion. This does certainly not happen often with papers originating from China, and George Lewith, the UK collaborator in this article, is also not known for his critical attitude towards alternative medicine. But there are other, less encouraging issues here to mention.
In the discussion section of their paper, the authors mention that the CNEDL has been approved by the Chinese Ministry of Public Health and is currently regarded as the accepted reference point for the medicines used in China. They also explain that the CNEDL was officially launched and implemented in August 2009. The CNEDL is now up-dated every 3 years, and its 2012 edition contains 520 medicines, including 203 CPHMs. The CPHMs listed in CNEDL cover 137 herbal remedies for internal medicine, 11 for surgery, 20 for gynaecology, 7 for ophthalmology, 13 for otorhinolaryngology and 15 for orthopaedics and traumatology.
Moreover, the authors inform us that about 3,100 medical and clinical experts had been recruited to evaluate the safety, effectiveness and costs of CPHMs. The selection process of medicines into CNEDL was strictly in accordance with the principle that they ‘must be preventive and curative, safe and effective, affordable, easy to use, think highly of both Chinese and Western medicine’. A detailed procedure for evaluation is, however, not available because the files are confidential.
The authors finally state that their paper demonstrates that the selection of CPHMs into the CNEDL is less likely to be ‘evidence-based’ and revealed the sharp contrast between the policy and priority given to by the Chinese government to Traditional Chinese Medicine(TCM).
This surely must be a benign judgement, if there ever was one! I would say that the facts disclosed in this review show that TCM seems to exist in a strange universe where commercial interests are officially allowed to reign supreme over patients’ interests and public health.
Few alternative remedies are more popular than colloidal silver, i.e. tiny particles of silver suspended in a liquid, and few represent more irresponsible quackery. It is widely promoted as a veritable panacea. Take this website (one of thousands), for instance; it advertises colloidal silver in the most glowing terms:
Here are some of the diseases against which Colloidal Silver has been used successfully Acne, Allergies, Appendicitis, Arthritis, Blood parasites, Bubonic plague, Burns (colloidal silver is one of the few treatments that can keep severe burn patients alive), Cancer, Cholera, Conjunctivitis, Diabetes, Gonorrhoea, Hay Fever, Herpes, Leprosy, Leukaemia, Malaria, Meningitis, Parasitic Infections both viral and fungal, Pneumonia, Rheumatism, Ringworm, Scarlet Fever, Septic conditions of eyes, ears, mouth, throat, Shingles, Skin Cancer, Syphilis, all viruses, warts and stomach ulcer.In addition it also has veterinary uses, such as for canine Parvo virus. You’ll also find Colloidal Silver very handy in the garden since it can be used against bacterial, fungal / viral attacks on plants.It would also appear highly unlikely that any germ warfare agents could survive an encounter with CS, as viruses such as E Bola and Hanta are in the end merely viruses and bacteria.Colloidal Silver is non-toxic, making it safe for both children, adults and pets. Colloidal Silver is in fact a pre 1938 healing modality, making it exempt from FDA jurisdiction.
So why haven’t you heard of it? It’s suspected that the user friendly economics of Colloidal Silver may have something to do with its low profile in the media. Colloidal Silver shines a spotlight on the over expensive and deadly nature of the pharmaceutical industry, who are larger than the Pentagon economically.
That’s right, plenty of bogus claims (it goes without saying that there is no good evidence to support any of them) and, for good measure, some conspiracy theory as well – the perfect mix for making a fast buck!
But sometimes things do not work out as planned. The following text was recently published on the website of Essex County Council:
A man claiming to sell a cure for cancer has been fined £750 following an investigation by Essex Trading Standards. Steven Cook, 54, of East Road, West Mersea, was charged with an offence under the Cancer Act after suggesting Colloidal Silver was a treatment for cancer.
Mr Cook pleaded guilty at Colchester Magistrates’ Court on Friday 12 September. Magistrates imposed a fine of £750 and ordered him to pay £1,500 costs. Cllr Roger Hirst, Essex County Council’s cabinet member for Trading Standards, said: “Trading Standards’ advice to people who are considering whether to take any substance not prescribed for a medical purpose, either preventative or as a treatment, is to consult their doctor first.
“I hope the public feel safer knowing that Essex Trading Standards will take action where traders are trying to sell products which are neither medically proven nor safe.”
Mr Cook runs a website, www.colloidalsilveruk.com, selling various products containing silver. One of the products on sale was “Ultimate Colloidal Silver”, a liquid containing silver that Mr Cook made in his own home. Trading Standards said the website implied that the product can cure cancer – and this is an offence under the Cancer Act. Mr Cook has now updated the website and removed any claims that colloidal silver can cure some cancers.
So, there is some hope! Occasionally, fraudsters are being found out and punished. But the bad news, of course, is that this sort of thing occurs far too rarely and when it does happen, the punishment is far too lenient. Consequently, the public’s protection from fraudsters exploiting the most vulnerable patients is woefully insufficient.
In the past, I have been involved in several court cases where patients had complained about mistreatment by charlatans. Similarly I have acted as an expert witness for the General Medical Council in similar circumstances.
So, it is true, quacks are sometimes being held to account by their victims. But, generally speaking, patients seem to complain very rarely when they fall in the hands of even the most incompetent of quacks.
Here is one telling reminder showing how long it can take until a complaint is finally filed.
Dr Julian Kenyon is, according to his website, an integrated medicine physician and Medical Director of the Dove Clinic for Integrated Medicine, Winchester and London. Dr Julian Kenyon is Founder-Chairman of the British Medical Acupuncture Society in 1980 and Co-Founder of the Centre for the Study of Complementary Medicine in Southampton and London where he worked for many years before starting The Dove Clinic in 2000. He is also Founder/President of the British Society for Integrated Medicine and is an established authority in the field of complementary treatment approaches for a wide range of medical conditions. He has written approximately 20 books and has had many academic papers published in peer review journals* and has several patents to his name. He graduated from the University of Liverpool with a Bachelor of Medicine and Surgery and subsequently with a research degree, Doctor of Medicine. In 1972, he was appointed a Primary Fellow of the Royal College of Surgeons, Edinburgh.
*[I found only 4 on Medline]
Kenyon has been on sceptics’ radar for a very long time. For instance, he is one of the few UK doctors who use ‘LIVE BLOOD ANALYSIS’, a bogus diagnostic method that can harm patients through false-negative or false-positive diagnoses. A 2003 undercover investigation for BBC 1 South’s ‘Inside Out’ accused Dr Julian Kenyon of using yet another spurious diagnostic test at his clinic near Winchester. Kenyon has, for many years, been working together with George Lewith, another of the country’s ‘leading’ complementary doctors. In 1994, the two published an article about their co-operation; here is its abstract:
This paper outlines the main research effort that has taken place within the Centre for the Study of Complementary Medicine over the last 10 years. It demonstrates the Centre’s expertise and interest in a whole variety of areas, including the social implications and development of complementary medicine, clinical trial methodology, the evaluation of complementary medical machinery, the effects of electromagnetic fields on health and the investigation of the subtle energetic processes involved in complementary medicine. Our future plans are outlined.
Lewith and Kenyon have been using a technique called electrodermal testing for more than 20 years. Considering the fact that the two doctors authored a BMJ paper which concluded that electrodermal machines couldn’t detect environmental allergies, this seems more than a little surprising.
Using secret filming, ‘Inside Out’ showed Dr Kenyon testing a six-year-old boy and then deciding that he is sensitive to dust mites. Later, Dr Kenyon insists that he made his diagnosis purely on the boy’s symptoms and that he didn’t use the machine to test for dust mites. The BBC then took the boy for a conventional skin prick test, which suggested he didn’t have any allergies at all. But Dr Kenyon then says the conventional test may not be accurate: “He may be one of the 10% who actually are negative to the skin tests but benefit from measures to reduce dust mite exposure.”
Despite this very public disclosure, Kenyon was able to practice unrestrictedly for many years.
In December 2014, it was reported in the Hampshire Chronicle that Dr Kenyon eventually did, after a complaint from a patient, end up in front of the General Medical Council’s conduct tribunal. The panel heard that, after a 20-minute consultation, which cost £300, Dr Kenyon told one terminally-ill man with late-stage cancer: “I am not claiming we can cure you, but there is a strong possibility that we would be able to increase your median survival time with the relatively low-risk approaches described here.” He also made bold statements about the treatment’s supposed benefits to an undercover reporter who posed as the husband of a woman with breast cancer.
After considering the full details of the case, Ben Fitzgerald, for the General Medical Council, had called for Dr Kenyon to be suspended, but the panel’s chairman Dr Surendra Kumar said Dr Kenyon’s misconduct was not serious enough to warrant a ban. The panel eventually imposed restrictions on Kenyon’s licence lasting for 12 months.
I estimate that patients are exposed to quackery from doctors and alternative practitioners thousands of times every day. Why then, I ask myself, do so few of them complain? Here are some of the possible answers to this important question:
- They do not dare to.
- They feel embarrassed.
- They don’t know how to.
- They cannot be bothered and fear the agro.
- They fail to identify quackery and fall for the nonsense they are being told.
- They even might perceive benefit from treatments which, in fact, are pure quackery.
Whatever the reasons, I think it is regrettable that not far more quacks are held to account – regardless of whether the charlatan in question as studied medicine or not. If you disagree, consider this: not filing a complaint means that many more patients will be put at risk.
Guest post by Nick Ross
If you’re a fan of Edzard Ernst – and who with a rational mind would not be – then you will be a fan of HealthWatch.
Edzard is a distinguished supporter. Do join us. I can’t promise much in return except that you will be part of a small and noble organisation that campaigns for treatments that work – in other words for evidence based medicine. Oh, and you get a regular Newsletter, which is actually rather good.
HealthWatch was inspired 25 years ago by Professor Michael Baum, the breast cancer surgeon who was incandescent that so many women presented to his clinic late, doomed and with suppurating sores, because they had been persuaded to try ‘alternative treatment’ rather than the real thing.
But like Edzard (and indeed like Michael Baum), HealthWatch keeps an open mind. If there are reliable data to show that an apparently weirdo treatment works, hallelujah. If there is evidence that an orthodox one doesn’t then it deserves a raspberry. HealthWatch has worked to expose quacks and swindlers and to get the Advertising Standards Authority to do its job regulating against false claims and flimflam. It has fought the NHS to have women given fair and balanced advice about the perils of mass screening. It has campaigned with Sense About Science, English Pen and Index to protect whistleblowing scientists from vexatious libel laws, and it has joined the AllTrials battle for transparency in drug trials. It has an annual competition for medical and nursing students to encourage critical analysis of clinical research protocols, and it stages the annual HealthWatch Award and Lecture which has featured Edzard (in 2005) and a galaxy of other champions of scepticism and good evidence including Sir Iain Chalmers, Richard Smith, David Colquhoun, Tim Harford, John Diamond, Richard Doll, Peter Wilmshurst, Ray Tallis, Ben Goldacre, Fiona Godlee and, last year, Simon Singh. We are shortly to sponsor a national debate on Lord Saatchi’s controversial Medical innovation Bill.
But we need new blood. Do please check us out. Be careful, because since we first registered our name a host of brazen copycats have emerged, not least Her Majesty’s Government with ‘Healthwatch England’ which is part of the Care Quality Commission. We have had to put ‘uk’ at the end of our web address to retain our identity. So take the link to http://www.healthwatch-uk.org/, or better still take out a (very modestly priced) subscription.
As Edmund Burke might well have said, all it takes for quackery to flourish is that good men and women do nothing.
As promised, I will try with this post to explain my reservations regarding the new meta-analysis suggesting that individualised homeopathic remedies are superior to placebos. Before I start, however, I want to thank all those who have commented on various issues; it is well worth reading the numerous and diverse comments.
To remind us of the actual meta-analysis, it might be useful to re-publish its abstract (the full article is also available online):
A rigorous and focused systematic review and meta-analysis of randomised controlled trials (RCTs) of individualised homeopathic treatment has not previously been undertaken. We tested the hypothesis that the outcome of an individualised homeopathic treatment approach using homeopathic medicines is distinguishable from that of placebos.
The review’s methods, including literature search strategy, data extraction, assessment of risk of bias and statistical analysis, were strictly protocol-based. Judgment in seven assessment domains enabled a trial’s risk of bias to be designated as low, unclear or high. A trial was judged to comprise ‘reliable evidence’ if its risk of bias was low or was unclear in one specified domain. ‘Effect size’ was reported as odds ratio (OR), with arithmetic transformation for continuous data carried out as required; OR > 1 signified an effect favouring homeopathy.
Thirty-two eligible RCTs studied 24 different medical conditions in total. Twelve trials were classed ‘uncertain risk of bias’, three of which displayed relatively minor uncertainty and were designated reliable evidence; 20 trials were classed ‘high risk of bias’. Twenty-two trials had extractable data and were subjected to meta-analysis; OR = 1.53 (95% confidence interval (CI) 1.22 to 1.91). For the three trials with reliable evidence, sensitivity analysis revealed OR = 1.98 (95% CI 1.16 to 3.38).
Medicines prescribed in individualised homeopathy may have small, specific treatment effects. Findings are consistent with sub-group data available in a previous ‘global’ systematic review. The low or unclear overall quality of the evidence prompts caution in interpreting the findings. New high-quality RCT research is necessary to enable more decisive interpretation.
Since my team had published an RCTs of individualised homeopathy, it seems only natural that my interest focussed on why the study (even though identified by Mathie et al) had not been included in the meta-analysis. Our study had provided no evidence that adjunctive homeopathic remedies, as prescribed by experienced homeopathic practitioners, are superior to placebo in improving the quality of life of children with mild to moderate asthma in addition to conventional treatment in primary care.
I was convinced that this trial had been rigorous and thus puzzled why, despite receiving ‘full marks’ from the reviewers, they had not included it in their meta-analysis. I thus wrote to Mathie, the lead author of the meta-analysis, and he explained: For your trial (White et al. 2003), under domain V of assessment, we were unable to extract data for meta-analysis, and so it was attributed high risk of bias, as specified by the Cochrane judgmental criteria. Our designated main outcome was the CAQ, for which we needed to know (or could at least estimate) a mean and SD for both the baseline and the end-point of the study. Since your paper reported only the change from baseline in Table 3 or in the main text, it is not possible to derive the necessary end-point for analysis.
It took a while and several further emails until I understood: our study did report both the primary (Table 2 quality of life) and secondary outcome measure (Table 3 severity of symptoms). The primary outcome measure was reported in full detail such that a meta-analysis would have been possible. The secondary outcome measure was also reported but not in full detail, and the data provided by us would not lend themselves to meta-analyses. By electing not our primary but our secondary outcome measure for their meta-analysis, Mathie et al were able to claim that they were unable to use our study and reject it for their meta-analysis.
Why did they do that?
The answer is simple: in their methods section, they specify that they used outcome measures “based on a pre-specified hierarchical list in order of greatest to least importance, recommended by the WHO“. This, I would argue is deeply flawed: the most important outcome measure of a study is usually the one for which the study was designed, not the one that some guys at the WHO feel might be important (incidentally, the WHO list was never meant to be applied to meta-analyses in that way).
By following rigidly their published protocol, the authors of the meta-analysis managed to exclude our negative trial. Thus they did everything right – or did they?
Well, I think they committed several serious mistakes.
- Firstly, they wrote the protocol, which forced them to exclude our study. Following a protocol is not a virtue in itself; if the protocol is nonsensical it even is the opposite. Had they proceeded as is normal in such cases and used our primary outcome measure in their meta-analyses, it is most likely that their overall results would not have been in favour of homeopathy.
- Secondly, they awarded our study a malus point for the criterium ‘selective outcome reporting’. This is clearly a wrong decision: we did report the severity-outcome, albeit not in sufficient detail for their meta-analysis. Had they not committed this misjudgment, our RCT would have been the only one with an ‘A’ rating. This would have very clearly highlighted the nonsense of excluding the best-rated trial from meta-analysis.
There are several other oddities as well. For instance, Mathie et al judge our study to be NOT free of vested interest. I asked Mathie why they had done this and was told it is because we accepted free trial medication from a homeopathic pharmacy. I would argue that my team was far less plagued by vested interest than the authors of their three best (and of course positive) trials who, as I happen to know, are consultants for homeopathic manufacturers.
And all of this is just in relation to our own study. Norbert Aust has uncovered similar irregularities with other trials and I take the liberty of quoting his comments posted previously again here:
I have reason to believe that this review and metaanalysis in biased in favor of homeopathy. To check this, I compared two studies (1) Jacobs 1994 about the treatment of childhood diarrhea in Nicaragua, (2) Walach 1997 about homeopathic threatment of headaches. The Jacobs study is one of the three that provided ‘reliable evidence’, Walach’s study earned a poor C2.2 rating and was not included in the meta-analyses. Jacobs’ results were in favour of homeopathy, Walach’s not.
For the domains where the rating of Walach’s study was less than that of the Jacobs study, please find citations from the original studies or my short summaries for the point in question.
Domain I: Sequence generation:
“The remedy selected was then mailed to a notary public who held a stock of placebos. The notary threw a dice and mailed either the homeopathic remedy or an appropriate placebo. The notary was provided with a blank randomisation list.”
Rating: UNCLEAR (Medium risk of bias)
“For each of these medications, there was a box of tubes in sequentially numbered order which had been previously randomized into treatment or control medication using a random numbers table in blocks of four”
Rating: YES (Low risk of bias)
Domain IIIb: Blinding of outcome assessor
“The notary was provided with a blank randomization list which was an absolutely unique document. It was only handed out after the biometrician (WG) had deposited all coded original data as a printout at the notary’s office. (…) Data entry was performed blindly by personnel not involved in the study. ”
Rating: UNCLEAR (Medium risk of bias)
“All statistical analyses were done before breaking the randomisation code, using the program …”
Rating: YES (Low risk of bias)
Domain V: Selective outcome reporting
Study protocol was published in 1991 prior to enrollment of participants, all primary outcome variables were reported with respect to all participants and the endpoints.
Rating: NO (high risk of bias)
No prior publication of protocol, but a pilot study exists. However this was published in 1993 only after the trial was performed in 1991. Primary outcome defined (duration of diarrhea), reported but table and graph do not match, secondary outcome (number of unformed stools on day 3) seems defined post hoc, for this is the only one point in time, this outcome yielded a significant result.
Rating: YES (low risk of bias)
Domain VI: Other sources of bias:
Rating: NO (high risk of bias), no details given
Imbalance of group properties (size, weight and age of children), that might have some impact on course of disease, high impact of parallel therapy (rehydration) by far exceeding effect size of homeopathic treatment
Rating: YES (low risk of bias), no details given
In a nutshell: I fail to see the basis for the different ratings in the studies themselves. I assume bias of the authors of the review.
So, what about the question posed in the title of this article? The meta-analysis is clearly not a ‘proof of concept’. But is it proof for misconduct? I asked Mathie and he answered as follows: No, your statement does not reflect the situation at all. As for each and every paper, we selected the main outcome measure for your trial using the objective WHO classification approach (in which quality of life is clearly of lower rank than severity). This is all clearly described in our prospective protocol. Under no circumstances did we approach this matter retrospectively, in the way you are implying.
Some nasty sceptics might have assumed that the handful of rigorous studies with negative results were well-known to most researchers of homeopathy. In this situation, it would have been hugely tempting to write the protocol such that these studies must be excluded. I am thrilled to be told that the authors of the current new meta-analysis (who declared all sorts of vested interests at the end of the article) resisted this temptation.