‘What Doctors Don’t Tell You’ (WDDTY) have been shown to be strangely economical with the truth many times before (for instance here, here and here). Now they have published an article entitled ‘Ombudsman investigates ‘flawed’ homeopathic study that claimed it doesn’t work’ It attacks in no uncertain terms the ‘NHMRC Statement on Homeopathy and NHMRC Information Paper – Evidence on the effectiveness of homeopathy for treating health conditions’ which I believe to be a sound evaluation of homeopathy and therefore have mentioned repeatedly on this blog. Here is what WDDTY stated:
START OF QUOTE
A major and influential review of homeopathy concluded that the controversial therapy doesn’t work—but it was so riddled with error and bad science that it’s sparked an official ombudsman investigation.
The world’s media announced that homeopathy was a scam after the Australia government’s National Health and Medical Research Council (NHMRC) published its findings in 2015 that “there are no health conditions for which there is reliable evidence that homeopathy is effective.”
But now the Commonwealth Ombudsman is investigating the review’s procedures after receiving reports of inaccuracies, mishandling of evidence and conflicts of interest.
The review has been triggered by the Australian Homeopathic Association (AHA), supported by the Homeopathic Research Institute (HRI), which began questioning the review’s processes after several solid studies that demonstrated homeopathy’s benefits had been overlooked.
The NHMRC review team set arbitrary parameters that only studies that involved more than 150 people—and which met standards that even drug trials rarely achieve—would be considered. Those requirements reduced the number of qualifying studies to just five—from an initial pool of more than 1,800 trials—and none of these showed that homeopathy was effective.
One of the NHMRC’s own reviewers produced a mysterious first report that has never been published, and hasn’t been released despite Freedom of Information requests.
And the AHA has discovered that Prof Peter Brooks, chair of the NHMRC committee that carried out the homeopathy review, never declared that he was a member of the anti-homeopathy lobby group, Friends of Science in Medicine.
There are solid studies that demonstrate homeopathy is effective against childhood diarrhea, sinusitis and hay fever—but they all involve fewer than 150 people, said HRI chief executive Rachel Roberts. “The public has a right to know that there are high quality studies showing homeopathy works for some medical conditions—information that was lost only due to NHMRC’s mishandling of the evidence.”
The homeopaths aren’t alone in challenging the NHMRC review: Australia’s independent Cochrane Centre said its conclusions are not an accurate reflection of the evidence, and a second expert also said he felt “uncertain of the definitive nature of the report’s conclusions.”
END OF QUOTE
As it happens, I am in contact with the lead author of this report, Paul Glasziou, not least because he very kindly wrote the foreword for my book HOMEOPATHY, THE UNDILUTED FACTS. So, we corresponded and discussed the latest WDDTY diatribe. Thus I am now in a position to put a few things straight (I hope Paul does not mind).
ISSUE 1. – The NHMRC review team set arbitrary parameters that only studies involving more than 150 people—and which met standards that even drug trials rarely achieve—would be considered.
The truth is that report focused on systematic reviews of trials, not individual trials. The 57 included systematic reviews found 176 individual trials which covered 61 conditions: an average of about 3 trials per condition. But some conditions only had 1 trial, and one small trial would, of course, not be considered a reasonable basis for reliable conclusions. GRADE – the international standard for assessing evidence – downgrades reviews for “imprecision” – the GRADE Handbook suggests “whenever there are sample sizes that are less than 400, review authors and guideline developers should certainly consider rating down for imprecision.” Hence the criterion of 150 which the Australians decided to use is considerably more lenient than the current GRADE guideline.
ISSUE 2 – Those requirements reduced the number of qualifying studies to just five—from an initial pool of more than 1,800 trials—and none of these showed that homeopathy was effective.
This is simply not correct. The report found 57 systematic reviews that contained 176 individual trials, not 5. These 176 trials, which covered 61 conditions, formed the body of evidence for the NHMRC report’s conclusions.
ISSUE 3 – There are solid studies that demonstrate homeopathy is effective against childhood diarrhoea, sinusitis and hay fever—but they all involve fewer than 150 people, said HRI chief executive Rachel Roberts.
The NHMRC report focused on systematic reviews that covered all trials for individual conditions. Given the conventional p-value of 0.05, one would expect 1 in 20 single trials to be “false positives”. So with 176 trials, we expect about 9 “false positive” trials. But using systematic reviews that combine all trials for individual conditions, reduces this risk of false positives. Most national evidence review bodies require more than 1 trial, e.g, the FDA requires 2 positive trials, whereas many others require a systematic review which has at least 2 trials. Replication of findings is obviously a cornerstone of science.
ISSUE 4 The homeopaths aren’t alone in challenging the NHMRC review: Australia’s independent Cochrane Centre said its conclusions are not an accurate reflection of the evidence, and a second expert also said he felt “uncertain of the definitive nature of the report’s conclusions.”
The truth is that the Cochrane Centre, which provided an independent check during the processes of the NHMRC review, concluded that “Overall, the conclusions arising from the review appear justified based on the evidence presented.”
I REST MY CASE.
19 Responses to Ombudsman investigates ‘flawed’ homeopathic study
Leave a Reply to Logos-Bios Cancel reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
The ombudsman will, of course, conclude that the systemic review was conducted correctly and that its findings are entirely appropriate. The homeopaths will continue to bleat because, of course, the laws of statistics don’t apply to them in the same way that the laws of mathematics, physics, chemistry, biology and, indeed, logic don’t either. And Lynne and Bryan at WDDTY will mysteriously forget to report on the conclusions. So it goes.
you should set up shop as a clairvoyant
Where did they get the Cochrane quote from, and who is the `second expert’? I think we should be told.
The Australian report
“In March 2015, the Australian National Health and Medical Research Council (NHMRC) published an Information Paper on homeopathy, commonly referred to as ‘The Australian Report’.1
This document concludes that “…there are no health conditions for which there is reliable evidence that homeopathy is effective”.
This report triggered headlines around the world suggesting NHMRC had found that homeopathy doesn’t work for any condition.3
Australian Report key facts
An extensive detailed investigation by the Australian Homeopathic Association (AHA) into NHMRC’s conduct, combined with an in-depth scientific analysis of the report by HRI, revealed evidence of serious procedural and scientific misconduct in producing this report:
NHMRC did the review twice. They rejected the first report, despite it being undertaken by a reputable scientist who is an author of NHMRC’s own guidelines on how to conduct reviews.
The existence of the first report has never been disclosed to the public – it was only discovered by AHA through Freedom of Information (FOI) requests.
NHMRC said the results of their 2015 report were based on a “rigorous assessment of over 1800 studies”. In fact results were based on only 176 studies.
NHMRC used a method that has never been used in any other review, before or since. NHMRC decided that for trials to be ‘reliable’ they had to have at least 150 participants and reach an unusually high threshold for quality. This is despite the fact that NHMRC itself routinely conducts studies with less than 150 participants.
These unprecedented and arbitrary rules meant the results of 171 of the trials were completely disregarded as being ‘unreliable’ leaving only 5 trials NHMRC considered to be ‘reliable’. This explains how they could conclude that there was no ‘reliable’ evidence.
Professor Peter Brooks, Chair of the NHMRC committee that conducted the 2015 review, initially failed to declare that he was a member of the anti-homeopathy lobby group ‘Friends of Science in Medicine’.
In violation of NHMRC’s own guidelines there was not one homeopathy expert on the committee.
HRI is not alone in criticising the accuracy of NHMRC’s findings. FOI requests have brought to light that two independent experts also raised concerns over the report’s conclusions during peer review, prior to final publication. The Australasian Cochrane Centre commented that for some conditions, “…. ‘no reliable evidence’ does not seem an accurate reflection of the body of evidence”; a second expert felt “uncertain of the definitive nature of the Report’s conclusions”. NHMRC chose not to act on this feedback and did not amend their conclusions.
Complaint submitted to Commonwealth Ombudsman
In August 2016 HRI’s in-depth scientific analysis was used as part of a submission of complaint to the Commonwealth Ombudsman brought by Complementary Medicines Australia, Australian Homoeopathic Association and Australian Traditional Medicine Society.”
The above quote was what I could find with a quick Nexis search.
So what is new here? Source?
In Manitoba, Canada, the provincial government,s Minister of health has asked the province,s Health Professions Advisory Council to investigate the risks and benefits of high neck manipulation. The government has recognizes a number of acts, called reserved acts that pose a significant risk to the public and that might require conditions, restrictions and limitations. High velocity, low amplitude thrust of the spine within its anatomical range of motion is one of the reserved acts and the Manitoba chiropractors have applied for authority to perform that reserved act. The Council requested submissions on this topic from other regulatory bodies in the province including physicians and surgeons and physiotherapists. The conclusions of this investigation will be reflected in the scope of practice for our provincial chiropractors. The conclusions will be reflected as law and our little group in Manitoba….The Manitoba Chiropractic Stroke Survivors …..has obviously made a submission. It has been four months with no further correspondence from the Council or the government.
Our Group has recommended a complete ban on high velocity low amplitude high neck manipulation…..or at the very least urged the government to enforce conditions, restrictions and limitations,
We need help, expert help in attaining our goal. I suspect this will be a one and only chance to legislate high velocity low amplitude high neck manipulation.
Pchevrier
Off-topic post, but I’ll engage. It’s unfortunate when any person suffers significant ramifications from a healthcare procedure or an underlying developmental defect or infirmity. It’s quite an academic stretch, but a very human one, to wish for the banning of the alleged procedure which had purportedly instigated the medical condition. The extremely low risk of chiropractic cervical SMT has been well established in the literature and has been discussed on this blog often. It is not reasonable to ascribe more risk to a procedure than that which has been demonstrated in numerous studies. Otherwise, many procedures known by many to be helpful might be forbidden based on anecdotal grounds.
An example of such a reflexive call for banning a procedure might be the instantiation of one from the realm of bariatric surgery:
A nursing assistant died while undergoing stomach reduction surgery, and the hospital said Thursday it is suspending such operations while it investigates. Brett Davey, spokesman for Roger Williams Medical Center, would not say went wrong during the surgery or whether any doctors would be disciplined.
Robert Mesa Jr., 27, who worked at the hospital, died Tuesday about a half-hour into the laparoscopic gastric bypass operation, Davey said.
It was the third death among the 340 gastric bypass procedures performed at the hospital over the past three years, chairman of surgery Paul Liu said.
Liu said the hospital will stop performing both open and laparoscopic procedures. The newer laparoscopic technique uses smaller incisions and leads to less scarring and quicker recovery.
He said the doctor who performed the surgery has not been suspended. He would not identify the doctor, but said he’s “very experienced.”
Last month, Boston’s Brigham and Women’s Hospital stopped performing the laparoscopic procedure after a 38-year-old woman there died after the staple gun apparently misfired during the procedure.
“I think what’s happening is the fact that it’s become so visible with celebrities and others who have had it done … the numbers who are having it done are going up dramatically, and I think the true risk of this procedure is becoming more known,” said Dr. Vincent Pera, director of the weight management program at Miriam Hospital in Providence.” Indeed…perhaps Geir should take note!
Pera said the death rate for gastric bypass surgery is one in 200(and Geir complains about the 1 in 5+ million deaths with SMT).
Gastric bypass surgery aids weight loss in a morbidly obese patient by shrinking the stomach from the size of a football to the size of an egg; unfortunately, some surgeons market the procedure to improve the aesthetics of the patient who is considering or being “soft-sold” the surgery.
The smaller stomach pouch is created by stitching or stapling part of the stomach closed. A part of the small intestine is bypassed to reduce nutrient absorption, causing the patient to lose weight. Typically, patients lose 70 percent to 80 percent of their excess body weight in about a year.
But the surgery comes with risks, including postoperative complications that can include blood clots floating to the lungs or stomach juices leaking into the blood and causing infections.
Though the procedure is commonly known as stomach stapling, the hospital would not confirm whether staples were used during Messa’s procedure.
Davey said Messa underwent about three months of preparation for the surgery, including consulting with a cardiologist, psychiatrist and dietitian and attending a three-hour seminar on the risks and benefits of the procedure.
Despite these precautions, Messa still died. BWH discontinued the performing of this procedure due to safety concerns.
Obviously such surgeries do carry significant risks, but there are many people who claim they have benefitted them. Should such surgeries be banned because of cases as I have above described?
“The extremely low risk of chiropractic cervical SMT has been well established in the literature and has been discussed on this blog often.”
oh really?
A gastric bypass would cost in around $20000. Not bad huh?
Bariatric surgery has known risks that are clearly explained to the patients who in our state require strict follow-up guidelines on healthy weight loss diets and exercise programs and are proven to have failed this before they are allowed to undergo the surgery. There are many clear benefits for those who elect to have this procedure and the risks are minimized with an experienced surgeon and team. On the other hand there are serious risks with cervical manipulation with close to no real benefits beyond psychological manipulation, like with voodo, witch craft, palm reading, homeopathy, faith healing and the like. It should be against the law, especially in kids.
Why is this idiot still copy-pasting irrelevant, off-topic old news? I try to skip anything that comes from Logos-Bios but Dr. Cox’s comment led me to read this rubbish .
The news item pasted there is really sad but I guess you can find all sorts of criminal incompetence in the US? Not only in surgery as the world is getting increasingly aware of (blink-blink).
The story describes a clinic that seems to be run by bungling amateurs trying to preform advanced surgery. Three deaths and only 340 cases in three years is a sure sign that they should not be doing bariatric surgery. Low-volume centres are the dangerous ones, they simply do not develop and maintain the competence necessary. Saying the death rate of bariatric surgery as 1 per 200 is a blatant lie by at least a factor of ten.
Having a patient die 30 minutes into the procedure and blaming it on a stapler malfunction is a sure indication of incompetence. They probably couldn’t handle an intraoperative hemorrhage, the spleen is a real “killer” if you don’t know how to avoid it. Or the anesthesia bungled their part?
We are usually finished with a gastric bypass in 30 minutes and we count our cases in thousands, not hundreds.
I have previously provided information about the publicly available, confirmed morbidity and mortality rates in our part of the world and am not going to repeat that.
Those who are interested can find trustworthy information for the US situation on the ASMBS site
Can we please have a button on this blog that hides “Logos-Bios” comments? There are some of us who are tired of its pompous, irrelevant prattle? I may of course have missed (=skipped) something but I cannot recall seeing anything for a long time that contributed to the actual subject being discussed. The overlong comment in question here, certainly does not contribute anything to a discussion about homeopathy or its research.
It’s laughable that Geir would criticize my response to Pchevrier’s off-topic comments….and then proceed to discuss his “previously provided information” regarding bariatric surgery’s alleged mortality rates. At least he again has admitted that “modern medicine” allows “bungling amateurs” to perform complex surgeries. I pointed out some time ago that some of Geir’s citations which purport an infinitesimal 1-year post-surgical mortality rate included proceduresperformed by only the most skilled surgeons, not “average” or “bungling amateur” surgeons. Of course, Geir must have “missed” (=skipped) those posts. Not surprising from a fellow who loves to criticize professions but is shockingly appalled when criticisms are directed at his own…..CONVENIENT!
Serious risks of cervical SMT? Compare the alleged risks to treating patients per the quack-scam GSK/Paxil outrage which was foisted upon innocent patients by “modern medicine.” “No real benefits,” you stated. You perhaps were erroneously thinking of the IMATCH program within “modern medicine.” You’ll recall it…..patients do report satisfaction with its results and it does generate $30K and more in revenues for medical practitioners; it appears to be a “win, win” program. Unfortunately it has minimal research justification or evidence for its use. By your “logic” such would necessarily be classified as comparable to voodoo, witch craft, palm reading, faith healing, and the like, and be against the law. However, I’m sure you would not endorse such comparisons because IMATCH is staffed primarily with medical professionals. Isn’t this so, Cox?
you seem to be a bit slow on the uptake: how often has it been pointed out here that it is the risk/benefit balance that matters?
What was the risk-benefit of prescribing Paxil to children, Edzard?
Take Paxil, when prescribed for children, for example. It would seem that “modern medicine” itself is guilty of misconduct…..and its ramifications often are dire.
A major reanalysis just published in The BMJ of tens of thousands of pages of original trial documents from GlaxoSmithKline’s infamous Study 329, has concluded that the antidepressant paroxetine is neither safe nor effective in adolescents with depression. This conclusion, drawn by independent researchers, is in direct contrast to that of the trial’s original journal publication in 2001, which had proclaimed paroxetine “generally well tolerated and effective.” The new paper, published under the restoring invisible and abandoned trials (RIAT) initiative, has reignited calls for retraction of the original study, putting additional pressure on academic and professional institutions to publicly address the many allegations of wrongdoing.
Troubled from the start
Few studies have sustained as much criticism as Study 329, a placebo controlled, randomized trial of paroxetine and imipramine carried out by SmithKline Beecham (which became GlaxoSmithKline (GSK) in 2000). In 2002, a US Food and Drug Administration officer who formally reviewed the trial reported that “on balance, this trial should be considered as a failed trial, in that neither active treatment group showed superiority over placebo by a statistically significant margin. Yet this same year, according to the New York State Attorney General’s office, which sued GSK, over two million prescriptions were written for children and adolescents in the United States, all off-label, after a marketing campaign that characterized Study 329 as demonstrating “REMARKABLE Efficacy and Safety.”
The disparity between what the manufacturer and study authors claim the trial found and what other parties say the data show was an important element in the US Department of Justice’s criminal charges against GSK. Imagine that….a drug company(pillar of “modern medicine”) failed to disclose pertinent research findings and instead allowed and encouraged physicians to prescribe the “no benefit”(for children) drug to millions of their patients. Paxil has been associated with hundreds, if not thousands, of serious side effects, even death. How many years passed before the recommendations for Paxil’s use were modified?
Please respond cogently, if possible.
I’ve got no dog in this race. Perhaps nothing is new. The info I posted was from a simple Nexis search:
HRI – The Australian report – Homeopathy Research Institute