scientific misconduct
Guest post by Christian Lehmann
It’s the end of February. We see the first death, in the Oise department, near Paris, of a French citizen who has not recently travelled abroad. For doctors concerned about what is happening in China, this is the red alert. In spite of of the little notices posted by the health minister, Agnes Buzyn, at airports, the coronavirus has made it onto French soil. Nobody knows at that point how it will spread. Almost nobody, apart from those responsible for it, yet knows that France has completely run down its stocks of masks. Doctors themselves do know that the health service has only held out, for as long as it has, on the backs of its care personnel. Some are assessing the scale of what is to come.
The announcement by Didier Raoult about the spectacular effectiveness of a synthetic antimalarial, chloroquine, has brought enormous relief, followed immediately for many of us health professionals by growing doubts about an accumulation of errors: Raoult denies any toxicity, urges people to “fall upon” a medication requiring sensitive handling. When we locate the Chinese article on which Didier Raoult is basing his crisis communication, we are stupefied. No need for specialised knowledge in statistical methodology to understand that there is something seriously wrong. No numerical data. Nobody knows what dosage has been given, to what type of patient, nor how many have been treated. The article has not been “peer reviewed”, that is to say reviewed by professional equals; decoded, it has the effect of a simple announcement. So of course at this chaotic time we tell ourselves that, given a revelation of such importance, the Chinese wanted to act as quickly as possible, to inform the whole world. And Didier Raoult, who routinely advises, as he explains with delicious modesty, the Chinese, « the world’s best virologists », has probably been entitled to the first fruits of this revelation.
On Youtube, on 28 February, he posts a weird interview, “Why would the Chinese be mistaken?”, in which he repeatedly takes up his interviewer with obvious irritation. “No, that’s not the question that you should be asking me. You should be asking me….” An informal group of doctors and tweeters pass around the link. We are rubbing our eyes in disbelief. What Didier Raoult is passing off as an interview is nothing more then an audience accorded to one of his media aides. We advise him, sarcastically, to make a professional cut of the video before broadcasting it. An hour later the video disappears and returns in a more professional form which could create the illusion of a genuine interview. And rapidly, in the Press which is beginning to turn its microphones towards the Professor from Marseille, he modifies his stance, without ever acknowledging the radical changes.
Chloroquine, spectacular and miraculous only yesterday, disappears as if by magic, replaced from one day to the next by hydroxychloroquine (Plaquenil), a different medicine, less common. Though its chemical structure is close to that of the antimalarial medication, hydroxychloroquine is used primarily in rheumatic conditions such as rheumatoid polyarthritis, or immune conditions such as lupus. So at least it isn’t lying around in large quantities in medicine cabinets. And its cardiac toxicity, very real, is slightly lower then that of chloroquine. Didier Raoult puts forward HCQ as an immense discovery, continuing in his usual manner to ridicule his detractors. “The doctors who criticise me are neither in my field nor up to my weight”. He flays the inaction of embittered petty health officials, only fit to follow the diktats of the authorities, who, bogged down in their catastrophic crisis management, dare not intervene. And his posturing as a refractory Gaul, a loudmouth taking on the system, gains sympathy, from those to whom he gives hope, from those who understand that the State does not tell them everything, and from those looking for a hero to fit in with their stereotypes: the man on his own against the establishment, the White Knight taking on Big Pharma, the Hippocratic colossus besieged by hordes of soulless ants.
No one among those who hold out their microphones to him, not one asks him the question which we are all asking, GPs, cardiologists, pharmaceutical specialists, emergency specialists, resuscitation specialists – by what sleight of hand has Didier Raoult exchanged his miracle medicine, in 48 hours, openly and publicly? And how is it that no one has noticed the sleight-of-hand? Has this man who makes such a big deal of his image on social networks suddenly become aware of the risk of being confronted about chloroquine with a justifiable public outcry and with deaths by self-medication?
While the World Health Organisation is sounding alarm bells, in the context of overall mistrust with regard to scientific opinion, of confrontation with regard to government, of growing awareness ( belated and sometimes disproportionate) of the influence of Big Pharma, and as the initial fear gives way to real panic for some with the registration of each new case, Didier Raoult piles up Facebook likes, fans, sites to his glory. And for us, fearful, begins the long registration of flagrant mistruths delivered as revealed truths, which this professor will never have the honestly to set right.
For Didier Raoult, a minimum of intellectual integrity would demand that he admits having changed horses in midstream. That he admits that the concern of his despised detractors was well founded, with respect to chloroquine to which many have access without knowing its dangers ( Nivaquine is very often used in suicides). And, because Didier Raoult withdraws nothing, he continues to stash away all the profits of his media coverage. Every supporter of the Wise Man of Marseille piles in with testimony. Their brother, sister, uncle, the father-in-law of their hairdresser has been taking the Professor’s medicine ( Which one? ) for eight years in Africa and has never had a problem, so that’s the real proof that his detractors are just jealous, or, even worse, backed by “the lobbies”.
And untiringly we repeat the fundamental truths:
- Yes chloroquine has existed for years
- Yes it is widely used
- But for a different treatment, the prevention of malaria
- And in dosages 5 to 10 times smaller
- And in large dosages it causes cardiac arrest
- And it has never been effective in fighting a virus
- Not this virus nor any other
- And the same is true for hydroxychloroquine
- In fact it’s rather the opposite
In fact what is being patiently stated by the upholders of the scientific method is very counter-intuitive, almost inaudible, because they are telling worried and disorientated people, who have put their trust and their hope in one man, that in his assertions………nothing makes sense.
Guest post by Kevin Smith
A family member of my household has been aghast to receive in the post yesterday a letter suggesting that, if they develop symptoms of coronavirus, they should take homeopathic remedies.
If this had been from some quack pharmacy doing a random mailshot, it would have been bad enough. But, astonishingly, it has come from the NHS! The letter is not on headed notepaper and is unsigned (it is in the format of a ‘factsheet’), thus is doesn’t contain the sender’s address; however, the envelope’s address label displays both my family member’s NHS number and the name of their GP practice. Moreover, the franking refers to a PO Box number that is owned by the NHS teaching hospital in our area. So it has certainly come from the NHS.
I believe that the family member who received it has been targeted because, in the past, a GP referral had been made for them to consult an NHS homeopath at this hospital.
Yes, very sadly, homeopaths have managed to exist within the NHS in the local area. I had assumed that, with the NHS recently cracking down on homeopathy, such quacks would have been excised – but this looks not to be the case, given the sending of this letter.
Here’s the text of the letter. Read it and see if you are as astonished – indeed as enraged – as I certainly am, and as is the family member to whom it was sent.
Guidance on Coronavirus (updated)
Prevention:
Daily probiotics, Regular handwashing, Stat dose of Covid-19 nosode 200c if it becomes available, Vit C & Zinc supplementation
Stress avoidance (Constitutional homeopathic prescribing & lifestyle)
Avoid incidental paracetamol use (ie no symptomatics for stress headaches etc)
Contact:
Add Ecchinacea, tincture 5 drops in water, twice daily, for no more than 4 consecutive days
Prodromal (ie before symptoms emerge):
Avoidance of incidental paracetamol use.
Stop work. Rest. Isolation. (+ Gelsemium 2 hourly, and/or Covid-19 nosode if it becomes available)
If you develop symptoms of Coronavirus: then avoid Paracetamol, Ibuprofen or Aspirin and take one of the following every hour, sucked in the mouth:
Camphora 30c (tablets or pillules) chills, cough, changeable fever
Bryonia 30c (tablets or pillules) fever, painful dry cough
Arsenicum album 30c (tablets or pillules) washed out feeling, chilliness, restless or agitated
Veratrum album 30c (diarrhoea, chills and fatigue)
Bryonia and Camphora are the most commonly indicated for Covid-19 from experience so far. Order them directly from one of the UK Homeopathic Pharmacies listed.
7 grammes / 8 grammes = 60 tablets or pillules
14 grammes / 15 grammes = 120 tablets or pillules
That’s the front page of the letter. Overleaf, it lists 11 homeopathic suppliers (across the UK), complete with contact details.
Additionally, the letter was accompanied by a pink slip, containing the following text:
If you find that you need to use any of the treatments outlined here, it is very important that you provide detailed feedback to us, so that we can adapt and improve our advice to others if necessary. Email (feedback only) [email protected]
What to make of this communication? Remember, this was from the NHS! What to do about it? COMMENTS WELCOME!
The ‘Corona-Virus Quackery Club’ (CVQC) is enjoying a fast-growing membership. As mentioned in previous posts, it consists of:
Chiropractors have been keen to join since weeks. They have a long tradition of claiming that their ‘adjustments’ boost the immune system, and therefore it was to be expected that they also jump on the corona-bandwagon.
Some chiropractors seem to believe that the corona-virus pandemic is a fine business opportunity or, as one put it, the perfect opportunity to have a heart to heart with patients about their immune and nervous systems! Remember, if germs automatically caused disease, the human race wouldn’t be around to debate the issue. Many forget that Louis Pasteur, the father of the germ theory recanted his belief. On his deathbed he observed, “It’s the soil, not the seed.” In other words, without the right environment, germs can do little harm.
Chiropractors and other health care workers are at greater risk due to patient or client interactions and are encouraged to take extra precautions when it comes to cleaning and disinfecting surfaces and skin or close contact.
“Every chiropractic practice has been touched by coronavirus [fears],” says Bill Esteb, DC, who has created and is circulating a coronavirus and chiropractic guide on how to avoid contracting the virus.
“We wanted to create a tool that chiropractors could use as a conversation springboard. Chiropractors need to remind their patients that germs don’t automatically cause disease. And that ‘catching’ the coronavirus, or anything else, requires a hospitable environment.”
The only way to catch anything, says Esteb, is to become a hospitable host. Flipping the message, Esteb in his coronavirus and chiropractic guide says here is “How to Catch the Coronavirus”:
- Eat a Poor Diet — Make sure your body lacks the vitamins, minerals, enzymes and micronutrients needed to keep itself in good repair.
- Avoid Adequate Rest — Stay up late and use sugar, tobacco, coffee and energy drinks as needed.
- Become Dehydrated — Reduce the effectiveness of your natural defense mechanisms by shunning adequate water.
- Stop Exercising — Reduce the efficiency of your lymphatic system, which requires movement to circulate this important germ-fighting fluid.
- Think Negative Thoughts — Worry that you’ll be a victim. Closely monitor news reports about outbreaks, fearing the advancing pandemic.
- Rarely Wash Your Hands — Use your dirty hands and fingers to rub your eyes, pick your nose or wipe your lips.
- Skip Your Chiropractic Adjustments — Handicap your nervous system, the master system that controls your entire body. Wait until symptoms are clearly present.
“Following these suggestions is the way to become a suitable host for any number of germs or microbes,” Esteb says. “The tongue-in-check approach keeps the subject light. It stimulates more instructive patient conversations. It helps reduce appointment cancellations.
“Most people have an inappropriate fear of germs. And while this poster and patient handout won’t eliminate it, use it to explore the value of ongoing chiropractic care as a preventive strategy.”
——–
The Internet is full with messages of this type. Here is just one example: The best defense for the Corona Virus is to be healthy when you are exposed to the virus. Get adjusted to boost your immune system. Check out this video blog on what you can do to be healthy and prepare your body to fight off the corona virus.
——–
Perhaps the worst excesses can be found on Twitter:
It seems that some people are pushing the notion that Boiron’s homeopathic product
Oscillococcinum®
might be helpful for the prevention and/or treatment of the Corona virus infection. To get an idea how implausible this assumption is, read my previous post on the subject.
The website of Boiron, the producer of the product, seems undeterred by plausibility and states the following:
Clinical studies show Boiron Oscillococcinum (Oscillo®) reduces the duration and severity of flu-like symptoms when taken at the onset of symptoms.1-2 Oscillo does not cause drowsiness and has no known or reported drug interactions.
Uses*
- Temporarily relieves flu-like symptoms such as body aches, headache, fever, chills and fatigue
- Non-drowsy; no drug interactions
- Easy-to-take, quick-dissolving pellets
- For everyone 2 years of age and older
- Make sure your patients always have Oscillococcinum on hand—it works best when taken at the first sign of symptoms. Help your patients feel better before they feel worse.
While this text does not state that Oscillococcinum works for the coronavirus, one could easily read it as implying it, particularly if one also considers this tweet:
Oscillococcinum USA
@OscilloUSA
Getting sick when travelling can ruin the best of vacations. Take non-drowsy Oscillococcinum the moment you feel body aches, headache, fever, chills or fatigue coming on. http://bit.ly/2BGCmCz
________________________________________________________________________________
On the Internet we find many much more direct claims. Take this website, for instance:
The commonly indicated Homeopathic remedies for Coronavirus are: –
• OSCILLOCOCCINUM
• Arsenic Album
• INFLUENZINUM
**However, for best results contact a Qualified Homeopathic doctor so that correct medicines can be prescribed.
And even some politicians promote such irresponsible nonsense.
________________________________________________________________________________
All the claims about Oscillococcinum have one thing in common: they are not evidence based! Any notion that it might work against the coronavirus is pure fantasy. And the above statement by Boiron is based on two cherry-picked studies. The totality of the evidence, however, does not show that Oscillococcinum is effective. The current Cochrane review says about its effectiveness: There is insufficient good evidence to enable robust conclusions to be made about Oscillococcinum(®) in the prevention or treatment of influenza and influenza-like illness. Our findings do not rule out the possibility that Oscillococcinum(®) could have a clinically useful treatment effect but, given the low quality of the eligible studies, the evidence is not compelling. There was no evidence of clinically important harms due to Oscillococcinum(®).
The reason, I guess, why this conclusion is not more forthright stating THERE IS NO GOOD EVIDENCE THAT OSCILLOCOCCINUM HAS ANY EFFECT can be found in the list of conflicts of interest of the paper’s authors:
All three review authors are research‐active in the field of homeopathy. They were members of the International Scientific Committee for Homeopathic Investigations (ISCHI), whose membership also included two employees of Boiron (the manufacturers of Oscillococcinum®), and whose committee activities ceased in July 2013. Progress with the Cochrane Review on Oscillococcinum® was presented briefly at ISCHI meetings in 2010 and 2011. The drafting of this Cochrane Review was carried out independently of those communications and of the authors’ other ongoing research activity. ISCHI has not run or sponsored any research on Oscillococcinum®.
Robert T Mathie: Dr Mathie is Research Development Adviser, British Homeopathic Association. He was a member of the International Scientific Committee on Homeopathic Investigations, which ceased its committee activities in July 2013. Joyce Frye: Part of Dr Frye’s salary was supported by a research grant from the Standard Homeopathic Company, paid to her employer, the Center for Integrative Medicine, Department of Family Medicine, University of Maryland, USA. Support ended in June 2013 when Dr Frye resigned from the University of Maryland. Standard Homeopathic Company does not manufacture Oscillococcinum or any similar product, and had no interest in the outcome of the review. Dr Frye received honoraria from the International Scientific Committee on Homeopathic Investigations, which was dissolved in July 2013. Peter Fisher: I am Expert Adviser on Complementary and Alternative Medicine to the National Institute for Health and Clinical Excellence (NICE), which may take an interest in the evidence in this review. I am Editor in Chief of an international, peer‐reviewed journal dedicated to homeopathy. All payments and reimbursements for lectures have been from universities or professional or learned societies. None of these lectures has been dedicated to the subject of this review. Some meetings have been supported by grants from commercial interests, including the manufacturer of the product that is the subject of the review.
So, to be clear: oscillococcinum does not help against the corona or any other virus. Those who claim otherwise are either mistaken, or have a commercial interest, or both.
A team of chiropractic researchers conducted a review of the safety of spinal manipulative therapy (SMT) in children under 10 years. They aimed to:
1) describe adverse events;
2) report the incidence of adverse events;
3) determine whether SMT increases the risk of adverse events compared to other interventions.
They searched MEDLINE, CINAHL, and Index to Chiropractic Literature from January 1, 1990 to August 1, 2019. Eligible studies were case reports/series, cohort studies and randomized controlled trials. Studies of high and acceptable methodological quality were included.
Most adverse events are mild (e.g., increased crying, soreness). One case report describes a severe adverse event (rib fracture in a 21-day-old) and another an indirect harm in a 4-month-old. The incidence of mild adverse events ranges from 0.3% (95% CI: 0.06, 1.82) to 22.22% (95% CI: 6.32, 54.74). Whether SMT increases the risk of adverse events in children is unknown.
The authors concluded that the risk of moderate and severe adverse events is unknown in children treated with SMT. It is unclear whether SMT increases the risk of adverse events in children < 10 years.
Thanks to their ingenious methodology, the authors managed to miss 11 of the 13 studies included in the review by Vohra et al which reported 9 serious adverse events and 20 cases of delayed diagnosis associated with SMT. Another review reported 15 serious adverse events and 775 mild to moderate adverse events following manual therapy. As far as I can see, the authors of the new review make just one reasonable point:
We recommend the implementation of a population-based active surveillance program to measure the incidence of severe and serious adverse events following SMT treatment in this population.
In the absence of such a surveillance system, any incidence figures are not just guess-work but also a depiction of the tip of a much bigger iceberg. So, why do the authors of this review not make this point clearly and powerfully? Why does the review read mostly like an attempt to white-wash a thorny subject? Why do they not provide a breakdown of the adverse events according to profession? The answer to these questions can be found at the very end of the paper:
This study was supported by the College of Chiropractors of British Columbia to Ontario Tech University. The College of Chiropractors of British Columbia was not involved in the design, conduct or interpretation of the research that informed the research. This research was undertaken, in part, thanks to funding from the Canada Research Chairs program to Pierre Côté who holds the Canada Research Chair in Disability Prevention and Rehabilitation at Ontario Tech University, and from the Canadian Chiropractic Research Foundation to Carol Cancelliere who holds a Research Chair in Knowledge Translation in the Faculty of Health Sciences at Ontario Tech University.
This study was supported by the College of Chiropractors of British Columbia to Ontario Tech University. The College of Chiropractors of British Columbia was not involved in the design, conduct or interpretation of the research that informed the research. This research was undertaken, in part, thanks to funding from the Canada Research Chairs program to Pierre Côté who holds the Canada Research Chair in Disability Prevention and Rehabilitation at Ontario Tech University, and funding from the Canadian Chiropractic Research Foundation to Carol Cancelliere who holds a Research Chair in Knowledge Translation in the Faculty of Health Sciences at Ontario Tech University.
I have often felt that chiropractic is similar to a cult. An investigation by cult members into the dealings of a cult is not the most productive of concepts, I guess.
About 85% of German children are treated with herbal remedies. Yet, little is known about the effects of such interventions. A new study might tell us more.
This analysis accessed 2063 datasets from the paediatric population in the PhytoVIS data base, screening for information on indication, gender, treatment, co-medication and tolerability. The results suggest that the majority of patients was treated with herbal medicine for the following conditions:
- common cold,
- fever,
- digestive complaints,
- skin diseases,
- sleep disturbances
- anxiety.
The perceived effect of the therapy was rated in 84% of the patients as very good or good without adverse events.
The authors concluded that the results confirm the good clinical effects and safety of herbal medicinal products in this patient population and show that they are widely used in Germany.
If you are a fan of herbal medicine, you will be jubilant. If, on the other hand, you are a critical thinker or a responsible healthcare professional, you might wonder what this database is, why it was set up and how exactly these findings were produced. Here are some details:
The data were collected by means of a retrospective, anonymous, one-off survey consisting of 20 questions on the user’s experience with herbal remedies. The questions included complaints/ disease, information on drug use, concomitant factors/diseases as well as basic patient data. Trained interviewers performed the interviews in pharmacies and doctor’s offices. Data were collected in the Western Part of Germany between April 2014 and December 2016. The only inclusion criterion was the intake of herbal drugs in the last 8 weeks before the individual interview. The primary endpoint was the effect and tolerability of the products according to the user.
And who participated in this survey? If I understand it correctly, the survey is based on a convenience sample of parents using herbal remedies. This means that those parents who had a positive experience tended to volunteer, while those with a negative experience were absent or tended to refuse. (Thus the survey is not far from the scenario I often use where people in a hamburger restaurant are questioned whether they like hamburgers.)
So, there are two very obvious factors other than the effectiveness of herbal remedies determining the results:
- selection bias,
- lack of objective outcome measure.
This means that conclusions about the clinical effects of herbal remedies in paediatric patients are quite simply not possible on the basis of this survey. So, why do the authors nevertheless draw such conclusions (without a critical discussion of the limitations of their survey)?
Could it have something to do with the sponsor of the research?
The PhytoVIS study was funded by the Kooperation Phytopharmaka GbR Bonn, Germany.
Or could it have something to do with the affiliations of the paper’s authors:
1 Institute of Pharmacy, University of Leipzig, Brüderstr. 34, 04103, Leipzig, Germny. [email protected].
2 Kooperation Phytopharmaka GbR, Plittersdorfer Str. 218, 573, Bonn, Germany. [email protected].
3 Institute of Medical Statistics and Computational Biology, Faculty of Medicine, University of Cologne, Kerpener Str. 62, 50937, Cologne, Germany.
4 ClinNovis GmbH, Genter Str. 7, 50672, Cologne, Germany.
5 Bayer Consumer Health, Research & Development, Phytomedicines Supply and Development Center, Steigerwald Arzneimittelwerk GmbH, Havelstr. 5, 64295, Darmstadt, Germany.
6 Kooperation Phytopharmaka GbR, Plittersdorfer Str. 218, 53173, Bonn, Germany.
7 Institute of Pharmaceutical Biology, Goethe University Frankfurt, Max-von-Laue-Str. 9, 60438, Frankfurt, Germany.
8 Chair of Naturopathy, University Medicine Rostock, Ernst-Heydemann Str. 6, 18057, Rostock, Germany.
WHAT DO YOU THINK?
An enthusiast of homeopathy recently posted an overview of systematic reviews of homeopathy concluding that the data we do have point towards homeopathy as having an effect greater than that of placebo:
In recent decades, homeopathy has been examined via a number of clinical trials, the number of which now allow meta-analysis. As we can see from the study findings, the type of homeopathy research (ie, individualized vs non-individualized, placebo-controlled vs non-placebo-controlled) can have a strong influence on the results, although trial quality also has a strong effect.
All meta-analyses performed in at least a somewhat open and rigorous manner have found statistically significant effects. This suggests that homeopathy has a greater-than-placebo effect, or at least a strong trend in that direction, when using data from the totality of homeopathy research, or from individualized, placebo-controlled trials. The meta-analyses with questionable methodology, one of which is undergoing government investigation for academic irregularities, have produced negative results, which have been demonstrated to be a direct result of their exclusion of vast swathes of the homeopathic clinical trial literature (based on arbitrary and unexplained criteria), as well as of their failure to differentiate – as Mathie has done – different types of homeopathic research.
The clinical data are flawed. Issues with methodology used in homeopathy RCTs, combined with a lack of research funding, have produced a lack of high-quality trials and data. However, the data we do have point towards homeopathy as having an effect greater than that of placebo.
There can be no argument with this conclusion, aside from possible new data emerging. Anyone who disputes this is going against the existing set of the highest-quality evidence on homeopathy.
His overview is based on the following publications:
| Kleijnen, 19911 | All types of homeopathy (eg, single remedy vs combination). Methodological quality assessed; 105 trials. Results: Positive trend, regardless of type of homeopathy; 81 trials were positive, 24 showed no effect. |
| Linde, 19972 | All types of homeopathy. Out of 185 trials, 119 met inclusion criteria; 89 of these had extractable data. Results: OR = 2.45 (95% CI 2.05-2.93). |
| Ernst, 19983 | Individualized homeopathy; 5 trials determined to be high-quality. Results: OR = 0. |
| Linde, 19985 | Individualized homeopathy; 32 trials, 19 of which had extractable data. Results: OR = 1.62 for all trials (95% CI 1.17-2.23). Only high-quality trials produced no significant trend. |
| Cucherat, 20009 | All types of homeopathy; 118 trials, 16 of which met inclusion criteria. Used unusual method of combining p values. Results: All trials = p< 0.000036. Less than 10% dropouts: p<0.084; less than 5% dropouts (higher standards than most trials considered reliable): p<0.08 (non-significant). |
| Shang, 200511 | All types of homeopathy; only 8 trials selected from 21 high-quality trials of 110 selected with unusual criteria. Results: OR = 0.88 (0.65-1.19). Result strongly disputed by statisticians. |
| Mathie, 201413 | Individualized homeopathy; of the analysis pooled data from 22 higher-quality, individualized, double-blind RCTs. Results: OR = 1.53 (1.22-1.91) for all trials pooled; OR = 1.93 (1.16-3.38) for the 3 reliable trials. |
| NHMRC, 201516 | Out of 176 studies, 171 were excluded, leaving only 5 for the study. Investigators used unprecedented methods, did not combine data, and are currently under investigation for outcome shopping. Results: Negative results. |
| Mathie, 201720 | Non-individualized homeopathy; very few higher-quality trials. Results: For 54 trials with extractable data, SMD = -0.33 (-0.44, -0.21). When these were adjusted for publication bias, SMD = -0.16 (-0.46,-0.09). The 3 high-quality trials had non-significant results: SMD = -0.18 (-0.46, +0.09). |
| Mathie, 201821 | Individualized, other-than-placebo-controlled trials; 11 trials found, 8 with extractable data. Results: 4 heterogeneous comparative trials showed a non-significant difference. One trial in this group was positive. Three heterogeneous trials with additive homeopathy showed a statistically significant SMD. No definitive conclusion possible due to trial heterogeneity, poor quality, and low number of trials. |
| Mathie, 201922 | Non-individualized, other-than-placebo-controlled trials; 17 RCTs found, 14 with high risk of bias. Results: Significant heterogeneity prevented much comparison; 3 comparable trials showed a non-significant SMD. |
Apart from getting the wrong end of the stick when interpreting the results of these papers (see for instance here, and here), there are other rather embarrassing flaws in this overview:
- Many older systematic reviews were omitted (including about 10 of my own papers). This is relevant because the author of the above review went beck until 1991 to find the reviews he included.
- Several new papers were missing as well. This is relevant because the author evidently included reviews up to 2019. Here are the key passages from the conclusions of some of them:
… homoeopathy as a whole may be considered as a placebo treatment.
We found no evidence to support the efficacy of homeopathic medicinal products
I am, of course, not saying that this overview amounts to anything like a systematic review. It merely gives you a flavour how trustworthy proponents of homeopathy are when they pretend to provide us with an objective evaluation of the best available evidence.
A 2020 review entitled ‘Prevention, Treatment and Management of Tuberculosis through Combinational Approaches of Different Indian Systems of Medicine’ stated that recent research suggests that Homeopathic treatment along with the antibiotics synergise the effect of antibiotics while reaching to its site of action. This is a surprising finding, if there ever was one. Therefore, I looked a bit closer with a view of determining what the original data were that led to this conclusion.
Here is the unaltered section on homeopathy from the review:
Homeopathy is a special, organic, holistic medicine process which activates the healing responses of the body without any known contraindications or common side effects. In his book Organon of Medicine, aka Organon of Healing Art, the founder of Homeopathy, Samuel Hahnemann, developed these principles. Homoeopathy does not have holistic treatment and hence is incapable of curing TB because of homeopathic portions of water, which is not enough to treat TB, but recent research suggests the use of homeopathic treatment along with the antibiotics, which results in synergising the effect of antibiotics while reaching to its site of action. As standard MDR-TB medicines are second-generation antibiotics taken for 24–27 months, thus, apart from conventional treatment, homeopathy in MDR-TB tends to improve outcome. Hence, it increases the bioavailability of the isoniazid and rifampicin, which resulted in the reduction of tuberculosis therapy. This study has been proving by analysing following parameters as below:
There was no significant difference in smear conversion from positive to negative at 95 % CI between homeopathy (H) and individualised standard treatment regimen (SR) + placebo (P). However, the conversion culture conversion from positive to negative in SR + H was seen in 29 (48.3 %) patients and 23 (38.3 %) patients in SR + P group (p = 0.269) (ITT) and 27 (55.1 %); 21 (42.8 %), p = 0.225 (PP) implying that as compared to SR + P group, culture conversion in SR + H group was more by 10 %. So the difference, although favourable to the SR + H group is not statistically significant as shown in table 4.
All patients had far advanced lung disease as evident from the extensive infiltration, cavitation ns and fibrosis/collapse. Chest X-ray (CXR) were further assessed using RAT (Radiological Assessment Tool), examples of one case rated as + 5 and one case as _4. Significant improvements were seen statistically in CXR in the SR + H 37 (61.7 %); as compared to SR + P 20 (33.3 %), p = 0.002 at 95 % CI (ITT). Homeopathy system having all possibility to enhance the treatment and management of the TB [13].
To be clear: there is no further mention or discussion of homeopathy in this paper. The above-quoted section (its multiple falsehoods are so obvious that I probably don’t need to mention them here) is thus the only evidence provided for backing up the claim that Homeopathic treatment along with the antibiotics synergise the effect of antibiotics while reaching to its site of action. I fail to see how the information provided supports the claim. Moreover, I am not aware of any sound evidence that would support the claim that homeopathy acts synergistically to antibiotics.
In the reference list, I found one of my own articles cited as reference 13 (see above). It refers to this article: Ernst E. A systematic review of systematic reviews of homeopathy. Dr J Clin Pharmacol 2002; 54: 577–582. Here is its abstract:
Homeopathy remains one of the most controversial subjects in therapeutics. This article is an attempt to clarify its effectiveness based on recent systematic reviews. Electronic databases were searched for systematic reviews/meta-analysis on the subject. Seventeen articles fulfilled the inclusion/exclusion criteria. Six of them related to re-analyses of one landmark meta-analysis. Collectively they implied that the overall positive result of this meta-analysis is not supported by a critical analysis of the data. Eleven independent systematic reviews were located. Collectively they failed to provide strong evidence in favour of homeopathy. In particular, there was no condition which responds convincingly better to homeopathic treatment than to placebo or other control interventions. Similarly, there was no homeopathic remedy that was demonstrated to yield clinical effects that are convincingly different from placebo. It is concluded that the best clinical evidence for homeopathy available to date does not warrant positive recommendations for its use in clinical practice.
I fail to see how my paper backs up the very odd sentence: Homeopathy system having all possibility to enhance the treatment and management of the TB.
This is much more than sloppiness. In fact, it is one of the clearest cases of scientific misconduct I have ever come across. What is more, if its message would get adopted, it has the potential to do considerable harm. The authors of the paper in question, Priyanka Sharma , Ramesh K. Goyal, and Mukesh Nandave, declared no conflicts of interest; their affiliation is provided as ‘Departments of Pharmacology and Toxicology, Delhi Pharmaceutical Sciences and Research University, New Delhi, India’. The journal that published their review is ‘Drug Research‘, a publication which I always had thought was reputable.
I am not sure what I should do next.
Any advice?
I have been sceptical about Craniosacral Therapy (CST) several times (see for instance here, here and here). Now, a new paper might change all this:
The systematic review assessed the evidence of Craniosacral Therapy (CST) for the treatment of chronic pain. Randomized clinical trials (RCTs) assessing the effects of CST in chronic pain patients were eligible. Pain intensity and functional disability were the primary outcomes. Risk of bias was assessed using the Cochrane tool.
Ten RCTs with a total of 681 patients suffering from neck and back pain, migraine, headache, fibromyalgia, epicondylitis, and pelvic girdle pain were included.
Compared to treatment as usual, CST showed greater post intervention effects on:
- pain intensity (SMD=-0.32, 95%CI=[−0.61,-0.02])
- disability (SMD=-0.58, 95%CI=[−0.92,-0.24]).
Compared to manual/non-manual sham, CST showed greater post intervention effects on:
- pain intensity (SMD=-0.63, 95%CI=[−0.90,-0.37])
- disability (SMD=-0.54, 95%CI=[−0.81,-0.28]) ;
Compared to active manual treatments, CST showed greater post intervention effects on:
- pain intensity (SMD=-0.53, 95%CI=[−0.89,-0.16])
- disability (SMD=-0.58, 95%CI=[−0.95,-0.21]) .
At six months, CST showed greater effects on pain intensity (SMD=-0.59, 95%CI=[−0.99,-0.19]) and disability (SMD=-0.53, 95%CI=[−0.87,-0.19]) versus sham. Secondary outcomes were all significantly more improved in CST patients than in other groups, except for six-month mental quality of life versus sham. Sensitivity analyses revealed robust effects of CST against most risk of bias domains. Five of the 10 RCTs reported safety data. No serious adverse events occurred. Minor adverse events were equally distributed between the groups.
The authors concluded that, in patients with chronic pain, this meta-analysis suggests significant and robust effects of CST on pain and function lasting up to six months. More RCTs strictly following CONSORT are needed to further corroborate the effects and safety of CST on chronic pain.
Robust effects! This looks almost convincing, particularly to an uncritical proponent of so-called alternative medicine (SCAM). However, a bit of critical thinking quickly discloses numerous problems, not with this (technically well-made) review, but with the interpretation of its results and the conclusions. Let me mention a few that spring into my mind:
- The literature searches were concluded in August 2018; why publish the paper only in 2020? Meanwhile, there might have been further studies which would render the review outdated even on the day it was published. (I know that there are many reasons for such a delay, but a responsible journal editor must insist on an update of the searches before publication.)
- Comparisons to ‘treatment as usual’ do not control for the potentially important placebo effects of CST and thus tell us nothing about the effectiveness of CST per se.
- The same applies to comparisons to ‘active’ manual treatments and ‘non-manual’ sham (the purpose of a sham is to blind patients; a non-manual sham defies this purpose).
- This leaves us with exactly two trials employing a sham that might have been sufficiently credible to be able to fool patients into believing that they were receiving the verum.
- One of these trials (ref 44) is far too flimsy to be taken seriously: it was tiny (n=23), did not adequately blind patients, and failed to mention adverse effects (thus violating research ethics [I cannot take such trials seriously]).
- The other trial (ref 41) is by the same research group as the review, and the authors award themselves a higher quality score than any other of the primary studies (perhaps even correctly, because the other trials are even worse). Yet, their study has considerable weaknesses which they fail to discuss: it was small (n=54), there was no check to see whether patient-blinding was successful, and – as with all the CST studies – the therapist was, of course, no blind. The latter point is crucial, I think, because patients can easily be influenced by the therapists via verbal or non-verbal communication to report the findings favoured by the therapist. This means that the small effects seen in such studies are likely to be due to this residual bias and thus have nothing to do with the intervention per se.
- Despite the fact that the review findings depend critically on their own primary study, the authors of the review declared that they have no conflict of interest.
Considering all this plus the rather important fact that CST completely lacks biological plausibility, I do not think that the conclusions of the review are warranted. I much prefer the ones from my own systematic review of 2012. It included 6 RCTs (all of which were burdened with a high risk of bias) and concluded that the notion that CST is associated with more than non‐specific effects is not based on evidence from rigorous RCTs.
So, why do the review authors first go to the trouble of conducting a technically sound systematic review and meta-analysis and then fail utterly to interpret its findings critically? I might have an answer to this question. Back in 2016, I included the head of this research group, Gustav Dobos, into my ‘hall of fame’ because he is one of the many SCAM researchers who never seem to publish a negative result. This is what I then wrote about him:
Dobos seems to be an ‘all-rounder’ whose research tackles a wide range of alternative treatments. That is perhaps unremarkable – but what I do find remarkable is the impression that, whatever he researches, the results turn out to be pretty positive. This might imply one of two things, in my view:
- all alternative therapies are effective,
- the ‘Trustworthiness Index’ of Prof Dobos is unusual.
I let my readers chose which possibility they deem to be more likely.
Yesterday, we discussed a paper concluding (amongst other things) that there are insufficient high‐quality RCTs to judge the efficacy of acupuncture for cancer‐related pain. Today, we are looking at one that overtly contradicts this verdict.
This systematic review (published in JAMA Oncology) evaluated the existing randomized clinical trials (RCTs) for evidence of the association of acupuncture and acupressure with reduction in cancer pain. Randomized clinical trials that compared acupuncture and acupressure with a sham control, analgesic therapy, or usual care for managing cancer pain were included. The primary outcome was pain intensity measured by the Brief Pain Inventory, Numerical Rating Scale, Visual Analog Scale, or Verbal Rating Scale.
A total of 17 RCTs (with 1111 patients) were included, and data from 14 RCTs (with 920 patients) were used in the meta-analysis. Seven sham-controlled RCTs (35%) were notable for their high quality, being judged to have a low risk of bias for all of their domains, and showed that real (compared with sham) acupuncture was associated with reduced pain intensity. A favourable association was also seen when acupuncture and acupressure were combined with analgesic therapy in 6 RCTs for reducing pain intensity and in 2 RCTs for reducing opioid dose. The evidence grade was moderate because of the substantial heterogeneity among studies.
The authors concluded that this systematic review and meta-analysis found that acupuncture and/or acupressure was significantly associated with reduced cancer pain and decreased use of analgesics, although the evidence level was moderate. This finding suggests that more rigorous trials are needed to identify the association of acupuncture and acupressure with specific types of cancer pain and to integrate such evidence into clinical care to reduce opioid use.
So, which of the two conclusions should we trust?
Personally, I find the JAMA paper unimpressive to the point of being suspect. Here are some of my reasons:
- About half of the primary studies are Chinese; and we have seen repeatedly that they are unreliable and report only positive results.
- Many of the trials are published in Chinese and can thus not be checked by non-Chinese readers (nor, presumably, by the experts who acted as peer-reviewers for JAMA Oncology).
- I have my doubts about the rigor of the peer-review of some of the journals that published the primary studies included in the review.
- One paper included in the review is even a mere doctoral thesis which usually is not peer-reviewed in the usual sense.
- The authors state that they included only clinical trials that compared acupuncture and acupressure with a sham control, analgesic therapy, or usual care. However, this is evidently not true; many of the studies had the infamous A+B versus B design comparing acupuncture plus a conventional therapy against the conventional therapy. As we have discussed ad nauseam on this blog, such trials cannot produce a negative finding even if ‘A’ is a placebo.
- Contrary to what the authors claim, the quality of most of the included studies was extremely poor, as far as I can see.
- One included paper which I cannot access is entitled ‘Clinical observation on 30 cases of moderate and severe cancer pain of bone metastasis treated by auricular acupressure‘. Are the review authors seriously claiming that this is an RCT?
The more I study the details of the JAMA Oncology paper, the more I feel it might be worth a complaint to the editor with a view of initiating a thorough investigation and a possible retraction.
