MD, PhD, FMedSci, FSB, FRCP, FRCPEd

scientific misconduct

Aromatherapy is popular and pleasant – but does it have real health effects? The last time I tried to find an answer to this question was in 2012. At that time, our systematic review concluded that “the evidence is not sufficiently convincing that aromatherapy is an effective therapy for any condition.” But 5 years can be a long time in research, and more up-to-date information would perhaps be helpful.

This systematic review of 2017 aimed to provide an analysis of the clinical evidence on the efficacy of aromatherapy specifically for depressive symptoms on any type of patients. The authors searched 5 databases for relevant studies Outcome measures included scales measuring depressive symptoms levels. Twelve randomized controlled trials (RCTs) were included. Aromatherapy was administered by inhalation (5 studies) or massage (7 studies). Seven RCTs showed improvement in depressive symptoms. The quality of half of the studies was low, and the administration protocols varied considerably among the studies. Different assessment tools were employed in the studies. In 6 of the RCTs, aromatherapy was compared to no intervention.

Despite these caveats, the authors concluded that aromatherapy showed potential to be used as an effective therapeutic option for the relief of depressive symptoms in a wide variety of subjects. Particularly, aromatherapy massage showed to have more beneficial effects than inhalation aromatherapy.

Apart from the poor English, this paper is irritating because of the almost total lack of critical input. Given that half of the trials were of poor quality (only one was given the full points on the quality scale) and many totally failed to control for placebo-effects, I think that calling aromatherapy an effective therapeutic option for the relief of depressive symptoms is simply not warranted. In fact, it is highly misleading and, given the fact that depression is a life-threatening condition, it seems unethical and dangerous.

Considering these facts, my conclusion remains that “the evidence is not sufficiently convincing that aromatherapy is an effective therapy for any condition, including depression.”

Dana Ullman is an indefatigable promotor of bogus claims and an unwitting contributor of hilarity. Therefore he has become a regular feature of this blog (see for instance here, here and here). His latest laughable assertion is that lead and other poisonings can be successfully treated with homeopathy.

Just to make sure: lead poisoning is no joke. The greatest risk is to brain development in babies, where irreversible damage can occur. Higher levels can damage the kidneys and nervous system in both children and adults. Very high lead levels may cause seizures, unconsciousness and death.

In view of this, Ullman’s claim is surprising, to say the least. In order to persuade the unsuspecting public of his notion, Ullman first cites a review of basic research on homeopathy and toxins published in Human and Experimental Toxicology. “Of forty high-quality studies, 27 showed positive results from homeopathic treatment”, Ullman states.

Now, now, now Dana!

Has your mom not taught you that telling porkies is forbidden?

Or did you perhaps miss this line in the article’s abstract? “The quality of evidence in these studies was low with only 43% achieving one half of the maximum possible quality score and only 31% reported in a fashion that permitted re-evaluation of the data. Very few studies were independently replicated using comparable models.”

Hardly ‘high quality studies’, wouldn’t you agree?

But this review was of pre-clinical studies; what about the much more important clinical evidence?

Here Ullman cites one trial where a potentized homeopathic remedy, Arsenicum Album 30C, was administered to  55 people who were entered into a double-blind, placebo-controlled trial. According to Ullman, the homeopathically treated group “experienced higher excretion of arsenic in their urine for the first eleven days, compared to those given a placebo.”

Na, na, na, Dana, this is getting serious!!!

Another porky – and not even a little one.

The authors of this study clearly stated that, at the end of the 11-day RCT, there was no significant difference between the homeopathy and the placebo group: “The differences in the concentration between the two groups (drug versus placebo) were generally a little higher during the first week, but subsequently the differences were not so palpable, particularly at the 11th day.” And for those who are a bit slow on the uptake, they even included a graph that makes it abundantly clear.

The only other clinical study cited by Ullman in support of his surprising claim is a double-blind randomized trial which was conducted with 131 workers who suffered lead poisoning at the Ajax battery plant in Bauru, São Paulo State, Brazil. Subjects were prescribed homeopathic doses of lead (Plumbum metallicum 15C) or placebo which they took orally for 35 days. The results of this RCT show that homeopathy is not better than placebo.

So, we seem to have all of two RCTs on the subject (I did a quick Medline-search and also found no further RCTs), and both are negative.

Anyone who is not given to compulsive porky-telling would, I guess, conclude from this evidence that people suffering from lead poisoning should urgently see conventional experts and avoid homeopaths at all costs – not so Dana Ullman who boldly concludes his article with these words:

“As an adjunct to conventional medical treatment, professional homeopathic care is recommended for people who have been exposed (or think they have been exposed) to toxic substances… Even if you do not have a professional homeopath in your town, many homeopathic practitioners “see” their patients via Skype or do consultations over the telephone. Unlike acupuncturists, who put needles in you, or chiropractors, who adjust your spine, homeopaths are not “hands-on”: they simply need to conduct a detailed interview… If your symptoms are serious or potentially serious, it is important to see a professional homeopath and/or physician. While a homeopath will commonly prescribe a safe homeopathic dose of the toxic substance to which one was exposed, the homeopath may instead decide that a different substance more closely matches the patient’s unique symptoms…”

It takes a lot these days to make me speechless but there, Dana, you almost succeeded!

Yes, homeopaths are incredibly fond of the notion that homeopathy has been proven to work in numerous population studies of outbreaks of infectious diseases. The argument is bound to come up in any discussion with a ‘well-informed’ homeopathy fan. Therefore, it might be worth addressing it once and for all.

This website offers a fairly good summary of what homeopaths consider to be convincing evidence. It also provides links to the original articles which is valuable for all who want to study them in full detail. I will therefore present the crucial passage here unchanged.

START OF QUOTE

By the end of year 2014, there have been 19 papers published on Epidemiological studies on 7 epidemic diseases (scarlet fever, typhus fever, Cholera, Dengue, meningococcal, influenza and Leptospirosis) in 11 peer-reviewed (beyond year 1893) journals in evidence of Homeopathy including 2 Randomised Controlled Trials.

1. Samuel Hahnemann, “The Cure and prevention of scarlet fever”, Zeitschrift für Praktischen Medizin (Journal of Practical Medicine), 1801, Republished in Lesser Writings. B.Jain Publishing, New Delhi

Preventive use of homeopathy was first applied in 1799 during an epidemic of scarlet fever in Königslütter, Germany, when Dr. Hahnemann prescribed a single dose of Belladonna, as the remedy of the genus epidemicus to susceptible children in the town with more than 95% success rate. In this paper, he also specified how the Belladonna has to be potentised to 1/24,000,000 dilution. His recommended dose of Belladonna was 0.0416 nanograms to be repeated every 72 hrs. This is the first recorded nano dose of medicine used in treatment of any disease [6]. It was another 125 years before Gladys Henry and George Frederick developed a vaccine for scarlet fever in 1924.

2. Samuel Hahnemann, “Scarlet fever and Purpura miliaris, two different diseases”, Zeitschrift für Praktischen Medizin, vol. 24, part. 1, 1806

3. Samuel Hahnemann, “Observations on scarlet fever”, Allgemeine Reichanzeiger (General Reich Gazette), No. 160, Germany, 1808

4. Samuel Hahnemann, “Reply to a question about the prophylactic for scarlet fever”, Zeitschrift für Praktischen Medizin, vol. 27, part. 4, p. 152-156, 1808

5. Samuel Hahnemann, “Treatment of typhus & fever at present prevailing”, Allgemeine Reichanzeiger, No. 6, Jan. 1814.

6. Hufeland, Prophylactic powers of Belladonna against Scarlet Fever , The Lancet, 1829
The proper use of belladonna has, in most cases, prevented infection. Numerous observations have shown that, by the general use of belladonna, epidemics of scarlet fever have actually been arrested. In those few instances where the use of belladonna was insufficient to prevent infection, the disease has been invariably slight. The Prussian (German Empire) Government ordered the use of the prophylactic during all scarlet fever epidemics

7. Samuel Hahnemann, “Cure and prevention of Asiatic cholera”, Archiv für die homöopathische Heilkunst (Archives for the Homoeopathic Healing Art), Vol. 11, part 1, 1831.
Cuprum 30c once every week as preventive medicine

8. Samuel Hahnemann, “On the contagiousness of cholera”. British Homoeopathic Journal, Vol. 7, 1849

9. Samuel Hahnemann, “Appeal to Thinking Philanthropists Respecting the Mode of Propagation of the Asiatic Cholera”, 20 pages, 1831. Republished in British Homoeopathic Journal, Oct 1849.

He said, “On board ships – in those confined spaces, filled with mouldy watery vapours, the cholera-miasm finds a favourable element for its multiplication, and grows into an enormously increased brood of those excessively minute, invisible, living creatures, so inimical to human life, of which the contagious matter of the cholera most probably consists millions of those miasmatic animated beings, which, at first developed on the broad marshy banks or the tepid Ganges– on board these ships, I say, this concentrated aggravated miasm kills several of the crew …” [7].
It was another 59 years (1890) before Koch saw these organisms, and later on orthodox medicine gave them the name ‘germs’

10. Charles Woodhull Eaton, The Facts about Variolinum, Transactions of the American Institute of Homoeopathy, 1907
2806 patients were treated prophylactically with Variolinum 30 (a nosode) for prevention of smallpox in Iowa. Of the 547 patients definitely exposed, only 14 developed the disease. Efficacy rate of 97.5%

11. Taylor Smith A, Poliomyelitis and prophylaxis British Homoeopathic Journal, 1950
In 1950 during an epidemic of poliomyelitis, Dr Taylor Smith of Johannesburg, South Africa protected 82 people with homoeopathic Lathyrus sativus. Of the 82 so immunised, 12 came into direct contact with disease. None were infected.

12. Oscillococcinum 200c in the treatment of influenza during epidemic in France from 1984-1987, British Journal of Clinical Pharmacology (1989)
A DBRPCT, Oscillococcinum 200c taken twice daily for 5 days significantly increased the rate of cure within two days (n=487, 237 treated and 241 on placebo), absence of symptoms at 48 hours, relative risk estimate significantly favour homeopathy (p=0.048), no pain and no fever (p=0.048), recovery rate (headache, stiffness, articular pain, shivering reduction) at 48 hours better in homeopathy group (p=0.032)

13. Bernard Leary, Cholera 1854 Update, British Homoeopathic Journal, 1994
Sir William Wilde, the well-known allopathic doctor of Dublin, which in his work entitled “Austria and its Institutions”, wrote: “Upon comparing the report of the treatment of Cholera in the Homeopathic hospital testified to by two allopathic medical inspectors appointed by Government with that of the treatment of the same disease in the other hospitals of Vienna during the same period the epidemic of 1836, it appeared that while two-thirds of the cases treated by Dr. Fleischmann the physician of the Homeopathic hospital, recovered, two-thirds of those treated by the ordinary methods in the other hospitals died.”

14. Meningococcinum – its protective effect against meningococcal disease, Homeopathy Links, 2001 (2001)
A total of 65,826 people between the ages of 0–20 were immunised homeopathically to protect against meningococcal disease while 23,532 were not. Over a year period, 4 out of 65,826 protected homeopathically developed meningococcal infection. 20 out of 23,532 not protected developed meningococcal infection. Based on the infection rate in the unprotected group, 58 cases of infection could have been expected in the homeopathically protected group. Instead, there were only four cases of meningococcal infection. Statistical analysis showed that homeopathic immunisation offered 95% protection in the first six months and 91% protection over the year against meningococcal disease. [8]

15. Contribution of homeopathy to the control of an outbreak of dengue epidemic in Macaé, Rio de Janeiro, Brazil in 2007-8 , International Journal of High Dilution Research, 2008
In a campaign ‘Homeopathy campaign against dengue’ by Brazilian Govt, “156,000 doses of homeopathic remedy were freely distributed in April and May 2007 to asymptomatic patients and 129 doses to symptomatic patients treated in outpatient clinics, according to the notion of genus epidemicus . The remedy used was a homeopathic complex against dengue containing Phosphorus 30c, Crotalus horridus 30c and Eupatorium perfoliatum 30c. The incidence of the disease in the first three months of 2008 fell 93% by comparison to the corresponding period in 2007, whereas in the rest of the State of Rio de Janeiro there was an increase of 128%.”

16. Marino R. Eupatorium perfoliatum 30c for the Dengue Epidemics in Brazil in 2007. International Journal of High Dilution Research, 2008
In May 2001, prophylactic use of Eupatorium perfoliatum 30c single dose was given during a dengue outbreak to 40% of residents in the most highly affected neighbourhood which resulted in significant decrease in dengue incidence by 81.5% (p<0.0001) when compared with those neighbourhoods that did not receive homeopathic prophylaxis.

17. Bracho et. al. Application of 200C potency of bacteria for Leptospirosis epidemic control in Cuba 2007-8 (2010)
Conducted by the Finlay Institute, a vaccines producer in Cuba gave 2.308562 million (70% of the target population above the age of 1 year) people in Cuba given two doses (1 dose=5 drops) of 200C potency of a nosode prepared from Leptospirosis bacteria, each (7-9 days apart), for protection against Leptospirosis (fever+jaundice+ inflammation in kidney+enlargement of spleen) with 84% decrease in disease incidence and only 10 reported cases. Dramatic decrease in morbidity within two weeks and zero morbidity of hospitalised patients, non-treated (8.8 millions) area saw an increase in number of cases from 309 cases in 2007 to 376 in 2008 representing a 21% increase. The cost of homeopathic immunization =1/15th of conventional vaccine.

18. Effect of individualized homoeopathic treatment in influenza like illness, Indian Journal of Research in Homeopathy (2013)
A multicenter, single blind, randomized, placebo controlled study to evaluate the effect of homoeopathic medicines in the treatment of Influenza like illness and to compare the efficacy of LM (50 millisimal) potency vis-à-vis centesimal (C) potency. In LM group (n=152), C group (n=147) or placebo (n=148) group. The study revealed the significant effect of individualized homoeopathic treatment in the patients suffering from ILI with no marked difference between LM and Centesimal groups. The medicines which were commonly prescribed were: Arsenic album, Bryonia alba, Rhus tox., Belladonna, Nux vomica, Sepia, Phosphorus, Gelsemium, Sulphur, Natrum mur. and Aconitum napellus. [9]

19. Reevaluation of the Effectiveness of Homoeoprophylaxis Against Leptospirosis in Cuba in 2007-8, Journal of Evidence-based Complementary & Alternative Medicine (2014)
The results support the previous conclusions that homoeoprophylaxis can be used to effectively immunize people against targeted infectious diseases such as leptospirosis.

References
[1] Iman Navab, Lives saved by Homeopathy in Epidemics and Pandemics, https://drnancymalik.wordpress.com/2013/01/23/epidemics-and-pandemics/

[2] Reshu Agarwal, Natural History of Disease and Homeopathy at different levels of Intervention, http://www.homeorizon.com/homeopathic-articles/homeopathic-philosophy/disease-history

[3] Homoeopathy- Science of Gentle Healing, Deptt. of AYUSH, Ministry of Health & Family Welfare, Govt, of India, 2013, http://www.ccrhindia.org/Dossier/content/page22.html

[4] Conversation with David Little, http://hpathy.com/homeopathy-papers/conversations-with-david-little/

[5] Nancy Malik, Principles of Homeopathy Explained, 2015, https://drnancymalik.wordpress.com/article/homeopathy-explained/

[6] Nancy Malik, Recent Advances in Nanoparticle Research in Homeopathy, Homeopathy 4 Everyone, Vol.12, Issue 6, 18 June 2015, http://hpathy.com/scientific-research/recent-advances-in-nanoparticle-research-in-homeopathy/

[7] Samuel Hahnemann, “Appeal to Thinking Philanthropists Respecting the Mode of Propagation of the Asiatic Cholera”, 20 pages, 1831, Translated by R E Dudgeon, M.D. in The Lesser Writings of Samuel Hahnemann, 1851, B Jain Publishers, reproduced edition, 2002, p. 758

[8] Fran Sheffield, Homeoprophylaxis: Human Records, Studies and Trials, 2014, http://homeopathyplus.com/Homeoprophylaxis-Human-Records-Studies-Trials.pdf

[9] Homoeopathy in Flu-like Illness- Factsheet, Central Council for Research in Homoeopathy, Deptt. of AYUSH, Ministry of Health & Family Welfare, Govt, of India, 2015, http://ccrhindia.org/pdf/swineflu.pdf

END OF QUOTE

Whenever I read articles of this nature, I get a little embarrassed. It seems obvious to me that the authors of such reviews have done some ‘research’ and believe strongly in the correctness in what they write. It embarrasses me to see how such people, full of good will, can be so naïve, ignorant and wrong. They clearly fail to understand several crucial issues. To me. this seems like someone such as me lecturing others about car mechanics, quantum physics or kite flying. I have no idea about these subjects, and therefore it would be idiotic to lecture others about them. But homeopaths tend to be different! And this is when my embarrassment quickly turns into anger: articles like the above spread nonsense and misguide people about important issues. THEY ARE DANGEROUS! There is little room for embarrassment and plenty of room for criticism. So, let’s criticise the notions advanced above.

In my recent book, I briefly touched upon epidemics in relation to homeopathy:

Epidemics are outbreaks of disease occurring at the same time in one geographical area and affecting large number of people. In homeopathy, epidemics are important because, in its early days, they seemed to provide evidence for the notion that homeopathy is effective. The results of homeopathic treatment seemed often better than those obtained by conventional means. Today we know that this was not necessarily due to the effects of homeopathy per se, but might have been a false impression caused by bias and confounding.

This tells us the main reason why the much-treasured epidemiological evidence of homeopaths is far from compelling. The review above does not mention these caveats at all. But it is lousy also for a whole host of other reasons, for instance:

  • The text contains several errors (which I find too petty to correct here).
  • The list of studies is the result of cherry-picking the evidence.
  • It confuses what epidemiological studies are; RCTs are certainly not epidemiological studies, for instance.
  • It also omits some of the most important epidemiological studies suggesting homeopathy works.
  • It cites texts that are clearly not epidemiological studies.
  • Several studies are on prevention of illness rather than on treatment.
  • Some studies do not even employ homeopathy at all.

In the typical epidemiological case/control study, one large group of patients [A] is retrospectively compared to another group [B]. By large, I mean with a sample size of thousands of patients. In our case, group A has been treated homeopathically, while group B received the treatments available at the time. It is true that several of such reports seemed to suggest that homeopathy works. But this does by no means prove anything; the result might have been due to a range of circumstances, for instance:

  • group A might have been less ill than group B,
  • group A might have been richer and therefore better nourished,
  • group A might have benefitted from better hygiene in the homeopathic hospital,
  • group A might have received better care, e. g. hydration,
  • group B might have received treatments that made the situation not better but worse.

Because these are RETROSPECTIVE studies, there is no way to account for these and many other factors that might have influenced the outcome. This means that epidemiological studies of this nature can generate interesting results which, in turn, need testing in properly controlled studies where these confounding factors are adequately controlled for. Without such tests, they are next to worthless for recommendations regarding clinical practice.

As it happens, the above author also included two RCT in the review (these are NOT epidemiological studies, as I already mentioned). Let’s have a quick look at them.

The first RCT is flawed for a range of reasons and has been criticised many times before. Even its authors state that “the result cannot be explained given our present state of knowledge, but it calls for further rigorously designed clinical studies.” More importantly, the current Cochrane review of Oscillococcinum, the remedy used in this study, concluded: “There is insufficient good evidence to enable robust conclusions to be made about Oscillococcinum® in the prevention or treatment of influenza and influenza-like illness.”

The second RCT is equally flawed; for instance, its results could be due to the concomitant use of paracetamol, and it seems as though the study was not double blind. The findings of this RCT have so far not been confirmed by an independent replication.

What puzzles me most with these regularly voiced notions about the ‘epidemiological evidence’ for homeopathy is not the deplorable ineptitude of those who promote them, but it is this: do homeopaths really believe that conventional medics and scientists would ignore such evidence, if it were sound or even just encouraging? This assumes that all healthcare professionals (except homeopaths) are corrupt and cynical enough not to follow up leads with the potential to change medicine for ever. It assumes that we would supress knowledge that could save the lives of millions for the sole reason that we are against homeopathy or bribed by ‘BIG PHARMA’.

Surely, this shows more clearly than anything else how deluded homeopaths really are!!!

 

Can intercessory prayer improve the symptoms of sick people?

Why should it? It’s utterly implausible!

Because the clinical evidence says so?

No, the current Cochrane review concluded that [the] findings are equivocal and, although some of the results of individual studies suggest a positive effect of intercessory prayer, the majority do not and the evidence does not support a recommendation either in favour or against the use of intercessory prayer. We are not convinced that further trials of this intervention should be undertaken and would prefer to see any resources available for such a trial used to investigate other questions in health care.

Yet, not all seem to agree with this; and some even continue to investigate prayer as a medical therpy.

For this new study (published in EBCAM), the Iranian investigators randomly assigned 92 patients in 2 groups to receive either 40 mg of propranolol twice a day for 2 month (group “A”) or 40 mg of propranolol twice a day for 2 months with prayer (group “B”). At the beginning of study and 3 months after intervention, patients’ pain was measured using the visual analogue scale.

All patients who participate in present study were Muslim. At the beginning of study and before intervention, the mean score of pain in patients in groups A and B were 5.7 ± 1.6 and 6.5 ± 1.9, respectively. According to results of independent t test, mean score of pain intensity at the beginning of study were similar between patients in 2 groups (P > .05). Three month after intervention, mean score of pain intensity decreased in patients in both groups. At this time, the mean scores of pain intensity were 5.4 ± 1.1 and 4.2 ± 2.3 in patients in groups A and B, respectively. This difference between groups was statistically significant (P < .001).

figure

The above figure shows the pain score in patients before and after the intervention.

The authors concluded that the present study revealed that prayer can be used as a nonpharmacologic pain coping strategy in addition to pharmacologic intervention for this group of patients.

Extraordinary claims require extraordinary proof. This study is, in fact, extraordinary – but only in the sense of being extraordinarily poor, or at least it is extraordinary in its quality of reporting. For instance, all we learn in the full text article about the two treatments applied to the patient groups is this: “The prayer group participated in an 8-week, weekly, intercessory prayer program with each session lasting 45 minutes. Pain reduction was measured at baseline and after 3 months, by registered nurses who were specialist in pain management and did not know which patients were in which groups (control or intervention), using a visual analogue scale.”

Intercessory prayer is the act of praying on behalf of others. This mans that the patients receiving prayer might have been unaware of being ‘treated’. In this case, the patients could have been adequately blinded. But this is not made clear in the article.

More importantly perhaps, the authors fail to provide any numeric results. All that we are given is the above figure. It is not possible therefore to run any type of check on the data. We are simply asked to believe what the authors have written. I for one have great difficulties in doing so. All I do believe in relation to this article is that

  • the journal EBCAM is utter trash,
  • constantly publishing rubbish is unethical and a disservice to everyone,
  • prayer does not need further research of this nature,
  • and poor studies often generate false-positive findings.

This meta-analysis was performed “to ascertain the effectiveness of oral aloe vera consumption on the reduction of fasting blood glucose (FBG) and hemoglobin A1c (HbA1c).”

PubMed, CINAHL, Natural Medicines Comprehensive Database, and Natural Standard databases were searched. The searches were limited to clinical trials or observational studies conducted in humans and published in English. Studies of aloe vera’s effect on FBG, HbA1c, homeostasis model assessment-estimated insulin resistance (HOMA-IR), fasting serum insulin, fructosamine, and oral glucose tolerance test (OGTT) in prediabetic and diabetic populations were examined.

Nine studies were included in the FBG parameter (n = 283); 5 of these studies included HbA1c data (n = 89). Aloe vera decreased FBG by 46.6 mg/dL (p < 0.0001) and HbA1c by 1.05% (p = 0.004). Significant reductions of both endpoints were maintained in all subgroup analyses. Additionally, the data suggested that patients with an FBG ≥200 mg/dL may see a greater benefit. A mean FBG reduction of 109.9 mg/dL was observed in this population (p ≤ 0.0001). There was evidence of publication bias with FBG but not with HbA1c.

The authors concluded that the results of this meta-analysis support the use of oral aloe vera for significantly reducing both FBG (46.6 mg/dL) and HbA1c (1.05%) in prediabetic and diabetic patients. However, given the current overall quality and relative scarcity of data, further clinical studies that are more robust and better controlled are warranted to confirm and further explore these findings.

Oh no, the results do not support the use of aloe vera at all!!

Why?

Because this ‘meta-analysis’ is of unacceptably poor quality. Here are just some of the flaws that render it totally useless, particularly for issuing advice such as above:

  • The authors included uncontrolled observational studies which make no attempt to control for non-specific effects.
  • In several studies, the use of concomitant anti-diabetic medications was allowed; therefore it is not possible to establish cause and effect by aloe vera.
  • The search strategy was woefully inadequate; for instance non-English publications were not considered.
  • There was no assessment of the scientific rigor of the included studies; this totally invalidates the reliably of the conclusions.
  • The included studies used preparations of widely different aloe vera preparations, and there is no way of knowing the does of the active ingredients.

Diabetes is a serious condition that affects millions worldwide. If some of these patients are sufficiently gullible to follow the conclusions of this paper, they might be dead within a matter of days. This makes this article one of the most dangerous papers that I have seen in the ‘peer-reviewed’ literature of alternative medicine.

Who publishes such utter and irresponsible rubbish?

You may well ask.

The journal has been discussed on this blog  before for the junk that regularly appears in its pages, and so has its editor in chief. The authors (and the reviewers) are not known to me, but one thing is for sure: they don’t know the first thing about conducting a decent systematic review/meta-analysis.

The common cold is one of the indications for which homeopathy is deemed to be effective… by homeopaths that is! Non-homeopaths are understandably critical about this claim, not least because there is no good evidence for it. But, hold on, there is a new study which might change all this.

This study was recently published in COMPLEMENTARY THERAPIES IN MEDICINE which is supposed to be one of the better journals in this area. According to its authors, it was conducted “to determine if a homeopathic syrup was effective in treating cold symptoms in preschool children.” Children diagnosed with an upper respiratory tract infection were randomized to receive a commercial homeopathic cold syrup containing allium cepa 6X, hepar sulf calc 12X, natrum muriaticum 6X, phosphorous 12X, pulsatilla 6X, sulphur 12X, and hydrastasis 6X or placebo. Parents administered the study medication as needed for 3 days. The primary outcome was change in symptoms one hour after each dose. Parents also assessed the severity of each of the symptoms of runny nose, cough, congestion and sneezing at baseline and twice daily for 3 days, using a 4-point rating scale. A composite cold score was calculated by combining the values for each of the four symptoms. Among 261 eligible participants, data on 957 doses of study medication in 154 children were analyzed. There was no significant difference in improvement one hour after the dose for any symptom between the two groups. Analysis of twice daily data on the severity of cold symptoms compared to baseline values found that improvements in sneezing, cough and the composite cold score were significantly greater at both the first and second assessments among those receiving the cold syrup compared to placebo recipients.

The authors concluded that the homeopathic syrup appeared to be effective in reducing the severity of cold symptoms in the first day after beginning treatment.

Where to start? There are so many problems with this study that I find it difficult to chose the most crucial ones:

  • The study had a clearly defined primary endpoint; it was not affected by the homeopathic treatment which doubtlessly makes the study a negative trial. The only correct conclusion therefore is that THE HOMEOPATHIC SYRUP FAILED TO AFFECT THE PRIMARY OUTCOME MEASURE OF THIS STUDY. THEREFORE THE TRIAL DID NOT PRODUCE ANY EVIDENCE TO ASSUME THAT THE EXPERIMENTAL TREATMENT WAS EFFICACIOUS.
  • I don’t think that many of the primary or secondary outcome measures are validated or reliable.
  • All the positive results reported in the abstract and the article relate to secondary endpoints which are purely explanatory by nature. They should, in my view, not be mentioned in the conclusions at all.
  • The fact that some results turned out to be positive can be explained by the fact that the investigators ran dozens of tests for statistical significance which means that, by simple chance, some will turn out to produce a positive result.
  • A further explanation for the seemingly positive results might be the fact disclosed in the text of the article that the children in the homeopathy group received more conventional drugs than those in the placebo group.
  • Whatever the reason for these positive results, they certainly had nothing to do with the homeopathic syrup.
  • The study was funded by the company producing the syrup and for which one of the authors was employed as a consultant. This might be an explanation for the abominably poor science. In other words, this paper is not an exercise in testing a hypothesis but one in marketing.

While I might forgive the company for trying to maximise their sales figures, I do find it harder to forgive the authors, reviewers and editors for publishing such overtly false conclusions. In my view, they are all guilty of scientific misconduct.

This randomized, double-blind study evaluated the efficacy of a homeopathic treatment in preventing excessive weight gain during pregnancy in overweight or obese women who were suspected of having a common mental disorder. For the homeopathic group (n=62), 9 homeopathic remedies were pre-selected: (1) Pulsatilla nigricans, (2) Sepia succus, (3) Lycopodium clavatum, (4) sulphur, (5) Lachesis trigonocephalus, (6) Nux vomica, (7) Calcarea carbonica, (8) phosphorus; and (9) Conium maculatum. From those 9 drugs, one was prioritized for administration for each participant. After the first appointment, a re-selection or selection of a new, more appropriate drug occurred, using the list of preselected drugs. The dosage was 6 drops orally 2 ×/day, in the morning and at night, on 4 consecutive days each week, with an interval of 3 d between doses, up until the next appointment medical appointment. The control group (n=72) took placebos. Both groups also received a diet orientation.

Weight change during pregnancy was defined as the difference between the body mass index (BMI) at the initial evaluation and that recorded at the final evaluation, adjusted for 40 weeks of gestation. In addition, the APGAR index in the newborn  (a measure of the health of the baby) was evaluated. The mean variation between baseline BMI and BMI at week 40 of gestation was +4.95 kg/m2 in the control group and +5.05 kg/m2 in the homeopathy group. The difference between the two groups was not significant. APGAR 10 at 5 min (59.6% in the homeopathy group and 36.4% in the control group) was statistically significant (P = .016).

The authors concluded that homeopathy does not appear to prevent excessive body mass gain in pregnant women who are overweight or obese and suspected of having a common mental disorder. Homeopathy did not change the APGAR score to modified clinical attention at delivery room. However, the evidence observed at APGAR 10 at minute 5 suggests that  homeopathy had a modulating effect on the vitality of newborns, warranting further studies designed to investigate it.

I have seen many odd studies in my time, but this must be one of the oddest?

  • What is the rationale for assuming that homeopathy might affect body weight?
  • Why take pregnant women with a weight problem who were suspected of having a common mental disorder?
  • Why try to turn a clearly negative result into a finding that is (at least partly) positive?

The last point seems the most important one to me. The primary outcome measure of this study (weight gain) was clearly defined and was not affected by the therapy. Yet the authors feel it justified to add to their conclusions that homeopathy had a modulating effect on the vitality of newborns (almost certainly nothing but a chance finding).

Are they for real?

I suppose they are: they are real pseudo-scientific promoters of quackery!

Dietary and herbal supplements (DHS) are currently popular. They are being promoted as being natural and therefore safe – an assumption that is clearly wrong: some DHS can contain toxic substances or they might cause interactions with drugs or other DHS.

This study explored whether adverse events were actually associated with such interactions and examined specific characteristics among inpatient DHS users prone to such adverse events. It was designed as a cross-sectional survey of 947 patients hospitalized in 12 departments of a tertiary academic medical centre in Haifa, Israel. It evaluated the rate of DHS use among inpatients, the potential for interactions, and actual adverse events during hospitalization associated with DHS use. It also assessed whether DHS consumption was documented in patients’ medical files. Statistical analysis was used to delineate DHS users at risk for adverse events associated with interactions with conventional drugs or other DHS.

The results show that about half of all patients took DHS. In 17 (3.7%) of the 458 DHS users, an adverse event may have been caused by DHS-drug-DHS interactions. According to the Drug Interaction Probability Scale, 14 interactions “probably” caused the adverse events, and 11 “possibly” caused them. Interactions occurred more frequently in older patients (p = 0.025, 95% CI: 2.26-19.7), patients born outside Israel (p = 0.025, 95% CI: 0.03-0.42), those with ophthalmologic (p = 0.032, 95% CI: 0.02-0.37) or gastrointestinal (p = 0.008, 95% CI: 0.05-0.46) comorbidities, and those using higher numbers of DHS (p < 0.0001, 95% CI: 0.52-2.48) or drugs (p = 0.027, 95% CI: 0.23-3.77).

The authors concluded that approximately one in 55 hospitalizations in this study may have been caused by adverse events associated with DHS-drug-DHS interactions. To minimize the actual occurrence of adverse events, medical staff education regarding DHS should be improved.

This seems to be a good study and it generated interesting findings on an important topic.

Why do I have nevertheless a problem with it?

The answer is simple but not pleasant: very similar results have been published almost simultaneously in more than one journal. The link above is to an article in the BR J CLIN PHARMACOL of October this year. The following text is from the abstract of an article in INTERN EMERG MED also of October this year:

Of 927 patients who agreed to answer the questionnaire, 458 (49.4 %) reported the use of 89 different DHS. Potential DHS-DHS interactions were identified in 12.9 % of DHS users. Three interactions were associated with the actual occurrence of adverse events. Patients at risk of DHS-DHS interactions included females (p = 0.026) and patients with greater numbers of concomitant medications (p < 0.0001) and of consumed DHS (p < 0.0001). In 88.9 % of DHS users, DHS use was not reported in medical files and only 18 % of the DHS involved in interactions were documented. Potential DHS-DHS interactions are common in inpatients, and may lead to hospitalization or worsen existing medical conditions. The causal relationship between potential interactions and actual adverse events requires further study.

END OF QUOTE

And to my surprise, I also found a third article also from the October issue of INTERN EMERG MED reporting on this survey. Here is part of its abstract:

DHS users were determined via a questionnaire. The Natural Medicine database was used to search for potential DHS-drug interactions for identified DHS, and the clinical significance was evaluated using Lexi-interact online interaction analysis. Medical files were assessed for documentation of DHS use. Univariate and multivariate logistic regression analyses were used to characterize potential risk factors for DHS-drug interactions. Of 927 patients consenting to answer the questionnaire, 458 (49 %) reported DHS use. Of these, 215 (47 %) had at least one potential interaction during hospitalization (759 interactions). Of these interactions, 116 (15 %) were potentially clinically significant. Older age [OR = 1.02 (1.01-1.04), p = 0.002], males [OR = 2.11 (1.35-3.29), p = 0.001] and increased number of used DHS [OR = 4.28 (2.28-8.03), p < 0.001] or drugs [OR = 1.95 (1.17-3.26), p = 0.011] were associated with potential interactions in DHS users. Physicians documented only 16.5 % of DHS involved in these interactions in patients’ medical files. In conclusion, a substantial number of inpatients use DHS with potential interactions with concomitant medications. Medical staff should be aware of this, question patients on DHS usage and check for such interactions.

END OF QUOTE

What is the difference between the three articles? The first one in INTERN EMERG MED authored by Levy I, Attias S, Ben Arye E, Goldstein L, Schiff E evaluated “potential DHS-DHS interactions among inpatients”. The second one in INTERN EMERG MED also authored by Levy I, Attias S, Ben Arye E, Goldstein L, Schiff E evaluated “potentially dangerous interactions of DHS with prescribed medications among inpatients”. Finally the one in BR J CLIN PHARMACOL also authored by Levy I, Attias S, Ben-Arye E, Goldstein L, Schiff E  assessed in addition the interactions between DHS and prescription drugs.

Dual publications are usually considered to be a violation of research ethics. Publication of different aspects of one single data-set in multiple articles is called ‘salami-slicing’ and is often considered to be poor form.

My question to you, the reader of this post, is: What type of scientific misconduct do we have here?

On 25 and 26 May of this year I wrote two posts about an acupuncture trial that, in my view, was dodgy. To refresh your memory, here is the relevant part of the 2nd post:

This new study was designed as a randomized, sham-controlled trial of acupuncture for persistent allergic rhinitis in adults investigated possible modulation of mucosal immune responses. A total of 151 individuals were randomized into real and sham acupuncture groups (who received twice-weekly treatments for 8 weeks) and a no acupuncture group. Various cytokines, neurotrophins, proinflammatory neuropeptides, and immunoglobulins were measured in saliva or plasma from baseline to 4-week follow-up.

Statistically significant reduction in allergen specific IgE for house dust mite was seen only in the real acupuncture group. A mean (SE) statistically significant down-regulation was also seen in pro-inflammatory neuropeptide substance P (SP) 18 to 24 hours after the first treatment. No significant changes were seen in the other neuropeptides, neurotrophins, or cytokines tested. Nasal obstruction, nasal itch, sneezing, runny nose, eye itch, and unrefreshed sleep improved significantly in the real acupuncture group (post-nasal drip and sinus pain did not) and continued to improve up to 4-week follow-up.

The authors concluded that acupuncture modulated mucosal immune response in the upper airway in adults with persistent allergic rhinitis. This modulation appears to be associated with down-regulation of allergen specific IgE for house dust mite, which this study is the first to report. Improvements in nasal itch, eye itch, and sneezing after acupuncture are suggestive of down-regulation of transient receptor potential vanilloid 1.

…the trial itself raises a number of questions:

  1. Which was the primary outcome measure of this trial?
  2. What was the power of the study, and how was it calculated?
  3. For which outcome measures was the power calculated?
  4. How were the subjective endpoints quantified?
  5. Were validated instruments used for the subjective endpoints?
  6. What type of sham was used?
  7. Are the reported results the findings of comparisons between verum and sham, or verum and no acupuncture, or intra-group changes in the verum group?
  8. What other treatments did each group of patients receive?
  9. Does anyone really think that this trial shows that “acupuncture is a safe, effective and cost-effective treatment for allergic rhinitis”?

In the comments section, the author wrote: “after you have read the full text and answered most of your questions for yourself, it might then be a more appropriate time to engage in any meaningful discussion, if that is in fact your intent”, and I asked him to send me his paper. As he does not seem to have the intention to do so, I will answer the questions myself and encourage everyone to have a close look at the full paper [which I can supply on request].

  1. The myriad of lab tests were defined as primary outcome measures.
  2. Two sentences are offered, but they do not allow me to reconstruct how this was done.
  3. No details are provided.
  4. Most were quantified with a 3 point scale.
  5. Mostly not.
  6. Needle insertion at non-acupoints.
  7. The results are a mixture of inter- and intra-group differences.
  8. Patients were allowed to use conventional treatments and the frequency of this use was reported in patient diaries.
  9. I don’t think so.

So, here is my interpretation of this study:

  • It lacked power for many outcome measures, certainly the clinical ones.
  • There were hardly any differences between the real and the sham acupuncture group.
  • Most of the relevant results were based on intra-group changes, rather than comparing sham with real acupuncture, a fact, which is obfuscated in the abstract.
  • In a controlled trial fluctuations within one group must never be interpreted as caused by the treatment.
  • There were dozens of tests for statistical significance, and there seems to be no correction for multiple testing.
  • Thus the few significant results that emerged when comparing sham with real acupuncture might easily be false positives.
  • Patient-blinding seems questionable.
  • McDonald as the only therapist of the study might be suspected to have influenced his patients through verbal and non-verbal communications.

I am sure there are many more flaws, particularly in the stats, and I leave it to others to identify them. The ones I found are, however, already serious enough, in my view, to call for a withdrawal of this paper. Essentially, the authors seem to have presented a study with largely negative findings as a trial with positive results showing that acupuncture is an effective therapy for allergic rhinitis. Subsequently, McDonald went on social media to inflate his findings even more. One might easily ask: is this scientific misconduct or just poor science?

END OF QUOTE

This and the previous post created lots of discussion and comments. However, the question whether the study in question amounted to scientific misconduct was never satisfactorily resolved. Therefore, I decided to write to the editor of ‘Ann Allergy Asthma Immunol‘ where the trial had been published. He answered by saying I would need to file an official complaint for him to address the issue. On 13 June, I therefore sent him the following email:

Thank you for your letter of 3/6/2016 suggesting I make a formal complaint about the paper entitled ‘EFFECT OF ACUPUNCTURE ON HOUSE DUST MITE…’ [ Ann Allergy Asthma Immunol 2016] by McDonald et al. I herewith wish to file such a complaint.

The article in question reports an RCT of acupuncture for persistent allergic rhinitis. It followed a parallel group design with 3 groups receiving the following interventions:

1.       Acupuncture

2.       Sham-acupuncture

3.       No treatment

There was a plethora of outcome measures and time points on which they were measured. A broad range of parameters was defined as primary endpoints.

The conclusion reached by the authors essentially was that acupuncture affected several outcome measures in a positive sense, thus supporting the notion that acupuncture is efficacious [“Symptoms and quality of life improved significantly and were still continuing to improve 4 weeks after treatment ceased.”] This conclusion, however, is misleading and needs correcting.

The main reasons for this are as follows:

·         Despite the fact that the authors did many dozens of statistical tests for significance, they did not correct for this multiplicity of tests. Consequently, some or most of the significant results are likely to be false positive.

·         Many of the positive results of this paper were not obtained by comparing one group to another but by doing before/after comparisons within one group. This approach defies the principle of a controlled clinical trial. For doing intra-group comparisons, we obviously do not need any control group at all. The findings from intra-group comparisons are prominently reported in the paper, for instance in the abstract, giving the impression that they originate from inter-group comparisons. One has to read the paper very carefully to find that, when inter-group comparisons were conducted, their results did NOT confirm the findings from the reported intra-group comparisons. As this is the case for most of the symptomatic endpoints, the impression given is seriously misleading and needs urgent correction.

On the whole, the article is a masterpiece of obfuscation and misrepresentation of the actual data. I urge you to consider the harm than can be done by such a misleading publication. In my view, the best way to address this problem is to withdraw the article.

I look forward to your decision.

Regards

E Ernst

END OF QUOTE

I had to send several reminders but my most recent one prompted the following response dated 7/11/ 2016:

Dear Professor Ernst,
Thank you for your patience while we worked through the process of considering your complaints levied on the article entitled ‘EFFECT OF ACUPUNCTURE ON HOUSE DUST MITE…’ [ Ann Allergy Asthma Immunol 2016] by McDonald et al. I considered the points that you made in your previous letter, sought input from our editorial team (including our biostatistics editor) , our publisher and the authors themselves. I sent the the  charges ( point by point) anonymously to the authors and allowed them time to respond which they did. I had their responses reviewed by selected editors and , as a result of this process, have decided not to pursue any corrective or punitive action based upon the following :
 
  1. Our editorial team recognizes that this is not the best clinical trial we have published in the Annals of Allergy, Asthma and Immunology. However, neither is is the worst. As in most published research studies, there are always things that could have been done better to make it a stronger paper. Never-the-less, the criticism falls fall short of any sort of remedy that would include withdrawal of the manuscript.
  2. Regarding your accusation that the multiple positive endpoint resulted in the authors making specific therapeutic claims, our assessment is that no specific therapeutic claim was made but rather the authors maintained that the data support the value of acupuncture in improving symptoms and quality of life in patients with AR. We do not believe there was overreach in those statements.
  3. The authors’ stated intent was to show immune changes associated with clinical markers of improvement in the active acupuncture group compared to controls. The authors maintain (and our editors agree) that their data assessments were primarily based upon three statistical tests not “dozens” (as stated in your original letter of complaint).  The power analysis and sample size calculations were presented to us and deemed adequate , making the probability of a type I error quite low.
  4. The authors acknowledge in their paper that there could be limitations to their data interpretation based upon potential disparities between intra- and intergroup comparisons. The editors felt their transparency was adequately disclosed.
In summary, as editor-in-chief of the Annals of Allergy, Asthma and Immunology, I did not find sufficient merit in your charges to initiate any corrective or punitive action for this manuscript. I understand you will strongly disagree with this decision and I regret that. However, in the final analysis, my primary intent is to preserve the objectivity, fairness, and integrity of our journal and its review process. I believe I have accomplished that in this instance.
 
Sincerely
END OF QUOTE
This seems to settle the issue: the study in question does not involve scientific misconduct!
Or does it?
I would be grateful for the view of the readers of this post.

I have warned you before to be sceptical about Chinese studies. This is what I posted on this blog more than 2 years ago, for instance:

Imagine an area of therapeutics where 100% of all findings of hypothesis-testing research are positive, i.e. come to the conclusion that the treatment in question is effective. Theoretically, this could mean that the therapy is a miracle cure which is useful for every single condition in every single setting. But sadly, there are no miracle cures. Therefore something must be badly and worryingly amiss with the research in an area that generates 100% positive results.

Acupuncture is such an area; we and others have shown that Chinese trials of acupuncture hardly ever produce a negative finding. In other words, one does not need to read the paper, one already knows that it is positive – even more extreme: one does not need to conduct the study, one already knows the result before the research has started. But you might not believe my research nor that of others. We might be chauvinist bastards who want to discredit Chinese science. In this case, you might perhaps believe Chinese researchers.

In this systematic review, all randomized controlled trials (RCTs) of acupuncture published in Chinese journals were identified by a team of Chinese scientists. A total of 840 RCTs were found, including 727 RCTs comparing acupuncture with conventional treatment, 51 RCTs with no treatment controls, and 62 RCTs with sham-acupuncture controls. Among theses 840 RCTs, 838 studies (99.8%) reported positive results from primary outcomes and two trials (0.2%) reported negative results. The percentages of RCTs concealment of the information on withdraws or sample size calculations were 43.7%, 5.9%, 4.9%, 9.9%, and 1.7% respectively.

The authors concluded that publication bias might be major issue in RCTs on acupuncture published in Chinese journals reported, which is related to high risk of bias. We suggest that all trials should be prospectively registered in international trial registry in future.

END OF QUOTE

Now an even more compelling reason emerged for taking evidence from China with a pinch of salt:

A recent survey of clinical trials in China has revealed fraudulent practice on a massive scale. China’s food and drug regulator carried out a one-year review of clinical trials. They concluded that more than 80 percent of clinical data is “fabricated“. The review evaluated data from 1,622 clinical trial programs of new pharmaceutical drugs awaiting regulator approval for mass production. Officials are now warning that further evidence malpractice could still emerge in the scandal.
According to the report, much of the data gathered in clinical trials are incomplete, failed to meet analysis requirements or were untraceable. Some companies were suspected of deliberately hiding or deleting records of adverse effects, and tampering with data that did not meet expectations.

“Clinical data fabrication was an open secret even before the inspection,” the paper quoted an unnamed hospital chief as saying. Contract research organizations seem have become “accomplices in data fabrication due to cutthroat competition and economic motivation.”

A doctor at a top hospital in the northern city of Xian said the problem doesn’t lie with insufficient regulations governing clinical trials data, but with the failure to implement them. “There are national standards for clinical trials in the development of Western pharmaceuticals,” he said. “Clinical trials must be carried out in three phases, and they must be assessed at the very least for safety,” he said. “But I don’t know what happened here.”

Public safety problems in China aren’t limited to the pharmaceutical industry and the figure of 80 percent is unlikely to surprise many in a country where citizens routinely engage in the bulk-buying of overseas-made goods like infant formula powder. Guangdong-based rights activist Mai Ke said there is an all-pervasive culture of fakery across all products made in the country. “It’s not just the medicines,” Mai said. “In China, everything is fake, and if there’s a profit in pharmaceuticals, then someone’s going to fake them too.” He said the problem also extends to traditional Chinese medicines, which are widely used in conjunction with Western pharmaceuticals across the healthcare system.
“It’s just harder to regulate the fakes with traditional medicines than it is with Western pharmaceuticals, which have strict manufacturing guidelines,” he said.

According to Luo, academic ethics is an underdeveloped field in China, leading to an academic culture that is accepting of manipulation of data. “I don’t think that the 80 percent figure is overstated,” Luo said.

And what should we conclude from all this?

I find it very difficult to reach a verdict that does not sound hopelessly chauvinistic but feel that we have little choice but to distrust the evidence that originates from China. At the very minimum, I think, we must scrutinise it thoroughly; whenever it looks too good to be true, we ought to discard it as unreliable and await independent replications.

Gravityscan Badge

Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted.


Click here for a comprehensive list of recent comments.

Categories