As promised, I would like to correct the errors in my previous assessment of this paper. To remind everyone:
This systematic review evaluated individualized homeopathy as a treatment for children with attention deficit and hyperactivity disorder (ADHD) when compared to placebo or usual care alone.
Thirty-seven online sources were searched up to March 2021. Studies investigating the effects of individualized homeopathy against any control in ADHD were eligible. Data were extracted to a predefined excel sheet independently by two reviewers.
Six studies were analyzed:
- 5 were RCTs
- 2 were controlled against standard treatments;
- 4 were placebo-controlled and double-blinded.
The meta-analysis revealed a significant effect size across studies of Hedges’ g = 0.542 (95% CI 0.311-0.772; z = 4,61; p < 0.001) against any control and of g = 0.605 (95% CI 0.05-1.16; z = 2.16, p = 0.03) against placebo. The effect estimations are based on studies with an average sample size of 52 participants.
The authors concluded that individualized homeopathy showed a clinically relevant and statistically robust effect in the treatment of ADHD.
Now that I was able to access the full papers, I would like to offer a thorough analysis.
To get included in the review, primary studies had to be:
- Published after 1980,
- Investigating an individualized homeopathic intervention in childhood ADHD,
- Comparing the intervention to a control condition (placebo, standard care or treatment as usual, both of which are referred to as “active control”) in a randomized or non-randomized parallel-group study
design with one or more arms.
Six studies were included:
- Fibert, P., Peasgood, T. & Relton, C. Rethinking ADHD intervention trials: feasibility testing of two treatments and a methodology. Eur. J. Pediatr. 178, 983–993 (2019). – DOI
- Fibert, P., Relton, C., Heirs, M. & Bowden, D. A comparative consecutive case series of 20 children with a diagnosis of ADHD receiving homeopathic treatment, compared with 10 children receiving usual care. Homeopathy 105, 194–201 (2016). – DOI
- Jacobs, J., Williams, A. L., Girard, C., Njike, V. Y. & Katz, D. Homeopathy for attention-deficit/hyperactivity disorder: a pilot randomized-controlled trial. J. Altern. Complement. Med. 11, 799–806 (2005). – DOI
- Jones, M. The efficacy of homoeopathic simillimum in the treatment of attention-deficit/hyperactivity disorder (AD/HD) in schoolgoing children aged 6-11 years. https://openscholar.dut.ac.za/bitstream/10321/534/1/Jones_2009.pdf (2009).
- Frei, H. et al. Homeopathic treatment of children with attention deficit hyperactivity disorder: a randomised, double blind, placebo controlled crossover trial. Eur. J. Pediatr. 164, 758–767 (2005). – DOI
- Oberai, P. et al. Homoeopathic management of attention deficit hyperactivity disorder: a randomised placebo-controlled pilot trial. Indian J. Res. Homoeopathy 7, 158–167 (2013).
Exclusion criteria were:
- Homeopathic intervention not individualized,
- Serious methodological flaws, such as incidental unblinding, failure to report important data, or insufficient data for meta-analysis.
One study was excluded:
- Lamont, J. Homoeopathic treatment of attention deficit hyperactivity disorder. Br. Homeopathic J. 86, 196–200 (1997). – DOI
I will first make several points about Walach’s systematic review itself and then have a look at the primary studies that it included. Finally, I will try to draw some conclusions.
The review authors state in their introduction that “beneficial effects of this intervention [homeopathy] have been shown for various kinds of medical conditions, including child diarrhea, supportive care in cancer, fibromyalgia, or ADHD.” In other words, already in the introduction, they disclose their strong pro-homeopathy bias; it would, of course, not be difficult to find investigations that contradict their optimism.
Despite the stated inclusion/exclusion criteria, the authors did include the Frei-study that did not follow a parallel-group design (see also below).
The authors included two active-controlled studies both of which did not report the type of treatment received by the control group. In other words, these trials failed to report important data which was a stated exclusion criterium (see below).
In their discussion section, the authors state that “all included studies employed individualized homeopathy and were of comparable, solid quality, hence a lack of methodological rigor is unlikely the reason for the difference between homeopathy and controls…” This, I think, is grossly misleading; even according to the authors’ own assessments, one study was deemed to have a high risk of bias and in two studies the risk of bias was “unclear”.
The overall positive effect of homeopathy demonstrated by the review was determined almost exclusively by the study of Oberai et al (p-value = 0.000). In fact, the studies by Jones and by Jacobs were negative, and the one by Frei was borderline positive with a p-value of 0.46. The authors address this crucial issue repeatedly and claim that excluding Oberai et al would still generate an overall positive meta-analytic result. Yet, they do not mention that the overall result would no longer be clinically relevant.
Looking at the included primary studies, I should make the following points:
- The two Filbert studies, as mentioned, failed to report important data and should, according to the stated exclusion criteria, not have been included.
- The study by Jacobs was a pilot study and generated negative findings.
- The study by Jones is a non-peer-reviewed thesis. In my view, it should never have been included.
- The study by Frei was a cross-over trial. According to the exclusion/inclusion criteria of the authors, it should not have been included.
- The study by Oberai et al is the trial that has by far the largest effect size and thus is the driver of the overall result of the review. It is therefore important to have a closer look at it.
Here is the abstract:
Objective: To evaluate the usefulness of individualised homoeopathic medicines in treatment of Attention Deficit Hyperactivity Disorder (ADHD).
Design: Randomised placebo-controlled single-blind pilot trial.
Setting: Central Research Institute (Homoeopathy), Kottayam, Kerala, India from June 2009 to November 2011.
Participants: Children aged 6-15 years meeting the Diagnostic Statistical Manual of mental disorders (DSM-IV) criteria for ADHD.
Interventions: A total of 61 patients (Homoeopathy = 30, placebo = 31) were randomised to receive either individualised homoeopathic medicine in fifty millesimal (LM) potency or placebo for a period of one year.
Outcome measures: Conner’s Parent Rating Scale-Revised: Short (CPRS-R (S)), Clinical Global Impression-Severity Scale (CGI-SS), Clinical Global Impression- Improvement Scale (CGI-IS) and Academic performance.
Results: A total of 54 patients (homoeopathy = 27, placebo = 27) were analysed under modified intention to treat (ITT). All patients in homoeopathy group showed better outcome in baseline adjusted General Linear Model (GLM) repeated measures ANCOVA for oppositional, cognition problems, hyperactivity and ADHD Index (domains of CPRS-R (S)) and CGI-IS at T3, T6, T9 and T12 (P = 0.0001). The mean baseline-adjusted treatment difference between groups at month 12 from baseline for all individual outcome measures favoured homoeopathy group; Oppositional (−16.4, 95% CI – 20.5 to − 12.2, P = 0.0001), Cognition problems (−15.5, 95% CI − 19.2 to − 11.8, P = 0.0001), Hyperactivity (−20.6, 95% CI − 25.6 to − 15.4, P = 0.0001), ADHD I (−15.6, 95% CI − 19.5 to − 11.6, P = 0.0001), Academic performance 14.4%, 95% CI 8.3 to 20.5, P = 0.0001), CGISS (−1.6, 95% CI − 1.9 to − 1.2, P = 0.0001), CGIIS (−1.6, 95% CI − 2.3 to -0.9, P = 0.0001).
Conclusion: This pilot study provides evidence to support the therapeutic effects of individualised homoeopathic medicines in ADHD children. However, the results need to be validated in multi-center randomised double-blind placebo-controlled clinical trial.
Here are a few points of concern related to the Oberai et al:
- The trial was a mere pilot study.
- Despite the fact that it is now 9 years old, the authors never published a definitive trial.
- The study was published in an obscure journal that is not Medline-listed.
- The study is very poorly reported.
- It is unclear how the diagnosis of ADHD for including the patients was verified.
- The control patients were treated for one year with a placebo and no other therapies. In my view, this is not ethical.
- The method of randomization is unclear.
- The authors state that acute symptoms were treated throughout the study period with homeopathy, even in the control group. This seems odd and defies the principle of a placebo-controlled trial.
- The authors state that only the patients were blind, not the investigators. This opens the door wide for all sorts of biases. It is, for example, likely that it also de-blinded the patients (the verum could be adjusted and changed, while the placebo remained constant).
All in all, this paper is of poor quality, Its findings are far from trustworthy and were not meant to be definitive. According to the following exclusion criteria, it should have been excluded:
- It had several serious methodological flaws.
- It did not blind the investigators.
- It is likely that patients were de-blinded.
- It failed to report important data.
So, why did Walach and his co-authors include it?
Could it be because, without the Oberai-study, the overall findings of the review would at best have turned out to be borderline significant and not clinically relevant?