Since 1997, several meta-analyses (MAs) of placebo-controlled randomised efficacy trials of homoeopathy for any indication (PRETHAIs) have been published with different methods, results and conclusions. To date, a formal assessment of these MAs has not been performed. The main objective of this systematic review of MAs of PRETHAIs was to evaluate the efficacy of homoeopathic treatment.
The inclusion criteria were as follows: MAs of PRETHAIs in humans; all ages, countries, settings, publication languages; and MAs published from 1 Jan. 1990 to 30 Apr. 2023. The exclusion criteria were as follows: systematic reviews without MAs; MAs restricted to age or gender groups, specific indications, or specific homoeopathic treatments; and MAs that did not assess efficacy. We searched 8 electronic databases up to 14 Dec. 2020, with an update search in 6 databases up to 30 April 2023.
The primary outcome was the effect estimate for all included trials in each MA and after restricting the sample to trials with high methodological quality, according to predefined criteria. The risk of bias for each MA was assessed by the ROBIS (Risk Of Bias In Systematic reviews) tool. The quality of evidence was assessed by the GRADE framework. Statistical analyses were performed to determine the proportion of MAs showing a significant positive effect of homoeopathy vs. no significant difference.
Six MAs were included, covering individualised homoeopathy (I-HOM, n = 2), nonindividualised homoeopathy (NI-HOM, n = 1) and all homoeopathy types (ALL-HOM = I-HOM + NI-HOM, n = 3). The MAs comprised between 16 and 110 trials, and the included trials were published from 1943–2014. The median trial sample size ranged from 45 to 97 patients. The risk of bias (low/unclear/high) was rated as low for three MAs and high for three MAs.
Effect estimates for all trials in each MA showed a significant positive effect of homoeopathy compared to placebo (5 of 5 MAs, no data in 1 MA). Sensitivity analyses with sample restriction to high-quality trials were available from 4 MAs; the effect remained significant in 3 of the MAs (2 MAs assessed ALL-HOM, 1 MA assessed I-HOM) and was no longer significant in 1 MA (which assessed NI-HOM).
The authors concluded that the quality of evidence for positive effects of homoeopathy beyond placebo (high/moderate/low/very low) was high for I-HOM and moderate for ALL-HOM and NI-HOM. There was no support for the alternative hypothesis of no outcome difference between homoeopathy and placebo. The available MAs of PRETHAIs reveal significant positive effects of homoeopathy beyond placebo. This is in accordance with laboratory experiments showing partially replicable effects of homoeopathically potentised preparations in physico-chemical, in vitro, plant-based and animal-based test systems.
Reading this SR, I got the impression that it was designed to generate a positive result. The 6 included MAs are marginally positive (mainly due to publication bias and other artefacts) and thus very well known to fans of homeopathy. The authors of this paper must therefore have expected that combining them in a review would generate an overall positive finding.
The first question I asked myself while studying this paper was: who would want to do an SR of MAs (a most peculiar exercise); why not at least an SR of SRs which is already an unusual project but would make at least some sense. (An SR is a review that includes all studies that match a set of pre-definied criteria. A MA is a special form of SR where statistical pooling was possible.) The answer is, I fear, simple: this would not have generated a positive result: here are now dozens of SRs of homeopathy and most are not positive (as discussed regularly on this blog)
The authors themselves provide no real justification for their bizarre approach. All they tell us about it is this:
One approach is to focus on a specifc indication (e.g., depression , acute respiratory tract infections in children ) while often including open-label trials and observational studies. In this approach, data synthesis is grouped by design, thus yielding information about homoeopathy in patient care. Te opposite approach is to include all indications while restricting study designs to placebo-controlled trials and aggregating results in an MAs, thus yielding information about the specifc efects of homoeopathy beyond those of placebo. A major reason for using this approach has been the claim that ‘homoeopathy violates natural laws and thus any efect must be a placebo efect’ .
Since 1997, at least six MAs of placebo-controlled homoeopathy trials for any condition have been published [6–11]. These MAs have difered in their methods for trial inclusion, data synthesis and assessment of risk of bias; furthermore, their results and conclusions have been inconsistent. During this period, there have been substantial advancements in methodology and quality standards for MAs and other SRs [12–15], including SRs of SRs (also called overviews or umbrella reviews) [16–18]. To our knowledge, a formal SR of MAs of randomised placebo-controlled homoeopathy trials for any condition has not been performed. Herein, we report such an SR.
What the authors actually did is this: they knew of the 6 MA; they also knew that they arrived at cautiously positive conclusions; finally they also were aware of the fact that, obviously, the 6 MA included more or less almost the same primary studies. So, by pooling the MAs, they generated a positive result which was no longer marginally positive but strongly. Anyone looking through this strategy (which in effect multiplies the results of many primary studies by factor 6) must realize that this methods creates a false-positive impression.
My suspicion that this paper is a deliberate attempt at misleading us about the ‘efficacy’ (actually it should be effectiveness) is strengthened by further facts:
- One of the two MAs by Mathie et al excluded primary studies that reported positive findings (i.e. mine and the one by Walach et al)
- Funding: Open Access funding enabled and organized by Projekt DEAL. Funding specifcally for this SR was provided by Christophorus-Stiftung (No. 393 CST), Stiftung Marion Meyenburg (Date 24.09.2020), Dr. Hauschka Stiftung (Date 16.11.2020) and Gesellschaft für Pluralität im Gesundheitswesen (Dates 11.06.2021, 22.06.2021). General funding for IFAEMM was provided by the Software-AG Stiftung (SE-P 13544). The funders had no infuence on the writing of the protocol or on the planning, conduct and publication of this SR.
- Competing interests: In the past 3 years, HJH has received research grants from two manufacturers of anthroposophic medicinal products (Wala Heilmittel GmbH, Bad Boll/ Eckwälden, Germany; Weleda AG, Arlesheim Switzerland). Anthroposophic medicine is not based on the homoeopathic simile principle or on drug provings, but some anthroposophic medicinal products are potentized. The two manufacturers had no involvement with the present SR. Anthroposophic medicinal products were not part of the intervention in any of the trials evaluated in the MAs of this SR (Suppl. Table 15). DSR has received a development grant from Heel GmbH (manufacturer of homoeopathic products) for online training in case report writing. AG, KvA and HK declare that they have no competing interests.
- Author details: 1) Institute for Applied Epistemology and Medical Methodology at Witten/ Herdecke University (IFAEMM), Freiburg, Germany. 2) Faculty of Health, Department of Medicine, Chair of Medical Theory, Integrative and Anthroposophic Medicine, Witten/Herdecke University, Witten, Germany. 3) Maryland University of Integrative Health (MUIH), Laurel, MD, USA. 4) Homeopathic Pharmacopoeia Convention of the United States (HPCUS), Southeastern, PA, USA.
Personally, I do not find it surprising that these authors bend over backwards to publish something positive about homeopathy (such things happen in homeopathy all the time). However, I do find it astonishing that an allegedly decent journal passes such pseudoscience for publication as though it is serious science.