Since 1997, several meta-analyses (MAs) of placebo-controlled randomised efficacy trials of homoeopathy for any indication (PRETHAIs) have been published with different methods, results and conclusions. To date, a formal assessment of these MAs has not been performed. The main objective of this systematic review of MAs of PRETHAIs was to evaluate the efficacy of homoeopathic treatment.

The inclusion criteria were as follows: MAs of PRETHAIs in humans; all ages, countries, settings, publication languages; and MAs published from 1 Jan. 1990 to 30 Apr. 2023. The exclusion criteria were as follows: systematic reviews without MAs; MAs restricted to age or gender groups, specific indications, or specific homoeopathic treatments; and MAs that did not assess efficacy. We searched 8 electronic databases up to 14 Dec. 2020, with an update search in 6 databases up to 30 April 2023.

The primary outcome was the effect estimate for all included trials in each MA and after restricting the sample to trials with high methodological quality, according to predefined criteria. The risk of bias for each MA was assessed by the ROBIS (Risk Of Bias In Systematic reviews) tool. The quality of evidence was assessed by the GRADE framework. Statistical analyses were performed to determine the proportion of MAs showing a significant positive effect of homoeopathy vs. no significant difference.

Six MAs were included, covering individualised homoeopathy (I-HOM, n = 2), nonindividualised homoeopathy (NI-HOM, n = 1) and all homoeopathy types (ALL-HOM = I-HOM + NI-HOM, n = 3). The MAs comprised between 16 and 110 trials, and the included trials were published from 1943–2014. The median trial sample size ranged from 45 to 97 patients. The risk of bias (low/unclear/high) was rated as low for three MAs and high for three MAs.

Effect estimates for all trials in each MA showed a significant positive effect of homoeopathy compared to placebo (5 of 5 MAs, no data in 1 MA). Sensitivity analyses with sample restriction to high-quality trials were available from 4 MAs; the effect remained significant in 3 of the MAs (2 MAs assessed ALL-HOM, 1 MA assessed I-HOM) and was no longer significant in 1 MA (which assessed NI-HOM).

The authors concluded that the quality of evidence for positive effects of homoeopathy beyond placebo (high/moderate/low/very low) was high for I-HOM and moderate for ALL-HOM and NI-HOM. There was no support for the alternative hypothesis of no outcome difference between homoeopathy and placebo. The available MAs of PRETHAIs reveal significant positive effects of homoeopathy beyond placebo. This is in accordance with laboratory experiments showing partially replicable effects of homoeopathically potentised preparations in physico-chemical, in vitro, plant-based and animal-based test systems.

Reading this SR, I got the impression that it was designed to generate a positive result. The 6 included MAs are marginally positive (mainly due to publication bias and other artefacts) and thus very well known to fans of homeopathy. The authors of this paper must therefore have expected that combining them in a review would generate an overall positive finding.

The first question I asked myself while studying this paper was: who would want to do an SR of MAs (a most peculiar exercise); why not at least an SR of SRs which is already an unusual project but would make at least some sense. (An SR is a review that includes all studies that match a set of pre-definied criteria. A MA is a special form of SR where statistical pooling was possible.) The answer is, I fear, simple: this would not have generated a positive result: here are now dozens of SRs of homeopathy and most are not positive (as discussed regularly on this blog)

The authors themselves provide no real justification for their bizarre approach. All they tell us about it is this:

One approach is to focus on a specifc indication (e.g., depression [4], acute respiratory tract infections in children [5]) while often including open-label trials and observational studies. In this approach, data synthesis is grouped by design, thus yielding information about homoeopathy in patient care. Te opposite approach is to include all indications while restricting study designs to placebo-controlled trials and aggregating results in an MAs, thus yielding information about the specifc efects of homoeopathy beyond those of placebo. A major reason for using this approach has been the claim that ‘homoeopathy violates natural laws and thus any efect must be a placebo efect’ [6].
Since 1997, at least six MAs of placebo-controlled homoeopathy trials for any condition have been published [6–11]. These MAs have difered in their methods for trial inclusion, data synthesis and assessment of risk of bias; furthermore, their results and conclusions have been inconsistent. During this period, there have been substantial advancements in methodology and quality standards for MAs and other SRs [12–15], including SRs of SRs (also called overviews or umbrella reviews) [16–18]. To our knowledge, a formal SR of MAs of randomised placebo-controlled homoeopathy trials for any condition has not been performed. Herein, we report such an SR.

What the authors actually did is this: they knew of the 6 MA; they also knew that they arrived at cautiously positive conclusions; finally they also were aware of the fact that, obviously, the 6 MA included more or less almost the same primary studies. So, by pooling the MAs, they generated a positive result which was no longer marginally positive but strongly. Anyone looking through this strategy (which in effect multiplies the results of many primary studies by factor 6) must realize that this methods creates a false-positive impression.

My suspicion that this paper is a deliberate attempt at misleading us about the ‘efficacy’ (actually it should be effectiveness) is strengthened by further facts:

  • One of the two MAs by Mathie et al excluded primary studies that reported positive findings (i.e. mine and the one by Walach et al)
  • Funding: Open Access funding enabled and organized by Projekt DEAL. Funding specifcally for this SR was provided by Christophorus-Stiftung (No. 393 CST), Stiftung Marion Meyenburg (Date 24.09.2020), Dr. Hauschka Stiftung (Date 16.11.2020) and Gesellschaft für Pluralität im Gesundheitswesen (Dates 11.06.2021, 22.06.2021). General funding for IFAEMM was provided by the Software-AG Stiftung (SE-P 13544). The funders had no infuence on the writing of the protocol or on the planning, conduct and publication of this SR.
  • Competing interests: In the past 3 years, HJH has received research grants from two manufacturers of anthroposophic medicinal products (Wala Heilmittel GmbH, Bad Boll/ Eckwälden, Germany; Weleda AG, Arlesheim Switzerland). Anthroposophic medicine is not based on the homoeopathic simile principle or on drug provings, but some anthroposophic medicinal products are potentized. The two manufacturers had no involvement with the present SR. Anthroposophic medicinal products were not part of the intervention in any of the trials evaluated in the MAs of this SR (Suppl. Table 15). DSR has received a development grant from Heel GmbH (manufacturer of homoeopathic products) for online training in case report writing. AG, KvA and HK declare that they have no competing interests.
  • Author details: 1) Institute for Applied Epistemology and Medical Methodology at Witten/ Herdecke University (IFAEMM), Freiburg, Germany. 2) Faculty of Health, Department of Medicine, Chair of Medical Theory, Integrative and Anthroposophic Medicine, Witten/Herdecke University, Witten, Germany. 3) Maryland University of Integrative Health (MUIH), Laurel, MD, USA. 4) Homeopathic Pharmacopoeia Convention of the United States (HPCUS), Southeastern, PA, USA.

Personally, I do not find it surprising that these authors bend over backwards to publish something positive about homeopathy (such things happen in homeopathy all the time). However, I do find it astonishing that an allegedly decent journal passes such pseudoscience for publication as though it is serious science.

12 Responses to A new systematic review of homoeopathy reported a positive result – but I can’t take it seriously!

  • And let’s wait for Dana to come along and jabber some inanities.. ten, nine, eight, seven, six..

  • When it comes to meta-analysis one should keep one thing in mind. No matter how high you stack the cow pies, you will never get a cream pie.

  • Interesting. The IFAEMM is not part of the University Witten/Herdecke (whose reputation is also not the best due to their proximity to SCAM). It is only “registered society” (eingetragener Verein).

    The Institute for Applied Epistemology and Medical Methodology e. V. is an independent scientific institution founded in Freiburg in 1994.
    The institute stands in continuation of the medical-social and scientific impulse of Gerhard Kienle (1923-1983) – anthroposophist, founder of the community hospital Herdecke and the first German university in free sponsorship (Witten/Herdecke) and pioneer for the legal recognition and acceptance of complementary medicine methods and a pluralistic conception of medicine.

    This impulse elevates both in science and in practice the widest possible consideration of the dignity and the truth and freedom capacity of human individuality to the maxim in all cognitive, action and organizational life.

    Accordingly, the tasks of the Institute for Applied Epistemology and Medical Methodology e. V. lie in the field of the development and application of scientific and medical research methods, in which the dignity and truth and freedom of the human being are given special consideration. A special concern is to realize in all research projects the reference to the epistemological foundations of the respective field.

    The work of the Institute is emphatically based on the ideas of occidental philosophers, scientists and physicians; in particular Plato, Aristotle, Galileo, Descartes, Newton, Goethe, Kant, Fichte, Hegel, Steiner, Wegman, Wittgenstein, Fleck, Weiss, Bertalanffy, Duncker, Martini, Feinstein, Goodwin.

    The Institute pursues exclusively scientific purposes and does not seek profits. It is party-politically and denominationally neutral. It is financed by free donations and foundations. As far as compatible with the aims of the association, contract research is also carried out. It is directed by Dr. med. Helmut Kiene.

  • “Since 1997, several meta-analyses (MAs) of placebo-controlled randomised efficacy trials of homoeopathy for any indication (PRETHAIs) have been published with different methods, results and conclusions.

    The exclusion criteria were as follows: systematic reviews without MAs; MAs restricted to age or gender groups, specific indications, or specific homoeopathic treatments;…”
    [My bolding]

    So, they went fishing for any indication of efficacy from only non-specific treatments. Yet they managed to find only that “the effect remained significant in 3 of the [6 carefully selected] MAs”.

    Good grief, that’s pathetic !

    • If they find 3 of the 6 carefully selected MAs in the final part, is simply, Mr. Pete, they selected the best global MA’s published in peer reviewed journals! So, Shang, Cucherat and Linde 1998 MA’s were ranked as low quality evidence (High Risk of Bias). We have evidence that the “best” quantitave evidence managed by “so called skeptics” is in the best case, an example of low quality and outcome bias (post festum hypothesis). Then, as UK Report 2010 and EASAC’s reports are enterely based on Shang et al MA’s, we can with confidence discard the conclusions make by Ernst (the leading anti homeopathy expert consulted by House of Lords).

  • The main aim of SRs and MAs are to make a decision when conflicting conclusions are made in RCT published at different parts of the world.

    When MAs on Homeopathy medicine published till date gives varied results, it make sense to conduct a SR on them.

    The pooled result is valid and excellent! A real tribute to this age old system.

    Actually a Review is to be done pooling all the data since the inception of homeopathy! Yes the data is there..

  • This “SR of MAs” seems to me like cooking without water or sewing without thread. By which we used to refer to educational programmes, even at universities, that required time and effort but were obviously on the verge of being meaningless.

  • I found following article in German commenting on the review:
    I haven’t really checked the claims and I am not even in position to judge it with my backround. One of the points seems to agree with point made in the article here – some studies actually counted multiple times which even I can tell is a serious flaw. There are more details which can be useful.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Subscribe via email

Enter your email address to receive notifications of new blog posts by email.

Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted but you must tick the box: “Save my name, email, and website in this browser for the next time I comment.”

The most recent comments from all posts can be seen here.