The indefatigable Robert Mathie has published another systematic review/meta-analysis, and yet again he failed to come up with a convincingly positive result. This new paper reviews randomised controlled trials (RCTs) of individualised homeopathic treatment (IHT) in which the control (comparator) group was other than placebo (OTP). Its stated aim was to determine the comparative effectiveness of IHT on health-related outcomes in adults and children for any clinical condition that has been the subject of at least one OTP-controlled trial.
For each eligible trial, published in the peer-reviewed literature up to the end of 2015, the authors assessed its risk of bias (internal validity) using the Cochrane tool, and its relative pragmatic or explanatory attitude (external validity) using the 10-domain PRECIS tool. All RCTs were categorised according to whether they examined IHT as an alternative treatment (study design Ia), adjunctively with another intervention (design Ib), or compared with a no-intervention group (design II). For each RCT, the researchers identified a ‘main outcome measure’ to use in meta-analysis: ‘relative effect size’ was reported as odds ratio (OR; values >1 favouring homeopathy) or standardised mean difference (SMD; values < 0 favouring homeopathy).
Eleven RCTs, representing 11 different medical conditions, were eligible for inclusion in this systematic review. Five of the RCTs (four of which in design Ib) were judged to have pragmatic study attitude, two were explanatory, and four were equally pragmatic and explanatory. Ten trials were rated ‘high risk of bias’ overall: one of these, a pragmatic study with design Ib, had high risk of bias solely regarding participant blinding (a bias that is intrinsic to such trials); the other trial was rated ‘uncertain risk of bias’ overall. Eight trials had data that were extractable for meta-analysis: for 4 heterogeneous trials with design Ia, the pooled OR was statistically non-significant; collectively for three clinically heterogeneous trials with design Ib, there was a statistically significant SMD favouring adjunctive IHT; in the remaining trial of design 1a, IHT was non-inferior to fluoxetine in the treatment of depression.
The authors concluded that due to the low quality, the small number and the heterogeneity of studies, the current data preclude a decisive conclusion about the comparative effectiveness of IHT. Generalisability of findings is limited by the variable external validity identified overall; the most pragmatic study attitude was associated with RCTs of adjunctive IHT. Future OTP-controlled trials in homeopathy should aim, as far as possible, to promote both internal validity and external validity.
Considering that almost all of the authors are known proponents of homeopathy – Mathie himself is employed by the London-based ‘Homeopathy Research Institute’ – one has to applaud their rigour and enthusiasm in publishing negative findings about their trade. But why so complicated? I would have thought that a much simpler conclusion would have been clearer: THESE ANALYSES FAILED TO GENERATE EVIDENCE TO SUGGEST THAT HOMEOPATHY IS EFFECTIVE.