This review assessed the magnitude of reporting bias in trials assessing homeopathic treatments and its impact on evidence syntheses.
A cross-sectional study and meta-analysis. Two persons independently searched Clinicaltrials.gov, the EU Clinical Trials Register and the International Clinical Trials Registry Platform up to April 2019 to identify registered homeopathy trials. To determine whether registered trials were published and to detect published but unregistered trials, two persons independently searched PubMed, Allied and Complementary Medicine Database, Embase and Google Scholar up to April 2021. For meta-analyses, the authors used random effects models to determine the impact of unregistered studies on meta-analytic results.
The investigators reported the proportion of registered but unpublished trials and the proportion of published but unregistered trials. They also assessed whether primary outcomes were consistent between registration and publication
Since 2002, almost 38% of registered homeopathy trials have remained unpublished, and 53% of published randomised controlled trials (RCTs) have not been registered. Retrospective registration was more common than prospective registration. Furthermore, 25% of primary outcomes were altered or changed compared with the registry. Although we could detect a statistically significant trend toward an increase of registrations of homeopathy trials (p=0.001), almost 30% of RCTs published during the past 5 years had not been registered.
A meta-analysis stratified by registration status of RCTs revealed substantially larger treatment effects of unregistered RCTs (SMD: −0.53, 95% CI −0.87 to −0.20) than registered RCTs (SMD: −0.14, 95% CI −0.35 to 0.07).
The authors concluded that registration of published trials was infrequent, many registered trials were not published and primary outcomes were often altered or changed. This likely affects the validity of the body of evidence of homeopathic literature and may overestimate the true treatment effect of homeopathic remedies.
An obvious investigation to do (why did I not have this idea?)!
And a finding that will surprise few (except fans of homeopathy who will, of course, dispute it).
The authors also mention that reporting biases are likely to have a substantial impact on the estimated treatment effect of homeopathy. Using data from a highly cited meta-analysis of homeopathy RCTs, our example showed that unregistered trials yielded substantially larger treatment effects than registered trials. They also caution that, because of the reporting biases identified in their analysis, effect estimates of meta-analyses of homeopathy trials might substantially overestimate the true treatment effect of homeopathic remedies and need to be interpreted cautiously.
In other words, the few reviews suggesting that homeopathy works beyond placebo (and are thus celebrated by homeopaths) are most likely false-positive. And the many reviews showing that homeopathy does not work would demonstrate this fact even clearer if the reporting bias had been accounted for.
Or, to put it bluntly: