The objective of this systematic review was to investigate trustworthiness, i.e. the methodological rigor, transparency, good governance, research integrity, and absence of misconduct, of randomized clinical trials (RCTs) of osteopathic manual therapy.
The team searched MEDLINE, EMBASE, CINAHL, AMED, PEDro, ostmed.dr, and Chiroindex for RCTs evaluating osteopathic treatments (January 2021-June 2024). Risk of bias (RoB) was assessed using Cochrane tool 2, while trustworthiness was assessed with the Cochrane Pregnancy and Childbirth Screening Tool and the REAPPRAISED checklist. Journal trustworthiness, misleading representations in abstracts (“spin”), and results plausibility (via meta-analysis) were also assessed. Findings were synthesized descriptively.
Sixty-one RCTs were included (median sample size 45, interquartile range (IQR) 30-76), largely studying healthy volunteers (29%). The findings are sobering:
- Most had high RoB (74%).
- Only 7% acknowledged potential conflicts from authors’ professional ties.
- Only 27% of contactable authors engaged with reviewers.
- Only 7 abstracts (12%) were free of spin.
- Methodological concerns included poor missing data handling (31%), selective analyses (38%), unacknowledged multiple testing (36%), and outcome switching (12%).
- Meta-analysis found two outliers and 5 further with very large effects.
- 19% of the studies provided inadequate data for pooling.
- Limitations included incomplete reports and lack of validated trustworthiness assessment tools.
The authors concluded that adherence to best practices in osteopathic RCTs needs improvement to enhance evidence-based decision-making, reduce research waste, and enhance reproducibility. Further research should explore whether these findings apply to other small, under-resourced fields.
Considering that many of the authors are affiliated to osteopathic institutes, this is a remarkably critical review. Its results cannot surprise anyone who regularly studies the literature in this field. What is surprising, however, is the conclusion: ADHERENCE TO BEST PRACTICES NEEDS IMPROVEMENT. This is undoubtedly true, but it is not the most direct conclusion of what the data show. In my mind, it should be something more akin to this:
MOST OSTEOPATHIC RCTs ARE OF POOR QUALITY AND THEIR FINDINGS ARE THUS UNRELIABLE.
5 Responses to MOST OSTEOPATHIC RCTs ARE OF POOR QUALITY AND THEIR FINDINGS ARE THUS UNRELIABLE
Leave a Reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
Thank you Mr Ernst. This is very sobering. One question comes to my mind- why do they bother conducting poor quality research, when it’s an obvious waste of material and nonmaterial resources and it creates confusion? This doesnt seem intelligent.
And thank you for your work, big time.
whay do they do it?
several reasons [in my experience]:
incompetence – they are amateurs without adequate training in research,
promotion – they know that poor research is likely to yield the result they want,
money – good research is more expensive than poor research.
Thank you.
In your experience, do these reasons extend also to those who review and then publish poor research? If so, then this seems like a very corrupted or very poorly functioning system and public implications seem horrifying long term. Then we need strong guardians, i’m guessing you’re one of them. Does this have an end?
Thank you
In general: the poorer the research the lower-ranked the journal that it ends up in and the less rigorous the peer review process [what very often happens is that the authors are invited to name their peer-reviewers, and one does not need much imagination to see what the result of it then is]
Horrifying, it’s almost a crime in broad daylight, should be illegal. I want to do something about it too, guess I’ll start with educating myself about good practices for high quality research. The book you recommended to me recently is a good starting point. Thanks again.