Writing systematic reviews of TCM has just become a whole lot easier

Don’t get me wrong, I have nothing against systematic reviews. Quite to the contrary, I am sure they are an important source of information for patients, doctors, scientists, policy makers and others – after all, I have published more than 300 of such papers!

Having said that, I do dislike a certain type of systematic review, namely systematic reviews by Chinese authors evaluating TCM therapies and arriving at misleading conclusions. Such papers are currently swamping the marked.

At first glance, they look fine. On closer scrutiny, however, most turn out to be stereotypically useless, boring and promotional. The type of article I mean starts by stating its objective which usually is to evaluate the evidence for a traditional Chinese therapy as a treatment of a condition which few people in their right mind would treat with any form of TCM. It continues with details about the methodologies employed and then, in the results section, informs the reader that x studies were included in the review which mostly reported encouraging results but were wide open to bias. And then comes the crucial bit: THE CONCLUSIONS.

They are as predictable as they are misleading. let me give you two examples only published in the last few days.

The first review drew the following conclusions: This systematic review suggests that Chinese Herbal Medicine as an adjunctive therapy can improve cognitive impairment and enhance immediate response and quality of life in Senile Vascular Dementia patients. However, because of limitations of methodological quality in the included studies, further research of rigorous design is needed.

The second review concluded that the evidence that external application of traditional Chinese medicine is an effective treatment for venous ulcers is encouraging, but not conclusive due to the low methodological quality of the RCTs. Therefore, more high-quality RCTs with larger sample sizes are required.

Why does that sort of thing frustrate me so much? Because it is utterly meaningless and potentially harmful:

• I don’t know what treatments the authors are talking about.
• Even if I managed to dig deeper, I cannot get the information because practically all the primary studies are published in obscure journals in Chinese language.
• Even if I  did read Chinese, I do not feel motivated to assess the primary studies because we know they are all of very poor quality – too flimsy to bother.
• Even if they were formally of good quality, I would have my doubts about their reliability; remember: 100% of these trials report positive findings!
• Most crucially, I am frustrated because conclusions of this nature are deeply misleading and potentially harmful. They give the impression that there might be ‘something in it’, and that it (whatever ‘it’ might be) could be well worth trying. This may give false hope to patients and can send the rest of us on a wild goose chase.

So, to ease the task of future authors of such papers, I decided give them a text for a proper EVIDENCE-BASED conclusion which they can adapt to fit every review. This will save them time and, more importantly perhaps, it will save everyone who might be tempted to read such futile articles the effort to study them in detail. Here is my suggestion for a conclusion soundly based on the evidence, not matter what TCM subject the review is about:

OUR SYSTEMATIC REVIEW HAS SHOWN THAT THERAPY ‘X’ AS A TREATMENT OF CONDITION ‘Y’ IS CURRENTLY NOT SUPPORTED BY SOUND EVIDENCE.