Cochrane reviews have the reputation to be the most reliable evidence available anywhere. They are supposed to be independent, rigorous, transparent and up-to-date. Usually, this reputation is justified, in my view. But do the 54 Cochrane reviews of acupuncture quoted in my previous post live up to it?
If one had to put the entire body of evidence in a nutshell, it would probably look something like this:
TOTAL NUMBER OF SYSTEMATIC REVIEWS = 54
POSITIVE CONCLUSIONS BASED ON MORE THAN ONE HIGH QUALITY STUDY = 2
FAILURE TO REACH CONVINCINGLY POSITIVE CONCLUSIONS = 52
The two positive reviews are on:
1) prevention of migraine
2) prevention of tension-type headache
Both of the positive reviews are by Linde et al.
Allow me to raise just a few further critical points:
- If I counted correctly, 19 of the 54 reviews are authored entirely by Chinese authors. Why could this be a problem? One reason could be that many Chinese authors seem to be biased in favour of acupuncture. Another reason could be that data fabrication is rife in China.
- Many if not most of the primary studies are published in Chinese. This means that it is impossible for most non-Chinese co-authors of the review as well as for the referees of the paper to check the accuracy of the data extraction.
- I counted a total of 15 reviews which were by authors who one could categorise as outspoken enthusiasts of acupuncture. In these cases, one might be concerned about the trustworthiness of the review’s conclusion.
- Many (some would say most) of the reviews cover subject areas which are frankly bizarre. Who would, for instance, consider acupuncture a plausible treatment for Glaucoma, Mumps or chronic hepatitis B?
- Despite almost all of the reviews demonstrating that there is no good reason to recommend acupuncture for the condition in question, hardly any of them draw a transparent, helpful and clear conclusion. One example might suffice: the review of acupuncture for hordeolum concluded that “Low‐certainty evidence suggests that acupuncture with or without conventional treatments may provide short‐term benefits for treating acute hordeolum…” Its Chinese authors reached this conclusion on the basis of 6 primary studies (all from China) which were all of lousy quality. In such a case, the only justified conclusion would be, in my view, something like this: THERE IS NO RELIABLE EVIDENCE …
Despite these serious limitations and avoidable confusions, the totality of the evidence from these 54 Cochrane reviews does send an important message: there is hardly a single condition for which acupuncture is clearly, convincingly and indisputably effective. What I find most regrettable, however, is that the Cochrane Collaboration allowed the often biased review authors to obscure this crucial message so thoroughly. One needs a healthy portion of critical thinking to get through to the truth here – and how many fans of acupuncture possess such a thing?