Cochrane reviews have the reputation to be the most reliable evidence available anywhere. They are supposed to be independent, rigorous, transparent and up-to-date. Usually, this reputation is justified, in my view. But do the 54 Cochrane reviews of acupuncture quoted in my previous post live up to it?
If one had to put the entire body of evidence in a nutshell, it would probably look something like this:
TOTAL NUMBER OF SYSTEMATIC REVIEWS = 54
POSITIVE CONCLUSIONS BASED ON MORE THAN ONE HIGH QUALITY STUDY = 2
FAILURE TO REACH CONVINCINGLY POSITIVE CONCLUSIONS = 52
The two positive reviews are on:
1) prevention of migraine
2) prevention of tension-type headache
Both of the positive reviews are by Linde et al.
Allow me to raise just a few further critical points:
- If I counted correctly, 19 of the 54 reviews are authored entirely by Chinese authors. Why could this be a problem? One reason could be that many Chinese authors seem to be biased in favour of acupuncture. Another reason could be that data fabrication is rife in China.
- Many if not most of the primary studies are published in Chinese. This means that it is impossible for most non-Chinese co-authors of the review as well as for the referees of the paper to check the accuracy of the data extraction.
- I counted a total of 15 reviews which were by authors who one could categorise as outspoken enthusiasts of acupuncture. In these cases, one might be concerned about the trustworthiness of the review’s conclusion.
- Many (some would say most) of the reviews cover subject areas which are frankly bizarre. Who would, for instance, consider acupuncture a plausible treatment for Glaucoma, Mumps or chronic hepatitis B?
- Despite almost all of the reviews demonstrating that there is no good reason to recommend acupuncture for the condition in question, hardly any of them draw a transparent, helpful and clear conclusion. One example might suffice: the review of acupuncture for hordeolum concluded that “Low‐certainty evidence suggests that acupuncture with or without conventional treatments may provide short‐term benefits for treating acute hordeolum…” Its Chinese authors reached this conclusion on the basis of 6 primary studies (all from China) which were all of lousy quality. In such a case, the only justified conclusion would be, in my view, something like this: THERE IS NO RELIABLE EVIDENCE …
Despite these serious limitations and avoidable confusions, the totality of the evidence from these 54 Cochrane reviews does send an important message: there is hardly a single condition for which acupuncture is clearly, convincingly and indisputably effective. What I find most regrettable, however, is that the Cochrane Collaboration allowed the often biased review authors to obscure this crucial message so thoroughly. One needs a healthy portion of critical thinking to get through to the truth here – and how many fans of acupuncture possess such a thing?
12 Responses to Acupuncture: an update of the most reliable evidence – PART 3
Leave a Reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
Excellent, Edzard! We all would like to see institutions as the Cochrane Library, the Pubmed data base and the NCIH as reliable sources of scientific truths. You showed that such is unfortunately not the case.
I think the scientific truth is there; pity that one has to dig a bit to find it. it should be in the open for everyone to see.
That is true. If one looks at the whole picture and scrutinises the small – very small – number of positive reviews one gets a very clear picture.
A large body of evidence shows acupuncture does exactly what one would expect it to do.
It produces placebo expectation benefits, not without the cost of harm.
It’s nonsense on stilts albeit popular nonsense.
The continuation of acupuncture research is also nonsense. I would call it nothing but a great SCAM in itself.
Thank you this Edzard. It would be interesting looking at those systematic reviews and examining how they would rate on AMSTAR 2. There is one published examination of SR using AMSTAR 2, and we have one accepted as well. Both show that the bulk of those reviews (that’s the review methods, not the results) would give very low or low confidence in the results.
It’s not just acupuncture, it’s the quality of the reviews themselves.
[Almeida MO, Yamato TP, Parreira PDCS, Costa LOP, Kamper S, Saragiotto BT. Overall confidence in the results of systematic reviews on exercise therapy for chronic low back pain: a cross-sectional analysis using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) 2 tool. Braz J Phys Ther. 2019: pii: S1413-3555(18)30599-9.]
Whereupon the Chinese (government at least) will ignore it!
Have you read (yesterday’s Times) that the Chinese government is threatening to jail doctors who oppose Traditional Chinese Medicine!
I.e. If they say “There is no evidence for benefit from (TCM, acupuncture, moxibustion…)” – they could go to jail!
End of evidence-based medicine, indeed, of science, in China.
Sigh.
Not the end of science. 5G tech. Bugging?
TCM is big business and good public relations – the average person is not that criticially minded. TCM is so user friendly compared with nasty-hurty “western” medicine. Placebos are certainly user friendly. OTOH drugs like Artemisinin, scientifically developed from herbs aren’t nasty-hurty or placebos. They are TCM too. Where TCM is merely placebo – though not admitted – then that nasty western medicine may be used.
Promotion of popular and lucrative TCM for public relations and economic reasons inside and outside China serves the interests of the ruling Party well.
They will use science when it suits their purposes and bullshit similarly.
We criticize the lack of scientific rigor of the Chinese authors and reviewers, but no one criticizes that in several European countries acupuncture is included in the public health system and that many doctors, particularly in Austria and Germany, practice it, promote it and prescribe it, at the expense of public money.
Nobody?
[this blog is full with posts about this and related subjects!]
Yes, sorry, you’re right. You and a few more, but not enough.
Thanks for your research and for your posts.
no worries
And also meant to say that nothing much has changed over the past 20 years, with little attention paid to previous work:
Systematic review of systematic reviews of acupuncture published 1996-2005
C J Derry, S Derry, H J McQuay, R A Moore
PMID: 16956145 PMCID: PMC4953522 DOI: 10.7861/clinmedicine.6-4-381
Systematic reviews of acupuncture have tended to support its use, but few applied rigorous inclusion criteria. We tested the credibility of conclusions of systematic reviews of acupuncture published since 1996 by applying rigorous inclusion criteria. Reinterpretation used randomised and double blind trials with valid outcomes or design, and with information available from at least four trials or from 200 patients. Qualified support for acupuncture was originally reported in 12 out of 35 systematic reviews, and strong support was found in another six. Applying stricter inclusion criteria, however, showed that none of the 35 reviews supported acupuncture, predominantly because there were too few patients in the randomised, double blind studies. Six reviews with more than 200 patients in randomised, double blind studies had good evidence of no benefit. Systematic reviews of acupuncture have overstated effectiveness by including studies likely to be biased. They provide no robust evidence that acupuncture works for any indication.
good point – I agree