systematic review

I have said it often, and I say it again: I do like well-conducted systematic reviews; and Cochrane reviews are usually the best, i. e. most transparent, most thorough and least biased. Thus, I was pleased to see a new Cochrane review of acupuncture aimed at assessing the benefits and harms of acupuncture in patients with hip OA.

The authors included randomized controlled trials (RCTs) that compared acupuncture with sham acupuncture, another active treatment, or no specific treatment; and RCTs that evaluated acupuncture as an addition to another treatment. Major outcomes were pain and function at the short term (i.e. < 3 months after randomization) and adverse events.

Six RCTs with 413 participants were included. Four RCTs included only people with OA of the hip, and two included a mix of people with OA of the hip and knee. All RCTs included primarily older participants, with a mean age range from 61 to 67 years, and a mean duration of hip OA pain from two to eight years. Approximately two-thirds of participants were women. Two RCTs compared acupuncture versus sham acupuncture; the other four RCTs were not blinded. All results were evaluated at short-term (i.e. four to nine weeks after randomization).In the two RCTs that compared acupuncture to sham acupuncture, the sham acupuncture control interventions were judged believable, but each sham acupuncture intervention was also judged to have a risk of weak acupuncture-specific effects, due to placement of non-penetrating needles at the correct acupuncture points in one RCT, and the use of penetrating needles not inserted at the correct points in the other RCT. For these two sham-controlled RCTs, the risk of bias was low for all outcomes.

The combined analysis of two sham-controlled RCTs gave moderate quality evidence of little or no effect in reduction in pain for acupuncture relative to sham acupuncture. Due to the small sample sizes in the studies, the confidence interval includes both the possibility of moderate benefit and the possibility of no effect of acupuncture (120 participants; Standardized Mean Difference (SMD) -0.13, (95% Confidence Interval (CI) -0.49 to 0.22); 2.1 points greater improvement with acupuncture compared to sham acupuncture on 100 point scale (i.e., absolute percent change -2.1% (95% CI -7.9% to 3.6%)); relative percent change -4.1% (95% CI -15.6% to 7.0%)). Estimates of effect were similar for function (120 participants; SMD -0.15, (95% CI -0.51 to 0.21)). No pooled estimate, representative of the two sham-controlled RCTs, could be calculated or reported for the quality of life outcome.

The four other RCTs were unblinded comparative effectiveness RCTs, which compared (additional) acupuncture to four different active control treatments. There was low quality evidence that addition of acupuncture to the routine primary care that RCT participants were receiving from their physicians was associated with statistically significant and clinically relevant benefits, compared to the routine primary physician care alone, in pain (1 RCT; 137 participants; mean percent difference -22.9% (95% CI -29.2% to -16.6%); relative percent difference -46.5% (95% CI -59.3% to -33.7%)) and function (mean percent difference -19.0% (95% CI -24.41 to -13.59); relative percent difference -38.6% (95% CI -49.6% to -27.6%)). There was no statistically significant difference for mental quality of life and acupuncture showed a small, significant benefit for physical quality of life.

The effects of acupuncture compared with either advice plus exercise or NSAIDs are uncertain. The authors are also uncertain whether acupuncture plus patient education improves pain, function, and quality of life, when compared to patient education alone.

In general, the overall quality of the evidence for the four comparative effectiveness RCTs was low to very low, mainly due to the potential for biased reporting of patient-assessed outcomes due to lack of blinding and sparse data.

Information on safety was reported in 4 RCTs. Two RCTs reported minor side effects of acupuncture, which were primarily minor bruising, bleeding, or pain at needle insertion sites.

The authors concluded that acupuncture probably has little or no effect in reducing pain or improving function relative to sham acupuncture in people with hip osteoarthritis. Due to the small sample size in the studies, the confidence intervals include both the possibility of moderate benefits and the possibility of no effect of acupuncture. One unblinded trial found that acupuncture as an addition to routine primary physician care was associated with benefits on pain and function. However, these reported benefits are likely due at least partially to RCT participants’ greater expectations of benefit from acupuncture. Possible side effects associated with acupuncture treatment were minor.

This is an excellent review of data that (because of contradictions, methodological limitations, heterogeneity etc.) are not easy to evaluate fairly. The review shows that previous verdicts about acupuncture for osteoarthritis might have been too optimistic. Acupuncture has no or only very small specific therapeutic effects. As we have much better therapeutic options for this condition, it means that acupuncture can no longer be recommended as an effective therapy.

That surely must be big news in the little world of acupuncture!

I have been personally involved in several similar reviews:

In 1997, I concluded that the most rigorous studies suggest that acupuncture is not superior to sham-needling in reducing pain of osteoarthritis: both alleviate symptoms to roughly the same degree.

In 2006, the balance of evidence seemed to have shifted and more positive data had emerged; consequently our review concluded that sham-controlled RCTs suggest specific effects of acupuncture for pain control in patients with peripheral joint OA. Considering its favourable safety profile acupuncture seems an option worthy of consideration particularly for knee OA. Further studies are required particularly for manual or electro-acupuncture in hip OA.

Now, it seems that my initial conclusion of 1996 was more realistic. To me this is a fascinating highlight on the fact that in EBM, we change our minds based on the current best evidence. By contrast, in alternative medicine, as we have often lamented on this blog, minds do not easily change and all too often dogma seems to reign.

The new Cochrane review is important in several ways. Firstly, it affirms an appropriately high standard for such reviews. Secondly, it originates from a research team that has, in the past, been outspokenly pro-acupuncture; it is therefore unlikely that the largely negative findings were due to an anti-acupuncture bias. Thirdly – and most importantly – osteoarthritis has been THE condition for which even critical reviewers had to admit that there was at least some good, positive evidence.

It seems therefore, that yet again a beautiful theory has been slain by an ugly fact.

Does acupuncture increase birth rates after IVF?

You might be correct when pointing out that this is a rhetorical question.

Why should acupuncture increase the live birth rates after in vitro fertilization (IVF)?

Because it re-balances yin and yang?

Give me a break!!!

Yet acupuncture is widely used by women undergoing IVF, and therefore, we perhaps ought to know whether it works.

Laudably someone has conducted a trial so that we know the answer.

This study determined the efficacy of acupuncture compared with a sham acupuncture control performed during IVF on live births. It was designed as a single-blind, parallel-group RCT, including 848 women undergoing a fresh IVF cycle, and conducted at 16 IVF centres in Australia and New Zealand between June 29, 2011, and October 23, 2015, with 10 months of pregnancy follow-up until August 2016.

The women received either acupuncture (n = 424) or a sham acupuncture control (n = 424). The first treatment was administered between days 6 to 8 of follicle stimulation, and two treatments were administered prior to and following embryo transfer. The sham control used a non-invasive needle placed away from acupuncture points. The primary outcome was live birth, defined as the delivery of one or more living infants at greater than 20 weeks’ gestation or birth weight of at least 400 g.

Among the 848 women, 24 withdrew consent, and 824 were included in the study, 607 proceeded to an embryo transfer, and 809 (98.2%) had data available on live birth outcomes. Live births occurred among 74 of 405 women (18.3%) receiving acupuncture compared with 72 of 404 women (17.8%) receiving sham control.

The authors concluded that among women undergoing IVF, administration of acupuncture vs sham acupuncture at the time of ovarian stimulation and embryo transfer resulted in no significant difference in live birth rates. These findings do not support the use of acupuncture to improve the rate of live births among women undergoing IVF.

This is a clear result and technically a fairly decent study. I say ‘fairly decent’ because, had the result been positive, one would have to question the efficacy blinding as well as the fact that the acupuncturists might have (inadvertently?) influenced their verum-patients such that they were less anxious and thus produced better outcomes. Moreover, the trial was under-powered, and its publication so long after the end of the study is odd, in my view.

There have, of course, been plenty of trials and even systematic reviews of this topic. Here are the conclusions of the three most recent reviews:

Yet the authors state that “the evidence for efficacy is conflicting”.


The above conclusions seem crystal clear and not at all conflicting!

Is it because the authors needed to justify the no doubt huge costs for their study?

Is it because conducting such a trial while the evidence is already clear (and negative) is arguably not ethical?

Is it because the authors needed this alleged ‘uncertainty’ for getting their trial in a major journal?

I am, of course, not sure – but I am quite sure of one thing: the evidence that acupuncture is useless for IVF was already pretty clear when they started their study.

And pretending otherwise amounts to telling porkies, doesn’t it?

And telling porkies is unethical, isn’t it?

I have often criticised papers published by chiropractors.

Not today!

This article is excellent and I therefore quote extensively from it.

The objective of this systematic review was to investigate, if there is any evidence that spinal manipulations/chiropractic care can be used in primary prevention (PP) and/or early secondary prevention in diseases other than musculoskeletal conditions. The authors conducted extensive literature searches to locate all studies in this area. Of the 13.099 titles scrutinized, 13 articles were included (8 clinical studies and 5 population studies). They dealt with various disorders of public health importance such as diastolic blood pressure, blood test immunological markers, and mortality. Only two clinical studies could be used for data synthesis. None showed any effect of spinal manipulation/chiropractic treatment.

The authors concluded that they found no evidence in the literature of an effect of chiropractic treatment in the scope of PP or early secondary prevention for disease in general. Chiropractors have to assume their role as evidence-based clinicians and the leaders of the profession must accept that it is harmful to the profession to imply a public health importance in relation to the prevention of such diseases through manipulative therapy/chiropractic treatment.

In addition to this courageous conclusion (the paper is authored by a chiropractor and published in a chiro journal), the authors make the following comments:

Beliefs that a spinal subluxation can cause a multitude of diseases and that its removal can prevent them is clearly at odds with present-day concepts, as the aetiology of most diseases today is considered to be multi-causal, rarely mono-causal. It therefore seems naïve when chiropractors attempt to control the combined effects of environmental, social, biological including genetic as well as noxious lifestyle factors through the simple treatment of the spine. In addition, there is presently no obvious emphasis on the spine and the peripheral nervous system as the governing organ in relation to most pathologies of the human body.

The ‘subluxation model’ can be summarized through several concepts, each with its obvious weakness. According to the first three, (i) disturbances in the spine (frequently called ‘subluxations’) exist and (ii) these can cause a multitude of diseases. (iii) These subluxations can be detected in a chiropractic examination, even before symptoms arise. However, to date, the subluxation has been elusive, as there is no proof for its existence. Statements that there is a causal link between subluxations and various diseases should therefore not be made. The fourth and fifth concepts deal with the treatment, namely (iv) that chiropractic adjustments can remove subluxations, (v) resulting in improved health status. However, even if there were an improvement of a condition following treatment, this does not mean that the underlying theory is correct. In other words, any improvement may or may not be caused by the treatment, and even if so, it does not automatically validate the underlying theory that subluxations cause disease…

Although at first look there appears to be a literature on this subject, it is apparent that most authors lack knowledge in research methodology. The two methodologically acceptable studies in our review were found in PubMed, whereas most of the others were identified in the non-indexed literature. We therefore conclude that it may not be worthwhile in the future to search extensively the non-indexed chiropractic literature for high quality research articles.

One misunderstanding requires some explanations; case reports are usually not considered suitable evidence for effect of treatment, even if the cases relate to patients who ‘recovered’ with treatment. The reasons for this are multiple, such as:

  • Individual cases, usually picked out on the basis of their uniqueness, do not reflect general patterns.
  • Individual successful cases, even if correctly interpreted must be validated in a ‘proper’ research design, which usually means that presumed effect must be tested in a properly powered and designed randomized controlled trial.
  • One or two successful cases may reflect a true but very unusual recovery, and such cases are more likely to be written up and published as clinicians do not take the time to marvel over and spend time on writing and publishing all the other unsuccessful treatment attempts.
  • Recovery may be co-incidental, caused by some other aspect in the patient’s life or it may simply reflect the natural course of the disease, such as natural remission or the regression towards the mean, which in human physiology means that low values tend to increase and high values decrease over time.
  • Cases are usually captured at the end because the results indicate success, meaning that the clinical file has to be reconstructed, because tests were used for clinical reasons and not for research reasons (i.e. recorded by the treating clinician during an ordinary clinical session) and therefore usually not objective and reproducible.
  • The presumed results of the treatment of the disease is communicated from the patient to the treating clinician and not to a third, neutral person and obviously this link is not blinded, so the clinician is both biased in favour of his own treatment and aware of which treatment was given, and so is the patient, which may result in overly positive reporting. The patient wants to please the sympathetic clinician and the clinician is proud of his own work and overestimates the results.
  • The long-term effects are usually not known.
  • Further, and most importantly, there is no control group, so it is impossible to compare the results to an untreated or otherwise treated person or group of persons.

Nevertheless, it is common to see case reports in some research journals and in communities with readers/practitioners without a firmly established research culture it is often considered a good thing to ‘start’ by publishing case reports.

Case reports are useful for other reasons, such as indicating the need for further clinical studies in a specific patient population, describing a clinical presentation or treatment approach, explaining particular procedures, discussing cases, and referring to the evidence behind a clinical process, but they should not be used to make people believe that there is an effect of treatment…

For groups of chiropractors, prevention of disease through chiropractic treatment makes perfect sense, yet the credible literature is void of evidence thereof. Still, the majority of chiropractors practising this way probably believe that there is plenty of evidence in the literature. Clearly, if the chiropractic profession wishes to maintain credibility, it is time seriously to face this issue. Presently, there seems to be no reason why political associations and educational institutions should recommend spinal care to prevent disease in general, unless relevant and acceptable research evidence can be produced to support such activities. In order to be allowed to continue this practice, proper and relevant research is therefore needed…

All chiropractors who want to update their knowledge or to have an evidence-based practice will search new information on the internet. If they are not trained to read the scientific literature, they might trust any article. In this situation, it is logical that the ‘believers’ will choose ‘attractive’ articles and trust the results, without checking the quality of the studies. It is therefore important to educate chiropractors to become relatively competent consumers of research, so they will not assume that every published article is a verity in itself…



I am so glad that some experts within the chiropractic community are now publishing statements like these.

This was long overdue.

How was it possible that so many chiropractors so far failed to become competent consumers of research?

Do they and their professional organisations not know that this is deeply unethical?

Actually, I fear they do and did so for a long time.

Why then did they not do anything about it ages ago?

I fear, the answer is as easy as it is disappointing:

If chiropractors systematically trained to become research-competent, the chiropractic profession would cease to exist; they would become a limited version of physiotherapists. There is simply not enough positive evidence to justify chiropractic. In other words, as chiropractic wants to survive, it has little choice other than remaining ignorant of the current best evidence.

Generally speaking, Cochrane reviews provide the best (most rigorous, transparent and independent) evidence on the effectiveness of medical or surgical interventions. It is therefore important to ask what they tell us about homeopathy. In 2010, I did exactly that and published it as an overview of the current best evidence. At the time, there were 6 relevant Cochrane reviews. They covered the following conditions: cancer, attention-deficit hyperactivity disorder, asthma, dementia, influenza and induction of labour. And their results were clear: they did not show that homeopathic medicines have effects beyond placebo.

Now a further Cochrane review has been published.

Does it change this situation?

This systematic review assessed the effectiveness and safety of oral homeopathic medicinal products compared with placebo or conventional therapy to prevent and treat acute respiratory tract infections (ARTIs) in children. The researchers conducted extensive literature searches, checked references, and contacted study authors to identify additional studies. They included all double-blind, randomised controlled trials (RCTs) or double-blind cluster-RCTs comparing oral homeopathy medicinal products with identical placebo or self selected conventional treatments to prevent or treat ARTIs in children aged 0 to 16 years.

Eight RCTs of 1562 children receiving oral homeopathic medicinal products or a control treatment (placebo or conventional treatment) for upper respiratory tract infections (URTIs). Four treatment studies examined the effect on recovery from URTIs, and four studies investigated the effect on preventing URTIs after one to three months of treatment and followed up for the remainder of the year. Two treatment and two prevention studies involved homeopaths individualising treatment for children. The other studies used predetermined, non-individualised treatments. All studies involved highly diluted homeopathic medicinal products.

Several key limitations to the included studies were identified, in particular methodological inconsistencies and high attrition rates, failure to conduct intention-to-treat analysis, selective reporting, and apparent protocol deviations. The authors deemed three studies to be at high risk of bias in at least one domain, and many had additional domains with unclear risk of bias. Three studies received funding from homeopathy manufacturers; one reported support from a non-government organisation; two received government support; one was co-sponsored by a university; and one did not report funding support.

Methodological inconsistencies and significant clinical and statistical heterogeneity precluded robust quantitative meta-analysis. Only four outcomes were common to more than one study and could be combined for analysis. Odds ratios (OR) were generally small with wide confidence intervals (CI), and the contributing studies found conflicting effects, so there was little certainty that the efficacy of the intervention could be ascertained.

All studies assessed as at low risk of bias showed no benefit from oral homeopathic medicinal products; trials at uncertain and high risk of bias reported beneficial effects. The authors found low-quality evidence that non-individualised homeopathic medicinal products confer little preventive effect on ARTIs (OR 1.14, 95% CI 0.83 to 1.57). They also found low-quality evidence from two individualised prevention studies that homeopathy has little impact on the need for antibiotic usage (N = 369) (OR 0.79, 95% CI 0.35 to 1.76).

The authors also assessed adverse events, hospitalisation rates and length of stay, days off school (or work for parents), and quality of life, but were not able to pool data from any of these secondary outcomes. There is insufficient evidence from two pooled individualised treatment studies (N = 155) to determine the effect of homeopathy on short-term cure (OR 1.31, 95% CI 0.09 to 19.54; very low-quality evidence) and long-term cure rates (OR 1.01, 95% CI 0.10 to 9.96; very low-quality evidence). Adverse events were reported inconsistently; however, serious events were not reported. One study found an increase in the occurrence of non-severe adverse events in the treatment group.

The authors concluded that pooling of two prevention and two treatment studies did not show any benefit of homeopathic medicinal products compared to placebo on recurrence of ARTI or cure rates in children. We found no evidence to support the efficacy of homeopathic medicinal products for ARTIs in children. Adverse events were poorly reported, so conclusions about safety could not be drawn.

In their paper, the authors state that “there are no established explanatory models for how highly diluted homeopathic medicinal products might work. For this reason, homeopathy remains highly controversial because the key concepts governing this form of medicine are not consistent with the established laws of conventional therapeutics.” In other words, there is no reason why highly diluted homeopathic remedies should work. Yet, remarkably, when asked what conditions responds best to homeopathy, most homeopaths would probably include ARTI of children.

The authors also point out that “The results of this review are consistent with all previous systematic reviews on homeopathy. Funders and study investigators contemplating any further research in this area need to consider whether further research will advance our knowledge, given the uncertain mechanism of action and debate about how the lack of a measurable dose can make them effective. The studies we identified did not use a uniform approach to choosing and measuring outcomes or assigning appropriate time points for outcome measurement. The use of validated symptom scales would facilitate future meta-analyses. It is unclear if there is any benefit from individualised (classical) homeopathy over the use of commercially available products.”

Even though I agree with the authors on most of their views and comment their excellent work, I would be more outspoken regarding the need of further research. In my view, it would be a foolish, wasteful and therefore unethical activity to fund, plan or conduct further research in this area.

This week, I find it hard to decide where to focus; with all the fuzz about ‘Homeopathy Awareness Week’ it is easy to forget that our friends, the chiros are celebrating  Chiropractic Awareness Week (9-15 April). On this occasion, the British Chiropractic Association (BCA), for instance, want people to keep moving to make a positive impact on managing and preventing back and neck pain.

Good advice! In a recent post, I even have concluded that people should “walk (slowly and cautiously) to the office of their preferred therapist, have a little rest there (say hello to the staff perhaps) and then walk straight back home.” The reason for my advice is based on the fact that there is precious little evidence that the spinal manipulations of chiropractors make much difference plus some worrying indications that they may cause serious damage.

It seems to me that, by focussing their PR away from spinal manipulations and towards the many other things chiropractors sometimes do – they often call this ‘adjunctive therapies’ – there is a tacit admission here that the hallmark intervention of chiros (spinal manipulation) is of dubious value.

A recent article entitled ‘Spinal Manipulative Therapy and Other Conservative Treatments for Low Back Pain: A Guideline From the Canadian Chiropractic Guideline Initiative’ seems to confirm this impression. Its objective was to develop a clinical practice guideline on the management of acute and chronic low back pain (LBP) in adults. The specific aim was to develop a guideline to provide best practice recommendations on the initial assessment and monitoring of people with low back pain and address the use of spinal manipulation therapy (SMT) compared with other commonly used conservative treatments.

The topic areas were chosen based on an Agency for Healthcare Research and Quality comparative effectiveness review, specific to spinal manipulation as a non-pharmacological intervention. The panel updated the search strategies in Medline and assessed admissible systematic reviews and randomized controlled trials. Evidence profiles were used to summarize judgments of the evidence quality and link recommendations to the supporting evidence. Using the Evidence to Decision Framework, the guideline panel determined the certainty of evidence and strength of the recommendations. Consensus was achieved using a modified Delphi technique. The guideline was peer reviewed by an 8-member multidisciplinary external committee.

For patients with acute (0-3 months) back pain, we suggest offering advice (posture, staying active), reassurance, education and self-management strategies in addition to SMT, usual medical care when deemed beneficial, or a combination of SMT and usual medical care to improve pain and disability. For patients with chronic (>3 months) back pain, we suggest offering advice and education, SMT or SMT as part of a multimodal therapy (exercise, myofascial therapy or usual medical care when deemed beneficial). For patients with chronic back-related leg pain, we suggest offering advice and education along with SMT and home exercise (positioning and stabilization exercises).

The authors concluded that a multimodal approach including SMT, other commonly used active interventions, self-management advice, and exercise is an effective treatment strategy for acute and chronic back pain, with or without leg pain.

I find this paper most interesting and revealing. Considering that it originates from the ‘Canadian Chiropractic Guideline Initiative’, it is remarkably shy about recommending SMT – after all their vision is “To enhance the health of Canadians by fostering excellence in chiropractic care.” They are thus not likely to be overly critical of the treatment chiropractors use most, i. e. SMT.

Perhaps this is also the reason why, in their conclusion, they seem to have rather a large blind spot, namely the risks of SMT. I have commented on this issue more often than I care to remember. Most recently, I posted this:

The reason why my stance, as expressed on this blog and elsewhere, is often critical about certain alternative therapies is thus obvious and transparent. For none of them (except for massage) is the risk/benefit balance positive. And for spinal manipulation, it even turns out to be negative. It goes almost without saying that responsible advice must be to avoid treatments for which the benefits do not demonstrably outweigh the risks.



An announcement (it’s in German, I’m afraid) proudly declaring that ‘homeopathy fulfils the criteria of evidence-based medicine‘ caught my attention.

Here is the story:

In 2016, Dr. Melanie Wölk, did a ‘Master of Science’* at the ‘Donau University’ in Krems, Austria investigating the question whether homeopathy follows the rules of evidence-based medicine (EBM). She arrived at the conclusion that YES, IT DOES! This pleased the leading Austrian manufacturer of homeopathics (Dr Peithner) so much and so durably that, on 23 March 2018, he gave her a ‘scientific’ award (the annual Peithner award) for her ‘research’.

So far so good.

Her paper is unpublished, or at least not available on Medline; therefore, I am unable to evaluate it directly. All I know about it from the announcement is that she did her ‘research at the ‘Zentrum für Traditionelle Chinesische Medizin und Komplementärmedizin‘ of the said university. A quick Medline search revealed that this unit has never published anything, not a single paper, it seems! Disappointed I search for Dr. Christine Schauhuber, the leader of the unit; and again I find no Medline-listed publications in her name. My interim conclusion is thus that this institution might not be at the cutting edge of science.

But what do we know about Dr. Melanie Wölk’s award-winning master thesis *?

The announcement tells us that she investigated all RCTs published between 2010 and 2016. In addition, she evaluated:

On that basis, she arrived at her positive verdict – not just tentatively, but without doubt (“Das Ergebnis steht fest”).

Dr Peithner, the owner of the company and awarder of the prize, was quoted stating that this is a very important piece of work for homeopathy; it shows yet again what we see in our daily routine, namely that homeopathics are effective. Wölk’s investigation demonstrates furthermore that high-quality trials of homeopathy do exist, and that it is time to end the witch-hunt aimed at discrediting an effective therapy. Conventional medicine and homeopathy ought to finally work hand in hand – for the benefit of our patients. (“Für die Homöopathie ist das eine sehr wichtige Arbeit, die wieder zeigt, was wir in der ärztlichen Praxis täglich erleben, nämlich dass homöopathische Arzneimittel wirken. Wölks Untersuchung zeigt weiters deutlich, dass es sehr wohl hochqualitative Homöopathie-Studien gibt und es an der Zeit ist, die Hexenjagd zu beenden, mit der eine wirksame medizinische Therapie diskreditiert werden soll. Konventionelle Medizin und Homöopathie sollten endlich Hand in Hand arbeiten – zum Wohle der Patientinnen und Patienten.”)

I do hope that Dr Wölk uses the prize money (by no means a fortune; see photo) to buy some time for publishing her work (one of my teachers, all those years ago, used to say ‘unpublished research is no research’) so that we can all benefit from it. Until it becomes available, I should perhaps mention that the description of her methodology (publications between 2010 and 2016 [plus a few other papers that nicely fitted the arguments?]; including one Linde review and not his more recent re-analysis [see above]) does not inspire me to think that Dr Wölk’s research was anywhere near rigorous, systematic or complete. In the same vein, I am tempted to point out that the Swiss report is probably the very last document I would select, if I wanted to generate an objective picture about the value of homeopathy.

Taking all this into account, I conclude that we seem to be dealing here with a

  • pseudo-prize (given by a commercial firm to further its business) for a piece of
  • pseudo-research (the project seems to have been aimed to white-wash homeopathy) into
  • pseudo-medicine (a treatment that has been tested extensively but has not been shown to work beyond placebo).

*Wölk, Melanie: Eminenz oder Evidenz: Die Homöopathie auf dem Prüfstand der Evidence based Medicine. Masterarbeit zur Erlangung des akademischen Abschlusses Master of Science im Universitätslehrgang Natural Medicine. Donau-Universität Krems, Department für Gesundheitswissenschaften und Biomedizin. Krems, Mai 2016.

We all know that there is a plethora of interventions for and specialists in low back pain (chiropractors, osteopaths, massage therapists, physiotherapists etc., etc.); and, depending whether you are an optimist or a pessimist, each of these therapies is as good or as useless as the next. Today, a widely-publicised series of articles in the Lancet confirms that none of the current options is optimal:

Almost everyone will have low back pain at some point in their lives. It can affect anyone at any age, and it is increasing—disability due to back pain has risen by more than 50% since 1990. Low back pain is becoming more prevalent in low-income and middle-income countries (LMICs) much more rapidly than in high-income countries. The cause is not always clear, apart from in people with, for example, malignant disease, spinal malformations, or spinal injury. Treatment varies widely around the world, from bed rest, mainly in LMICs, to surgery and the use of dangerous drugs such as opioids, usually in high-income countries.

The Lancet publishes three papers on low back pain, by an international group of authors led by Prof Rachelle Buchbinder, Monash University, Melbourne, Australia, which address the issues around the disorder and call for worldwide recognition of the disability associated with the disorder and the removal of harmful practices. In the first paper, Jan Hartvigsen, Mark Hancock, and colleagues draw our attention to the complexity of the condition and the contributors to it, such as psychological, social, and biophysical factors, and especially to the problems faced by LMICs. In the second paper, Nadine Foster, Christopher Maher, and their colleagues outline recommendations for treatment and the scarcity of research into prevention of low back pain. The last paper is a call for action by Rachelle Buchbinder and her colleagues. They say that persistence of disability associated with low back pain needs to be recognised and that it cannot be separated from social and economic factors and personal and cultural beliefs about back pain.

Overview of interventions endorsed for non-specific low back pain in evidence-based clinical practice guidelines (Danish, US, and UK guidelines)

In this situation, it makes sense, I think, to opt for a treatment (amongst similarly effective/ineffective therapies) that is at least safe, cheap and readily available. This automatically rules out chiropractic, osteopathy and many others. Exercise, however, does come to mind – but what type of exercise?

The aim of this meta-analysis of randomized controlled trials was to gain insight into the effectiveness of walking intervention on pain, disability, and quality of life in patients with chronic low back pain (LBP) at post intervention and follow ups.

Six electronic databases (PubMed, Science Direct, Web of Science, Scopus, PEDro and The Cochrane library) were searched from 1980 to October 2017. Randomized controlled trials (RCTs) in patients with chronic LBP were included, if they compared the effects of walking intervention to non-pharmacological interventions. Pain, disability, and quality of life were the primary health outcomes.

Nine RCTs were suitable for meta-analysis. Data was analysed according to the duration of follow-up (short-term, < 3 months; intermediate-term, between 3 and 12 months; long-term, > 12 months). Low- to moderate-quality evidence suggests that walking intervention in patients with chronic LBP was as effective as other non-pharmacological interventions on pain and disability reduction in both short- and intermediate-term follow ups.

The authors concluded that, unless supplementary high-quality studies provide different evidence, walking, which is easy to perform and highly accessible, can be recommended in the management of chronic LBP to reduce pain and disability.

I know – this will hardly please the legions of therapists who earn their daily bread with pretending their therapy is the best for LBP. But healthcare is clearly not about the welfare of the therapists, it is/should be about patients. And patients should surely welcome this evidence. I know, walking is not always easy for people with severe LBP, but it seems effective and it is safe, free and available to everyone.

My advice to patients is therefore to walk (slowly and cautiously) to the office of their preferred therapist, have a little rest there (say hello to the staff perhaps) and then walk straight back home.

The plethora of dodgy meta-analyses in alternative medicine has been the subject of a recent post – so this one is a mere update of a regular lament.

This new meta-analysis was to evaluate evidence for the effectiveness of acupuncture in the treatment of lumbar disc herniation (LDH). (Call me pedantic, but I prefer meta-analyses that evaluate the evidence FOR AND AGAINST a therapy.) Electronic databases were searched to identify RCTs of acupuncture for LDH, and 30 RCTs involving 3503 participants were included; 29 were published in Chinese and one in English, and all trialists were Chinese.

The results showed that acupuncture had a higher total effective rate than lumbar traction, ibuprofen, diclofenac sodium and meloxicam. Acupuncture was also superior to lumbar traction and diclofenac sodium in terms of pain measured with visual analogue scales (VAS). The total effective rate in 5 trials was greater for acupuncture than for mannitol plus dexamethasone and mecobalamin, ibuprofen plus fugui gutong capsule, loxoprofen, mannitol plus dexamethasone and huoxue zhitong decoction, respectively. Two trials showed a superior effect of acupuncture in VAS scores compared with ibuprofen or mannitol plus dexamethasone, respectively.

The authors from the College of Traditional Chinese Medicine, Jinan University, Guangzhou, Guangdong, China, concluded that acupuncture showed a more favourable effect in the treatment of LDH than lumbar traction, ibuprofen, diclofenac sodium, meloxicam, mannitol plus dexamethasone and mecobalamin, fugui gutong capsule plus ibuprofen, mannitol plus dexamethasone, loxoprofen and huoxue zhitong decoction. However, further rigorously designed, large-scale RCTs are needed to confirm these findings.

Why do I call this meta-analysis ‘dodgy’? I have several reasons, 10 to be exact:

  1. There is no plausible mechanism by which acupuncture might cure LDH.
  2. The types of acupuncture used in these trials was far from uniform and  included manual acupuncture (MA) in 13 studies, electro-acupuncture (EA) in 10 studies, and warm needle acupuncture (WNA) in 7 studies. Arguably, these are different interventions that cannot be lumped together.
  3. The trials were mostly of very poor quality, as depicted in the table above. For instance, 18 studies failed to mention the methods used for randomisation. I have previously shown that some Chinese studies use the terms ‘randomisation’ and ‘RCT’ even in the absence of a control group.
  4. None of the trials made any attempt to control for placebo effects.
  5. None of the trials were conducted against sham acupuncture.
  6. Only 10 studies 10 trials reported dropouts or withdrawals.
  7. Only two trials reported adverse reactions.
  8. None of these shortcomings were critically discussed in the paper.
  9. Despite their affiliation, the authors state that they have no conflicts of interest.
  10. All trials were conducted in China, and, on this blog, we have discussed repeatedly that acupuncture trials from China never report negative results.

And why do I find the journal ‘dodgy’?

Because any journal that publishes such a paper is likely to be sub-standard. In the case of ‘Acupuncture in Medicine’, the official journal of the British Medical Acupuncture Society, I see such appalling articles published far too frequently to believe that the present paper is just a regrettable, one-off mistake. What makes this issue particularly embarrassing is, of course, the fact that the journal belongs to the BMJ group.

… but we never really thought that science publishing was about anything other than money, did we?

What an odd title, you might think.

Systematic reviews are the most reliable evidence we presently have!

Yes, this is my often-voiced and honestly-held opinion but, like any other type of research, systematic reviews can be badly abused; and when this happens, they can seriously mislead us.

new paper by someone who knows more about these issues than most of us, John Ioannidis from Stanford university, should make us think. It aimed at exploring the growth of published systematic reviews and meta‐analyses and at estimating how often they are redundant, misleading, or serving conflicted interests. Ioannidis demonstrated that publication of systematic reviews and meta‐analyses has increased rapidly. In the period January 1, 1986, to December 4, 2015, PubMed tags 266,782 items as “systematic reviews” and 58,611 as “meta‐analyses.” Annual publications between 1991 and 2014 increased 2,728% for systematic reviews and 2,635% for meta‐analyses versus only 153% for all PubMed‐indexed items. Ioannidis believes that probably more systematic reviews of trials than new randomized trials are published annually. Most topics addressed by meta‐analyses of randomized trials have overlapping, redundant meta‐analyses; same‐topic meta‐analyses may exceed 20 sometimes.

Some fields produce massive numbers of meta‐analyses; for example, 185 meta‐analyses of antidepressants for depression were published between 2007 and 2014. These meta‐analyses are often produced either by industry employees or by authors with industry ties and results are aligned with sponsor interests. China has rapidly become the most prolific producer of English‐language, PubMed‐indexed meta‐analyses. The most massive presence of Chinese meta‐analyses is on genetic associations (63% of global production in 2014), where almost all results are misleading since they combine fragmented information from mostly abandoned era of candidate genes. Furthermore, many contracting companies working on evidence synthesis receive industry contracts to produce meta‐analyses, many of which probably remain unpublished. Many other meta‐analyses have serious flaws. Of the remaining, most have weak or insufficient evidence to inform decision making. Few systematic reviews and meta‐analyses are both non‐misleading and useful.

The author concluded that the production of systematic reviews and meta‐analyses has reached epidemic proportions. Possibly, the large majority of produced systematic reviews and meta‐analyses are unnecessary, misleading, and/or conflicted.

Ioannidis makes the following ‘Policy Points’:

  • Currently, there is massive production of unnecessary, misleading, and conflicted systematic reviews and meta‐analyses. Instead of promoting evidence‐based medicine and health care, these instruments often serve mostly as easily produced publishable units or marketing tools.
  • Suboptimal systematic reviews and meta‐analyses can be harmful given the major prestige and influence these types of studies have acquired.
  • The publication of systematic reviews and meta‐analyses should be realigned to remove biases and vested interests and to integrate them better with the primary production of evidence.

Obviously, Ioannidis did not have alternative medicine in mind when he researched and published this article. But he easily could have! Virtually everything he stated in his paper does apply to it. In some areas of alternative medicine, things are even worse than Ioannidis describes.

Take TCM, for instance. I have previously looked at some of the many systematic reviews of TCM that currently flood Medline, based on Chinese studies. This is what I concluded at the time:

Why does that sort of thing frustrate me so much? Because it is utterly meaningless and potentially harmful:

  • I don’t know what treatments the authors are talking about.
  • Even if I managed to dig deeper, I cannot get the information because practically all the primary studies are published in obscure journals in Chinese language.
  • Even if I  did read Chinese, I do not feel motivated to assess the primary studies because we know they are all of very poor quality – too flimsy to bother.
  • Even if they were formally of good quality, I would have my doubts about their reliability; remember: 100% of these trials report positive findings!
  • Most crucially, I am frustrated because conclusions of this nature are deeply misleading and potentially harmful. They give the impression that there might be ‘something in it’, and that it (whatever ‘it’ might be) could be well worth trying. This may give false hope to patients and can send the rest of us on a wild goose chase.

So, to ease the task of future authors of such papers, I decided give them a text for a proper EVIDENCE-BASED conclusion which they can adapt to fit every review. This will save them time and, more importantly perhaps, it will save everyone who might be tempted to read such futile articles the effort to study them in detail. Here is my suggestion for a conclusion soundly based on the evidence, not matter what TCM subject the review is about:


On another occasion, I stated that I am getting very tired of conclusions stating ‘…XY MAY BE EFFECTIVE/HELPFUL/USEFUL/WORTH A TRY…’ It is obvious that the therapy in question MAY be effective, otherwise one would surely not conduct a systematic review. If a review fails to produce good evidence, it is the authors’ ethical, moral and scientific obligation to state this clearly. If they don’t, they simply misuse science for promotion and mislead the public. Strictly speaking, this amounts to scientific misconduct.

In yet another post on the subject of systematic reviews, I wrote that if you have rubbish trials, you can produce a rubbish review and publish it in a rubbish journal (perhaps I should have added ‘rubbish researchers).

And finally this post about a systematic review of acupuncture: it is almost needless to mention that the findings (presented in a host of hardly understandable tables) suggest that acupuncture is of proven or possible effectiveness/efficacy for a very wide array of conditions. It also goes without saying that there is no critical discussion, for instance, of the fact that most of the included evidence originated from China, and that it has been shown over and over again that Chinese acupuncture research never seems to produce negative results.

The main point surely is that the problem of shoddy systematic reviews applies to a depressingly large degree to all areas of alternative medicine, and this is misleading us all.

So, what can be done about it?

My preferred (but sadly unrealistic) solution would be this:


Research is not fundamentally different from other professional activities; to do it well, one needs adequate training; and doing it badly can cause untold damage.

Today, the BMJ published our ‘head to head‘ article on the above question. Dr Mike Cummings argues the pro-part, while Prof Asbjorn Horbjardsson and I argue against the notion.

The pro arguments essentially are the well-rehearsed points acupuncture-fans like to advance:

  • Some guidelines do recommend acupuncture.
  • Sham acupuncture is not a valid comparator.
  • The largest meta-analysis shows a small effect.
  • Acupuncture is not implausible.
  • It improves quality of life.

Cummings concludes as follows: In summary, the pragmatic view sees acupuncture as a relatively safe and moderately effective intervention for a wide range of common chronic pain conditions. It has a plausible set of neurophysiological mechanisms supported by basic science.12 For those patients who choose it and who respond well, it considerably improves health related quality of life, and it has much lower long term risk for them than non-steroidal anti-inflammatory drugs. It may be especially useful for chronic musculoskeletal pain and osteoarthritis in elderly patients, who are at particularly high risk from adverse drug reactions.

Our arguments are also not new; essentially, we stress that:

  • The effects of acupuncture are too small to be clinically relevant.
  • They are probably not even caused by acupuncture, but the result of residual bias.
  • Pragmatic trials are of little value in defining efficacy.
  • Acupuncture is not free of risks.
  • Regular acupuncture treatments are expensive.
  • There is no generally accepted, plausible mechanism.

We concluded that after decades of research and hundreds of acupuncture pain trials, including thousands of patients, we still have no clear mechanism of action, insufficient evidence for clinically worthwhile benefit, and possible harms. Therefore, doctors should not recommend acupuncture for pain.

Neither Asbjorn nor I have any conflicts of interests to declare.

Dr Cummings, by contrast, states that he is the salaried medical director of the British Medical Acupuncture Society, which is a membership organisation and charity established to stimulate and promote the use and scientific understanding of acupuncture as part of the practice of medicine for the public benefit. He is an associate editor for Acupuncture in Medicine, published by BMJ. He has a modest private income from lecturing outside the UK, royalties from textbooks, and a partnership teaching veterinary surgeons in Western veterinary acupuncture. He has participated in a NICE guideline development group as an expert adviser discussing acupuncture. He has used Western medical acupuncture in clinical practice following a chance observation as a medical officer in the Royal Air Force in 1989.


Please, do let us know by posting a comment here, or directly at the BMJ article (better), or both (best).

Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted.

Click here for a comprehensive list of recent comments.