systematic review

1 2 3 18

Mastitis is a common disease in dairies. Numerous non-antimicrobial drugs and treatment strategies have been recommended for this condition. Homeopaths in particular have long claimed that their highly diluted remedies are an effective option, and I have reported repeatedly about the evidence – see here, here, and here, for instance. Even though it is far from positive, evangelic homeopaths like our friend Dana Ullman or naïve quackery-fans like Prince Charles claim that it is “as effective as antibiotics, the mastitis treatment of choice”.

So, who is right?

I am biased, homeopaths insist.

Ullman is a joke, any rational thinker must admit.

Prince Charles? … no comment.

What we need is an independent body to look at the data.

A new systematic review did exactly that. Its authors are highly respected and come from institutions that are not likely to promote bogus claims:

  • Département de Sciences Cliniques, Faculté de Médecine Vétérinaire, Université de Montréal, Canada
  • Département de Sciences Cliniques, Faculté de Médecine Vétérinaire, Université de Montréal, Canada
  • Canadian Bovine Mastitis and Milk Quality Research Network, Canada
  • Canadian Bovine Mastitis and Milk Quality Research Network, Canada
  • Sherbrooke Research and Development Centre, Agriculture and Agri-Food Canada
  • Canadian Bovine Mastitis and Milk Quality Research Network, Canada
  • Département de Pathologie et Microbiologie, Faculté de Médecine Vétérinaire, Université de Montreal, Canada.

It was performed with studies written in English or French selected from CAB Abstracts, PubMed, and Web of Science. All treatments other than conventional antimicrobials for clinical mastitis during lactation were retained. Only studies comparing the treatment under investigation to a negative or positive control, or both, were included. Outcomes evaluated were clinical and bacteriological cure rates and milk production. Selection of the study, data extraction, and assessment of risk of bias was performed by 3 reviewers. Assessment of risk of bias was evaluated using the Cochrane Collaboration tool for systematic review of interventions.

A total of 2,451 manuscripts were first identified and 39 manuscripts corresponding to 41 studies were included. Among these, 22 were clinical trials, 18 were experimental studies, and one was an observational study. The treatments evaluated were conventional anti-inflammatory drugs (n = 14), oxytocin with or without frequent milk out (n = 5), biologics (n = 9), homeopathy (n = 5), botanicals (n = 4), probiotics (n = 2), and other alternative products (n = 2). All trials had at least one unclear or high risk of bias. Most trials (n = 13) did not observe significant differences in clinical or bacteriological cure rates in comparison with negative or positive controls. Few studies evaluated the effect of treatment on milk yield. In general, the power of the different studies was very low, thus precluding conclusions on non-inferiority or non-superiority of the treatments investigated. No evidence-based recommendations could be given for the use of an alternative or non-antimicrobial conventional treatment for clinical mastitis.

The authors concluded that homeopathic treatments are not efficient for management of clinical mastitis.

Will this finally stop homeopaths from claiming that their placebos work for mastitis?

I would not count on it!

Insomnia is a ‘gold standard’ indication for alternative therapies of all types. In fact, it is difficult to find a single of these treatments that are not being touted for this indication. Consequently, it has become a nice little earner for alternative therapists (hence ‘gold standard’).

But how good is the evidence suggesting that any alternative therapy is effective for insomnia?

Whenever I have discussed this issue on my blog, the conclusion was that the evidence is less than convincing or even negative. Similarly, whenever I conducted proper systematic reviews in this area, the evidence turned out to be weak or negative. Here are four of the conclusions we drew at the time:

“But this ERNST fellow cannot be trusted, he is not objective!”, I hear some of my detractors shout.

But is he really?

Would an independent, high-level panel of experts arrive at more positive conclusions?

Let’s find out!

This European guideline for the diagnosis and treatment of insomnia recently provided recommendations for the management of adult patients with insomnia. The guideline is based on a systematic review of relevant meta-analyses published till June 2016. The GRADE (Grading of Recommendations Assessment, Development and Evaluation) system was used to grade the evidence and guide recommendations.

The findings and recommendations are as follows:

  • Cognitive behavioural therapy for insomnia is recommended as the first-line treatment for chronic insomnia in adults of any age (strong recommendation, high-quality evidence).
  • A pharmacological intervention can be offered if cognitive behavioural therapy for insomnia is not sufficiently effective or not available. Benzodiazepines, benzodiazepine receptor agonists and some antidepressants are effective in the short-term treatment of insomnia (≤4 weeks; weak recommendation, moderate-quality evidence). Antihistamines, antipsychotics, melatonin and phytotherapeutics are not recommended for insomnia treatment (strong to weak recommendations, low- to very-low-quality evidence).
  • Light therapy and exercise need to be further evaluated to judge their usefulness in the treatment of insomnia (weak recommendation, low-quality evidence).
  • Complementary and alternative treatments (e.g. homeopathy, acupuncture) are not recommended for insomnia treatment (weak recommendation, very-low-quality evidence).

I think, I can rest my case.

Shinrin-yoku means “taking in the forest atmosphere” or “forest bathing.” It was developed in Japan during the 1980s and has, according to its proponents, become a cornerstone of preventive health care and healing in Japanese medicine. Researchers primarily in Japan and South Korea have established a robust body of scientific literature on the health benefits of spending time under the canopy of a living forest… there have been many scientific studies that are demonstrating the mechanisms behind the healing effects of simply being in wild and natural areas. (some of this research is available here). For example, many trees give off organic compounds that support our “NK” (natural killer) cells that are part of our immune system’s way of fighting cancer.

The claimed benefits of Shinrin-yoku are remarkable:

  • Boosted immune system functioning, with an increase in the count of the body’s Natural Killer (NK) cells.
  • Reduced blood pressure
  • Reduced stress
  • Improved mood
  • Increased ability to focus, even in children with ADHD
  • Accelerated recovery from surgery or illness
  • Increased energy level
  • Improved sleep
  • Deeper and clearer intuition
  • Increased flow of energy
  • Increased capacity to communicate with the land and its species
  • Increased flow of eros/life force
  • Deepening of friendships
  • Overall increase in sense of happiness

But is any of this really true?

The aim of this state-of-the-art review was to summarise empirical research conducted on the physiological and psychological effects of Shinrin-Yoku. Research published from 2007 to 2017 was considered. A total of 64 studies met the inclusion criteria. According to the authors, they show that health benefits associated with the immersion in nature continue to be currently researched. Longitudinal research, conducted worldwide, is needed to produce new evidence of the relationships associated with Shinrin-Yoku and clinical therapeutic effects. Nature therapy as a health-promotion method and potential universal health model is implicated for the reduction of reported modern-day “stress-state” and “technostress.”



A look at the primary studies reveals that they are usually small and of poor quality.

Perhaps a brand new  review aimed more specifically at evaluating preventive or therapeutic effects of Shinrin-Yoku on blood pressure can tell us more. The authors considered all published, randomized, controlled trials, cohort studies, and comparative studies that evaluated the effects of the forest environment on changes in systolic blood pressure. Twenty trials involving 732 participants were reviewed. Systolic and diastolic blood pressure of patients submitted to the forest environment was significantly lower than that of controls. The authors concluded that this systematic review shows a significant effect of Shinrin-yoku on reduction of blood pressure.

I find this paper odd as well:

  • it lacks important methodological detail;
  • the authors included not just controlled clinical trials but all sorts of ‘studies’;
  • there is no assessment of the methodological rigor of the primary trials (from what I could see, they were mostly too poor to draw any conclusions from them).

What does all of this mean?

I have no problems in assuming that relaxation in a forest is beneficial in many ways and a nice experience.

But why call this a therapy?

It is relaxation!

Why make so many unsubstantiated claims?

And why study it in such obviously flawed ways?

All this does, I fear, is giving science a bad name.

The UK ‘Faculty of Homeopathy’ (FoH) is the professional body of British doctors who specialise in homeopathy. As doctors, FoH members have been to medical school and should know about evidence, science etc., I had always thought. But perhaps I was mistaken?

The FoH has a website with an interesting new post entitled ‘Scientific evidence and Homeopathy’. Here I have copied the section on CLINICAL TRIALS OF HOMEOPATHY. I have read it several times and must admit: it is a masterpiece, in my view – not a masterpiece in accurate reporting, but a masterpiece in misleading the public. The first and most obvious thing that struck me is the fact that is cites not a single clinical trial. But read for yourself (the numbers in round brackets were inserted by me and refer to my comments below):


By August 2017 1,138 clinical trials of homeopathy had been published (1). Details can be found on the CORE-HOM database also maintained by the Carstens Foundation and accessible without charge:

Four (2) systematic review/meta-analyses of homeopathy for all conditions have been published.[26],[27],[28]  Of these, three (3) reached a positive conclusion: that there is evidence that homeopathy is clinically effective (4). The exception is the review by Shang et al.46  This meta-analysis was controversial, particularly because its conclusions were based on only eight clinical trials whose identity was concealed until several months after the publication, precluding informed examination of its results (5) (6). The only undisputed conclusion (7) of this paper is that clinical trials of homeopathy are of higher quality than matched trials of conventional medicine: of 110 clinical trials each of homeopathy and conventional medicine, 21 trials of homeopathy but only 9 trials of conventional medicine were of ‘higher quality’.[29] [30]

A leading Swedish medical researcher (8) remarked: To conclude that homeopathy lacks clinical effect, more than 90% of the available clinical trials had to be dis­regarded.  Alternatively, flawed statistical methods had to be applied.”[31] Higher quality equates to less risk of bias, Mathie et al analysed randomized clinical trials of individualized homeopathy, showing that the highest quality trials yielded positive results (9).[32]

Systematic reviews of randomized controlled trials of homeopathy in specific clinical situations have also yielded positive results, including: allergies and upper respiratory tract infections (2 systematic reviews),[33],[34] (10) (11) Arnica in knee surgery,[35] (12) Childhood diarrhoea,[36] Post-operative ileus,[37] (13) Rheumatic diseases,[38] (14) Seasonal allergic rhinitis (hay fever) (2 systematic reviews),[39] [40] (15) (16) and vertigo.[41] (17)



  1. This is a wild exaggeration which was made possible by counting all sorts of clinical reports as ‘clinical trials’. A clinical trial  “follows a pre-defined plan or protocol to evaluate the effects of a medical or behavioral intervention on health outcomes.” This would exclude most observational studies, case series, case reports. However, the figure cited here includes such reports.
  2. The author cites only three!
  3. Does the author mean ‘two’?
  4. This is not quite true! I have dedicated an entire post to this issue.
  5. True, the Shang meta-analysis has been criticised – but exclusively by homeopaths who, for obvious reasons, were unable to accept its negative findings. In fact, it is a solid piece of research.
  6. Why does the author not mention the most recent systematic review of homeopathy?  Perhaps because it concluded: Homeopathy should not be used to treat health conditions that are chronic, serious, or could become serious. People who choose homeopathy may put their health at risk if they reject or delay treatments for which there is good evidence for safety and effectiveness. People who are considering whether to use homeopathy should first get advice from a registered health practitioner. Those who use homeopathy should tell their health practitioner and should keep taking any prescribed treatments.
  7. Really? Undisputed? Even by the logic of the author’s last sentence, this would be disputed.
  8. The ‘leading researcher’ is Prof Hahn who has featured many times on my blog. He seems to be more than a little unhinged when it comes to the topic of homeopathy.
  9. The author forgot to mention that Mathie – who was sponsored by the British Homeopathic Association – included this little caveat in his conclusions: The low or unclear overall quality of the evidence prompts caution in interpreting the findings.
  10. Reference 33 is the infamous ‘Swiss report’ that has been shown to be fatally flawed over and over again.
  11. Reference 34 refers to a review that fails to adhere to almost all the criteria of a systematic review.
  12. This review concluded: In all three trials, patients receiving homeopathic arnica showed a trend towards less postoperative swelling compared to patients receiving placebo. However, a significant difference in favour of homeopathic arnica was only found in the CLR trial. Only a deluded homeopath can call this a ‘positive result’.
  13. This is a systematic review by my team. It showed that several flawed trials produced a false positive result, while the only large multicentre trial was negative. Our conclusions therefore include the statement that  several caveats preclude a definitive judgment. Only a deluded homeopath can call this a ‘positive result’.
  14. This reference refers to the following abstract: Despite a growing interest in uncovering the basic mechanisms of arthritis, medical treatment remains symptomatic. Current medical treatments do not consistently halt the long-term progression of these diseases, and surgery may still be needed to restore mechanical function in large joints. Patients with rheumatic syndromes often seek alternative therapies, with homeopathy being one of the most frequent. Homeopathy is one of the most frequently used complementary therapies worldwide. Only a deluded homeopath can call this a ‘positive result’.
  15. The first reference refers to a paper where the author analysed three of his own studies.
  16. Reference 40 refers to a review that fails to adhere to almost all the criteria of a systematic review.
  17. This reference refers to a review of Vertigoheel@ that includes observational studies. One of its authors was an employee of the manufacturer of the product. Vertigoheel is not a homeopathic remedy (it does not adhere to the ‘like cures like’ principle) but a homotoxicologic product. Homotoxicology is a method inspired by homeopathy which was developed by Hans Heinrich Reckeweg (1905 – 1985). He believed that all or most illness is caused by an overload of toxins in the body. The toxins originate, according to Reckeweg, both from the environment and from the malfunction of physiological processes within the body. His treatment consists mainly in applying homeopathic remedies which usually consist of combinations of single remedies, because health cannot be achieved without ridding the body of toxins. The largest manufacturer and promoter of remedies used in homotoxicology is the German firm Heel. Our own systematic review of RCTs of homotoxicology included 7 trials which were mostly of a high methodological standard, according to the Jadad score. The trials tested the efficacy of seven different medicines for seven different indications. The results were positive in all but one study. Important flaws were found in all trials. These render the results of the primary studies less reliable than their high Jadad scores might suggest. Despite mostly positive findings and high ratings on the Jadad score, the placebo-controlled, randomised clinical trials of homotoxicology fail to demonstrate the efficacy of this therapeutic approach.


What do we make of all this?

To say that it is disappointing would, I think, be an understatement. The FoH is not supposed to be a lobby group of amateurs ignorant of science and evidence; it is a recognised professional organisation who must behave ethically. Patients and consumers should be able to trust the FoH. The fact that the FoH publish misinformation on such a scale should, in my view, be a matter for the General Medical Council.

On this blog, we have often discussed the risks of spinal manipulation. As I see it, the information we have at present suggests that

  • mild to moderate adverse effects are extremely frequent and occur in about half of all patients;
  • serious adverse effects are being reported regularly;
  • the occur usually with chiropractic manipulations of the neck (which are not of proven efficacy for any condition) and often relate to vascular accidents;
  • the consequences can be permanent neurological deficits and even deaths;
  • under-reporting of such cases might be considerable and therefore precise incidence figures are not available;
  • there is no system to accurately monitor the risks;
  • chiropractors are in denial of these problems.

Considering the seriousness of these issues, it is important to do more rigorous research. Therefore, any new paper published on this subject is welcome. A recent article might shed new light on the topic.

The objective of this systematic review was to identify characteristics of 1) patients, 2) practitioners, 3) treatment process and 4) adverse events (AE) occurring after cervical spinal manipulation (CSM) or cervical mobilization. A systematic searches were performed in 6 electronic databases up to December 2014. Of the initial 1043 articles thus located, 144 were included, containing 227 cases. 117 cases described male patients with a mean age of 45 and a mean age of 39 for females. Most patients were treated by chiropractors (66%). Manipulation was reported in 95% of the cases, and neck pain was the most frequent indication for the treatment. Cervical arterial dissection (CAD) was reported in 57%  of the cases and 45.8% had immediate onset symptoms. The overall distribution of gender for CAD was 55% for female. Patient characteristics were described poorly. No clear patient profile, related to the risk of AE after CSM, could be extracted, except that women seemed more at risk for CAD. The authors of this review concluded that there seems to be under-reporting of cases. Further research should focus on a more uniform and complete registration of AE using standardized terminology.

This article provides little new information; but it does confirm what I have been saying since many years: NECK MANIPULATIONS ARE ASSOCIATED WITH SERIOUS RISKS AND SHOULD THEREFORE BE AVOIDED.

An article by Rabbi Yair Hoffman for the Five Towns Jewish Times caught my eye. Here are a few excerpts:

“I am sorry, Mrs. Ploni, but the muscle testing we performed on you indicates that your compatibility with your spouse is a 1 out of a possible 10 on the scale.”

“Your son being around his father is bad for his energy levels. You should seek to minimize it.”

“Your husband was born normal, but something happened to his energy levels on account of the vaccinations he received as a child. It is not really his fault, but he is not good for you.”

Welcome to the world of Applied Kinesiology (AK) or health Kinesiology… Incredibly, there are people who now base most of their life decisions on something called “muscle testing.” Practitioners believe or state that the body’s energy levels can reveal remarkable information, from when a bride should get married to whether the next Kinesiology appointment should be in one week or two weeks. Prices for a 45 minute appointment can range from $125 to $250 a session. One doctor who is familiar with people who engage in such pursuits remarked, “You have no idea how many inroads this craziness has made in our community.”

… AK (applied kinesiology) is system that evaluates structural, chemical, and mental aspects of health by using “manual muscle testing (MMT)” along with other conventional diagnostic methods. The belief of AK adherents is that every organ dysfunction is accompanied by a weakness in a specific corresponding muscle… Treatments include joint manipulation and mobilization, myofascial, cranial and meridian therapies, clinical nutrition, and dietary counselling. A manual muscle test is conducted by having the patient resist using the target muscle or muscle group while the practitioner applies force the other way. A smooth response is called a “strong muscle” and a response that was not appropriate is called a “weak response.” Like some Ouiji board out of the 1970’s, Applied Kinesiology is used to ask “Yes or No” questions about issues ranging from what type of Parnassa courses one should be taking, to what Torah music tapes one should listen to, to whether a therapist is worthwhile to see or not.

“They take everything with such seriousness – they look at it as if it is Torah from Sinai,” remarked one person familiar with such patients. One spouse of an AK patient was shocked to hear that a diagnosis was made concerning himself through the muscle testing of his wife – without the practitioner having ever met him… And the lines at the office of the AK practitioner are long. One husband holds a crying baby for three hours, while his wife attends a 45 minute session. Why so long? The AK practitioner let other patients ahead – because of emergency needs…


The article  is a reminder how much nonsense happens in the name of alternative medicine. AK is one of the modalities that is exemplary:

  • it is utterly implausible;
  • there is no good evidence that it works.

The only systematic review of AK was published in 2008 by a team known to be strongly in favour of alternative medicine. It included 22 relevant studies. Their methodology was poor. The authors concluded that there is insufficient evidence for diagnostic accuracy within kinesiology, the validity of muscle response and the effectiveness of kinesiology for any condition. 

Some AK fans might now say: absence of evidence is not evidence of absence!!! There is no evidence that AK does not work, and therefore we should give it the benefit of the doubt and use it.

This, of course, is absolute BS! Firstly, the onus is on those who claim that AK works to prove their assumption. Secondly, in responsible healthcare, we are obliged to employ those modalities for which the evidence is positive, while avoiding those for which the evidence fails to be positive.


It used to be called ‘good bedside manners’. The term is an umbrella for a range of attitudes and behaviours including compassion, empathy and conveying positive messages. What could be more obvious than the assumption that good bedside manners are better than bad ones?

But as sceptics, we need to doubt obvious assumptions! Where is the evidence? we need to ask. So, where is the evidence that positive messages have any clinical effects? A meta-analysis has tackled the issue, and the results are noteworthy.

The researchers aimed to estimate the efficacy of positive messages for pain reduction. They included RCTs of the effects of positive messages. Their primary outcome measures were differences in patient- or observer reported pain between groups who were given positive messages and those who were not. Of the 16 RCTs (1703 patients) that met the inclusion criteria, 12 trials had sufficient data for meta-analysis. The pooled standardized effect size was −0.31 (95% CI −0.61 to −0.01, P = 0.04, I² = 82%). The effect size remained positive but not statistically significant after we excluded studies considered to have a high risk of bias (standard effect size −0.17, 95% CI −0.54 to 0.19, P = 0.36, I² = 84%). The authors concluded that care of patients with chronic or acute pain may be enhanced when clinicians deliver positive messages about possible clinical outcomes. However, we have identified several limitations of the present study that suggest caution when interpreting the results. We recommend further high quality studies to confirm (or falsify) our result.

The 1st author of this paper published a comment in which he stated that our recent mega-study with 12 randomized trials confirmed that doctors who use positive language reduce patient pain by a similar amount to drugs. Other trials show that positive messages can:

• help Parkinson’s patients move their hands faster,
• increase ‘peak flow’ (a measure of how much air is breathed) in asthma patients,
• improve the diameter of arteries in heart surgery patients, and
• reduce the amount of pain medication patients use.

The way a positive message seems to help is biological. When a patient anticipates a good thing happening (for example that their pain will go away), this activates parts of the brain that help the body make its own drugs like endorphins. A positive doctor may also help a patient relax which can also improve health.

I am not sure that this is entirely correct. When the authors excluded the methodologically weak and therefore unreliable studies, the effect was no longer significant. That is to say, it was likely due to chance.

And what about the other papers cited above? I am not sure about them either. Firstly, they do not necessarily show that positive messages are effective. Secondly, there is just one study for each claim, and one swallow does not make a summer; we would need independent replications.

So, am I saying that being positive as a clinician is ineffective? No! I am saying that the evidence is too flimsy to be sure. And possibly, this means that the effect of positive messages is smaller than we all thought.

This recently published report provides updated clinical practice guidelines from the Society for Integrative Oncology on the use of integrative therapies for specific clinical indications during and after breast cancer treatment, including anxiety/stress, depression/mood disorders, fatigue, quality of life/physical functioning, chemotherapy-induced nausea and vomiting, lymphedema, chemotherapy-induced peripheral neuropathy, pain, and sleep disturbance.

The practice guidelines are based on a systematic literature review from 1990 through 2015. The recommendations are as follows:

  • Music therapy, meditation, stress management, and yoga are recommended for anxiety/stress reduction.
  • Meditation, relaxation, yoga, massage, and music therapy are recommended for depression/mood disorders.
  • Meditation and yoga are recommended to improve quality of life.
  • Acupressure and acupuncture are recommended for reducing chemotherapy-induced nausea and vomiting.
  • Acetyl-L-carnitine is not recommended to prevent chemotherapy-induced peripheral neuropathy due to a possibility of harm.
  • No strong evidence supports the use of ingested dietary supplements to manage breast cancer treatment-related side effects.

The authors conclude that there is a growing body of evidence supporting the use of integrative therapies, especially mind-body therapies, as effective supportive care strategies during breast cancer treatment. Many integrative practices, however, remain understudied, with insufficient evidence to be definitively recommended or avoided.

I have to admit that I am puzzled by this paper.

The first obvious point to make is that these treatments are not ‘integrative therapies’; they are alternative or complementary and I fail to see what is integrative about them.

The second point is that the positive recommendations are based on often poor-quality studies which did not control for placebo effects.

The third point is that the negative recommendations are woefully incomplete. There are many more alternative therapies for which there is no strong evidence.

The forth point is the conclusion implying that treatment supported by insufficient evidence should be avoided. I would not claim that any of the mentioned treatments is backed by SUFFICIENT evidence. Therefore, we should avoid them all, one might argue.

But these concerns are perhaps relatively trivial or far-fetched. More important is the fact that a very similar article been published in 2014. Here is the abstract:


The majority of breast cancer patients use complementary and/or integrative therapies during and beyond cancer treatment to manage symptoms, prevent toxicities, and improve quality of life. Practice guidelines are needed to inform clinicians and patients about safe and effective therapies.


Following the Institute of Medicine’s guideline development process, a systematic review identified randomized controlled trials testing the use of integrative therapies for supportive care in patients receiving breast cancer treatment. Trials were included if the majority of participants had breast cancer and/or breast cancer patient results were reported separately, and outcomes were clinically relevant. Recommendations were organized by outcome and graded based upon a modified version of the US Preventive Services Task Force grading system.


The search (January 1, 1990–December 31, 2013) identified 4900 articles, of which 203 were eligible for analysis. Meditation, yoga, and relaxation with imagery are recommended for routine use for common conditions, including anxiety and mood disorders (Grade A). Stress management, yoga, massage, music therapy, energy conservation, and meditation are recommended for stress reduction, anxiety, depression, fatigue, and quality of life (Grade B). Many interventions (n = 32) had weaker evidence of benefit (Grade C). Some interventions (n = 7) were deemed unlikely to provide any benefit (Grade D). Notably, only one intervention, acetyl-l-carnitine for the prevention of taxane-induced neuropathy, was identified as likely harmful (Grade H) as it was found to increase neuropathy. The majority of intervention/modality combinations (n = 138) did not have sufficient evidence to form specific recommendations (Grade I).


Specific integrative therapies can be recommended as evidence-based supportive care options during breast cancer treatment. Most integrative therapies require further investigation via well-designed controlled trials with meaningful outcomes.

I have harshly criticised this review on this blog in 2016. For instance, I voiced concern about the authors declaration of conflicts of interest and stated:


I know none of the authors (Heather Greenlee, Lynda G. Balneaves, Linda E. Carlson, Misha Cohen, Gary Deng, Dawn Hershman, Matthew Mumber, Jane Perlmutter, Dugald Seely, Ananda Sen, Suzanna M. Zick, Debu Tripathy) of the document personally. They made the following collective statement about their conflicts of interest: “There are no financial conflicts of interest to disclose. We note that some authors have conducted/authored some of the studies included in the review.” I am a little puzzled to hear that they have no financial conflicts of interest (do not most of them earn their living by practising integrative medicine? Yes they do! The article informs us that: “A multidisciplinary panel of experts in oncology and integrative medicine was assembled to prepare these clinical practice guidelines. Panel members have expertise in medical oncology, radiation oncology, nursing, psychology, naturopathic medicine, traditional Chinese medicine, acupuncture, epidemiology, biostatistics, and patient advocacy.”). I also suspect they have other, potentially much stronger conflicts of interest. They belong to a group of people who seem to religiously believe in the largely nonsensical concept of integrative medicine

The just-published update has a different statement about conflicts of interest:

DISCLOSURES: Linda E. Carlson reports book royalties from New Harbinger and the American Psychological Association. Misha R. Cohen reports royalties from Health Concerns Inc., outside the submitted work. Matthew Mumber owns stock in I Thrive. All remaining authors report no conflicts of interest.

Is this much better than the previous statement? Was the previous statement therefore false?

I wonder.

What do you think?

I have repeatedly cautioned about the often poor quality of research into alternative medicine. This seems particularly necessary with studies of acupuncture, and especially true for such research carried out in China. I have also frequently noted that certain ‘CAM journals’ are notoriously prone to publishing rubbish. So, what can we expect from a paper that:

  • is on alternative medicine,
  • focusses on acupuncture,
  • is authored by Chinese researchers,
  • was published in the Journal of Alternative and Complementary Medicine (JACM)?

The answer is PROBABLY NOT A LOT!

As if for confirming my prediction, The JACM just published this systematic review. It reports pairwise and network meta-analyses to determine the effectiveness of acupuncture and acupuncture-related techniques for the treatment of psoriasis. A total of 13 RCTs were included. The methodological quality of these studies was ‘not rigorous’ according to the authors – in fact, it was lousy. Acupoint stimulation seemed to be more effective than non-acupoint stimulation. The short-term treatment effect was superior to the long-term effect (as one would expect with placebo). Network meta-analysis suggested that acupressure or acupoint catgut embedding generate superior effects compared to medications. It was noted that acupressure was the most effective treatment of all the acupuncture-like therapies.

The authors concluded that acupuncture-related techniques could be considered as an alternative or adjuvant therapy for psoriasis in short term, especially of acupressure and acupoint catgut embedding. This study recommends further well-designed, methodologically rigorous, and more head-to-head randomized trials to explore the effects of acupuncture-related techniques for treating psoriasis.

And what is wrong with that?


  • The review is of very poor quality.
  • The primary studies are even worse.
  • The English language is defective to the point of being not understandable.
  • The conclusions are misleading.

Correct conclusions should read something like this: Due to the paucity and the poor quality of the clinical trials, this review could not determine whether acupuncture and similar therapies are effective for psoriasis.

And then there is, of course, the question about plausibility. How plausible is the assumption that acupuncture might affect a genetic autoimmune disease like psoriasis. The answer, I think, is that the assumption is highly unlikely.

In the above review, most of the 13 primary RCTs were from China. One of the few studies not conducted in China is this one:

56 patients suffering from long-standing plaque psoriasis were randomized to receive either active treatment (electrostimulation by needles placed intramuscularly, plus ear-acupuncture) or placebo (sham, ‘minimal acupuncture‘) twice weekly for 10 weeks. The severity of the skin lesions was scored (PASI) before, during, and 3 months after therapy. After 10 weeks of treatment the PASI mean value had decreased from 9.6 to 8.3 in the ‘active’ group and from 9.2 to 6.9 in the placebo group (p < 0.05 for both groups). These effects are less than the usual placebo effect of about 30%. There were no statistically significant differences between the outcomes in the two groups during or 3 months after therapy. The patient’s own opinion about the results showed no preference for ‘active’ therapy. It was also clear from the answers that the blinded nature of the study had not been discovered by the patients. In conclusion, classical acupuncture is not superior to sham (placebo) ‘minimal acupuncture‘ in the treatment of psoriasis.

Somehow, I trust these conclusions more than the ones from the review!

And somehow, I get very tired of journal editors failing to do their job of rejecting papers that evidently are embarrassing, unethical rubbish.

Systematic reviews are aimed at summarising and critically evaluating the evidence on a specific research question. They are the highest level of evidence and are more reliable than anything else we have. Therefore, they represent a most useful tool for both clinicians and researchers.

But there are, of course, exceptions. Take, for instance, this recent systematic review by researchers from the

  • Texas Chiropractic College, Pasadena, the Palmer Center for Chiropractic Research, Palmer College of Chiropractic, Davenport,
  • Department of Planning, Policy and Design, University of California, Irvine,
  • VA Puget Sound Health Care System, Tacoma,
  • New York Chiropractic College, Seneca Falls,
  • Logan University College of Chiropractic, Chesterfield,
  • University of Western States, Portland.

Its purpose was to evaluate the effectiveness of conservative non-drug, non-surgical interventions, either alone or in combination, for conditions of the shoulder. The review was conducted from March 2016 to November 2016 in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), and was registered with PROSPERO. Eligibility criteria included randomized controlled trials (RCTs), systematic reviews, or meta-analyses studying adult patients with a shoulder diagnosis. Interventions qualified if they did not involve prescription medication or surgical procedures, although these could be used in the comparison group or groups. At least 2 independent reviewers assessed the quality of each study using the Scottish Intercollegiate Guidelines Network checklists. Shoulder conditions addressed were

  • shoulder impingement syndrome (SIS),
  • rotator cuff-associated disorders (RCs),
  • adhesive capsulitis (AC),
  • nonspecific shoulder pain.

Twenty-five systematic reviews and 44 RCTs met inclusion criteria. Low- to moderate-quality evidence supported the use of manual therapies for all 4 shoulder conditions. Exercise, particularly combined with physical therapy protocols, was beneficial for SIS and AC. For SIS, moderate evidence supported several passive modalities. For RC, physical therapy protocols were found beneficial but not superior to surgery in the long term. Moderate evidence supported extracorporeal shockwave therapy for calcific tendinitis RC. Low-level laser was the only modality for which there was moderate evidence supporting its use for all 4 conditions.

The authors concluded that the findings of this literature review may help inform practitioners who use conservative methods (eg, doctors of chiropractic, physical therapists, and other manual therapists) regarding the levels of evidence for modalities used for common shoulder conditions.

This review has so many defects that it would be boring to list them here.

The PRISMA guidelines  – I happen to be a co-author – state, for instance, that the abstract (the above text is from the abstract) should provide a structured summary including, as applicable: background; objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions and implications of key findings; systematic review registration number. It is obvious that the review authors have omitted several of these items.

And that is just the abstract!  There is much, much more to criticise in this paper.

The most striking deficit, in my view, is the useless conclusion: the one from the abstract (the part of the paper that will be read most widely) could have been written before the review had even been started. It is therefore not based on the data presented. Crucially it does not match the stated aim of this review (“to evaluate the effectiveness of conservative…interventions”).

But why? Why did the authors bother to follow PRISMA? Why did they formulate this bizarre conclusion in their abstract? Why did they do a review in the first place?

I fear, the answers might be embarrassingly simple:

  • They only pretended to follow PRISMA guidelines because that gives their review a veneer of respectability.
  • They formulated the conclusions because otherwise they would have needed to state that the evidence for manual therapy is less than convincing.
  • They conducted the review to promote chiropractic, and when the data were not as they had hoped for, they just back-paddled in an attempt to hide the truth as much as possible.

If this were an isolated case, I would not have bothered to mention it. But sadly, in the realm of chiropractic (and alternative medicine in general) we currently witness a plethora of rubbish reviews (published by rubbish journals). To the naïve observer, they might look rigorous and therefore they will be taken seriously. The end-effect of this pollution of the literature with rubbish is that we get a false-positive impression about the validity of the treatments in question. Consequently, we will see a host of wrong decisions on all levels of healthcare.


I only see one solution: completely disregard certain journals that have been identified to regularly publish nonsense. Sadly, the wider medical community is far from having arrived at this point. As far as I can see, the problem has not even been identified yet as a serious issue that needs addressing. For the foreseeable future, we will probably have to live with this type of pollution of our medical literature.

1 2 3 18

Gravityscan Badge

Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted.

Click here for a comprehensive list of recent comments.