systematic review

As the data suggesting that homeopathy is effective for improving health is – to put it mildly – less than convincing, a frantic search is currently on amongst homeopaths and their followers to identify a specific condition for which the evidence is stronger than for all conditions pooled into one big analysis. If they could show that it works for just one disease, they could celebrate this finding and henceforth use it for refuting doubters stating that highly diluted homeopathic remedies are pure placebos. One such condition is allergic rhinitis; there have been several trials suggesting that homeopathy might be effective for it, and therefore it is only logical that homeopathy-promoters want to summarise these data in order to silence sceptics once and for all.

A new paper ought to be seen in this vein. It is systematic review by the Mathie group with the stated aim “to evaluate the efficacy and effectiveness of homeopathic intervention in the treatment of seasonal or perennial allergic rhinitis (AR).”

Randomized controlled trials evaluating all forms of homeopathic treatment for AR were included in a systematic review (SR) of studies published up to and including December 2015. Two authors independently screened potential studies, extracted data, and assessed risk of bias. Primary outcomes included symptom improvement and total quality-of-life score. Treatment effect size was quantified as mean difference (continuous data), or by risk ratio (RR) and odds ratio (dichotomous data), with 95% confidence intervals (CI). Meta-analysis was performed after assessing heterogeneity and risk of bias.

Eleven studies were eligible for SR. All trials were placebo-controlled except one. Six trials used the treatment approach known as isopathy, but they were unsuitable for meta-analysis due to problems of heterogeneity and data extraction. The overall standard of methods and reporting was poor: 8/11 trials were assessed as “high risk of bias”; only one trial, on isopathy for seasonal AR, possessed reliable evidence. Three trials of variable quality (all using Galphimia glauca for seasonal AR) were included in the meta-analysis: nasal symptom relief at 2 and 4 weeks (RR = 1.48 [95% CI 1.24-1.77] and 1.27 [95% CI 1.10-1.46], respectively) favoured homeopathy compared with placebo; ocular symptom relief at 2 and 4 weeks also favoured homeopathy (RR = 1.55 [95% CI 1.33-1.80] and 1.37 [95% CI 1.21-1.56], respectively). The single trial with reliable evidence had a small positive treatment effect without statistical significance. A homeopathic and a conventional nasal spray produced equivalent improvements in nasal and ocular symptoms.

The authors concluded that the low or uncertain overall quality of the evidence warrants caution in drawing firm conclusions about intervention effects. Use of either Galphimia glauca or a homeopathic nasal spray may have small beneficial effects on the nasal and ocular symptoms of AR. The efficacy of isopathic treatment of AR is unclear.

Extracts of Galphimia glauca (GG) have been used traditionally in South America for the treatment of allergic conditions, with some reports suggesting effectiveness. A 1997 meta-analysis of 11 clinical trials (most of them of very poor quality) of homeopathic GG suggested this therapy to be effective in the treatment of AR. In 2011, I published a review (FACT 2011, 16 200-203) focussed exclusively on the remarkable set of RCTs of homeopathic Galphimia glauca (GG). My conclusions were as follows: three of the four currently available placebo-controlled RCTs of homeopathic GG suggest this therapy is an effective symptomatic treatment for hay fever. There are, however, important caveats. Most essentially, independent replication would be required before GG can be considered for the routine treatment of hay fever. Since then, no new studies have emerged.

I am citing this for two main reasons:

  • There is nothing homeopathic about the principle of using GG for allergic conditions; according to homeopathic theory GG extracts would need to cause allergies for GG to have potential as a homeopathic allergy remedy. Arguably, the GG trials should therefore have been excluded from this meta-analysis for not following the homeopathic principal of ‘like cures like’.
  • All the RCTs of GG were done by the same German research group. There is not a single independent replication of their findings!

Seen from this perspective, the conclusion by Mathie et al, that the use of either Galphimia glauca … may have small beneficial effects on the nasal and ocular symptoms of AR, seems more than a little over-optimistic.

The question whether spinal manipulative therapy (SMT) is effective for acute low back pain is still discussed controversially. Chiropractors (they use SMT more regularly than other professionals) try everything to make us believe it does work, while the evidence is far less certain. Therefore, it is worth considering the best and most up-to-date data.

The  aim of this paper was to systematically review studies of the effectiveness and harms of SMT for acute (≤6 weeks) low back pain. The research question was straight forward: Is the use of SMT in the management of acute (≤6 weeks) low back pain associated with improvements in pain or function?

A through literature search was conducted to locate all relevant papers. Study quality was assessed using the Cochrane Back and Neck (CBN) Risk of Bias tool. The evidence was assessed using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) criteria. The main outcome measures were pain (measured by either the 100-mm visual analog scale, 11-point numeric rating scale, or other numeric pain scale), function (measured by the 24-point Roland Morris Disability Questionnaire or Oswestry Disability Index [range, 0-100]), or any harms measured within 6 weeks.

Of 26 eligible RCTs identified, 15 RCTs (1711 patients) provided moderate-quality evidence that SMT has a statistically significant association with improvements in pain (pooled mean improvement in the 100-mm visual analog pain scale, −9.95 [95% CI, −15.6 to −4.3]). Twelve RCTs (1381 patients) produced moderate-quality evidence that SMT has a statistically significant association with improvements in function (pooled mean effect size, −0.39 [95% CI, −0.71 to −0.07]). Heterogeneity was not explained by type of clinician performing SMT, type of manipulation, study quality, or whether SMT was given alone or as part of a package of therapies. No RCT reported any serious adverse event. Minor transient adverse events such as increased pain, muscle stiffness, and headache were reported 50% to 67% of the time in large case series of patients treated with SMT.

The authors concluded that among patients with acute low back pain, spinal manipulative therapy was associated with modest improvements in pain and function at up to 6 weeks, with transient minor musculoskeletal harms. However, heterogeneity in study results was large.

This meta-analysis has been celebrated by chiropractors around the world as a triumph for their hallmark therapy, SMT. But there have also been more cautionary voices – not least from the lead author of the paper. Patients undergoing spinal manipulation experienced a decline of 1 point in their pain rating, says Dr. Paul Shekelle, an internist with the West Los Angeles Veterans Affairs Medical Center and the Rand Corporation who headed the study. That’s about the same amount of pain relief as from NSAIDs, over-the-counter nonsteroidal anti-inflammatory medication, such as ibuprofen. The study also found spinal manipulation modestly improved function. On average, patients reported greater ease and comfort engaging in two day-to-day activities — such as finding they could walk more quickly, were having less difficulty turning over in bed or were sleeping more soundly.

It’s not clear exactly how spinal manipulation relieves back pain. But it may reposition the small joints in the spine in a way that causes less pain, according to Dr. Richard Deyo, an internist and professor of evidence-based medicine at the Oregon Health and Science University. Deyo wrote an editorial published along with the study. Another possibility, Deyo says, is that spinal manipulation may restore some material in the disk between the vertebrae, or it may simply relax muscles, which could be important. There may also be mind-body interaction that comes from the “laying of hands” or a trusting relationship between patients and their health care provider, he says.

Deyo notes that there are many possible treatments for lower back pain, including oral medicine, injected medicine, corsets, traction, surgery, acupuncture and massage therapy. But of about 200 treatment options, “no single treatment is clearly superior,” he says.

In another comment by Paul Ingraham the critical tone was much clearer: “Claiming it as a victory is one of the best examples I’ve ever seen of making lemonade out of science lemons! But I can understand the mistake, because the review itself does seem positive at first glance: the benefits of SMT are disingenuously summarized as “statistically significant” in the abstract, with no mention of clinical significance (effect size; see Statistical Significance Abuse). So the abstract sounds like good news to anyone but the most wary readers, while deep in the main text the same results are eventually conceded to be “clinically modest.” But even even that seems excessively generous: personally, I need at least a 2-point improvement in pain on a scale of 10 to consider it a “modest” improvement! This is not a clearly positive review: it shows weak evidence of minor efficacy, based on “significant unexplained heterogeneity” in the results. That is, the results were all over the place — but without any impressive benefits reported by any study — and the mixture can’t be explained by any obvious, measurable factor. This probably means there’s just a lot of noise in the data, too many things that are at least as influential as the treatment itself. Or — more optimistically — it could mean that SMT is “just” disappointingly mediocre on average, but might have more potent benefits in a minority of cases (that no one seems to be able to reliably identify). Far from being good news, this review continues a strong trend (eg Rubinstein 2012) of damning SMT with faint praise, and also adds evidence of backfiring to mix. Although fortunately “no RCT reported any serious adverse event,” it seems that minor harms were legion: “increased pain, muscle stiffness, and headache were reported 50% to 67% of the time in large case series of patients treated with SMT.” That’s a lot of undesirable outcomes. So the average patient has a roughly fifty-fifty chance of up to roughly maybe a 20% improvement… or feeling worse to some unknown degree! That does not sound like a good deal to me. It certainly doesn’t sound like good medicine.”


As I have made clear in many previous posts, I do fully agree with these latter statements and would add just three points:

  1. We know that many of the SMT studies completely neglect reporting adverse effects. Therefore it is hardly surprising that no serious complications were on record. Yet, we know that they do occur with sad regularity.
  2. None of the studies controlled for placebo effects. It is therefore possible – I would say even likely – that a large chunk of the observed benefit is not due to SMT per se but to a placebo response.
  3. It seems more than questionable whether the benefits of SMT outweigh its risks.

‘What Doctors Don’t Tell You’ (WDDTY) have been shown to be strangely economical with the truth many times before (for instance here, here and here). Now they have published an article entitled ‘Ombudsman investigates ‘flawed’ homeopathic study that claimed it doesn’t work’ It attacks in no uncertain terms the ‘NHMRC Statement on Homeopathy and NHMRC Information Paper – Evidence on the effectiveness of homeopathy for treating health conditions’ which I believe to be a sound evaluation of homeopathy and therefore have mentioned repeatedly on this blog. Here is what WDDTY stated:


A major and influential review of homeopathy concluded that the controversial therapy doesn’t work—but it was so riddled with error and bad science that it’s sparked an official ombudsman investigation.

The world’s media announced that homeopathy was a scam after the Australia government’s National Health and Medical Research Council (NHMRC) published its findings in 2015 that “there are no health conditions for which there is reliable evidence that homeopathy is effective.”

But now the Commonwealth Ombudsman is investigating the review’s procedures after receiving reports of inaccuracies, mishandling of evidence and conflicts of interest.

The review has been triggered by the Australian Homeopathic Association (AHA), supported by the Homeopathic Research Institute (HRI), which began questioning the review’s processes after several solid studies that demonstrated homeopathy’s benefits had been overlooked.

The NHMRC review team set arbitrary parameters that only studies that involved more than 150 people—and which met standards that even drug trials rarely achieve—would be considered. Those requirements reduced the number of qualifying studies to just five—from an initial pool of more than 1,800 trials—and none of these showed that homeopathy was effective.

One of the NHMRC’s own reviewers produced a mysterious first report that has never been published, and hasn’t been released despite Freedom of Information requests.

And the AHA has discovered that Prof Peter Brooks, chair of the NHMRC committee that carried out the homeopathy review, never declared that he was a member of the anti-homeopathy lobby group, Friends of Science in Medicine.

There are solid studies that demonstrate homeopathy is effective against childhood diarrhea, sinusitis and hay fever—but they all involve fewer than 150 people, said HRI chief executive Rachel Roberts. “The public has a right to know that there are high quality studies showing homeopathy works for some medical conditions—information that was lost only due to NHMRC’s mishandling of the evidence.”

The homeopaths aren’t alone in challenging the NHMRC review: Australia’s independent Cochrane Centre said its conclusions are not an accurate reflection of the evidence, and a second expert also said he felt “uncertain of the definitive nature of the report’s conclusions.”


As it happens, I am in contact with the lead author of this report, Paul Glasziou, not least because he very kindly wrote the foreword for my book HOMEOPATHY, THE UNDILUTED FACTS. So, we corresponded and discussed the latest WDDTY diatribe. Thus I am now in a position to put a few things straight (I hope Paul does not mind).

ISSUE 1. – The NHMRC review team set arbitrary parameters that only studies involving more than 150 people—and which met standards that even drug trials rarely achieve—would be considered.

The truth is that report focused on systematic reviews of trials, not individual trials. The 57 included systematic reviews found 176 individual trials which covered 61 conditions: an average of about 3 trials per condition. But some conditions only had 1 trial, and one small trial would, of course, not be considered a reasonable basis for reliable conclusions. GRADE – the international standard for assessing evidence – downgrades reviews for “imprecision” – the GRADE Handbook suggests “whenever there are sample sizes that are less than 400, review authors and guideline developers should certainly consider rating down for imprecision.” Hence the criterion of 150 which the Australians decided to use is considerably more lenient than the current GRADE guideline.

ISSUE 2 – Those requirements reduced the number of qualifying studies to just five—from an initial pool of more than 1,800 trials—and none of these showed that homeopathy was effective.

This is simply not correct. The report found 57 systematic reviews that contained 176 individual trials, not 5. These 176 trials, which covered 61 conditions, formed the body of evidence for the NHMRC report’s conclusions.

ISSUE 3 – There are solid studies that demonstrate homeopathy is effective against childhood diarrhoea, sinusitis and hay fever—but they all involve fewer than 150 people, said HRI chief executive Rachel Roberts.

The NHMRC report focused on systematic reviews that covered all trials for individual conditions. Given the conventional p-value of 0.05, one would expect 1 in 20 single trials to be “false positives”. So with 176 trials, we expect about 9 “false positive” trials. But using systematic reviews that combine all trials for individual conditions, reduces this risk of false positives. Most national evidence review bodies require more than 1 trial, e.g, the FDA requires 2 positive trials, whereas many others require a systematic review which has at least 2 trials. Replication of findings is obviously a cornerstone of science.

ISSUE 4 The homeopaths aren’t alone in challenging the NHMRC review: Australia’s independent Cochrane Centre said its conclusions are not an accurate reflection of the evidence, and a second expert also said he felt “uncertain of the definitive nature of the report’s conclusions.”

The truth is that the Cochrane Centre, which provided an independent check during the processes of the NHMRC review, concluded that “Overall, the conclusions arising from the review appear justified based on the evidence presented.”



This new systematic review by proponents of homeopathy (and supported by a grant from the Manchester Homeopathic Clinic) tested the null hypothesis that “the main outcome of treatment using a non-individualised (standardised) homeopathic medicine is indistinguishable from that of placebo“. An additional aim was to quantify any condition-specific effects of non-individualised homeopathic treatment. In reporting this paper, I will stay very close to the published text hoping that this avoids both misunderstandings and accusations of bias on my side:

Literature search strategy, data extraction and statistical analysis followed the methods described in a pre-published protocol. A trial comprised ‘reliable evidence’ if its risk of bias was low or it was unclear in one specified domain of assessment. ‘Effect size’ was reported as standardised mean difference (SMD), with arithmetic transformation for dichotomous data carried out as required; a negative SMD indicated an effect favouring homeopathy.

The authors excluded the following types of trials: studies of crossover design; of radionically prepared homeopathic medicines; of homeopathic prophylaxis; of homeopathy combined with other (complementary or conventional) intervention; for other specified reasons. The final explicit exclusion criterion was that there was obviously no blinding of participants and practitioners to the assigned intervention.

Forty-eight different clinical conditions were represented in 75 eligible RCTs; 49 were classed as ‘high risk of bias’ and 23 as ‘uncertain risk of bias’; the remaining three trials displayed sufficiently low risk of bias to be designated reliable evidence. Fifty-four trials had extractable data: pooled SMD was -0.33 (95% confidence interval (CI) -0.44, -0.21), which was attenuated to -0.16 (95% CI -0.31, -0.02) after adjustment for publication bias. The three trials with reliable evidence yielded a non-significant pooled SMD: -0.18 (95% CI -0.46, 0.09). There was no single clinical condition for which meta-analysis produced reliable evidence.

A meta-regression was performed to test specifically for within-group differences for each sub-group. The results showed that there were no significant differences between studies that were and were not:

  • included in previous meta-analyses (p = 0.447);
  • pilot studies (p = 0.316);
  • greater than the median sample (p = 0.298);
  • potency ≥ 12C (p = 0.221);
  • imputed for meta-analysis (p = 0.384);
  • free from vested interest (p = 0.391);
  • acute/chronic (p = 0.796);
  • different types of homeopathy (p = 0.217).

After removal of ‘C’-rated trials, the pooled SMD still favoured homeopathy for all sub-groups, but was statistically non-significant for 10 of the 18 (included in previous meta-analysis; pilot study; sample size > median; potency ≥12C; data imputed; free of vested interest; not free of vested interest; combination medicine; single medicine; chronic condition). There remained no significant differences between sub-groups—with the exception of the analysis for sample size > median (p = 0.028).

Meta-analyses were possible for eight clinical conditions, each analysis comprising two to 5 trials. A statistically significant pooled SMD, favouring homeopathy, was observed for influenza (N = 2), irritable bowel syndrome (N = 2), and seasonal allergic rhinitis (N = 5). Each of the other five clinical conditions (allergic asthma, arsenic toxicity, infertility due to amenorrhoea, muscle soreness, post-operative pain) showed non-significant findings. Removal of ‘C’-rated trials negated the statistically significant effect for seasonal allergic rhinitis and left the non-significant effect for post-operative pain unchanged; no higher-rated trials were available for additional analysis of arsenic toxicity, infertility due to amenorrhoea or irritable bowel syndrome. There were no ‘C’-rated trials to remove for allergic asthma, influenza, or muscle soreness. Thus, influenza was the only clinical condition for which higher-rated trials indicated a statistically significant effect; neither of its contributing trials, however, comprised reliable evidence.

The authors concluded that the quality of the body of evidence is low. A meta-analysis of all extractable data leads to rejection of our null hypothesis, but analysis of a small sub-group of reliable evidence does not support that rejection. Reliable evidence is lacking in condition-specific meta-analyses, precluding relevant conclusions. Better designed and more rigorous RCTs are needed in order to develop an evidence base that can decisively provide reliable effect estimates of non-individualised homeopathic treatment.

I am sure that this paper will lead to lively discussions in the comments section of this blog. I will therefore restrict my comments to a bare minimum.

In my view, this new meta-analysis essentially yield a negative result and confirms most previous, similar reviews.

  • It confirms Linde’s conclusion that “insufficient evidence from these studies that homeopathy is clearly efficacious for any single clinical condition”.
  • It confirms Linde’s conclusion that “there was clear evidence that studies with better methodological quality tended to yield less positive results”.
  • It confirms Kleinjen’s conclusion that “most trials are of low methodological quality”.
  • It also confirms the results of the meta-analysis by Shang et al (much-maligned by homeopaths) than “finding is compatible with the notion that the clinical effects of homoeopathy are placebo effects.”
  • Finally, it confirms the conclusion of the analysis of the Australian National Health and Medical Research Council: “Homeopathy should not be used to treat health conditions that are chronic, serious, or could become serious. People who choose homeopathy may put their health at risk if they reject or delay treatments for which there is good evidence for safety and effectiveness. People who are considering whether to use homeopathy should first get advice from a registered health practitioner. Those who use homeopathy should tell their health practitioner and should keep taking any prescribed treatments.”

Another not entirely unimportant point that often gets missed in these discussions is this: even if we believe (which I do not) the most optimistic interpretation of these (and similar data) by homeopaths, we ought to point out that there is no evidence whatsoever that homeopathy cures anything. At the very best it provides marginal symptomatic relief. Yet, the claim of homeopaths that we hear constantly is that homeopathy is a causal and curative therapy.

The first author of the new meta-analysis is an employee of the Homeopathy Research Institute. We might therefore forgive him that he he repeatedly insists on dwelling on largely irrelevant (i. e. based on unreliable primary studies) findings. It seems obvious that firm conclusions can only be based on reliable data. I therefore disregard those analyses and conclusions that include such studies.

In the discussion, the authors of the new meta-analysis confirm my interpretation this by stating that they “reject the null hypothesis (non-individualised homeopathy is indistinguishable from placebo) on the basis of pooling all studies, but fail to reject the null hypothesis on the basis of the reliable evidence only.” And, in the long version of their conclusions, we find this remarkable statement: “Our meta-analysis of the current reliable evidence base therefore fails to reject the null hypothesis that the outcome of treatment using a non-individualised homeopathic medicine is not distinguishable from that using placebo.” A most torturous way of stating the obvious: the more reliable data show no difference between homeopathy and placebo.

Tui Na is a massage technique that is based on the Taoist principles of TCM. It involves a range of manipulations usually performed by an operator’s finger, hand, elbow, knee, or foot applied to muscle or soft tissue at specific parts of the body. According to one website of TCM-proponents “Tui Na makes use of various hand techniques in combination with acupuncture and other manipulation techniques. To enhance the healing process, the practitioner may recommend the use of Chinese herbs. Many of the techniques used in this massage resemble that of a western massage like gliding, kneading, vibration, tapping, friction, pulling, rolling, pressing and shaking. In Tui Na massage, the muscles and tendons are massaged with the help of hands, and an acupressure technique is applied to directly affect the flow of Qi at different acupressure points of the body, thus facilitating the healing process. It removes the blockages and keeps the energy moving through the meridians as well as the muscles. A typical session of Tui Na massage may vary from thirty minutes to an hour. The session timings may vary depending on the patient’s needs and condition. The best part of the therapy is that it relaxes as well as energizes the person. The main benefit of Tui Na massage is that it focuses on the specific problem, whether it is an acute or a chronic pain associated with the joints, muscles or a skeletal system. This technique is very beneficial in reducing the pain of neck, shoulders, hips, back, arms, highs, legs and ankle disorders. It is a very effective therapy for arthritis, pain, sciatica and muscle spasms. Other benefits of this massage therapy include alleviation of the stress related disorders like insomnia, constipation, headaches and other disorders related to digestive, respiratory and reproductive systems. The greatest advantage of Tui Na is that it focuses on maintaining overall balance with both physical and mental health. Any one who wants to avoid the side effects of drugs or a chemical based treatment can adopt this effective massage technique to alleviate their pain. Tui Na massage therapy is now becoming a more common therapy method due to its focus on specific problems rather than providing a general treatment.”

This clearly begs the question IS IT EFFECTIVE?

This systematic review assessed the evidence of Tui Na for cervical radiculopathy. Seven databases were searched. Randomised controlled trials (RCTs) incorporating Tui Na alone or Tui Na combined with conventional treatment were included. Five studies involving 448 patients were found. The pooled analysis from the 3 trials indicated that Tui Na alone showed a significant lowering immediate effects on pain score with moderate heterogeneity compared to cervical traction. The meta-analysis from 2 trials revealed significant immediate effects of Tui Na plus cervical traction in improving pain score with no heterogeneity compared to cervical traction alone. None of the RCTs mentioned adverse effects. There was very low quality or low quality evidence to support the results.

The authors concluded that “Tui Na alone or Tui Na plus cervical traction may be helpful to cervical radiculopathy patients, but supportive evidence seems generally weak. Future clinical studies with low risk of bias and adequate follow-up design are recommended.”

In my view, this is a misleading conclusion. A correct one would have been: THE CURRENT EVIDENCE IS INSUFFICIENT TO DRAW ANY CONCLUSIONS ABOUT THE EFFECTIVENESS OF TUI NA.


Here are some of the most obvious reasons:

Personally, I am getting very tired of conclusions stating ‘…XY MAY BE EFFECTIVE/HELPFUL/USEFUL/WORTH A TRY…’ It is obvious that the therapy in question MAY be effective, otherwise one would surely not conduct a systematic review. If a review fails to produce good evidence, it is the authors’ ethical, moral and scientific obligation to state this clearly. If they don’t, they simply misuse science for promotion and mislead the public. Strictly speaking, this amounts to scientific misconduct.

Is spinal manipulative therapy (SMT) dangerous? This question has kept us on this blog busy for quite some time now. To me, there is little doubt that SMT can cause adverse effects some of which are serious. But many chiropractors seem totally unconvinced. Perhaps this new overview of reviews might help to clarify the issue. Its aim was to elucidate and quantify the risk of serious adverse events (SAEs) associated with SMT.

The authors searched five electronic databases from inception to December 8, 2015 and included reviews on any type of studies, patients, and SMT technique. The primary outcome was SAEs. The quality of the included reviews was assessed using a measurement tool to assess systematic reviews (AMSTAR). Since there were insufficient data for calculating incidence rates of SAEs, they used an alternative approach; the conclusions regarding safety of SMT were extracted for each review, and the communicated opinion were judged by two reviewers independently as safe, harmful, or neutral/unclear. Risk ratios (RRs) of a review communicating that SMT is safe and meeting the requirements for each AMSTAR item, were calculated.

A total of 283 eligible reviews were identified, but only 118 provided data for synthesis. The most frequently described adverse events (AEs) were stroke, headache, and vertebral artery dissection. Fifty-four reviews (46%) expressed that SMT is safe, 15 (13%) expressed that SMT is harmful, and 49 reviews (42%) were neutral or unclear. Thirteen reviews reported incidence estimates for SAEs, roughly ranging from 1 in 20,000 to 1 in 250,000,000 manipulations. Low methodological quality was present, with a median of 4 of 11 AMSTAR items met (interquartile range, 3 to 6). Reviews meeting the requirements for each of the AMSTAR items (i.e. good internal validity) had a higher chance of expressing that SMT is safe.

The authors concluded that it is currently not possible to provide an overall conclusion about the safety of SMT; however, the types of SAEs reported can indeed be significant, sustaining that some risk is present. High quality research and consistent reporting of AEs and SAEs are needed.

This article is valuable, if only for the wealth of information one can extract from it. There are, however, numerous problems. One is that the overview included mostly reviews of the effectiveness of SMT for various conditions. We know that studies of SMT often do not even mention AEs. If such studies are then pooled in a review, they inevitably generate an impression of safety. But this would, of course, be a false-positive result!

The authors of the overview are aware of this problem and address it in the following paragraph: “When only considering the subset of reviews, where the objective was to investigate AEs (37 reviews), then 8 reviews (22%) expressed that SMT is safe, 13 reviews (35%) expressed that SMT is harmful and 16 reviews (43%) were neutral or unclear regarding the safety of SMT. Hence, there is a tendency that a bigger proportion of these reviews are expressing that SMT is harmful compared to the full sample of reviews…”

To my surprise, I found several of my own reviews in the ‘neutral or unclear’ category. Here are the verbatim conclusions of three of them:

  1. It is concluded that serious cerebrovascular complications of spinal manipulation continue to be reported.
  2. The most common serious adverse events are vertebrobasilar accidents, disk herniation, and cauda equina syndrome.
  3. These data indicate that mild and transient adverse events seem to be frequent. Serious adverse events are probably rare but their incidence can only be estimated at present.

I find it puzzling how this could be classified as neutral or unclear. The solution of the puzzle might lie in the methodology used: “we appraised the communicated opinions of each review concerning the safety of SMT based on their conclusions regarding the AEs and SAEs. This was done by two reviewers independently (SMN, LK), who judged the communicated opinions as either ‘safe’, ‘neutral/unclear’ or ‘harmful’, based on the qualitative impression the reviewers had when reading the conclusions. The reviewers had no opinion about the safety/harmfulness of SMT before commencing the judgements. Cohen’s weighted Kappa was calculated for the agreement between the reviewers, with a value of 0.40–0.59 indicating ‘fair agreement’, 0.60–0.74 indicating ‘good agreement’ and ≥0.75 indicating ‘excellent agreement’. Disagreements were resolved by a third reviewer (MH).”

In other words, the categorisation was done on the basis of subjective judgements of two researchers. It seems obvious that, if their attitude was favourable towards SMT, their judgements would be influenced. The three examples from my own work cited above indicates to me that their verdicts were indeed far from objective.

So what is the main message here? In my view, it can be summarized in the following quote from the overview: “a bigger proportion of these reviews are expressing that SMT is harmful …”

Yes, yes, yes – I know that, if you are a chiropractor (or other practitioner using mostly SMT), you are unlikely to agree with this!

Perhaps you can agree with this statement then:

As long as there is reasonable doubt about the safety of SMT, and as long as we cannot be sure that SMT generates more good than harm, we should be very cautious using it for routine healthcare and do rigorous research to determine the truth (it’s called the precautionary principle and applies to all types of healthcare).

Therapeutic Touch is a therapy mostly popular with nurses. We have discussed it before, for instance here, here, here and here. To call it implausible would be an understatement. But what does the clinical evidence tell us? Does it work?

This literature review by Iranian authors was aimed at critically evaluating the data from clinical trials examining the clinical efficacy of therapeutic touch as a supportive care modality in adult patients with cancer.

Four electronic databases were searched from the year 1990 to 2015 to locate potentially relevant peer-reviewed articles using the key words therapeutic touch, touch therapy, neoplasm, cancer, and CAM. Additionally, relevant journals and references of all the located articles were manually searched for other potentially relevant studies.

The number of 334 articles was found on the basis of the key words, of which 17 articles related to the clinical trial were examined in accordance with the objectives of the study. A total of 6 articles were in the final dataset in which several examples of the positive effects of healing touch on pain, nausea, anxiety and fatigue, and life quality and also on biochemical parameters were observed.

The authors concluded that, based on the results of this study, an affirmation can be made regarding the use of TT, as a non-invasive intervention for improving the health status in patients with cancer. Moreover, therapeutic touch was proved to be a useful strategy for adult patients with cancer.

This review is badly designed and poorly reported. Crucially, its conclusions are not credible. Contrary to what the authors stated when formulating their aims, the methods lack any attempt of critically evaluating the primary data.

A systematic review is more than a process of ‘pea counting’. It requires a rigorous assessment of the risk of bias of the included studies. If that crucial step is absent, the article is next to worthless and the review degenerates into a promotional excercise. Sadly, this is the case with the present review.

You may think that this is relatively trivial (“Who cares what a few feeble-minded nurses do?”), but I would disagree: if the medical literature continues to be polluted by such irresponsible trash, many people (nurses, journalists, healthcare decision makers, researchers) who may not be in a position to see the fatal flaws of such pseudo-reviews will arrive at the wrong conclusions and make wrong decisions. This will inevitably contribute to a hindrance of progress and, in certain circumstances, must endanger the well-being or even the life of vulnerable patients.

The aim of this paper was to systematically review effectiveness, safety, and robustness of evidence for complementary and alternative medicine in managing premature ejaculation (PE). Nine databases were searched through September 2015. Randomized controlled trials (RCTs) evaluating complementary and alternative medicine for PE were included. Studies were included if they reported on intravaginal ejaculatory latency time (IELT) and/or another validated PE measurement. Adverse effects were summarized.

Ten RCTs were included. Two assessed acupuncture, five assessed Chinese herbal medicine, one assessed Ayurvedic herbal medicine, and two assessed topical “severance secret” cream. Risk of bias was unclear in all studies because of unclear allocation concealment or blinding, and only five studies reported stopwatch-measured IELT. Acupuncture slightly increased IELT over placebo in one study (mean difference [MD] = 0.55 minute, P = .001). In another study, Ayurvedic herbal medicine slightly increased IELT over placebo (MD = 0.80 minute, P = .001). Topical severance secret cream increased IELT over placebo in two studies (MD = 8.60 minutes, P < .001), although inclusion criteria were broad (IELT < 3 minutes). Three studies comparing Chinese herbal medicine with selective serotonin reuptake inhibitors (SSRIs) favored SSRIs (MD = 1.01 minutes, P = .02). However, combination treatment with Chinese medicine plus SSRIs improved IELT over SSRIs alone (two studies; MD = 1.92 minutes, P < .00001) and over Chinese medicine alone (two studies; MD = 2.52 minutes, P < .00001). Adverse effects were not consistently assessed but where reported were generally mild.

The authors concluded that there is preliminary evidence for the effectiveness of acupuncture, Chinese herbal medicine, Ayurvedic herbal medicine, and topical severance secret cream in improving IELT and other outcomes. However, results are based on clinically heterogeneous studies of unclear quality. There are sparse data on adverse effects or potential for drug interactions. Further well-conducted randomized controlled trials would be valuable.

One has to be an optimist to agree that this constitutes ‘preliminary evidence for the effectiveness of acupuncture, Chinese herbal medicine, Ayurvedic herbal medicine, and topical severance secret cream in improving IELT and other outcomes.’ In the discussion section, the authors stress that  “…all 10 studies were classed as having an overall unclear risk of bias because of unclear reporting of allocation concealment (all 10 studies) and unclear blinding of participants and personnel (five studies).” This hardly allows even a preliminary conclusion, in my view.

So, what DOES this review show? I think it demonstrates that

  • alternative therapies are being touted and occasionally tested for even the most unlikely conditions,
  • the quality of the studies is generally too poor to justify the research (particularly in an area as intrusive as PE),
  • clinical trials often seem to be used not for finding answers but for promotion,
  • in alternative medicine, trialists regularly violate research ethics by failing to report adverse effects.



The love-affair of many nurses with complementary medicine is well-known. We have discussed it many times on this blog – see for instance here, here and here. Yet the reasons for it remain somewhat mysterious, I find. Therefore I was interested to see a new paper on the subject.

The aim of this ‘meta-synthesis‘ was to review, critically, appraise and synthesize the existing qualitative research to develop a new, more substantial interpretation of nurses’ attitudes regarding the, use of complementary therapies by patients. Fifteen articles were included in the review.

Five themes emerged from the data relating to nurses’ attitude towards complementary therapies:

  1. the strengths and weaknesses of conventional medicine;
  2. complementary therapies as a way to enhance nursing practice;
  3. patient empowerment and patient-centeredness;
  4. cultural barriers and enablers to integration;
  5. structural barriers and enablers to integration.

Nurses’ support for complementary therapies, the authors of this article claim, is not an attempt to challenge mainstream medicine but rather an endeavour to improve the quality of care available to patients. There are, however, a number of barriers to nurses’ support including institutional culture and clinical context, as well as time and knowledge limitations.

The authors concluded that some nurses promote complementary therapies as an opportunity to personalise care and practice in a humanistic way. Yet, nurses have very limited education in this field and a lack of professional frameworks to assist them. The nursing profession needs to consider how to address current deficiencies in meeting the growing use of complementary therapies by patients.

In my view, there are two most remarkable misunderstandings here:

  1. While it is undoubtedly laudable that nurses “endeavour to improve the quality of care available to patients”, it has to be said that such an endeavour does not require complementary medicine. Are they implying that with conventional medicine the quality of care cannot be improved?
  2. I fail to understand why the lack of good evidential support for most complementary therapies did not emerge as a prominent theme. Are nurses not concerned about the (lack of) evidence that underpins their actions?

Aromatherapy is popular and pleasant – but does it have real health effects? The last time I tried to find an answer to this question was in 2012. At that time, our systematic review concluded that “the evidence is not sufficiently convincing that aromatherapy is an effective therapy for any condition.” But 5 years can be a long time in research, and more up-to-date information would perhaps be helpful.

This systematic review of 2017 aimed to provide an analysis of the clinical evidence on the efficacy of aromatherapy specifically for depressive symptoms on any type of patients. The authors searched 5 databases for relevant studies Outcome measures included scales measuring depressive symptoms levels. Twelve randomized controlled trials (RCTs) were included. Aromatherapy was administered by inhalation (5 studies) or massage (7 studies). Seven RCTs showed improvement in depressive symptoms. The quality of half of the studies was low, and the administration protocols varied considerably among the studies. Different assessment tools were employed in the studies. In 6 of the RCTs, aromatherapy was compared to no intervention.

Despite these caveats, the authors concluded that aromatherapy showed potential to be used as an effective therapeutic option for the relief of depressive symptoms in a wide variety of subjects. Particularly, aromatherapy massage showed to have more beneficial effects than inhalation aromatherapy.

Apart from the poor English, this paper is irritating because of the almost total lack of critical input. Given that half of the trials were of poor quality (only one was given the full points on the quality scale) and many totally failed to control for placebo-effects, I think that calling aromatherapy an effective therapeutic option for the relief of depressive symptoms is simply not warranted. In fact, it is highly misleading and, given the fact that depression is a life-threatening condition, it seems unethical and dangerous.

Considering these facts, my conclusion remains that “the evidence is not sufficiently convincing that aromatherapy is an effective therapy for any condition, including depression.”

Gravityscan Badge

Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted.

Click here for a comprehensive list of recent comments.