clinical trial

I have blogged about the herbal antidepressant before; for instance about the fact that it can cause potentially dangerous herb-drug interactions. When taken alone, however, it seems to be both safe and efficacious in reducing the symptoms of depression. This notion has just been confirmed yet again.

A new systematic review evaluated St. John’s wort (SJW) for the treatment of Major Depressive Disorder (MDD). The objectives of this review were to (1) evaluate the efficacy and safety of SJW in adults with MDD compared to placebo and active comparator and (2) evaluate whether the effects vary by severity of MDD.

The authors searched 9 electronic databases and existing reviews to November 2014. Two independent reviewers screened the citations, abstracted the data, and assessed the risk of bias. They included randomized controlled trials (RCTs) examining the effect of at least a 4-week administration of SJW on depression outcomes against placebo or active comparator in adults with MDD. Risk of bias was assessed using the Cochrane Risk of Bias tool and USPSTF criteria. Quality of evidence (QoE) was assessed using the GRADE approach.

Thirty-five studies examining 6993 patients met inclusion criteria; 8 studies evaluated a SJW extract that combined 0.3 % hypericin and 1-4 % hyperforin. SJW was associated with more treatment responders than placebo (relative risk [RR] 1.53; 95 % confidence interval [CI] 1.19, 1.97; I(2) 79 %; 18 RCTs; N = 2922, moderate QoE; standardized mean differences [SMD] 0.49; CI 0.23, 0.74; 16 RCTs; I(2) 89 %, N = 2888, moderate QoE). Compared to antidepressants, SJW participants were less likely to experience adverse events (OR 0.67; CI 0.56, 0.81; 11 RCTs; moderate QoE) with no difference in treatment effectiveness (RR 1.01; CI 0.90, 1.14; 17 RCTs, I(2) 52 %, moderate QoE; SMD -0.03; CI -0.21, 0.15; 14 RCTs; I(2) 74 %; N = 2248, moderate QoE) in mild and moderate depression.

The authors concluded that SJW monotherapy for mild and moderate depression is superior to placebo in improving depression symptoms and not significantly different from antidepressant medication. However, evidence of heterogeneity and a lack of research on severe depression reduce the quality of the evidence. Adverse events reported in RCTs were comparable to placebo and fewer compared with antidepressants. However, assessments were limited due to poor reporting of adverse events and studies were not designed to assess rare events. Consequently, the findings should be interpreted with caution.

This is an excellent review from a reputable and independent team. The findings are therefore trustworthy.

Does that mean that we can now recommend SJW for patients suffering from depression?

Perhaps – but we need to keep an eye on the interaction issue. As a sole treatment, SJW is much safer than conventional antidepressants. But if a patient takes other medicines, we ought to be very careful.

Other currently unresolved issues are the questions of which extract and which dose. At present, there is not enough evidence to provide conclusive answers to either of these, and therefore the enthusiasm of many doctors for prescribing SJW is understandably limited.

Irrespective of these problems, I have to say that SJW is without question one of the biggest ‘success stories’ from the realm of alternative medicine. Pity that there are not more of them!

I have warned you before to be sceptical about Chinese studies. This is what I posted on this blog more than 2 years ago, for instance:

Imagine an area of therapeutics where 100% of all findings of hypothesis-testing research are positive, i.e. come to the conclusion that the treatment in question is effective. Theoretically, this could mean that the therapy is a miracle cure which is useful for every single condition in every single setting. But sadly, there are no miracle cures. Therefore something must be badly and worryingly amiss with the research in an area that generates 100% positive results.

Acupuncture is such an area; we and others have shown that Chinese trials of acupuncture hardly ever produce a negative finding. In other words, one does not need to read the paper, one already knows that it is positive – even more extreme: one does not need to conduct the study, one already knows the result before the research has started. But you might not believe my research nor that of others. We might be chauvinist bastards who want to discredit Chinese science. In this case, you might perhaps believe Chinese researchers.

In this systematic review, all randomized controlled trials (RCTs) of acupuncture published in Chinese journals were identified by a team of Chinese scientists. A total of 840 RCTs were found, including 727 RCTs comparing acupuncture with conventional treatment, 51 RCTs with no treatment controls, and 62 RCTs with sham-acupuncture controls. Among theses 840 RCTs, 838 studies (99.8%) reported positive results from primary outcomes and two trials (0.2%) reported negative results. The percentages of RCTs concealment of the information on withdraws or sample size calculations were 43.7%, 5.9%, 4.9%, 9.9%, and 1.7% respectively.

The authors concluded that publication bias might be major issue in RCTs on acupuncture published in Chinese journals reported, which is related to high risk of bias. We suggest that all trials should be prospectively registered in international trial registry in future.


Now an even more compelling reason emerged for taking evidence from China with a pinch of salt:

A recent survey of clinical trials in China has revealed fraudulent practice on a massive scale. China’s food and drug regulator carried out a one-year review of clinical trials. They concluded that more than 80 percent of clinical data is “fabricated“. The review evaluated data from 1,622 clinical trial programs of new pharmaceutical drugs awaiting regulator approval for mass production. Officials are now warning that further evidence malpractice could still emerge in the scandal.
According to the report, much of the data gathered in clinical trials are incomplete, failed to meet analysis requirements or were untraceable. Some companies were suspected of deliberately hiding or deleting records of adverse effects, and tampering with data that did not meet expectations.

“Clinical data fabrication was an open secret even before the inspection,” the paper quoted an unnamed hospital chief as saying. Contract research organizations seem have become “accomplices in data fabrication due to cutthroat competition and economic motivation.”

A doctor at a top hospital in the northern city of Xian said the problem doesn’t lie with insufficient regulations governing clinical trials data, but with the failure to implement them. “There are national standards for clinical trials in the development of Western pharmaceuticals,” he said. “Clinical trials must be carried out in three phases, and they must be assessed at the very least for safety,” he said. “But I don’t know what happened here.”

Public safety problems in China aren’t limited to the pharmaceutical industry and the figure of 80 percent is unlikely to surprise many in a country where citizens routinely engage in the bulk-buying of overseas-made goods like infant formula powder. Guangdong-based rights activist Mai Ke said there is an all-pervasive culture of fakery across all products made in the country. “It’s not just the medicines,” Mai said. “In China, everything is fake, and if there’s a profit in pharmaceuticals, then someone’s going to fake them too.” He said the problem also extends to traditional Chinese medicines, which are widely used in conjunction with Western pharmaceuticals across the healthcare system.
“It’s just harder to regulate the fakes with traditional medicines than it is with Western pharmaceuticals, which have strict manufacturing guidelines,” he said.

According to Luo, academic ethics is an underdeveloped field in China, leading to an academic culture that is accepting of manipulation of data. “I don’t think that the 80 percent figure is overstated,” Luo said.

And what should we conclude from all this?

I find it very difficult to reach a verdict that does not sound hopelessly chauvinistic but feel that we have little choice but to distrust the evidence that originates from China. At the very minimum, I think, we must scrutinise it thoroughly; whenever it looks too good to be true, we ought to discard it as unreliable and await independent replications.

In a recent PJ article, Michael Marshall from the ‘Good Thinking Society’ asked “WHY ON EARTH IS THE NHS SPENDING EVEN A SINGLE PENNY ON HOMEOPATHY?”. A jolly good question, given the overwhelmingly negative evidence, I thought  – but one that must be uncomfortable to homeopaths. Sure enough, a proponent of homeopathy, Jeanette Lindsay from Glasgow, has objected to Marshall’s arguments in a short comment which is a fairly typical defence of homeopathy; I therefore take the liberty of reproducing it here (the 12 references in her text were added by me and refer to my footnotes below):

I wonder if people such as Michael Marshall (The Pharmaceutical Journal 2016;297:101), who would refuse [1] patients the option of NHS homeopathic treatment, have considered the plight of people failed by evidence-based medicine ? [2] Where are those with chronic, disabling conditions to turn when the medicines available on the NHS do not work, or worse, are positively harmful? [3]

Take the instance of a woman with multiple drug allergies who has no means of treating her severe inflammatory arthritis and no suitable analgesia. [4] It has been demonstrated that disease states with immune system involvement are particularly susceptible to the placebo effect but how does one induce this? Current thinking precludes treatment with placebo medicines but it so happens that homeopathic remedies would appear, from the results of clinical trials [5], to be a good substitute. [6] Used properly, there is a good chance that in this case homeopathic treatment may achieve a real therapeutic effect. [7]

Patients who cannot tolerate allopathic [8] treatment do not just go away because they cannot take the prescribed medicine. [9] They suffer and surely deserve a better range of options [10] than those provided by the current obsession with evidence-based medicine. [11] The availability of homeopathic treatment is important and should not be denied until better alternatives become commonplace. [12]

[1] Michael Marshall does not ‘refuse’ homeopathy on the NHS; that is not in his power. He merely questions whether NHS funds should not be spent on treatments that demonstrably do more good than harm.

[2] I am sure he as carefully considered such patients.

[3] Depending on the exact circumstances, such patients have many options: for instance, they could change their physician, have their diagnosis re-considered, or try a non-drug treatment.

[4] An allergy to one drug is rarely (I would even say never) associated with allergies to all drugs for any given condition. Even if this were the case, there are several non-drug treatments for arthritis or other diseases.

[5] I think this is fantasy; there is no good evidence from clinical trials to show that homeopathy is efficacious for either inflammatory or degenerative arthritis.

[6] Is this an admission that homeopathic remedies are placebos?

[7] I am not aware of sound evidence to support this statement.

[8] ‘Allopathic’ is a derogatory term introduced by Hahnemann to defame conventional medicine.

[9] I have never seen a patient who could not tolerate any prescription medicine. I suspect this is fantasy again.

[10] Patients deserve the optimal therapy available for their conditions – that is a therapy that demonstrably generates more good than harm. Homeopathy is clearly not in this category.

[11] An obsession? Yes, perhaps it is an obsession for some dedicated healthcare professionals to provide the best possible treatments for their patients. But the way it is put here, it sounds as though this was something despicable. I would argue that such an ‘obsession’ would be most commendable.

[12] For practically all conditions, symptoms, illesses and diseases that afflict mankind, better alternatives than homeopathy have been available since about 150 years.

It seems to me that Jeanette Lindsay has been harshly disappointed by conventional medicine. Perhaps this is why, one day, she consulted a homeopath and received the empathy, understanding and compassion that she needed to get better. Many homeopaths excel at these qualities; and this is the main reason why their patients swear by them, even though their remedies are pure placebos.

My advice to such patients is: find a physician who has time, empathy and compassion. They do exist! Once you have found such a doctor, you can benefit from the compassion and empathy just as you may have benefitted from the homeopath’s compassion and empathy. But in addition to these benefits (and contrary to what you got from your homeopath), you will also be able to profit from the efficacy of the treatments prescribed.

To put it simply: homeopaths can help patients via non-specific therapeutic effects; responsible physicians can help patients via non-specific therapeutic effects plus the specific effects of the treatments they prescribe.


Some osteopaths – similar to their chiropractic, naturopathic, homeopathic, etc. colleagues – claim they can treat almost any condition under the sun. Even gynaecological ones? Sure! But is the claim true? Let’s find out.

The aim of this recent review was to evaluate the effects of the osteopathic manipulative treatment (OMT) on women with gynaecological and obstetric disorders. An extensive search from inception to April 2014 was conducted on MEDLINE, Embase, the Cochrane library using MeSH and free terms. Clinical studies investigating the effect of OMT in gynaecologic and obstetric conditions were included as well as unpublished works. Reviews and personal contributions were excluded. Studies were screened for population, outcome, results and adverse effects by two independent reviewers using an ad-hoc data extraction form. The high heterogeneity of the studies led to a narrative review.

In total, 24 studies were included. They addressed the following conditions: back pain and low back functioning in pregnancy, pain and drug use during labor and delivery, infertility and subfertility, dysmenorrhea, symptoms of (peri)menopause and pelvic pain. Overall, OMT was considered to be effective for pregnancy related back pain. For all other gynaecological and obstetrical conditions the evidence was considered to be uncertain. Only three studies mentioned adverse events after OMT.

The authors concluded that, although positive effects were found, the heterogeneity of study designs, the low number of studies and the high risk of bias of included trials prevented any indication on the effect of osteopathic care. Further investigation with more pragmatic methodology, better and detailed description of interventions and systematic reporting of adverse events are recommended in order to obtain solid and generalizable results.

Given the fact that the lead authors of this review come from the “Accademia Italiana Osteopatia Tradizionale, Pescara, Italy, we can probably answer the question in the title of this blog with a straight NO. I see no reason why OMT should work for gynaecological conditions, and I am not in the least surprised to read that there is no clinical evidence for this notion. Sadly, this is unlikely to stop osteopaths to claim otherwise and continue to prey on the desperate and the gullible.

One might thus say that this review is totally unremarkable – but I would beg to differ: it highlights yet again one very important finding, namely the fact that trials of alternative therapies far too often fail to report adverse effects. I have stated this often already, but I will say it again: THIS OMISSION IS A VIOLATION OF RESEARCH ETHICS WHICH GIVES US A FALSE POSITIVE OVERALL PICTURE OF THE RISKS ASSOCIATED WITH ALTERNATIVE MEDICINE.


Deepak Chopra rarely publishes in medical journals (I suppose, he has better things to do). I was therefore intrigued when I saw a recent article of which he is a co-author.

The ‘study‘ in question allegedly examined the effects of a comprehensive residential mind–body program on well-being. The authors describe it as “a quasi-randomized trial comparing the effects of participation in a 6-day Ayurvedic system of medicine-based comprehensive residential program with a 6-day residential vacation at the same retreat location.” They included 69 healthy women and men who received the Ayurvedic intervention addressing physical and emotional well-being through group meditation and yoga, massage, diet, adaptogenic herbs, lectures, and journaling. Key components of the program include physical cleansing through ingestion of herbs, fiber, and oils that support the body’s natural detoxification pathways and facilitate healthy elimination; two Ayurvedic meals daily (breakfast and lunch) that provide a light plant-based diet; daily Ayurvedic oil massage treatments; and heating treatments through the use of sauna and/or steam. The program includes lectures on Ayurvedic principles and lifestyle as well as lectures on meditation and yoga philosophy. The study group also participated in twice-daily group meditation and daily yoga and practiced breathing exercises (pranayama) as well as emotional expression through a process of journaling and emotional support. During the program, participants received a 1-hour integrative medical consultation with a physician and follow-up with an Ayurvedic health educator.

The control group simply had a vacation without any of the above therapies in the same resort. They were asked to do what they would normally do on a resort vacation with the additional following restrictions: they were asked not to engage in more exercise than they would in their normal lifestyle and to refrain from using La Costa Resort spa services. They were also asked not to drink ginger tea or take Gingko biloba during the 2 days before and during the study week.

Recruitment was via email announcements on the University of California San Diego faculty and staff and Chopra Center for Wellbeing list-servers. Study flyers stated that the week-long Self-Directed Biological Transformation Initiative (SBTI) study would be conducted at the Chopra Center for Wellbeing, located at the La Costa Resort in Carlsbad, California, in order to learn more about the psychosocial and physiologic effects of the 6-day Perfect Health (PH) Program compared with a 6-day stay at the La Costa Resort. The study participants were not blinded, and site investigators and study personnel knew to which group participants were assigned.

Participants in the Ayurvedic program showed significant and sustained increases in ratings of spirituality and gratitude compared with the vacation group, which showed no change. The Ayurvedic participants also showed increased ratings for self-compassion as well as less anxiety at the 1-month follow-up.

The authors arrived at the following conclusion: Findings suggest that a short-term intensive program providing holistic instruction and experience in mind–body healing practices can lead to significant and sustained increases in perceived well-being and that relaxation alone is not enough to improve certain aspects of well-being.

This ‘study’ had ethical approval from the University of California San Diego and was supported by the Fred Foundation, the MCJ Amelior Foundation, the National Philanthropic Trust, the Walton Family Foundation, and the Chopra Foundation. The paper’s first author is director of research at the Chopra Foundation. Deepak Chopra is the co-founder of The Chopra Center for Wellbeing.

Did I promise too much?

Isn’t this paper hilarious?

Just for the record, let me formulate a short conclusion that actually fits the data from this ‘study’: Lots of TLC, attention and empathy does make some people feel better.

This is hardly something one needs to write home about; and certainly nothing to do a study on!

But which journal would publish such unadulterated advertising?

On this blog, I have mentioned the JACM several times before. Recently, I wrote about the new man in charge of it. I concluded stating WATCH THIS SPACE.

I think the wait is now over – this paper is from the latest issue of the JACM, and I am sure we all agree that the new editor has just shown us of what he is made and where he wants to take his journal.

Just as I thought that this cannot get any better, it did! It did so in the form of a second paper which is evidently reporting from the same ‘study’. Here is its abstract unaltered in its full beauty:

The effects of integrative medicine practices such as meditation and Ayurveda on human physiology are not fully understood. The aim of this study was to identify altered metabolomic profiles following an Ayurveda-based intervention. In the experimental group, 65 healthy male and female subjects participated in a 6-day Panchakarma-based Ayurvedic intervention which included herbs, vegetarian diet, meditation, yoga, and massage. A set of 12 plasma phosphatidylcholines decreased (adjusted p < 0.01) post-intervention in the experimental (n = 65) compared to control group (n = 54) after Bonferroni correction for multiple testing; within these compounds, the phosphatidylcholine with the greatest decrease in abundance was PC ae C36:4 (delta = -0.34). Application of a 10% FDR revealed an additional 57 metabolites that were differentially abundant between groups. Pathway analysis suggests that the intervention results in changes in metabolites across many pathways such as phospholipid biosynthesis, choline metabolism, and lipoprotein metabolism. The observed plasma metabolomic alterations may reflect a Panchakarma-induced modulation of metabotypes. Panchakarma promoted statistically significant changes in plasma levels of phosphatidylcholines, sphingomyelins and others in just 6 days. Forthcoming studies that integrate metabolomics with genomic, microbiome and physiological parameters may facilitate a broader systems-level understanding and mechanistic insights into these integrative practices that are employed to promote health and well-being.

Now that I managed to stop laughing about the first paper, I am not just amused but also puzzled by the amount of contradictions the second article seems to cause. Were there 65 or 69 individuals in the experimental group? Was the study randomised, quasi-randomised or not randomised? All of these versions are implied at different parts of the articles. It turns out that they randomised some patients, while allocating others without randomisation – and this clearly means the study was NOT randomised. Was the aim of the study ‘to identify altered metabolomic profiles following an Ayurveda-based intervention’ or ‘to examine the effects of a comprehensive residential mind–body program on well-being’?

I am sure that others will find further contradictions and implausibilites, if they look hard enough.

The funniest inconsistency, in my opinion, is that Deepak Chopra does not even seem to be sure to which university department he belongs. Is it the ‘Department of Family Medicine and Public Health, University of California, San Diego, La Jolla, CA.’ as indicated in the 1st paper or is it the ‘Department of Family and Preventive Medicine, University of California San Diego, La Jolla, California, USA’ as listed in the 2nd article?

Does he know from which planet he is?


After > 200 years of existence, homeopathy still remains unproven – in fact, most rational thinkers would call it disproven. Today only homeopaths doubt this statement; they work hard to find a water-tight proof that might show the doubters to be wrong.

What is better suited for this purpose than a few rigorous animal experiments?

Engystol® is a popular homeopathic product promoted as an anti-viral agent manufactured by Heel GmbH, Baden-Baden, Germany. In several in vivo and in vitro studies, it apparently affected an immune response. This new study was to “evaluate the innate and adaptive immuno-modulatory effects of oral Engystol® (1 or 10 tablets/L water consumed), prior to and post antigenic challenge in a mouse model with a well-characterized and clinically measureable immune system.”

The investigators first evaluated the murine immune response when oral Engystol® was given alone for 28 days. to mice. The animals were then challenged with an antigen-specific H5N1 HA vaccine while on Engystol® for an additional 33 days. Serum and supernatants from cultured splenic lymphocytes were collected and screened with a 32-cytokine panel. Serum vaccine epitope-specific IgG titers plus T cell and B cell phenotypes from splenic tissue were also evaluated.

The results showed that Engystol® alone did not alter immunity. However, upon vaccine challenge, Engystol® decreased CD4+/CD8+ ratios, altered select cytokines/chemokines, and anti-H5N1 HA IgG titers were increased in the group of mice receiving 10 tablet/L.

The authors concluded that “these data suggest that Engystol® can modulate immunity upon antigenic challenge.”

Engystol is being advertised as “a homeopathic preparation which has been scientifically proven to significantly reduce the duration and severity of symptoms during an acute viral infection and help protect from subsequent infections.” I was unable find good evidence for this claim and therefore have to assume that it is bogus. The only human trial I was able to locate was this one:


To compare the effects of a complex homeopathic preparation (Engystol; Heel GmbH, Baden-Baden, Germany) with those of conventional therapies with antihistamines, antitussives, and nonsteroidal antiinflammatory drugs on upper respiratory symptoms of the common cold in a setting closely related to everyday clinical practice.


Nonrandomized, observational study over a treatment period of maximally two weeks.


Eighty-five general and homeopathic practices in Germany.


Three hundred ninety-seven patients with upper respiratory symptoms of the common cold.


Engystol-based therapy or common over-the-counter treatments for the common cold. Patients receiving this homeopathic treatment were allowed other short-term medications, but long-term use of analgesics, antibiotics, and antiinflammatory agents was not permitted. Patients were allowed nonpharmacological therapies such as vitamins, thermotherapies, and others.


The effects of treatment were evaluated on the variables fatigue, sensation of illness, chill/tremor, aching joints, overall severity of illness, sum of all clinical variables, temperature, and time to symptomatic improvement.


Both treatment regimens provided significant symptomatic relief, and this homeopathic treatment was noninferior in a noninferiority analysis. Significantly more patients (P < .05) using Engystol-based therapy reported improvement within 3 days (77.1% vs 61.7% for the control group). No adverse events were reported in any of the treatment groups.


This homeopathic treatment may be a useful component of an integrated symptomatic therapy for the common cold in patients and practitioners choosing an integrative approach to medical care.

Let me comment on the human study first. It is an excellent example of the bias that can be introduced by non-randomization. The patients in the homeopathic group obviously were those who chose to be treated homeopathically. Consequently they had high expectations in this therapy. Consequently they reported better results than the control group. In other words the reported outcomes have nothing to do with the homeopathic remedy.

But what about the animal study? Animals, we hear so often, do not exhibit a placebo response. Does that render this investigation any more reliable?

The answer, I am afraid is no.

The animal study in question had no control group at all. Therefore a myriad of factors could have caused the observed result. This study is very far from a poof of homeopathy!

But even if the findings of the two studies had not been the result of bias and confounding, I would be more than cautious about viewing them as anything near conclusive. The reason lies in the nature of this particular homeopathic remedy.

Engystol® contains Vincetoxicum hirundinaria (D6), Vincetoxicum hirundinaria (D10), Vincetoxicum hirundinaria (D30), sulphur (D4) and sulphur (D10). In other words, it is one of those combination remedies which are not sufficiently dilute to be devoid of active molecules. Sulphur D4, for instance, means that the remedy contains one part of sulphur in 10 000 parts of diluent. It is conceivable, even likely that such a concentration might affect certain immune parameters, I think.

And my conclusion from all this?

The proof of homeopathy – if it ever came – would need to be based on investigations that are more rigorous than these two rather pathetic studies.

For some time now, the research activity in and around alternative medicine has been seemingly buoyant. In each of the last 4 years, Medline listed around 2 000 articles is the category of ‘complementary alternative medicine’. This will surely look impressive to many!

Why then did I write ‘seemingly’? To comprehend this a little better, we should have some comparisons. Here are numbers of Medline-listed articles published in 2015 for a few other areas:

  • Surgery: 176 277
  • Psychology: 65 679
  • Internal medicine: 36 998
  • Obstetrics/gynaecology: 13 818
  • Pharmacology: 194 322
  • Paediatrics: 30 646

Now you see, I hope, why the 2 049 Medline-listed articles in the category of ‘complementary alternative medicine’ are only seemingly impressive. But what about specific alternative therapies? Here are numbers of Medline-listed articles published in 2015 for some major alternative treatments:

  • Homeopathy: 181
  • Herbal medicine: 1 572
  • Chiropractic: 314
  • Acupuncture: 1 784
  • Naturopathy: 45
  • Dietary supplements: 5 199

These figures are perhaps interesting but not easy to interpret. They might indicate that certain sections of alternative medicine are more open to scientific scrutiny than others. Or do they show that for some areas there are more research funds and expertise than others? I am not sure I know the answer.

If we look a little closer at the research activity in defined alternative therapies, we are bound to get disappointed. I have recently done this for homeopathy and for acupuncture and reached rather gloomy conclusions.

In the case of homeopathy the were:

  1. The research activity into homeopathy is currently very subdued.
  2. Arguably the main research question of efficacy does not seem to concern researchers of homeopathy all that much.
  3. There is an almost irritating abundance of papers that are data-free and thrive on opinion (my category of ‘other papers’).
  4. Given all this, I find it hard to imagine that this area of investigation is going to generate much relevant new knowledge or clinical progress.

And in the case of acupuncture, I stated:

  • Too little research is focussed on the two big questions: efficacy and safety.
  • In relation to the meagre output in RCTs, there are too many systematic reviews.
  • As long as we cannot be sure that acupuncture is more than a placebo, all these pre-clinical studies seem a bit out of place.
  • The vast majority of the articles were in low or very low impact journals.
  • There was only one paper that I would consider outstanding.

And what about the quality of the research into alternative medicine?

Well, this is a sad and depressing tale! If you doubt it, read my previous post or indeed any of the other ~500 which I have written on this particular subject in the past.

In alternative medicine, good evidence is like gold dust and good evidence showing that alternative therapies are efficacious is even rarer. Therefore, I was delighted to come across a brand-new article from an institution that should stand for reliable information: the NIH, no less.

According to its authors, this new article “examines the clinical trial evidence for the efficacy and safety of several specific approaches—acupuncture, manipulation, massage therapy, relaxation techniques including meditation, selected natural product supplements (chondroitin, glucosamine, methylsulfonylmethane, S-adenosylmethionine), tai chi, and yoga—as used to manage chronic pain and related disability associated with back pain, fibromyalgia, osteoarthritis, neck pain, and severe headaches or migraines.”

The results of this huge undertaking are complex, of course, but in a nutshell they are at least partly positive for alternative medicine. Specifically, the authors state that “based on a preponderance of positive trials vs negative trials, current evidence suggests that the following complementary approaches may help some patients manage their painful health conditions: acupuncture and yoga for back pain; acupuncture and tai chi for OA of the knee; massage therapy for neck pain with adequate doses and for short-term benefit; and relaxation techniques for severe headaches and migraine. Weaker evidence suggests that massage therapy, SM, and osteopathic manipulation might also be of some benefit to those with back pain, and relaxation approaches and tai chi might help those with fibromyalgia.”

This is excellent news! Finally, we have data from an authoritative source showing that some alternative treatments can be recommended for common pain conditions.

Hold on, not so fast! Yes, the NIH is a most respectable organisation, but we must not blindly accept anything of importance just because it appears to come form a reputable source. Let’s look a bit closer at the actual evidence provided by the authors of this paper.

Reading the article carefully, it is impossible not to get troubled. Here are a few points that concern me most:

  • the safety of a therapy cannot be evaluated on the basis of data from RCTs (particularly as it has been shown repeatedly that trials of alternative therapies often fail to report adverse effects); much larger samples are needed for that; any statements about safety in the aims of the paper are therefore misplaced;
  • the authors talk about efficacy but seem to mean effectiveness;
  • the authors only included RCTs from the US which must result in a skewed and incomplete picture;
  • the article is from the National Center for Complementary and Integrative Health which is part of the NIH but which has been criticised repeatedly for being biased in favour of alternative medicine;
  • not all of the authors seem to be NIH staff, and I cannot find a declaration of conflicts of interest;
  • the discussion of the paper totally lacks any critical thinking;
  • there is no assessment of the quality of the trials included in this review.

My last point is by far the most important. A summary of this nature that fails to take into account the numerous limitations of the primary data is, I think, as good as worthless. As I know most of the RCTs included in the analyses, I predict that the overall picture generated by this review would have changed substantially, if the risks of bias in the primary studies had been accounted for.

Personally, I find it lamentable that such a potentially worthy exercise ended up employing such lousy methodology. Perhaps even more lamentable is the fact that the NIH (or one of its Centers) can descend that low; to mislead the public in this way borders on scientific misconduct and is, in my view, unethical and unacceptable.

Cranio-sacral therapy has been a subject on this blog before, for instance here, here and here. The authors of this single-blind, randomized trial explain in the introduction of their paper that “cranio-sacral therapy is an alternative and complementary therapy based on the theory that restricted movement at the cranial sutures of the skull negatively affect rhythmic impulses conveyed through the cerebral spinal fluid from the cranium to the sacrum. Restriction within the cranio-sacral system can affect its components: the brain, spinal cord, and protective membranes. The brain is said to produce involuntary, rhythmic movements within the skull. This movement involves dilation and contraction of the ventricles of the brain,  which produce the circulation of the cerebral spinal fluid. The theory states that this fluctuation mechanism causes reciprocal tension within the membranes, transmitting motion to the cranial bones and the sacrum. Cranio-sacral therapy and cranial osteopathic manual therapy originate from the observations made by William G. Sutherland, who said that the bones of the human skeleton have mobility. These techniques are based mainly on the study of anatomic and physiologic mechanisms in the skull and their relation to the body as a whole, which includes a system of diagnostic and therapeutic techniques aimed at treatment and prevention of diseases. These techniques are  based on the so-called primary respiratory movement, which is manifested in the mobility of the cranial bones, sacrum,  dura, central nervous system, and cerebrospinal fluid. The main difference between the two therapies is that cranial osteopathy, in addition to a phase that works in the direction of the lesion (called the functional phase), also uses a phase that worsens the injury, which is called structural phase.”

With this study, the researchers wanted to evaluate the effects of cranio-sacral therapy on disability, pain intensity, quality of life, and mobility in patients with low back pain. Sixty-four patients with chronic non-specific low back pain were assigned to an experimental group receiving 10 sessions of craniosacral therapy, or to the control group receiving 10 sessions of classic massage. Craniosacral therapy took 50 minutes and was conducted as follows: With pelvic diaphragm release, palms are placed in transverse position on the superior aspect of the pubic bone, under  the L5–S1 sacrum, and finger pads are placed on spinal processes.  With respiratory diaphragm release, palms are placed transverse under T12/L1 so that the spine lies along the start of fingers and the border of palm, and the anterior hand is placed on the breastbone. For thoracic inlet release, the thumb and index finger are placed on the opposite sides of the clavicle, with the posterior hand/palm of the hand cupping C7/T1. For the hyoid release, the thumb and index finger are placed on the hyoid, with the index finger on the occiput and the cupping finger pads on the cervical vertebrae. With the sacral technique for stabilizing L5/sacrum, the fingers contact the sulcus and the palm of the hand is in contact with the distal part of the sacral bone. The non-dominant hand of the therapist rested over the pelvis, with one hand on one iliac crest and the elbow/forearm of the other side over the other iliac crest. For CV-4 still point induction, thenar pads are placed under the occipital protuberance, avoiding mastoid sutures.  Classic massage protocol was compounded by the following sequence techniques of soft tissue massage on the  low back: effleurage, petrissage, friction, and kneading. The maneuvers are performed with surface pressure, followed by deep pressure and ending with surface pressure again. The techniques took 30 minutes.

Disability (Roland Morris Disability Questionnaire RMQ, and Oswestry Disability Index) was the primary endpoint. Other outcome measures included the pain intensity (10-point numeric pain rating scale), kinesiophobia (Tampa Scale of Kinesiophobia), isometric endurance of trunk flexor muscles (McQuade test), lumbar mobility in flexion, hemoglobin oxygen saturation, systolic blood pressure, diastolic blood pressure, hemodynamic measures (cardiac index), and biochemical analyses of interstitial fluid. All outcomes were measured at baseline, after treatment, and one-month follow-up.

No statistically significant differences were seen between groups for the main outcome of the study, the RMQ. However, patients receiving craniosacral therapy experienced greater improvement in pain intensity (p ≤ 0.008), hemoglobin oxygen saturation (p ≤ 0.028), and systolic blood pressure (p ≤ 0.029) at immediate- and medium-term and serum potassium (p = 0.023) level and magnesium (p = 0.012) at short-term than those receiving classic massage.

The authors concluded that 10 sessions of cranio-sacral therapy resulted in a statistically greater improvement in pain intensity, hemoglobin oxygen saturation, systolic blood pressure, serum potassium, and magnesium level than did 10 sessions of classic massage in patients with low back pain.

Given the results of this study, the conclusion is surprising. The primary outcome measure failed to show an inter-group difference; in other words, the results of this RCT were essentially negative. To use secondary endpoints – most of which are irrelevant for the study’s aim – in order to draw a positive conclusion seems odd, if not misleading. These positive findings are most likely due to the lack of patient-blinding or to the 200 min longer attention received by the verum patients. They are thus next to meaningless.

In my view, this publication is yet another example of an attempt to turn a negative into a positive result. This phenomenon seems embarrassingly frequent in alternative medicine. It goes without saying that it is not just misleading but also dishonest and unethical.

A new Cochrane review evaluated the effectiveness and safety of Chinese herbal medicines (CHM) in the treatment of menopausal symptoms. Its authors conducted a thorough search for randomised controlled trials (RCTs) comparing the effectiveness of CHM with placebo, hormone therapy (HT), pharmaceutical drugs, acupuncture, or another CHM formula in women suffering from menopausal symptoms.

Two review authors independently assessed 864 studies for eligibility. Data extractions were performed by them with disagreements resolved through group discussion and clarification of data or direct contact with the study authors. Data analyses were performed in accordance with Cochrane Collaboration guidelines.

In total, 22 RCTs (2902 women) could be included. When CHM was compared with placebo (8 RCTs), there was little or no evidence of a difference between the groups for the following outcomes: hot flushes per day (MD 0.00, 95% CI -0.88 to 0.89; 2 trials, 199 women; moderate quality evidence); hot flushes per day assessed by an overall hot flush score in which a difference of one point equates to one mild hot flush per day (MD -0.81 points, 95% CI -2.08 to 0.45; 3 RCTs, 263 women; low quality evidence); and overall vasomotor symptoms per month measured by the Menopause-Specific Quality of Life questionnaire (MENQOL, scale 0 to 6) (MD -0.42 points; 95% CI -1.52 to 0.68; 3 RCTs, 256 women; low quality evidence). In addition, results from individual studies suggested there was no evidence of a difference between the groups for daily hot flushes assessed by severity (MD -0.70 points, 95% CI -1.00, -0.40; 1 RCT, 108 women; moderate quality evidence); or overall monthly hot flushes scores (MD -2.80 points, 95% CI -8.93 to 3.33; 1 RCT, 84 women; very low quality evidence); or overall daily night sweats scores (MD 0.07 points, 95% CI -0.19 to 0.33, 1 RCT, 64 women; low quality evidence); or overall monthly night sweats scores (MD 1.30 points, 95% CI -1.76 to 4.36, 1 RCT, 84 women; very low quality evidence). However, one study reported that overall monthly vasomotor symptom scores were lower in the CHM group (MD -4.79 points, 95% CI -5.52 to -4.06; 1 RCT, 69 women; low quality evidence).

When CHM was compared with HT (10 RCTs), only two RCTs reported monthly vasomotor symptoms using MENQOL. It was uncertain whether CHM reduces vasomotor symptoms (MD 0.47 points, 95% CI -0.50 to 1.44; 2 RCTs, 127 women; very low quality evidence).

Adverse effects were not fully reported in the included studies. Adverse events reported by women taking CHM included mild diarrhoea, breast tenderness, gastric discomfort and an unpleasant taste. Effects were inconclusive because of imprecise estimates of effects: CHM versus placebo (RR 1.51; 95% CI 0.69 to 3.33; 7 trials, 705 women; I² = 40%); CHM versus HT (RR 0.96; 95% CI 0.66 to 1.39; 2 RCTs, 864 women; I² = 0%); and CHM versus specific conventional medications (such as Fluoxetine and Estazolam) (RR 0.20; 95% CI 0.03 to 1.17; 2 RCTs, 139 women; I² = 61%).

The authors concluded: We found insufficient evidence that Chinese herbal medicines were any more or less effective than placebo or HT for the relief of vasomotor symptoms. Effects on safety were inconclusive. The quality of the evidence ranged from very low to moderate; there is a need for well-designed randomised controlled studies.

This review seems well done and reports clear findings. The fact that there was insufficient evidence for CHM is probably no surprise to most readers of this blog. However, I would like to draw your attention to a finding that could easily be missed: most of the primary studies failed to mention adverse effects; to be perfectly clear: they did not state “there were no adverse effects”, but they simply did not mention the subject of adverse effects at all.

In my view, this is a breach of research ethics. I have been banging on about this phenomenon for some time now, because I think it is important. Many if not most clinical trials in this area neglect reporting adverse effects. This means that we get an entirely misleading impression about the safety of the treatments in question. Reviewers of such studies are bound to conclude that they seem to be safe, while, in fact, researchers have only been withholding crucial information from us.

The solution to this fast-growing problem would be simple: trialists must be forced to fully report adverse effects. This is less complicated that it might seem: journal editors must insist that all authors fully report adverse effects of alternative treatments. Even if there were none at all – a very unlikely proposition if you think about it – they must disclose this fact.

Recent Comments

Note that comments can now be edited for up to five minutes after they are first submitted.

Click here for a comprehensive list of recent comments.