As I often said, I find it regrettable that sceptics often say THERE IS NOT A SINGLE STUDY THAT SHOWS HOMEOPATHY TO BE EFFECTIVE (or something to that extent). This is quite simply not true, and it gives homeopathy-fans the occasion to suggest sceptics wrong. The truth is that THE TOTALITY OF THE MOST RELIABLE EVIDENCE FAILS TO SUGGEST THAT HIGHLY DILUTED HOMEOPATHIC REMEDIES ARE EFFECTIVE BEYOND PLACEBO. As a message for consumers, this is a little more complex, but I believe that it’s worth being well-informed and truthful.
And that also means admitting that a few apparently rigorous trials of homeopathy exist and some of them show positive results. Today, I want to focus on this small set of studies.
How can a rigorous trial of a highly diluted homeopathic remedy yield a positive result? As far as I can see, there are several possibilities:
- Homeopathy does work after all, and we have not fully understood the laws of physics, chemistry etc. Homeopaths favour this option, of course, but I find it extremely unlikely, and most rational thinkers would discard this possibility outright. It is not that we don’t quite understand homeopathy’s mechanism; the fact is that we understand that there cannot be a mechanism that is in line with the laws of nature.
- The trial in question is the victim of some undetected error.
- The result has come about by chance. Of 100 trials, 5 would produce a positive result at the 5% probability level purely by chance.
- The researchers have cheated.
When we critically assess any given trial, we attempt, in a way, to determine which of the 4 solutions apply. But unfortunately we always have to contend with what the authors of the trial tell us. Publications never provide all the details we need for this purpose, and we are often left speculating which of the explanations might apply. Whatever it is, we assume the result is false-positive.
Naturally, this assumption is hard to accept for homeopaths; they merely conclude that we are biased against homeopathy and conclude that, however, rigorous a study of homeopathy is, sceptics will not accept its result, if it turns out to be positive.
But there might be a way to settle the argument and get some more objective verdict, I think. We only need to remind ourselves of a crucially important principle in all science: INDEPENDENT REPLICATION. To be convincing, a scientific paper needs to provide evidence that the results are reproducible. In medicine, it unquestionably is wise to accept a new finding only after it has been confirmed by other, independent researchers. Only if we have at least one (better several) independent replications, can we be reasonably sure that the result in question is true and not false-positive due to bias, chance, error or fraud.
And this is, I believe, the extremely odd phenomenon about the ‘positive’ and apparently rigorous studies of homeopathic remedies. Let’s look at the recent meta-analysis of Mathie et al. The authors found several studies that were both positive and fairly rigorous. These trials differ in many respects (e. g. remedies used, conditions treated) but they have, as far as I can see, one important feature in common: THEY HAVE NOT BEEN INDEPENDENTLY REPLICATED.
If that is not astounding, I don’t know what is!
Think of it: faced with a finding that flies in the face of science and would, if true, revolutionise much of medicine, scientists should jump with excitement. Yet, in reality, nobody seems to take the trouble to check whether it is the truth or an error.
To explain this absurdity more fully, let’s take just one of these trials as an example, one related to a common and serious condition: COPD
The study is by Prof Frass and was published in 2005 – surely long enough ago for plenty of independent replications to emerge. Its results showed that potentized (C30) potassium dichromate decreases the amount of tracheal secretions was reduced, extubation could be performed significantly earlier, and the length of stay was significantly shorter. This is a scientific as well as clinical sensation, if there ever was one!
The RCT was published in one of the leading journals on this subject (Chest) which is read by most specialists in the field, and it was at the time widely reported. Even today, there is hardly an interview with Prof Frass in which he does not boast about this trial with truly sensational results (only last week, I saw one). If Frass is correct, his findings would revolutionise the lives of thousands of seriously suffering patients at the very brink of death. In other words, it is inconceivable that Frass’ result has not been replicated!
But it hasn’t; at least there is nothing in Medline.
Why not? A risk-free, cheap, universally available and easy to administer treatment for such a severe, life-threatening condition would normally be picked up instantly. There should not be one, but dozens of independent replications by now. There should be several RCTs testing Frass’ therapy and at least one systematic review of these studies telling us clearly what is what.
But instead there is a deafening silence.
For heaven sakes, why?
The only logical explanation is that many centres around the world did try Frass’ therapy. Most likely they found it does not work and soon dismissed it. Others might even have gone to the trouble of conducting a formal study of Frass’ ‘sensational’ therapy and found it to be ineffective. Subsequently they felt too silly to submit it for publication – who would not laugh at them, if they said they trailed a remedy that was diluted 1: 1000000000000000000000000000000000000000000000000000000000000 and found it to be worthless? Others might have written up their study and submitted it for publication, but got rejected by all reputable journals in the field because the editors felt that comparing one placebo to another placebo is not real science.
And this is roughly, how it went with the other ‘positive’ and seemingly rigorous studies of homeopathy as well, I suspect.
Regardless of whether I am correct or not, the fact is that there are no independent replications (if readers know any, please let me know).
Once a sufficiently long period of time has lapsed and no replications of a ‘sensational’ finding did not emerge, the finding becomes unbelievable or bogus – no rational thinker can possibly believe such a results (I for one have not yet met an intensive care specialist who believes Frass’ findings, for instance). Subsequently, it is quietly dropped into the waste-basket of science where it no longer obstructs progress.
The absence of independent replications is therefore a most useful mechanism by which science rids itself of falsehoods.
It seems that homeopathy is such a falsehood.
We have repeatedly discussed the journal ‘Evidence-Based Complementary and Alternative Medicine’ (see for instance here and here). The journal has recently done something remarkable and seemingly laudable: it retracted an article titled “Psorinum Therapy in Treating Stomach, Gall Bladder, Pancreatic, and Liver Cancers: A Prospective Clinical Study” due to concerns about the ethics, authorship, quality of reporting, and misleading conclusions.***
Aradeep and Ashim Chatterjee own and manage the Critical Cancer Management Research Centre and Clinic (CCMRCC), the private clinic to which they are affiliated. The methods state “The study protocol was approved by the Institutional Review Board (IRB approval Number: 2001–05) of the CCMRCC” in 2001, but a 2014 review of Psorinum therapy said CCMRCC was founded in 2008. The study states “The participants received the drug Psorinum along with allopathic and homeopathic supportive treatments without trying conventional or any other investigational cancer treatments”; withholding conventional cancer treatment raises ethical concerns.
We asked the authors and their institutions for documentation of the ethics approval, the study protocol, and a blank copy of the informed consent form. However, the corresponding author, Aradeep Chatterjee, was reported to have been arrested in June 2017 for allegedly practising medicine without the correct qualifications and his co-author and father Ashim Chatterjee was reported to have been arrested in August; the Chatterjees and their legal representative did not respond to our queries. The co-authors Syamsundar Mandal, Sudin Bhattacharya, and Bishnu Mukhopadhyay said they did not agree to be authors of the article and were not aware of its submission; co-author Jaydip Biswas did not respond.
A member of the editorial board noted that although the discussion stated that “The limitation of this study is that it did not have any placebo or treatment control arm; therefore, it cannot be concluded that Psorinum Therapy is effective in improving the survival and the quality of life of the participants due to the academic rigours of the scientific clinical trials”, the abstract was misleading because it implied Psorinum therapy is effective in cancer treatment. The study design was described as a “prospective observational clinical trial”, but it cannot have been both observational and a clinical trial.
(*** while I wrote this blog (13/3/18) the abstract of this paper was still available on Medline without a retraction notice)
In case you wonder what ‘psorinum therapy’ is, this website explains:
A cancer specialist and Psorinum clinical researcher, Aurodeep Chaterjee, believes Psorinum Therapy is less time consuming and more economical for treatment of cancer. ‘The advantage of this treatment is that the patient can continue this treatment while staying home and the hospitalization is less required,’ said Chaterjee. He added that it’s an immunotherapy treatment in which the medicine is in liquid form and the technique of consumption is oral.
Though no chemo or radiation sessions are required in it but they can be used parallel to it depending upon the stage of the cancer. He claimed that more than 30 types of cancers could be treated from this therapy. Some of them include gastrointestinal cancer, liver cancer, gall bladder cancer, ovarian cancer, stomach cancer, etc. The process requires two months duration in which the patient has to undergo 12 cycles and the cost is just Rs 5000. Moli Rapoor 55, software engineer from USA who is suffering from ovarian cancer said on Thursday (June 20) that after three chemo cycles when her cancer did not cure after being diagnosed in 2008, she decided to take up Psorinum therapy.
I am sorry, but the retraction of such a paper is far less laudable than it seems – it should not have been retracted, but it should have never been published in the first place. There are multiple points where the reviewers’ and editors’ alarm bells should have started ringing loud and clear. Take, for instance, this note at the end of the paper:
Dr. Rabindranath Chatterjee Memorial Cancer Trust provided funding for this study.
Conflict of Interests
The authors declare that they have no conflict of interests.
What this story shows, in my view, is that the journal ‘Evidence-Based Complementary and Alternative Medicine’ (EBCAM) operates an unacceptably poor system of peer-review, and is led by an editor who seems to shut both eyes when deciding about publication or rejection. And why would an editor shut his/her eyes to abuse? Perhaps the journal’s interesting business model provides an explanation? Here is what I wrote about it previously:
What I fail to understand is why so many researchers send their papers to this journal. In 2015, EBCAM published just under 1000 (983 to be exact) papers. This is not far from half of all Medline-listed articles on alternative medicine (2056 in total).
To appreciate these figures – and this is where it gets not just puzzling but intriguing, in my view – we need to know that EBCAM charges a publication fee of US$ 2500. That means the journal has an income of about US$ 2 500 000 per annum!
END OF QUOTE
To put it in a nutshell: in healthcare, fraud and greed can cause enormous harm.
The UK ACUPPUNCTURE RESEARCH RESOURCE CENTRE (ARRC) is a specialist resource for acupuncture research information; the only such resource in the land. It is funded by the British Acupuncture Council (BAcC) and was established in 1994 by the BAcC in partnership with the Foundation for Research in Traditional Chinese Medicine.
The ARRC organise an annual meeting. This year’s meeting is special because it is their 20th! It is scheduled to take place in London on 17th March. In case you are already busy that day, or you want to save the £120 registration fee, I have copied for you the programme below and am even able to inform you about the content of each lecture.
- Hugh MacPherson – Celebrating twenty years of acupuncture research
- Lee Hullender Rubin – The Impact of Whole Systems Traditional Chinese Medicine on In Vitro Fertilization Outcomes – A Retrospective Cohort Study
- Robert Davis – Beyond Efficacy: Conducting and translating research for policy-makers considering acupuncture reimbursement in a small, rural US state
- Lee Hullender Rubin – Acupuncture Augmentation of Lidocaine Treatment of Provoked, localized Vulvodynia – a Feasability and Acceptability Pilot Study
- Florian Beissner – A TCM-based psychotherapy with acupuncture for endometriosis
- Beverley De Valois – Using moxa on St 36 to reduce chemotherapy-induced pancytopenia: a feasibility study
- Ian Appleyard – Warm needle acupuncture for osteoarthritis of the knee: a pilot study
- Ed Fraser – Stand Easy: An Evaluation of the acceptability and effectiveness of acupuncture as a treatment for post-traumatic stress disorder for veterans in Norfolk
Having attended plenty of such meetings in my time, I can give you a fairly good idea about the contents of the 8 lectures. Below, I provide succinct (and slightly satirical) summaries of what the presenters will tell their audience on the 17th:
- Despite difficult circumstances, we (the ARRC) have done very well indeed. We managed to publish lots of papers, and we made sure that not a single one reported a negative result. That would be bad for business. We are optimistic about the future provided we get some funding, of course.
- Whole Systems Traditional Chinese Medicine has a profoundly positive effect on the outcomes of In Vitro fertilization. We are totally balled over! Only the most pedantic sceptics would have reservations and might argue that the study had no controls and was retrospective. But who cares, we believe in positive results, and therefore, we never listen to criticism.
- Because efficacy is a sticky issue in the realm of acupuncture, it is much wiser to tackle policy makers by persuading them that they can save money (lots of it), if they implement the abundant use of acupuncture. The evidence for this notion is flimsy to say the least, but policy makers do not understand the science (and neither do we).
- Our study showed that Acupuncture Augmentation of Lidocaine Treatment is extremely good for vulvodynia. We are very impressed, over the moon even. Of course, this was a feasibility study and we should really only conclude that a full study may be feasible, but let’s not be nit-picking.
- Based on my very extensive experience, I am able to confirm that TCM-based psychotherapy with acupuncture is an excellent therapy for endometriosis. Rigorous, controlled clinical trials do not exist, but my findings are so clear that, quite honestly, we do not need them.
- Using moxa on St 36 to reduce chemotherapy-induced pancytopenia is feasible. Isn’t that lovely?
- My trial of warm needle acupuncture for osteoarthritis of the knee showed most encouraging results. Of course, this was only a pilot study, and from it we should really only conclude that a proper study may be feasible, but let’s not be holier than thou!
- Our results demonstrate that acupuncture as a treatment for post-traumatic stress disorder is amazingly effective. A breakthrough! What is more, veterans found it most acceptable. The study is not rigorous, but I don’t mind. I advocate this treatment to be rolled out nationally as a matter of urgency.
So, there you are; that’s all you need to know about the 20th annual meeting of the ARRC.
You don’t need to go.
I have thus saved you £120!
No, I don’t expect thanks – I prefer, if you would send half of this amount (£60) to my account.
Clinical trials are a most useful tool, but they can easily be abused. It is not difficult to misuse them in such a way that even the most useless treatment appears to be effective. Sadly, this sort of thing happens all too often in the realm of alternative medicine. Take for instance this recently published trial of homeopathy.
The objective of this study was to investigate the usefulness of classical homeopathy for the prevention of recurrent urinary tract infections (UTI) in patients with spinal cord injury (SCI). Patients were admitted to this trial, if they had chronic SCI and had previously suffered from at least three UTI/year. They were treated either with a standardized prophylaxis alone, or with a standardized prophylaxis in combination with homeopathy. The number of UTIs, general and specific quality of life (QoL), and satisfaction with homeopathic treatment were assessed prospectively over the period of one year. Ten patients were in the control group and 25 patients received adjunctive homeopathic treatment. The median number of self-reported UTI in the homeopathy group decreased significantly, whereas it remained unchanged in the control group. The domain incontinence impact of the KHQ improved significantly, whereas the general QoL did not change. The satisfaction with homeopathic care was high.
The authors concluded that adjunctive homeopathic treatment lead to a significant decrease of UTI in SCI patients. Therefore, classical homeopathy could be considered in SCI patients with recurrent UTI.
Where to begin?
Here are just some of the most obvious flaws of and concerns with this study:
- There is no plausible rationale to even plan such a study.
- The sample size was far too small for allowing generalizable conclusions.
- There was no adequate randomisation and patients were able to chose the homeopathy option.
- The study seems to lack objective outcome measures.
- The study design did not allow to control for non-specific effects; therefore, it seems likely that the observed outcomes are unrelated to the homeopathic treatments but are caused by placebo and other non-specific effects.
- Even if the study had been rigorous, we would need independent replications before we draw such definitive conclusions.
- Two of the authors are homeopaths, and it is in their clinics that the study took place.
- Some of the authors have previously published a very similar paper – except that this ‘case series’ included no control group at all.
- The latter paper seems to have been published more than once.
- Of this paper, one of the authors claimed that ” the usefulness of classical homeopathy as an adjunctive measure for UTI prophylaxis in patients with NLUTD due to SCI has been demonstrated in a case series”. He seems to be unaware of the fact that a case series cannot possible lend itself to demonstrate this.
- I do wonder: did they just add a control group to their case series thus pretending it became a controlled clinical trial?
What strikes me most with such pseudo-research is its abundance and the naivety – or should I call it ignorance? – of the enthusiasts who conduct it. Most of them, I am fairly sure do not mean to do harm; but by Jove they do!
The question whether spinal manipulative therapy (SMT) has any specific therapeutic effects is still open. This fact must irritate ardent chiropractors, and they therefore try everything to dispel our doubts. One way would be to demonstrate a dose-effect relationship between SMT and the clinical outcome. But, for several reasons, this is not an easy task.
This RCT was aimed at identifying the dose-response relationship between visits for SMT and chronic cervicogenic headache (CGH) outcomes; to evaluate the efficacy of SMT by comparison with a light massage control.
The study included 256 adults with chronic CGH. The primary outcome was days with CGH in the prior 4 weeks evaluated at the 12- and 24-week primary endpoints. Secondary outcomes included CGH days at remaining endpoints, pain intensity, disability, perceived improvement, medication use, and patient satisfaction. Participants were randomized to 4 different dose levels of chiropractic SMT: 0, 6, 12, or 18 sessions. They were treated 3 times per week for 6 weeks and received a focused light-massage control at sessions when SMT was not assigned. Linear dose effects and comparisons to the no-manipulation control group were evaluated at 6, 12, 24, 39, and 52 weeks.
A linear dose-response was observed for all follow-ups, a reduction of approximately 1 CGH day/4 weeks per additional 6 SMT visits (p<.05); a maximal effective dose could not be determined. CGH days/4 weeks were reduced from about 16 to 8 for the highest and most effective dose of 18 SMT visits. Mean differences in CGH days/4 weeks between 18 SMT visits and control were -3.3 (p=.004) and -2.9 (p=.017) at the primary endpoints, and similar in magnitude at the remaining endpoints (p<.05). Differences between other SMT doses and control were smaller in magnitude (p > .05). CGH intensity showed no important improvement nor differed by dose. Other secondary outcomes were generally supportive of the primary.
The authors concluded that there was a linear dose-response relationship between SMT visits and days with CGH. For the highest and most effective dose of 18 SMT visits, CGH days were reduced by half, and about 3 more days per month than for the light-massage control.
This trial would make sense, if the effectiveness of SMT for CGH had been a well-documented fact, and if the study had rigorously controlled for placebo-effects.
But guess what?
Neither of these conditions were met.
A recent review concluded that there are few published randomized controlled trials analyzing the effectiveness of spinal manipulation and/or mobilization for TTH, CeH, and M in the last decade. In addition, the methodological quality of these papers is typically low. Clearly, there is a need for high-quality randomized controlled trials assessing the effectiveness of these interventions in these headache disorders. And this is by no means the only article making such statements; similar reviews arrive at similar conclusions. In turn, this means that the effects observed after SMT are not necessarily specific effects due to SMT but could easily be due to placebo or other non-specific effects. In order to avoid confusion, one would need a credible placebo – one that closely mimics SMT – and make sure that patients were ‘blinded’. But ‘light massage’ clearly does not mimic SMT, and patients obviously were aware of which interventions they received.
So, an alternative – and I think at least as plausible – conclusion of the data provided by this new RCT is this:
Chiropractic SMT is associated with a powerful placebo response which, of course, obeys a dose-effect relationship. Thus these findings are in keeping with the notion that SMT is a placebo.
And why would the researchers – who stress that they have no conflicts of interest – mislead us by making this alternative interpretation of their findings not abundantly clear?
I fear, the reason might be simple: they also seem to mislead us about their conflicts of interest: they are mostly chiropractors with a long track record of publishing promotional papers masquerading as research. What, I ask myself, could be a stronger conflict of interest?
(Pity that a high-impact journal like SPINE did not spot these [not so little] flaws)
The chiropractor Oakley Smith had graduated under D D Palmer in 1899. Smith was a former Iowa medical student who also had investigated Andrew Still’s osteopathy in Kirksville, before going to Palmer in Davenport. Eventually, Smith came to reject the Palmer concept of vertebral subluxation and developed his own concept of “the connective tissue doctrine” or naprapathy. Today, naprapathy is a popular form of manual therapy, particularly in Scandinavia and the US.
But what exactly is naprapathy? This website explains it quite well: Naprapathy is defined as a system of specific examination, diagnostics, manual treatment and rehabilitation of pain and dysfunction in the neuromusculoskeletal system. The therapy is aimed at restoring function through treatment of the connective tissue, muscle- and neural tissues within or surrounding the spine and other joints. Naprapathic treatment consists of combinations of manual techniques for instance spinal manipulation and mobilization, neural mobilization and Naprapathic soft tissue techniques, in additional to the manual techniques Naprapaths uses different types of electrotherapy, such as ultrasound, radial shockwave therapy and TENS. The manual techniques are often combined with advice regarding physical activity and ergonomics as well as medical rehabilitation training in order to decrease pain and disability and increase work ability and quality of life. A Dr. of Naprapathy is specialized in the diagnosis of structural and functional neuromusculoskeletal disorders, treatment and rehabilitation of patients with problems of such origin as well as to differentiate pain of other origin.
DOCTOR OF NAPRAPATHY? I hear you shout.
Yes, in the US, the title exists: The National College of Naprapathic Medicine is chartered by the State of Illinois and recognized by the State Board of Higher Education to grant the degree, Doctor of Naprapathy (D.N.). Graduates of the College are eligible to take the Naprapathic Medicine examination for licensure in the State of Illinois. The D.N. Degree requires:
- 66 hours – Basic Sciences
- 64 hours – Naprapathic Sciences
- 60 hours – Clinical Internship
Things become even stranger when we ask, what does the evidence show?
I found all of three clinical trials on Medline.
A 2016 clinical trial was designed to compare the treatment effect on pain intensity, pain related disability and perceived recovery from a) naprapathic manual therapy (spinal manipulation, spinal mobilization, stretching and massage) to b) naprapathic manual therapy without spinal manipulation and to c) naprapathic manual therapy without stretching for male and female patients seeking care for back and/or neck pain.
Participants were recruited among patients, ages 18-65, seeking care at the educational clinic of Naprapathögskolan – the Scandinavian College of Naprapathic Manual Medicine in Stockholm. The patients (n = 1057) were randomized to one of three treatment arms a) manual therapy (i.e. spinal manipulation, spinal mobilization, stretching and massage), b) manual therapy excluding spinal manipulation and c) manual therapy excluding stretching. The primary outcomes were minimal clinically important improvement in pain intensity and pain related disability. Treatments were provided by naprapath students in the seventh semester of eight total semesters. Generalized estimating equations and logistic regression were used to examine the association between the treatments and the outcomes.
At 12 weeks follow-up, 64% had a minimal clinically important improvement in pain intensity and 42% in pain related disability. The corresponding chances to be improved at the 52 weeks follow-up were 58% and 40% respectively. No systematic differences in effect when excluding spinal manipulation and stretching respectively from the treatment were found over 1 year follow-up, concerning minimal clinically important improvement in pain intensity (p = 0.41) and pain related disability (p = 0.85) and perceived recovery (p = 0.98). Neither were there disparities in effect when male and female patients were analyzed separately.
The authors concluded that the effect of manual therapy for male and female patients seeking care for neck and/or back pain at an educational clinic is similar regardless if spinal manipulation or if stretching is excluded from the treatment option.
Even though this study is touted as showing that naprapathy works by advocates, in all honesty, it tells us as good as nothing about the effect of naprapathy. The data are completely consistent with the interpretation that all of the outcomes were to the natural history of the conditions, regression towards the mean, placebo, etc. and entirely unrelated to any specific effects of naprapathy.
A 2010 study by the same group was to compare the long-term effects (up to one year) of naprapathic manual therapy and evidence-based advice on staying active regarding non-specific back and/or neck pain.
Subjects with non-specific pain/disability in the back and/or neck lasting for at least two weeks (n = 409), recruited at public companies in Sweden, were included in this pragmatic randomized controlled trial. The two interventions compared were naprapathic manual therapy such as spinal manipulation/mobilization, massage and stretching, (Index Group), and advice to stay active and on how to cope with pain, provided by a physician (Control Group). Pain intensity, disability and health status were measured by questionnaires.
89% completed the 26-week follow-up and 85% the 52-week follow-up. A higher proportion in the Index Group had a clinically important decrease in pain (risk difference (RD) = 21%, 95% CI: 10-30) and disability (RD = 11%, 95% CI: 4-22) at 26-week, as well as at 52-week follow-ups (pain: RD = 17%, 95% CI: 7-27 and disability: RD = 17%, 95% CI: 5-28). The differences between the groups in pain and disability considered over one year were statistically significant favoring naprapathy (p < or = 0.005). There were also significant differences in improvement in bodily pain and social function (subscales of SF-36 health status) favoring the Index Group.
The authors concluded that combined manual therapy, like naprapathy, is effective in the short and in the long term, and might be considered for patients with non-specific back and/or neck pain.
This study is hardly impressive either. The results are consistent with the interpretation that the extra attention and care given to the index group was the cause of the observed outcomes, unrelated to ant specific effects of naprapathy.
The last study was published in 2017 again by the same group. It was designed to compare naprapathic manual therapy with evidence-based care for back or neck pain regarding pain, disability, and perceived recovery.
Four hundred and nine patients with pain and disability in the back or neck lasting for at least 2 weeks, recruited at 2 large public companies in Sweden in 2005, were included in this randomized controlled trial. The 2 interventions were naprapathy, including spinal manipulation/mobilization, massage, and stretching (Index Group) and support and advice to stay active and how to cope with pain, according to the best scientific evidence available, provided by a physician (Control Group). Pain, disability, and perceived recovery were measured by questionnaires at baseline and after 3, 7, and 12 weeks.
At 7-week and 12-week follow-ups, statistically significant differences between the groups were found in all outcomes favoring the Index Group. At 12-week follow-up, a higher proportion in the naprapathy group had improved regarding pain [risk difference (RD)=27%, 95% confidence interval (CI): 17-37], disability (RD=18%, 95% CI: 7-28), and perceived recovery (RD=44%, 95% CI: 35-53). Separate analysis of neck pain and back pain patients showed similar results.
The authors thought that this trial suggests that combined manual therapy, like naprapathy, might be an alternative to consider for back and neck pain patients.
As the study suffers from the same limitations as the one above (in fact, it might be a different analysis of the same trial), they might be mistaken. I see no good reason to assume that any of the three studies provide good evidence for the effectiveness of naprapathy.
So, what should we conclude from all this?
If you ask me, naprapathy is something between chiropractic (without some of the woo) and physiotherapy (without its expertise). There is no good evidence that it works. Crucially, there is no evidence that it is superior to other therapeutic options.
I was going to finish on a positive note stating that ‘at least the ‘naprapathologists’ (I refuse to even consider the title of ‘doctor of naprapathy’) do not claim to treat conditions other than musculoskeletal problems’. But then I found this advertisement of a ‘naprapathologist’ on Twitter:
And now, I am going to finish by stating that A LOT OF NAPRAPATHY LOOKS VERY MUCH LIKE QUACKERY TO ME.
Sipjeondaebo-tang is an East Asian herbal supplement containing Angelica root (Angelicae Gigantis Radix), the rhizome of Cnidium officinale Makino (Cnidii Rhizoma), Radix Paeoniae, Rehmannia glutinosa root (Rehmanniae Radix Preparata), Ginseng root (Ginseng Radix Alba), Atractylodes lancea root (Atractylodis Rhizoma Alba), the dried sclerotia of Poria cocos (Poria cocos Sclerotium), Licorice root (Glycyrrhizae Radix), Astragalus root (Astragali Radix), and the dried bark of Cinnamomum verum (Cinnamomi Cortex).
But does this herbal mixture actually work? Korean researchers wanted to find out.
The purpose of their study was to examine the feasibility of Sipjeondaebo-tang (Juzen-taiho-to, Shi-Quan-Da-Bu-Tang) for cancer-related anorexia. A total of 32 participants with cancer anorexia were randomized to either Sipjeondaebo-tang group or placebo group. Participants were given 3 g of Sipjeondaebo-tang or placebo 3 times a day for 4 weeks. The primary outcome was a change in the Anorexia/Cachexia Subscale of Functional Assessment of Anorexia/Cachexia Therapy (FAACT). The secondary outcomes included Visual Analogue Scale (VAS) of anorexia, FAACT scale, and laboratory tests.
The results showed that anorexia and quality of life measured by FAACT and VAS were improved after 4 weeks of Sipjeondaebo-tang treatment. However, there was no significant difference between changes of Sipjeondaebo-tang group and placebo group.
From this, the authors of the study concluded that sipjeondaebo-tang appears to have potential benefit for anorexia management in patients with cancer. Further large-scale studies are needed to ensure the efficacy.
Well, isn’t this just great? Faced with a squarely negative result, one simply ignores it and draws a positive conclusion!
As we all know – and as trialists certainly must know – controlled trials are designed to compare the outcomes of two groups. Changes within one of the groups can be caused by several factors unrelated to the therapy and are therefore largely irrelevant. This means that “no significant difference between changes of Sipjeondaebo-tang group and placebo group” indicates that the herbal mixture had no effect. In turn this means that a conclusion stating that “sipjeondaebo-tang appears to have potential benefit for anorexia” is just fraudulent.
This level of scientific misconduct is remarkable, even for the notoriously poor Evid Based Complement Alternat Med.
I strongly suggest that:
- The journal is de-listed from Medline because similarly misleading nonsense has been coming out of this rag for some time.
- The paper is withdrawn because it can only mislead vulnerable patients.
Cranio-sacral therapy is firstly implausible, and secondly it lacks evidence of effectiveness (see for instance here, here, here and here). Yet, some researchers are nevertheless not deterred to test it in clinical trials. While this fact alone might be seen as embarrassing, the study below is a particular and personal embarrassment to me, in fact, I am shocked by it and write these lines with considerable regret.
Why? Bear with me, I will explain later.
The purpose of this trial was to evaluate the effectiveness of osteopathic manipulative treatment and osteopathy in the cranial field in temporomandibular disorders. Forty female subjects with temporomandibular disorders lasting at least three months were included. At enrollment, subjects were randomly assigned into two groups: (1) osteopathic manipulative treatment group (n=20) and (2) osteopathy in the cranial field [craniosacral therapy for you and me] group (n=20). Examinations were performed at baseline (E0) and at the end of the last treatment (E1), and consisted of subjective pain intensity with the Visual Analog Scale, Helkimo Index and SF-36 Health Survey. Subjects had five treatments, once a week. 36 subjects completed the study.
Patients in both groups showed significant reduction in Visual Analog Scale score (osteopathic manipulative treatment group: p = 0.001; osteopathy in the cranial field group: p< 0.001), Helkimo Index (osteopathic manipulative treatment group: p = 0.02; osteopathy in the cranial field group: p = 0.003) and a significant improvement in the SF-36 Health Survey – subscale “Bodily Pain” (osteopathic manipulative treatment group: p = 0.04; osteopathy in the cranial field group: p = 0.007) after five treatments (E1). All subjects (n = 36) also showed significant improvements in the above named parameters after five treatments (E1): Visual Analog Scale score (p< 0.001), Helkimo Index (p< 0.001), SF-36 Health Survey – subscale “Bodily Pain” (p = 0.001). The differences between the two groups were not statistically significant for any of the three endpoints.
The authors concluded that both therapeutic modalities had similar clinical results. The findings of this pilot trial support the use of osteopathic manipulative treatment and osteopathy in the cranial field as an effective treatment modality in patients with temporomandibular disorders. The positive results in both treatment groups should encourage further research on osteopathic manipulative treatment and osteopathy in the cranial field and support the importance of an interdisciplinary collaboration in patients with temporomandibular disorders. Implications for rehabilitation Temporomandibular disorders are the second most prevalent musculoskeletal condition with a negative impact on physical and psychological factors. There are a variety of options to treat temporomandibular disorders. This pilot study demonstrates the reduction of pain, the improvement of temporomandibular joint dysfunction and the positive impact on quality of life after osteopathic manipulative treatment and osteopathy in the cranial field. Our findings support the use of osteopathic manipulative treatment and osteopathy in the cranial field and should encourage further research on osteopathic manipulative treatment and osteopathy in the cranial field in patients with temporomandibular disorders. Rehabilitation experts should consider osteopathic manipulative treatment and osteopathy in the cranial field as a beneficial treatment option for temporomandibular disorders.
This study has so many flaws that I don’t know where to begin. Here are some of the more obvious ones:
- There is, as already mentioned, no rationale for this study. I can see no reason why craniosacral therapy should work for the condition. Without such a rationale, the study should never even have been conceived.
- Technically, this RCTs an equivalence study comparing one therapy against another. As such it needs to be much larger to generate a meaningful result and it also would require a different statistical approach.
- The authors mislabelled their trial a ‘pilot study’. However, a pilot study “is a preliminary small-scale study that researchers conduct in order to help them decide how best to conduct a large-scale research project. Using a pilot study, a researcher can identify or refine a research question, figure out what methods are best for pursuing it, and estimate how much time and resources will be necessary to complete the larger version, among other things.” It is not normally a study suited for evaluating the effectiveness of a therapy.
- Any trial that compares one therapy of unknown effectiveness to another of unknown effectiveness is a complete and utter nonsense. Equivalent studies can only ever make sense, if one of the two treatments is of proven effectiveness – think of it as a mathematical equation: one equation with two unknowns is unsolvable.
- Controlled studies such as RCTs are for comparing the outcomes of two or more groups, and only between-group differences are meaningful results of such trials.
- The ‘positive results’ which the authors mention in their conclusions are meaningless because they are based on such within-group changes and nobody can know what caused them: the natural history of the condition, regression towards the mean, placebo-effects, or other non-specific effects – take your pick.
- The conclusions are a bonanza of nonsensical platitudes and misleading claims which do not follow from the data.
As regular readers of this blog will doubtlessly have noticed, I have seen plenty of similarly flawed pseudo-research before – so, why does this paper upset me so much? The reason is personal, I am afraid: even though I do not know any of the authors in person, I know their institution more than well. The study comes from the Department of Physical Medicine and Rehabilitation, Medical University of Vienna, Austria. I was head of this department before I left in 1993 to take up the Exeter post. And I had hoped that, even after 25 years, a bit of the spirit, attitude, knowhow, critical thinking and scientific rigor – all of which I tried so hard to implant in my Viennese department at the time – would have survived.
Perhaps I was wrong.
Some say that Chinese herbal medicine offers a solution.
This Chinese multi-centre RCT included 588 mothers considering breastfeeding. The intervention group received the Chinese herbal mixture Zengru Gao, while the control group received no therapy. The primary outcomes were the percentages of fully and partially breastfeeding mothers, and a secondary outcome was baby’s daily formula intake.
At day 3 and 7 after delivery, significant differences were found in favour of Zengru Gao group on the percentage of full/ partial breastfeeding. At day 7, the percentage of full/ partial breastfeeding of the active group increased to 71.48%/20.70% versus 58.67%/30.26% in the control group, the differences remained significant. No statistically significant differences were detected on primary measures at day. While intake of formula differed between groups at day 1 and 3, this difference did not achieve statistical significance, but this difference was apparent by day 7.
The authors concluded that the Chinese Herbal medicine Zengru Gao enhanced breastfeeding success during one week postpartum. The approach is acceptable to participants and merits further evaluation.
To the naïve observer, this study might look rigorous, but it is a seriously flawed RCT. Here are just some of its most obvious limitations:
- All we get in the methods section is this explanation: Participants were randomly allocated to the blank control group or the intervention group: Zengru Gao, orally, 30 g a time and 3 times a day. This seems to indicate that the control group got no treatment at all which means there was no blinding nor placebo control. The authors even comment on this point in the discussion section of their paper stating that because we included new mothers who received no treatment as a control group, we were able to prove that the improvement in breastfeeding was not due to the placebo effect. However, this is a totally nonsensical argument.
- The experimental treatment is not reproducible. The authors state: Zengru Gao, a Chinese herbal formula, which is composed of 8 herbs: Semen Vaccariae, Medulla Tetrapanacis, Radix Rehmanniae Praeparata, Radix Angelicae Sinensis, Radix Paeoniae Alba,Rhizoma Chuanxiong, Herba Leonuri, Radix Trichosanthis. This is not enough information to replicate the study outside China where the mixture is not commercially available.
- The primary outcome was the percentage of fully, and partially breastfeeding mothers. Breastfeeding was defined as mother’s milk given by direct breast feeding. Full breastfeeding meant that no other types of milk or solids were given. Partially breastfeeding meant that sustained latch with deep rhythmic sucking through the length of the feed, with some pause, on either/ or both breasts. We are not being told how the endpoint was quantified. Presumably women kept diaries. We cannot guess how accurate this process was.
- As far as I can see, there was no correction for multiple testing for statistical significance. This means that some or all of the significant results might be false-positive.
- There is insufficient data to show that the herbal mixture is safe for the mothers and the babies. At the very minimum, the researchers should have measured essential safety parameters. This omission is a gross violation of research ethics.
- Towards the end of the paper, we find the following statement: The authors would like to thank the Research and Development Department of Zhangzhou Pien Tze Huang Pharmaceutical co., Ltd. … The authors declare that they have no competing interests. And the 1st and 3rd authors are “affiliated with” Guangzhou Hipower Pharmaceutical Technology Co., Ltd, Guangzhou, China, i. e. work for the manufacturer of the mixture. This does clearly not make any sense whatsoever.
I have seen too many flawed studies of alternative medicine to be shocked or even surprised by this level of incompetence and nonsense. Yet, I still find it lamentable. But, in my view, the worst is that supposedly peer-reviewed journals such as ‘BMC Complement Altern Med’ publish such overt rubbish.
It would be easy to shrug one’s shoulder and bin the paper. But the effect of such fatally flawed research is too serious for that. In our recent book MORE HARM THAN GOOD? THE MORAL MAZE OF COMPLEMENTARY AND ALTERNATIVE MEDICINE, we discuss that such flawed science amounts to a violation of medical ethics: CAM journals allocate peer review tasks to a narrow range of CAM enthusiasts who often have been chosen by the authors of the article in question. The raison d’être of CAM journals and CAM researchers is inextricably tied to a belief in CAM, resulting in a self-referential situation which is permissive to the acceptance of weak or ﬂawed reports of clinical effectiveness… Defective research—whether at the design, execution, analysis, or reporting stage—corrupts the repository of reliable medical knowledge. Ultimately, this leads to suboptimal and erroneous treatment decisions…
Many hard-nosed sceptics might claim that there is no herbal treatment for upper respiratory infections that makes the slightest difference difference. But is this assumption really correct?
According to my own research of 2004, it is not. Here is the abstract of our systematic review:
Acute respiratory infections represent a significant cause of over-prescription of antibiotics and are one of the major reasons for absence from work. The leaves of Andrographis paniculata (Burm. f.) Wall ex Nees (Acanthaceae) are used as a medicinal herb in the treatment of infectious diseases. Systematic literature searches were conducted in six computerised databases and the reference lists of all papers located were checked for further relevant publications. Information was also requested from manufacturers, the spontaneous reporting schemes of the World Health Organisation and national drug safety bodies. No language restrictions were imposed. Seven double-blind, controlled trials (n = 896) met the inclusion criteria for evaluation of efficacy. All trials scored at least three, out of a maximum of five, for methodological quality on the Jadad scale. Collectively, the data suggest that A. paniculata is superior to placebo in alleviating the subjective symptoms of uncomplicated upper respiratory tract infection. There is also preliminary evidence of a preventative effect. Adverse events reported following administration of A. paniculata were generally mild and infrequent. There were few spontaneous reports of adverse events. A. paniculata may be a safe and efficacious treatment for the relief of symptoms of uncomplicated upper respiratory tract infection; more research is warranted.
A. Paniculata (Burm.f.) Wall ex Nees (Acanthaceae family), also known as nemone chinensi, Chuān Xīn Lián, has traditionally been used in Indian and Chinese herbal medicine mostly as an antipyretic for relieving and reducing the severity and duration of symptoms of common colds and alleviating fever, cough and sore throats, or as a tonic to aid convalescence after uncomplicated respiratory tract infections. The active constituents of A. paniculata include the diterpene, lactones commonly known as the andrographolides which have shown anti-inflammatory, antiviral, anti-allergic, and immune-stimulatory activities. A. Paniculata has also been shown, in vitro, to be effective against avian influenza A (H9N2 and H5N1) and human influenza A H1N1 viruses, possibly through blocking the binding of viral hemagglutinin to cells, or by inhibiting H1N1 virus-induced cell death.
But our systematic review was published 14 years ago!
We need more up-to-date information!
And I am pleased to report that a recent paper provided exactly that.
This systematic review included published and unpublished RCTs. Quasi-RCTs, crossover trials, controlled before and after studies, interrupted time series (ITS) studies, and non-experimental studies were not included due to their potential high risk of bias.
Thirty-three trials involving 7175 patients with ARTIs were included. Their methodological quality was restricted as randomisation was not well documented; 73% of the trials included were not blinded; where ITT analysis were performed, loss to follow-up data were counted as no effect; and most trials were published without a protocol available.
Findings suggested limited but consistent evidence that A. Paniculata improved cough and sore throat when compared with placebo. A. Paniculata (alone or plus usual care) had a statistically significant effect in improving overall symptoms of ARTIs when compared to placebo, usual care, and other herbal therapies. A. Paniculata in pillule tended to be more effective in improving overall symptoms over A. Paniculata in tablet. Evidence also suggested that A. Paniculata (alone or plus usual care) shortens the duration of cough, sore throat and sick leave/time to resolution when compared versus usual care. Reduction in antibiotic usage was seldom evaluated in the included trials.
The authors concluded that A. Paniculata appears beneficial and safe for relieving ARTI symptoms and shortening time to symptom resolution. However, these findings should be interpreted cautiously owing to poor study quality and heterogeneity. Well-designed trials evaluating the effectiveness and potential to reduce antibiotic use of A. Paniculata are warranted.
In case you wonder about conflicts of interest: there were none with my 2004 paper, and the authors of the new review state that this paper presents independent research funded by the National Institute for Health Research School for Primary Care Research (NIHR SPCR). The views expressed are those of the author(s) and not necessarily those of the NIHR, the NHS or the Department of Health.
Yes, the RCTs are not all of top quality.
And yes, the effect size is not huge.
But maybe – just maybe – we do have here an alternative therapy that does help against a condition for which conventional drugs are fairly useless!?!