Reiki is a Japanese technique administered by “laying on hands” and is based on the idea that an unseen “life force energy” flows through us and is what causes us to be alive. If one’s “life force energy” is low, then we are more likely to get sick or feel stress, and if it is high, we are more capable of being happy and healthy (because it is such a clear-cut case of nonsense, we have discussed Reiki regularly; see for instance here, here, here, here, here, here, and here).
But nonsense does not stop researchers from conducting trials. In this new clinical trial, conducted in Physiotherapy Clinic of Khatam Al-Anbia Hospital in Iran, 60 patients with pain due to inter-vertebral disc herniation (IVDH) were randomly assigned to one of three groups.
- The Reiki group received three 15-minute Reiki sessions on consecutive days by a master of Reiki plus Indomethacin and Methocarbamol (as in group 3).
- The physiotherapy group underwent 7 to 10 sessions of physiotherapy of 60 to 90 minutes (heat therapy, TENS, pelvic traction, and physical exercises) plus Indomethacin and Methocarbamol (as in group 3).
- The drug group received Indomethacin capsules 75 mg and Methocarbamol tablets 500 mg every 8 hours daily for one week.
The severity of pain and the activities of daily living (ADL) were measured using visual analogue scales (VAS) and ADL-Instrumental ADL questionnaire before and after the intervention. A significant difference was found in pain intensity and ADL improvement between Reiki and the drug therapy. No significant difference between the Reiki and physiotherapy groups were noted.
The authors concluded that Reiki and physiotherapy are effective methods in managing pain and improving ADL in patients with IVDH; however, Reiki is more cost-effective and faster treatment method than physiotherapy.
This RCT seems fairly well-panned and conducted, and its results are straight forward. My only problem with it is how the findings are interpreted.
The study design was such that there was no blinding or control for placebo effects. Therefore, the observed outcomes can be interpreted in more than one way. In my view, by far the most plausible explanation is that Reiki (being an exotic, impressive intervention that generates plenty of expectation) produced a powerful placebo effect. Physiotherapy (being entirely normal and routine), on the other hand, was only marginally successful. It is regrettable that the authors do not even consider this interpretation of their results. They should have remembered that a clinical trial test the null-hypothesis (the experimental treatment is not better that the comparator) which can be rejected only, if there is no other reasonable explanation for the results produced.
If I am correct, the conclusions should be re-written as follows:
The addition of Reiki to drug treatment generated better outcomes than drug therapy alone. Physiotherapy was only marginally effective. The effects of Reiki are most likely not due to the treatment per se but to a classical placebo response.
As I often said, I find it regrettable that sceptics often say THERE IS NOT A SINGLE STUDY THAT SHOWS HOMEOPATHY TO BE EFFECTIVE (or something to that extent). This is quite simply not true, and it gives homeopathy-fans the occasion to suggest sceptics wrong. The truth is that THE TOTALITY OF THE MOST RELIABLE EVIDENCE FAILS TO SUGGEST THAT HIGHLY DILUTED HOMEOPATHIC REMEDIES ARE EFFECTIVE BEYOND PLACEBO. As a message for consumers, this is a little more complex, but I believe that it’s worth being well-informed and truthful.
And that also means admitting that a few apparently rigorous trials of homeopathy exist and some of them show positive results. Today, I want to focus on this small set of studies.
How can a rigorous trial of a highly diluted homeopathic remedy yield a positive result? As far as I can see, there are several possibilities:
- Homeopathy does work after all, and we have not fully understood the laws of physics, chemistry etc. Homeopaths favour this option, of course, but I find it extremely unlikely, and most rational thinkers would discard this possibility outright. It is not that we don’t quite understand homeopathy’s mechanism; the fact is that we understand that there cannot be a mechanism that is in line with the laws of nature.
- The trial in question is the victim of some undetected error.
- The result has come about by chance. Of 100 trials, 5 would produce a positive result at the 5% probability level purely by chance.
- The researchers have cheated.
When we critically assess any given trial, we attempt, in a way, to determine which of the 4 solutions apply. But unfortunately we always have to contend with what the authors of the trial tell us. Publications never provide all the details we need for this purpose, and we are often left speculating which of the explanations might apply. Whatever it is, we assume the result is false-positive.
Naturally, this assumption is hard to accept for homeopaths; they merely conclude that we are biased against homeopathy and conclude that, however, rigorous a study of homeopathy is, sceptics will not accept its result, if it turns out to be positive.
But there might be a way to settle the argument and get some more objective verdict, I think. We only need to remind ourselves of a crucially important principle in all science: INDEPENDENT REPLICATION. To be convincing, a scientific paper needs to provide evidence that the results are reproducible. In medicine, it unquestionably is wise to accept a new finding only after it has been confirmed by other, independent researchers. Only if we have at least one (better several) independent replications, can we be reasonably sure that the result in question is true and not false-positive due to bias, chance, error or fraud.
And this is, I believe, the extremely odd phenomenon about the ‘positive’ and apparently rigorous studies of homeopathic remedies. Let’s look at the recent meta-analysis of Mathie et al. The authors found several studies that were both positive and fairly rigorous. These trials differ in many respects (e. g. remedies used, conditions treated) but they have, as far as I can see, one important feature in common: THEY HAVE NOT BEEN INDEPENDENTLY REPLICATED.
If that is not astounding, I don’t know what is!
Think of it: faced with a finding that flies in the face of science and would, if true, revolutionise much of medicine, scientists should jump with excitement. Yet, in reality, nobody seems to take the trouble to check whether it is the truth or an error.
To explain this absurdity more fully, let’s take just one of these trials as an example, one related to a common and serious condition: COPD
The study is by Prof Frass and was published in 2005 – surely long enough ago for plenty of independent replications to emerge. Its results showed that potentized (C30) potassium dichromate decreases the amount of tracheal secretions was reduced, extubation could be performed significantly earlier, and the length of stay was significantly shorter. This is a scientific as well as clinical sensation, if there ever was one!
The RCT was published in one of the leading journals on this subject (Chest) which is read by most specialists in the field, and it was at the time widely reported. Even today, there is hardly an interview with Prof Frass in which he does not boast about this trial with truly sensational results (only last week, I saw one). If Frass is correct, his findings would revolutionise the lives of thousands of seriously suffering patients at the very brink of death. In other words, it is inconceivable that Frass’ result has not been replicated!
But it hasn’t; at least there is nothing in Medline.
Why not? A risk-free, cheap, universally available and easy to administer treatment for such a severe, life-threatening condition would normally be picked up instantly. There should not be one, but dozens of independent replications by now. There should be several RCTs testing Frass’ therapy and at least one systematic review of these studies telling us clearly what is what.
But instead there is a deafening silence.
For heaven sakes, why?
The only logical explanation is that many centres around the world did try Frass’ therapy. Most likely they found it does not work and soon dismissed it. Others might even have gone to the trouble of conducting a formal study of Frass’ ‘sensational’ therapy and found it to be ineffective. Subsequently they felt too silly to submit it for publication – who would not laugh at them, if they said they trailed a remedy that was diluted 1: 1000000000000000000000000000000000000000000000000000000000000 and found it to be worthless? Others might have written up their study and submitted it for publication, but got rejected by all reputable journals in the field because the editors felt that comparing one placebo to another placebo is not real science.
And this is roughly, how it went with the other ‘positive’ and seemingly rigorous studies of homeopathy as well, I suspect.
Regardless of whether I am correct or not, the fact is that there are no independent replications (if readers know any, please let me know).
Once a sufficiently long period of time has lapsed and no replications of a ‘sensational’ finding did not emerge, the finding becomes unbelievable or bogus – no rational thinker can possibly believe such a results (I for one have not yet met an intensive care specialist who believes Frass’ findings, for instance). Subsequently, it is quietly dropped into the waste-basket of science where it no longer obstructs progress.
The absence of independent replications is therefore a most useful mechanism by which science rids itself of falsehoods.
It seems that homeopathy is such a falsehood.
A few days ago, the German TV ‘FACT’ broadcast a film (it is in German, the bit on homeopathy starts at ~min 20) about a young woman who had her breast cancer first operated but then decided to forfeit subsequent conventional treatments. Instead she chose homeopathy which she received from Dr Jens Wurster at the ‘Clinica Sta Croce‘ in Lucano/Switzerland.
Elsewhere Dr Wurster stated this: Contrary to chemotherapy and radiation, we offer a therapy with homeopathy that supports the patient’s immune system. The basic approach of orthodox medicine is to consider the tumor as a local disease and to treat it aggressively, what leads to a weakening of the immune system. However, when analyzing all studies on cured cancer cases it becomes evident that the immune system is always the decisive factor. When the immune system is enabled to recognize tumor cells, it will also be able to combat them… When homeopathic treatment is successful in rebuilding the immune system and reestablishing the basic regulation of the organism then tumors can disappear again. I’ve treated more than 1000 cancer patients homeopathically and we could even cure or considerably ameliorate the quality of life for several years in some, advanced and metastasizing cases.
The recent TV programme showed a doctor at this establishment confirming that homeopathy alone can cure cancer. Dr Wurster (who currently seems to be a star amongst European homeopaths) is seen lecturing at the 2017 World Congress of Homeopathic Physicians in Leipzig and stating that a ‘particularly rigorous study’ conducted by conventional scientists (the senior author is Harald Walach!, hardly a conventional scientist in my book) proved homeopathy to be effective for cancer. Specifically, he stated that this study showed that ‘homeopathy offers a great advantage in terms of quality of life even for patients suffering from advanced cancers’.
This study did, of course, interest me. So, I located it and had a look. Here is the abstract:
Many cancer patients seek homeopathy as a complementary therapy. It has rarely been studied systematically, whether homeopathic care is of benefit for cancer patients.
We conducted a prospective observational study with cancer patients in two differently treated cohorts: one cohort with patients under complementary homeopathic treatment (HG; n = 259), and one cohort with conventionally treated cancer patients (CG; n = 380). For a direct comparison, matched pairs with patients of the same tumour entity and comparable prognosis were to be formed. Main outcome parameter: change of quality of life (FACT-G, FACIT-Sp) after 3 months. Secondary outcome parameters: change of quality of life (FACT-G, FACIT-Sp) after a year, as well as impairment by fatigue (MFI) and by anxiety and depression (HADS).
HG: FACT-G, or FACIT-Sp, respectively improved statistically significantly in the first three months, from 75.6 (SD 14.6) to 81.1 (SD 16.9), or from 32.1 (SD 8.2) to 34.9 (SD 8.32), respectively. After 12 months, a further increase to 84.1 (SD 15.5) or 35.2 (SD 8.6) was found. Fatigue (MFI) decreased; anxiety and depression (HADS) did not change. CG: FACT-G remained constant in the first three months: 75.3 (SD 17.3) at t0, and 76.6 (SD 16.6) at t1. After 12 months, there was a slight increase to 78.9 (SD 18.1). FACIT-Sp scores improved significantly from t0 (31.0 – SD 8.9) to t1 (32.1 – SD 8.9) and declined again after a year (31.6 – SD 9.4). For fatigue, anxiety, and depression, no relevant changes were found. 120 patients of HG and 206 patients of CG met our criteria for matched-pairs selection. Due to large differences between the two patient populations, however, only 11 matched pairs could be formed. This is not sufficient for a comparative study.
In our prospective study, we observed an improvement of quality of life as well as a tendency of fatigue symptoms to decrease in cancer patients under complementary homeopathic treatment. It would take considerably larger samples to find matched pairs suitable for comparison in order to establish a definite causal relation between these effects and homeopathic treatment.
Even the abstract makes several points very clear, and the full text confirms further embarrassing details:
- The patients in this study received homeopathy in addition to standard care (the patient shown in the film only had homeopathy until it was too late, and she subsequently died, aged 33).
- The study compared A+B with B alone (A=homeopathy, B= standard care). It is hardly surprising that the additional attention of A leads to an improvement in quality of life. It is arguably even unethical to conduct a clinical trial to demonstrate such an obvious outcome.
- The authors of this paper caution that it is not possible to conclude that a causal relationship between homeopathy and the outcome exists.
- This is true not just because of the small sample size, but also because of the fact that the two groups had not been allocated randomly and therefore are bound to differ in a whole host of variables that have not or cannot be measured.
- Harald Walach, the senior author of this paper, held a position which was funded by Heel, Baden-Baden, one of Germany’s largest manufacturer of homeopathics.
- The H.W.& J.Hector Foundation, Germany, and the Samueli Institute, provided the funding for this study.
In the film, one of the co-authors of this paper, the oncologist HH Bartsch from Freiburg, states that Dr Wurster’s interpretation of this study is ‘dishonest’.
I am inclined to agree.
The pro arguments essentially are the well-rehearsed points acupuncture-fans like to advance:
- Some guidelines do recommend acupuncture.
- Sham acupuncture is not a valid comparator.
- The largest meta-analysis shows a small effect.
- Acupuncture is not implausible.
- It improves quality of life.
Cummings concludes as follows: In summary, the pragmatic view sees acupuncture as a relatively safe and moderately effective intervention for a wide range of common chronic pain conditions. It has a plausible set of neurophysiological mechanisms supported by basic science.12 For those patients who choose it and who respond well, it considerably improves health related quality of life, and it has much lower long term risk for them than non-steroidal anti-inflammatory drugs. It may be especially useful for chronic musculoskeletal pain and osteoarthritis in elderly patients, who are at particularly high risk from adverse drug reactions.
Our arguments are also not new; essentially, we stress that:
- The effects of acupuncture are too small to be clinically relevant.
- They are probably not even caused by acupuncture, but the result of residual bias.
- Pragmatic trials are of little value in defining efficacy.
- Acupuncture is not free of risks.
- Regular acupuncture treatments are expensive.
- There is no generally accepted, plausible mechanism.
We concluded that after decades of research and hundreds of acupuncture pain trials, including thousands of patients, we still have no clear mechanism of action, insufficient evidence for clinically worthwhile benefit, and possible harms. Therefore, doctors should not recommend acupuncture for pain.
Neither Asbjorn nor I have any conflicts of interests to declare.
Dr Cummings, by contrast, states that he is the salaried medical director of the British Medical Acupuncture Society, which is a membership organisation and charity established to stimulate and promote the use and scientific understanding of acupuncture as part of the practice of medicine for the public benefit. He is an associate editor for Acupuncture in Medicine, published by BMJ. He has a modest private income from lecturing outside the UK, royalties from textbooks, and a partnership teaching veterinary surgeons in Western veterinary acupuncture. He has participated in a NICE guideline development group as an expert adviser discussing acupuncture. He has used Western medical acupuncture in clinical practice following a chance observation as a medical officer in the Royal Air Force in 1989.
My question to you is this: WHICH OF THE TWO POSITION IS THE MORE REASONABLE ONE?
Please, do let us know by posting a comment here, or directly at the BMJ article (better), or both (best).
Clinical trials are a most useful tool, but they can easily be abused. It is not difficult to misuse them in such a way that even the most useless treatment appears to be effective. Sadly, this sort of thing happens all too often in the realm of alternative medicine. Take for instance this recently published trial of homeopathy.
The objective of this study was to investigate the usefulness of classical homeopathy for the prevention of recurrent urinary tract infections (UTI) in patients with spinal cord injury (SCI). Patients were admitted to this trial, if they had chronic SCI and had previously suffered from at least three UTI/year. They were treated either with a standardized prophylaxis alone, or with a standardized prophylaxis in combination with homeopathy. The number of UTIs, general and specific quality of life (QoL), and satisfaction with homeopathic treatment were assessed prospectively over the period of one year. Ten patients were in the control group and 25 patients received adjunctive homeopathic treatment. The median number of self-reported UTI in the homeopathy group decreased significantly, whereas it remained unchanged in the control group. The domain incontinence impact of the KHQ improved significantly, whereas the general QoL did not change. The satisfaction with homeopathic care was high.
The authors concluded that adjunctive homeopathic treatment lead to a significant decrease of UTI in SCI patients. Therefore, classical homeopathy could be considered in SCI patients with recurrent UTI.
Where to begin?
Here are just some of the most obvious flaws of and concerns with this study:
- There is no plausible rationale to even plan such a study.
- The sample size was far too small for allowing generalizable conclusions.
- There was no adequate randomisation and patients were able to chose the homeopathy option.
- The study seems to lack objective outcome measures.
- The study design did not allow to control for non-specific effects; therefore, it seems likely that the observed outcomes are unrelated to the homeopathic treatments but are caused by placebo and other non-specific effects.
- Even if the study had been rigorous, we would need independent replications before we draw such definitive conclusions.
- Two of the authors are homeopaths, and it is in their clinics that the study took place.
- Some of the authors have previously published a very similar paper – except that this ‘case series’ included no control group at all.
- The latter paper seems to have been published more than once.
- Of this paper, one of the authors claimed that ” the usefulness of classical homeopathy as an adjunctive measure for UTI prophylaxis in patients with NLUTD due to SCI has been demonstrated in a case series”. He seems to be unaware of the fact that a case series cannot possible lend itself to demonstrate this.
- I do wonder: did they just add a control group to their case series thus pretending it became a controlled clinical trial?
What strikes me most with such pseudo-research is its abundance and the naivety – or should I call it ignorance? – of the enthusiasts who conduct it. Most of them, I am fairly sure do not mean to do harm; but by Jove they do!
Do chiropractors even know the difference between promotion and research?
Probably a rhetorical question.
Personally, I have seen them doing so much pseudo-research that I doubt they recognise the real thing, even if they fell over it.
Here is a recent example that stands for many, many more such ‘research’ projects (some of which have been discussed on this blog).
But first a few sentences on the background of this new ‘study’.
The UD chiropractic profession is currently on the ‘opioid over-use bandwagon’ hoping that this move might promote their trade. Most chiropractors have always been against using (any type of) pharmaceutical treatment and advise their patients accordingly. D D Palmer, the founder of chiropractic, was adamant that drugs are to be avoided; he stated for instance that Drugs are delusive; they do not adjust anything. And “as the Founder intended, chiropractic has existed as a drug-free healthcare profession for better than 120 years.” To this day, chiropractors are educated and trained to argue against non-drug treatments and regularly claim that chiropractic is a drug-free alternative to traditional medicine.
Considering this background, this new piece of (pseudo) research is baffling, in my view.
The objective of this investigation was to evaluate the association between utilization of chiropractic services and the use of prescription opioid medications. The authors used a retrospective cohort design to analyse health insurance claims data. The data source was the all payer claims database administered by the State of New Hampshire. The authors chose New Hampshire because health claims data were readily available for research, and in 2015, New Hampshire had the second-highest age-adjusted rate of drug overdose deaths in the United States.
The study population comprised New Hampshire residents aged 18-99 years, enrolled in a health plan, and with at least two clinical office visits within 90 days for a primary diagnosis of low-back pain. The authors excluded subjects with a diagnosis of cancer. They measured likelihood of opioid prescription fill among recipients of services delivered by chiropractors compared with a control group of patients not consulting a chiropractor. They also compared the cohorts with regard to rates of prescription fills for opioids and associated charges.
The adjusted likelihood of filling a prescription for an opioid analgesic was 55% lower among chiropractic compared to non-chiropractic patients. Average charges per person for opioid prescriptions were also significantly lower among the former group.
The authors concluded that among New Hampshire adults with office visits for noncancer low-back pain, the likelihood of filling a prescription for an opioid analgesic was significantly lower for recipients of services delivered by doctors of chiropractic compared with nonrecipients. The underlying cause of this correlation remains unknown, indicating the need for further investigation.
The underlying cause remains unknown???
Let me speculate, or even better, let me extrapolate by drawing an analogy:
Employees by a large Hamburger chain set out to study the association between utilization of Hamburger restaurant services and vegetarianism. The authors used a retrospective cohort design. The study population comprised New Hampshire residents aged 18-99 years, who had entered the premises of a Hamburger restaurant within 90 days for a primary purpose of eating. The authors excluded subjects with a diagnosis of cancer. They measured the likelihood of vegetarianism among recipients of services delivered by Hamburger restaurants compared with a control group of individuals not using meat-dispensing facilities. They also compared the cohorts with regard to the money spent in Hamburger restaurants.
The adjusted likelihood of being a vegetarian was 55% lower among the experimental group compared to controls. The average money spent per person in Hamburger restaurants were also significantly lower among the Hamburger group.
The authors concluded that among New Hampshire adults visiting Hamburger restaurants, the likelihood of vegetarianism was significantly lower for consumers frequenting Hamburger restaurants compared with those who failed to frequent such places. The underlying cause of this correlation remains unknown, indicating the need for further investigation.
The question whether spinal manipulative therapy (SMT) has any specific therapeutic effects is still open. This fact must irritate ardent chiropractors, and they therefore try everything to dispel our doubts. One way would be to demonstrate a dose-effect relationship between SMT and the clinical outcome. But, for several reasons, this is not an easy task.
This RCT was aimed at identifying the dose-response relationship between visits for SMT and chronic cervicogenic headache (CGH) outcomes; to evaluate the efficacy of SMT by comparison with a light massage control.
The study included 256 adults with chronic CGH. The primary outcome was days with CGH in the prior 4 weeks evaluated at the 12- and 24-week primary endpoints. Secondary outcomes included CGH days at remaining endpoints, pain intensity, disability, perceived improvement, medication use, and patient satisfaction. Participants were randomized to 4 different dose levels of chiropractic SMT: 0, 6, 12, or 18 sessions. They were treated 3 times per week for 6 weeks and received a focused light-massage control at sessions when SMT was not assigned. Linear dose effects and comparisons to the no-manipulation control group were evaluated at 6, 12, 24, 39, and 52 weeks.
A linear dose-response was observed for all follow-ups, a reduction of approximately 1 CGH day/4 weeks per additional 6 SMT visits (p<.05); a maximal effective dose could not be determined. CGH days/4 weeks were reduced from about 16 to 8 for the highest and most effective dose of 18 SMT visits. Mean differences in CGH days/4 weeks between 18 SMT visits and control were -3.3 (p=.004) and -2.9 (p=.017) at the primary endpoints, and similar in magnitude at the remaining endpoints (p<.05). Differences between other SMT doses and control were smaller in magnitude (p > .05). CGH intensity showed no important improvement nor differed by dose. Other secondary outcomes were generally supportive of the primary.
The authors concluded that there was a linear dose-response relationship between SMT visits and days with CGH. For the highest and most effective dose of 18 SMT visits, CGH days were reduced by half, and about 3 more days per month than for the light-massage control.
This trial would make sense, if the effectiveness of SMT for CGH had been a well-documented fact, and if the study had rigorously controlled for placebo-effects.
But guess what?
Neither of these conditions were met.
A recent review concluded that there are few published randomized controlled trials analyzing the effectiveness of spinal manipulation and/or mobilization for TTH, CeH, and M in the last decade. In addition, the methodological quality of these papers is typically low. Clearly, there is a need for high-quality randomized controlled trials assessing the effectiveness of these interventions in these headache disorders. And this is by no means the only article making such statements; similar reviews arrive at similar conclusions. In turn, this means that the effects observed after SMT are not necessarily specific effects due to SMT but could easily be due to placebo or other non-specific effects. In order to avoid confusion, one would need a credible placebo – one that closely mimics SMT – and make sure that patients were ‘blinded’. But ‘light massage’ clearly does not mimic SMT, and patients obviously were aware of which interventions they received.
So, an alternative – and I think at least as plausible – conclusion of the data provided by this new RCT is this:
Chiropractic SMT is associated with a powerful placebo response which, of course, obeys a dose-effect relationship. Thus these findings are in keeping with the notion that SMT is a placebo.
And why would the researchers – who stress that they have no conflicts of interest – mislead us by making this alternative interpretation of their findings not abundantly clear?
I fear, the reason might be simple: they also seem to mislead us about their conflicts of interest: they are mostly chiropractors with a long track record of publishing promotional papers masquerading as research. What, I ask myself, could be a stronger conflict of interest?
(Pity that a high-impact journal like SPINE did not spot these [not so little] flaws)
Lock 10 bright people into a room and tell them they will not be let out until they come up with the silliest idea in healthcare. It is not unlikely, I think, that they might come up with the concept of visceral osteopathy.
In case you wonder what visceral osteopathy (or visceral manipulation) is, one ‘expert’ explains it neatly: Visceral Osteopathy is an expansion of the general principles of osteopathy which includes a special understanding of the organs, blood vessels and nerves of the body (the viscera). Visceral Osteopathy relieves imbalances and restrictions in the interconnections between the motions of all the organs and structures of the body. Jean-Piere Barral RPT, DO built on the principles of Andrew Taylor Still DO and William Garner Sutherland DO, to create this method of detailed assessment and highly specific manipulation. Those who wish to practice Visceral Osteopathy train intensively through a series of post-graduate studies. The ability to address the specific visceral causes of somatic dysfunction allows the practitioner to address such conditions as gastroesophageal reflux disease (GERD), irritable bowel (IBS), and even infertility caused by mechanical restriction.
But, as I have pointed out many times before, the fact that a treatment is based on erroneous assumptions does not necessarily mean that it does not work. What we need to decide is evidence. And here we are lucky; a recent paper provides just that.
The purpose of this systematic review was to identify and critically appraise the scientific literature concerning the reliability of diagnosis and the clinical efficacy of techniques used in visceral osteopathy.
Only inter-rater reliability studies including at least two raters or the intra-rater reliability studies including at least two assessments by the same rater were included. For efficacy studies, only randomized-controlled-trials (RCT) or crossover studies on unhealthy subjects (any condition, duration and outcome) were included. Risk of bias was determined using a modified version of the quality appraisal tool for studies of diagnostic reliability (QAREL) in reliability studies. For the efficacy studies, the Cochrane risk of bias tool was used to assess their methodological design. Two authors performed data extraction and analysis.
Extensive searches located 8 reliability studies and 6 efficacy trials that could be included in this review. The analysis of reliability studies showed that the diagnostic techniques used in visceral osteopathy are unreliable. Regarding efficacy studies, the least biased study showed no significant difference for the main outcome. The main risks of bias found in the included studies were due to the absence of blinding of the examiners, an unsuitable statistical method or an absence of primary study outcome.
The authors (who by the way declared no conflicts of interest) concluded that the results of the systematic review lead us to conclude that well-conducted and sound evidence on the reliability and the efficacy of techniques in visceral osteopathy is absent.
It is hard not to appreciate the scientific rigor of this review or to agree with the conclusions drawn by the French authors.
But what consequences should we draw from all this?
The authors of this paper state that more and better research is needed. Somehow, I doubt this. Visceral osteopathy is not plausible and the best evidence available to date does not show it works. In my view, this means that we should declare it an obsolete aberration of medical history.
To this, the proponents of visceral osteopathy will probably say that they have tons of experience and have witnessed wonderful cures etc. This I do not doubt; however, the things they saw were not due to the effects of visceral osteopathy, they were due to chance, placebo, regression towards the mean, the natural history of the diseases treated etc., etc. And sometimes, experience is nothing more that the ability to repeat a mistake over and over again.
- If it looks like a placebo,
- if it behaves like a placebo,
- if it tests like a placebo,
IT MOST LIKELY IS A PLACEBO!!!
And what is wrong with a placebo, if it helps patients?
GIVE ME A BREAK!
WE HAVE ALREADY DISCUSSED THIS AD NAUSEAM. JUST READ SOME OF THE PREVIOUS POSTS ON THIS SUBJECT.
Sipjeondaebo-tang is an East Asian herbal supplement containing Angelica root (Angelicae Gigantis Radix), the rhizome of Cnidium officinale Makino (Cnidii Rhizoma), Radix Paeoniae, Rehmannia glutinosa root (Rehmanniae Radix Preparata), Ginseng root (Ginseng Radix Alba), Atractylodes lancea root (Atractylodis Rhizoma Alba), the dried sclerotia of Poria cocos (Poria cocos Sclerotium), Licorice root (Glycyrrhizae Radix), Astragalus root (Astragali Radix), and the dried bark of Cinnamomum verum (Cinnamomi Cortex).
But does this herbal mixture actually work? Korean researchers wanted to find out.
The purpose of their study was to examine the feasibility of Sipjeondaebo-tang (Juzen-taiho-to, Shi-Quan-Da-Bu-Tang) for cancer-related anorexia. A total of 32 participants with cancer anorexia were randomized to either Sipjeondaebo-tang group or placebo group. Participants were given 3 g of Sipjeondaebo-tang or placebo 3 times a day for 4 weeks. The primary outcome was a change in the Anorexia/Cachexia Subscale of Functional Assessment of Anorexia/Cachexia Therapy (FAACT). The secondary outcomes included Visual Analogue Scale (VAS) of anorexia, FAACT scale, and laboratory tests.
The results showed that anorexia and quality of life measured by FAACT and VAS were improved after 4 weeks of Sipjeondaebo-tang treatment. However, there was no significant difference between changes of Sipjeondaebo-tang group and placebo group.
From this, the authors of the study concluded that sipjeondaebo-tang appears to have potential benefit for anorexia management in patients with cancer. Further large-scale studies are needed to ensure the efficacy.
Well, isn’t this just great? Faced with a squarely negative result, one simply ignores it and draws a positive conclusion!
As we all know – and as trialists certainly must know – controlled trials are designed to compare the outcomes of two groups. Changes within one of the groups can be caused by several factors unrelated to the therapy and are therefore largely irrelevant. This means that “no significant difference between changes of Sipjeondaebo-tang group and placebo group” indicates that the herbal mixture had no effect. In turn this means that a conclusion stating that “sipjeondaebo-tang appears to have potential benefit for anorexia” is just fraudulent.
This level of scientific misconduct is remarkable, even for the notoriously poor Evid Based Complement Alternat Med.
I strongly suggest that:
- The journal is de-listed from Medline because similarly misleading nonsense has been coming out of this rag for some time.
- The paper is withdrawn because it can only mislead vulnerable patients.
Cranio-sacral therapy is firstly implausible, and secondly it lacks evidence of effectiveness (see for instance here, here, here and here). Yet, some researchers are nevertheless not deterred to test it in clinical trials. While this fact alone might be seen as embarrassing, the study below is a particular and personal embarrassment to me, in fact, I am shocked by it and write these lines with considerable regret.
Why? Bear with me, I will explain later.
The purpose of this trial was to evaluate the effectiveness of osteopathic manipulative treatment and osteopathy in the cranial field in temporomandibular disorders. Forty female subjects with temporomandibular disorders lasting at least three months were included. At enrollment, subjects were randomly assigned into two groups: (1) osteopathic manipulative treatment group (n=20) and (2) osteopathy in the cranial field [craniosacral therapy for you and me] group (n=20). Examinations were performed at baseline (E0) and at the end of the last treatment (E1), and consisted of subjective pain intensity with the Visual Analog Scale, Helkimo Index and SF-36 Health Survey. Subjects had five treatments, once a week. 36 subjects completed the study.
Patients in both groups showed significant reduction in Visual Analog Scale score (osteopathic manipulative treatment group: p = 0.001; osteopathy in the cranial field group: p< 0.001), Helkimo Index (osteopathic manipulative treatment group: p = 0.02; osteopathy in the cranial field group: p = 0.003) and a significant improvement in the SF-36 Health Survey – subscale “Bodily Pain” (osteopathic manipulative treatment group: p = 0.04; osteopathy in the cranial field group: p = 0.007) after five treatments (E1). All subjects (n = 36) also showed significant improvements in the above named parameters after five treatments (E1): Visual Analog Scale score (p< 0.001), Helkimo Index (p< 0.001), SF-36 Health Survey – subscale “Bodily Pain” (p = 0.001). The differences between the two groups were not statistically significant for any of the three endpoints.
The authors concluded that both therapeutic modalities had similar clinical results. The findings of this pilot trial support the use of osteopathic manipulative treatment and osteopathy in the cranial field as an effective treatment modality in patients with temporomandibular disorders. The positive results in both treatment groups should encourage further research on osteopathic manipulative treatment and osteopathy in the cranial field and support the importance of an interdisciplinary collaboration in patients with temporomandibular disorders. Implications for rehabilitation Temporomandibular disorders are the second most prevalent musculoskeletal condition with a negative impact on physical and psychological factors. There are a variety of options to treat temporomandibular disorders. This pilot study demonstrates the reduction of pain, the improvement of temporomandibular joint dysfunction and the positive impact on quality of life after osteopathic manipulative treatment and osteopathy in the cranial field. Our findings support the use of osteopathic manipulative treatment and osteopathy in the cranial field and should encourage further research on osteopathic manipulative treatment and osteopathy in the cranial field in patients with temporomandibular disorders. Rehabilitation experts should consider osteopathic manipulative treatment and osteopathy in the cranial field as a beneficial treatment option for temporomandibular disorders.
This study has so many flaws that I don’t know where to begin. Here are some of the more obvious ones:
- There is, as already mentioned, no rationale for this study. I can see no reason why craniosacral therapy should work for the condition. Without such a rationale, the study should never even have been conceived.
- Technically, this RCTs an equivalence study comparing one therapy against another. As such it needs to be much larger to generate a meaningful result and it also would require a different statistical approach.
- The authors mislabelled their trial a ‘pilot study’. However, a pilot study “is a preliminary small-scale study that researchers conduct in order to help them decide how best to conduct a large-scale research project. Using a pilot study, a researcher can identify or refine a research question, figure out what methods are best for pursuing it, and estimate how much time and resources will be necessary to complete the larger version, among other things.” It is not normally a study suited for evaluating the effectiveness of a therapy.
- Any trial that compares one therapy of unknown effectiveness to another of unknown effectiveness is a complete and utter nonsense. Equivalent studies can only ever make sense, if one of the two treatments is of proven effectiveness – think of it as a mathematical equation: one equation with two unknowns is unsolvable.
- Controlled studies such as RCTs are for comparing the outcomes of two or more groups, and only between-group differences are meaningful results of such trials.
- The ‘positive results’ which the authors mention in their conclusions are meaningless because they are based on such within-group changes and nobody can know what caused them: the natural history of the condition, regression towards the mean, placebo-effects, or other non-specific effects – take your pick.
- The conclusions are a bonanza of nonsensical platitudes and misleading claims which do not follow from the data.
As regular readers of this blog will doubtlessly have noticed, I have seen plenty of similarly flawed pseudo-research before – so, why does this paper upset me so much? The reason is personal, I am afraid: even though I do not know any of the authors in person, I know their institution more than well. The study comes from the Department of Physical Medicine and Rehabilitation, Medical University of Vienna, Austria. I was head of this department before I left in 1993 to take up the Exeter post. And I had hoped that, even after 25 years, a bit of the spirit, attitude, knowhow, critical thinking and scientific rigor – all of which I tried so hard to implant in my Viennese department at the time – would have survived.
Perhaps I was wrong.