false positive
Chiropractors, like other alternative practitioners, use their own unique diagnostic tools for identifying the health problems of their patients. One such test is the Kemp’s test, a manual test used by most chiropractors to diagnose problems with lumbar facet joints. The chiropractor rotates the torso of the patient, while her pelvis is fixed; if manual counter-rotative resistance on one side of the pelvis by the chiropractor causes lumbar pain for the patient, it is interpreted as a sign of lumbar facet joint dysfunction which, in turn would be treated with spinal manipulation.
All diagnostic tests have to fulfil certain criteria in order to be useful. It is therefore interesting to ask whether the Kemp’s test meets these criteria. This is precisely the question addressed in a recent paper. Its objective was to evaluate the existing literature regarding the accuracy of the Kemp’s test in the diagnosis of facet joint pain compared to a reference standard.
All diagnostic accuracy studies comparing the Kemp’s test with an acceptable reference standard were located and included in the review. Subsequently, all studies were scored for quality and internal validity.
Five articles met the inclusion criteria. Only two studies had a low risk of bias, and three had a low concern regarding applicability. Pooling of data from studies using similar methods revealed that the test’s negative predictive value was the only diagnostic accuracy measure above 50% (56.8%, 59.9%).
The authors concluded that currently, the literature supporting the use of the Kemp’s test is limited and indicates that it has poor diagnostic accuracy. It is debatable whether clinicians should continue to use this test to diagnose facet joint pain.
The problem with chiropractic diagnostic methods is not confined to the Kemp’s test, but extends to most tests employed by chiropractors. Why should this matter?
If diagnostic methods are not reliable, they produce either false-positive or false-negative findings. When a false-negative diagnosis is made, the chiropractor might not treat a condition that needs attention. Much more common in chiropractic routine, I guess, are false-positive diagnoses. This means chiropractors frequently treat conditions which the patient does not have. This, in turn, is not just a waste of money and time but also, if the ensuing treatment is associated with risks, an unnecessary exposure of patients to getting harmed.
The authors of this review, chiropractors from Canada, should be praised for tackling this subject. However, their conclusion that “it is debatable whether clinicians should continue to use this test to diagnose facet joint pain” is in itself highly debatable: the use of nonsensical diagnostic tools can only result in nonsense and should therefore be disallowed.
Most of the underlying assumptions of alternative medicine (AM) lack plausibility. Whenever this is the case, so the argument put forward by an international team of researchers in a recent paper, there are difficulties involved in obtaining a valid statistical significance in clinical studies.
Using a mostly statistical approach, they argue that, since the prior probability of a research hypothesis is directly related to its scientific plausibility, the commonly used frequentist statistics, which do not account for this probability, are unsuitable for studies exploring matters in various degree disconnected from science. Any statistical significance obtained in this field should be considered with great caution and may be better applied to more plausible hypotheses (like placebo effect) than the specific efficacy of the intervention.
The researchers conclude that, since achieving meaningful statistical significance is an essential step in the validation of medical interventions, AM practices, producing only outcomes inherently resistant to statistical validation, appear not to belong to modern evidence-based medicine.
To emphasize their arguments, the researchers make the following additional points:
- It is often forgotten that frequentist statistics, commonly used in clinical trials, provides only indirect evidence in support of the hypothesis examined.
- The p-value inherently tends to exaggerate the support for the hypothesis tested, especially if the scientific plausibility of the hypothesis is low.
- When the rationale for a clinical intervention is disconnected from the basic principles of science, as in case of complementary alternative medicines, any positive result obtained in clinical studies is more reasonably ascribable to hypotheses (generally to placebo effect) other than the hypothesis on trial, which commonly is the specific efficacy of the intervention.
- Since meaningful statistical significance as a rule is an essential step to validation of a medical intervention, complementary alternative medicine cannot be considered evidence-based.
Further explanations can be found in the discussion of the article where the authors argue that the quality of the hypothesis tested should be consistent with sound logic and science and therefore have a reasonable prior probability of being correct. As a rule of thumb, assuming a “neutral” attitude towards the null hypothesis (odds = 1:1), a p-value of 0.01 or, better, 0.001 should suffice to give a satisfactory posterior probability of 0.035 and 0.005 respectively.
In the area of AM, hypotheses often are entirely inconsistent with logic and frequently fly in the face of science. Four examples can demonstrate this instantly and sufficiently, I think:
- Homeopathic remedies which contain not a single ‘active’ molecule are not likely to generate biological effects.
- Healing ‘energy’ of Reiki masters has no basis in science.
- Meridians of acupuncture are pure imagination.
- Chiropractic subluxation have never been shown to exist.
Positive results from clinical trials of implausible forms of AM are thus either due to chance, bias or must be attributed to more credible causes such as the placebo effect. Since the achievement of meaningful statistical significance is an essential step in the validation of medical interventions, unless some authentic scientific support to AM is provided, one has to conclude that AM cannot be considered as evidence-based.
Such arguments are by no means new; they have been voiced over and over again. Essentially, they amount to the old adage: IF YOU CLAIM THAT YOU HAVE A CAT IN YOUR GARDEN, A SIMPLE PICTURE MAY SUFFICE. IF YOU CLAIM THERE IS A UNICORN IN YOUR GARDEN, YOU NEED SOMETHING MORE CONVINCING. An extraordinary claim requires an extraordinary proof! Put into the context of the current discussion about AM, this means that the usual level of clinical evidence is likely to be very misleading as long as it totally neglects the biological plausibility of the prior hypothesis.
Proponents of AM do not like to hear such arguments. They usually insist on what we might call a ‘level playing field’ and fail to see why their assumptions require not only a higher level of evidence but also a reasonable scientific hypothesis. They forget that the playing field is not even to start with; to understand the situation better, they should read this excellent article. Perhaps its elegant statistical approach will convince them – but I would not hold my breath.
Bach Flower Remedies are the brain child of Dr Edward Bach who, as an ex-homeopath, invented his very own highly diluted remedies. Like homeopathic medicines, they are devoid of active molecules and are claimed to work via some non-defined ‘energy’. Consequently, the evidence for these treatments is squarely negative: my systematic review analysed the data of all 7 RCTs of human patients or volunteers that were available in 2010. All but one were placebo-controlled. All placebo-controlled trials failed to demonstrate efficacy. I concluded that the most reliable clinical trials do not show any differences between flower remedies and placebos.
But now, a new investigation has become available. The aim of this study was to evaluate the effect of Bach flower Rescue Remedy on the control of risk factors for cardiovascular disease in rats.
A randomized longitudinal experimental study was conducted on 18 Wistar rats which were randomly divided into three groups of six animals each and orogastrically dosed with either 200μl of water (group A, control), or 100μl of water and 100μl of Bach flower remedy (group B), or 200μl of Bach flower remedy (group C) every 2 days, for 20 days. All animals were fed standard rat chow and water ad libitum.
Urine volume, body weight, feces weight, and food intake were measured every 2 days. On day 20, tests of glycemia, hyperuricemia, triglycerides, high-density lipoprotein (HDL) cholesterol, and total cholesterol were performed, and the anatomy and histopathology of the heart, liver and kidneys were evaluated. Data were analyzed using Tukey’s test at a significance level of 5%.
No significant differences were found in food intake, feces weight, urine volume and uric acid levels between groups. Group C had a significantly lower body weight gain than group A and lower glycemia compared with groups A and B. Groups B and C had significantly higher HDL-cholesterol and lower triglycerides than controls. Animals had mild hepatic steatosis, but no cardiac or renal damage was observed in the three groups.
From these results, the authors conclude that Bach flower Rescue Remedy was effective in controlling glycemia, triglycerides, and HDL-cholesterol and may serve as a strategy for reducing risk factors for cardiovascular disease in rats. This study provides some preliminary “proof of concept” data that Bach Rescue Remedy may exert some biological effects.
If ever there was a bizarre study, it must be this one:
- As far as I know, nobody has ever claimed that Rescue Remedy modified cardiovascular risk factors.
- It seems debatable whether the observed changes are all positive as far as the cardiovascular risk is concerned.
- It seems odd that a remedy that does not contain active molecules is associated with some sort of dose-effect response.
- The modification of cardiovascular risk factors in rats might be of little relevance for humans.
- A strategy for reducing cardiovascular risk factors in rats seems a strange idea.
- Even the authors cannot offer a mechanism of action [other than pure magic].
Does this study tell us anything of value? The authors are keen to point out that it provides a preliminary proof of concept for Rescue Remedy having biological effects. Somehow, I doubt that this conclusion will convince many of my readers.
Linus Carl Pauling (1901 – 1994), the American scientist, peace activist, author, and educator who won two Nobel prizes, was one of the most influential chemists in history and ranks among the most important scientists of the 20th century. Linus Pauling’s work on vitamin C, however, generated considerable controversy. Pauling wrote many papers and a popular book, Cancer and Vitamin C. Vitamin C, we know today, protects cells from oxidative DNA damage and might thereby block carcinogenesis. Pauling popularised the regular intake of vitamin C; eventually he published two studies of end-stage cancer patients; their results apparently showed that vitamin C quadrupled survival times. A re-evaluation, however, found that the vitamin C groups were less sick on entry to the study. Later clinical trials concluded that there was no benefit to high-dose vitamin C. Since then, the established opinion is that the best evidence does not support a role for high dose vitamin C in the treatment of cancer. Despite all this, high dose IV vitamin C is in unexpectedly wide use by CAM practitioners.
Yesterday, new evidence has been published in the highly respected journal ‘Nature’; does it vindicate Pauling and his followers?
Chinese oncologists conducted a meta-analysis to assess the association between vitamin C intake and the risk to acquire lung cancer. Pertinent studies were identified by a searches of several electronic databases through December of 2013. Random-effect model was used to combine the data for analysis. Publication bias was estimated using Begg’s funnel plot and Egger’s regression asymmetry test.
Eighteen articles reporting 21 studies involving 8938 lung cancer cases were included in this meta-analysis. Pooled results suggested that highest vitamin C intake level versus lowest level was significantly associated with the risk of lung cancer. The effect was largest in investigations from the United States and in prospective studies. A linear dose-response relationship was found, with the risk of lung cancer decreasing by 7% for every 100 mg/day increase in the intake of vitamin C . No publication bias was found.
The authors conclude that their analysis suggested that the higher intake of vitamin C might have a protective effect against lung cancer, especially in the United States, although this conclusion needs to be confirmed.
Does this finding vindicate Pauling’s theory? Not really.
Even though the above-quoted conclusions seem to suggest a causal link, we are, in fact, far from having established one. The meta-analysis pooled mainly epidemiological data from various studies. Such investigations are doubtlessly valuable but they are fraught with uncertainties and cannot prove causality. For instance, there could be dozens of factors that have confounded these data in such a way that they produce a misleading result. The simplest explanation of the meta-analytic results might be that people who have a very high vitamin C intake tend to have generally healthier life-styles than those who take less vitamin C. When conducting a meta-analysis, one does, of course, try to account for such factors; but in many cases the necessary information to do that is not available, and therefore uncertainty persists.
In other words, the authors were certainly correct when stating that their findings needed to be confirmed. Pauling’s theory cannot be vindicated by such reports – in fact, the authors do not even mention Pauling with one word.
Dodgy science abounds in alternative medicine; this is perhaps particularly true for homeopathy. A brand-new trial seems to confirm this view.
The aim of this study was to test the hypothesis that homeopathy (H) enhances the effects of scaling and root planing (SRP) in patients with chronic periodontitis (CP).
The researchers, dentists from Brazil, randomised 50 patients with CP to one of two treatment groups: SRP (C-G) or SRP + H (H-G). Assessments were made at baseline and after 3 and 12 months of treatment. The local and systemic responses to the treatments were evaluated after one year of follow-up. The results showed that both groups displayed significant improvements, however, the H-G group performed significantly better than C-G group.
The authors concluded that homeopathic medicines, as an adjunctive to SRP, can provide significant local and systemic improvements for CP patients.
Really? I am afraid, I disagree!
Homeopathic medicines might have nothing whatsoever to do with this result. Much more likely is the possibility that the findings are caused by other factors such as:
- placebo-effects,
- patients’ expectations,
- improved compliance with other health-related measures,
- the researchers’ expectations,
- the extra attention given to the patients in the H-G group,
- disappointment of the C-G patients for not receiving the additional care,
- a mixture of all or some of the above.
I should stress that it would not have been difficult to plan the study in such a way that these factors were eliminated as sources of bias or confounding. But this study was conducted according to the A+B versus B design which we have discussed repeatedly on this blog. In such trials, A is the experimental treatment (homeopathy) and B is the standard care (scaling and root planning). Unless A is an overtly harmful therapy, it is simply not conceivable that A+B does not generate better results than B alone. The simplest way to comprehend this argument is to imagine A and B are two different amounts of money: it is impossible that A+B is not more that B!
It is unclear to me what relevant research question such a study design actually does answer (if anyone knows, please tell me). It seems obvious, however, that it cannot test the hypothesis that homeopathy (H) enhances the effects of scaling and root planing (SRP). This does not necessarily mean that the design is necessarily useless. But at the very minimum, one would need an adequate research question (one that matches this design) and adequate conclusions based on the findings.
The fact that the conclusions drawn from a dodgy trial are inadequate and misleading could be seen as merely a mild irritation. The facts that, in homeopathy, such poor science and misleading conclusions emerge all too regularly, and that journals continue to publish such rubbish are not just mildly irritating; they are annoying and worrying – annoying because such pseudo-science constitutes an unethical waste of scarce resources; worrying because it almost inevitably leads to wrong decisions in health care.
Readers of this blog will know that few alternative treatments are more controversial and less plausible than homeopathy. Therefore they might be interested to read about the latest attempt of homeopathy-enthusiasts to convince the public that, despite all the clinical evidence to the contrary, homeopathy does work.
The new article was published in German by Swiss urologist and is a case-report describing a patient suffering from paralytic ileus. This condition is a typical complication of ileocystoplasty of the bladder, the operation the patient had undergone. The patient had also been suffering from a spinal cord injury which, due to a pre-existing neurogenic bowel dysfunction, increases the risk of paralytic ileus.
The paraplegic patient developed a massive paralytic ileus after ileocystoplasty and surgical revision. Conventional stimulation of bowel function was unsuccessful. But after adjunctive homeopathic treatment normalization of bowel function was achieved.
The authors conclude that adjunctive homeopathic therapy is a promising treatment option in patients with complex bowel dysfunction after abdominal surgery who do not adequately respond to conventional treatment.
YES, you did read correctly: homeopathic therapy is a promising treatment…
In case anyone doubts that this is more than a trifle too optimistic, let me suggest three much more plausible reasons why the patient’s bowel function finally normalised:
- It could have been a spontaneous recovery (in most cases, even severe ones, this is what happens).
- It could have been all the conventional treatments aimed at stimulating bowel function.
- It could have been a mixture of the two.
The article made me curious, and I checked whether the authors had previously published other material on homeopathy. Thus I found two further articles in a very similar vein:
We present the clinical course of a patient with an epididymal abscess caused by multiresistant bacteria. As the patient declined surgical intervention, a conservative approach was induced with intravenous antibiotic treatment. As the clinical findings did not ameliorate, adjunctive homeopathic treatment was used. Under combined treatment, laboratory parameters returned to normal, and the epididymal abscess was rapidly shrinking. After 1 week, merely a subcutaneous liquid structure was detected. Fine-needle aspiration revealed sterile purulent liquid, which was confirmed by microbiological testing when the subcutaneous abscess was drained. Postoperative course was uneventful.
As the risk for recurrent epididymitis is high in persons with spinal cord injury, an organ-preserving approach is justified even in severe cases. Homeopathic treatment was a valuable adjunctive treatment in the above-mentioned case. Therefore, prospective studies are needed to further elucidate the future opportunities and limitations of classical homeopathy in the treatment of urinary tract infections.
Recurrent urinary tract infections (UTI) in patients with spinal cord injury are a frequent clinical problem. Often, preventive measures are not successful. We present the case reports of five patients with recurrent UTI who received additional homeopathic treatment. Of these patients, three remained free of UTI, whereas UTI frequency was reduced in two patients. Our initial experience with homeopathic prevention of UTI is encouraging. For an evidence-based evaluation of this concept, prospective studies are required.
It seems clear that all of the three more plausible explanations for the patients’ recovery listed above also apply to these two cases.
One might not be far off speculating that J Pannek, the first author of all these three articles, is a fan of homeopathy (this suspicion is confirmed by a link between him and the HOMEOPATHY RESEARCH INSTITUE: Prof Jürgen Pannek on the use of homeopathy for prophylaxis of UTI’s in patients with neurogenic bladder dysfunction). If that is so, I wonder why he does not conduct a controlled trial, rather than publishing case-report after case-report of apparently successful homeopathic treatments. Does he perhaps fear that his effects might dissolve into thin air under controlled conditions?
Case-reports of this nature can, of course, be interesting and some might even deserve to be published. But it would be imperative to draw the correct conclusions. Looking at the three articles above, I get the impression that, as time goes by, the conclusions of Prof Pannek et al (no, I know nobody from this group of authors personally) are growing more and more firm on less and less safe ground.
In my view, responsible authors should have concluded much more cautiously and reasonably. In the case of the paralytic ileus, for instance, they should not have gone further than stating something like this: adjunctive homeopathic therapy might turn out to be a promising treatment option for such patients. Despite the implausibility of homeopathy, this case-report might deserve to be followed up with a controlled clinical trial. Without such evidence, firm conclusions are clearly not possible.
Blinding patients in clinical trials is a key methodological procedure for minimizing bias and thus making sure that the results are reliable. In alternative medicine, blinding is not always straight forward, and many studies are therefore not patient-blinded. We all know that this can introduce bias into a trial, but how large is its effect on study outcomes?
This was the research question addressed by a recent systematic review of randomized clinical trials with one sub-study (i.e. experimental vs control) involving blinded patients and another, otherwise identical, sub-study involving non-blinded patients. Within each trial, the researchers compared the difference in effect sizes (i.e. standardized mean differences) between the two sub-studies. A difference <0 indicates that non-blinded patients generated a more optimistic effect estimate. The researchers then pooled the differences with random-effects inverse variance meta-analysis, and explored reasons for heterogeneity.
The main analysis included 12 trials with a total of 3869 patients. Ten of these RCTs were studies of acupuncture. The average difference in effect size for patient-reported outcomes was -0.56 (95% confidence interval -0.71 to -0.41), (I(2 )= 60%, P = 0.004), indicating that non-blinded patients exaggerated the effect size by an average of 0.56 standard deviation, but with considerable variation. Two of the 12 trials also used observer-reported outcomes, showing no indication of exaggerated effects due lack of patient blinding.
There was an even larger effect size difference in the 10 acupuncture trials [-0.63 (-0.77 to -0.49)], than in the two non-acupuncture trials [-0.17 (-0.41 to 0.07)]. Lack of patient blinding was also associated with increased attrition rates and the use of co-interventions: ratio of control group attrition risk 1.79 (1.18 to 2.70), and ratio of control group co-intervention risk 1.55 (0.99 to 2.43).
The authors conclude that this study provides empirical evidence of pronounced bias due to lack of patient blinding in complementary/alternative randomized clinical trials with patient-reported outcomes.
This is a timely, rigorous and important analysis. In alternative medicine, we currently see a proliferation of trials that are not patient-blinded. We always suspected that they are at a high risk of generating false-positive results – now we know that this is, in fact, the case.
What should we do with this insight? In my view, the following steps would be wise:
- Take the findings from the existing trials that are devoid of patient-blinding with more than just a pinch of salt.
- Discourage the funding of future studies that fail to include patient-blinding.
- If patient-blinding is truly and demonstrably impossible – which is not often the case – make sure that the trialists at least include blinding of the assessors of the primary outcome measures.
When someone has completed a scientific project, it is customary to publish it [‘unpublished science is no science’, someone once told me many years ago]. To do so, he needs to write it up and submit it to a scientific journal. The editor of this journal will then submit it to a process called ‘peer review’.
What does ‘peer review’ entail? Well, it means that 2-3 experts are asked to critically assess the paper in question, make suggestions as to how it can be improved and submit a recommendation as to whether or not the article deserves to be published.
Peer review has many pitfalls but, so far, nobody has come up with a solution that is convincingly better. Many scientists are under pressure to publish [‘publish or perish’], and therefore some people resort to cheating. A most spectacular case of fraudulent peer review has been reported recently in this press release:
SAGE statement on Journal of Vibration and Control
London, UK (08 July 2014) – SAGE announces the retraction of 60 articles implicated in a peer review and citation ring at the Journal of Vibration and Control (JVC). The full extent of the peer review ring has been uncovered following a 14 month SAGE-led investigation, and centres on the strongly suspected misconduct of Peter Chen, formerly of National Pingtung University of Education, Taiwan (NPUE) and possibly other authors at this institution.
In 2013 the then Editor-in-Chief of JVC, Professor Ali H. Nayfeh,and SAGE became aware of a potential peer review ring involving assumed and fabricated identities used to manipulate the online submission system SAGE Track powered by ScholarOne Manuscripts™. Immediate action was taken to prevent JVC from being exploited further, and a complex investigation throughout 2013 and 2014 was undertaken with the full cooperation of Professor Nayfeh and subsequently NPUE.
In total 60 articles have been retracted from JVC after evidence led to at least one author or reviewer being implicated in the peer review ring. Now that the investigation is complete, and the authors have been notified of the findings, we are in a position to make this statement.
While investigating the JVC papers submitted and reviewed by Peter Chen, it was discovered that the author had created various aliases on SAGE Track, providing different email addresses to set up more than one account. Consequently, SAGE scrutinised further the co-authors of and reviewers selected for Peter Chen’s papers, these names appeared to form part of a peer review ring. The investigation also revealed that on at least one occasion, the author Peter Chen reviewed his own paper under one of the aliases he had created.
Unbelievable? Perhaps, but sadly it is true; some scientists seem to be criminally ingenious when it comes to getting their dodgy articles into peer-reviewed journals.
And what does this have to do with ALTERNATIVE MEDICINE, you may well ask. The Journal of Vibration and Control is not even medical and certainly would never consider publishing articles on alternative medicine. Such papers go to one of the many [I estimate more that 1000] journals that cover either alternative medicine in general or any of the modalities that fall under this wide umbrella. Most of these journals, of course, pride themselves with being peer-reviewed – and, at least nominally, that is correct.
I have been on the editorial board of most of the more important journals in alternative medicine, and I cannot help thinking that their peer review process is not all that dissimilar from the fraudulent scheme set up by Peter Chen and disclosed above. What happens in alternative medicine is roughly as follows:
- a researcher submits a paper for publication,
- the editor sends it out for peer review,
- the peer reviewers are either those suggested by the original author or members of the editorial board of the journal,
- in either case, the reviewers are more than likely to be uncritical and recommend publication,
- in the end, peer review turns out to be a farcical window dressing exercise with no consequence,
- thus even very poor research and pseudo-research are being published abundantly.
The editorial boards of journals of alternative medicine tend to be devoid of experts who are critical about the subject at hand. If you think that I am exaggerating, have a look at the editorial board members of ‘HOMEOPATHY’ (or any other journal of alternative medicine) and tell me who might qualify as a critic of homeopathy. When the editor, Peter Fisher, recently fired me from his board because he felt I had tarnished the image of homeopathy, this panel lost the only person who understood the subject matter and, at the same time, was critical about it (the fact that the website still lists me as an editorial board member is merely a reflection of how slow things are in the world of homeopathy: Fisher fired me more than a year ago).
The point I am trying to make is simple: peer review is never a perfect method but when it is set up to be deliberately uncritical, it cannot possibly fulfil its function to prevent the publication of dodgy research. In this case, the quality of the science will be inadequate and generate false-positive messages that mislead the public.
Reiki is a Japanese technique which, according to a proponent, … is administered by “laying on hands” and is based on the idea that an unseen “life force energy” flows through us and is what causes us to be alive. If one’s “life force energy” is low, then we are more likely to get sick or feel stress, and if it is high, we are more capable of being happy and healthy…
A treatment feels like a wonderful glowing radiance that flows through and around you. Reiki treats the whole person including body, emotions, mind and spirit creating many beneficial effects that include relaxation and feelings of peace, security and wellbeing. Many have reported miraculous results.
Reiki is a simple, natural and safe method of spiritual healing and self-improvement that everyone can use. It has been effective in helping virtually every known illness and malady and always creates a beneficial effect. It also works in conjunction with all other medical or therapeutic techniques to relieve side effects and promote recovery [my emphasis].
Many websites give much more specific information about the health effects of Reiki:
Some Of The Reiki Healing Health Benefits
- Creates deep relaxation and aids the body to release stress and tension,
- It accelerates the body’s self-healing abilities,
- Aids better sleep,
- Reduces blood pressure
- Can help with acute (injuries) and chronic problems (asthma, eczema, headaches, etc.) and aides the breaking of addictions,
- Helps relieve pain,
- Removes energy blockages, adjusts the energy flow of the endocrine system bringing the body into balance and harmony,
- Assists the body in cleaning itself from toxins,
- Reduces some of the side effects of drugs and helps the body to recover from drug therapy after surgery and chemotherapy,
- Supports the immune system,
- Increases vitality and postpones the aging process,
- Raises the vibrational frequency of the body,
- Helps spiritual growth and emotional clearing.
With such remarkable claims being made, I had to look into this extraordinary treatment.
In 2008, I had a co-worker in my team who was (still is, I think) a Reiki healer. He also happened to be a decent scientist, and we thus decided to conduct a systematic review summarising the evidence for the effectiveness of Reiki. We searched the literature using 23 databases from their respective inceptions through to November 2007 (search again 23 January 2008) without language restrictions. Methodological quality was assessed using the Jadad score. The searches identified 205 potentially relevant studies. Nine randomised clinical trials (RCTs) met our inclusion criteria. Two RCTs suggested beneficial effects of Reiki compared with sham control on depression, while one RCT did not report intergroup differences. For pain and anxiety, one RCT showed intergroup differences compared with sham control. For stress and hopelessness, a further RCT reported effects of Reiki and distant Reiki compared with distant sham control. For functional recovery after ischaemic stroke there were no intergroup differences compared with sham. There was also no difference for anxiety between groups of pregnant women undergoing amniocentesis. For diabetic neuropathy there were no effects of reiki on pain. A further RCT failed to show the effects of Reiki for anxiety and depression in women undergoing breast biopsy compared with conventional care.
Overall, the trial data for any one condition were scarce and independent replications were not available for any condition. Most trials suffered from methodological flaws such as small sample size, inadequate study design and poor reporting. We therefore concluded that the evidence is insufficient to suggest that Reiki is an effective treatment for any condition. Therefore the value of Reiki remains unproven.
But this was in 2008! In the meantime, the evidence might have changed. Here are two recent publications which, I think, are worth having a look at:
The first article is a case-report of a nine-year-old female patient with a history of perinatal stroke, seizures, and type-I diabetes was treated for six weeks with Reiki. At the end of this treatment period, there was a decrease in stress in both the child and the mother, as measured by a modified Perceived Stress Scale and a Perceived Stress Scale, respectively. No change was noted in the child’s overall sense of well-being, as measured by a global questionnaire. However, there was a positive change in sleep patterns on 33.3% of the nights as reported on a sleep log kept by the mother. The child and the Reiki Master (a Reiki practitioner who has completed all three levels of Reiki certification training, trains and certifies individuals in the practice of Reiki, and provides Reiki to individuals) experienced warmth and tingling sensations on the same area of the child during the Reiki 7 minutes of each session. There were no reports of seizures during the study period.
The author concluded that Reiki is a useful adjunct for children with increased stress levels and sleep disturbances secondary to their medical condition. Further research is warranted to evaluate the use of Reiki in children, particularly with a large sample size, and to evaluate the long-term use of Reiki and its effects on adequate sleep.
In my view, this article is relevant because it typifies the type of research that is being done in this area and the conclusions that are being drawn from it. It should be clear to anyone who has the slightest ability of critical thinking that a case report of this nature tells us as good as nothing about the effectiveness of a therapy. Considering that Reiki is just about the least plausible intervention anyone can think of, the child’s condition in all likelihood improved not because of the Reiki healing but because of a myriad of unrelated factors; just think of placebo-effects, regression towards the mean, natural history of the condition, concomitant treatments, etc.
The plausibility of energy/biofield/spiritual healing such as Reiki is also the focus of the second remarkable article that was just published. It reports a systematic review of studies designed to examine whether bio-field therapists undergo physiological changes as they enter the healing state (remember: the Reiki healer in the above study experienced ‘warmth and tingling sensations’ during therapies). If reproducible changes could be identified, the authors argue, they might serve as markers to reveal events that correlate with the healing process.
Databases were searched for controlled or non-controlled studies of bio-field therapies in which physiological measurements were made on practitioners in a healing state. Design and reporting criteria, developed in part to reflect the pilot nature of the included studies, were applied using a yes (1.0), partial (0.5), or no (0) scoring system.
Of 67 identified studies, the inclusion criteria were met by 22, 10 of which involved human patients. Overall, the studies were of moderate to poor quality and many omitted information about the training and experience of the healer. The most frequently measured biomarkers were electroencephalography (EEG) and heart rate variability (HRV). EEG changes were inconsistent and not specific to bio-field therapies. HRV results suggest an aroused physiology for Reconnective Healing, Bruyere healing, and Hawaiian healing, but no changes were detected for Reiki or Therapeutic Touch.
The authors of this paper concluded that despite a decades-long research interest in identifying healing-related biomarkers in bio-field healers, little robust evidence of unique physiological changes has emerged to define the healers׳ state.
Now, let me guess why this is so. One does not need to be a rocket scientist to come up with the suggestion that no robust evidence for Reiki and all the other nonsensical forms of healing can be found for one disarmingly simple reason: NO SUCH EFFECTS EXIST.
There is some (albeit not compelling) evidence to suggest that chiropractic spinal manipulation might be effective for treating non-specific back pain. But what about specific back pain, such as the one caused by a herniated disc? Some experts believe that, in patients suffering from such a condition, manipulations are contra-indicated (because the latter can cause the former), while others think that manipulation might be an effective treatment option (although the evidence is far from compelling). Who is correct? The issue can only be resolved with data from well-designed clinical investigations. A new trial might therefore enlighten us.
The stated purposes of this study were:
- to evaluate patients with low-back pain (LBP) and leg pain due to magnetic resonance imaging-confirmed disc herniation treated with high-velocity, low-amplitude spinal manipulation in terms of their short-, medium-, and long-term outcomes of self-reported global impression of change and pain levels
- to determine if outcomes differ between acute and chronic patients using.
The researchers conducted a ‘prospective cohort outcomes study‘ with 148 patients with LBP, leg pain, and physical examination abnormalities with concordant lumbar disc herniations. Baseline numerical rating scale (NRS) data for LBP, leg pain, and the Oswestry questionnaire were obtained. The specific lumbar spinal manipulation was dependent upon whether the disc herniation was intraforaminal or paramedian as seen on the magnetic resonance images and was performed by a chiropractor. Outcomes included the patient’s global impression of change scale for overall improvement, the NRS for LBP, leg pain, and the Oswestry questionnaire at 2 weeks, 1, 3, and 6 months, and 1 year. The proportion of patients reporting “improvement” on the patient’s global impression of change scale was calculated for all patients and for acute vs chronic patients. Pre-treatment and post-treatment NRS scores were compared using the paired t test. Baseline and follow-up Oswestry scores were compared using the Wilcoxon test. Numerical rating scale and Oswestry scores for acute vs chronic patients were compared using the unpaired t test for NRS scores and the Mann-Whitney U test for Oswestry scores.
Significant improvements for all outcomes at all time points were reported. At 3 months, 91% of patients were “improved”, and 88% were “improved” after 1 year. Acute patients improved faster by 3 months than did chronic patients. 81.8% of chronic patients 89.2% felt “improved” at 1 year. No adverse events were reported.
The researchers concluded that a large percentage of acute and importantly chronic lumbar disc herniation patients treated with chiropractic spinal manipulation reported clinically relevant improvement.
Does this new study meaningfully contribute to our knowledge about the effectiveness of chiropractic manipulation for back pain caused by herniated discs? The short answer to this question is NO.
A longer answer might be that the report does tell us something relevant about the quality of this research project. We know from countless studies that ~50% of patients experience adverse effects after spinal manipulations by a chiropractor. This means that any report claiming that NO ADVERSE EFFECTS WERE REPORTED is puzzling to a degree that we have to seriously question its quality or even honesty. In this context, it is relevant to mention that a recent review of the evidence concluded that a cause-effect relationship exists between the manipulative treatment and the development of disc herniation.
The positive outcomes reported in this new study could, of course, be due to a range of factors which are unrelated to the manipulations administered by the chiropractors:
- placebo-effects
- natural history of disc herniation
- regression towards the mean
- other treatments employed by the patients
- social desirability
To be able to say with any degree of certainty that the manipulations had anything to do with the observed positive outcomes would require an entirely different study-design. Should we assume that this is not known in the world of chiropractic? Or should we consider that chiropractors shy away from rigorous research because they fear its results?
The term prospective cohort outcomes study, seems to be a chiropractic invention (cohort studies are by definition prospective, and observational studies are usually prospective). It seems that, behind this long and impressive word, one can easily hide the fact that this study design fails to make the slightest attempt of controlling for non-specific effects; the term sounds scientific – but when we analyse what it means, we discover that this methodology is little more than a self-serving consumer survey. Most scientists would call such an investigation quite simply an OBSERVATIONAL STUDY.
I think it is time that chiropractors start doing proper research which actually does answer some of the many open questions regarding spinal manipulation.
