A new RCT of Reiki healing has been published by US authors from the following institutions: Union Institute & University, Psychology Program, Brattleboro, VT, Coyote Institute, Augusta and Bangor, ME, Eastern Maine Medical Center and Acadia Hospital, Bangor, ME, University of New England College of Osteopathic Medicine, Biddeford, ME, Coyote Institute, Orono, ME. The purpose of this study was to determine if 30 minutes of healing touch could reduce burnout in community mental health clinicians.
The authors utilized a crossover design to explore the efficacy of Reiki versus sham Reiki, a pseudo treatment performed by volunteers who had no experience with Reiki and pretended to be healers vis-à-vis the patients. This sham control intervention was designed to mimic true Reiki.
Subjects were randomized to whether they started with Reiki or sham. The Maslach Burnout Inventory-Human Services Survey (MBI-HSS) and the Measure Your Medical Outcome Profile Version 2 (MYMOP-2) were used as outcome measures. Multilevel modeling was used to represent the relations among variables.
The results showed that real Reiki was significantly better than sham Reiki in reducing burnout among community mental health clinicians. Reiki was significant in reducing depersonalization, but only among single people. Reiki reduced the primary symptom on the MYMOP also only among single people.
The authors concluded that the effects of Reiki were differentiated from sham Reiki. Reiki could be helpful in community mental health settings for the mental health of the practitioners.
My team has published on Reiki (see here and here, for instance), and on this blog I have repeatedly been expressed my doubts that Reiki is more than an elaborate placebo (see here and here, for instance). Do these new results mean that I need to eat my words and henceforth praise the wonders of Reiki? No, I don’t think so!
Having conducted studies on ‘energy healing’ myself, I know only too well of the many pitfalls and possibilities of generating false-positive findings with such research. This new study has many flaws, but we need not look far to find the reason for the surprising and implausible finding. Here is my explanation why this study suggests one placebo to be superior to another placebo.
The researchers had to recruit 16 Reiki healers and several non-Reiki volunteers to perform the interventions on the small group of patients. It goes without saying that the Reiki healers were highly motivated to demonstrate the value of their therapy. This means they (unintentionally?) used verbal and non-verbal communication to maximise the placebo effect of their treatment. The sham healers, of course, lacked such motivation. In my view, this seemingly trivial difference alone is capable of producing the false-positive result above.
There are, of course, ways of minimising the danger of such confounding. In our own study of ‘energy healing’ with sham healers as controls, for instance, we instructed both the healers and the sham healers to abstain from all communication with their patients, we filmed each session to make sure, and we asked each patient to guess which treatment they had received. None of these safeguards were incorporated in the present study – I wonder why!
The press officers of journals like to send out press-releases of articles which are deemed to be particularly good and important. Sadly, it is not often that articles on alternative medicine fulfil these criteria. I was therefore excited to receive this press-release which seemed encouraging, to say the least:
Medical evidence supports the potential for acupuncture to be significantly more effective in the treatment of dermatologic conditions such as dermatitis, pruritus, and urticaria than alternative treatment options, “placebo acupuncture,” or no treatment, according to a review of the medical literature published in The Journal of Alternative and Complementary Medicine, a peer-reviewed publication from Mary Ann Liebert, Inc., publishers…
The abstract was equally promising:
Objectives: Acupuncture is a form of Traditional Chinese Medicine that has been used to treat a broad range of medical conditions, including dermatologic disorders. This systematic review aims to synthesize the evidence on the use of acupuncture as a primary treatment modality for dermatologic conditions.
Methods: A systematic search of MEDLINE, EMBASE, and the Cochrane Central Register was performed. Studies were limited to clinical trials, controlled studies, case reports, comparative studies, and systematic reviews published in the English language. Studies involving moxibustion, electroacupuncture, or blood-letting were excluded.
Results: Twenty-four studies met inclusion criteria. Among these, 16 were randomized controlled trials, 6 were prospective observational studies, and 2 were case reports. Acupuncture was used to treat atopic dermatitis, urticaria, pruritus, acne, chloasma, neurodermatitis, dermatitis herpetiformis, hyperhidrosis, human papillomavirus wart, breast inflammation, and facial elasticity. In 17 of 24 studies, acupuncture showed statistically significant improvements in outcome measurements compared with placebo acupuncture, alternative treatment options, and no intervention.
Conclusions: Acupuncture improves outcome measures in the treatment of dermatitis, chloasma, pruritus, urticaria, hyperhidrosis, and facial elasticity. Future studies should ideally be double-blinded and standardize the control intervention.
One has to read the actual full text article to understand that the evidence presented here is dodgy to the extreme. In fact, one has to go into the tedious details of the methods section to find the reasons why: All searches were limited to clinical trials, controlled studies, case reports, comparative studies, and systematic reviews published in the English language.
There are many more weaknesses of this review, but the inclusion of uncontrolled studies and even anecdotes is, in my view, a virtual death sentence to its credibility. It means that no general conclusions about the effectiveness of acupuncture, such as the authors have decided to make, are possible.
Such overt exaggerations are sadly no rarities in the realm of alternative medicine. I think, this begs a number of serious questions:
- Does this cross the line between flawed research and scientific misconduct?
- Why did the reviewers not pick up these flaws?
- Why did the editor pass this article for publication?
- How can the publisher tolerate such dubious behaviour?
- Should this journal (which I have commented on before here and which is one with the highest impact factor of all the alt med journals) be de-listed from Medline?
I don’t think that we will get answers from the people responsible for this disgrace, but I would like to learn my readers’ opinions.
A paper entitled ‘Real world research: a complementary method to establish the effectiveness of acupuncture’ caught my attention recently. I find it quite remarkable and think it might stimulate some discussion on this blog. Here is its abstract:
Acupuncture has been widely used in the management of a variety of diseases for thousands of years, and many relevant randomized controlled trials have been published. In recent years, many randomized controlled trials have provided controversial or less-than-convincing evidence that supports the efficacy of acupuncture. The clinical effectiveness of acupuncture in Western countries remains controversial.
Acupuncture is a complex intervention involving needling components, specific non-needling components, and generic components. Common problems that have contributed to the equivocal findings in acupuncture randomized controlled trials were imperfections regarding acupuncture treatment and inappropriate placebo/sham controls. In addition, some inherent limitations were also present in the design and implementation of current acupuncture randomized controlled trials such as weak external validity. The current designs of randomized controlled trials of acupuncture need to be further developed. In contrast to examining efficacy and adverse reaction in a “sterilized” environment in a narrowly defined population, real world research assesses the effectiveness and safety of an intervention in a much wider population in real world practice. For this reason, real world research might be a feasible and meaningful method for acupuncture assessment. Randomized controlled trials are important in verifying the efficacy of acupuncture treatment, but the authors believe that real world research, if designed and conducted appropriately, can complement randomized controlled trials to establish the effectiveness of acupuncture. Furthermore, the integrative model that can incorporate randomized controlled trial and real world research which can complement each other and potentially provide more objective and persuasive evidence.
In the article itself, the authors list seven criteria for what they consider good research into acupuncture:
- Acupuncture should be regarded as complex and individualized treatment;
- The study aim (whether to assess the efficacy of acupuncture needling or the effectiveness of acupuncture treatment) should be clearly defined and differentiated;
- Pattern identification should be clearly specified, and non-needling components should also be considered;
- The treatment protocol should have some degree of flexibility to allow for individualization;
- The placebo or sham acupuncture should be appropriate: knowing “what to avoid” and “what to mimic” in placebos/shams;
- In addition to “hard evidence”, one should consider patient-reported outcomes, economic evaluations, patient preferences and the effect of expectancy;
- The use of qualitative research (e.g., interview) to explore some missing areas (e.g., experience of practitioners and patient-practitioner relationship) in acupuncture research.
Furthermore, the authors list the advantages of their RWR-concept:
- In RWR, interventions are tailored to the patients’ specific conditions, in contrast to standardized treatment. As a result, conclusions based on RWR consider all aspects of acupuncture that affect the effectiveness.
- At an operational level, patients’ choice of the treatment(s) decreases the difficulties in recruiting and retaining patients during the data collection period.
- The study sample in RWR is much more representative of the real world situation (similar to the section of the population that receives the treatment). The study, therefore, has higher external validity.
- RWR tends to have a larger sample size and longer follow-up period than RCT, and thus is more appropriate for assessing the safety of acupuncture.
The authors make much of their notion that acupuncture is a COMPLEX INTERVENTION; specifically they claim the following: Acupuncture treatment includes three aspects: needling, specific non-needling components drove by acupuncture theory, and generic components not unique to acupuncture treatment. In addition, acupuncture treatment should be performed on the basis of the patient condition and traditional Chinese medicine (TCM) theory.
There is so much BS here that it is hard to decide where to begin refuting. As the assumption of acupuncture or other alternative therapies being COMPLEX INTERVENTIONS (and therefore exempt from rigorous tests) is highly prevalent in this field, let me try to just briefly tackle this one.
The last time I saw a patient and prescribed a drug treatment I did all of the following:
- I greeted her, asked her to sit down and tried to make her feel relaxed.
- I first had a quick chat about something trivial.
- I then asked why she had come to see me.
- I started to take notes.
- I inquired about the exact nature and the history of her problem.
- I then asked her about her general medical history, family history and her life-style.
- I also asked about any psychological problems that might relate to her symptoms.
- I then conducted a physical examination.
- Subsequently we discussed what her diagnosis might be.
- I told her what my working diagnosis was.
- I ordered a few tests to either confirm or refute it and explained them to her.
- We decided that she should come back and see me in a few days when her tests had come back.
- In order to ease her symptoms in the meanwhile, I gave her a prescription for a drug.
- We discussed this treatment, how and when she should take it, adverse effects etc.
- We also discussed other therapeutic options, in case the prescribed treatment was in any way unsatisfactory.
- I reassured her by telling her that her condition did not seem to be serious and stressed that I was confident to be able to help her.
- She left my office.
The point I am trying to make is: prescribing an entirely straight forward drug treatment is also a COMPLEX INTERVENTION. In fact, I know of no treatment that is NOT complex.
Does that mean that drugs and all other interventions are exempt from being tested in rigorous RCTs? Should we allow drug companies to adopt the RWR too? Any old placebo would pass that test and could be made to look effective using RWR. In the example above, my compassion, care and reassurance would alleviate my patient’s symptoms, even if the prescription I gave her was complete rubbish.
So why should acupuncture (or any other alternative therapy) not be tested in proper RCTs? I fear, the reason is that RCTs might show that it is not as effective as its proponents had hoped. The conclusion about the RWR is thus embarrassingly simple: proponents of alternative medicine want double standards because single standards would risk to disclose the truth.
If you talk to advocates of homeopathy, you are bound to hear claims that are false or misleading; in fact, you hear them so regularly that you might begin to doubt the truth. For those who have such doubts or are in need of some correct counter-arguments, I have listed here those 12 bogus claims which, in my experience, are most common together with short, suitable, and factual rebuttals.
1) THERE IS NOTHING MYSTERIOUS ABOUT HOMEOPATHY’S MODE OF ACTION, IT WORKS LIKE VACCINATIONS
This argument is used by enthusiasts in response the fact that most homeopathic remedies are too highly diluted to have pharmacological effects. Vaccines are also highly diluted and they are, of course, very effective; therefore, so the bogus notion, there is nothing odd about homeopathy.
The argument is wrong on several levels; the easiest way to refute, I think, it is to point out that vaccines contain measurable amounts of material and lead to measurable changes in the immune system. By contrast, the typical homeopathic remedy (beyond the C12 potency) contains not a single molecule of an active substance and leads to no measurable changes in any system.
2) SIGNIFICANTLY MORE CONTROLLED CLINICAL TRIALS OF HOMEOPATHY ARE POSITIVE THAN NEGATIVE
Several websites of homeopathic organisations make this claim and even provide simple statistics to back it up. Consequently, many homeopathy fans have adopted it.
The statistics they present show that x % of studies are positive, y % are negative and z % are neutral; the whole point is that x is larger than y. The percentage figures may even be correct but they rely on the spurious definitions used: positive = superior to placebo, negative = placebo superior to homeopathy, neutral = no difference between homeopathy and placebo. The latter category was created so that homeopathy comes out trumps.
For all intents and purposes, a study where the experimental treatment is no better than placebo is not a study neutral but a negative result. Thus the negative category in such statistics must be y + z which is, of course, larger than x. In other words, the majority of trials is, in truth, negative.
3) HOMEOPATHY IS SUPPORTED BY NOBEL PRIZE WINNERS
I don’t know of a single Nobel Prize winner who has stated or implied that homeopathy works better than a placebo. Some have tried to find a mechanism of action for homeopathy by doing some basic research and have published theories about it. None of those has been accepted by science.
And if there ever should be a Nobel Prize winner or similarly brilliant person who supports homeopathy, this would merely show that even bright individuals can make mistakes!
4) HOMEOPATHY IS SAFE
Tell that to the child that has just been reported to have died because her parents used homeopathy for an ear infection which (could have been easily treated with antibiotics but) degenerated into a brain abscess with homeopathic therapy. There are many more such tragic cases than I care to remember.
The risks of homeopathy are, of course, minor compared to many conventional treatments, but the risk/benefit balance of homeopathy can never be positive because, unlike those high risk conventional treatments, it has no benefit.
5) HOMEOPATHY DOES NOT LEND ITSELF TO BEING TESTED IN CLINICAL TRIALS
The best way to disprove this argument is to point out that ~ 250 controlled clinical trials are currently available. Every homeopath on the planet boasts about clinical trials – provided they are positive.
6) HOMEOPATHY WORKS VIA QUANTUM ENTANGLEMENT
I do not understand quantum mechanics and, I suspect, neither do the homeopaths who use this argument. But physicists who do understand this subject well are keen to stress that homeopathy cannot be explained in this way.
7) THERE IS NO PROOF THAT HOMEOPATHY DOES NOT WORK
The absence of evidence is not the same as evidence of absence, homeopaths like to exclaim. And they are, of course, correct! However, they forget that, science cannot prove a negative and that, in routine health care, we do not even look for a proof of ineffectiveness. We use those treatments that have a positive proof of effectiveness – everything else is irresponsible.
8) EVEN IF HOMEOPATHY WERE JUST A PLACEBO, IT STILL HELPS PATIENTS AND IS THEREFORE A USEFUL TREATMENT
It is true, of course, that placebo effects can help patients. But it is not true that, for generating a placebo response, we need a placebo. If a clinician administers an effective treatment with compassion, the patient will benefit from a placebo response plus from the specific effects of the treatment. Only giving placebos is therefore tantamount to cheating the patient.
9) THERE IS A WORLDWIDE CONSPIRACY AGAINST HOMEOPATHY
In a way, this argument merely suggests that homeopathic remedies are ineffective in treating paranoia. I have not ever seen a jot of evidence for it – and neither can anyone who uses this claim produce any.
10) YOU NEED TO BE A HOMEOPATH TO BE ABLE TO UNDERSTAND AND ADEQUATELY JUDGE THE VALUE OF HOMEOPATHY
With this notion, homeopaths want to claim that the critics of homeopathy are incompetent. It is like saying that only people who believe in god are allowed to criticise religion. By definition, homeopaths are believers, and therefore they are unlikely to be free of bias when judging the value of homeopathy. Homeopathy is a health technology that must be evaluated like all other health technologies: by independent scientists who know their job.
11) HOMEOPATHY HAS BEEN PROVEN TO WORK FOR LITTLE CHILDREN AND ANIMALS
The argument here is that animals and children cannot possibly respond to placebo. Therefore homeopathy must be more than a placebo.
This notion is twice wrong. Firstly, both animals and children can respond to placebo, if only ‘by proxy’, i.e. via their carers. Secondly, if we consider the totality of the reliable data, we find that neither for children nor for animals is the evidence convincingly positive.
12) HOMEOPATHY HAS BEEN USED VERY SUCCESSFULLY IN MAJOR EPIDEMICS, AND THAT FACT IS PROOF ENOUGH FOR ITS EFFICACY
Yes, there are some rather fascinating historical accounts which homeopaths interpret in this fashion. But if we look a little closer, we invariably find explanations which are much more plausible than the assumption of homeopathy’s effectiveness. Epidemiological observations of this nature can almost never establish cause and effect, and the clinical outcome could have been due to a myriad of confounders unrelated to homeopathy.
Necessity, they say, is the mother of invention. The meaning of this proverb is fairly clear:
- In the Oxford Dictionary the proverb has been defined as– when the need for something becomes imperative, you are forced to find ways of getting or achieving it.
- According to the Cambridge Dictionary, this is “an expression that means that if you really need to do something, you will think of a way of doing it.”
- Finally, the Longman dictionary has defined the proverb as– “if someone really needs to do something, they will find a way of doing it.”
In the world of chiropractic the proverb acquires a special meaning: chiropractic relies almost entirely on inventions. A few examples have to suffice:
- first, instead of pathophysiology, they invented subluxations,
- this required the invention of adjustments which were needed for their imagined subluxation,
- then they invented the ‘inate’,
- then they invented the idea that all sorts of conditions are caused by subluxations and therefore require adjustments,
- finally, they invented the notion that regular adjustments are needed for a healthy person to stay healthy.
I was reminded of the unique inventive capacity of chiropractic when I came across the website of the Foundation for Chiropractic Progress (F4CP). The F4CP is, according to their own statements, a not-for-profit organization dedicated to raising awareness about the value of chiropractic care (which is, of course, another invention).
Experts at the F4CP point out that a growing number of professional athletic teams utilize chiropractic care to maximize overall health and maintain peak performance. “Repetitive motion injuries, including shoulder tendinitis, elbow, lower back pain and muscle spasms, are common conditions and injuries among professional baseball players that can be successfully prevented, managed and treated with chiropractic care,” says Hirad N. Bagy, DC. “Chiropractic adjustments, in conjunction with soft tissue mobilization, provide athletes with proper structure, function and balance to reduce the risk of injury, accelerate recovery time and improve overall performance,” he continues – and he must know, because he has received specialized training and certifications specific to sports medicine, which include the Graston Technique®, Active Release Technique®, Myofascial Release Technique, Impact Concussion Testing and Functional Dry Needling. Dr. Bagy continues: “A number of athletes that I treat regularly understand the importance of chiropractic maintenance care, and also seek treatment when an injury arises. Through the restoration of proper bio-mechanics, doctors of chiropractic are now positioned as key health care providers throughout all of the sports teams that I work with.”
BRAVO! We are impressed! So much so, that we almost forgot to ask: “Is there any evidence for all of these therapeutic claims?”
Just as well! Because had we asked and perhaps even did a bit of research, we would have found that almost none of these far-reaching claims are evidence-based.
But who would be so petty? Instead of criticising the incessant flow of bogus claims made by chiropractors worldwide, we should really admire their remarkable skill of invention:
- When the need for profit becomes imperative, CHIROPRACTORS are forced to find ways of getting or achieving it.
- If CHIROPRACTORS really need to do something, they will think of a way of doing it.
- If a CHIROPRACTOR really needs money, he will advocate ‘maintenance care’.
AND THAT’S WHAT IS CALLED ‘CHIROPRACTIC PROGRESS’!
If you ask a chiropractor, you will probably be told that chiropractic spinal manipulation is a safe treatment. Unfortunately this is not quite true, as regular readers of this blog will appreciate. About half of all patients suffer mild to moderate adverse effects after chiropractic treatments and, in addition, many instances of much more serious complications have been documented, including rare cases of Horner syndrome. It results from an interruption of the sympathetic nerve supply to the eye and is characterized by the classic triad of miosis (ie, constricted pupil), partial ptosis, and loss of hemifacial sweating (ie, anhidrosis).
Danish neurologists recently reported the case of a 60-year-old man with no relevant medical history who was admitted to the Department of Neurology with drooping of his right upper eyelid and an ipsilateral contracted pupil, combined with pain, weakness, and numbness in his upper right limb.
The patient had experienced thoracic back pain of moderate intensity with radiating right-sided belt-like chest pain for 7 days. When the discomfort suddenly intensified, he sought chiropractic treatment. Following manipulations of the thoracic and cervical spine, the pain intensity initially lessened. Approximately one hour after chiropractic treatment, the patient experienced the eye and upper limb symptoms described above, for which he sought medical assistance three days later.
A detailed neurologic examination revealed moderate right-sided ptosis and miosis, no facial anhidrosis, decreased strength of the intrinsic and opponens muscles of the right hand, and reduced cutaneous sensation corresponding to the T1 dermatome, with inability to discriminate pain and light touch. The remaining clinical examination, routine blood tests, and vital parameters were unremarkable.
Brain CT scan and CT angiography including the aortic arch and neck vessels were performed and ruled out cerebral stroke and carotid artery dissection, respectively. As clinical signs of Horner syndrome and a concomitant radiating pain to the medial arm were considered suggestive of either a lower brachial plexopathy, i.e., due to a Pancoast tumor, or a radiculopathy, chest X-ray and electroneurography (ENG) were performed. No apical pulmonary pathology was detected. ENG of the right medial cutaneous antebrachial nerve demonstrated a normal sensory action potential (SAP), consistent with the lesion being located proximally to the dorsal spinal root ganglion, thus suggestive of a spinal nerve root lesion. A subsequent MRI of the thoracic spine showed a para-median herniation of the T1-T2 intervertebral disc compressing the right T1 spinal nerve root.
The patient received no surgery, and follow-up examination 6 months later revealed near-complete recovery, with only mild paraesthesia in the T1 segment of his right arm and a subtle ptosis remaining.
Horner syndrome due to a herniated thoracic disc has only been reported 6 times in the English language literature, though never preceded by chiropractic manipulation.
One of the most frequent causes of Horner syndrome is carotid artery dissection, which may occur spontaneously or due to local trauma to the neck region. Chiropractic manipulation as an independent risk factor for neck artery dissection and a consequent stroke is a controversial topic, though multiple cases of Horner syndrome due to ICA dissections subsequent to chiropractic manipulation have been reported. In the patient described here, an ICA dissection was considered unlikely due to the concomitant prominent radiating medial brachialgia and was furthermore ruled out by a CT angiogram of the neck vessels.
This patient experienced the onset of a Horner syndrome and ipsilateral upper limb symptoms shortly after chiropractic treatment, suggesting the cervico-thoracic manipulation as the cause of or at least worsening factor in the T1-T2 disc herniation. Several cases of disc herniations following chiropractic treatment have been reported.
While the definite pathophysiologic mechanism to explain this patient’s Horner syndrome remains unclear, it seems, according to the authors of this case-report, evident that manipulations as a minimum altered the configuration of an already existing disc protrusion.
You may feel that homeopaths are bizarre, irrational, perhaps even stupid – but you cannot deny their tenacity. Since 200 years, they are trying to convince us that their treatments are effective beyond placebo. And they seem to get more and more bold with their claims: while they used to suggest that homeopathy was effective for trivial conditions like a common cold, they now have their eyes on much more ambitious things. Two recent studies, for instance, claim that homeopathic remedies can help cancer patients.
The aim of the first study was to evaluate whether homeopathy influenced global health status and subjective wellbeing when used as an adjunct to conventional cancer therapy.
In this pragmatic randomized controlled trial, 410 patients, who were treated by standard anti-neoplastic therapy, were randomized to receive or not receive classical homeopathic adjunctive therapy in addition to standard therapy. The main outcome measures were global health status and subjective wellbeing as assessed by the patients. At each of three visits (one baseline, two follow-up visits), patients filled in two questionnaires for quantification of these endpoints.
The results show that 373 patients yielded at least one of three measurements. The improvement of global health status between visits 1 and 3 was significantly stronger in the homeopathy group by 7.7 (95% CI 2.3-13.0, p=0.005) when compared with the control group. A significant group difference was also observed with respect to subjective wellbeing by 14.7 (95% CI 8.5-21.0, p<0.001) in favor of the homeopathic as compared with the control group. Control patients showed a significant improvement only in subjective wellbeing between their first and third visits.
Our homeopaths concluded that the results suggest that the global health status and subjective wellbeing of cancer patients improve significantly when adjunct classical homeopathic treatment is administered in addition to conventional therapy.
The second study is a little more modest; it had the aim to explore the benefits of a three-month course of individualised homeopathy (IH) for survivors of cancer.
Fifteen survivors of any type of cancer were recruited by a walk-in cancer support centre. Conventional treatment had to have taken place within the last three years. Patients scored their total, physical and emotional wellbeing using the Functional Assessment of Chronic Illness Therapy for Cancer (FACIT-G) before and after receiving four IH sessions.
The results showed that 11 women had statistically positive results for emotional, physical and total wellbeing based on FACIT-G scores.
And the conclusion: Findings support previous research, suggesting CAM or individualised homeopathy could be beneficial for survivors of cancer.
As I said: one has to admire their tenacity, perhaps also their chutzpa – but not their understanding of science or their intelligence. If they were able to think critically, they could only arrive at one conclusion: STUDY DESIGNS THAT ARE WIDE OPEN TO BIAS ARE LIKELY TO DELIVER BIASED RESULTS.
The second study is a mere observation without a control group. The reported outcomes could be due to placebo, expectation, extra attention or social desirability. We obviously need an RCT! But the first study was an RCT!!! Its results are therefore more convincing, aren’t they?
No, not at all. I can repeat my sentence from above: The reported outcomes could be due to placebo, expectation, extra attention or social desirability. And if you don’t believe it, please read what I have posted about the infamous ‘A+B versus B’ trial design (here and here and here and here and here for instance).
My point is that such a study, while looking rigorous to the naïve reader (after all, it’s an RCT!!!), is just as inconclusive when it comes to establishing cause and effect as a simple case series which (almost) everyone knows by now to be utterly useless for that purpose. The fact that the A+B versus B design is nevertheless being used over and over again in alternative medicine for drawing causal conclusions amounts to deceit – and deceit is unethical, as we all know.
My overall conclusion about all this:
QUACKS LOVE THIS STUDY DESIGN BECAUSE IT NEVER FAILS TO PRODUCE FALSE POSITIVE RESULTS.
A recent comment to a post of mine (by a well-known and experienced German alt med researcher) made the following bold statement aimed directly at me and at my apparent lack of understanding research methodology:
C´mon , as researcher you should know the difference between efficacy and effectiveness. This is pharmacological basic knowledge. Specific (efficacy) + nonspecific effects = effectiveness. And, in fact, everything can be effective – because of non-specific or placebo-like effects. That does not mean that efficacy is existent.
The point he wanted to make is that outcome studies – studies without a control group where the researcher simply observe the outcome of a particular treatment in a ‘real life’ situation – suffice to demonstrate the effectiveness of therapeutic interventions. This belief is very wide-spread in alternative medicine and tends to mislead all concerned. It is therefore worth re-visiting this issue here in an attempt to create some clarity.
When a patient’s condition improves after receiving a therapy, it is very tempting to feel that this improvement reflects the effectiveness of the intervention (as the researcher mentioned above obviously does). Tempting but wrong: there are many other factors involved as well, for instance:
- the placebo effect (mainly based on conditioning and expectation),
- the therapeutic relationship with the clinician (empathy, compassion etc.),
- the regression towards the mean (outliers tend to return to the mean value),
- the natural history of the patient’s condition (most conditions get better even without treatment),
- social desirability (patients tend to say they are better to please their friendly clinician),
- concomitant treatments (patients often use treatments other than the prescribed one without telling their clinician).
So, how does this fit into the statement above ‘Specific (efficacy) + nonspecific effects = effectiveness’? Even if this formula were correct, it would not mean that outcome studies of the nature described demonstrate the effectiveness of a therapy. It all depends, of course, on what we call ‘non-specific’ effects. We all agree that placebo-effects belong to this category. Probably, most experts also would include the therapeutic relationship and the regression towards the mean under this umbrella. But the last three points from my list are clearly not non-specific effects of the therapy; they are therapy-independent determinants of the clinical outcome.
The most important factor here is usually the natural history of the disease. Some people find it hard to imagine what this term actually means. Here is a little joke which, I hope, will make its meaning clear and memorable.
CONVERATION BETWEEN TWO HOSPITAL DOCTORS:
Doc A: The patient from room 12 is much better today.
Doc B: Yes, we stared his treatment just in time; a day later and he would have been cured without it!
I am sure that most of my readers now understand (and never forget) that clinical improvement cannot be equated with the effectiveness of the treatment administered (they might thus be immune to the misleading messages they are constantly exposed to). Yet, I am not at all sure that all ‘alternativists’ have got it.
In my last post, I claimed that researchers of alternative medicine tend to be less than rigorous. I did not link this statement to any evidence at all. Perhaps I should have at least provided an example!? As it happens, I just came across a brand new paper which nicely demonstrates what I meant.
According to its authors, this non-interventional study was performed to generate data on safety and treatment effects of a complex homeopathic drug. They treated 1050 outpatients suffering from common cold with a commercially available homeopathic remedy for 8 days. The study was conducted in 64 German outpatient practices of medical doctors trained in CAM. Tolerability, compliance and the treatment effects were assessed by the physicians and by patient diaries. Adverse events were collected and assessed with specific attention to homeopathic aggravation and proving symptoms. Each adverse effect was additionally evaluated by an advisory board of experts.
The physicians detected 60 adverse events from 46 patients (4.4%). Adverse drug reactions occurred in 14 patients (1.3%). Six patients showed proving symptoms (0.57%) and only one homeopathic aggravation (0.1%) appeared. The rate of compliance was 84% for all groups. The global assessment of the treatment effects resulted in the verdict “good” and “very good” in 84.9% of all patients.
The authors concluded that the homeopathic complex drug was shown to be safe and effective for children and adults likewise. Adverse reactions specifically related to homeopathic principles are very rare. All observed events recovered quickly and were of mild to moderate intensity.
So why do I think this is ‘positively barmy’?
The study had no control group. This means that there is no way anyone can attribute the observed ‘treatment effects’ to the homeopathic remedy. There are many other phenomena that may have caused or contributed to it, e. g.:
- a placebo effect
- the natural history of the condition
- regression to the mean
- other treatments which the patients took but did not declare
- the empathic encounter with the physician
- social desirability
To plan a study with the aim as stated above and to draw the conclusion as cited above is naïve and unprofessional (to say the least) on the part of the researchers (I often wonder where, in such cases, the boundary between incompetence and research misconduct might lie). To pass such a paper through the peer review process is negligent on the part of the reviewers. To publish the article is irresponsible on the part of the editor.
In a nut-shell: COLLECTIVELY, THIS IS ‘POSITIVELY BARMY’!!!
Yoga is a popular form of alternative medicine. Evidence for its effectiveness is scarce and generally far from convincing. But at least it is safe! At least this is what yoga enthusiasts would claim. Unfortunately, this is not entirely true; adverse events have also been reported with some regularity. Their frequency is, however, not known.
A new study was aimed at filling this gap. It was conducted to elucidate the frequencies and characteristics of adverse events of yoga performed in classes and the risk factors of such events.
The subjects were 2508 people taking yoga classes and 271 yoga therapists conducting the classes. A survey for yoga class attendees was performed on adverse events that occurred during a yoga class on the survey day. A survey for yoga therapists was performed on adverse events that the therapists had observed in their students to date. Adverse events were defined as “undesirable symptoms or responses that occurred during a yoga class”.
Among 2508 yoga class attendees, 1343 (53.5%) had chronic diseases and 1063 (42.3%) were receiving medication at hospitals. There were 687 class attendees (27.8%) who reported some type of undesirable symptoms after taking a yoga class. Musculoskeletal symptoms such as myalgia were the most common symptoms, involving 297 cases, followed by neurological symptoms and respiratory symptoms. Most adverse events (63.8%) were mild and did not interfere with class participation. The risk factors for adverse events were examined, and the odds ratios for adverse events were significantly higher in attendees with chronic disease, poor physical condition on the survey day, or a feeling that the class was physically and mentally stressful. In particular, the occurrence of severe adverse events that interfered with subsequent yoga practice was high among elderly participants (70 years or older) and those with chronic musculoskeletal diseases.
The authors concluded that the results of this large-scale survey demonstrated that approximately 30% of yoga class attendees had experienced some type of adverse event. Although the majority had mild symptoms, the survey results indicated that attendees with chronic diseases were more likely to experience adverse events associated with their disease. Therefore, special attention is necessary when yoga is introduced to patients with stress-related, chronic diseases.
I find these findings interesting and thought-provoking. The main question that they raise is, I think, the flowing: ARE THERE ANY CONDITIONS FOR WHICH YOGA DEMONSTRABLY GENERATES MORE GOOD THAN HARM?