clinical trial
This new RCT by researchers from the National Institute of Complementary Medicine in Sydney, Australia was aimed at ‘examining the effect of changing treatment timing and the use of manual, electro acupuncture on the symptoms of primary dysmenorrhea’. It had four arms:
- low frequency manual acupuncture (LF-MA),
- high frequency manual acupuncture (HF-MA),
- low frequency electro acupuncture (LF-EA)
- and high frequency electro acupuncture (HF-EA).
A total of 74 women were given 12 treatments over three menstrual cycles, either once per week (LF groups) or three times in the week prior to menses (HF groups). All groups received a treatment in the first 48 hours of menses. The primary outcome was the reduction in peak menstrual pain at 12 months from trial entry.
During the treatment period and 9 month follow-up all groups showed statistically significant reductions in peak and average menstrual pain compared to baseline. However, there were no differences between groups. Health related quality of life increased significantly in 6 domains in groups having high frequency of treatment compared to two domains in low frequency groups. Manual acupuncture groups required less analgesic medication than electro-acupuncture groups. HF-MA was most effective in reducing secondary menstrual symptoms compared to both–EA groups.
The authors concluded that acupuncture treatment reduced menstrual pain intensity and duration after three months of treatment and this was sustained for up to one year after trial entry. The effect of changing mode of stimulation or frequency of treatment on menstrual pain was not significant. This may be due to a lack of power. The role of acupuncture stimulation on menstrual pain needs to be investigated in appropriately powered randomised controlled trials.
If I were not used to reading rubbish research of alternative medicine in general and acupuncture in particular, this RCT would amaze me – not so much because of its design, execution, or write-up, but primarily because of its conclusion (why, oh why, I ask myself, did PLOS ONE publish this paper?). They are, I think, utterly barmy.
Let me explain:
- “acupuncture treatment reduced menstrual pain intensity” – oh no, it didn’t; at least this is not what the study proves; the fact that pain was perceived as less could be due to a host of factors, for instance regression towards the mean, or social desirability; as there was no proper control group, nobody can tell;
- the lack of difference between treatments “may be due to a lack of power”. Yes, but more likely it is due to the fact that all versions of a placebo therapy generate similar outcomes.
- “acupuncture stimulation on menstrual pain needs to be investigated in appropriately powered randomised controlled trials”. Why? Because the authors have a quasi-religious belief in acupuncture? And if they have, why did they not design their study ‘appropriately’?
The best conclusion I can suggest for this daft trial is this: IN THIS STUDY, THE PRIMARY ENDPOINT SHOWED NO DIFFERENCE BETWEEN THE 4 TREATMENT GROUPS. THE RESULTS ARE THEREFORE FULLY COMPATIBLE WITH THE NOTION THAT ACUPUNCTURE IS A PLACEBO THERAPY.
Something along these lines would, in my view, have been honest and scientific. Sadly, in acupuncture research, we very rarely get such honest science and the ‘National Institute of Complementary Medicine in Sydney, Australia’ has no track record of being the laudable exception to this rule.
I have repeatedly cautioned about the often poor quality of research into alternative medicine. This seems particularly necessary with studies of acupuncture, and especially true for such research carried out in China. I have also frequently noted that certain ‘CAM journals’ are notoriously prone to publishing rubbish. So, what can we expect from a paper that:
- is on alternative medicine,
- focusses on acupuncture,
- is authored by Chinese researchers,
- was published in the Journal of Alternative and Complementary Medicine (JACM)?
The answer is PROBABLY NOT A LOT!
As if for confirming my prediction, The JACM just published this systematic review. It reports pairwise and network meta-analyses to determine the effectiveness of acupuncture and acupuncture-related techniques for the treatment of psoriasis. A total of 13 RCTs were included. The methodological quality of these studies was ‘not rigorous’ according to the authors – in fact, it was lousy. Acupoint stimulation seemed to be more effective than non-acupoint stimulation. The short-term treatment effect was superior to the long-term effect (as one would expect with placebo). Network meta-analysis suggested that acupressure or acupoint catgut embedding generate superior effects compared to medications. It was noted that acupressure was the most effective treatment of all the acupuncture-like therapies.
The authors concluded that acupuncture-related techniques could be considered as an alternative or adjuvant therapy for psoriasis in short term, especially of acupressure and acupoint catgut embedding. This study recommends further well-designed, methodologically rigorous, and more head-to-head randomized trials to explore the effects of acupuncture-related techniques for treating psoriasis.
And what is wrong with that?
EVERYTHING!
- The review is of very poor quality.
- The primary studies are even worse.
- The English language is defective to the point of being not understandable.
- The conclusions are misleading.
Correct conclusions should read something like this: Due to the paucity and the poor quality of the clinical trials, this review could not determine whether acupuncture and similar therapies are effective for psoriasis.
And then there is, of course, the question about plausibility. How plausible is the assumption that acupuncture might affect a genetic autoimmune disease like psoriasis. The answer, I think, is that the assumption is highly unlikely.
In the above review, most of the 13 primary RCTs were from China. One of the few studies not conducted in China is this one:
56 patients suffering from long-standing plaque psoriasis were randomized to receive either active treatment (electrostimulation by needles placed intramuscularly, plus ear-acupuncture) or placebo (sham, ‘minimal acupuncture‘) twice weekly for 10 weeks. The severity of the skin lesions was scored (PASI) before, during, and 3 months after therapy. After 10 weeks of treatment the PASI mean value had decreased from 9.6 to 8.3 in the ‘active’ group and from 9.2 to 6.9 in the placebo group (p < 0.05 for both groups). These effects are less than the usual placebo effect of about 30%. There were no statistically significant differences between the outcomes in the two groups during or 3 months after therapy. The patient’s own opinion about the results showed no preference for ‘active’ therapy. It was also clear from the answers that the blinded nature of the study had not been discovered by the patients. In conclusion, classical acupuncture is not superior to sham (placebo) ‘minimal acupuncture‘ in the treatment of psoriasis.
Somehow, I trust these conclusions more than the ones from the review!
And somehow, I get very tired of journal editors failing to do their job of rejecting papers that evidently are embarrassing, unethical rubbish.
Many garlic supplements are heavily marketed as a treatment of infections.
But are they really effective?
To answer this question, we clearly need clinical trials.
The aim of this RCT was to examine the impact of garlic tablets on nosocomial infections in hospitalized patients in intensive care units. It was carried out on 94 patients, admitted to the intensive care units in Kashani and Al-Zahra hospitals. Patients were randomised into case and control groups. The case group administered one 400 mg garlic tablet (Garlic tablets 400 mg, Gol Darou Company) daily for 6 days and the control group received placebo. During the study, inflammatory blood factors and infection occurrence in the two groups were compared. During the study period, 78 intravenous catheter tips were sent to laboratory for culture of which 37 cases were in the intervention group and 41 in the control group. Culture results of Catheter tips was positive in 5 cases all of which were in the control group. Frequency distribution of catheter tip culture was significantly higher in the control group than that of the intervention group. The authors concluded that garlic supplementation has shown to be effective in patients admitted to ICU, who are highly susceptible to nosocomial infection, and it can be used for the prevention of septicemia and urinary tract infections. However, further research with larger sample size is needed.
The trouble is not just that this trial was less than rigorous, but that there are so very few similar investigations to confirm or refute the anti-infectious activities of garlic.
In this study, healthy human participants (n = 120), between 21 and 50 y of age, were recruited for a randomized, double-blind, placebo-controlled parallel-intervention study to consume 2.56 g aged garlic extract (AGE)/d or placebo supplements for 90 d during the cold and flu season. Peripheral blood mononuclear cells were isolated before and after consumption, and γδ-T and NK cell function was assessed by flow cytometry. The effect on cold and flu symptoms was determined by using daily diary records of self-reported illnesses. After 45 d of AGE consumption, γδ-T and NK cells proliferated better and were more activated than cells from the placebo group. After 90 d, although the number of illnesses was not significantly different, the AGE group showed reduced cold and flu severity, with a reduction in the number of symptoms, the number of days participants functioned suboptimally, and the number of work/school days missed. The authors concluded that AGE supplementation may enhance immune cell function and may be partly responsible for the reduced severity of colds and flu reported. The results also suggest that the immune system functions well with AGE supplementation, perhaps with less accompanying inflammation.
There is plenty of in vitro evidence to suggest that garlic and its compounds have anti-bacterial, anti-viral and anti-fungal effects. Yet, for a range of reasons, this may not translate into clinical effects. To find out, we need clinical trials. So far, such investigations were almost entirely missing.
The two recent studies above are, I think, a good start. They are far from perfect but their findings are nevertheless mildly encouraging. For once, I do agree with the standard conclusion in alternative medicine:
More and better clinical trials are justified.
I was surprised to receive this email yesterday: “Hello Edzard Ernst, You may remember I got in touch last week regarding losing a loved one to the ravages of drugs or alcohol. I just wanted to remind you that Narconon is here to help. For over fifty years Narconon drug and alcohol rehabilitation centres have been successfully reversing the tide of addiction for men and woman from all walks of life. The Narconon programme has saved them from the misery of addiction, and the potential of an early grave. We not only address the cause of the addiction, we resolve them…”
The email was signed by a man from ‘Narconon International’. First I thought someone has been counting the empty bottles in my bin, then I read it again and noticed the word ‘NARCONON’ and remembered that I once wrote about it. A quick search located my article from THE GUARDIAN 2012:
Imagine a therapy that “enables an individual to rid himself of the harmful effects of drugs, toxins and other chemicals that lodge in the body and create a biochemical barrier to spiritual well-being“. If you were told that the treatment was entirely natural and had already “enabled hundreds of thousands to free themselves from the harmful effects of drugs and toxins and so achieve spiritual gains”, wouldn’t you be tempted to try it?
Who doesn’t want a body free of nasty chemicals? And who wouldn’t be delighted at the chance to counter a growing threat to an “advancement in mental … wellbeing”?
These claims are being made for the “Purification Rundown” (“Purif” for short) and the closely related Narconon detox programmes, which mainly consist of regular exercise, sauna and nutrition, with industrial doses of vitamins and minerals added for good measure. Some of the claims are quite specific: the Purif programme is supposed to increase your IQ, reduce the level of cancer-causing agents in your body, and even enable you to lose weight easily and quickly. The Narconon programme is more specifically targeted at drug and alcohol dependency and is claimed to have an impressive success rate of 75%.
Both programmes were developed by L Ron Hubbard (1911-1986) and are currently marketed by the Church of Scientology. The CoS is not generally known to be an organisation that promotes healthcare programmes. Hubbard, the pulp-fiction writer who founded the CoS, portrayed himself somewhat over-optimistically as a pioneer, innovator and nuclear physicist.
He taught his followers that, at their core, humans contain a “thetan”. After creating the universe, thetans accidentally became trapped in physical bodies and, through scientology, we can restore the immortal, omnipotent, god-like powers of the “thetan” within us. Weird stuff that is the preserve of Hollywood eccentrics, you might think, but perhaps the CoS’s detox-ventures are an attempt to conquer new territory?
A typical course of treatment lasts several weeks and consists of many hours of exercise and sauna every day. This regimen is supplemented with megadoses of vitamins and minerals, which can cause problems. Niacin, one vitamin that is given in high doses as part of the regimen, can be particularly dangerous. The US National Institutes of Health warns that at high doses it can cause “liver problems, gout, ulcers of the digestive tract, loss of vision, high blood sugar, irregular heartbeat, and other serious problems.” It should not be taken by people who already have liver damage.
Seven fatalities of people undergoing the Narconon programme are currently being investigated in Oklahoma, although the CoS says these deaths are not connected with the treatment regimen itself.
Whatever the truth regarding these deaths, a review of the evidence about the treatment regimen’s effectiveness – carried out by the Norwegian Knowledge Centre for the Health Services in 2008 – found no good evidence that the Narconon programme works:
There is currently no reliable evidence for the effectiveness of Narconon as a primary or secondary drug prevention program. This is partly due to the insufficient research evidence about Narconon and partly due to the non-experimental nature of the few studies that exist.
The claim that such detox treatments eliminate toxins from the body is, of course, easily testable. All we would need to do is define what toxin we are talking about and measure the change in levels of that toxin compared with a control group of volunteers who did not receive the detox.
But such studies are not available. Why? Do the marketing men believe in their own claims? Maybe they feel that profits and evidence are like fire and water? Or possibly the thetans have an aversion to science?
If you think that the Purif, Narconon or any other form of alternative detox eliminates toxins, you might be mistaken. Most clients have lost some money, many have lost their ability to think straight, some may even have lost their lives. But there is no reliable evidence that they have actually lost any toxins.
END OF MY 2012 ARTICLE
In 2012, I found no evidence to suggest that NARCONON works. Now, I looked again and found this article reporting a non-randomised, controlled study:
“In 2004, Narconon International developed a multi-module, universal prevention curriculum for high school ages based on drug abuse etiology, program quality management data, prevention theory and best practices. We review the curriculum and its rationale and test its ability to change drug use behavior, perceptions of risk/benefits, and general knowledge. After informed parental consent, approximately 1000 Oklahoma and Hawai’i high school students completed a modified Center for Substance Abuse Prevention (CSAP) Participant Outcome Measures for Discretionary Programs survey at three testing points: baseline, one month later, and six month follow-up. Schools assigned to experimental conditions scheduled the Narconon curriculum between the baseline and one-month follow-up test; schools in control conditions received drug education after the six-month follow-up. Student responses were analyzed controlling for baseline differences using analysis of covariance. At six month follow-up, youths who received the Narconon drug education curriculum showed reduced drug use compared with controls across all drug categories tested. The strongest effects were seen in all tobacco products and cigarette frequency followed by marijuana. There were also significant reductions measured for alcohol and amphetamines. The program also produced changes in knowledge, attitudes and perception of risk. The eight-module Narconon curriculum has thorough grounding in substance abuse etiology and prevention theory. Incorporating several historically successful prevention strategies this curriculum reduced drug use among youths.”
The question arises: would I send anyone to the NARCONON programme?
My answer is NO!
Not because the trial is lousy (which it is) and not because the programme is too expensive (which it is); I would not send anyone to any institution that has even the slightest links to Scientology.
This study tested chondroitin sulfate 800 mg/day (CS) pharmaceutical-grade in the management of symptomatic knee osteoarthritis. It was designed as a prospective, randomised, 6-month, 3-arm, double-blind, double-dummy, placebo and celecoxib (200 mg/day)-controlled trial. The primary endpoints were changes in pain on a Visual Analogue Scale (VAS) and in the Lequesne Index (LI). Minimal-Clinically Important Improvement (MCII), Patient-Acceptable Symptoms State (PASS) were used as secondary endpoints.
A total of 604 patients, diagnosed according to American College of Rheumalogy (ACR) criteria, were recruited in five European countries and followed for 182 days. CS and celecoxib showed a greater significant reduction in pain and LI than placebo. In the intention-to-treat (ITT) population, pain reduction in VAS at day 182 in the CS group (−42.6 mm) and in celecoxib group (−39.5 mm) was significantly greater than the placebo group (−33.3 mm) (p=0.001 for CS and p=0.009 for celecoxib). No difference observed between CS and celecoxib. Similar trend for the LI, as reduction in this metric in the CS group (−4.7) and celecoxib group (−4.6) was significantly greater than the placebo group (−3.7) (p=0.023 for CS and p=0.015 for celecoxib). Again, no difference was observed between CS and celecoxib. Both secondary endpoints (MCII and PASS) at day 182 improved significantly in the CS and celecoxib groups. All treatments demonstrated excellent safety profiles.
The authors concluded that a 800 mg/day pharmaceutical-grade CS is superior to placebo and similar to celecoxib in reducing pain and improving function over 6 months in symptomatic knee osteoarthritis (OA) patients. This formulation of CS should be considered a first-line treatment in the medical management of knee OA.
In my view, this is a good study with clear and useful results: CS seems to be efficacious and safe. Another recent study confirmed the superiority of CS over celecoxib at reducing cartilage volume loss in knee OA patients.
The current Cochrane review does not yet account for the new data; it concluded cautiously positive: A review of randomized trials of mostly low quality reveals that chondroitin (alone or in combination with glucosamine) was better than placebo in improving pain in participants with osteoarthritis in short-term studies. The benefit was small to moderate with an 8 point greater improvement in pain (range 0 to 100) and a 2 point greater improvement in Lequesne’s index (range 0 to 24), both seeming clinically meaningful. These differences persisted in some sensitivity analyses and not others. Chondroitin had a lower risk of serious adverse events compared with control. More high-quality studies are needed to explore the role of chondroitin in the treatment of osteoarthritis. The combination of some efficacy and low risk associated with chondroitin may explain its popularity among patients as an over-the-counter supplement.
The call for more high quality trials was justified but has now been answered. In my view, CS can be considered an evidence-based option in the management of OA.
I am sure you always wanted to know what animal chiropractic is all about!
This website explains it quite well:
START OF QUOTES
…Animal chiropractic (veterinary spinal manipulative therapy) focuses on the preservation and health/wellness of the neuro-musculo-skeletal system. Chiropractic is the science that is centered around the relationship between the spine and the nervous system. The spine is your body’s foundation and the nervous system, including your brain, spinal cord and nerves, controls your entire body. They must work together harmoniously to improve one’s general health and their ability heal. If the systems are not functioning to their highest potential you may experience changes in digestion, heart and lung function, reproduction and most evidently musculature. When adjacent joints are in an abnormal position, called a subluxation, the nervous system and all that it controls will be negatively impacted. If these subluxations are not corrected, they can result in prolonged inappropriate stimulation of nerves. This could result in reduced function internally, musculo-skeletal dysfunction and pain.
Spinal manipulation is the art of restoring full and pain free range of motion to joints and can greatly benefit an animal after they have experienced subluxations. The veterinarian will use their hands to palpate joints both statically and in motion. By doing this, they can determine where the animal is experiencing decreased motion or misaligned joints. Once identified, an adjustment can be performed. An adjustment or spinal manipulation is a gentle, specific, quick and low force thrust that will be applied at an angle specific to the different areas of motion in the spine and extremities. Only a certified animal chiropractor will understand the complexity involved in adjustments and can best assess if an animal can benefit from chiropractic care.
Many animals can benefit from this alternative therapy. If you notice that your animal has a particularly sensitive spot somewhere on their body, is walking or trotting differently and or not performing to the same ability they have previously, they may be a candidate for a chiropractic assessment. However, an animal does not need to be sick or injured to benefit from chiropractic care. Animals in good health or ones used for sporting activities are also prime candidates for chiropractic care. By maintaining your pet’s proper spinal alignment and mobility they will attain optimal function of muscles, nerves and tissues that support the joints. When the body can move freely your pet will experience improved mobility, stance and flexibility, which can evolve into improved agility, endurance and overall performance. Finally, many people have never considered that chiropractic care can also benefit their animal by boosting their immune response. It can aid in providing a healthier metabolism and a vibrant nervous system which all facilitate your animal’s natural ability to heal themselves from within. Chiropractic care can enhance the quality of your pet’s life ensuring many active and healthy years to come.
…during veterinary school I began the process of researching how to become an animal chiropractor or veterinary spinal manipulative therapist. As I researched further, I noticed that this specialized profession has grown. It became apparent that one should be certified by either the College of Animal Chiropractors or American Veterinary Chiropractic Association to practice on animals… It was surprising to find out that there are only four programs in the USA and Canada that are approved by both organizations. The courses consisted of over 200 hours of intensive study and hands on learning followed by certification testing…
END OF QUOTE
Yes, I did shorten the quote a bit but, rest assured, I did not cut out a single word about the efficacy of animal chiropractic. Even if I had wanted to, I couldn’t: there is no mention of it in the article.
I wonder why!
Looking into Medline, I found several reports related to the subject:
- One study suggested an association between chiropractic findings in the lumbar vertebrae and urinary incontinence and retention in dogs.
- A case report highlighted the potential benefits of combining traditional medical management with chiropractic treatment and physical therapy techniques for management of severe acute-onset torticollis in a giraffe.
- A review explained that there is limited evidence supporting the effectiveness of spinal mobilization and manipulation in animals.
- An observational study suggested that chiropractic manipulations elicit slight but significant changes in thoracolumbar and pelvic kinematics.
- A comparative study measured the spinal mechanical nociceptive thresholds in 38 horses, and showed that they increased by 27, 12 and 8% in the chiropractic, massage and phenylbutazone groups, respectively.
… and that was basically it. Not a single study to suggest that chiropractic is effective for specific conditions of animals.
Frustrated, I went on the site of the ‘College of Animal Chiropractic’; surely there I would find the evidence I was looking for. They offer lots of platitudes and this largely nonsensical statement:
“When a joint become restricted in its range of motion(hypomobile or ‘locked-up’), through trauma, repetitive injury, degenerative changes, or structural stresses, the surrounding tissues are affected. This, in turn, further affects the joints ability to move freely and sensitive structures are activated causing the area to be sensitive or painful. Nerves are the communication links between all tissues in the body to the brain and spinal cord; when joint dysfunction is present, messages to other areas are also affected, which can lead to pain, weakness, reduced function, and compensatory changes. Animal chiropractic focuses on the restoration of movement and the promotion of heath by restoring normal joint mechanics and soft-tissue function, thus, normalizing neurological patterns that facilitate healing . The main tool an animal chiropractor uses to restore joint motion is called an “adjustment”, or veterinary spinal manipulation. This gentle, specialized, manual skill, involves the application of a quick, low-force maneuver that is directed to a specific area of a joint at a specific angle. A certified animal chiropractor understands these joint angles intimately and can best asses if an animal can benefit from chiropractic care, and, is the only professional who is qualified to adjust your pet.”
But no evidence!
By now I was desperate. My last hope was the ‘American Veterinary Chiropractic Association’. All I found there, however, was this: the “American Veterinary Chiropractic Association (AVCA) is a professional membership group promoting animal chiropractic to professionals and the public, and acting as the certifying agency for doctors who have undergone post-graduate animal chiropractic training.”
Not a jot of evidence!
The assumption that animal chiropractic is effective seems to rely on the evidence from human studies…
… and we all know how solid that body of evidence is!
My conclusion from all this: chiropractors treating animals and those treating humans have one important characteristic in common.
THEY HAPPILY PROMOTE BOGUS TREATMENTS.
How often have I pointed out that most studies of chiropractic (and other alternative therapies) are overtly unethical because they fail to report adverse events? And if you think this is merely my opinion, you are mistaken. This new analysis by a team of chiropractors aimed to describe the extent of adverse events reporting in published RCTs of Spinal Manipulative Therapy (SMT), and to determine whether the quality of reporting has improved since publication of the 2010 Consolidated Standards Of Reporting Trials (CONSORT) statement.
The Physiotherapy Evidence Database and the Cochrane Central Register of Controlled Trials were searched for RCTs involving SMT. Domains of interest included classifications of adverse events, completeness of adverse events reporting, nomenclature used to describe the events, methodological quality of the study, and details of the publishing journal. Data were analysed using descriptive statistics. Frequencies and proportions of trials reporting on each of the specified domains above were calculated. Differences in proportions between pre- and post-CONSORT trials were calculated with 95% confidence intervals using standard methods, and statistical comparisons were analysed using tests for equality of proportions with continuity correction.
Of 7,398 records identified in the electronic searches, 368 articles were eligible for inclusion in this review. Adverse events were reported in 140 (38.0%) articles. There was a significant increase in the reporting of adverse events post-CONSORT (p=.001). There were two major adverse events reported (0.3%). Only 22 articles (15.7%) reported on adverse events in the abstract. There were no differences in reporting of adverse events post-CONSORT for any of the chosen parameters.
The authors concluded that although there has been an increase in reporting adverse events since the introduction of the 2010 CONSORT guidelines, the current level should be seen as inadequate and unacceptable. We recommend that authors adhere to the CONSORT statement when reporting adverse events associated with RCTs that involve SMT.
We conducted a very similar analysis back in 2012. Specifically, we evaluated all 60 RCTs of chiropractic SMT published between 2000 and 2011 and found that 29 of them did not mention adverse effects at all. Sixteen RCTs reported that no adverse effects had occurred (which I find hard to believe since reliable data show that about 50% of patients experience adverse effects after consulting a chiropractor). Complete information on incidence, severity, duration, frequency and method of reporting of adverse effects was included in only one RCT. Conflicts of interests were not mentioned by the majority of authors. Our conclusion was that adverse effects are poorly reported in recent RCTs of chiropractic manipulations.
The new paper suggests that the situation has improved a little, yet it is still wholly unacceptable. To conduct a clinical trial and fail to mention adverse effects is not, as the authors of the new article suggest, against current guidelines; it is a clear and flagrant violation of medical ethics. I blame the authors of such papers, the reviewers and the journal editors for behaving dishonourably and urge them to get their act together.
The effects of such non-reporting are obvious: anyone looking at the evidence (for instance via systematic reviews) will get a false-positive impression of the safety of SMT. Consequently, chiropractors are able to claim that very few adverse effects have been reported in the literature, therefore our hallmark therapy SMT is demonstrably safe. Those who claim otherwise are quite simply alarmist.
A recent post discussed a ‘STATE OF THE ART REVIEW’ from the BMJ. When I wrote it, I did not know that there was more to come. It seems that the BMJ is planning an entire series on the state of the art of BS! The new paper certainly looks like it:
Headaches, including primary headaches such as migraine and tension-type headache, are a common clinical problem. Complementary and integrative medicine (CIM), formerly known as complementary and alternative medicine (CAM), uses evidence informed modalities to assist in the health and healing of patients. CIM commonly includes the use of nutrition, movement practices, manual therapy, traditional Chinese medicine, and mind-body strategies. This review summarizes the literature on the use of CIM for primary headache and is based on five meta-analyses, seven systematic reviews, and 34 randomized controlled trials (RCTs). The overall quality of the evidence for CIM in headache management is generally low and occasionally moderate. Available evidence suggests that traditional Chinese medicine including acupuncture, massage, yoga, biofeedback, and meditation have a positive effect on migraine and tension headaches. Spinal manipulation, chiropractic care, some supplements and botanicals, diet alteration, and hydrotherapy may also be beneficial in migraine headache. CIM has not been studied or it is not effective for cluster headache. Further research is needed to determine the most effective role for CIM in patients with headache.
My BS-detector struggled with the following statements:
- integrative medicine (CIM), formerly known as complementary and alternative medicine (CAM) – the fact that CIM is a nonsensical new term has been already mentioned in the previous post;
- evidence informed modalities – another new term! evidence-BASED would be too much? because it would require using standards that do not apply to CIM? double standards promoted by the BMJ, what next?
- CIM commonly includes the use of nutrition – yes, so does any healthcare or indeed life!
- the overall quality of the evidence for CIM in headache management is generally low and occasionally moderate – in this case, no conclusions should be drawn from it (see below);
- evidence suggests that traditional Chinese medicine including acupuncture, massage, yoga, biofeedback, and meditation have a positive effect on migraine and tension headaches – no, it doesn’t (see above)!
- further research is needed to determine the most effective role for CIM in patients with headache – this sentence does not even make the slightest sense to me; have the reviewers of this article been asleep?
And this is just the abstract!
The full text provides enough BS to fertilise many acres of farmland!
Moreover, the article is badly researched, cherry-picked, poorly constructed, devoid of critical input, and poorly written. Is there anything good about it? You tell me – I did not find much!
My BS-detector finally broke when we came to the conclusions:
The use of CIM therapies has the potential to empower patients and help them take an active role in their care. Many CIM modalities, including mind-body therapies, are both self selected and self administered after an education period. This, coupled with patients’ increased desire to incorporate integrative medicine, should prompt healthcare providers to consider and discuss its inclusion in the overall management strategy. Low to moderate quality evidence exists for the effectiveness of some CIM therapies in the management of primary headache. The evidence for and use of CIM is continuously changing so healthcare professionals should direct their patients to reliable and updated resources, such as NCCIH.
WHAT IS HAPPENING TO THE BMJ?
IT USED TO BE A GOOD JOURNAL!
The website of BMJ Clinical Evidence seems to be popular with fans of alternative medicine (FAMs). That sounds like good news: it’s an excellent source, and one can learn a lot about EBM when studying it. But there is a problem: FAMs don’t seem to really study it (alternatively they do not have the power of comprehension to understand the data); they merely pounce on this figure and cite it endlessly:
They interpret it to mean that only 11% of what conventional clinicians do is based on sound evidence. This is water on their mills, because now they feel able to claim:
THE MAJORITY OF WHAT CONVENTIONAL CLINICIANS DO IS NOT EVIDENCE-BASED. SO, WHY DO SO-CALLED RATIONAL THINKERS EXPECT ALTERNATIVE THERAPIES TO BE EVIDENCE-BASED? IF WE NEEDED PROOF THAT THEY ARE HYPOCRITES, HERE IT IS!!!
The question is: are these FAMs correct?
The answer is: no!
They are merely using a logical fallacy (tu quoque); what is worse, they use it based on misunderstanding the actual data summarised in the above figure.
Let’s look at this in a little more detail.
The first thing we need to understand the methodologies used by ‘Clinical Evidence’ and what the different categories in the graph mean. Here is the explanation:
So, arguably the top three categories amounting to 42% signify some evidential support (if we decided to be more rigorous and merely included the two top categories, we would still arrive at 35%). This is not great, but we must remember two things here:
- EBM is fairly new;
- lots of people are working hard to improve the evidence base of medicine so that, in future, these figures will be better (by contrast, in alternative medicine, no similar progress is noticeable).
The second thing that strikes me is that, in alternative medicine, these figures would surely be much, much worse. I am not aware of reliable estimates, but I guess that the percentages might be one dimension smaller.
The third thing to mention is that the figures do not cover the entire spectrum of treatments available today but are based on ~ 3000 selected therapies. It is unclear how they were chosen, presumably the choice is pragmatic and based on the information available. If an up-to date systematic review has been published and provided the necessary information, the therapy was included. This means that the figures include not just mainstream but also plenty of alternative treatments (to the best of my knowledge ‘Clinical Evidence’ makes no distinction between the two). It is thus nonsensical to claim that the data highlight the weakness of the evidence in conventional medicine. It is even possible that the figures would be better, if alternative treatments had been excluded (I estimate that around 2 000 systematic reviews of alternative therapies have been published [I am the author of ~400 of them!]).
The fourth and possibly the most important thing to mention is that the percentage figures in the graph are certainly NOT a reflection of what percentage of treatments used in routine care are based on good evidence. In conventional practice, clinicians would, of course, select where possible those treatments with the best evidence base, while leaving the less well documented ones aside. In other words, they will use the ones in the two top categories much more frequently than those from the other categories.
At this stage, I hear some FAMs say: how does he know that?
Because several studies have been published that investigated this issue in some detail. They have monitored what percentage of interventions used by conventional clinicians in their daily practice are based on good evidence. In 2004, I reviewed these studies; here is the crucial passage from my paper:
“The most conclusive answer comes from a UK survey by Gill et al who retrospectively reviewed 122 consecutive general practice consultations. They found that 81% of the prescribed treatments were based on evidence and 30% were based on randomised controlled trials (RCTs). A similar study conducted in a UK university hospital outpatient department of general medicine arrived at comparable figures; 82% of the interventions were based on evidence, 53% on RCTs. Other relevant data originate from abroad. In Sweden, 84% of internal medicine interventions were based on evidence and 50% on RCTs. In Spain these percentages were 55 and 38%, respectively. Imrie and Ramey pooled a total of 15 studies across all medical disciplines, and found that, on average, 76% of medical treatments are supported by some form of compelling evidence — the lowest was that mentioned above (55%),6 and the highest (97%) was achieved in anaesthesia in Britain. Collectively these data suggest that, in terms of evidence-base, general practice is much better than its reputation.”
My conclusions from all this:
FAMs should study the BMJ Clinical Evidence more thoroughly. If they did, they might comprehend that the claims they tend to make about the data shown there are, in fact, bogus. In addition, they might even learn a thing or two about EBM which might eventually improve the quality of the debate.
The new guidelines by the American College of Physicians entitled ‘Noninvasive Treatments for Acute, Subacute, and Chronic Low Back Pain: A Clinical Practice Guideline From the American College of Physicians’ have already been the subject of the previous post. Today, I want to have a closer look at a small section of these guidelines which, I think, is crucial. It is entitled ‘HARMS OF NONPHARMACOLOGIC THERAPIES’. I have taken the liberty of copying it below:
“Evidence on adverse events from the included RCTs and systematic reviews was limited, and the quality of evidence for all available harms data is low. Harms were poorly reported (if they were reported at all) for most of the interventions.
Low-quality evidence showed no reported harms or serious adverse events associated with tai chi, psychological interventions, multidisciplinary rehabilitation, ultrasound, acupuncture, lumbar support, or traction (9,95,150,170–174). Low-quality evidence showed that when harms were reported for exercise, they were often related to muscle soreness and increased pain, and no serious harms were reported. All reported harms associated with yoga were mild to moderate (119). Low-quality evidence showed that none of the RCTs reported any serious adverse events with massage, although 2 RCTs reported soreness during or after massage therapy (175,176). Adverse events associated with spinal manipulation included muscle soreness or transient increases in pain (134). There were few adverse events reported and no clear differences between MCE and controls. Transcutaneous electrical nerve stimulation was associated with an increased risk for skin site reaction but not serious adverse events (177). Two RCTs (178,179) showed an increased risk for skin flushing with heat compared with no heat or placebo, and no serious adverse events were reported. There were no data on cold therapy. Evidence was insufficient to determine harms of electrical muscle stimulation, LLLT, percutaneous electrical nerve stimulation, interferential therapy, short-wave diathermy, and taping.”
The first thing that strikes me is the brevity of the section. Surely, guidelines of this nature must include a full discussion of the risks of the treatments in question!
The second thing that is noteworthy is the fact that the authors confirm the fact I have been banging on about for years: clinical trials of alternative therapies far too often fail to mention adverse effects. I have often pointed out that the failure to report adverse effects in clinical trials is an unacceptable violation of medical ethics. By contrast, the guideline authors seem not to feel strongly about this omission.
The third thing that is noteworthy is that the guidelines evaluate the harms of the treatments purely on the basis of the adverse effects reported in the clinical trials and systematic reviews included in their efficacy assessments. This is nonsensical for at least two reasons:
- The guideline authors themselves are aware that the trials very often fail to mention adverse effects.
- For any assessment of harm, one has to go far beyond the evidence of clinical trials, because trials tend to be too small to pick up rare adverse effects, and because they are always conducted under optimally controlled conditions where adverse effects are less likely to occur than in real life.
Together, these features of the assessment of harms explain why the guideline authors arrive at conclusions which are oddly misguided; I would even feel that they resemble a white-wash. Here are two of the most overt misjudgements:
- no harms associated with acupuncture,
- only trivial harm associated with spinal manipulations.
The best evidence we have today shows that acupuncture leads to mild adverse effects in about 10% of all cases and is also associated with very severe complications (e.g. pneumothorax, cardiac tamponade, infections, deaths) in an unknown number of patients. More details can be found for instance here, here, here and here.
And the best evidence available shows that spinal manipulation leads to moderately severe adverse effects in ~50% of all cases. In addition, we know of hundreds of cases of very severe complications resulting in stroke, permanent neurological deficits or deaths. More details can be found for instance here, here, here and here.
In the introduction, I stated that this small section of the guidelines is crucial.
Why?
The reason is simple: any responsible therapeutic decision has to be based not just on the efficacy of the treatment in question but on its risk/benefit balance. The evidence shows that the risks of some alternative therapies can be considerable, a fact that is almost totally neglected in the guidelines. Therefore, the recommendations of the new guidelines by the American College of Physicians entitled ‘Noninvasive Treatments for Acute, Subacute, and Chronic Low Back Pain: A Clinical Practice Guideline From the American College of Physicians’ are in several aspects not entirely correct and need to be reconsidered.
