A new update of the current Cochrane review assessed the benefits and harms of spinal manipulative therapy (SMT) for the treatment of chronic low back pain. The authors included all randomised controlled trials (RCTs) examining the effect of spinal manipulation or mobilisation in adults (≥18 years) with chronic low back pain with or without referred pain. Studies that exclusively examined sciatica were excluded.
The effect of SMT was compared with recommended therapies, non-recommended therapies, sham (placebo) SMT, and SMT as an adjuvant therapy. Main outcomes were pain and back specific functional status, examined as mean differences and standardised mean differences (SMD), respectively. Outcomes were examined at 1, 6, and 12 months.
Forty-seven RCTs including a total of 9211 participants were identified. Most trials compared SMT with recommended therapies. In 16 RCTs, the therapists were chiropractors, in 14 they were physiotherapists, and in 5 they were osteopaths. They used high velocity manipulations in 18 RCTs, low velocity manipulations in 12 studies and a combination of the two in 20 trials.
Moderate quality evidence suggested that SMT has similar effects to other recommended therapies for short term pain relief and a small, clinically better improvement in function. High quality evidence suggested that, compared with non-recommended therapies, SMT results in small, not clinically better effects for short term pain relief and small to moderate clinically better improvement in function.
In general, these results were similar for the intermediate and long term outcomes as were the effects of SMT as an adjuvant therapy.
Low quality evidence suggested that SMT does not result in a statistically better effect than sham SMT at one month. Additionally, very low quality evidence suggested that SMT does not result in a statistically better effect than sham SMT at six and 12 months. Low quality evidence suggested that SMT results in a moderate to strong statistically significant and clinically better effect than sham SMT at one month. Additionally, very low quality evidence suggested that SMT does not result in a statistically significant better effect than sham SMT at six and 12 months.
(Mean difference in reduction of pain at 1, 3, 6, and 12 months (0-100; 0=no pain, 100 maximum pain) for spinal manipulative therapy (SMT) versus recommended therapies in review of the effects of SMT for chronic low back pain. Pooled mean differences calculated by DerSimonian-Laird random effects model.)
About half of the studies examined adverse and serious adverse events, but in most of these it was unclear how and whether these events were registered systematically. Most of the observed adverse events were musculoskeletal related, transient in nature, and of mild to moderate severity. One study with a low risk of selection bias and powered to examine risk (n=183) found no increased risk of an adverse event or duration of the event compared with sham SMT. In one study, the Data Safety Monitoring Board judged one serious adverse event to be possibly related to SMT.
The authors concluded that SMT produces similar effects to recommended therapies for chronic low back pain, whereas SMT seems to be better than non-recommended interventions for improvement in function in the short term. Clinicians should inform their patients of the potential risks of adverse events associated with SMT.
This paper is currently being celebrated (mostly) by chiropractors who think that it vindicates their treatments as being both effective and safe. However, I am not sure that this is entirely true. Here are a few reasons for my scepticism:
- SMT is as good as other recommended treatments for back problems – this may be so but, as no good treatment for back pain has yet been found, this really means is that SMT is as BAD as other recommended therapies.
- If we have a handful of equally good/bad treatments, it stand to reason that we must use other criteria to identify the one that is best suited – criteria like safety and cost. If we do that, it becomes very clear that SMT cannot be named as the treatment of choice.
- Less than half the RCTs reported adverse effects. This means that these studies were violating ethical standards of publication. I do not see how we can trust such deeply flawed trials.
- Any adverse effects of SMT were minor, restricted to the short term and mainly centred on musculoskeletal effects such as soreness and stiffness – this is how some naïve chiro-promoters already comment on the findings of this review. In view of the fact that more than half the studies ‘forgot’ to report adverse events and that two serious adverse events did occur, this is a misleading and potentially dangerous statement and a good example how, in the world of chiropractic, research is often mistaken for marketing.
- Less than half of the studies (45% (n=21/47)) used both an adequate sequence generation and an adequate allocation procedure.
- Only 5 studies (10% (n=5/47)) attempted to blind patients to the assigned intervention by providing a sham treatment, while in one study it was unclear.
- Only about half of the studies (57% (n=27/47)) provided an adequate overview of withdrawals or drop-outs and kept these to a minimum.
- Crucially, this review produced no good evidence to show that SMT has effects beyond placebo. This means the modest effects emerging from some trials can be explained by being due to placebo.
- The lead author of this review (SMR), a chiropractor, does not seem to be free of important conflicts of interest: SMR received personal grants from the European Chiropractors’ Union (ECU), the European Centre for Chiropractic Research Excellence (ECCRE), the Belgian Chiropractic Association (BVC) and the Netherlands Chiropractic Association (NCA) for his position at the Vrije Universiteit Amsterdam. He also received funding for a research project on chiropractic care for the elderly from the European Centre for Chiropractic Research and Excellence (ECCRE).
- The second author (AdeZ) who also is a chiropractor received a grant from the European Chiropractors’ Union (ECU), for an independent study on the effects of SMT.
After carefully considering the new review, my conclusion is the same as stated often before: SMT is not supported by convincing evidence for back (or other) problems and does not qualify as the treatment of choice.
Osteopathy is a tricky subject:
- Osteopathic manipulations/mobilisations are advocated mainly for spinal complaints.
- Yet many osteopaths use them also for a myriad of non-spinal conditions.
- Osteopathy comprises two entirely different professions; in the US, osteopaths are very similar to medically trained doctors, and many hardly ever employ osteopathic manual techniques; outside the US, osteopaths are alternative practitioners who use mainly osteopathic techniques and believe in the obsolete gospel of their guru Andrew Taylor Still (this post relates to the latter type of osteopathy).
- The question whether osteopathic manual therapies are effective is still open – even for the indication that osteopaths treat most, spinal complaints.
- Like chiropractors, osteopaths now insist that osteopathy is not a treatment but a profession; the transparent reason for this argument is to gain more wriggle-room when faced with negative evidence regarding they hallmark treatment of osteopathic manipulation/mobilisation.
A new paper authored by osteopaths is an attempt to shed more light on the effectiveness of osteopathy. The aim of this systematic review evaluated the impact of osteopathic care for spinal complaints. Only randomized controlled trials conducted in high-income Western countries were considered. Two authors independently screened the titles and abstracts. Primary outcomes included ‘pain’ and ‘functional status’, while secondary outcomes included ‘medication use’ and ‘health status’.
Nineteen studies were included and qualitatively synthesized. Nine studies were from the US, followed by Germany with 7 studies. The majority of studies (n = 13) focused on low back pain.
In general, mixed findings related to the impact of osteopathic care on primary and secondary outcomes were observed. For the primary outcomes, a clear distinction between US and European studies was found, where the latter RCTs reported positive results more frequently. Studies were characterized by substantial methodological differences in sample sizes, number of treatments, control groups, and follow-up.
The authors concluded that “the findings of the current literature review suggested that osteopathic care may improve pain and functional status in patients suffering from spinal complaints. A clear distinction was observed between studies conducted in the US and those in Europe, in favor of the latter. Today, no clear conclusions of the impact of osteopathic care for spinal complaints can be drawn. Further studies with larger study samples also assessing the long-term impact of osteopathic care for spinal complaints are required to further strengthen the body of evidence.”
Some of the most obvious weaknesses of this review include the following:
- In none of the studies employed blinding of patients, care provider or outcome assessor occurred, or it was unclear. Blinding of outcome assessors is easily implemented and should be standard in any RCT.
- In three studies, the study groups differed to some extent at baseline indicating that randomisation was not successful..
- Five studies were derived from the ‘grey literature’ and were therefore not peer-reviewed.
- One study (the UK BEAM trial) employed not just osteopaths but also chiropractors and physiotherapists for administering the spinal manipulations. It is therefore hardly an adequate test of osteopathy.
- The study was funded by an unrestricted grant from the GNRPO, the umbrella organization of the ‘Belgian Professional Associations for Osteopaths’.
Considering this last point, the authors’ honesty in admitting that no clear conclusions of the impact of osteopathic care for spinal complaints can be drawn is remarkable and deserves praise.
Considering that the evidence for osteopathy is even far worse for non-spinal conditions (numerous trials exist for all sorts of other conditions, but they tend to be flimsy and usually lack independent replications), it is fair to conclude that osteopathy is NOT an evidence-based therapy.
Collagen is a fibrillar protein of the conjunctive and connective tissues in the human body, essentially skin, joints, and bones. Due to its abundance in our bodies, its strength and its relation with skin aging, collagen has gained great interest as an oral dietary supplement as well as an ingredient in cosmetics. Collagen fibres get damaged with the pass of time, losing thickness and strength which has been linked to skin aging phenomena. Collagen can be obtained from natural sources such as plants and animals or by recombinant protein production systems. Because of its increased use, the collagen market is worth billions. The question therefore arises: is it worth it?
This 2019 systematic review assessed all available randomized-controlled trials using collagen supplementation for treatment efficacy regarding skin quality, anti-aging benefits, and potential application in medical dermatology. Eleven studies with a total of 805 patients were included. Eight studies used collagen hydrolysate, 2.5g/d to 10g/d, for 8 to 24 weeks, for the treatment of pressure ulcers, xerosis, skin aging, and cellulite. Two studies used collagen tripeptide, 3g/d for 4 to 12 weeks, with notable improvement in skin elasticity and hydration. Lastly, one study using collagen dipeptide suggested anti-aging efficacy is proportionate to collagen dipeptide content.
The authors concluded that preliminary results are promising for the short and long-term use of oral collagen supplements for wound healing and skin aging. Oral collagen supplements also increase skin elasticity, hydration, and dermal collagen density. Collagen supplementation is generally safe with no reported adverse events. Further studies are needed to elucidate medical use in skin barrier diseases such as atopic dermatitis and to determine optimal dosing regimens.
These conclusions are similar to those of a similar but smaller review of 2015 which concluded that the oral supplementation with collagen peptides is efficacious to improve hallmarks of skin aging.
And what about the many other claims that are currently being made for oral collagen?
A 2006 review of collagen for osteoarthritis concluded that a growing body of evidence provides a rationale for the use of collagen hydrolysate for patients with OA. It is hoped that ongoing and future research will clarify how collagen hydrolysate provides its clinical effects and determine which populations are most appropriate for treatment with this supplement. For other indication, the evidence seems less conclusive.
So, what should we make of this collective evidence. My interpretation is that, of course, there are caveats. For instance, most studies are small and not as rigorous as one would hope. But the existing evidence is nevertheless intriguing (and much more compelling than that for most other supplements). Moreover, there seem to be very few adverse effects with oral usage (don’t inject the stuff for cosmetic purposes, as often recommended!). Therefore, I feel that collagen might be one of the few dietary supplements worth keeping an eye on.
So-called alternative medicine (SCAM) for animals is popular. A recent survey suggested that 76% of US dog and cat owners use some form of SCAM. Another survey showed that about one quarter of all US veterinary medical schools run educational programs in SCAM. Amazon currently offers more that 4000 books on the subject.
The range of SCAMs advocated for use in animals is huge and similar to that promoted for use in humans; the most commonly employed practices seem to include acupuncture, chiropractic, energy healing, homeopathy (as discussed in the previous post) and dietary supplements. In this article, I will briefly discuss the remaining 4 categories.
Acupuncture is the insertion of needles at acupuncture points on the skin for therapeutic purposes. Many acupuncturists claim that, because it is over 2 000 years old, acupuncture has ‘stood the test of time’ and its long history proves acupuncture’s efficacy and safety. However, a long history of usage proves very little and might even just demonstrate that acupuncture is based on the pre-scientific myths that dominated our ancient past.
There are many different forms of acupuncture. Acupuncture points can allegedly be stimulated not just by inserting needles (the most common way) but also with heat, electrical currents, ultrasound, pressure, bee-stings, injections, light, colour, etc. Then there is body acupuncture, ear acupuncture and even tongue acupuncture. Traditional Chinese acupuncture is based on the Taoist philosophy of the balance between two life-forces, ‘yin and yang’. In contrast, medical acupuncturists tend to cite neurophysiological theories as to how acupuncture might work; even though some of these may appear plausible, they nevertheless are mere theories and constitute no proof for acupuncture’s validity.
The therapeutic claims made for acupuncture are legion. According to the traditional view, acupuncture is useful for virtually every condition. According to ‘Western’ acupuncturists, acupuncture is effective mostly for chronic pain. Acupuncture has, for instance, been used to improve mobility in dogs with musculoskeletal pain, to relieve pain associated with cervical neurological disease in dogs, for respiratory resuscitation of new-born kittens, and for treatment of certain immune-mediated disorders in small animals.
While the use of acupuncture seems to gain popularity, the evidence fails to support this. Our systematic review of acupuncture (to the best of my knowledge the only one on the subject) in animals included 14 randomized controlled trials and 17 non-randomized controlled studies. The methodologic quality of these trials was variable but, on average, it was low. For cutaneous pain and diarrhoea, encouraging evidence emerged that might warrant further investigation. Single studies reported some positive inter-group differences for spinal cord injury, Cushing’s syndrome, lung function, hepatitis, and rumen acidosis. However, these trials require independent replication. We concluded that, overall, there is no compelling evidence to recommend or reject acupuncture for any condition in domestic animals. Some encouraging data do exist that warrant further investigation in independent rigorous trials.
Serious complications of acupuncture are on record and have repeatedly been discussed on this blog: acupuncture needles can, for instance, injure vital organs like the lungs or the heart, and they can introduce infections into the body, e. g. hepatitis. About 100 human fatalities after acupuncture have been reported in the medical literature – a figure which, due to lack of a monitoring system, may disclose just the tip of an iceberg. Information on adverse effects of acupuncture in animals is currently not available.
Given that there is no good evidence that acupuncture works in animals, the risk/benefit balance of acupuncture cannot be positive.
Chiropractic was created by D D Palmer (1845-1913), an American magnetic healer who, in 1895, manipulated the neck of a deaf janitor, allegedly curing his deafness. Chiropractic was initially promoted as a cure-all by Palmer who claimed that 95% of diseases were due to subluxations of spinal joints. Subluxations became the cornerstone of chiropractic ‘philosophy’, and chiropractors who adhere to Palmer’s gospel diagnose subluxation in nearly 100% of the population – even in individuals who are completely disease and symptom-free. Yet subluxations, as understood by chiropractors, do not exist.
There is no good evidence that chiropractic spinal manipulation might be effective for animals. A review of the evidence for different forms of manual therapies for managing acute or chronic pain syndromes in horses concluded that further research is needed to assess the efficacy of specific manual therapy techniques and their contribution to multimodal protocols for managing specific somatic pain conditions in horses. For other animal species or other health conditions, the evidence is even less convincing.
In humans, spinal manipulation is associated with serious complications (regularly discussed in previous posts), usually caused by neck manipulation damaging the vertebral artery resulting in a stroke and even death. Several hundred such cases have been documented in the medical literature – but, as there is no system in place to monitor such events, the true figure is almost certainly much larger. To the best of my knowledge, similar events have not been reported in animals.
Since there is no good evidence that chiropractic spinal manipulations work in animals, the risk/benefit balance of chiropractic fails to be positive.
Energy healing is an umbrella term for a range of paranormal healing practices, e. g. Reiki, Therapeutic Touch, Johrei healing, faith healing. Their common denominator is the belief in an ‘energy’ that can be used for therapeutic purposes. Forms of energy healing have existed in many ancient cultures. The ‘New Age’ movement has brought about a revival of these ideas, and today ‘energy’ healing systems are amongst the most popular alternative therapies in many countries.
Energy healing relies on the esoteric belief in some form of ‘energy’ which refers to some life force such as chi in Traditional Chinese Medicine, or prana in Ayurvedic medicine. Some proponents employ terminology from quantum physics and other ‘cutting-edge’ science to give their treatments a scientific flair which, upon closer scrutiny, turns out to be little more than a veneer of pseudo-science.
Considering its implausibility, energy healing has attracted a surprisingly high level of research activity in the form of clinical trials on human patients. Generally speaking, the methodologically best trials of energy healing fail to demonstrate that it generates effects beyond placebo. There are few studies of energy healing in animals, and those that are available are frequently less than rigorous (see for instance here and here). Overall, there is no good evidence to suggest that ‘energy’ healing is effective in animals.
Even though energy healing is per se harmless, it can do untold damage, not least because it can lead to neglect of effective treatments and it undermines rationality in our societies. Its risk/benefit balance therefore fails to be positive.
Dietary supplements for veterinary use form a category of remedies that, in most countries, is a regulatory grey area. Supplements can contain all sorts of ingredients, from minerals and vitamins to plants and synthetic substances. Therefore, generalisations across all types of supplements are impossible. The therapeutic claims that are being made for supplements are numerous and often unsubstantiated. Although they are usually promoted as natural and safe, dietary supplements do not have necessarily either of these qualities. For example, in the following situations, supplements can be harmful:
- Combining one supplement with another supplement or with prescribed medicines
- Substituting supplements for prescription medicines
- Overdosing some supplements, such as vitamin A, vitamin D, or iron
Examples of currently most popular supplements for use in animals include chondroitin, glucosamine, probiotics, vitamins, minerals, lutein, L-carnitine, taurine, amino acids, enzymes, St John’s wort, evening primrose oil, garlic and many other herbal remedies. For many supplements taken orally, the bioavailability might be low. There is a paucity of studies testing the efficacy of dietary supplements in animals. Three recent exceptions (all of which require independent replication) are:
- A trial showing that the dietary supplementation with Maca increased sperm production in stallions.
- A study demonstrating that curcumin supplementation appeared to reduce arthritis pain in dogs.
- An investigation suggesting that royal jelly supplementation can improve the egg quality of hens.
Dietary supplements are promoted as being free of direct risks. On closer inspection, this notion turns out to be little more than an advertising slogan. As discussed repeatedly on this blog, some supplements contain toxic materials, contaminants or adulterants and thus have the potential to do harm. A report rightly concluded that many challenges stand in the way of determining whether or not animal dietary supplements are safe and at what dosage. Supplements considered safe in humans and other cross-species are not always safe in horses, dogs, and cats. An adverse event reporting system is badly needed. And finally, regulations dealing with animal dietary supplements are in disarray. Clear and precise regulations are needed to allow only safe dietary supplements on the market.
It is impossible to generalise about the risk/benefit balance of dietary supplements; however, caution is advisable.
SCAM for animals is an important subject, not least because of the current popularity of many treatments that fall under this umbrella. For most therapies, the evidence is woefully incomplete. This means that most SCAMs are unproven. Arguably, it is unethical to use unproven medicines in routine veterinary care.
I was invited several months ago to write this article for VETERINARY RECORD. It was submitted to peer review and subsequently I withdrew my submission. The above post is a slightly revised version of the original (in which I used the term ‘alternative medicine’ rather than ‘SCAM’) which also included a section on homeopathy (see my previous post). The reason for the decision to withdraw this article was the following comment by the managing editor of VETERINARY RECORD: A good number of vets use these therapies and a more balanced view that still sets out their efficacy (or otherwise) would be more useful for the readership.
Ever since Samuel Hahnemann, the German physician who invented homeopathy, gave a lecture on the subject in the mid-1810s, homeopathy has been used for treating animals. Initially, veterinary medical schools tended to reject homoeopathy as implausible, and the number of veterinary homeopaths remained small. In the 1920ies, however, veterinary homoeopathy was revived in Germany, and in 1936, members of the “Studiengemeinschaft für tierärztliche Homöopathie” (Study Group for Veterinary Homoeopathy) started to investigate homeopathy systematically.
Today, veterinary homeopathy is popular not least because of the general boom in so-called alternative medicine (SCAM). Prince Charles is just one of many prominent advocates who claims to treat animals with homeopathy. In many countries, veterinary homeopaths have their own professional organisations, while elsewhere veterinarians are banned from practicing homeopathy. In the UK, only veterinarians are currently allowed to use homeopathy on animals (but ironically, anyone regardless of background can use it on human patients).
Considering the implausibility of its assumptions, it seems unlikely that homeopathic remedies can be anything other than placebos. Yet homeopaths and their followers regularly produce clinical trials that seem to suggest efficacy. Today, there are about 500 controlled clinical trials of homeopathy (mostly on humans), and it is no surprise that, purely by chance, some of them show positive results. To avoid being misled by random findings, cherry-picking, or flawed science, we ought to critically evaluate the totality of the available evidence. In other words, we should rely not on single studies but on systematic reviews of all reliable trials.
A 2015 systematic review by ardent homeopaths tested the hypothesis that the outcome of veterinary homeopathic treatments is distinguishable from placebos. A total of 15 trials could be included, but only two comprised reliable evidence without overt vested interest. The authors concluded that there is “very limited evidence that clinical intervention in animals using homeopathic medicines is distinguishable from corresponding intervention using placebos.”
A more recent systematic review compared the efficacy of homeopathy to that of antibiotics in cattle, pigs and poultry. A total number of 52 trials were included of which 28 were in favour of homeopathy and 22 showed no effect. No study had been independently replicated. The authors concluded that “the use of homeopathy cannot claim to have sufficient prognostic validity where efficacy is concerned.”
Discussing this somewhat unclear and contradictory findings of trials of homeopathy for animals, Lee et al concluded that “…it is overwhelmingly likely that small effects observed in the RCTs and systematic reviews are the result of residual bias in the trials.” To this, I might add that ‘publication bias’, i. e. the phenomenon that negative trials often remain unpublished, might be the reason why systematic reviews of homeopathy are never entirely negative.
In recent years, several scientific bodies have assessed the evidence on homeopathy and published statements about it. Here are the key passages from some of these ‘official verdicts’:
“The principles of homeopathy contradict known chemical, physical and biological laws and persuasive scientific trials proving its effectiveness are not available”
Russian Academy of Sciences, Russia
“Homeopathy should not be used to treat health conditions that are chronic, serious, or could become serious. People who choose homeopathy may put their health at risk if they reject or delay treatments for which there is good evidence for safety and effectiveness.”
National Health and Medical Research Council, Australia
“These products are not supported by scientific evidence.”
Health Canada, Canada
“Homeopathic remedies don’t meet the criteria of evidence-based medicine.”
Hungarian Academy of Sciences, Hungary
“The incorporation of anthroposophical and homeopathic products in the Swedish directive on medicinal products would run counter to several of the fundamental principles regarding medicinal products and evidence-based medicine.”
Swedish Academy of Sciences, Sweden
“There is little evidence to support homeopathy as an effective treatment for any specific condition”
National Centre for Complementary and Integrative Health, USA
“There is no good-quality evidence that homeopathy is effective as a treatment for any health condition”
“Homeopathic remedies perform no better than placebos, and the principles on which homeopathy is based are “scientifically implausible””
House of Commons Science and Technology Committee, UK
“Homeopathy has not definitively proven its efficacy in any specific indication or clinical situation.”
Ministry of Health, Spain
“… homeopathy should be treated as one of the unscientific methods of the so called ‘alternative medicine’, which proposes worthless products without scientifically proven efficacy.”
National Medical Council, Poland
“… there is no valid empirical proof of the efficacy of homeopathy beyond the placebo effect.”
Federaal Kenniscentrum voor de Gezondheidszorg, Belgium
As they are usually far too dilute to contain anything, homeopathic remedies are generally harmless, provided they are produced according to good manufacturing practice (which is not always the case). Unfortunately, however, this harmlessness does not necessarily apply to homeopathy in general. When employed to replace an effective therapy, even the most innocent but ineffective treatment can become life-threatening. Since homeopaths recommend their remedies for even the most serious conditions, this is by no means a theoretical consideration. I have therefore often stated that HOMEOPATHICS MIGHT BE HARMLESS, BUT HOMEOPATHS CERTAINLY ARE NOT.
It follows that an independent risk/beneﬁt analysis of homeopathy fails to arrive at a positive conclusion. In other words, homeopathy has not been shown to generate more good than harm. In turn, this means that homeopathy has no place in veterinary (or human) evidence-based medicine.
Did we not have a flurry of systematic reviews of homeopathy in recent months?
And were they not a great disappointment to homeopaths and their patients?
Just as we thought that this is more than enough evidence to show that homeopathy is not effective, here comes another one.
This new review evaluated RCTs of non-individualised homeopathic treatment (NIHT) in which the control group received treatments other than placebo (OTP). Specifically, its aim was to determine the comparative effectiveness of NIHT on health-related outcomes for any given condition.
For each eligible trial, published in the peer-reviewed literature up to the end of 2016, the authors assessed its risk of bias (internal validity) using the seven-domain Cochrane tool, and its relative pragmatic or explanatory attitude (external validity) using the 10-domain PRECIS tool. The researchers grouped RCTs by whether these examined homeopathy as an alternative treatment (study design 1a), adjunctively with another intervention (design 1b), or compared with no intervention (design 2). RCTs were sub-categorised as superiority trials or equivalence/non-inferiority trials. For each RCT, a single ‘main outcome measure’ was selected to use in meta-analysis.
Seventeen RCTs, representing 15 different medical conditions, were eligible for inclusion. Three of the trials were more pragmatic than explanatory, two were more explanatory than pragmatic, and 12 were equally pragmatic and explanatory. Fourteen trials were rated ‘high risk of bias’ overall; the other three trials were rated ‘uncertain risk of bias’ overall. Ten trials had data that were extractable for meta-analysis. Significant heterogeneity undermined the planned meta-analyses or their meaningful interpretation. For the three equivalence or non-inferiority trials with extractable data, the small, non-significant, pooled effect size was consistent with a conclusion that NIHT did not differ from treatment by a comparator (Ginkgo biloba or betahistine) for vertigo or (cromolyn sodium) for seasonal allergic rhinitis.
The authors concluded that the current data preclude a decisive conclusion about the comparative effectiveness of NIHT. Generalisability of findings is restricted by the limited external validity identified overall. The highest intrinsic quality was observed in the equivalence and non-inferiority trials of NIHT.
I do admire the authors’ tenacity in meta-analysing homeopathy trials and empathise with their sadness of the multitude of negative results they thus have to publish. However, I do disagree with their conclusions. In my view, at least two firm conclusions ARE possible:
- This dataset confirms yet again that the methodological quality of homeopathy trials is lousy.
- The totality of the trial evidence analysed here fails to show that non-individualised homeopathy is effective.
In case you wonder why the authors are not more outspoken about their own findings, perhaps you need to read their statement of conflicts of interest:
Authors RTM, YYYF, PV and AKLT are (or were) associated with a homeopathy organisation whose significant aim is to clarify and extend an evidence base in homeopathy. RTM holds an independent research consultancy contract with the Deutsche Homöopathie-Union, Karlsruhe, Germany. YYYF and AKLT belong to Living Homeopathy Ltd., which has contributed funding to some (but not this current) HRI project work. RTM and PV have no other relationships or activities that could appear to have influenced the submitted work. JRTD had no support from any organisation for the submitted work; in the last 3 years, and for activities outside the submitted study, he received personal fees, royalties or out-of-pocket expenses for advisory work, invitational lectures, use of rating scales, published book chapters or committee membership; he receives royalties from Springer Publishing Company for his book, A Century of Homeopaths: Their Influence on Medicine and Health. JTRD has no other relationships or activities that could appear to have influenced the submitted study.
If one had wanted to add insult to injury, one could have added that, if, despite such conflicts of interest, the overall result of this new review turned out to be not positive, the evidence must be truly negative.
Simply put, in the realm of SCAM, we seem to have two types of people:
- those who don’t care a hoot about evidence;
- those who try their best to follow the evidence.
The first group is replete with SCAM enthusiasts who make their decisions based purely on habit, emotion, intuition etc. They are beyond my reach, I fear. It is almost exclusively the second group for whom I write this blog.
And that could be relatively easy, if the evidence were always accessible, understandable, straight forward, conclusive and convincing. But sadly, in SCAM (as in most other areas of healthcare), the evidence is full of apparent and real contradictions. In this situation, it is often difficult even for experts to understand what is going on; for lay people this must be immeasurably more confusing. Yet, it is the lay consumers who often will take the decision to use or not use this or that SCAM. They therefore need our help.
What can consumers do when they are confronted with contradictory evidence?
How can they distinguish right from wrong?
- Some articles claim that homeopathy works – others say it is just a placebo therapy.
- Some experts claim that chiropractic is safe – others say it can do serious harm.
- Some articles claim that SCAM-practitioners are competent – others say this is not true.
- Some experts claim that SCAM is the future – others stress that it is obsolete.
What can a lay person with no or very little understanding of science do to see through this fog of contradictions?
Let me try to provide consumers with a step by step approach to get closer to the truth by asking a few incisive questions:
- WHERE DID YOU READ THE CLAIM? If it was in a newspaper, magazine, website, etc. take it with a pinch of salt (double the dose of salt, if it’s from the Daily Mail).
- CAN YOU RETRACE THE CLAIM TO A SCIENTIFIC PAPER? This might challenge you skills as a detective, but it is always well-worth finding the original source of a therapeutic claim in order to judge its credibility. If no good source can be found, I advise caution.
- IN WHICH MEDICAL JOURNAL WAS THE CLAIM PUBLISHED? Be aware of the fact that there are dozens of SCAM-journals that would publish virtually any rubbish.
- WHO ARE THE AUTHORS OF THE SCIENTIFIC PAPER? It might be difficult for a lay person to evaluate their credibility. But there might be certain pointers; for instance, authors affiliated to a university tend to be more credible than SCAM-practitioners who have no such affiliations or authors working for a lobby-group.
- WHAT SORT OF ARTICLE IS THE ORIGINAL SOURCE OF THE CLAIM? Is it a proper experimental study or a mere opinion piece? If possible, try to find a good-quality (perhaps even a Cochrane) review on the subject.
- ARE THERE OTHER RESEARCHERS WHO HAVE ARRIVED AT SIMILAR CONCLUSIONS? If the claim is based on just one solitary piece of research or opinion, it clearly weighs less than a consensus of experts.
- DO PUBLICATIONS EXIST THAT DISAGREE WITH THE CLAIM? Even if there are several scientific papers from different teams of researchers supporting the claim, it is important to find out whether the claim is shared by all experts in the field.
Eventually, you might get a good impression about the veracity of the claim. But sometimes you also might end up with a bunch of systematic reviews of which several support, while others reject the claim. And all of them could look similarly credible to your untrained eyes. Does that mean your attempt to find the truth of the matter has been frustrated?
In this case, you would probably consider the following options:
- You could do a simple ‘pea count’; this would tell you whether the majority of reviews is pro or contra the claim. However, this might be your worst bet for arriving at a sound conclusion. The quantity of the evidence usually is far less important than its quality.
- If you have no training to judge the quality of a review, you might just go with the most recent and up-to-date review. This, however, would also be fraught with problems, as you can, of course, not be sure that the most recent one is also the least biased assessment.
- Perhaps you can somehow get an impression about the respectability of the source. If, for instance, there is a recent Cochrane review, I advise to go with that one.
- Look up the profession of the authors of the review. The pope is unlikely to condemn Catholicism; likewise, you will find very few homeopaths who are critical of homeopathy, or chiropractors who are critical of chiropractic, etc. I know this is a very crude ‘last resort’ for replacing an authorative evaluation of the claim. But, if that’s all you have, it is better than nothing. Ask yourself who can normally be trusted more, the SCAM-practitioner or lobbyist who makes a living from the claim or an independent academic who has no such conflict of interest?
If all of this does not help you to decide whether a therapeutic claim is trustworthy or not, my advice has always been to reflect on this: IF IT SOUNDS TOO GOOD TO BE TRUE, IT PROBABLY IS.
Highly diluted homeopathic remedies are pure placebos! This is what the best evidence clearly shows. Ergo they cannot be shown in a rigorous study to have effects that differ from placebo. But now there is a study that seems to contradict this widely accepted conclusion.
Can someone please help me to understand what is going on?
In this double-blind, placebo-controlled RCT, 60 patients suffering from insomnia were treated either individualised homeopathy (IH) or placebo for 3 months. Patient-administered sleep diary and Insomnia Severity Index (ISI) were used the primary and secondary outcomes respectively, measured at baseline, and after 3 months.
Five patients dropped out (verum:2,control:3).Intention to treat sample (n=60) was analysed. Trial arms were comparable at baseline. In the verum group, except sleep diary item 3 (P= 0.371), rest of the outcomes improved significantly (all P < 0.01). In the control group, there were significant improvements in diary item 6 and ISI score (P < 0.01) and just significant improvement in item 5 (P= 0.018). Group differences were significant for items 4, 5 and 6(P < 0.01) and just significant (P= 0.014) for ISI score with moderate to large effect sizes; but non-significant (P > 0.01) for rest of the outcomes.
The authors concluded that in this double-blind, randomized, prospective, placebo-controlled, two parallel arms clinical trial conducted on 60 patients suffering from insomnia, there was statistically significant difference measured in sleep efficiency, total sleep time, time in bed, and ISI score in favour of homeopathy over placebo with moderate to large effect sizes. Group differences were non-significant for rest of the outcomes(i.e. latency to fall asleep, minutes awake in middle of night and minutes awake too early). Individualized homeopathy seemed to produce significantly better effect than placebo. Independent replications and adequately powered trials with enhanced methodological rigor are warranted.
I have studied this article in some detail; its methodology is nicely and fully described in the original paper. To my amazement, I cannot find a flaw that is worth mentioning. Sure, the sample was small, the treatment time short, the outcome measure subjective, the paper comes from a dubious journal, the authors have a clear conflict of interest, even though they deny it – but none of these limitations has the potential to conclusively explain the positive result.
In view of what I stated above and considering what the clinical evidence so far tells us, this is most puzzling.
A 2010 systematic review authored by proponents of homeopathy included 4 RCTs comparing homeopathic medicines to placebo. All involved small patient numbers and were of low methodological quality. None demonstrated a statistically significant difference in outcomes between groups.
My own 2011 not Medline-listed review (Focus on Alternative and Complementary Therapies Volume 16(3) September 2011 195–199) included several additional studies. Here is its abstract:
The aim of this review was the critical evaluation of evidence for the effectiveness of homeopathy for insomnia and sleep-related disorders. A search of MEDLINE, AMED, CINAHL, EMBASE and Cochrane Central Register was conducted to find RCTs using any form of homeopathy for the treatment of insomnia or sleep-related disorders. Data were extracted according to pre-defined criteria; risk of bias was assessed using Cochrane criteria. Six randomised, placebo-controlled trials met the inclusion criteria. Two studies used individualised homeopathy, and four used standardised homeopathic treatment. All studies had significant flaws; small sample size was the most prevalent limitation. The results of one study suggested that homeopathic remedies were superior to placebo; however, five trials found no significant differences between homeopathy and placebo for any of the main outcomes. Evidence from RCTs does not show homeopathy to be an effective treatment for insomnia and sleep-related disorders.
It follows that the new trial contradicts previously published evidence. In addition, it clearly lacks plausibility, as the remedies used were highly diluted and therefore should be pure placebos. So, what could be the explanation of the new, positive result?
As far as I can see, there are the following possibilities:
- some undetected/undisclosed bias,
- homeopathy works after all.
I would be most grateful, if someone could help solving this puzzle for me (if needed, I can send you the full text of the new article for assessment).
Mistletoe treatment of cancer patients was the idea of Rudolf Steiner. Mistletoe grows on a host tree like a parasite and eventually might kill it. This seems similar to a cancer killing a patient, and Steiner – influenced by the homeopathic ‘like cures like’ notion – thought that mistletoe should thus be an ideal treatment of all cancers. Despite the naivety of this concept, it somehow did catch on, and mistletoe has now become the number one cancer SCAM in Europe which is spreading fast also to the US and other countries.
But, as we all know, the fact that a therapy lacks plausibility does not necessarily mean that it is clinically useless. To decide, we need clinical trials; and to be sure, we need rigorous reviews of all reliable trials. Two such papers have just been published.
The aim of the systematic review was to give an extensive overview about current state of research concerning mistletoe therapy of oncologic patients regarding survival, quality of life and safety.
The authors extensive literature searches identified 3647 hits and 28 publications with 2639 patients were finally included in this review. Mistletoe was used in bladder cancer, breast cancer, other gynecological cancers (cervical cancer, corpus uteri cancer, and ovarian cancer), colorectal cancer, other gastrointestinal cancer (gastric cancer and pancreatic cancer), glioma, head and neck cancer, lung cancer, melanoma and osteosarcoma. In nearly all studies, mistletoe was added to a conventional therapy. Patient relevant endpoints were overall survival (14 studies, n = 1054), progression- or disease-free survival or tumor response (10 studies, n = 1091). Most studies did not show any effect of mistletoe on survival. Especially high quality studies did not show any benefit.
The authors concluded that, with respect to survival, a thorough review of the literature does not provide any indication to prescribe mistletoe to patients.
The aim of the second systematic review by the same team was to give an extensive overview about the current state of evidence concerning mistletoe therapy of oncologic patients regarding quality of life and side effects of cancer treatments. The same studies were used for this analysis as in the first review. Regarding quality of life, 17 publications reported results. Studies with better methodological quality showed less or no effects on quality of life.
The authors concluded that with respect to quality of life or reduction of treatment-associated side effects, a thorough review of the literature does not provide any indication to prescribe mistletoe to patients with cancer.
In 2003, we published a systematic review of the same subject. Here is its abstract:
Mistletoe extracts are widely used in the treatment of cancer. The results of clinical trials are however highly inconsistent. We therefore conducted a systematic review of all randomised clinical trials of this unconventional therapy. Eight databases were searched to identify all studies that met our inclusion/exclusion criteria. Data were independently validated and extracted by 2 authors and checked by the 3rd according to predefined criteria. Statistical pooling was not possible because of the heterogeneity of the primary studies. Therefore a narrative systematic review was conducted. Ten trials could be included. Most of the studies had considerable weaknesses in terms of study design, reporting or both. Some of the weaker studies implied benefits of mistletoe extracts, particularly in terms of quality of life. None of the methodologically stronger trials exhibited efficacy in terms of quality of life, survival or other outcome measures. Rigorous trials of mistletoe extracts fail to demonstrate efficacy of this therapy.
As we see, 16 years and 18 additional trials have changed nothing!
I therefore think that it is time to call it a day. We should stop the funding for further research into this dead-end alley. More importantly, we must stop giving false hope to cancer patients. All that mistletoe therapy truly does is to support a multi-million Euro industry.
Today is WORLD CANCER DAY. A good reason, I feel, to remind everyone of the existence of CAM-CANCER, an initiative that I have been involved with from its start (in fact, I was one of its initiators). Essentially, we – that is an international team of CAM-experts – conduct systematic reviews of CAMs often advertised for cancer. We then offer them as a free web resource providing the public with evidence-based information about all sorts of CAMs for cancer.
CAM-Cancer follows a strict methodology to produce CAM-summaries of high quality. Writing, review and editorial processes all follow pre-defined methods and the CAM-Cancer editorial team and Executive Committee ensure that CAM-summaries comply with the guidelines and templates. We are independent from commercial funders and strive to be as objective as possible. Most of the experts are more enthusiastic about the value of CAM than I am, but we do our very best to avoid letting sentiments get in the way of rigorous scientific assessments.
So far, we have managed to publish a respectably large and diverse array of summaries. Here is the full list:
Let me pick out just one of the summaries, Gerson therapy. This topic has led to fierce debates on my blog. The ‘key points’ of the CAM-CANCER summary are as follows:
- Gerson therapy uses a special diet, supplements and coffee enemas with the aim of detoxifying and stimulating the body’s metabolism.
- No substantial evidence exists in the scientific literature to support the claims that the Gerson therapy is an effective alternative therapy for cancer.
- Some evidence exists to suggest that elements of the therapy (coffee enemas in particular) are potentially dangerous if used excessively.
- The specific safety problems, advice to stop conventional cancer therapies and the lack of substantial evidence for efficacy outweigh any benefits associated with the Gerson therapy.
I think this is clear enough and it certainly corresponds well with what I previously wrote about Gerson on this blog. The style of presentation might be different, but the information and conclusions are almost identical.
Altogether, our CAM-CANCER summaries are well-informed, concise, and strictly evidence-based. On this WORLD CANCER DAY, I therefore warmly recommend them to everyone and sincerely hope you make good use of them, for instance, by telling other interested parties about this little-known but precious resource.