Do you think that chiropractic is effective for asthma? I don’t – in fact, I know it isn’t because, in 2009, I have published a systematic review of the available RCTs which showed quite clearly that the best evidence suggested chiropractic was ineffective for that condition.
But this is clearly not true, might some enthusiasts reply. What is more, they can even refer to a 2010 systematic review which indicates that chiropractic is effective; its conclusions speak a very clear language: …the eight retrieved studies indicated that chiropractic care showed improvements in subjective measures and, to a lesser degree objective measures… How on earth can this be?
I would not be surprised, if chiropractors claimed the discrepancy is due to the fact that Prof Ernst is biased. Others might point out that the more recent review includes more studies and thus ought to be more reliable. The newer review does, in fact, have about twice the number of studies than mine.
How come? Were plenty of new RCTs published during the 12 months that lay between the two publications? The answer is NO. But why then the discrepant conclusions?
The answer is much less puzzling than you might think. The ‘alchemists of alternative medicine’ regularly succeed in smuggling non-evidence into such reviews in order to beautify the overall picture and confirm their wishful thinking. The case of chiropractic for asthma does by no means stand alone, but it is a classic example of how we are being misled by charlatans.
Anyone who reads the full text of the two reviews mentioned above will find that they do, in fact, include exactly the same amount of RCTs. The reason why they arrive at different conclusions is simple: the enthusiasts’ review added NON-EVIDENCE to the existing RCTs. To be precise, the authors included one case series, one case study, one survey, two randomized controlled trials (RCTs), one randomized patient and observer blinded cross-over trial, one single blind cross study design, and one self-reported impairment questionnaire.
Now, there is nothing wrong with case reports, case series, or surveys – except THEY TELL US NOTHING ABOUT EFFECTIVENESS. I would bet my last shirt that the authors know all of that; yet they make fairly firm and positive conclusions about effectiveness. As the RCT-results collectively happen to be negative, they even pretend that case reports etc. outweigh the findings of RCTs.
And why do they do that? Because they are interested in the truth, or because they don’t mind using alchemy in order to mislead us? Your guess is as good as mine.
18 Responses to The alchemists of alternative medicine – part 6: smuggling in non-evidence
Leave a Reply to Mojo Cancel reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
In a world where there are so many studies to look at, many readers will not get past the abstract. This is particularly true for papers outside our field. Having an ever increasing volume of such evidence with the needed conclusions, will support a position with the public and lawmakers. Unlike yourself most health care professionals will not make the time to read these papers. They create an impression and for their purposes that is all that is necessary.
Thank you for taking the time to dig through this Bovine scat on our behalf. You also help us to better read published papers of all kinds.
The authors of this review actually concluded that “more evidence is required before any definitive statements can be made with respect to the clinical effectiveness of chiropractic care for patients with asthma and with respect to the most appropriate role chiropractors should play in the management of these patients”
I know – but such ‘disclaimers’ are routine in research publications. their conclusions were clearly far too positive considering that the collective RCT-data are negative.
By this reasoning, Andy, one could make a case for the continued study of the shape of the earth. If chiropractic was actually effective for anything, doesn’t it seem that we might know this by now? The bottom line is that there isn’t even any reason to think it is effective as it is based on a single unscientific observation by Palmer.
Even little children stop believing in Santa Claus once they are told the truth–or as is often the case, they discover it for themselves.
There’s a classic example here . It is really a study of numbers using homeopathy and patient (actually: parent of patient) satisfaction, yet it is being touted by some homeopathists as evidence for efficacy.
My area of special interest is in chronic syndromes (conditions characterized by a group of symptoms that frequently occur together) for which we currently have no medical cure. What I find particularly unsavoury is the alt med alchemists who cherry-pick amongst the symptoms, using methods Prof. Ernst is describing in this series of articles, to then make claims that their version of alchemy is backed by evidence to be efficacious in the treatment of a particular syndrome.
The alchemists use well-rehearsed rhetoric, misdirection, incomprehensible yet sciencey-sounding nonsense, endless appeals to authority, and false promises of further studies, to maximally overwhelm not just those who are chronically ill with the syndrome, but also their friends and families trying to support them (i.e. those who have the money to help pay for the proffered alchemy). I think of this as: The Insidious Creep of Alt Med — the double entendre of my term is the only aspect of it that could be considered to be somewhat therapeutic. It isn’t funny, but learning to laugh in the face of adversity and false promises is better than suffering from the horrible depression that so often results after being financially milked by a quack.
As someone with allergic-asthma, I cannot count the number of times some altie-inclined person has most sincerely offered me the “one true cure” for my allergies. Avoidance, while not very glamorous, seems to be the best avenue–now how do I sell that?!
Selling avoidance would be easy if we named it something like Triaxial Displacement Therapy. Shall I start drafting some woo and a three-tier mastery scheme to go with it?
Apologies, Irene: I meant to end that with a 🙂 to show that I was agreeing with you. My replies to you are never indented to be sarcastic or dismissive; they’re intended to be supportive and my sarcasm is directed towards only those who try to con us.
“…a systematic review of the available RCTs which showed quite clearly that the best evidence suggested chiropractic was ineffective for that condition.”
Most people will take that on face value & infer that as being proof that choropractic is ineffective for that condition. And the author clearly implied that, as ‘ineffective’ was written in boldface to emphasize its significance, and the phrase ‘showed quite clearly’ was added to make it sound even more dramatic.
However in all fairness, that systemic review was not definitive, and neither are most systematic reviews, since they look at what data is available at the particular time the review is accomplished, & frequently they only look at only a portion of the available data from sources they deem credible. At a future time, when another systemic review is conducted and more & better data is available, different results can be expected, so there is nothing definitive, & words to indicate that it is, is disingenuous.
Unlike the pharmaceutical companies who have lots more money & are mandated by law to perform testing on their drugs, chiropractors are much more fragmented with less money, & they are not mandated to do RCTs, which is why there are less RCTs available for systemic reviews. Since they are not mandated, who in the chiropractic community is going to take on the initiative to do the RCTs? Why should they? Who’s going to fund the RCTs? So instead of being critical, let’s look at the whole picture.
PURE GENIUS!
You just told me what my own SR showed [and gave me a completely free lesson about SRs in general]
THANKS
Given that Jon expects that we only need to wait for better data to see different results, totally ignoring that better studies of “alternative” treatments usually show that the interventions are even less useful than we thought, I don’t think he minds lecturing you on your own paper. Also notice that he seems to think being critical and focussing on the whole picture are different things.
Jon wrote: “Unlike the pharmaceutical companies who have lots more money & are mandated by law to perform testing on their drugs, chiropractors are much more fragmented with less money”
Well chiropractors are certainly not fragmented when it comes to marketing. For example, it’s strange how the chiropractic marketing group, the Foundation for Chiropractic Progress, received record financial support of over $500K in 2009 http://www.chiro.org/wordpress/?p=1151 when only a month before, the Foundation for Chiropractic Education and Research, filed for bankruptcy:
http://www.dynamicchiropractic.com/mpacms/dc/article.php?id=54144
Even more strange is that after 152 randomized controlled trials and 33,000 articles (since the inception of the Foundation for Chiropractic Education and Research in 1943), chiropractors are still unable to demonstrate that chiropractic spinal manipulation – for which there is only some very slim evidence for the short-lived relief of pain in back pain sufferers – is superior to exercise or taking a couple of paracetamol. Indeed, spinal manipulative therapy could be looking at ‘a slow death by data’…
http://www.bodyinmind.org/spinal-manipulative-therapy-a-slow-death-by-data/
The way to overcome this problem is, presumably, to mandate chiropractors to do RCTs by having the same rules for chiropractic as there are for pharmaceuticals – to only licence chiropractic for conditions where it has been demonstrated to work, so that chiropractors are “mandated to do RCTs” in the same way that pharmaceutical companies are.
Jon said:
How about the chiros?
In the profoundly dizzying world of alt med it seems that RCT is far too often taken to mean Recorded Client Testimonies, with the onus being on “science” to disprove each and every (poorly-)recorded client case notes.
In my experience with alt med, the reason that practitioners record clients’ full name and addresses might not always be for the reason that clients are led to believe. The random clients are indeed paying to partake, unwittingly, in ongoing trials that will never be completed: each trial result suggests that the evidence is inclusive therefore further studies are indicated. Perpetual motion does exist in reality, but only in the domain of business empires.
Sorry if that was too acidic, I haven’t yet found a local practitioner to balance my pH.
In reply to Alan Henness:
“How about the chiros?”
I agree with you 100%. The Danish chiro’s have done just that. They all voted a while ago to contribute a % of each consultation into a research fund. That is why there is good research coming out of Denmark now and they are taken seriously by the doctors. On average there are 300,000 chiro consultations in Australia every week. If $1 or even 50c was donated automatically when the patients health fund card was run through the HiCaps machine that is between 7.8 and 15.6 Million per year. Year after year after year! I would be happy to be part of such a scheme!
Better than relying on the ASRF who contribute Diddley Squat to research.
Correct answer!
I have already created an easy-to-use calculator to help with exactly this, allowing you to play with the numbers and see what funds could be very easily raised.
There are around 3,000 chiros in the UK; even making very conservative estimates about the average number of customers they have (4 per day) and what they charge (£50 a visit), a 1% levy (50p) on each visit would raise just under £1 million a year.
Using some higher numbers (3,000 chiros, 8 customers a day, £80 a visit, 5% levy) raises over £15 million each and every year. Just think what high quality research they could do for that tidy sum?
The only question is, why isn’t this being done?