Whenever a new trial of an alternative intervention emerges which fails to confirm the wishful thinking of the proponents of that therapy, the world of alternative medicine is in turmoil. What can be done about yet another piece of unfavourable evidence? The easiest solution would be to ignore it, of course – and this is precisely what is often tried. But this tactic usually proves to be unsatisfactory; it does not neutralise the new evidence, and each time someone brings it up, one has to stick one’s head back into the sand. Rather than denying its existence, it would be preferable to have a tool which invalidates the study in question once and for all.
The ‘fatal flaw’ solution is simpler than anticipated! Alternative treatments are ‘very special’, and this notion must be emphasised, blown up beyond all proportions and used cleverly to discredit studies with unfavourable outcomes: the trick is simply to claim that studies with unfavourable results have a ‘fatal flaw’ in the way the alternative treatment was applied. As only the experts in the ‘very special’ treatment in question are able to judge the adequacy of their therapy, nobody is allowed to doubt their verdict.
Take acupuncture, for instance; it is an ancient ‘art’ which only the very best will ever master – at least that is what we are being told. So, all the proponents need to do in order to invalidate a trial, is read the methods section of the paper in full detail and state ‘ex cathedra’ that the way acupuncture was done in this particular study is completely ridiculous. The wrong points were stimulated, or the right points were stimulated but not long enough [or too long], or the needling was too deep [or too shallow], or the type of stimulus employed was not as recommended by TCM experts, or the contra-indications were not observed etc. etc.
As nobody can tell a correct acupuncture from an incorrect one, this ‘fatal flaw’ method is fairly fool-proof. It is also ever so simple: acupuncture-fans do not necessarily study hard to find the ‘fatal flaw’, they only have to look at the result of a study – if it was favourable, the treatment was obviously done perfectly by highly experienced experts; if it was unfavourable, the therapists clearly must have been morons who picked up their acupuncture skills in a single weekend course. The reasons for this judgement can always be found or, if all else fails, invented.
And the end-result of the ‘fatal flaw’ method is most satisfactory; what is more, it can be applied to all alternative therapies – homeopathy, herbal medicine, reflexology, Reiki healing, colonic irrigation…the method works for all of them! What is even more, the ‘fatal flaw’ method is adaptable to other aspects of scientific investigations such that it fits every conceivable circumstance.
An article documenting the ‘fatal flaw’ has to be published, of course – but this is no problem! There are dozens of dodgy alternative medicine journals which are only too keen to print even the most far-fetched nonsense as long as it promotes alternative medicine in some way. Once this paper is published, the proponents of the therapy in question have a comfortable default position to rely on each time someone cites the unfavourable study: “WHAT NOT THAT STUDY AGAIN! THE TREATMENT HAS BEEN SHOWN TO BE ALL WRONG. NOBODY CAN EXPECT GOOD RESULTS FROM A THERAPY THAT WAS NOT CORRECTLY ADMINISTERED. IF YOU DON’T HAVE BETTER STUDIES TO SUPPORT YOUR ARGUMENTS, YOU BETTER SHUT UP.”
There might, in fact, be better studies – but chances are that the ‘other side’ has already documented a ‘fatal flaw’ in them too.
8 Responses to The alchemists of alternative medicine – part 2: the ‘fatal flaw’ method
Leave a Reply to Edzard Cancel reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
Attempting to debunk any form of anti-science or pseudoscience (I repeat myself) using “what” questions has been endlessly shown to be pointless. It always comes down to: “You cannot prove my pink unicorns do not exist therefore science must remain open-minded in order to avoid being laughably dogmatic.”
Debunking anti-science using “why” questions invariably reveals the truth:
Patient: “Why do I need to believe in your pink unicorns in order to overcome my illness?”
Fictional Truthful CAM Practitioner: “If you don’t believe in my pink unicorns I will lose business. Furthermore, I will attempt to sue you if you claim publicly that my pink unicorns do not exist: using the legal resources of Society X to which I sensibly joined to protect myself from scrutiny.”
I’m fully aware that “why” questions frequently result in stalemates over Big sCAM versus Big Pharma, but the scientific method and critical thinking skills are not patented methods: these are the most powerful methods we currently have of debunking the pink unicorns used in bogus business models and the advertising of related products and services.
In the UK we have the ASA and Trading Standards organisations to protect us. The quacks are desperately trying to crush these organisations by overwhelming them — the equivalent of an Internet distributed denial-of-service (DDoS) attack.
Trick or Treatment? initially inspired me to to answer my plethora of “how” questions. Soon after, it inspired me to start learning how to thoroughly research my ongoing “why” questions far beyond the realm of sCAM quackery. It’s been a very difficult journey, but the positive outcomes far outweigh the hardships 🙂
I think this article exemplifies some of the profound points made in The Demon-Haunted World by Carl Sagan.
What you seem to be saying is we can’t really do this with science so let’s have a bit of common sense and not give this nonsense any more air time. Is that it?
Bob, I think science alone cannot solve this problem, but I’m not suggesting for one moment that we cease giving nonsense air time. This long reply isn’t a rant, I’m attempting to highlight the problem from the perspective of those who the suffer most from the alt med empires.
Science has endlessly shown the effects of placebo and nocebo reactions, and it has shown alt med that is efficacious is called medicine. sCAM that has no effect beyond placebo is a collection of different ways to sell placebos (using various combinations of fallacies and theatrics): it is a collection of business models not a collection of scientific models[1]; it’s primary objective is to attract clients whereas the primary objective of medicine is to reduce disease and improve patient quality of life.
[1] the various hypotheses used to explain the treatments are not only anti-scientific, they are wholly incompatible with each other.
Time after time it has been demonstrated that as the power of a study increases, the effectiveness of the alt med treatment rapidly approaches that of placebo or sham treatment. Obviously, it is trivial to perform a series of low-powered studies to obtain statistically significant (but clinically meaningless) results from false positives and/or unsound methods. Most potential clients incorrectly interpret “Studies have shown treatment X to be effective…” as equivalent to “High-quality peer-reviewed medical studies have shown that treatment X is efficacious.”
One could argue that “buyer beware” applies to sCAM just as it does to most consumer products and services. However, this is a false argument because many of the buyers are at their most vulnerable, often desperately seeking a cure, therefore least able to make rational decisions. Indeed, the ASA CAP codes specifically address the issue of marketing to the elderly and vulnerable groups in society.
To the desperate, alt med gives the appearance of being a viable alternative healthcare system. Debunking it with science is useless because the (potential) clients have already discovered that science-based medicine is unable to cure their illness. The last thing a desperate ill person can start learning is critical thinking skills and the scientific method.
I think that the vulnerable would be far more interested to learn that alt med is not an efficacious alternative to medicine, it is a collection of businesses taking advantage of the vulnerable by changing for theatrical placebos that carry risks of causing serious harm.
Using science to debunk sCAM seems to be having little effect on the ever rising numbers of: partitioners; clients; and forms of woo being proffered. Science has been very useful in clamping down on unsubstantiated claims made in advertising and other media, but it’s a time-consuming Whac-A-Mole game being fully exploited by the vendors of woo.
Snake oil vendors have been successful in England for the last 300 years (Richard Stoughton’s Elixir was granted a patent in 1712). Their principles remain the same and each time their trade gets exposed for what it really is they just rename it and/or establish trade organisations and certification schemes to give it an air of credibility.
http://en.wikipedia.org/wiki/Snake_oil
I wish I knew the answer because I find it highly distressing to witness seriously ill people spending over £1000 on a long course of therapy that is not only useless for their condition, it has well-documented negative outcomes — especially the utter despair suffered when it finally dawns on the client that the treatment didn’t work and they’ve run out of money. This is a diabolical societal problem rather than a scientific problem per se.
Great posts Pete. Our journeys have followed similar paths. Darwin, Sagan, Ernst, Singh, Randi, and all the current science blogs have guided me on this path. It’s not always fun being the only skeptic in a room full of alties, but I am never sorry I refused to open my mind so far that my brains fell out.
Thank you Irene. Yes, our journeys have followed similar paths. Being the only skeptic in a room full of alties has become easier since they’ve learnt two things: I will definitely say something; and that they are totally unable to predict what I’ll say or when I’ll say it!
Apologies for my typos and why I wrote “by changing for” instead of “by charging for” is a complete mystery — perhaps my brains did fall out years ago 🙂
This article is itself fatally flawed, in that it implies that it could never be the therapist that is at fault, but always the therapy itself…. which is of course absurd.
This article is just a propaganda tool used to arbitrarily dismiss anyone who questions the validity of an unfavorable study.
you must have a lot of fantasy to read that into my article
Surely that is the interpretation that people would read into the article, which is the sign of a good author in being able to do that, as I’ve seen that argument used in the exact manner as I’ve characterized.