I have often asked myself whether it is right/necessary to scientifically test things which are entirely implausible. Should we, for instance test the effectiveness of treatments which have a very low prior probability of generating a positive effect such as paranormal healing, homeopathy or Bach flower remedies? If you believe in the principles of evidence-based medicine you might focus on the clinical evidence and see biological plausibility as secondary. If you are a basic scientist, you are likely to do the reverse.
A recent article addressed this issue. The author points out that evaluating the absurd is absurd. Specifically, he noted that the empirical evaluation of a therapy would normally assume a plausible rationale regarding the mechanism of action. However, examination of the historical background and underlying principles for reflexology, iridology, acupuncture, auricular acupuncture, and some herbal medicines, reveals a rationale founded on the principle of analogical correspondences, which is a common basis for magical thinking and pseudoscientific beliefs such as astrology and chiromancy. Where this is the case, it is suggested that subjecting these therapies to empirical evaluation may be tantamount to evaluating the absurd.
This makes a lot of sense – but is it really entirely true? Are there no legitimate reasons at all for testing alternative treatments that lack biological plausibility? Ten or twenty years ago, I would have disagreed with the notion that plausibility is an essential prerequisite for scientific testing; today, I have changed my mind a little, but not as much as to agree completely with the assumption. In other words, I still see more than one good reason why evaluating the absurd might be reasonable or even advisable.
- Using plausibility as the only arbiter of scientific ‘evaluability’, assumes that we understand everything about plausibility there is to know. Yet it might just be possible that we mis-categorise something as implausible simply because we are not yet fully aware of all the facts.
- Declaring something as plausible and another thing as implausible are not hard and fast verdicts but judgements which, at least to some degree, are subjective. Sceptics find the axioms of homeopathy utterly implausible, for instance – but ask a homeopath, and you will hear all sorts of explanations which, at least to them, sound plausible.
- If an implausible alternative treatment is in wide-spread use, we arguably have a responsibility to test it scientifically in order to demonstrate the truth about it (to those proponents of that therapy who are willing to accept that rigorous science can find the truth). If we fail to do this, it will be the enthusiasts of that therapy who conduct less than rigorous science and produce false positive results. In turn, this will give the impression that the treatment is effective and mislead consumers, politicians, journalists etc. Seen from this perspective, it might even be unethical to not do the science.
So, I am in two minds about this (which might be a reflection of the fact that, during different periods of my life, I have been a clinician, a basic scientist and a clinical researcher). I realise that plausibility and prior probability are important – much more so than I appreciated years ago. But I think they should not be the only criteria. The clinical evidence should not be pushed aside completely.
I’d be interested to learn your views on this tricky issue.
11 Responses to Evaluating the absurd: a waste of time or a necessary step towards progress?
Leave a Reply to Nick Nakorn Cancel reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
I think it is both a waste of time and a necessary step to progress to test the absurd. In engineering, for example, many lives have been saved by testing what were thought to be absurd ideas; ideas that are now ordinary. In medicine I expect germ theory was once thought absurd by many religious people and we know that many are still anti-vax. On the other hand, ideas are often absurd not because we don’t know about them but because we do. Homeopathy, if it were true, would overturn all the first principles of science so we can rule it out (it’s probably popular because it’s so anti-science) in the same way we can rule out perpetual motion machines. Every so often on facebook there will be ‘The Secret Energy Source The Oil Companies Don’t Want You to Know About!’ and it’s always nonsense because we know that thermodynamics does not allow entropy to work backwards. But quite often individuals (or occasionally newspapers) will spend their own cash exposing these hoaxes just to prevent gullible people from investing their life savings. So it all depends on the context; popularity, the ability to do damage, gaps in our knowledge and exposing charletons are all good reasons for testing the absurd while our current knowledge and lack of funds is a reason for dismissiing clearly fraudulent claims before they come too popular. Many wine growers are testing Biodynamics because of the fraudulent claims of the Biodynamic growers who charge considerably more to a gullible public.
Prior probability and prior plausibility can be very useful indicators, but it is simple to demonstrate that these criteria are often used absurdly…
Just after the Big Bang, the prior probability and plausibility of me writing this message today were both zero. They also remained zero for 13.8 billion years, but here I am writing this comment. The circa 10^80 atoms in our universe could have assembled themselves in an almost infinite number of different ways, but they didn’t: they assembled themselves into all the forms we now observe at this present moment in time.
The remit of science is to explain what we currently observe by working backwards through time to the beginning of the universe. It is absolutely not the remit of science to explain why the almost infinite number of other atom combinations did not occur instead.
Creating models of the universe running forwards in time belong to the domains of religion, pseudoscience, CAM, popular psychology (and a few other domains that I’m too polite to mention). These are the domains of unbounded fiction for which there is an annual global market value of hundreds of billions £/$.
Most often, it is evidence-based observations that lead to the creation of scientific explanations. Why? Because without solid and replicable evidence for an observation it is a deplorable waste of resources to attempt a scientific explanation or refutation.
Unfortunately, the ever expanding domains of quackery are lucrative simply by exploiting the general public’s severe bankruptcy in areas of the scientific method and critical thinking skills. Quackery also fully exploits the absurd political correctness still used in science.
Compare science to engineering: A scientist is compelled to write wishy-washy wording such as “The findings were inconclusive, which may indicate that further research is required.”; An engineer would write “The findings were total bollocks, the method/system doesn’t work so we’ll relegate it to history and focus on far more important tasks.”
I totally admire Dame Sally Davies for so clearly stating: “Homeopathy is rubbish.” It is indeed rubbish that has no place whatsoever in this 21st Century other than to milk maximal money from the gullible.
If the results of RCTs and systematic reviews evolve to be written in engineering speak instead of PC speak then the human race will have a good chance to move forward from its many hundreds and thousands of years of accumulated abject nonsense.
Our Moon walks, Mars rovers, and space telescopes are the ultimate testimony to the irrelevance of prior probability and prior plausibility when, and only when, each subsystem of the whole is based on either science or just solid evidence from experiments.
Science fiction has its place in books, cinemas, and home entertainment; it has no place in modern healthcare.
Yes, we engineers thin differently.
The problem, as I see it, there aremuch more RCTs on homeopathy giving positive evidence than could be explained by the 5 % false positive probability and publication bias. Even checking the quality (Jadad score) does not solve the problem. As the Shang – and other – metaanalyses show, there are a number of hogh quality trials that seem to prove efficiency.
So we have different modes of looking at it:
Homeopathic: Efficacy is proven by the tests, so science is wrong or incomplete in understanding the process.
Science: You need some additional information to properly use the method, be it prior probability or plausible physical model or whatever. You simply got to know if it works to find evidence that it works.
Engineering (especially with prior experience in quality management): There is something wrong with the method of the RCT or its application. And if this method or its application is uncapable to yield proper results, then this is the case in all fields, not just homeopathy. The way, the method is applied either has to be improved or the method itself has to be disarded for being useless and misleading.
One basic requirement: The method to check evidence should be able to detect if there is none, maybe with some reasonably small (!) tolerances. Imagine, the method to check the amount of alcohol in a driver’s blood needs prior proof, that the driver really is drunk, to work properly. Or the method to establish the safe load of a bridge would need prior proof, that the bridge actually is able to carry the load, otherwise the results will be useless. Ridiculous.
I am working on this one, and I guess I am up to something here.
It is ‘think’ in the first line…
You wrote that “You simply got to know if it works to find evidence that it works.”
It does not sound so …scientific what you are saying – I thought that whether a treatment works or not is supposed to be tested in RCT and later in meta analyses. Isn’t that the current and mainstream scientific method? If you knew the answer in advance— you would not need to investigate it. You know of course that for several conventional medications the mechanism of action is unknown.
According to what you are saying Maybe all RCT and meta analyses ( including Ernsts) say nothing about Homeopath’s efficacy and the health care decisions taken based on these are false and arbitrary, Based on what individual scientists think or” feel ” it is improbable; or simply based on their superstitions.
>>> If you knew the answer in advance— you would not need to investigate it.
Oh no! Now you found me out….
As it was said in this talk page : “In engineering, for example, many lives have been saved by testing what were thought to be absurd ideas; ideas that are now ordinary.”
In physics, criticism of the theory of relativity “was mainly expressed in the early years after its publication in the early 1900s, on scientific, pseudoscientific, philosophical, or ideological bases” – “Some of these criticisms had the support of reputable scientists. ”
Regarding homeopathy – some researches say it is all placebo. Others, equally notable, in reputable mainstream sources, say it is not placebo.
(It is not like astrology for which there is no reputable mainstream journal saying that something is real in the field.)
Therefore, not supporting more research for homeopathy and other alternative medicine systems , while notable scientist say -it is something there, is a totalitarian approach typically based on emotional and less rational basis. A good example is what Ernst says about the absurdity of certain ideas- as absurd being an ” objective” term.
” This whole pain thing that you seem to find is so important is a medical construct. Treat the pain and all is well. Not. So maybe you should think about this from a different viewpoint. The pain is not the problem. It is the alarm that something is wrong and needs to change in their life.”
WHERE DID YOU PICK UP SUCH NONSENSE?
in conventional medicine, the aim is to treat the cause of symptoms where possible.
Maybe you posted your reply to the wrong section?
A few months ago I wrote an article about this topic (Scientability — a concept for the handling of homeopathic
remedies by EbM, in: Z. Evid. Fortbild. Qual. Gesundh. wesen (ZEFQ) (2013) 107, 606—610). I argue that homeopathy should be investigated, but with appropriate kinds of studies. RCTs are much too susceptible for bias and mistakes. So they are easily abused by homeopaths.
This is the english abstract of the article:
Evidence-based medicine (EbM) has proved to be very useful in healthcare; thanks
to its methodology the reliability of our knowledge of the benefits and harms of interventions
can be assessed. This at least applies to interventions which are based on a plausible concept
for their mechanism of action and which have already achieved positive effects in experiments
and simple studies. However, for interventions whose concepts contradict scientific findings
EbM has proved to be unsuitable; it has not been able to prevent that they are still regarded as
effective amongst wide parts of the population and medical experts. Particularly homeopathy
has managed to even present itself as scientifically justified by using EbM. With the aim of
highlighting the speculative character of homeopathy and other procedures and of preventing
EbM from getting damaged, the concept of scientability is introduced in this article. This
concept only approves of clinical studies if the intervention that is to be tested does not
contradict definite scientific findings.
This concept was attacked both by supporters of EbM as well of supporters of Homeopathy.