Twenty years ago, when I started my Exeter job as a full-time researcher of complementary/alternative medicine (CAM), I defined the aim of my unit as applying science to CAM. At the time, this intention upset quite a few CAM-enthusiasts. One of the most prevalent arguments of CAM-proponents against my plan was that the study of CAM with rigorous science was quite simply an impossibility. They claimed that CAM included mind and body practices, holistic therapies, and other complex interventions which cannot not be put into the ‘straight jacket’ of conventional research, e. g. a controlled clinical trial. I spent the next few years showing that this notion was wrong. Gradually and hesitantly CAM researchers seemed to agree with my view – not all, of course, but first a few and then slowly, often reluctantly the majority of them.
What followed was a period during which several research groups started conducting rigorous tests of the hypotheses underlying CAM. All too often, the results turned out to be disappointing, to say the least: not only did most of the therapies in question fail to show efficacy, they were also by no means free of risks. Worst of all, perhaps, much of CAM was disclosed as being biologically implausible. The realization that rigorous scientific scrutiny often generated findings which were not what proponents had hoped for led to a sharp decline in the willingness of CAM-proponents to conduct rigorous tests of their hypotheses. Consequently, many asked whether science was such a good idea after all.
But that, in turn, created a new problem: once they had (at least nominally) committed themselves to science, how could they turn against it? The answer to this dilemma was easier that anticipated: the solution was to appear dedicated to science but, at the same time, to argue that, because CAM is subtle, holistic, complex etc., a different scientific approach was required. At this stage, I felt we had gone ‘full circle’ and had essentially arrived back where we were 20 years ago – except that CAM-proponents no longer rejected the scientific method outright but merely demanded different tools.
A recent article may serve as an example of this new and revised stance of CAM-proponents on science. Here proponents of alternative medicine argue that a challenge for research methodology in CAM/ICH* is the growing recognition that CAM/IHC practice often involves complex combination of novel interventions that include mind and body practices, holistic therapies, and others. Critics argue that the reductionist placebo controlled randomized control trial (RCT) model that works effectively for determining efficacy for most pharmaceutical or placebo trial RCTs may not be the most appropriate for determining effectiveness in clinical practice for either CAM/IHC or many of the interventions used in primary care, including health promotion practices. Therefore the reductionist methodology inherent in efficacy studies, and in particular in RCTs, may not be appropriate to study the outcomes for much of CAM/IHC, such as Traditional Korean Medicine (TKM) or other complex non-CAM/IHC interventions—especially those addressing comorbidities. In fact it can be argued that reductionist methodology may disrupt the very phenomenon, the whole system, that the research is attempting to capture and evaluate (i.e., the whole system in its naturalistic environment). Key issues that surround selection of the most appropriate methodology to evaluate complex interventions are well described in the Kings Fund report on IHC and also in the UK Medical Research Council (MRC) guidelines for evaluating complex interventions—guidelines which have been largely applied to the complexity of conventional primary care and care for patients with substantial comorbidity. These reports offer several potential solutions to the challenges inherent in studying CAM/IHC. [* IHC = integrated health care]
Let’s be clear and disclose what all of this actually means. The sequence of events, as I see it, can be summarized as follows:
- We are foremost ALTERNATIVE! Our treatments are far too unique to be subjected to reductionist research; we therefore reject science and insist on an ALTERNATIVE.
- We (well, some of us) have reconsidered our opposition and are prepared to test our hypotheses scientifically (NOT LEAST BECAUSE WE NEED THE RECOGNITION THAT THIS MIGHT BRING).
- We are dismayed to see that the results are mostly negative; science, it turns out, works against our interests.
- We need to reconsider our position.
- We find it inconceivable that our treatments do not work; all the negative scientific results must therefore be wrong.
- We always said that our treatments are unique; now we realize that they are far too holistic and complex to be submitted to reductionist scientific methods.
- We still believe in science (or at least want people to believe that we do) – but we need a different type of science.
- We insist that RCTs (and all other scientific methods that fail to demonstrate the value of CAM) are not adequate tools for testing complex interventions such as CAM.
- We have determined that reductionist research methods disturb our subtle treatments.
- We need pragmatic trials and similarly ‘soft’ methods that capture ‘real life’ situations, do justice to CAM and rarely produce a negative result.
What all of this really means is that, whenever the findings of research fail to disappoint CAM-proponents, the results are by definition false-negative. The obvious solution to this problem is to employ different (weaker) research methods, preferably those that cannot possibly generate a negative finding. Or, to put it bluntly: in CAM, science is acceptable only as long as it produces the desired results.
16 Responses to My summary of 20 years of alternative medicine research
Leave a Reply to kololodo Cancel reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
This is absolutely true, and highlights the difference between the scientific method and the pseudoscientific method.
The scientific method places objective validity at the apex. Any scientist knows that when an idea is tested, the result may come back negative, and when it does, it means your idea was probably wrong. In science, you are allowed to be wrong: there is no fooling nature. As Carl Sagan put it,:
The pseudoscientific method places doctrine at the apex. Your idea is assumed to be valid simply because it is your idea, and disconfirming results are waved away. Once you have adopted a doctrinal authority – such as Samuel Hahnemann – their word is law, and any science that conflicts is by definition wrong.
The primary feature of the scientific method is that it is self-correcting. Errors may creep in and even persist for a while, but they are corrected in time, inevitably. Think of the idea that ulcers are caused by stress. The pseudoscientific method is inherently incapable of self-correction, because the primacy of objective investigation is not only not recognised, but is openly rejected. Thus homeopaths differ on the use of imponderables, the validity of homeoprophylaxis, the Korsakovian dilution, even on whether succussion is required at all – but they all absolutely agree that homeopathy is uniquely valid. They cannot settle their doctrinal differences because any properly objective test to compare two schools of homeopathic thought, will find them both equally wrong. And that result cannot be accepted because it conflicts with doctrine.
This also explains why quacks are fixated on people like you, Simon Singh and Stephen Barrett. In their world, if you demolish the authority, then the doctrine may also fall, and the doctrine cannot be divorced form the authority. They do not understand that even if Einstein was found to have copied his papers from someone else, their conclusions would remain valid; if every skeptic currently alive were to abandon skepticism tomorrow, the doctrines of alternative medicine would remain every bit as wrong and the scientific basis of real medicine would remain every bit as sound.
Great comment! Your example of ulcers caused by stress is beautiful. It took medicine about 15 years for all practitioners to be convinced that peptic ulcers are bacterial infections. That’s an eyeblink compared to the centuries over which CAM pseudoscience grumbles on its inept way without accepting it has nothing dependable to offer.
In medicine and other sciences, and religious and political ideologies, it is stunning what lengths those who have made up their minds before seeing the evidence will go to twist and turn science and philosophy into something that will shore up what they already believe. As you, Dr. Ernst, once said, they use science as a lamppost–not to illuminate, but to lean on. This is wrong of them, but they will turn and say that anyone who says that is what they are doing are really doing it themselves. A.C. Grayling puts it, “There is a human tragedy in this: the more they suspect they might be wrong, the more fiercely they adhere.”
You clearly don’t truly understand the word Reductionism, how that relates to linear thinking vs whole/circular, nor BASIC quantum physics which has proven that scientific results are altered w/ conscious attention/scrutiny. You probably found some interesting, valid worthy standards, quality issues, things to be improved, (like the good that came from the Flexner Report), but to white wash the whole topic is juvenile and trite and hopefully people won’t fall for it. Spare me.
“white wash the whole topic …” ????
@Julie Tyler
As physicist Jim Al-Khalili said:
Would you mind to formulate your quantum mechanical approach? To make it clear, what I mean with ‘formulate’: come up with the equations on which your claims are based. Mathematics and not meaningless talk about the uncertainty principle or Schroedinger’s cat.
It’s a sure sign when a non-physicist uses the phrase “quantum physics” that they have no idea what quantum physics really is. Ask a physicist what your statement means and they will laugh in your face.
Julie Tyler, you clearly don’t know what is basic quantum physics which have never proven “that scientific result are altered by conscious attention/scrutiniy”. First, quantic laws regulates quantum world at quantum scale and not macro-medicine world (more like chemistry and classical physics) so they don’t go along with our situation here. Then you claim that the quantum state of particle change when “consciously” observed, well that’s oversimplifying. An isolated particle (note -isolated-) may have few possible quantum states and when you put light on it “choose” one state (that doesn’t mean the particle is really in few states at once). In classical physics the initial condition of the experiment are known (or at least sufficienly known) and determine the result of the experiment. In quantum physics the inital condition of the experiment are not known before we look at the particle and look at the particle mean throw a photon at it and a photon will interact with the particle and force a state (we call it decoherence). Then we can statistically predict wich state the particle will take with more measure, and this is perfectly reproductible, so the result are not altered by the “scrutinity” it’s just an non-deterministic approach provoked by decoherence. If the results were really affected by the “scrutinity” no quantum experiment would be reproductible and it would just may be the limit of physics but it’s not the case.
Please forgive a short cut and paste from “Science as Falsification” by Karl Popper:
“These considerations led me in the winter of 1919-20 to conclusions which I may now reformulate as follows.
It is easy to obtain confirmations, or verifications, for nearly every theory — if we look for confirmations.
Confirmations should count only if they are the result of risky predictions; that is to say, if, unenlightened by the theory in question, we should have expected an event which was incompatible with the theory — an event which would have refuted the theory.
Every “good” scientific theory is a prohibition: it forbids certain things to happen. The more a theory forbids, the better it is.
A theory which is not refutable by any conceivable event is non-scientific. Irrefutability is not a virtue of a theory (as people often think) but a vice.
Every genuine test of a theory is an attempt to falsify it, or to refute it. Testability is falsifiability; but there are degrees of testability: some theories are more testable, more exposed to refutation, than others; they take, as it were, greater risks.
Confirming evidence should not count except when it is the result of a genuine test of the theory; and this means that it can be presented as a serious but unsuccessful attempt to falsify the theory. (I now speak in such cases of “corroborating evidence.”)
Some genuinely testable theories, when found to be false, are still upheld by their admirers — for example by introducing ad hoc some auxiliary assumption, or by reinterpreting the theory ad hoc in such a way that it escapes refutation. Such a procedure is always possible, but it rescues the theory from refutation only at the price of destroying, or at least lowering, its scientific status. (I later described such a rescuing operation as a “conventionalist twist” or a “conventionalist stratagem.”)
One can sum up all this by saying that the criterion of the scientific status of a theory is its falsifiability, or refutability, or testability.”
If ever there has been a better example of the ad hoc reasoning or conventionalist twist, it is this new focus on anecdotes. By attempting to cloak themselves with scientific rigor, they have had to kill science, skin it, and turn it inside out.
1, do you know why people get sick of it scientific? — — Of course you do not know if you know could get the Nobel Prize in Medicine. You do not know where to disease, there is no basis to produce medicine theory, your theory mansion built on a sand fall apart at any time.
2, you have a medical theory into practice it? All kinds of pain, a variety syndrome? You do not know how it comes, do not know how to treat, do not know the results of treatment. In the hands of Chinese medicine theory it can be judged by the treatment, so even SARS, Chinese medicine has a history of 3,000 years can still be cured.
3, randomized, double-blind is science? — Of course not, but he is just statistics, it is like a simple order to solve quadratic equations, at the expense of all possible answers are carried out on behalf of checking into it, which is most stupid way. Ill is not a simple, linear logic, but with complex conditions and rules of knowledge, interaction and influence with its own life sciences. Not A-> B, but A-> C-> F -> ….-> B.
Your comments are hard to understand, but I think the answer to them is contained in Guy Chapman’s comment at the start of the thread.