The most widely used definition of EVIDENCE-BASED MEDICINE (EBM) is probably this one: The judicious use of the best current available scientific research in making decisions about the care of patients. Evidence-based medicine (EBM) is intended to integrate clinical expertise with the research evidence and patient values.
David Sackett’s own definition is a little different: Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research.
Even though the principles of EBM are now widely accepted, there are those who point out that EBM has its limitations. The major criticisms of EBM relate to five themes: reliance on empiricism, narrow definition of evidence, lack of evidence of efficacy, limited usefulness for individual patients, and threats to the autonomy of the doctor/patient relationship.
Advocates of alternative medicine have been particularly vocal in pointing out that EBM is not really applicable to their area. However, as their arguments were less than convincing, a new strategy for dealing with EBM seemed necessary. Some proponents of alternative medicine therefore are now trying to hoist EBM-advocates by their own petard.
In doing so they refer directly to the definitions of EBM and argue that EBM has to fulfil at least three criteria: 1) external best evidence, 2) clinical expertise and 3) patient values or preferences.
Using this argument, they thrive to demonstrate that almost everything in alternative medicine is evidence-based. Let me explain this with two deliberately extreme examples.
CRYSTAL THERAPY FOR CURING CANCER
There is, of course, not a jot of evidence for this. But there may well be the opinion held by crystal therapist that some cancer patients respond to their treatment. Thus the ‘best’ available evidence is clearly positive, they argue. Certainly the clinical expertise of these crystal therapists is positive. So, if a cancer patient wants crystal therapy, all three preconditions are fulfilled and CRYSTAL THERAPY IS ENTIRELY EVIDENCE-BASED.
CHIROPRACTIC FOR ASTHMA
Even the most optimistic chiropractor would find it hard to deny that the best evidence does not demonstrate the effectiveness of chiropractic for asthma. But never mind, the clinical expertise of the chiropractor may well be positive. If the patient has a preference for chiropractic, at least two of the three conditions are fulfilled. Therefore – on balance – chiropractic for asthma is [fairly] evidence-based.
The ‘HOISTING ON THE PETARD OF EBM’-method is thus a perfect technique for turning the principles of EBM upside down. Its application leads us straight back into the dark ages of medicine when anything was legitimate as long as some charlatan could convince his patients to endure his quackery and pay for it – if necessary with his life.
Do you think that chiropractic is effective for asthma? I don’t – in fact, I know it isn’t because, in 2009, I have published a systematic review of the available RCTs which showed quite clearly that the best evidence suggested chiropractic was ineffective for that condition.
But this is clearly not true, might some enthusiasts reply. What is more, they can even refer to a 2010 systematic review which indicates that chiropractic is effective; its conclusions speak a very clear language: …the eight retrieved studies indicated that chiropractic care showed improvements in subjective measures and, to a lesser degree objective measures… How on earth can this be?
I would not be surprised, if chiropractors claimed the discrepancy is due to the fact that Prof Ernst is biased. Others might point out that the more recent review includes more studies and thus ought to be more reliable. The newer review does, in fact, have about twice the number of studies than mine.
How come? Were plenty of new RCTs published during the 12 months that lay between the two publications? The answer is NO. But why then the discrepant conclusions?
The answer is much less puzzling than you might think. The ‘alchemists of alternative medicine’ regularly succeed in smuggling non-evidence into such reviews in order to beautify the overall picture and confirm their wishful thinking. The case of chiropractic for asthma does by no means stand alone, but it is a classic example of how we are being misled by charlatans.
Anyone who reads the full text of the two reviews mentioned above will find that they do, in fact, include exactly the same amount of RCTs. The reason why they arrive at different conclusions is simple: the enthusiasts’ review added NON-EVIDENCE to the existing RCTs. To be precise, the authors included one case series, one case study, one survey, two randomized controlled trials (RCTs), one randomized patient and observer blinded cross-over trial, one single blind cross study design, and one self-reported impairment questionnaire.
Now, there is nothing wrong with case reports, case series, or surveys – except THEY TELL US NOTHING ABOUT EFFECTIVENESS. I would bet my last shirt that the authors know all of that; yet they make fairly firm and positive conclusions about effectiveness. As the RCT-results collectively happen to be negative, they even pretend that case reports etc. outweigh the findings of RCTs.
And why do they do that? Because they are interested in the truth, or because they don’t mind using alchemy in order to mislead us? Your guess is as good as mine.
The efficacy or effectiveness of medical interventions is, of course, best tested in clinical trials. The principle of a clinical trial is fairly simple: typically, a group of patients is divided (preferably at random) into two subgroups, one (the ‘verum’ group) is treated with the experimental treatment and the other (the ‘control’ group) with another option (often a placebo), and the eventual outcomes of the two groups is compared. If done well, such studies are able to exclude biases and confounding factors such that their findings allow causal inference. In other words, they can tell us whether an outcome was caused by the intervention per se or by some other factor such as the natural history of the disease, regression towards the mean etc.
A clinical trial is a research tool for testing hypotheses; strictly speaking, it tests the ‘null-hypothesis’: “the experimental treatment generates the same outcomes as the treatment of the control group”. If the trial shows no difference between the outcomes of the two groups, the null-hypothesis is confirmed. In this case, we commonly speak of a negative result. If the experimental treatment was better than the control treatment, the null-hypothesis is rejected, and we commonly speak of a positive result. In other words, clinical trials can only generate positive or negative results, because the null-hypothesis must either be confirmed or rejected – there are no grey tones between the black of a negative and the white of a positive study.
For enthusiasts of alternative medicine, this can create a dilemma, particularly if there are lots of published studies with negative results. In this case, the totality of the available trial evidence is negative which means the treatment in question cannot be characterised as effective. It goes without saying that such an overall conclusion rubs the proponents of that therapy the wrong way. Consequently, they might look for ways to avoid this scenario.
One fairly obvious way of achieving this aim is to simply re-categorise the results. What, if we invented a new category? What, if we called some of the negative studies by a different name? What about NON-CONCLUSIVE?
That would be brilliant, wouldn’t it. We might end up with a simple statistic where the majority of the evidence is, after all, positive. And this, of course, would give the impression that the ineffective treatment in question is effective!
How exactly do we do this? We continue to call positive studies POSITIVE; we then call studies where the experimental treatment generated worst results than the control treatment (usually a placebo) NEGATIVE; and finally we call those studies where the experimental treatment created outcomes which were not different from placebo NON-CONCLUSIVE.
In the realm of alternative medicine, this ‘non-conclusive result’ method has recently become incredibly popular . Take homeopathy, for instance. The Faculty of Homeopathy proudly claim the following about clinical trials of homeopathy: Up to the end of 2011, there have been 164 peer-reviewed papers reporting randomised controlled trials (RCTs) in homeopathy. This represents research in 89 different medical conditions. Of those 164 RCT papers, 71 (43%) were positive, 9 (6%) negative and 80 (49%) non-conclusive.
This misleading nonsense was, of course, warmly received by homeopaths. The British Homeopathic Association, like many other organisations and individuals with an axe to grind lapped up the message and promptly repeated it: The body of evidence that exists shows that much more investigation is required – 43% of all the randomised controlled trials carried out have been positive, 6% negative and 49% inconclusive.
Let’s be clear what has happened here: the true percentage figures seem to show that 43% of studies (mostly of poor quality) suggest a positive result for homeopathy, while 57% of them (on average the ones of better quality) were negative. In other words, the majority of this evidence is negative. If we conducted a proper systematic review of this body of evidence, we would, of course, have to account for the quality of each study, and in this case we would have to conclude that homeopathy is not supported by sound evidence of effectiveness.
The little trick of applying the ‘NON-CONCLUSIVE’ method has thus turned this overall result upside down: black has become white! No wonder that it is so popular with proponents of all sorts of bogus treatments.
Chiropractors are notorious for their overuse and misuse of x-rays for non-specific back and neck pain as well as other conditions. A recent study from the US has shown that the rate of spine radiographs within 5 days of an initial patient visit to a chiropractor is 204 per 1000 new patient examinations. Considering that X-rays are not usually necessary for patients with non-specific back pain, such rates are far too high. Therefore, a team of US/Canadian researchers conducted a study to evaluate the impact of web-based dissemination of a diagnostic imaging guideline discouraging the use of spine x-rays among chiropractors.
They disseminated an imaging guideline online in April 2008. Administrative claims data were extracted between January 2006 and December 2010. Segmented regression analysis with autoregressive error was used to estimate the impact of guideline recommendations on the rate of spine x-rays. Sensitivity analysis considered the effect of two additional quality improvement strategies, a policy change and an education intervention.
The results show a significant change in the level of spine x-ray ordering weeks after introduction of the guidelines (-0.01; 95% confidence interval=-0.01, -0.002; p=.01), but no change in trend of the regression lines. The monthly mean rate of spine x-rays within 5 days of initial visit per new patient exams decreased by 10 per 1000, a 5.26% relative decrease after guideline dissemination.
The authors concluded that Web-based guideline dissemination was associated with an immediate reduction in spine x-ray claims. Sensitivity analysis suggests our results are robust. This passive strategy is likely cost-effective in a chiropractic network setting.
These findings are encouraging because they suggest that at least some chiropractors are capable of learning, even if this means altering their practice against their financial interests – after all, there is money to be earned with x-ray investigations! At the same time, the results indicate that, despite sound evidence, chiropractors still order far too many x-rays for non-specific back pain. I am not aware of any recent UK data on chiropractic x-ray usage, but judging from old evidence, it might be very high.
It would be interesting to know why chiropractors order spinal x-rays for patients with non-specific back pain or other conditions. A likely answer is that they need them for the diagnosis of spinal ‘subluxations’. To cite just one of thousands of chiropractors with the same opinion: spinography is a necessary part of the chiropractic examination. Detailed analysis of spinographic film and motion x-ray studies helps facilitate a specific and timely correction of vertebral subluxation by the Doctor of Chiropractic. The correction of a vertebral subluxation is called: Adjustment.
This, of course, merely highlights the futility of this practice: despite the fact that the concept is still deeply engrained in the teaching of chiropractic, ‘subluxation’ is a mystical entity or dogma which “is similar to the Santa Claus construct”, characterised by a “significant lack of evidence to fulfil the basic criteria of causation”. But even if chiropractic ‘subluxation’ were real, it would not be diagnosable with spinal x-ray investigations.
The inescapable conclusion from all this, I believe, is that the sooner chiropractors abandon their over-use of x-ray studies, the better for us all.
THERE WILL NEVER BE AN ALTERNATIVE CANCER CURE
This statement contradicts all those thousands of messages on the Internet that pretend otherwise. Far too many ‘entrepreneurs’ are trying to exploit desperate cancer patients by making claims about alternative cancer ‘cures’ ranging from shark oil to laetrile and from Essiac to mistletoe. The truth is that none of them are anything other than bogus.
Why? Let me explain.
If ever a curative cancer treatment emerged from the realm of alternative medicine that showed any promise at all, it would be very quickly researched by scientists and, if the results were positive, instantly adopted by mainstream oncology. The notion of an alternative cancer cure is therefore a contradiction in terms. It implies that oncologists are mean bastards who would, in the face of immense suffering, reject a promising cure simply because it did not originate from their own ranks.
BUT THAT DOES NOT NECESSARILY MEAN THAT ALTERNATIVE CANCER TREATMENTS ARE USELESS
So, let’s forget about alternative cancer ‘cures’ and let’s once and for all declare the people who sell or promote them as charlatans of the worst type. But some alternative therapies might nevertheless have a role in oncology – not as curative treatments but as supportive or palliative therapies.
The aim of supportive or palliative cancer care is not to cure the disease but to ease the suffering of cancer patients. According to my own research, promising evidence exists in this context, for instance, for massage, guided imagery, Co-enzyme Q10, acupuncture for nausea, and relaxation therapies. For other alternative therapies, the evidence is not supportive, e.g. reflexology, tai chi, homeopathy, spiritual healing, acupuncture for pain-relief, and aromatherapy.
So, in the realm of supportive and palliative care there is both encouraging as well as disappointing evidence. But what amazes me over and over again is the fact that the majority of cancer centres employing alternative therapies seem to bother very little about the evidence; they tend to use a weird mix of treatments regardless of whether they are backed by evidence or not. If patients like them, all is fine, they seem to think. I find this argument worrying.
Of course, every measure that increases the well-being of cancer patients must be welcome. But this should not mean that we disregard priorities or adopt any quackery that is on offer. In the interest of patients, we need to spend the available resources in the most effective ways. Those who argue that a bit of Reiki or reflexology, for example, is useful - if only via a non-specific (placebo) effects - seem to forget that we do not require quackery for patients to benefit from a placebo-response. An evidence-based treatment that is administered with kindness and compassion also generates specific non-specific effects. In addition, such treatments also generate specific effects. Therefore it would be a disservice to patients to merely rely on the non-specific effects of bogus treatments, even if the patients do experience some benefit from them.
ALTERNATIVE ‘PAMPERING’ AS A COMPENSATION FOR INADEQUACIES IN THE SYSTEM?
So, why are unproven or disproven treatments like Reiki or reflexology so popular for cancer palliation? This question has puzzled me for years, and I sometimes wonder whether some oncologists’ tolerance of quackery is not an attempt to compensate for any inadequacies within the routine service they deliver to their patients. Sub-standard care, unappetising food, insufficient pain-control, lack of time and compassion as well as other problems undoubtedly exist in some cancer units. It might be tempting to assume that such deficiencies can be compensated by a little pampering from a reflexologist or Reiki master. And it might be easier to hire a few alternative therapists for treating patients with agreeable yet ineffective interventions than to remedy the deficits that may exist in basic conventional care.
But this strategy would be wrong, unethical and counter-productive. Empathy, sympathy and compassion are core features of conventional care and must not be delegated to quacks.
A recent interview on alternative medicine for the German magazine DER SPIEGEL prompted well over 500 comments; even though, in the interview, I covered numerous alternative therapies, the discussion that followed focussed almost entirely on homeopathy. Yet again, many of the comments provided a reminder of the quasi-religious faith many people have in homeopathy.
There can, of course, be dozens of reasons for such strong convictions. Yet, in my experience, some seem to be more prevalent and important than others. During my last two decades in researching homeopathy, I think, I have identified several of the most important ones. In this post, I try to outline a typical sequence of events that eventually leads to a faith in homeopathy which is utterly immune to fact and reason.
The starting point of this journey towards homeopathy-worship is usually an impressive personal experience which is often akin to an epiphany (defined as a moment of sudden and great revelation or realization). I have met hundreds of advocates of homeopathy, and those who talk about this sort of thing invariably offer impressive stories about how they metamorphosed from being a ‘sceptic’ (yes, it is truly phenomenal how many believers insist that they started out as sceptics) into someone who was completely bowled over by homeopathy, and how that ‘moment of great revelation’ changed the rest of their lives. Very often, this ’Saulus-Paulus conversion’ relates to that person’s own (or a close friend’s) illness which allegedly was cured by homeopathy.
Rachel Roberts, chief executive of the Homeopathy Research Institute, provides as good an example of this sort of epiphany as anyone; in an article in THE GUARDIAN, she described her conversion to homeopathy with the following words:
I was a dedicated scientist about to begin a PhD in neuroscience when, out of the blue, homeopathy bit me on the proverbial bottom.
Science had been my passion since I began studying biology with Mr Hopkinson at the age of 11, and by the age of 21, when I attended the dinner party that altered the course of my life, I had still barely heard of it. The idea that I would one day become a homeopath would have seemed ludicrous.
That turning point is etched in my mind. A woman I’d known my entire life told me that a homeopath had successfully treated her when many months of conventional treatment had failed. As a sceptic, I scoffed, but was nonetheless a little intrigued.
She confessed that despite thinking homeopathy was a load of rubbish, she’d finally agreed to an appointment, to stop her daughter nagging. But she was genuinely shocked to find that, after one little pill, within days she felt significantly better. A second tablet, she said, “saw it off completely”.
I admit I ruined that dinner party. I interrogated her about every detail of her diagnosis, previous treatment, time scales, the lot. I thought it through logically – she was intelligent, she wasn’t lying, she had no previous inclination towards alternative medicine, and her reluctance would have diminished any placebo effect.
Scientists are supposed to make unprejudiced observations, then draw conclusions. As I thought about this, I was left with the highly uncomfortable conclusion that homeopathy appeared to have worked. I had to find out more.
So, I started reading about homeopathy, and what I discovered shifted my world for ever. I became convinced enough to hand my coveted PhD studentship over to my best friend and sign on for a three-year, full-time homeopathy training course.
Now, as an experienced homeopath, it is “science” that is biting me on the bottom. I know homeopathy works…
As I said, I have heard many strikingly similar accounts. Some of these tales seem a little too tall to be true and might be a trifle exaggerated, but the consistency of the picture that emerges from all of these stories is nevertheless extraordinary: people get started on a single anecdote which they are prepared to experience as an epiphanic turn-around. Subsequently, they are on a mission of confirming their new-found belief over and over again, until they become undoubting disciples for life.
So what? you might ask. But I do think this epiphany-like event at the outset of a homeopathic career is significant. In no other area of health care does the initial anecdote regularly play such a prominent role. People do not become believers in aspirin, for instance, on the basis of a ‘moment of great revelation’, they may take it because of the evidence. And, if there is a discrepancy between the external evidence and their own experience, as with homeopathy, most people would start to reflect: What other explanations exist to rationalise the anecdote? Invariably, there are many (placebo, natural history of the condition, concomitant events etc.).
Epiphany-stuck believers spends much time and effort to actively look for similar stories that seem to confirm the initial anecdote. They might, for instance, recommend or administer or prescribe homeopathy to others, many of whom would report positive outcomes. At the same time, all anecdotes that do not happen to fit the belief are brushed aside, forgotten, supressed, belittled, decried etc. This process leads to confirmation after confirmation after confirmation - and gradually builds up to what proponents of homeopathy would call ‘years of experience’. And ‘years of experience’ can, of course, not be wrong!
Again, believers neglect to question, doubt and rationalise their own perceptions. They ignore the fact that years of experience might just be little more than a suborn insistence on repeating one’s own mistakes. Even the most obvious confounders such as selective memory or alternative causes for positive clinical outcomes are quickly dismissed or not even considered at all.
Avoiding cognitive dissonance at all cost
But believers still has to somehow deal with the scientific facts about homeopathy; and these are, of course, grossly out of line with their belief. Thus the external evidence and the internal belief would inevitably clash creating a shrill cognitive dissonance. This must be avoided at all cost, as it might threaten the believer’s peace of mind. And the solution is amazingly simple: scientific evidence that does not confirm the believer’s conviction is ignored or, when this proves to be impossible, turned upside down.
Rachel Roberts’ account is most enlightening also in this repect:
And yet I keep reading reports in the media saying that homeopathy doesn’t work and that this scientific evidence doesn’t exist.
The facts, it seems, are being ignored. By the end of 2009, 142 randomised control trials (the gold standard in medical research) comparing homeopathy with placebo or conventional treatment had been published in peer-reviewed journals – 74 were able to draw firm conclusions: 63 were positive for homeopathy and 11 were negative. Five major systematic reviews have also been carried out to analyse the balance of evidence from RCTs of homeopathy – four were positive (Kleijnen, J, et al; Linde, K, et al; Linde, K, et al; Cucherat, M, et al) and one was negative (Shang, A et al). It’s usual to get mixed results when you look at a wide range of research results on one subject, and if these results were from trials measuring the efficacy of “normal” conventional drugs, ratios of 63:11 and 4:1 in favour of a treatment working would be considered pretty persuasive.
This statement is, in my view, a classic example of a desperate misinterpretation of the truth as a means of preventing the believer’s house of cards from collapsing. It even makes the hilarious claim that not the believers but the doubters “ignore” the facts.
In order to be able to adhere to her belief, Roberts needs to rely on a woefully biased white-wash from the ‘British Homeopathic Association’. And, in order to be on the safe side, she even quotes it misleadingly. The conclusion of the Cucherat review, for instance, can only be seen as positive by most blinkered of minds: There is some evidence that homeopathic treatments are more effective than placebo; however, the strength of this evidence is low because of the low methodological quality of the trials. Studies of high methodological quality were more likely to be negative than the lower quality studies. Further high quality studies are needed to confirm these results. Contrary to what Roberts states, there are at least a dozen more than 5 systematic reviews of homeopathy; my own systematic review of systematic reviews, for example, concluded that the best clinical evidence for homeopathy available to date does not warrant positive recommendations for its use in clinical practice.
It seems that, at this stage of a believer’s development, the truth gets all too happily sacrificed on the altar of faith. All these ‘ex-sceptics’ turned believers are now able to display is a rather comical parody of scepticism.
The delusional end-stage
The last stage in the career of a believer has been reached when hardly anything that he or she is convinced of resembles reality any longer. I don’t know much about Rachel Roberts, and she might not have reached this point yet; but there are many others who clearly have.
My two favourite examples of end-stage homeopathic delusionists are John Benneth and Dana Ullman. The final stage on the journey from ‘sceptic scientist’ to delusional disciple is characterised by an incessant stream of incoherent statements of vile nonsense that beggars belief. It is therefore easy to recognise and, because nobody can possibly take the delusionists seriously, they are best viewed as relatively harmless contributors to medical comedy.
Why does all of this matter?
Many homeopathy-fans are quasi-religious believers who, in my experience, have degressed way beyond reason. It is therefore a complete waste of time trying to reason with them. Initiated by a highly emotional epiphany, their faith cannot be shaken by rational arguments. Similar but usually less pronounced attitudes, I am afraid, can be observed in true believers of other alternative treatments as well (here I have chosen the example of homeopathy mainly because it is the area where things are most explicit).
True believers claim to have started out as sceptics and they often insist to be driven by a scientific mind. Yet I have never seen any evidence for these assumptions. On the contrary, for a relatively trivial episode to become a life-changing epiphany, the believer’s mind needs to be lamentably unscientific, unquestioning and simple.
In my experience, true believers will not change their mind; I have never seen this happening. However, progress might nevertheless be made, if we managed to instil a more (self-) questioning rationality and scientific attitudes into the minds of the next generations. In other words, we need better education in science and more training of critical thinking during their formative years.
Alternative medicine thrives in the realm of common chronic conditions which conventional medicine cannot cure and which respond well to treatment with placebos. Irritable bowel syndrome (IBS) is such a condition, and IBS-sufferers who are often frustrated with the symptomatic relief conventional medicine has to offer are only too keen to try any therapy that promises help. There is hardly an alternative therapy which does not claim to be the solution to IBS-symptoms: herbal medicine, mind-body interventions, homeopathy (the subject of my next post), acupuncture, even ‘MOXIBUSTION‘.
Moxibustion is a derivative of acupuncture; instead of needles, this method employs heat to stimulate acupuncture points. Proponents believe that the effects of moxibustion are roughly equivalent to those of acupuncture but many acupuncturists feel that they are less powerful. One website explains: Moxibustion is a traditional Chinese medicine technique that involves the burning of mugwort, a small, spongy herb, to facilitate healing. Moxibustion has been used throughout Asia for thousands of years; in fact, the actual Chinese character for acupuncture, translated literally, means “acupuncture-moxibustion.” The purpose of moxibustion, as with most forms of traditional Chinese medicine, is to strengthen the blood, stimulate the flow of qi, and maintain general health.
Many proponents of moxibustion claim that their treatment works for IBS. The evidence is, however, far less clear. Two recent meta-analyses might tell us more.
The first systematic review and meta-analysis was published by Korean researchers and aimed at critically evaluating the current evidence on moxibustion for improving global symptoms of IBS. The authors conducted extensive searches and found a total of 20 RCTs to be included in their analyses. The risk of bias in these studies was generally high. Compared with pharmacological medications, moxibustion significantly alleviated overall IBS symptoms but there was a moderate inconsistency among the 7 RCTs. Moxibustion combined with acupuncture was more effective than pharmacological therapy but a moderate inconsistency among the 4 studies was found. When moxibustion was added to pharmacological medications or herbal medicine, no additive benefit of moxibustion was shown compared with pharmacological medications or herbal medicine alone. One small sham-controlled trial found no difference between moxibustion and sham control in symptom severity. Moxibustion appeared to be associated with few adverse events but the evidence is limited due to poor reporting.
The authors concluded that moxibustion may provide benefit to IBS patients although the risk of bias in the included studies is relatively high. Future studies are necessary to confirm whether this finding is reproducible in carefully-designed and conducted trials and to firmly establish the place of moxibustion in current practice.
The way I see it, these conclusions are far too optimistic. There was only one RCT that controlled for placebo-effects, and the results of that study were negative. Thus I would conclude that some studies report effectiveness of moxibustion for IBS, yet the effects seem not to be caused by the treatment per se but are most likely due to a placebo-effect.
The second systematic review and meta-analysis was published by Chinese researchers and aimed at evaluating the clinical efficacy and safety of moxibustion and acupuncture in treatment of IBS. The authors included randomized and quasi-randomized clinical trials in their analyses and were able to include 11 trials. Their meta analysis suggests that the effectiveness of the combined methods of acupuncture and moxibustion is superior to conventional western medication treatment. The authors concluded that acupuncture-moxibustion for IBS is better than the conventional western medication treatment.
While the first meta-analysis was at least technically sound, the second seems to have too many flaws to mention: the search methodology was flimsy, many available studies were not included, their risk of bias was not assessed critically, the conclusions are based more on wishful thinking than on the available data, etc.
If we consider that moxibustion is a method of stimulating acupoints, we have to assume that it can at best be as effective as acupuncture, quite possibly slightly less. Thus it is relevant to see what the evidence tells us about acupuncture for IBS. The current Cochrane review of acupuncture for IBS shows that sham-controlled RCTs have found no benefits of acupuncture relative to a credible sham acupuncture control for IBS symptom severity or IBS-related quality of life.
I think I rest my case.
If we ask how effective spinal manipulation is as a treatment of back pain, we get all sorts of answers. Therapists who earn their money with it – mostly chiropractors, osteopaths and physiotherapists - are obviously convinced that it is effective. But if we consult more objective sources, the picture changes dramatically. The current Cochrane review, for instance, arrives at this conclusion: SMT is no more effective in participants with acute low-back pain than inert interventions, sham SMT, or when added to another intervention. SMT also appears to be no better than other recommended therapies.
Such reviews tend to pool all studies together regardless of the nature of the practitioner. But perhaps one type of clinician is better than the next? Certainly many chiropractors are on record claiming that they are the best at spinal manipulations. Yet it is conceivable that physiotherapists who do manipulations without being guided by the myth of ‘adjusting subluxations’ have an advantage over chiropractors. Three very recent systematic reviews might go some way to answer these questions.
The purpose of the first systematic review was to examine the effectiveness of spinal manipulations performed by physiotherapists for the treatment of patients with low back pain. The authors found 6 RCTs that met their inclusion criteria. The most commonly used outcomes were pain rating scales and disability indexes. Notable results included varying degrees of effect sizes favouring spinal manipulations and minimal adverse events resulting from this intervention. Additionally, the manipulation group in one study reported significantly less medication use, health care utilization, and lost work time. The authors concluded that there is evidence to support the use of spinal manipulation by physical therapists in clinical practice. Physical therapy spinal manipulation appears to be a safe intervention that improves clinical outcomes for patients with low back pain.
The second systematic Review was of osteopathic intervention for chronic, non-specific low back pain (CNSLBP). Only two trials met the authors’ inclusion criteria. They had a lack of methodological and clinical homogeneity, precluding a meta-analysis. The trials used different comparators with regards to the primary outcomes, the number of treatments, the duration of treatment and the duration of follow-up. The authors drew the following conclusions: There are only two studies assessing the effect of the manual therapy intervention applied by osteopathic clinicians in adults with CNSLBP. One trial concluded that the osteopathic intervention was similar in effect to a sham intervention, and the other suggests similarity of effect between osteopathic intervention, exercise and physiotherapy. Further clinical trials into this subject are required that have consistent and rigorous methods. These trials need to include an appropriate control and utilise an intervention that reflects actual practice.
The third systematic review sought to determine the benefits of chiropractic treatment and care for back pain on well-being, and aimed to explore to what extent chiropractic treatment and care improve quality of life. The authors identified 6 studies (4 RCTs and two observational studies) of varying quality. There was a high degree of inconsistency and lack of standardisation in measurement instruments and outcome measures. Three studies reported reduced use of other/extra treatments as a positive outcome; two studies reported a positive effect of chiropractic intervention on pain, and two studies reported a positive effect on disability. The authors concluded that it is difficult to defend any conclusion about the impact of chiropractic intervention on the quality of life, lifestyle, health and economic impact on chiropractic patients presenting with back pain.
Yes, yes, yes, I know: the three reviews are not exactly comparable; so we cannot draw firm conclusions from comparing them. Five points seem to emerge nevertheless:
- The evidence for spinal manipulation as a treatment for back pain is generally not brilliant, regardless of the type of therapist.
- There seem to be considerable differences according to the nature of the therapist.
- Physiotherapists seem to have relatively sound evidence to justify their manipulations.
- Chiropractors and osteopaths are not backed by evidence which is as reliable as they so often try to make us believe.
- Considering that the vast majority of serious complications after spinal manipulation has occurred with chiropractors, it would seem that chiropractors are the profession with the worst track record regarding manipulation for back pain.
Some sceptics are convinced that, in alternative medicine, there is no evidence. This assumption is wrong, I am afraid, and statements of this nature can actually play into the hands of apologists of bogus treatments: they can then easily demonstrate the sceptics to be mistaken or “biased”, as they would probably say. The truth is that there is plenty of evidence – and lots of it is positive, at least at first glance.
Alternative medicine researchers have been very industrious during the last two decades to build up a sizable body of ‘evidence’. Consequently, one often finds data even for the most bizarre and implausible treatments. Take, for instance, the claim that homeopathy is an effective treatment for cancer. Those who promote this assumption have no difficulties in locating some weird in-vitro study that seems to support their opinion. When sceptics subsequently counter that in-vitro experiments tell us nothing about the clinical situation, apologists quickly unearth what they consider to be sound clinical evidence.
An example is this prospective observational 2011 study of cancer patients from two differently treated cohorts: one cohort with patients under complementary homeopathic treatment (HG; n = 259), and one cohort with conventionally treated cancer patients (CG; n = 380). Its main outcome measures were the change of quality life after 3 months, after one year and impairment by fatigue, anxiety or depression. The results of this study show significant improvements in most of these endpoints, and the authors concluded that we observed an improvement of quality of life as well as a tendency of fatigue symptoms to decrease in cancer patients under complementary homeopathic treatment.
Another, in some ways even better example is this 2005 observational study of 6544 consecutive patients from the Bristol Homeopathic Hospital. Every patient attending the hospital outpatient unit for a follow-up appointment was included, commencing with their first follow-up attendance. Of these patients 70.7% (n = 4627) reported positive health changes, with 50.7% (n = 3318) recording their improvement as better or much better. The authors concluded that homeopathic intervention offered positive health changes to a substantial proportion of a large cohort of patients with a wide range of chronic diseases.
The principle that is being followed here is simple:
- believers in a bogus therapy conduct a clinical trial which is designed to generate an apparently positive finding;
- the fact that the study cannot tell us anything about cause and effect is cleverly hidden or belittled;
- they publish their findings in one of the many journals that specialise in this sort of nonsense;
- they make sure that advocates across the world learn about their results;
- the community of apologists of this treatment picks up the information without the slightest critical analysis;
- the researchers conduct more and more of such pseudo-research;
- nobody attempts to do some real science: the believers do not truly want to falsify their hypotheses, and the real scientists find it unreasonable to conduct research on utterly implausible interventions;
- thus the body of false or misleading ‘evidence’ grows and grows;
- proponents start publishing systematic reviews and meta-analyses of their studies which are devoid of critical input;
- too few critics point out that these reviews are fatally flawed – ‘rubbish in, rubbish out’!
- eventually politicians, journalists, health care professionals and other people who did not necessarily start out as believers in the bogus therapy are convinced that the body of evidence is impressive and justifies implementation;
- important health care decisions are thus based on data which are false and misleading.
So, what can be done to prevent that such pseudo-evidence is mistaken as solid proof which might eventually mislead many into believing that bogus treatments are based on reasonably sound data? I think the following measures would be helpful:
- authors should abstain from publishing over-enthusiastic conclusions which can all too easily be misinterpreted (given that the authors are believers in the therapy, this is not a realistic option);
- editors might consider rejecting studies which contribute next to nothing to our current knowledge (given that these studies are usually published in journals that are in the business of promoting alternative medicine at any cost, this option is also not realistic);
- if researchers report highly preliminary findings, there should be an obligation to do further studies in order to confirm or refute the initial results (not realistic either, I am afraid);
- in case this does not happen, editors should consider retracting the paper reporting unconfirmed preliminary findings (utterly unrealistic).
What then can REALISTICALLY be done? I wish I knew the answer! All I can think of is that sceptics should educate the rest of the population to think and analyse such ’evidence’ critically…but how realistic is that?
Has it ever occurred to you that much of the discussion about cause and effect in alternative medicine goes in circles without ever making progress? I have come to the conclusion that it does. Here I try to illustrate this point using the example of acupuncture, more precisely the endless discussion about how to best test acupuncture for efficacy. For those readers who like to misunderstand me I should explain that the sceptics’ view is in capital letters.
At the beginning there was the experience. Unaware of anatomy, physiology, pathology etc., people started sticking needles in other people’s skin, some 2000 years ago, and observed that they experienced relief of all sorts of symptoms.When an American journalist reported about this phenomenon in the 1970s, acupuncture became all the rage in the West. Acupuncture-fans then claimed that a 2000-year history is ample proof that acupuncture does work.
BUT ANECDOTES ARE NOTORIOUSLY UNRELIABLE!
Even the most enthusiastic advocates conceded that this is probably true. So they documented detailed case-series of lots of patients, calculated the average difference between the pre- and post-treatment severity of symptoms, submitted it to statistical tests, and published the notion that the effects of acupuncture are not just anecdotal; in fact, they are statistically significant, they said.
BUT THIS EFFECT COULD BE DUE TO THE NATURAL HISTORY OF THE CONDITION!
“True enough”, grumbled the acupuncture-fans and conducted the very first controlled clinical trials. Essentially they treated one group of patients with acupuncture while another group received conventional treatments as usual. When they analysed the results, they found that the acupuncture group had improved significantly more. “Now do you believe us?”, they asked triumphantly, “acupuncture is clearly effective”.
NO! THIS OUTCOME MIGHT BE DUE TO SELECTION BIAS. SUCH A STUDY-DESIGN CANNOT ESTABLISH CAUSE AND EFFECT.
The acupuncturists felt slightly embarrassed because they had not thought of that. They had allocated their patients to the treatment according to patients’ choice. Thus the expectation of the patients (or the clinician) to get relief from acupuncture might have been the reason for the difference in outcome. So they consulted an expert in trial-design and were advised to allocate not by choice but by chance. In other words, they repeated the previous study but randomised patients to the two groups. Amazingly, their RCT still found a significant difference favouring acupuncture over treatment as usual.
BUT THIS DIFFERENCE COULD BE CAUSED BY A PLACEBO-EFFECT!
Now the acupuncturists were in a bit of a pickle; as far as they could see, there was no good placebo for acupuncture! Eventually some methodologist-chap came up with the idea that, in order to mimic a placebo, they could simply stick needles into non-acupuncture points. When the acupuncturists tried that method, they found that there were improvements in both groups but the difference between real acupuncture and placebo was tiny and usually neither statistically significant nor clinically relevant.
NOW DO YOU CONCEDE THAT ACUPUNCTURE IS NOT AN EFFECTIVE TREATMENT?
Absolutely not! The results merely show that needling non-acupuncture points is not an adequate placebo. Obviously this intervention also sends a powerful signal to the brain which clearly makes it an effective intervention. What do you expect when you compare two effective treatments?
IF YOU REALLY THINK SO, YOU NEED TO PROVE IT AND DESIGN A PLACEBO THAT IS INERT.
At that stage, the acupuncturists came up with a placebo-needle that did not actually penetrate the skin; it worked like a mini stage dagger that telescopes into itself while giving the impression that it penetrated the skin just like the real thing. Surely this was an adequate placebo! The acupuncturists repeated their studies but, to their utter dismay, they found again that both groups improved and the difference in outcome between their new placebo and true acupuncture was minimal.
WE TOLD YOU THAT ACUPUNCTURE WAS NOT EFFECTIVE! DO YOU FINALLY AGREE?
Certainly not, they replied. We have thought long and hard about these intriguing findings and believe that they can be explained just like the last set of results: the non-penetrating needles touch the skin; this touch provides a stimulus powerful enough to have an effect on the brain; the non-penetrating placebo-needles are not inert and therefore the results merely depict a comparison of two effective treatments.
YOU MUST BE JOKING! HOW ARE YOU GOING TO PROVE THAT BIZARRE HYPOTHESIS?
We had many discussions and consensus meeting amongst the most brilliant brains in acupuncture about this issue and have arrived at the conclusion that your obsession with placebo, cause and effect etc. is ridiculous and entirely misplaced. In real life, we don’t use placebos. So, let’s instead address the ‘real life’ question: is acupuncture better than usual treatment? We have conducted pragmatic studies where one group of patients gets treatment as usual and the other group receives acupuncture in addition. These studies show that acupuncture is effective. This is all the evidence we need. Why can you not believe us?
NOW WE HAVE ARRIVED EXACTLY AT THE POINT WHERE WE HAVE BEEN A LONG TIME AGO. SUCH A STUDY-DESIGN CANNOT ESTABLISH CAUSE AND EFFECT. YOU OBVIOUSLY CANNOT DEMONSTRATE THAT ACUPUNCTURE CAUSES CLINICAL IMPROVEMENT. THEREFORE YOU OPT TO PRETEND THAT CAUSE AND EFFECT ARE IRRELEVANT. YOU USE SOME IMITATION OF SCIENCE TO ‘PROVE’ THAT YOUR PRECONCEIVED IDEAS ARE CORRECT. YOU DO NOT SEEM TO BE INTERESTED IN THE TRUTH ABOUT ACUPUNCTURE AT ALL.