MD, PhD, FMedSci, FSB, FRCP, FRCPEd

evidence

Chiropractors are notorious for their overuse and misuse of x-rays for non-specific back and neck pain as well as other conditions. A recent study from the US has shown that the rate of spine radiographs within 5 days of an initial patient visit to a chiropractor is 204 per 1000 new patient examinations. Considering that X-rays are not usually necessary for patients with non-specific back pain, such rates are far too high. Therefore, a team of US/Canadian researchers conducted a study to evaluate the impact of web-based dissemination of a diagnostic imaging guideline discouraging the use of spine x-rays among chiropractors.

They disseminated an imaging guideline online in April 2008. Administrative claims data were extracted between January 2006 and December 2010. Segmented regression analysis with autoregressive error was used to estimate the impact of guideline recommendations on the rate of spine x-rays. Sensitivity analysis considered the effect of two additional quality improvement strategies, a policy change and an education intervention.

The results show a significant change in the level of spine x-ray ordering weeks after introduction of the guidelines (-0.01; 95% confidence interval=-0.01, -0.002; p=.01), but no change in trend of the regression lines. The monthly mean rate of spine x-rays within 5 days of initial visit per new patient exams decreased by 10 per 1000, a 5.26% relative decrease after guideline dissemination.

The authors concluded that Web-based guideline dissemination was associated with an immediate reduction in spine x-ray claims. Sensitivity analysis suggests our results are robust. This passive strategy is likely cost-effective in a chiropractic network setting.

These findings are encouraging because they suggest that at least some chiropractors are capable of learning, even if this means altering their practice against their financial interests – after all, there is money to be earned with x-ray investigations! At the same time, the results indicate that, despite sound evidence, chiropractors still order far too many x-rays for non-specific back pain. I am not aware of any recent UK data on chiropractic x-ray usage, but judging from old evidence, it might be very high.

It would be interesting to know why chiropractors order spinal x-rays for patients with non-specific back pain or other conditions. A likely answer is that they need them for the diagnosis of spinal ‘subluxations’. To cite just one of thousands of chiropractors with the same opinion: spinography is a necessary part of the chiropractic examination. Detailed analysis of spinographic film and motion x-ray studies helps facilitate a specific and timely correction of vertebral subluxation by the Doctor of Chiropractic. The correction of a vertebral subluxation is called: Adjustment.

This, of course, merely highlights the futility of this practice: despite the fact that the concept is still deeply engrained in the teaching of chiropractic, ‘subluxation’ is a mystical entity or dogma which “is similar to the Santa Claus construct”, characterised by a “significant lack of evidence to fulfil the basic criteria of causation”. But even if chiropractic ‘subluxation’ were real, it would not be diagnosable with spinal x-ray investigations.

The inescapable conclusion from all this, I believe, is that the sooner chiropractors abandon their over-use of x-ray studies, the better for us all.

THERE WILL NEVER BE AN ALTERNATIVE CANCER CURE

This statement contradicts all those thousands of messages on the Internet that pretend otherwise. Far too many ‘entrepreneurs’ are trying to exploit desperate cancer patients by making claims about alternative cancer ‘cures’ ranging from shark oil to laetrile and from Essiac to mistletoe. The truth is that none of them are anything other than bogus.

Why? Let me explain.

If ever a curative cancer treatment emerged from the realm of alternative medicine that showed any promise at all, it would be very quickly researched by scientists and, if the results were positive, instantly adopted by mainstream oncology. The notion of an alternative cancer cure is therefore a contradiction in terms. It implies that oncologists are mean bastards who would, in the face of immense suffering, reject a promising cure simply because it did not originate from their own ranks.

BUT THAT DOES NOT NECESSARILY MEAN THAT ALTERNATIVE CANCER TREATMENTS ARE USELESS

So, let’s forget about alternative cancer ‘cures’ and let’s once and for all declare the people who sell or promote them as charlatans of the worst type. But some alternative therapies might nevertheless have a role in oncology – not as curative treatments but as supportive or palliative therapies.

The aim of supportive or palliative cancer care is not to cure the disease but to ease the suffering of cancer patients. According to my own research, promising evidence exists in this context, for instance, for massage, guided imagery, Co-enzyme Q10, acupuncture for nausea, and relaxation therapies. For other alternative therapies, the evidence is not supportive, e.g. reflexology, tai chi, homeopathy, spiritual healing, acupuncture for pain-relief, and aromatherapy.

So, in the realm of supportive and palliative care there is both encouraging as well as disappointing evidence. But what amazes me over and over again is the fact that the majority of cancer centres employing alternative therapies seem to bother very little about the evidence; they tend to use a weird mix of treatments regardless of whether they are backed by evidence or not. If patients like them, all is fine, they seem to think. I find this argument worrying.

Of course, every measure that increases the well-being of cancer patients must be welcome. But this should not mean that we disregard priorities or adopt any quackery that is on offer. In the interest of patients, we need to spend the available resources in the most effective ways. Those who argue that a bit of Reiki or reflexology, for example, is useful – if only via a non-specific (placebo) effects – seem to forget that we do not require quackery for patients to benefit from a placebo-response. An evidence-based treatment that is administered with kindness and compassion also generates specific non-specific effects. In addition, such treatments also generate specific effects. Therefore it would be a disservice to patients to merely rely on the non-specific effects of bogus treatments, even if the patients do experience some benefit from them.

ALTERNATIVE ‘PAMPERING’ AS A COMPENSATION FOR INADEQUACIES IN THE SYSTEM?

So, why are unproven or disproven treatments like Reiki or reflexology so popular for cancer palliation? This question has puzzled me for years, and I sometimes wonder whether some oncologists’ tolerance of quackery is not an attempt to compensate for any inadequacies within the routine service they deliver to their patients. Sub-standard care, unappetising food, insufficient pain-control, lack of time and compassion as well as other problems undoubtedly exist in some cancer units. It might be tempting to assume that such deficiencies can be compensated by a little pampering from a reflexologist or Reiki master. And it might be easier to hire a few alternative therapists for treating patients with agreeable yet ineffective interventions than to remedy the deficits that may exist in basic conventional care.

But this strategy would be wrong, unethical and counter-productive. Empathy, sympathy and compassion are core features of conventional care and must not be delegated to quacks.

A recent interview on alternative medicine for the German magazine DER SPIEGEL prompted well over 500 comments; even though, in the interview, I covered numerous alternative therapies, the discussion that followed focussed almost entirely on homeopathy. Yet again, many of the comments provided a reminder of the quasi-religious faith many people have in homeopathy.

There can, of course, be dozens of reasons for such strong convictions. Yet, in my experience, some seem to be more prevalent and important than others. During my last two decades in researching homeopathy, I think, I have identified several of the most important ones. In this post, I try to outline a typical sequence of events that eventually leads to a faith in homeopathy which is utterly immune to fact and reason.

The epiphany

The starting point of this journey towards homeopathy-worship is usually an impressive personal experience which is often akin to an epiphany (defined as a moment of sudden and great revelation or realization). I have met hundreds of advocates of homeopathy, and those who talk about this sort of thing invariably offer impressive stories about how they metamorphosed from being a ‘sceptic’ (yes, it is truly phenomenal how many believers insist that they started out as sceptics) into someone who was completely bowled over by homeopathy, and how that ‘moment of great revelation’ changed the rest of their lives. Very often, this ‘Saulus-Paulus conversion’ relates to that person’s own (or a close friend’s) illness which allegedly was cured by homeopathy.

Rachel Roberts, chief executive of the Homeopathy Research Institute, provides as good an example of this sort of epiphany as anyone; in an article in THE GUARDIAN, she described her conversion to homeopathy with the following words:

I was a dedicated scientist about to begin a PhD in neuroscience when, out of the blue, homeopathy bit me on the proverbial bottom.

Science had been my passion since I began studying biology with Mr Hopkinson at the age of 11, and by the age of 21, when I attended the dinner party that altered the course of my life, I had still barely heard of it. The idea that I would one day become a homeopath would have seemed ludicrous.

That turning point is etched in my mind. A woman I’d known my entire life told me that a homeopath had successfully treated her when many months of conventional treatment had failed. As a sceptic, I scoffed, but was nonetheless a little intrigued.

She confessed that despite thinking homeopathy was a load of rubbish, she’d finally agreed to an appointment, to stop her daughter nagging. But she was genuinely shocked to find that, after one little pill, within days she felt significantly better. A second tablet, she said, “saw it off completely”.

I admit I ruined that dinner party. I interrogated her about every detail of her diagnosis, previous treatment, time scales, the lot. I thought it through logically – she was intelligent, she wasn’t lying, she had no previous inclination towards alternative medicine, and her reluctance would have diminished any placebo effect.

Scientists are supposed to make unprejudiced observations, then draw conclusions. As I thought about this, I was left with the highly uncomfortable conclusion that homeopathy appeared to have worked. I had to find out more.

So, I started reading about homeopathy, and what I discovered shifted my world for ever. I became convinced enough to hand my coveted PhD studentship over to my best friend and sign on for a three-year, full-time homeopathy training course.

Now, as an experienced homeopath, it is “science” that is biting me on the bottom. I know homeopathy works…

As I said, I have heard many strikingly similar accounts. Some of these tales seem a little too tall to be true and might be a trifle exaggerated, but the consistency of the picture that emerges from all of these stories is nevertheless extraordinary: people get started on a single anecdote which they are prepared to experience as an epiphanic turn-around. Subsequently, they are on a mission of confirming their new-found belief over and over again, until they become undoubting disciples for life.

So what? you might ask. But I do think this epiphany-like event at the outset of a homeopathic career is significant. In no other area of health care does the initial anecdote regularly play such a prominent role. People do not become believers in aspirin, for instance, on the basis of a ‘moment of great revelation’, they may take it because of the evidence. And, if there is a discrepancy between the external evidence and their own experience, as with homeopathy, most people would start to reflect: What other explanations exist to rationalise the anecdote? Invariably, there are many (placebo, natural history of the condition, concomitant events etc.).

Confirmation bias

Epiphany-stuck believers spends much time and effort to actively look for similar stories that seem to confirm the initial anecdote. They might, for instance, recommend or administer or prescribe homeopathy to others, many of whom would report positive outcomes. At the same time, all anecdotes that do not happen to fit the belief are brushed aside, forgotten, supressed, belittled, decried etc. This process leads to confirmation after confirmation after confirmation – and gradually builds up to what proponents of homeopathy would call ‘years of experience’. And ‘years of experience’ can, of course, not be wrong!

Again, believers neglect to question, doubt and rationalise their own perceptions. They ignore the fact that years of experience might just be little more than a suborn insistence on repeating one’s own mistakes. Even the most obvious confounders such as selective memory or alternative causes for positive clinical outcomes are quickly dismissed or not even considered at all.

Avoiding cognitive dissonance at all cost

But believers still has to somehow deal with the scientific facts about homeopathy; and these are, of course, grossly out of line with their belief. Thus the external evidence and the internal belief would inevitably clash creating a shrill cognitive dissonance. This must be avoided at all cost, as it might threaten the believer’s peace of mind. And the solution is amazingly simple: scientific evidence that does not confirm the believer’s conviction is ignored or, when this proves to be impossible, turned upside down.

Rachel Roberts’ account is most enlightening also in this repect:

And yet I keep reading reports in the media saying that homeopathy doesn’t work and that this scientific evidence doesn’t exist.

The facts, it seems, are being ignored. By the end of 2009, 142 randomised control trials (the gold standard in medical research) comparing homeopathy with placebo or conventional treatment had been published in peer-reviewed journals – 74 were able to draw firm conclusions: 63 were positive for homeopathy and 11 were negative. Five major systematic reviews have also been carried out to analyse the balance of evidence from RCTs of homeopathy – four were positive (Kleijnen, J, et al; Linde, K, et al; Linde, K, et al; Cucherat, M, et al) and one was negative (Shang, A et al). It’s usual to get mixed results when you look at a wide range of research results on one subject, and if these results were from trials measuring the efficacy of “normal” conventional drugs, ratios of 63:11 and 4:1 in favour of a treatment working would be considered pretty persuasive.

This statement is, in my view, a classic example of a desperate misinterpretation of the truth as a means of preventing the believer’s house of cards from collapsing. It even makes the hilarious claim that not the believers but the doubters “ignore” the facts.

In order to be able to adhere to her belief, Roberts needs to rely on a woefully biased white-wash from the ‘British Homeopathic Association’. And, in order to be on the safe side, she even quotes it misleadingly. The conclusion of the Cucherat review, for instance, can only be seen as positive by most blinkered of minds: There is some evidence that homeopathic treatments are more effective than placebo; however, the strength of this evidence is low because of the low methodological quality of the trials. Studies of high methodological quality were more likely to be negative than the lower quality studies. Further high quality studies are needed to confirm these results. Contrary to what Roberts states, there are at least a dozen more than 5 systematic reviews of homeopathy; my own systematic review of systematic reviews, for example, concluded that the best clinical evidence for homeopathy available to date does not warrant positive recommendations for its use in clinical practice.

It seems that, at this stage of a believer’s development, the truth gets all too happily sacrificed on the altar of faith. All these ‘ex-sceptics’ turned believers are now able to display is a rather comical parody of scepticism.

The delusional end-stage

The last stage in the career of a believer has been reached when hardly anything that he or she is convinced of resembles reality any longer. I don’t know much about Rachel Roberts, and she might not have reached this point yet; but there are many others who clearly have.

My two favourite examples of end-stage homeopathic delusionists are John Benneth and Dana Ullman. The final stage on the journey from ‘sceptic scientist’ to delusional disciple is characterised by an incessant stream of incoherent statements of vile nonsense that beggars belief. It is therefore easy to recognise and, because nobody can possibly take the delusionists seriously, they are best viewed as relatively harmless contributors to medical comedy.

Why does all of this matter?

Many homeopathy-fans are quasi-religious believers who, in my experience, have degressed way beyond reason. It is therefore a complete waste of time trying to reason with them. Initiated by a highly emotional epiphany, their faith cannot be shaken by rational arguments. Similar but usually less pronounced attitudes, I am afraid, can be observed in true believers of other alternative treatments as well (here I have chosen the example of homeopathy mainly because it is the area where things are most explicit).

True believers claim to have started out as sceptics and they often insist to be driven by a scientific mind. Yet I have never seen any evidence for these assumptions. On the contrary, for a relatively trivial episode to become a life-changing epiphany, the believer’s mind needs to be lamentably unscientific, unquestioning and simple.

In my experience, true believers will not change their mind; I have never seen this happening. However, progress might nevertheless be made, if we managed to instil a more (self-) questioning rationality and scientific attitudes into the minds of the next generations. In other words, we need better education in science and more training of critical thinking during their formative years.

Alternative medicine thrives in the realm of common chronic conditions which conventional medicine cannot cure and which respond well to treatment with placebos. Irritable bowel syndrome (IBS) is such a condition, and IBS-sufferers who are often frustrated with the symptomatic relief conventional medicine has to offer are only too keen to try any therapy that promises help. There is hardly an alternative therapy which does not claim to be the solution to IBS-symptoms: herbal medicine, mind-body interventions, homeopathy (the subject of my next post), acupuncture, even ‘MOXIBUSTION‘.

Moxibustion is a derivative of acupuncture; instead of needles, this method employs heat to stimulate acupuncture points. Proponents believe that the effects of moxibustion are roughly equivalent to those of acupuncture but many acupuncturists feel that they are less powerful. One website explains: Moxibustion is a traditional Chinese medicine technique that involves the burning of mugwort, a small, spongy herb, to facilitate healing. Moxibustion has been used throughout Asia for thousands of years; in fact, the actual Chinese character for acupuncture, translated literally, means “acupuncture-moxibustion.” The purpose of moxibustion, as with most forms of traditional Chinese medicine, is to strengthen the blood, stimulate the flow of qi, and maintain general health.

Many proponents of moxibustion claim that their treatment works for IBS. The evidence is, however, far less clear. Two recent meta-analyses might tell us more.

The first systematic review and meta-analysis was published by Korean researchers and aimed at critically evaluating the current evidence on moxibustion for improving global symptoms of IBS. The authors conducted extensive searches and found a total of 20 RCTs to be included in their analyses. The risk of bias in these studies was generally high. Compared with pharmacological medications, moxibustion significantly alleviated overall IBS symptoms but there was a moderate inconsistency among the 7 RCTs. Moxibustion combined with acupuncture was more effective than pharmacological therapy but a moderate inconsistency among the 4 studies was found. When moxibustion was added to pharmacological medications or herbal medicine, no additive benefit of moxibustion was shown compared with pharmacological medications or herbal medicine alone. One small sham-controlled trial found no difference between moxibustion and sham control in symptom severity. Moxibustion appeared to be associated with few adverse events but the evidence is limited due to poor reporting.

The authors concluded that moxibustion may provide benefit to IBS patients although the risk of bias in the included studies is relatively high. Future studies are necessary to confirm whether this finding is reproducible in carefully-designed and conducted trials and to firmly establish the place of moxibustion in current practice.

The way I see it, these conclusions are far too optimistic. There was only one RCT that controlled for placebo-effects, and the results of that study were negative. Thus I would conclude that some studies report effectiveness of moxibustion for IBS, yet the effects seem not to be caused by the treatment per se but are most likely due to a placebo-effect.

The second systematic review and meta-analysis was published by Chinese researchers and aimed at evaluating the clinical efficacy and safety of moxibustion and acupuncture in treatment of IBS. The authors included randomized and quasi-randomized clinical trials in their analyses and were able to include 11 trials. Their meta analysis suggests that the effectiveness of the combined methods of acupuncture and moxibustion is superior to conventional western medication treatment. The authors concluded that acupuncture-moxibustion for IBS is better than the conventional western medication treatment.

While the first meta-analysis was at least technically sound, the second seems to have too many flaws to mention: the search methodology was flimsy, many available studies were not included, their risk of bias was not assessed critically, the conclusions are based more on wishful thinking than on the available data, etc.

If we consider that moxibustion is a method of stimulating acupoints, we have to assume that it can at best be as effective as acupuncture, quite possibly slightly less. Thus it is relevant to see what the evidence tells us about acupuncture for IBS. The current Cochrane review of acupuncture for IBS shows that sham-controlled RCTs have found no benefits of acupuncture relative to a credible sham acupuncture control for IBS symptom severity or IBS-related quality of life.

I think I rest my case.

If we ask how effective spinal manipulation is as a treatment of back pain, we get all sorts of answers. Therapists who earn their money with it – mostly chiropractors, osteopaths and physiotherapists – are obviously convinced that it is effective. But if we consult more objective sources, the picture changes dramatically. The current Cochrane review, for instance, arrives at this conclusion: SMT is no more effective in participants with acute low-back pain than inert interventions, sham SMT, or when added to another intervention. SMT also appears to be no better than other recommended therapies.

Such reviews tend to pool all studies together regardless of the nature of the practitioner. But perhaps one type of clinician is better than the next? Certainly many chiropractors are on record claiming that they are the best at spinal manipulations. Yet it is conceivable that physiotherapists who do manipulations without being guided by the myth of ‘adjusting subluxations’ have an advantage over chiropractors. Three very recent systematic reviews might go some way to answer these questions.

The purpose of the first systematic review was to examine the effectiveness of spinal manipulations performed by physiotherapists for the treatment of patients with low back pain. The authors found 6 RCTs that met their inclusion criteria. The most commonly used outcomes were pain rating scales and disability indexes. Notable results included varying degrees of effect sizes favouring spinal manipulations and minimal adverse events resulting from this intervention. Additionally, the manipulation group in one study reported significantly less medication use, health care utilization, and lost work time. The authors concluded that there is evidence to support the use of spinal manipulation by physical therapists in clinical practice. Physical therapy spinal manipulation appears to be a safe intervention that improves clinical outcomes for patients with low back pain.

The second systematic Review was of osteopathic intervention for chronic, non-specific low back pain (CNSLBP). Only two trials met the authors’ inclusion criteria. They had a lack of methodological and clinical homogeneity, precluding a meta-analysis. The trials used different comparators with regards to the primary outcomes, the number of treatments, the duration of treatment and the duration of follow-up. The authors drew the following conclusions: There are only two studies assessing the effect of the manual therapy intervention applied by osteopathic clinicians in adults with CNSLBP. One trial concluded that the osteopathic intervention was similar in effect to a sham intervention, and the other suggests similarity of effect between osteopathic intervention, exercise and physiotherapy. Further clinical trials into this subject are required that have consistent and rigorous methods. These trials need to include an appropriate control and utilise an intervention that reflects actual practice.

The third systematic review sought to determine the benefits of chiropractic treatment and care for back pain on well-being, and aimed to explore to what extent chiropractic treatment and care improve quality of life. The authors identified 6 studies (4 RCTs and two observational studies) of varying quality. There was a high degree of inconsistency and lack of standardisation in measurement instruments and outcome measures. Three studies reported reduced use of other/extra treatments as a positive outcome; two studies reported a positive effect of chiropractic intervention on pain, and two studies reported a positive effect on disability. The authors concluded that it is difficult to defend any conclusion about the impact of chiropractic intervention on the quality of life, lifestyle, health and economic impact on chiropractic patients presenting with back pain.

Yes, yes, yes, I know: the three reviews are not exactly comparable; so we cannot draw firm conclusions from comparing them. Five points seem to emerge nevertheless:

  1. The evidence for spinal manipulation as a treatment for back pain is generally not brilliant, regardless of the type of therapist.
  2. There seem to be considerable differences according to the nature of the therapist.
  3. Physiotherapists seem to have relatively sound evidence to justify their manipulations.
  4. Chiropractors and osteopaths are not backed by evidence which is as reliable as they so often try to make us believe.
  5. Considering that the vast majority of serious complications after spinal manipulation has occurred with chiropractors, it would seem that chiropractors are the profession with the worst track record regarding manipulation for back pain.

Some sceptics are convinced that, in alternative medicine, there is no evidence. This assumption is wrong, I am afraid, and statements of this nature can actually play into the hands of apologists of bogus treatments: they can then easily demonstrate the sceptics to be mistaken or “biased”, as they would probably say. The truth is that there is plenty of evidence – and lots of it is positive, at least at first glance.

Alternative medicine researchers have been very industrious during the last two decades to build up a sizable body of ‘evidence’. Consequently, one often finds data even for the most bizarre and implausible treatments. Take, for instance, the claim that homeopathy is an effective treatment for cancer. Those who promote this assumption have no difficulties in locating some weird in-vitro study that seems to support their opinion. When sceptics subsequently counter that in-vitro experiments tell us nothing about the clinical situation, apologists quickly unearth what they consider to be sound clinical evidence.

An example is this prospective observational 2011 study of cancer patients from two differently treated cohorts: one cohort with patients under complementary homeopathic treatment (HG; n = 259), and one cohort with conventionally treated cancer patients (CG; n = 380). Its main outcome measures were the change of quality life after 3 months, after one year and impairment by fatigue, anxiety or depression. The results of this study show significant improvements in most of these endpoints, and the authors concluded that we observed an improvement of quality of life as well as a tendency of fatigue symptoms to decrease in cancer patients under complementary homeopathic treatment.

Another, in some ways even better example is this 2005 observational study of 6544 consecutive patients from the Bristol Homeopathic Hospital. Every patient attending the hospital outpatient unit for a follow-up appointment was included, commencing with their first follow-up attendance. Of these patients 70.7% (n = 4627) reported positive health changes, with 50.7% (n = 3318) recording their improvement as better or much better. The authors concluded that homeopathic intervention offered positive health changes to a substantial proportion of a large cohort of patients with a wide range of chronic diseases.

The principle that is being followed here is simple:

  • believers in a bogus therapy conduct a clinical trial which is designed to generate an apparently positive finding;
  • the fact that the study cannot tell us anything about cause and effect is cleverly hidden or belittled;
  • they publish their findings in one of the many journals that specialise in this sort of nonsense;
  • they make sure that advocates across the world learn about their results;
  • the community of apologists of this treatment picks up the information without the slightest critical analysis;
  • the researchers conduct more and more of such pseudo-research;
  • nobody attempts to do some real science: the believers do not truly want to falsify their hypotheses, and the real scientists find it unreasonable to conduct research on utterly implausible interventions;
  • thus the body of false or misleading ‘evidence’ grows and grows;
  • proponents start publishing systematic reviews and meta-analyses of their studies which are devoid of critical input;
  • too few critics point out that these reviews are fatally flawed – ‘rubbish in, rubbish out’!
  • eventually politicians, journalists, health care professionals and other people who did not necessarily start out as believers in the bogus therapy are convinced that the body of evidence is impressive and justifies implementation;
  • important health care decisions are thus based on data which are false and misleading.

So, what can be done to prevent that such pseudo-evidence is mistaken as solid proof which might eventually mislead many into believing that bogus treatments are based on reasonably sound data? I think the following measures would be helpful:

  • authors should abstain from publishing over-enthusiastic conclusions which can all too easily be misinterpreted (given that the authors are believers in the therapy, this is not a realistic option);
  • editors might consider rejecting studies which contribute next to nothing to our current knowledge (given that these studies are usually published in journals that are in the business of promoting alternative medicine at any cost, this option is also not realistic);
  • if researchers report highly preliminary findings, there should be an obligation to do further studies in order to confirm or refute the initial results (not realistic either, I am afraid);
  • in case this does not happen, editors should consider retracting the paper reporting unconfirmed preliminary findings (utterly unrealistic).

What then can REALISTICALLY be done? I wish I knew the answer! All I can think of is that sceptics should educate the rest of the population to think and analyse such ‘evidence’ critically…but how realistic is that?

Has it ever occurred to you that much of the discussion about cause and effect in alternative medicine goes in circles without ever making progress? I have come to the conclusion that it does. Here I try to illustrate this point using the example of acupuncture, more precisely the endless discussion about how to best test acupuncture for efficacy. For those readers who like to misunderstand me I should explain that the sceptics’ view is in capital letters.

At the beginning there was the experience. Unaware of anatomy, physiology, pathology etc., people started sticking needles in other people’s skin, some 2000 years ago, and observed that they experienced relief of all sorts of symptoms.When an American journalist reported about this phenomenon in the 1970s, acupuncture became all the rage in the West. Acupuncture-fans then claimed that a 2000-year history is ample proof that acupuncture does work.

BUT ANECDOTES ARE NOTORIOUSLY UNRELIABLE!

Even the most enthusiastic advocates conceded that this is probably true. So they documented detailed case-series of lots of patients, calculated the average difference between the pre- and post-treatment severity of symptoms, submitted it to statistical tests, and published the notion that the effects of acupuncture are not just anecdotal; in fact, they are statistically significant, they said.

BUT THIS EFFECT COULD BE DUE TO THE NATURAL HISTORY OF THE CONDITION!

“True enough”, grumbled the acupuncture-fans and conducted the very first controlled clinical trials. Essentially they treated one group of patients with acupuncture while another group received conventional treatments as usual. When they analysed the results, they found that the acupuncture group had improved significantly more. “Now do you believe us?”, they asked triumphantly, “acupuncture is clearly effective”.

NO! THIS OUTCOME MIGHT BE DUE TO SELECTION BIAS. SUCH A STUDY-DESIGN CANNOT ESTABLISH CAUSE AND EFFECT.

The acupuncturists felt slightly embarrassed because they had not thought of that. They had allocated their patients to the treatment according to patients’ choice. Thus the expectation of the patients (or the clinician) to get relief from acupuncture might have been the reason for the difference in outcome. So they consulted an expert in trial-design and were advised to allocate not by choice but by chance. In other words, they repeated the previous study but randomised patients to the two groups. Amazingly, their RCT still found a significant difference favouring acupuncture over treatment as usual.

BUT THIS DIFFERENCE COULD BE CAUSED BY A PLACEBO-EFFECT!

Now the acupuncturists were in a bit of a pickle; as far as they could see, there was no good placebo for acupuncture! Eventually some methodologist-chap came up with the idea that, in order to mimic a placebo, they could simply stick needles into non-acupuncture points. When the acupuncturists tried that method, they found that there were improvements in both groups but the difference between real acupuncture and placebo was tiny and usually neither statistically significant nor clinically relevant.

NOW DO YOU CONCEDE THAT ACUPUNCTURE IS NOT AN EFFECTIVE TREATMENT?

Absolutely not! The results merely show that needling non-acupuncture points is not an adequate placebo. Obviously this intervention also sends a powerful signal to the brain which clearly makes it an effective intervention. What do you expect when you compare two effective treatments?

IF YOU REALLY THINK SO, YOU NEED TO PROVE IT AND DESIGN A PLACEBO THAT IS INERT.

At that stage, the acupuncturists came up with a placebo-needle that did not actually penetrate the skin; it worked like a mini stage dagger that telescopes into itself while giving the impression that it penetrated the skin just like the real thing. Surely this was an adequate placebo! The acupuncturists repeated their studies but, to their utter dismay, they found again that both groups improved and the difference in outcome between their new placebo and true acupuncture was minimal.

WE TOLD YOU THAT ACUPUNCTURE WAS NOT EFFECTIVE! DO YOU FINALLY AGREE?

Certainly not, they replied. We have thought long and hard about these intriguing findings and believe that they can be explained just like the last set of results: the non-penetrating needles touch the skin; this touch provides a stimulus powerful enough to have an effect on the brain; the non-penetrating placebo-needles are not inert and therefore the results merely depict a comparison of two effective treatments.

YOU MUST BE JOKING! HOW ARE YOU GOING TO PROVE THAT BIZARRE HYPOTHESIS?

We had many discussions and consensus meeting amongst the most brilliant brains in acupuncture about this issue and have arrived at the conclusion that your obsession with placebo, cause and effect etc. is ridiculous and entirely misplaced. In real life, we don’t use placebos. So, let’s instead address the ‘real life’ question: is acupuncture better than usual treatment? We have conducted pragmatic studies where one group of patients gets treatment as usual and the other group receives acupuncture in addition. These studies show that acupuncture is effective. This is all the evidence we need. Why can you not believe us?

NOW WE HAVE ARRIVED EXACTLY AT THE POINT WHERE WE HAVE BEEN A LONG TIME AGO. SUCH A STUDY-DESIGN CANNOT ESTABLISH CAUSE AND EFFECT. YOU OBVIOUSLY CANNOT DEMONSTRATE THAT ACUPUNCTURE CAUSES CLINICAL IMPROVEMENT. THEREFORE YOU OPT TO PRETEND THAT CAUSE AND EFFECT ARE IRRELEVANT. YOU USE SOME IMITATION OF SCIENCE TO ‘PROVE’ THAT YOUR PRECONCEIVED IDEAS ARE CORRECT. YOU DO NOT SEEM TO BE INTERESTED IN THE TRUTH ABOUT ACUPUNCTURE AT ALL.

In 1747, James Lind conducted what may well be the first documented controlled clinical trial in the history of medicine. He treated a small group of healthy sailors with a range of different remedies to see whether one of these regimen might be effective in preventing scurvy. The results showed that lemon and lime juice – effectively vitamin C – did the trick. This trial changed the world: it saved tens of thousands of lives, gave Britain the advantage at sea needed to become a dominant empire, and set medicine on the track to eventually become evidence-based.

Of course, Lind did not know that the effective principle in his lemon/lime juice was vitamin C. The Hungarian physiologist Albert Szent-Gyorgyi discovered vitamin C only ~200 years later and received the Nobel Prize for it in 1937. Since then, research has been buoyant, and vitamin C has been advocated for almost every condition one can think of. Looking at some of the claims made for it, I get the impression that more charlatans have jumped on the vitamin C band-waggon than the old vehicle can support. In alternative medicine, high-dose IV vitamin C is a popular variation of Lind’s concept, not least for the treatment of cancer.

Researchers from the NIH in the US surveyed attendees at annual CAM Conferences in 2006 and 2008, and determined sales of intravenous vitamin C by major U.S. manufacturers/distributors. They also queried practitioners for adverse effects, compiled published cases, and analyzed FDA’s Adverse Events Database. Of 199 survey respondents (out of 550), 172 practitioners had administered IV vitamin C to 11,233 patients in 2006 and to 8876 patients in 2008. The average dose was 28 grams every 4 days, with a mean of 22 treatments per patient. Estimated yearly doses used (as 25g/50ml vials) were 318,539 in 2006 and 354,647 in 2008. Manufacturers’ yearly sales were 750,000 and 855,000 vials, respectively. Common reasons for treatment included infection, cancer, and fatigue. Of 9,328 patients for whom data was available, 101 had adverse effects, mostly minor, including lethargy/fatigue in 59 patients, change in mental status in 21 patients and vein irritation/phlebitis in 6 patients. Publications documented serious adverse events, including two deaths. The FDA Adverse Events Database was uninformative.

The authors of this paper conclude that high dose IV vitamin C is in unexpectedly wide use by CAM practitioners. Other than the known complications of IV vitamin C in those with renal impairment or glucose 6 phosphate dehydrogenase deficiency, high dose intravenous vitamin C appears to be remarkably safe. Physicians should inquire about IV vitamin C use in patients with cancer, chronic, untreatable, or intractable conditions and be observant of unexpected harm, drug interactions, or benefit.

I find these results somewhat worrying. Desperate cancer patients are constantly being told that they can fight the disease with high-dose vitamin C, for instance on the >9 million (!) websites on this subject. One site, for instance, leaves little doubt about the efficacy of vitamin C as a treatment for cancer: First shown to be a powerful anti-cancer agent in 1971, it wasn’t until 20 years later that vitamin C started to be accepted by the mainstream medical profession. Eating a vitamin C-rich diet substantially reduces the risk of cancer, and high intakes – above 5000mg a day (the equivalent of 100 oranges) – substantially increases the life expectancy of cancer patients.

Statements like this one give false hope to cancer patients which is unethical and cruel and might hasten the death of many. The reality is quite different and provides little reason for such hope. Here are just a few conclusions from recent scientific analyses on this or closely related topics:

post-diagnosis vitamin C supplementation at the level observed in our population was not associated with survival

Evidence is insufficient to prove the presence or absence of benefits from use of multivitamin and mineral supplements to prevent cancer and chronic disease

We could not find evidence that antioxidant supplements prevent gastrointestinal cancers. On the contrary, they seem to increase overall mortality. The potential cancer preventive effect of selenium should be studied in adequately conducted randomised trial

The question whether the regular intake of high doses of vitamin C have a preventative effect for certain cancers is currently open. But there is no good reason to suggest that high dose IV vitamin C is an effective treatment for any cancer. To pretend otherwise, as so many alternative practitioners seem to do, is less than responsible – in fact, it is a hallmark for cancer quackery.

As I write these words, I am travelling back from a medical conference. The organisers had invited me to give a lecture which I concluded saying: “anyone in medicine not believing in evidence-based health care is in the wrong business”. This statement was meant to stimulate the discussion and provoke the audience who were perhaps just a little on the side of those who are not all that taken by science.

I may well have been right, because, in the coffee break, several doctors disputed my point; to paraphrase their arguments: “You don’t believe in the value of experience, you think that science is the way to know everything. But you are wrong! Philosophers and other people, who are a lot cleverer than you, tell us that science is not the way to real knowledge; and in some forms of medicine we have a wealth of experience which we cannot ignore. This is at least as important as scientific knowledge. Take TCM, for instance, thousands of years of tradition must mean something; in fact it tells us more than science will ever be able to. Qi-energy, for instance, is a concept based on experience, and science is useless at verifying it.”

I disagreed, of course. But I am afraid that I did not convince my colleagues. The appeal to tradition is amazingly powerful, so much so that even well-seasoned physicians fall for it. Yet it nevertheless is a fallacy, I am sure.

So what does experience tell us, how is it generated and why should it be unreliable?

On the level of the individual, experience emerges when a clinician makes similar observations several times in a row. This is so persuasive that few doctors are immune to the phenomenon. Let’s assume the experience is about acupuncture, more precisely about acupuncture for smoking cessation. The acupuncturist presumably has learnt during his training that his therapy works for that indication via stimulating the flow of Qi, and promptly tries it on several patients. Some of them come back for more and report that they find it easier to give up cigarettes after consulting him. This happens repeatedly, and our clinician forthwith is convinced – in fact, he knows – that acupuncture is effective for smoking cessation.

If we critically analyse this scenario, what does it tell us? It tells us very little of relevance, I am afraid. The scenario is entirely compatible with a whole host of explanations which have nothing to do with the effects of acupuncture per se:

  • Those patients who did not manage to stop smoking might not have returned. Only seeing his successes without his failures, the acupuncturist would have got the wrong end of the stick.
  • Human memory is selective such that the few patients who did come back and reported failure might easily get forgotten by the clinician. We all remember the good things and forget the disappointments, particularly if we are clinicians.
  • The placebo-effect might have played a dirty trick on the experience of our acupuncturist.
  • Some patients might have used nicotine patches that helped him to stop smoking without disclosing this fact to the acupuncturist who then, of course, attributed the benefit to his needling.
  • The acupuncturist – being a very kind and empathetic clinician – might have involuntarily induced some of his patients to show kindness in return and thus tell porkies about their smoking habits which would have created a false positive impression about the effectiveness of his treatment.
  • Being so empathetic, the acupuncturists would have provided lots of encouragement to stop smoking which, in some patients, might have been sufficient to kick the habit.

 

The long and short of all this is that our acupuncturist gradually got convinced by this interplay of factors that Qi exists and that acupuncture is an ineffective treatment. Hence forth he would bet his last shirt that he is right about this – after all, he has seen it with his own eyes, not just once but many times. And he will doubt anyone who shows him evidence that says otherwise. In fact, he is likely become very sceptical about scientific evidence in general – just like the doctors who talked to me after my lecture.

On a population level, such experience will be prevalent in not just one but most acupuncturists. Our clinician’s experience is certainly not unique; others will have made it too. In fact, as an acupuncturist, it is hard not to make it. Acupuncturists would have told everyone else about it, perhaps reported it on conferences or published it in articles or books. Experience of this nature is passed on from generation to generation, and soon someone will be able to demonstrate that acupuncture has been used ’effectively’ for smoking cessation since decades or centuries. The creation of a myth out of unreliable experience is thus complete.

Am I saying that experience of this nature is always and necessarily wrong or useless? No, I am not. It can be and often is correct. But, at the same time, it is frequently incorrect. It can serve as a valuable indicator but not more. Experience is not a tool for reliably informing us about the effectiveness of medical interventions. Experience based-medicine is an obsolete pseudo-medicine burdened with concepts that are counter-productive to optimal health care.

Philosophers and other people who are much cleverer than I am have been trying for some time to separate good from bad science and evidence from experience. Most recently, two philosophers, MASSIMO PIGLIUCCI and MAARTEN BOUDRY, commented specifically on this problem in relation to TCM. I leave you with some extensive quotes from what they wrote.

… pointing out that some traditional Chinese remedies (like drinking fresh turtle blood to alleviate cold symptoms) may in fact work, and therefore should not be dismissed as pseudoscience… risks confusing the possible effectiveness of folk remedies with the arbitrary theoretical-metaphysical baggage attached to it. There is no question that some folk remedies do work. The active ingredient of aspirin, for example, is derived from willow bark…

… claims about the existence of “Qi” energy, channeled through the human body by way of “meridians,” though, is a different matter. This sounds scientific, because it uses arcane jargon that gives the impression of articulating explanatory principles. But there is no way to test the existence of Qi and associated meridians, or to establish a viable research program based on those concepts, for the simple reason that talk of Qi and meridians only looks substantive, but it isn’t even in the ballpark of an empirically verifiable theory.

…the notion of Qi only mimics scientific notions such as enzyme actions on lipid compounds. This is a standard modus operandi of pseudoscience: it adopts the external trappings of science, but without the substance.

…The notion of Qi, again, is not really a theory in any meaningful sense of the word. It is just an evocative word to label a mysterious force of which we do not know and we are not told how to find out anything at all.

Still, one may reasonably object, what’s the harm in believing in Qi and related notions, if in fact the proposed remedies seem to help? Well, setting aside the obvious objections that the slaughtering of turtles might raise on ethical grounds, there are several issues to consider. To begin with, we can incorporate whatever serendipitous discoveries from folk medicine into modern scientific practice, as in the case of the willow bark turned aspirin. In this sense, there is no such thing as “alternative” medicine, there’s only stuff that works and stuff that doesn’t.

Second, if we are positing Qi and similar concepts, we are attempting to provide explanations for why some things work and others don’t. If these explanations are wrong, or unfounded as in the case of vacuous concepts like Qi, then we ought to correct or abandon them. Most importantly, pseudo-medical treatments often do not work, or are even positively harmful. If you take folk herbal “remedies,” for instance, while your body is fighting a serious infection, you may suffer severe, even fatal, consequences.

…Indulging in a bit of pseudoscience in some instances may be relatively innocuous, but the problem is that doing so lowers your defenses against more dangerous delusions that are based on similar confusions and fallacies. For instance, you may expose yourself and your loved ones to harm because your pseudoscientific proclivities lead you to accept notions that have been scientifically disproved, like the increasingly (and worryingly) popular idea that vaccines cause autism.

Philosophers nowadays recognize that there is no sharp line dividing sense from nonsense, and moreover that doctrines starting out in one camp may over time evolve into the other. For example, alchemy was a (somewhat) legitimate science in the times of Newton and Boyle, but it is now firmly pseudoscientific (movements in the opposite direction, from full-blown pseudoscience to genuine science, are notably rare)….

The borderlines between genuine science and pseudoscience may be fuzzy, but this should be even more of a call for careful distinctions, based on systematic facts and sound reasoning. To try a modicum of turtle blood here and a little aspirin there is not the hallmark of wisdom and even-mindedness. It is a dangerous gateway to superstition and irrationality

I regularly used to ask alternative practitioners what diseases they are good at treating. In fact, we once ran an entire research project dedicated to this question and found that their own impressions were generally based on wishful thinking rather than on evidence. The libel case of the BCA versus Simon Singh then brought this issue into the focus of the public eye, and consequently several professional organisations of alternative practitioners seem to have advised their members to be cautious about making unsubstantiated therapeutic claims. This could have been an important step into the right direction – unless, of course, a clever trick had not been devised to bypass the need for evidence. Today, when I ask alternative practitioners ‘what do you treat effectively?’ I tend to get answers like:

  • Alternative practitioners, unlike conventional clinicians, do not treat diseases.
  • I treat the whole person, not just the disease.
  • I treat people and their specific set of signs and symptoms, rather than disease labels (this actually is a quote from the comments section of one of my recent posts).
  • I focus on the totality of the symptoms; disease labels are irrelevant in the realm of my therapy.
  • Chiropractors adjust subluxations which are the root cause for most diseases.
  • Acupuncturists re-balance life energies which is a precondition for healing to commence irrespective of the disease.
  • Homeopaths treat the totality of symptoms so that the patient’s vital force can do the healing.
  • etc. etc.

All of these statements are deeply rooted in the long obsolete notions of vitalism, i.e. the assumption that a vital energy flows in all living organisms and is responsible for our health irrespective of the disease we happen to suffer from. But what do the answers to my question ‘what do you treat?’ really mean? If we analyse the above responses critically, they seem to imply that:

  1. Conventional clinicians do not treat patients but merely disease labels.
  2. Alternative practitioners can successfully treat any disease or condition.

Ad 1 In my view, it is arrogant and grossly unfair to claim that alternative practitioners work holistically, while conventional health care professionals do not. I have pointed out repeatedly that any good medicine always has been and always will be holistic. High-jacking holism as a specific characteristic for alternative medicine is misleading and an insult to all conventional clinicians who do their best to practice good medicine.

Ad 2 By claiming that they treat the whole person irrespective of her disease, alternative practitioners effectively try to give themselves a ‘carte blanche’ for treating any disease or any condition or any symptom. If a child has asthma, a chiropractor will find a subluxation, adjust it with spinal manipulation, and claim that the child’s condition will improve as a consequence of his treatment – NEVER MIND THE EVIDENCE. If a person wants to give up smoking, an acupuncturist will use acupuncture to re-balance her yin and yang claiming that this intervention will make smoking cessation more successful – NEVER MIND THE EVIDENCE. If a patient suffers from cancer, a homeopath might find a remedy that promotes her vital energy claiming that the cancer will subsequently be cured – NEVER MIND THE EVIDENCE which in all of the three cases is negative.

The claim of alternative practitioners to not treat disease labels but the whole patient is doubtlessly attractive to consumers and it is also extremely good for business. On closer inspection, however, it turns out to be a distraction from the fact that alternative practitioners treat everything and anything, usually without the slightest evidence that their interventions generates more good than harm. It allows alternative practitioners to live in a fool’s paradise of quackery where they believe themselves to be protected from any challenges and demands for evidence.

Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted.


Click here for a comprehensive list of recent comments.

Categories