Edzard Ernst

MD, PhD, FMedSci, FSB, FRCP, FRCPEd

If we go on the internet, we find no end of positive claims for TM. The official TM website, for instance, claims that more than 350 peer-reviewed research studies on the TM technique have been published in over 160 scientific journals. These studies were conducted at many US and international universities and research centers, including Harvard Medical School, Stanford Medical School, Yale Medical School, and UCLA Medical School.

This may well be true – but do those studies amount to more than a heap of beans? Let’s find out.

The objective of our Cochrane review was to determine the effectiveness of TM for the primary prevention of cardiovascular disease (CVD). We searched the following electronic databases: the Cochrane Central Register of Controlled Trials (CENTRAL) (2013, Issue 10); MEDLINE (Ovid) (1946 to week three November 2013); EMBASE Classic and EMBASE (Ovid) (1947 to week 48 2013); ISI Web of Science (1970 to 28 November 2013); and Database of Abstracts of Reviews of Effects (DARE) and Health Technology Assessment Database and Health Economics Evaluations Database (November 2013). We also searched the Allied and complementary Medicine Database (AMED) (inception to January 2014) and IndMed (inception to January 2014). We hand searched trial registers and reference lists of reviews and articles and contacted experts in the field. We applied no language restrictions.

We included randomised controlled trials (RCTs) of at least three months’ duration involving healthy adults or adults at high risk of CVD. Trials examined TM only and the comparison group was no intervention or minimal intervention. We excluded trials that involved multi-factorial interventions. Outcomes of interest were clinical CVD events (cardiovascular mortality, all-cause mortality and non-fatal events) and major CVD risk factors (e.g. blood pressure and blood lipids, occurrence of type 2 diabetes, quality of life, adverse events and costs). Two authors independently selected trials for inclusion, extracted data and assessed the risk of bias.

We identified 4 RCTs with a total of 430 participants for inclusion in this review. The included trials were small, short term (three months) and at risk of bias. In all studies, TM was practised for 15 to 20 minutes twice a day. None of the included studies reported all-cause mortality, cardiovascular mortality or non-fatal endpoints as trials were short term, but one study reported survival rate three years after the trial was completed. In view of the considerable statistical heterogeneity between the results of the studies for the only outcomes reported, systolic blood pressure (I2 = 72%) and diastolic blood pressure (I2 = 66%), we decided not to undertake a meta-analysis. None of the four trials reported blood lipids, occurrence of type 2 diabetes, adverse events, costs or quality of life.

We concluded that there are few trials with limited outcomes examining the effectiveness of TM for the primary prevention of CVD. Due to the limited evidence to date, we could draw no conclusions as to the effectiveness of TM for the primary prevention of CVD. There was considerable heterogeneity between trials and the included studies were small, short term and at overall serious risk of bias. More and larger long-term, high-quality trials are needed.

Even though I am a co-author of this review, I am not entirely sure that the last sentence of our conclusion is totally correct. The TM movement has, in my view, all the characteristics of a cult with all its the dangers that cults entail. This means, I think, we ought to be cautious about TM and sceptical about their research and results. At the risk of provoking harsh criticism, I would even say we should be distrustful of their aims and methods.

Hard to believe but, in the last 35 years, I have written or edited a total of 49 books; about half of them on alternative medicine and the rest on various subjects related to clinical medicine and research. Each time a new one comes out, I am excited, of course, but this one is special:

  • I have not written a book for several years.
  • I have worked on it much longer than on any book before.
  • Never before have I written a book with is so much about myself.
  • None of my previous book covered material that is as ‘sensitive’ as this one.

I started on this book shortly after TRICK OR TREATMENT had been published. Its initial working title was ALTERNATIVE MEDICINE: THE INSIDE STORY. My aim was to focus on the extraordinary things which had happened during my time in Exeter, to shed some light on the often not so quaint life in academia, and to show how bizarre the world of alternative medicine truly is. But several people who know about these things and who had glanced at the first draft chapters strongly advised me to radically change this concept. They told me that such a book could only work as a personal memoire.

Yet I was most reluctant to write about myself; I wanted to write about science, research as well as the obstacles which some people manage to put in their way. So, after much discussion and contemplation, I compromised and added the initial chapters which told the reader about my background and my work prior to the Exeter appointment. This brought in subjects like my research on ‘Nazi-medicine’ (which, I believe, is more important than that on alternative medicine) that seemed almost entirely unrelated to alternative medicine, and the whole thing began to look a bit disjointed, in my view. However, my advisers felt this was a step in the right direction and argued that my compromise was not enough; they wanted more about me as a person, my motivations, my background etc. Eventually I (partly) gave in and provided a bit more of what they seemed to want.

But I am clearly not a novelist, most of what I have ever written is medical stuff; my style is too much that of a scientist – dry and boring. In other words, my book seemed to be going nowhere. Just when, after years of hard work, I was about to throw it all in the bin, help came from a totally unexpected corner.

Louise Lubetkin (even today, I have never met her in person) had contributed several posts as ‘guest editor’ to this blog, and I very much liked her way with words. When she offered to have a look at my book, I was thrilled. It is largely thanks to her that my ‘memoire’ ever saw the light of day. She helped enormously with making it readable and with joining up the seemingly separate episodes describes in my book.

Finding a fitting title was far from easy. Nothing seemed to encapsulate its contents, and ‘A SCIENTIST IN WONDERLAND’, the title I eventually chose, is a bit of a compromise; the subtitle does describe it much better, I think: A MEMOIR OF SEARCHING FOR TRUTH AND FINDING TROUBLE.

Now that the book is about to be published, I am anxious as never before on similar occasions. I do, of course, not think for a minute that it will be anything near to a best-seller, but I want people with an interest in alternative medicine, academia or science to read it (get it from a library to save money) and foremost I want them to understand why I wrote it. For me, this is neither about settling scores nor about self-promotion, it is about telling a story which is important in more than one way.

- Chronic low back pain (CLBP) is a condition which affects so many people that it represents a huge burden to individual patients’ suffering as well as to society in terms of loss of work time and increased economic cost. The number of therapies that have been claimed to be effective for CLBP can hardly be counted. Two of the most common treatments are spinal manipulation and exercise.

The purpose of this systematic review was to determine the effectiveness of spinal manipulation vs prescribed exercise for patients diagnosed with CLBP. Only RCTs that compared head-to-head spinal manipulation to an exercise group were included in this review.

A search of the current literature was conducted using a keyword process in CINAHL, Cochrane Register of Controlled Trials Database, Medline, and Embase. The searches included studies available up to August 2014. Studies were included based on PICOS criteria 1) individuals with CLBP defined as lasting 12 weeks or longer; 2) spinal manipulation performed by a health care practitioner; 3) prescribed exercise for the treatment of CLBP and monitored by a health care practitioner; 4) measurable clinical outcomes for reducing pain, disability or improving function; 5) randomized controlled trials. The methodological quality of all included articles was determined using the criteria developed and used by the Physiotherapy Evidence Database (PEDro).

Only three RCTs met the inclusion criteria of this systematic review. The outcomes used in these studies included Disability Indexes, Pain Scales and function improvement scales. One RCT found spinal manipulation to be more effective than exercise, and the results of another RCT indicated the reverse. The third RCT found both interventions offering equal effects in the long term.

The author concluded that there is no conclusive evidence that clearly favours spinal manipulation or exercise as more effective in treatment of CLBP. More studies are needed to further explore which intervention is more effective.

Whenever there are uncounted treatments for a given condition, one has to ask oneself whether they are all similarly effective or equally ineffective. The present review does unfortunately not answer this question, but I fear the latter might be more true than the former.

Considering how much money we spend on treating CLBP, it is truly surprising to see that just three RCTs are available comparing two of the most commonly used treatments for this condition. Equally surprising is the fact that we simply cannot tell, on the basis of these data, which of the two therapies is more effective.

What consequences should we draw from this information. Obviously we need more high quality trials. But what should we do in the meantime?

Whenever two treatments are equally effective (or, in this case, perhaps equally ineffective?), we must consider other important criteria such as safety and cost. Regular chiropractic care (chiropractors use spinal manipulation on almost every patient, while osteopaths and physiotherapists employ it less frequently)  is neither cheap nor free of serious adverse effects such as strokes; regular exercise has none of these disadvantages. In view of these undeniable facts, it is hard not to come up with anything other than the following recommendation: until new and compelling evidence becomes available, exercise ought to be preferred over spinal manipulation as a treatment of CLBP – and consequently consulting a chiropractor should not be the first choice for CLBP patients.

Guest post by Nick Ross

If you’re a fan of Edzard Ernst – and who with a rational mind would not be – then you will be a fan of HealthWatch.

Edzard is a distinguished supporter. Do join us. I can’t promise much in return except that you will be part of a small and noble organisation that campaigns for treatments that work – in other words for evidence based medicine. Oh, and you get a regular Newsletter, which is actually rather good.

HealthWatch was inspired 25 years ago by Professor Michael Baum, the breast cancer surgeon who was incandescent that so many women presented to his clinic late, doomed and with suppurating sores, because they had been persuaded to try ‘alternative treatment’ rather than the real thing.

But like Edzard (and indeed like Michael Baum), HealthWatch keeps an open mind. If there are reliable data to show that an apparently weirdo treatment works, hallelujah. If there is evidence that an orthodox one doesn’t then it deserves a raspberry. HealthWatch has worked to expose quacks and swindlers and to get the Advertising Standards Authority to do its job regulating against false claims and flimflam. It has fought the NHS to have women given fair and balanced advice about the perils of mass screening. It has campaigned with Sense About Science, English Pen and Index to protect whistleblowing scientists from vexatious libel laws, and it has joined the AllTrials battle for transparency in drug trials. It has an annual competition for medical and nursing students to encourage critical analysis of clinical research protocols, and it stages the annual HealthWatch Award and Lecture which has featured Edzard (in 2005) and a galaxy of other champions of scepticism and good evidence including Sir Iain Chalmers, Richard Smith, David Colquhoun, Tim Harford, John Diamond, Richard Doll, Peter Wilmshurst, Ray Tallis, Ben Goldacre, Fiona Godlee and, last year, Simon Singh. We are shortly to sponsor a national debate on Lord Saatchi’s controversial Medical innovation Bill.

But we need new blood. Do please check us out. Be careful, because since we first registered our name a host of brazen copycats have emerged, not least Her Majesty’s Government with ‘Healthwatch England’ which is part of the Care Quality Commission. We have had to put ‘uk’ at the end of our web address to retain our identity. So take the link to http://www.healthwatch-uk.org/, or better still take out a (very modestly priced) subscription.

As Edmund Burke might well have said, all it takes for quackery to flourish is that good men and women do nothing.

I would have never thought that someone would be able to identify the author of the text I quoted in the previous post:

It is known that not just novel therapies but also traditional ones, such as homeopathy, suffer opposition and rejection by some doctors without having ever been subjected to serious tests. The doctor is in charge of medical treatment; he is thus responsible foremost for making sure all knowledge and all methods are employed for the benefit of public health…I ask the medical profession to consider even previously excluded therapies with an open mind. It is necessary that an unbiased evaluation takes place, not just of the theories but also of the clinical effectiveness of alternative medicine.

More often than once has science, when it relied on theory alone, arrived at verdicts which later had to be overturned – frequently this occurred only after long periods of time, after progress had been hindered and most acclaimed pioneers had suffered serious injustice. I do not need to remind you of the doctor who, more than 100 years ago, in fighting puerperal fever, discovered sepsis and asepsis but was laughed at and ousted by his colleagues throughout his lifetime. Yet nobody would today deny that this knowledge is most relevant to medicine and that it belongs to the basis of medicine. Insightful doctors, some of whom famous, have, during the recent years, spoken openly about the crisis in medicine and the dead end that health care has maneuvered itself into. It seems obvious that the solution is going in directions which embrace nature. Hardly any other form of science is so tightly bound to nature as is the science occupied with healing living creatures. The demand for holism is getting stronger and stronger, a general demand which has already been fruitful on the political level. For medicine, the challenge is to treat more than previously by influencing the whole organism when we aim to heal a diseased organ.

It is from the opening speech by Rudolf Hess on the occasion of the WORLD CONFERENCE ON HOMEOPATHY 1937, in Berlin. Hess, at the time Hitler’s deputy, was not the only Nazi-leader. I knew of the opening speech because, a few years ago, DER SPIEGEL published a theme issue on homeopathy, and they published a photo of the opening ceremony of this meeting. It shows many men in SS-uniform and, in the first row of the auditorium, we see Hess (as well as Himmler) ready to spring into action.

Hess in particular was besotted with alternative medicine which the Nazis elected to call NEUE DEUTSCHE HEILKUNDE. Somewhat to the dismay of today’s alternative medicine enthusiasts, I have repeatedly published on this aspect of alternative medicine’s past, and it also is an important part of my new book A SCIENTIST IN WONDERLAND which the lucky winner (my congratulations!) of my little competition to identify the author has won. The abstract of an 2001 article explains this history succinctly:

The aim of this article is to discuss complementary/alternative medicine (CAM) in the Third Reich. Based on a general movement towards all things natural, a powerful trend towards natural ways of healing had developed in the 19(th)century. By 1930 this had led to a situation where roughly as many lay practitioners of CAM existed in Germany as doctors. To re-unify German medicine under the banner of ‘Neue Deutsche Heilkunde’, the Nazi officials created the ‘Heilpraktiker’ – a profession which was meant to become extinct within one generation. The ‘flag ship’ of the ‘Neue Deutsche Heilkunde’ was the ‘Rudolf Hess Krankenhaus’ in Dresden. It represented a full integration of CAM and orthodox medicine. An example of systematic research into CAM is the Nazi government’s project to validate homoeopathy. Even though the data are now lost, the results of this research seem to have been negative. Even though there are some striking similarities between today’s CAM and yesterday’s ‘Neue Deutsche Heilkunde’ there are important differences. Most importantly, perhaps, today’s CAM is concerned with the welfare of the individual, whereas the ‘Neue Deutsche Heilkunde’ was aimed at ensuring the dominance of the Aryan race.

One fascinating aspect of this past is the fact that the NEUE DEUTSCHE HEILKUNDE was de facto the invention of what we today call ‘integrated medicine’. Then it was more like a ‘shot-gun marriage’, while today it seems to be driven more by political correctness and sloppy thinking. It did not work 70 years ago for the same reason that it will fail today: the integration of bogus (non-evidence based) treatments into conventional medicine must inevitably render health care not better but worse!

One does not need to be a rocket scientist to understand that, and Hess as well as other proponents of alternative medicine of his time had certainly got the idea. So they initiated the largest ever series of scientific tests of homeopathy. This research program was not just left to the homeopaths, who never had a reputation of being either rigorous or unbiased, but some of the best scientists of the era were recruited for it. The results vanished in the hands of the homeopaths during the turmoil of the war. But one eye-witness report of a homeopaths, Fritz Donner, makes it very clear: as it turned out, there was not a jot of evidence in favour of homeopathy.

And this, I think, is the other fascinating aspect of the story: homeopaths did not give up their plight to popularise homeopathy. On the contrary, they re-doubled their efforts to fool us all and to convince us with dodgy results (see recent posts on this blog) that homeopathy somehow does defy the laws of nature and is, in effect, very effective for all sorts of diseases.

My readers suggested all sorts of potential authors for the Hess speech; and they are right! It could have been written by any proponent of alternative medicine. This fact is amusing and depressing at the same time. Amusing because it discloses the lack of new ideas and arguments (even the same fallacies are being used). Depressing because it suggests that progress in alternative medicine is almost totally absent.

Recently I came across an interesting speech on alternative medicine which impressed me for a number of reasons. It made me think of a little game: the first person who correctly guesses who its author is, and posts the right answer as a comment on this blog, will receive a free copy of my new book A SCIENTIST IN WONDERLAND.

Here are 2 paragraphs from the speech in question:

It is known that not just novel therapies but also traditional ones, such as homeopathy, suffer opposition and rejection by some doctors without having ever been subjected to serious tests. The doctor is in charge of medical treatment; he is thus responsible foremost for making sure all knowledge and all methods are employed for the benefit of public health…I ask the medical profession to consider even previously excluded therapies with an open mind. It is necessary that an unbiased evaluation takes place, not just of the theories but also of the clinical effectiveness of alternative medicine.

More often than once has science, when it relied on theory alone, arrived at verdicts which later had to be overturned – frequently this occurred only after long periods of time, after progress had been hindered and most acclaimed pioneers had suffered serious injustice. I do not need to remind you of the doctor who, more than 100 years ago, in fighting puerperal fever, discovered sepsis and asepsis but was laughed at and ousted by his colleagues throughout his lifetime.  Yet nobody would today deny that this knowledge is most relevant to medicine and that it belongs to the basis of medicine. Insightful doctors, some of whom famous, have, during the recent years, spoken openly about the crisis in medicine and the dead end that health care has maneuvered itself into. It seems obvious that the solution is going in directions which embrace nature. Hardly any other form of science is so tightly bound to nature as is the science occupied with healing living creatures. The demand for holism is getting stronger and stronger, a general demand which has already been fruitful on the political level. For medicine, the challenge is to treat more than previously by influencing the whole organism when we aim to heal a diseased organ.

The very first article on a subject related to alternative medicine with a 2015 date that I came across is a case-report. I am afraid it will not delight our chiropractic friends who tend to deny that their main therapy can cause serious problems.

In this paper, US doctors tell the story of a young woman who developed headache, vomiting, diplopia, dizziness, and ataxia following a neck manipulation by her chiropractor. A computed tomography scan of the head was ordered and it revealed an infarct in the inferior half of the left cerebellar hemisphere and compression of the fourth ventricle causing moderately severe, acute obstructive hydrocephalus. Magnetic resonance angiography showed severe narrowing and low flow in the intracranial segment of the left distal vertebral artery. The patient was treated with mannitol and a ventriculostomy. Following these interventions, she made an excellent functional recovery.

The authors of the case-report draw the following conclusions: This report illustrates the potential hazards associated with neck trauma, including chiropractic manipulation. The vertebral arteries are at risk for aneurysm formation and/or dissection, which can cause acute stroke.

I can already hear the counter-arguments: this is not evidence, it’s an anecdote; the evidence from the Cassidy study shows there is no such risk!

Indeed the Cassidy study concluded that vertebral artery accident (VBA) stroke is a very rare event in the population. The increased risks of VBA stroke associated with chiropractic and primary care physician visits is likely due to patients with headache and neck pain from VBA dissection seeking care before their stroke. We found no evidence of excess risk of VBA stroke associated chiropractic care compared to primary care. That, of course, was what chiropractors longed to hear (and it is the main basis for their denial of risk) – so much so that Cassidy et al published the same results a second time (most experts feel that this is a violation of publication ethics).

But repeating arguments does not make them more true. What we should not forget is that the Cassidy study was but one of several case-control studies investigating this subject. And the totality of all such studies does not deny an association between neck manipulation and stroke.

Much more important is the fact that a re-analysis of the Cassidy data found that prior studies grossly misclassified cases of cervical dissection and mistakenly dismissed a causal association with manipulation. The authors of this new paper found a classification error of cases by Cassidy et al and they re-analysed the Cassidy data, which reported no association between spinal manipulation and cervical artery dissection (odds ratio [OR] 5 1.12, 95% CI .77-1.63). These re-calculated results reveal an odds ratio of 2.15 (95% CI.98-4.69). For patients less than 45 years of age, the OR was 6.91 (95% CI 2.59-13.74). The authors of the re-analysis conclude as follows: If our estimates of case misclassification are applicable outside the VA population, ORs for the association between SMT exposure and CAD are likely to be higher than those reported using the Rothwell/Cassidy strategy, particularly among younger populations. Future epidemiologic studies of this association should prioritize the accurate classification of cases and SMT exposure.
I think they are correct; but my conclusion of all this would be more pragmatic and much simpler: UNTIL WE HAVE CONVINCING EVIDENCE TO THE CONTRARY, WE HAVE TO ASSUME THAT CHIROPRACTIC NECK MANIPULATION CAN CAUSE A STROKE.

Moxibustion is an ancient variation of acupuncture using  moxa made from dried mugwort (Artemisia argyi). It has long played an important role in the traditional heath care systems of China and other Asian countries. More recently, it has become popular also in the West. Practitioners use moxa sticks indirectly to warm acupuncture needles, or burn it close to the patient’s skin. Essentially, moxibustion is a treatment where acupuncture points are stimulated mainly or exclusively by the heat of burning moxa.

Because of moxibustion’s long history of usage and the fact that it is employed in many countries for a very wide range of conditions, some might argue that it has stood the ‘test of time’ and should be considered to be a well-established therapy. More critical thinkers would, however, point out that this is not an argument but a classical fallacy.

My team at Exeter regularly had research fellows from Korea and other Asian countries, and we managed to develop a truly productive cooperation. It enabled us to conduct systematic reviews including the Asian literature – and this is how we got involved in an unusual amount of research into moxibustion which, after all, is a fairly exotic alternative therapy. In 2010, we began a series of systematic reviews of moxibustion.

One of the first such articles included 9 RCTs testing the effectiveness of this treatment for stroke rehabilitation. Three RCTs reported favorable effects of moxibustion plus standard care on motor function versus standard care alone Three randomized clinical trials compared the effects of moxibustion on activities of daily living alone but failed to show favorable effects of moxibustion.

Also in 2010, our systematic review of RCTs of moxibustion as a treatment of ulcerative colitis (UC) concluded that current evidence is insufficient to show that moxibustion is an effective treatment of UC. Most of included trials had high risk of bias. More rigorous studies seem warranted.

Our (2010) systematic review od RCTs of moxibustion as a therapy in cancer care found that the evidence was limited to suggest moxibustion is an effective supportive cancer care in nausea and vomiting. However, all studies had a high risk of bias so effectively there was not enough evidence to draw any conclusion.

Our (2010) systematic review of RCTs of moxibustion for treating hypertension concluded that there was insufficient evidence to suggest that moxibustion is an effective treatment for hypertension.

Our (2010) systematic review of RCTs of moxibustion for constipation concluded as follows: Given that the methodological quality of all RCTs was poor, the results from the present review are insufficient to suggest that moxibustion is an effective treatment for constipation. More rigorous studies are warranted.

Our (2010) systematic review found few RCTs were available that test the effectiveness of moxibustion in the management of pain, and most of the existing trials had a high risk of bias. Therefore, more rigorous studies are required before the effectiveness of moxibustion for the treatment of pain can be determined.

Our (2011) systematic review of 14 RCTs of moxibustion for rheumatic conditions failed to provide conclusive evidence for the effectiveness of moxibustion compared with drug therapy in rheumatic conditions.

The, so far, last article in this series has only just been published. The purpose of this systematic review was to assess the efficacy of moxibustion as a treatment of chemotherapy-induced leukopenia. Twelve databases were searched from their inception through June 2014, without a language restriction. Randomized clinical trials (RCTs) were included, if moxibustion was used as the sole treatment or as a part of a combination therapy with conventional drugs for leukopenia induced by chemotherapy. Cochrane criteria were used to assess the risk of bias.

Six RCTs with a total of 681 patients met our inclusion criteria. All of the included RCTs were associated with a high risk of bias. The trials included patients with various types of cancer receiving ongoing chemotherapy or after chemotherapy. The results of two RCTs suggested the effectiveness of moxibustion combined with chemotherapy vs. chemotherapy alone. In four RCTs, moxibustion was more effective than conventional drug therapy. Six RCTs showed that moxibustion was more effective than various types of control interventions in increasing white blood cell counts.

Our conclusion: there is low level of evidence based on these six trials that demonstrates the superiority of moxibustion over drug therapies in the treatment of chemotherapy-induced leukopenia. However, the number of trials, the total sample size, and the methodological quality are too low to draw firm conclusions. Future RCTs appear to be warranted.

Was all this research for nothing?

I know many people who would think so. However, I disagree. If nothing else, these articles demonstrated several facts quite clearly:

  • There is quite a bit of research even on the most exotic alternative therapy; sometimes one needs to look hard and include languages other than English.
  • Studies from China and other Asian counties very rarely report negative results; this fact casts a dark shadow on the credibility of such data.
  • The poor quality of trials in most areas of alternative medicine is lamentable and must be stimulus for researchers in this field to improve their act.
  • Authors of systematic reviews must resist the temptation to draw positive conclusions based on flawed primary data.
  • Moxibustion is a perfect example for demonstrating that the ‘test of time’ is no substitute for evidence.
  • As for moxibustion, it cannot currently be considered an evidence-based treatment for any condition.

I know, it’s not really original to come up with the 10000th article on “10 things…” – but you will have to forgive me, I read so many of these articles over the holiday period that I can’t help but jump on the already over-crowded bandwagon and compose yet another one.

So, here are 10 things which could, if implemented, bring considerable improvement in 2015 to my field of inquiry, alternative medicine.

  1. Consumers need to get better at acting as bull shit (BS) detectors. Let’s face it, much of what we read or hear about this subject is utter BS. Yet consumers frequently lap up even the worst drivel like it were some source of deep wisdom. They could save themselves so much money, if they learnt to be just a little bit more critical.
  2. Dr Oz should focus on being a heart surgeon. His TV show has been demonstrated far too often to be promoting dangerous quackery. Yet as a heart surgeon, he actually might do some good.
  3. Journalists ought to remember that they have a job that extends well beyond their ambition to sell copy. They have a responsibility to inform the public truthfully and responsibly.
  4. Book publishers should abstain from churning out book after book that does little else but mislead the public about alternative medicine in a way that all to often is dangerous to the readers’ health. The world does not need the 1000th book repeating nonsense on detox, wellness etc.!
  5. Alternative practitioners must realise that claiming that therapy x cures condition y is not just slightly over-optimistic (or based on ‘years of experience’); if the claim is not based on sound evidence, it is what most people would call an outright lie.
  6. Proponents of alternative medicine should learn that it is neither fair nor productive to fiercely attack everyone personally who disagrees with their enthusiasm for this or that form of alternative medicine. In fact, it merely highlights the acute lack of rational arguments.
  7. Researchers of alternative medicine have to remember how important it is to think critically – an uncritical scientist is at best a contradiction in terms and at worst a pseudo-scientist who is likely to cause harm.
  8. Authorities should amass the courage, the political power and the financial means of going after those charlatans who ruthlessly exploit the public by making a fast and easy buck on the gullibility of consumers. Only if there is the likelihood of hefty fines will we see a meaningful decrease in the current epidemic of alternative health fraud.
  9. Politicians should realise that alternative medicine is not just a trivial subject with which one might win votes, if one issues platitudes to please the majority; alternative medicine is used by so many people that it has become an important public health issue.
  10. Prince Charles need to learn how to control himself and abstain from meddling in health politics by using every conceivable occasion to promote what he thinks is ‘integrated medicine’ but which, in fact, can easily be disclosed to be quackery.

As you see, my list almost instantly turned into a wish-list, and the big questions that follow from it are:

  1. How could we increase the likelihood of these wishes to come true?
  2. And would there be anything left of alternative medicine, if all of these wishes miraculously became true in 2015?

I do not pretend to have the answers, but I do feel strongly that a healthy dose of critical thinking in all levels of education – from kindergartens to schools, from colleges to universities etc. – would be a good and necessary starting point.

I know, my list is not just a wish list, it also is a wishful thinking list. It would be hopelessly naïve to assume that major advances will be made in 2015. I am realistic, sometimes even quite pessimistic, about progress in alternative medicine. But this does not mean that I or anyone else should just give up. 2015 will be a year where at least one thing is certain: you will see me continuing me my fight for reason, critical analysis, rational debate and good evidence – and that’s a promise!

As promised, I will try with this post to explain my reservations regarding the new meta-analysis suggesting that individualised homeopathic remedies are superior to placebos. Before I start, however, I want to thank all those who have commented on various issues; it is well worth reading the numerous and diverse comments.

To remind us of the actual meta-analysis, it might be useful to re-publish its abstract (the full article is also available online):

BACKGROUND:

A rigorous and focused systematic review and meta-analysis of randomised controlled trials (RCTs) of individualised homeopathic treatment has not previously been undertaken. We tested the hypothesis that the outcome of an individualised homeopathic treatment approach using homeopathic medicines is distinguishable from that of placebos.

METHODS:

The review’s methods, including literature search strategy, data extraction, assessment of risk of bias and statistical analysis, were strictly protocol-based. Judgment in seven assessment domains enabled a trial’s risk of bias to be designated as low, unclear or high. A trial was judged to comprise ‘reliable evidence’ if its risk of bias was low or was unclear in one specified domain. ‘Effect size’ was reported as odds ratio (OR), with arithmetic transformation for continuous data carried out as required; OR > 1 signified an effect favouring homeopathy.

RESULTS:

Thirty-two eligible RCTs studied 24 different medical conditions in total. Twelve trials were classed ‘uncertain risk of bias’, three of which displayed relatively minor uncertainty and were designated reliable evidence; 20 trials were classed ‘high risk of bias’. Twenty-two trials had extractable data and were subjected to meta-analysis; OR = 1.53 (95% confidence interval (CI) 1.22 to 1.91). For the three trials with reliable evidence, sensitivity analysis revealed OR = 1.98 (95% CI 1.16 to 3.38).

CONCLUSIONS:

Medicines prescribed in individualised homeopathy may have small, specific treatment effects. Findings are consistent with sub-group data available in a previous ‘global’ systematic review. The low or unclear overall quality of the evidence prompts caution in interpreting the findings. New high-quality RCT research is necessary to enable more decisive interpretation.

Since my team had published an RCTs of individualised homeopathy, it seems only natural that my interest focussed on why the study (even though identified by Mathie et al) had not been included in the meta-analysis. Our study had provided no evidence that adjunctive homeopathic remedies, as prescribed by experienced homeopathic practitioners, are superior to placebo in improving the quality of life of children with mild to moderate asthma in addition to conventional treatment in primary care.

I was convinced that this trial had been rigorous and thus puzzled why, despite receiving ‘full marks’ from the reviewers, they had not included it in their meta-analysis. I thus wrote to Mathie, the lead author of the meta-analysis, and he explained: For your trial (White et al. 2003), under domain V of assessment, we were unable to extract data for meta-analysis, and so it was attributed high risk of bias, as specified by the Cochrane judgmental criteria. Our designated main outcome was the CAQ, for which we needed to know (or could at least estimate) a mean and SD for both the baseline and the end-point of the study. Since your paper reported only the change from baseline in Table 3 or in the main text, it is not possible to derive the necessary end-point for analysis.

It took a while and several further emails until I understood: our study did report both the primary (Table 2 quality of life) and secondary outcome measure (Table 3 severity of symptoms). The primary outcome measure was reported in full detail such that a meta-analysis would have been possible. The secondary outcome measure was also reported but not in full detail, and the data provided by us would not lend themselves to meta-analyses. By electing not our primary but our secondary outcome measure for their meta-analysis, Mathie et al were able to claim that they were unable to use our study and reject it for their meta-analysis.

Why did they do that?

The answer is simple: in their methods section, they specify that they used outcome measures “based on a pre-specified hierarchical list in order of greatest to least importance, recommended by the WHO“. This, I would argue is deeply flawed: the most important outcome measure of a study is usually the one for which the study was designed, not the one that some guys at the WHO feel might be important (incidentally, the WHO list was never meant to be applied to meta-analyses in that way).

By following rigidly their published protocol, the authors of the meta-analysis managed to exclude our negative trial. Thus they did everything right – or did they?

Well, I think they committed several serious mistakes.

  • Firstly, they wrote the protocol, which forced them to exclude our study. Following a protocol is not a virtue in itself; if the protocol is nonsensical it even is the opposite. Had they proceeded as is normal in such cases and used our primary outcome measure in their meta-analyses, it is most likely that their overall results would not have been in favour of homeopathy.
  • Secondly, they awarded our study a malus point for the criterium ‘selective outcome reporting’. This is clearly a wrong decision: we did report the severity-outcome, albeit not in sufficient detail for their meta-analysis. Had they not committed this misjudgment, our RCT would have been the only one with an ‘A’ rating. This would have very clearly highlighted the nonsense of excluding the best-rated trial from meta-analysis.

There are several other oddities as well. For instance, Mathie et al judge our study to be NOT free of vested interest. I asked Mathie why they had done this and was told it is because we accepted free trial medication from a homeopathic pharmacy. I would argue that my team was far less plagued by vested interest than the authors of their three best (and of course positive) trials who, as I happen to know, are consultants for homeopathic manufacturers.

And all of this is just in relation to our own study. Norbert Aust has uncovered similar irregularities with other trials and I take the liberty of quoting his comments posted previously again here:

I have reason to believe that this review and metaanalysis in biased in favor of homeopathy. To check this, I compared two studies (1) Jacobs 1994 about the treatment of childhood diarrhea in Nicaragua, (2) Walach 1997 about homeopathic threatment of headaches. The Jacobs study is one of the three that provided ‘reliable evidence’, Walach’s study earned a poor C2.2 rating and was not included in the meta-analyses. Jacobs’ results were in favour of homeopathy, Walach’s not.

For the domains where the rating of Walach’s study was less than that of the Jacobs study, please find citations from the original studies or my short summaries for the point in question.

Domain I: Sequence generation:
Walach:
“The remedy selected was then mailed to a notary public who held a stock of placebos. The notary threw a dice and mailed either the homeopathic remedy or an appropriate placebo. The notary was provided with a blank randomisation list.”
Rating: UNCLEAR (Medium risk of bias)

Jacobs:
“For each of these medications, there was a box of tubes in sequentially numbered order which had been previously randomized into treatment or control medication using a random numbers table in blocks of four”
Rating: YES (Low risk of bias)

Domain IIIb: Blinding of outcome assessor
Walach:
“The notary was provided with a blank randomization list which was an absolutely unique document. It was only handed out after the biometrician (WG) had deposited all coded original data as a printout at the notary’s office. (…) Data entry was performed blindly by personnel not involved in the study. ”
Rating: UNCLEAR (Medium risk of bias)

Jacobs:
“All statistical analyses were done before breaking the randomisation code, using the program …”
Rating: YES (Low risk of bias)

Domain V: Selective outcome reporting

Walach:
Study protocol was published in 1991 prior to enrollment of participants, all primary outcome variables were reported with respect to all participants and the endpoints.
Rating: NO (high risk of bias)

Jacobs:
No prior publication of protocol, but a pilot study exists. However this was published in 1993 only after the trial was performed in 1991. Primary outcome defined (duration of diarrhea), reported but table and graph do not match, secondary outcome (number of unformed stools on day 3) seems defined post hoc, for this is the only one point in time, this outcome yielded a significant result.
Rating: YES (low risk of bias)

Domain VI: Other sources of bias:

Walach:
Rating: NO (high risk of bias), no details given

Jacobs:
Imbalance of group properties (size, weight and age of children), that might have some impact on course of disease, high impact of parallel therapy (rehydration) by far exceeding effect size of homeopathic treatment
Rating: YES (low risk of bias), no details given

In a nutshell: I fail to see the basis for the different ratings in the studies themselves. I assume bias of the authors of the review.

Conclusion

So, what about the question posed in the title of this article? The meta-analysis is clearly not a ‘proof of concept’. But is it proof for misconduct? I asked Mathie and he answered as follows: No, your statement does not reflect the situation at all. As for each and every paper, we selected the main outcome measure for your trial using the objective WHO classification approach (in which quality of life is clearly of lower rank than severity). This is all clearly described in our prospective protocol. Under no circumstances did we approach this matter retrospectively, in the way you are implying. 

Some nasty sceptics might have assumed that the handful of rigorous studies with negative results were well-known to most researchers of homeopathy. In this situation, it would have been hugely tempting to write the protocol such that these studies must be excluded. I am thrilled to be told that the authors of the current new meta-analysis (who declared all sorts of vested interests at the end of the article) resisted this temptation.

Recent Comments

Click here for a comprehensive list of recent comments.

Categories