clinical trial

We have probably all fallen into the trap of thinking that something which has stood the ‘test of time’, i.e. something that has been used for centuries with apparent success, must be ok. In alternative medicine, this belief is extremely wide-spread, and one could argue that the entire sector is built on it. Influential proponents of ‘traditional’ medicine like Prince Charles do their best to strengthen this assumption. Sadly, however, it is easily disclosed as a classical fallacy: things that have stood the ‘test of time’ might work, of course, but the ‘test of time’ is never a proof of anything.

A recent study brought this message home loud and clear. This trial tested the efficacy of Rhodiola crenulata (R. crenulata), a traditional remedy which has been used widely in the Himalayan areas and in Tibet to prevent acute mountain sickness . As no scientific studies of this traditional treatment existed, the researchers conducted a double-blind, placebo-controlled crossover RCT to test its efficacy in acute mountain sickness prevention.

Healthy adult volunteers were randomized to two treatment sequences, receiving either 800 mg R. crenulata extract or placebo daily for 7 days before ascent and two days during mountaineering. After a three-month wash-out period, they were crossed over to the alternate treatment. On each occasion, the participants ascended rapidly from 250 m to 3421 m. The primary outcome measure was the incidence of acute mountain sickness with headache and at least one of the symptoms of nausea or vomiting, fatigue, dizziness, or difficulty sleeping.

One hundred and two participants completed the trial. No significant differences in the incidence of acute mountain sickness were found between R. crenulata extract and placebo groups. If anything, the incidence of severe acute mountain sickness with Rhodiola extract was slightly higher compared to the one with placebo: 35.3% vs. 29.4%.

R. crenulata extract was not effective in reducing the incidence or severity of acute mountain sickness as compared to placebo.

Similar examples could be found by the dozen. They demonstrate very clearly that the notion of the ‘test of time’ is erroneous: a treatment which has a long history of usage is not necessarily effective (or safe) – not only that, it might be dangerous. The true value of a therapy cannot be judged by experience, to be sure, we need rigorous clinical trials. Acute mountain sickness is a potentially life-threatening condition for which there are reasonably effective treatments. If people relied on the ‘ancient wisdom’ instead of using a therapy that actually works, they might pay for their error with their lives. The sooner alternative medicine proponents realise that, the better.

Acupressure is a treatment-variation of acupuncture; instead of sticking needles into the skin, pressure is applied over ‘acupuncture points’ which is supposed to provide a stimulus similar to needling. Therefore the effects of both treatments should theoretically be similar.

Acupressure could have several advantages over acupuncture:

  • it can be used for self-treatment
  • it is suitable for people with needle-phobia
  • it is painless
  • it is not invasive
  • it has less risks
  • it could be cheaper

But is acupressure really effective? What do the trial data tell us? Our own systematic review concluded that the effectiveness of acupressure is currently not well documented for any condition. But now there is a new study which might change this negative verdict.

The primary objective of this 3-armed RCT was to assess the effectiveness and cost-effectiveness of self-acupressure using wristbands compared with sham acupressure wristbands and standard care alone in the management of chemotherapy-induced nausea. 500 patients from outpatient chemotherapy clinics in three regions in the UK involving 14 different cancer units/centres were randomised to the wristband arm, the sham wristband arm and the standard care only arm. Participants were chemotherapy-naive cancer patients receiving chemotherapy of low, moderate and high emetogenic risk. The experimental group were given acupressure wristbands pressing the P6 point (anterior surface of the forearm). The Rhodes Index for Nausea/Vomiting, the Multinational Association of Supportive Care in Cancer (MASCC) Antiemesis Tool and the Functional Assessment of Cancer Therapy General (FACT-G) served as outcome measures. At baseline, participants completed measures of anxiety/depression, nausea/vomiting expectation and expectations from using the wristbands.

Data were available for 361 participants for the primary outcome. The primary outcome analysis (nausea in cycle 1) revealed no statistically significant differences between the three arms. The median nausea experience in patients using wristbands (both real and sham ones) was somewhat lower than that in the anti-emetics only group (median nausea experience scores for the four cycles: standard care arm 1.43, 1.71, 1.14, 1.14; sham acupressure arm 0.57, 0.71, 0.71, 0.43; acupressure arm 1.00, 0.93, 0.43, 0). Women responded more favourably to the use of sham acupressure wristbands than men (odds ratio 0.35 for men and 2.02 for women in the sham acupressure group; 1.27 for men and 1.17 for women in the acupressure group). No significant differences were detected in relation to vomiting outcomes, anxiety and quality of life. Some transient adverse effects were reported, including tightness in the area of the wristbands, feeling uncomfortable when wearing them and minor swelling in the wristband area (n = 6). There were no statistically significant differences in the costs associated with the use of real acupressure band.

26 subjects took part in qualitative interviews. Participants perceived the wristbands (both real and sham) as effective and helpful in managing their nausea during chemotherapy.

The authors concluded that there were no statistically significant differences between the three arms in terms of nausea, vomiting and quality of life, although apparent resource use was less in both the real acupressure arm and the sham acupressure arm compared with standard care only; therefore; no clear conclusions can be drawn about the use of acupressure wristbands in the management of chemotherapy-related nausea and vomiting. However, the study provided encouraging evidence in relation to an improved nausea experience and some indications of possible cost savings to warrant further consideration of acupressure both in practice and in further clinical trials.

I could argue about several of the methodological details of this study. But I resist the temptation in order to focus on just one single point which I find important and which has implications beyond the realm of acupressure.

Why on earth do the authors conclude that no clear conclusions can be drawn about the use of acupressure wristbands in the management of chemotherapy-related nausea and vomiting? The stated aim of this RCT was to assess the effectiveness and cost-effectiveness of self-acupressure using wristbands compared with sham acupressure wristbands and standard care. The results failed to show significant differences of the primary outcome measures, consequently the conclusion cannot be “unclear”, it has to be that ACUPRESSURE WRIST BANDS ARE NOT MORE EFFECTIVE THAN SHAM ACUPRESSURE WRIST BANDS AS AN ADJUNCT TO ANTI-EMETIC DRUG TREATMENT (or something to that extent).

As long as RCTs of alternative therapies are run by evangelic believers in the respective therapy, we are bound to regularly encounter this lamentable phenomenon of white-washing negative findings with an inadequate conclusion. In my view, this is not research or science, it is pseudo-research or pseudo-science. And it is much more than a nuisance or a trivial matter; it is a waste of research funds, a waste of patients’ good will that has reached a point where people will lose trust in alternative medicine research. Someone should really do a systematic study to identify those research teams that regularly commit such scientific misconduct and ensure that they are cut off public funding and support.

This post will probably work best, if you have read the previous one describing how the parallel universe of acupuncture research insists on going in circles in order to avoid admitting that their treatment might not be as effective as they pretend. The way they achieve this is fairly simple: they conduct trials that are designed in such a way that they cannot possibly produce a negative result.

A brand-new investigation which was recently vociferously touted via press releases etc. as a major advance in proving the effectiveness of acupuncture is an excellent case in point. According to its authors, the aim of this study was to evaluate acupuncture versus usual care and counselling versus usual care for patients who continue to experience depression in primary care. This sounds alright, but wait!

755 patients with depression were randomised to one of three arms to 1)acupuncture, 2)counselling, and 3)usual care alone. The primary outcome was the difference in mean Patient Health Questionnaire (PHQ-9) scores at 3 months with secondary analyses over 12 months follow-up. Analysis was by intention-to-treat. PHQ-9 data were available for 614 patients at 3 months and 572 patients at 12 months. Patients attended a mean of 10 sessions for acupuncture and 9 sessions for counselling. Compared to usual care, there was a statistically significant reduction in mean PHQ-9 depression scores at 3 and 12 months for acupuncture and counselling.

From this, the authors conclude that both interventions were associated with significantly reduced depression at 3 months when compared to usual care alone.

Acupuncture for depression? Really? Our own systematic review with co-authors who are the most ardent apologists of acupuncture I have come across showed that the evidence is inconsistent on whether manual acupuncture is superior to sham… Therefore, I thought it might be a good idea to have a closer look at this new study.

One needs to search this article very closely indeed to find out that the authors did not actually evaluate acupuncture versus usual care and counselling versus usual care at all, and that comparisons were not made between acupuncture, counselling, and usual care (hints like the use of the word “alone” are all we get to guess that the authors’ text is outrageously misleading). Not even the methods section informs us what really happened in this trial. You find this hard to believe? Here is the unabbreviated part of the article that describes the interventions applied:

Patients allocated to the acupuncture and counselling groups were offered up to 12 sessions usually on a weekly basis. Participating acupuncturists were registered with the British Acupuncture Council with at least 3 years post-qualification experience. An acupuncture treatment protocol was developed and subsequently refined in consultation with participating acupuncturists. It allowed for customised treatments within a standardised theory-driven framework. Counselling was provided by members of the British Association for Counselling and Psychotherapy who were accredited or were eligible for accreditation having completed 400 supervised hours post-qualification. A manualised protocol, using a humanistic approach, was based on competences independently developed for Skills for Health. Practitioners recorded in logbooks the number and length of sessions, treatment provided, and adverse events. Further details of the two interventions are presented in Tables S2 and S3. Usual care, both NHS and private, was available according to need and monitored for all patients in all three groups for the purposes of comparison.

It is only in the results tables that we can determine what treatments were actually given; and these were:

1) Acupuncture PLUS usual care (i.e. medication)

2) Counselling PLUS usual care

3) Usual care

Its almost a ‘no-brainer’ that, if you compare A+B to B (or in this three-armed study A+B vs C+B vs B), you find that the former is more than the latter – unless A is a negative, of course. As acupuncture has significant placebo-effects, it never can be a negative, and thus this trial is an entirely foregone conclusion. As, in alternative medicine, one seems to need experimental proof even for ‘no-brainers’, we have some time ago demonstrated that this common sense theory is correct by conducting a systematic review of all acupuncture trials with such a design. We concluded that the ‘A + B versus B’ design is prone to false positive results…What makes this whole thing even worse is the fact that I once presented our review in a lecture where the lead author of the new trial was in the audience; so there can be no excuse of not being aware of the ‘no-brainer’.

Some might argue that this is a pragmatic trial, that it would have been unethical to not give anti-depressants to depressed patients and that therefore it was not possible to design this study differently. However, none of these arguments are convincing, if you analyse them closely (I might leave that to the comment section, if there is interest in such aspects). At the very minimum, the authors should have explained in full detail what interventions were given; and that means disclosing these essentials even in the abstract (and press release) – the part of the publication that is most widely read and quoted.

It is arguably unethical to ask patients’ co-operation, use research funds etc. for a study, the results of which were known even before the first patient had been recruited. And it is surely dishonest to hide the true nature of the design so very sneakily in the final report.

In my view, this trial begs at least 5 questions:

1) How on earth did it pass the peer review process of one of the most highly reputed medical journals?

2) How did the protocol get ethics approval?

3) How did it get funding?

4) Does the scientific community really allow itself to be fooled by such pseudo-research?

5) What do I do to not get depressed by studies of acupuncture for depression?

Has it ever occurred to you that much of the discussion about cause and effect in alternative medicine goes in circles without ever making progress? I have come to the conclusion that it does. Here I try to illustrate this point using the example of acupuncture, more precisely the endless discussion about how to best test acupuncture for efficacy. For those readers who like to misunderstand me I should explain that the sceptics’ view is in capital letters.

At the beginning there was the experience. Unaware of anatomy, physiology, pathology etc., people started sticking needles in other people’s skin, some 2000 years ago, and observed that they experienced relief of all sorts of symptoms.When an American journalist reported about this phenomenon in the 1970s, acupuncture became all the rage in the West. Acupuncture-fans then claimed that a 2000-year history is ample proof that acupuncture does work.


Even the most enthusiastic advocates conceded that this is probably true. So they documented detailed case-series of lots of patients, calculated the average difference between the pre- and post-treatment severity of symptoms, submitted it to statistical tests, and published the notion that the effects of acupuncture are not just anecdotal; in fact, they are statistically significant, they said.


“True enough”, grumbled the acupuncture-fans and conducted the very first controlled clinical trials. Essentially they treated one group of patients with acupuncture while another group received conventional treatments as usual. When they analysed the results, they found that the acupuncture group had improved significantly more. “Now do you believe us?”, they asked triumphantly, “acupuncture is clearly effective”.


The acupuncturists felt slightly embarrassed because they had not thought of that. They had allocated their patients to the treatment according to patients’ choice. Thus the expectation of the patients (or the clinician) to get relief from acupuncture might have been the reason for the difference in outcome. So they consulted an expert in trial-design and were advised to allocate not by choice but by chance. In other words, they repeated the previous study but randomised patients to the two groups. Amazingly, their RCT still found a significant difference favouring acupuncture over treatment as usual.


Now the acupuncturists were in a bit of a pickle; as far as they could see, there was no good placebo for acupuncture! Eventually some methodologist-chap came up with the idea that, in order to mimic a placebo, they could simply stick needles into non-acupuncture points. When the acupuncturists tried that method, they found that there were improvements in both groups but the difference between real acupuncture and placebo was tiny and usually neither statistically significant nor clinically relevant.


Absolutely not! The results merely show that needling non-acupuncture points is not an adequate placebo. Obviously this intervention also sends a powerful signal to the brain which clearly makes it an effective intervention. What do you expect when you compare two effective treatments?


At that stage, the acupuncturists came up with a placebo-needle that did not actually penetrate the skin; it worked like a mini stage dagger that telescopes into itself while giving the impression that it penetrated the skin just like the real thing. Surely this was an adequate placebo! The acupuncturists repeated their studies but, to their utter dismay, they found again that both groups improved and the difference in outcome between their new placebo and true acupuncture was minimal.


Certainly not, they replied. We have thought long and hard about these intriguing findings and believe that they can be explained just like the last set of results: the non-penetrating needles touch the skin; this touch provides a stimulus powerful enough to have an effect on the brain; the non-penetrating placebo-needles are not inert and therefore the results merely depict a comparison of two effective treatments.


We had many discussions and consensus meeting amongst the most brilliant brains in acupuncture about this issue and have arrived at the conclusion that your obsession with placebo, cause and effect etc. is ridiculous and entirely misplaced. In real life, we don’t use placebos. So, let’s instead address the ‘real life’ question: is acupuncture better than usual treatment? We have conducted pragmatic studies where one group of patients gets treatment as usual and the other group receives acupuncture in addition. These studies show that acupuncture is effective. This is all the evidence we need. Why can you not believe us?


One cannot very well write a blog about alternative medicine without giving full credit to the biggest and probably most determined champion of quackery who ever hugged a tree. Prince Charles certainly has done more than anyone else I know to let unproven treatments infiltrate real medicine. To honour his unique achievements, I am here presenting a fictitious interview with him. It never did take place, of course, and the questions I put to him are pure imagination. However, the ‘answers’ are in a way quite real: they have been taken unaltered from various speeches he made and articles he wrote. To avoid being accused of using dodgy sources which might have quoted him inaccurately or sympathetically, I have exclusively used HRH’s very own official website as a source for his comments. It seems safe to assume that HRH identifies with them more fully than with the many other statements he made on this subject.

I have not changed a single word in his statements and I have tried to avoid quoting him out of context; I did, however, take the liberty of putting sentences side by side which do not always originate from the same speech or article, i.e. I have used quotes from different communications to appear as though they originally were in sequence. It will be clear from the text that the fictitious interview is dated before Charles’ Foundation folded because of money laundering and fraud.

It is, of course, hugely tempting to comment on the various statements by Charles. However, I have resisted this temptation; I wanted the reader to enjoy his wisdom in its pure and unadulterated beauty. Anyone who feels like it will have plenty of opportunity to post comments, if they so wish.

To make clear what is what, my questions appear in italics, while his ‘answers’ are in Roman typeface.


Q I believe you have no training in science or medicine; yet you have long felt yourself expert enough to champion bizarre forms of therapies which many of our readers might call quackery.

As you know by now, this is an area to which I attach the greatest importance and where I have tried to make a particular contribution. For many years, the NHS has found complementary medicine an uncomfortable bedfellow – at best regarded as ‘fringe’ and in some quarters as ‘quack’; never viewed as a substitute for conventional medicine and rarely as a genuine partner in providing therapy.

I look back to the rather “lukewarm” response I received in 1983 as President of the British Medical Association when I first spoke about integration and complementary and alternative medicine. We have clearly travelled a very long way since that time.

Q Alternative medicine is mainly used by those who can afford it; at present, little of it is available on the NHS. Why do you want to change this situation? 

The very popularity of non-conventional approaches suggests that people are either dissatisfied with the kind of orthodox treatment they are receiving, or find genuine relief in such therapies. Whatever the case, it is only reasonable to try to identify the factors that are contributing to their increased use. And if advantages are found, clearly they should not be limited only to those people who can pay, but should be made more widely available on the NHS.

Q If with a capital “I”?

I believe it is because complementary and alternative approaches to healthcare bring a different emphasis to bear which often unlocks an individual’s inner resources to aid recovery or help to manage living with a serious chronic illness. It is also because complementary and alternative therapies often offer more effective and less intrusive ways of treating illness.

Q Really? Are you sure that they are more effective that conventional treatments? What is your evidence for that?

In 1997 the Foundation for Integrated Medicine, of which I am the president and founder, identified research and development based on rigorous scientific evidence as one of the keys to the medical establishment’s acceptance of non-conventional approaches. I believed then, as I do now, that the move to a more integrated provision of healthcare would ultimately benefit patients and their families.

Q But belief is hardly a good substitute for evidence. In this context, it is interesting to note that chiropractors and osteopaths received the same status as doctors and nurses in the UK. Is this another of your achievements? Was it based on belief or on evidence?

True healing is a synergy that comes not by courtesy of a medical diploma.

Q What do you mean?

As we know, the professions of Osteopathy and Chiropractice are now regulated in the same way as doctors and dentists, with their own Acts of Parliament. I’m very proud to have played a tiny role in trying to push for that Act of Parliament over the years. It has also been reassuring to see the progress being made by the other main complementary professions and I look forward to the further development of regulatory frameworks enabling high standards of training, clinical practice and professional behaviour.

Q Some might argue that statutory regulation made them not more professional but merely improved their status and thus prevented asking question about evidence. Why did they need to be regulated in that way?

The House of Lord’s Select Committee Report on Complementary and Alternative Medicine in 2000, quite sensibly recommended that only complementary professions which were statutorily regulated, or which had well-established arrangements for voluntary self-regulation, should be made available through the NHS.

Q Integrated healthcare seems to be your new buzz-word, what does it mean? Is it more than a passing fad?

Integrated Healthcare is, I believe, here to stay. The public want it and need it. It is not a takeover of the orthodox by CAM or the other way around, but is rather the bringing together of the best from both for the ultimate benefit of the patient.

Q Your lobby-group, Foundation for Integrated Medicine, what has it ever done to justify its existence?

In 1997 the steering group of The Foundation for Integrated Medicine (FIM), of which I am proud to be president, published a discussion document ‘Integrated Healthcare – A Way Forward for the Next Five Years?’

Q Sorry to interrupt, but if so many people are already using them, why do you feel compelled to promote unproven treatments even more? Why is ‘a way forward’ in promotion actually needed? Why did we need a lobby group like FIM?

Homoeopaths, osteopaths, reflexologists, acupuncturists, T’ai chi instructors, art therapists, chiropractors, herbalists and aromatherapists: these practitioners were working alongside NHS colleagues in acute hospitals, on children’s wards, in nursing homes and in particular in primary healthcare, in GP practices and health clinics up and down the country.

Q Exactly! Why then even more promotion of unproven treatments?

All well and good, perhaps, but if there are advantages in this approach, clearly they should not be limited only to those who can pay.

Q Yes, if again with a capital “I”, presumably . Anyway, do you believe these therapies should be tested like other treatments?

One of the obstacles always raised is that it is very difficult to trial complementary therapies in the rigorous randomised way that mainstream medicine deems to be the gold standard. This is ironic as there are, of course, un-evaluated orthodox practices which continue to be funded by the NHS.

Q Are you an expert on research methodology as well?

At the same time, we should be mindful that clinically controlled trials alone are not the only pre-requisites to apply a healthcare intervention. Consumer-based surveys can explore WHY people choose complementary and alternative medicine and tease out the therapeutic powers of belief and trust

These “rationalist selves” would be enormously relieved to see the effectiveness of these treatments proven through the “double-blind randomized controlled trial” – the gold-standard of medical research. However, we know that some complementary and alternative medicine disciplines (and indeed other forms of medical or surgical intervention) do not lend themselves to this research method.

Q Are you sure? This sounds like something someone who is ignorant of research methodology has told you.

… it has been suggested that we need a research method for complementary treatment that is, to use that awful expression, “fit for purpose”. Something that is entirely practical – what has been called “applied” research – which takes into account the whole person and the whole treatment as it is actually given in the surgery or the hospital. Something that might offer us a better idea of the cost-effectiveness of any given approach. It would also help to provide the right sort of evidence that health service commissioners require when they decide which services they wish to commission for their patients.

Q Hmm – anyway, would you promote unproven treatments even for serious conditions like cancer?

Two surveys have indicated that up to eighty per cent of cancer patients try alternative or complementary treatments at some stage following diagnosis and seventy-five per cent of patients would like to see complementary medicine available on the N.H.S.

Q Yes, but why the promotion?

There is a major role for complementary medicine in bowel cancer – as a support to more conventional approaches – in helping to prevent it through lifestyle changes, helping to boost our immune systems and in helping sufferers to come to terms with, and maintain, a sense of control over their own lives and wellbeing. My own Foundation For Integrated Medicine is, for example, involved in finding ways to integrate the best of complementary and alternative medicine.

Q And what do you understand by “the best”? In medicine, this term should mean “the most effective”, shouldn’t it?

Many cancer patients have turned to an integrated approach to managing their health, finding complementary therapies such as acupuncture, aromatherapy, reflexology and massage therapy extremely therapeutic. I know of one patient who turned to Gerson Therapy having been told that she was suffering from terminal cancer, and would not survive another course of chemotherapy. Happily, seven years later she is alive and well. So it is therefore vital that, rather than dismissing such experiences, we should further investigate the beneficial nature of these treatments.

Q Gerson? Is it ethical to promote an unproven starvation diet for cancer? 

…many patients use and believe in Gerson Therapy, yet more evidence needs to be available as to who might benefit or what adverse effects there might be. But, surely, we need to take a wider view of the most appropriate types of research methodology – a wider view of what research will help patients.

Q You are a very wealthy man; will you put your own money into the research that you regularly demand?

Complementary medicine is gaining a toehold on the rockface of medical science.

Q I beg your pardon.

Complementary medicine’s toehold is literally that, and it’s an inescapable fact that clinical trials, of the calibre that medical science demands, cost money. Figures from the Department of Complementary Medicine at the University of Exeter show that less than 8p out of every £100 of NHS funds for medical research was spent on complementary medicine. In 1998-99 the Medical Research Council spent no money on it at all, and in 1999 only 0.05% of the total research budget of UK medical charities went to this area.

Q HmmNature; you are very fond of all things natural, aren’t you?

The garden is designed to remind people of our interconnectedness with Nature and of the beneficial medicinal properties She provides through countless plants, flowers and trees. Throughout the 20th century so much ancient, accumulated, traditional wisdom has been thrown away – whether in the fields of medicine, architecture, agriculture or education. The baby was thrown out with the bathwater, so this garden is designed to bring the baby back again and to remind us of that priceless, traditional knowledge before we lose that rich store of Nature’s healing gifts for the benefit of our descendants.

When you think about it, what on earth is the point of throwing away our lifeline; of abandoning the priceless knowledge and wisdom accumulated over 1,000’s of years relating to the treatment of the human condition by natural means? It is sheer folly it seems to me to forget that we are a part of Nature and to imagine we can survive on this Earth as if we were merely a mechanical process divorced from, and in opposition to, the unity of the world around us.

Q …and herbalism?

Medical herbalists talk about ‘synergy’, the result of a complex mix of active ingredients in a plant that create a more powerful therapeutic effect together than if isolated. It’s a concept that has a wider application. As the 17th century poet John Donne famously wrote, “No man is an Island, entire of itself; every man is a piece of the Continent, a part of the main.”

Q I am not sure I understand; what does that mean?

Medical herbalists, who make up their own preparations from combinations of fresh or dried plants, believe that this mix within individual herbs as well as in traditional mixtures of plant medicines creates what is called synergy, in which all the chemical components contribute to the remedy’s specific therapeutic effects.

At a time when farmers everywhere are struggling to make ends meet, the development of a natural pharmacy of organically grown herbs offers an alternative means of earning a living. Yet without protective measures, herbs are easily adulterated or their quality compromised.

Q …and homeopathy?

I went to open the new Glasgow Homeopathic Hospital for instance a couple of years ago, I met a whole lot of students who were studying homeopathy, I think, and I’ve never forgotten when they said to me ‘Are you interested in homeopathy’ and I thought – I don’t know, why do I bother?

Q And why exactly do you bother, if I may ask?

By allowing patients treatment choice, negative emotions can, in part, be alleviated. Many complementary practitioners provide time, empathy, hope and reassurance – skills that are referred to as the “human effect” – which can improve the confidence of cancer patients, alter mindsets and produce major positive changes in the immune system. As a result the “human effect” can greatly prolong life: it has been demonstrated that in a variety of cancers, such as breast cancer, that attitude of mind can not only raise the quality of life but in some cases can even prolong life. At the same time, we need specific treatments that are designed to improve the quality of patients’ lives, and to provide relief from the unpleasant symptoms of cancer – anxiety; pain; sleeplessness; skin irritation; poor appetite; nausea and depression, to name but a few.

Q At heart you seem to be a vitalist who believes in a vital force or energy that interconnects anything with everything and determines our health.

Research in the new field of psychoneuroimmunology – or mind-body medicine as it is sometimes called – is discovering that there is a constant interplay between our emotions, thoughts and actions and our body systems. It seems that the food we eat, the air we breathe, the exercise we take, our relationships with other people, all have a direct bearing on our health and natural healing processes. Complementary medicine has always known this and I believe it is one of the reasons for its enormous popularity.

Q Clarence House made several statements assuring the British public that you never overstep your constitutional role by trying to influence health politics; they were having us on, weren’t they?

A few days ago I launched an initiative to promote the provision of more complementary medicine in the NHS. For many years I have been working towards this goal.

Q Does that mean these statements were wrong?

I am convinced there is no better moment than now to create a real integration of our healthcare, particularly when there is talk of a Patient-Centred NHS. So much ill-health and disease is due to the misery, stress and alienation we see in our community.

The fish oil (FO) story began when a young Danish doctor noticed that there were no heart attacks in Greenland. Large epidemiological studies were initiated, mechanistic investigations followed, and a huge amount of fascinating data emerged. Today, we know more about FO than most other dietary supplements.

Fish oil contains large amounts of omega-3 fatty acids which are thought to be beneficial in treating hypertriglyceridemia,  preventing heart disease.  In addition, FO is often recommended for a wide variety of other conditions, such as  cancer, depression, and macular degeneration. Perhaps the most compelling evidence exists in the realm of inflammatory diseases; the mechanism of action of FO is well-studied and includes powerful anti-inflammatory properties.

Australian rheumatologists just published a study of FO supplements for patients suffering from rheumatoid arthritis (RA). Specifically, they examined  the effects of high versus low dose FO in early RA employing a ‘treat-to-target’ protocol of combination disease-modifying anti-rheumatic drugs (DMARDs).

Patients with chronic RA <12 months’ who were DMARD-naïve were enrolled and randomised 2:1 to FO at a high dose or plaacebo (low dose FO for masking). These groups were given 5.5 or 0.4 g/day, respectively, of  eicosapentaenoic acid + docosahexaenoic acid. All patients received methotrexate (MTX), sulphasalazine and hydroxychloroquine, and DMARD doses were adjusted according to an algorithm taking disease activity and toxicity into account. DAS28-erythrocyte sedimentation rate, modified Health Assessment Questionnaire (mHAQ) and remission were assessed three monthly. The primary outcome measure was failure of triple DMARD therapy.

The results indicate that, the FO group, failure of triple DMARD therapy was lower (HR=0.28 (95% CI 0.12 to 0.63; p=0.002) unadjusted and 0.24 (95% CI 0.10 to 0.54; p=0.0006) following adjustment for smoking history, shared epitope and baseline anti–cyclic citrullinated peptide. The rate of first American College of Rheumatology (ACR) remission was significantly greater in the FO compared with the control group (HRs=2.17 (95% CI 1.07 to 4.42; p=0.03) unadjusted and 2.09 (95% CI 1.02 to 4.30; p=0.04) adjusted). There were no differences between groups in MTX dose, DAS28 or mHAQ scores, or adverse events.

The authors conclude that FO was associated with benefits additional to those achieved by combination ‘treat-to-target’ DMARDs with similar MTX use. These included reduced triple DMARD failure and a higher rate of ACR remission.

These findings are most encouraging, particularly as they collaborate those of systematic reviews which concluded that evidence is seen for a fairly consistent, but modest, benefit of marine n-3 PUFAs on joint swelling and pain, duration of morning stiffness, global assessments of pain and disease activity, and use of non-steroidal anti-inflammatory drugs and …there is evidence from 6 of 14 randomized controlled trials supporting a favourable effect of n-3 LCP supplementation in decreasing joint inflammation in RA. And you don’t need to buy the supplements either; regularly eating lots of fatty fish like mackerel, sardine or salmon has the same effects.

So, here we have an alternative, ‘natural’, dietary supplement or diet that is supported by reasonably sound evidence for efficacy, that has very few adverse effects (the main one being contamination of the supplement with toxins), that generates a host of potentially useful effects on other organ systems, that is affordable, that has a plausible mechanism of action…. Hold on, I hear some people interrupting me, FO is not an alternative medicine, it is mainstream! Exactly, an alternative medicine that works is called….MEDICINE.

It was 20 years ago today that I started my job as ‘Professor of Complementary Medicine’ at the University of Exeter and became a full-time researcher of all matters related to alternative medicine. One issue that was discussed endlessly during these early days was the question whether alternative medicine can be investigated scientifically. There were many vociferous proponents of the view that it was too subtle, too individualised, too special for that and that it defied science in principle. Alternative medicine, they claimed, needed an alternative to science to be validated. I spent my time arguing the opposite, of course, and today there finally seems to be a consensus that alternative medicine can and should be submitted to scientific tests much like any other branch of health care.

Looking back at those debates, I think it is rather obvious why apologists of alternative medicine were so vehement about opposing scientific investigations: they suspected, perhaps even knew, that the results of such research would be mostly negative. Once the anti-scientists saw that they were fighting a lost battle, they changed their tune and adopted science – well sort of: they became pseudo-scientists (‘if you cannot beat them, join them’). Their aim was to prevent disaster, namely the documentation of alternative medicine’s uselessness by scientists. Meanwhile many of these ‘anti-scientists turned pseudo-scientists’ have made rather surprising careers out of their cunning role-change; professorships at respectable universities have mushroomed. Yes, pseudo-scientists have splendid prospects these days in the realm of alternative medicine.

The term ‘pseudo-scientist’ as I understand it describes a person who thinks he/she knows the truth about his/her subject well before he/she has done the actual research. A pseudo-scientist is keen to understand the rules of science in order to corrupt science; he/she aims at using the tools of science not to test his/her assumptions and hypotheses, but to prove that his/her preconceived ideas were correct.

So, how does one become a top pseudo-scientist? During the last 20 years, I have observed some of the careers with interest and think I know how it is done. Here are nine lessons which, if followed rigorously, will lead to success (… oh yes, in case I again have someone thick enough to complain about me misleading my readers: THIS POST IS SLIGHTLY TONGUE IN CHEEK).

  1. Throw yourself into qualitative research. For instance, focus groups are a safe bet. This type of pseudo-research is not really difficult to do: you assemble about 5 -10 people, let them express their opinions, record them, extract from the diversity of views what you recognise as your own opinion and call it a ‘common theme’, write the whole thing up, and – BINGO! – you have a publication. The beauty of this approach is manifold: 1) you can repeat this exercise ad nauseam until your publication list is of respectable length; there are plenty of alternative medicine journals who will hurry to publish your pseudo-research; 2) you can manipulate your findings at will, for instance, by selecting your sample (if you recruit people outside a health food shop, for instance, and direct your group wisely, you will find everything alternative medicine journals love to print); 3) you will never produce a paper that displeases the likes of Prince Charles (this is more important than you may think: even pseudo-science needs a sponsor [or would that be a pseudo-sponsor?]).
  2. Conduct surveys. These are very popular and highly respected/publishable projects in alternative medicine – and they are almost as quick and easy as focus groups. Do not get deterred by the fact that thousands of very similar investigations are already available. If, for instance, there already is one describing the alternative medicine usage by leg-amputated police-men in North Devon, and you nevertheless feel the urge of going into this area, you can safely follow your instinct: do a survey of leg-amputated police men in North Devon with a medical history of diabetes. There are no limits, and as long as you conclude that your participants used a lot of alternative medicine, were very satisfied with it, did not experience any adverse effects, thought it was value for money, and would recommend it to their neighbour, you have secured another publication in an alternative medicine journal.
  3. If, for some reason, this should not appeal to you, how about taking a sociological, anthropological or psychological approach? How about studying, for example, the differences in worldviews, the different belief systems, the different ways of knowing, the different concepts about illness, the different expectations, the unique spiritual dimensions, the amazing views on holism – all in different cultures, settings or countries? Invariably, you will, of course, conclude that one truth is at least as good as the next. This will make you popular with all the post-modernists who use alternative medicine as a playground for getting a few publications out. This approach will allow you to travel extensively and generally have a good time. Your papers might not win you a Nobel prize, but one cannot have everything.
  4. It could well be that, at one stage, your boss has a serious talk with you demanding that you start doing what (in his narrow mind) constitutes ‘real science’. He might be keen to get some brownie-points at the next RAE and could thus want you to actually test alternative treatments in terms of their safety and efficacy. Do not despair! Even then, there are plenty of possibilities to remain true to your pseudo-scientific principles. By now you are good at running surveys, and you could, for instance, take up your boss’ suggestion of studying the safety of your favourite alternative medicine with a survey of its users. You simply evaluate their experiences and opinions regarding adverse effects. But be careful, you are on somewhat thinner ice here; you don’t want to upset anyone by generating alarming findings. Make sure your sample is small enough for a false negative result, and that all participants are well-pleased with their alternative medicine. This might be merely a question of selecting your patients cleverly. The main thing is that your conclusion is positive. If you want to go the extra pseudo-scientific mile, mention in the discussion of your paper that your participants all felt that conventional drugs were very harmful.
  5. If your boss insists you tackle the daunting issue of therapeutic efficacy, there is no reason to give up pseudo-science either. You can always find patients who happened to have recovered spectacularly well from a life-threatening disease after receiving your favourite form of alternative medicine. Once you have identified such a person, you write up her experience in much detail and call it a ‘case report’. It requires a little skill to brush over the fact that the patient also had lots of conventional treatments, or that her diagnosis was assumed but never properly verified. As a pseudo-scientist, you will have to learn how to discretely make such irritating details vanish so that, in the final paper, they are no longer recognisable. Once you are familiar with this methodology, you can try to find a couple more such cases and publish them as a ‘best case series’ – I can guarantee that you will be all other pseudo-scientists’ hero!
  6. Your boss might point out, after you have published half a dozen such articles, that single cases are not really very conclusive. The antidote to this argument is simple: you do a large case series along the same lines. Here you can even show off your excellent statistical skills by calculating the statistical significance of the difference between the severity of the condition before the treatment and the one after it. As long as you show marked improvements, ignore all the many other factors involved in the outcome and conclude that these changes are undeniably the result of the treatment, you will be able to publish your paper without problems.
  7. As your boss seems to be obsessed with the RAE and all that, he might one day insist you conduct what he narrow-mindedly calls a ‘proper’ study; in other words, you might be forced to bite the bullet and learn how to plan and run an RCT. As your particular alternative therapy is not really effective, this could lead to serious embarrassment in form of a negative result, something that must be avoided at all cost. I therefore recommend you join for a few months a research group that has a proven track record in doing RCTs of utterly useless treatments without ever failing to conclude that it is highly effective. There are several of those units both in the UK and elsewhere, and their expertise is remarkable. They will teach you how to incorporate all the right design features into your study without there being the slightest risk of generating a negative result. A particularly popular solution is to conduct what they call a ‘pragmatic’ trial, I suggest you focus on this splendid innovation that never fails to produce anything but cheerfully positive findings.
  8. It is hardly possible that this strategy fails – but once every blue moon, all precautions turn out to be in vain, and even the most cunningly designed study of your bogus therapy might deliver a negative result. This is a challenge to any pseudo-scientist, but you can master it, provided you don’t lose your head. In such a rare case I recommend to run as many different statistical tests as you can find; chances are that one of them will nevertheless produce something vaguely positive. If even this method fails (and it hardly ever does), you can always home in on the fact that, in your efficacy study of your bogus treatment, not a single patient died. Who would be able to doubt that this is a positive outcome? Stress it clearly, select it as the main feature of your conclusions, and thus make the more disappointing findings disappear.
  9. Now that you are a fully-fledged pseudo-scientist who has produced one misleading or false positive result after the next, you may want a ‘proper’ confirmatory study of your pet-therapy. For this purpose run the same RCT over again, and again, and again. Eventually you want a meta-analysis of all RCTs ever published. As you are the only person who ever conducted studies on the bogus treatment in question, this should be quite easy: you pool the data of all your trials and, bob’s your uncle: a nice little summary of the totality of the data that shows beyond doubt that your therapy works. Now even your narrow-minded boss will be impressed.

These nine lessons can and should be modified to suit your particular situation, of course. Nothing here is written in stone. The one skill any pseudo-scientist must have is flexibility.

Every now and then, some smart arse is bound to attack you and claim that this is not rigorous science, that independent replications are required, that you are biased etc. etc. blah, blah, blah. Do not panic: either you ignore that person completely, or (in case there is a whole gang of nasty sceptics after you) you might just point out that:

  • your work follows a new paradigm; the one of your critics is now obsolete,
  • your detractors fail to understand the complexity of the subject and their comments merely reveal their ridiculous incompetence,
  • your critics are less than impartial, in fact, most are bought by BIG PHARMA,
  • you have a paper ‘in press’ that fully deals with all the criticism and explains how inappropriate it really is.

In closing, allow me a final word about publishing. There are hundreds of alternative medicine journals out there to chose from. They will love your papers because they are uncompromising promotional. These journals all have one thing in common: they are run by apologists of alternative medicine who abhor to read anything negative about alternative medicine. Consequently hardly a critical word about alternative medicine will ever appear in these journals. If you want to make double sure that your paper does not get criticised during the peer-review process (this would require a revision, and you don’t need extra work of that nature), you can suggest a friend for peer-reviewing it. In turn, you can offer to him/her that you do the same to him/her the next time he/she has an article to submit. This is how pseudo-scientists make sure that the body of pseudo-evidence for their pseudo-treatments is growing at a steady pace.

Researchers from the ‘International Centre for Allied Health Evidence’, University of South Australia in Adelaide wanted to determine whether massage therapy is an effective intervention for back pain. They carried out extensive literature searches to identify all systematic reviews on the subject, analysed them critically and evaluated their methodological quality. Nine systematic reviews were found. Their methodological quality varied from poor to excellent. The primary research informing these systematic reviews was generally considered to be weak quality. The findings indicated that massage may be an effective treatment option when compared to placebo or active treatment options such as relaxation, especially in the short term. There were conflicting and contradictory findings for the effectiveness of massage therapy as a treatment of non-specific low back pain when compared against other manual therapies such as mobilization, standard medical care, and acupuncture.

The authors concluded that there is an emerging body of evidence, albeit small, that supports the effectiveness of massage therapy for the treatment of non-specific low back pain in the short term. Due to common methodological flaws in the primary research, which informed the systematic reviews recommendations arising from this evidence base should be interpreted with caution.

My own systematic review from 1999 (which the authors of this systematic review of systematic reviews seem to have missed) concluded that massage seems to have some potential as a therapy for low back pain. Indeed, there seems to be unanimous agreement that massage therapy is a promising treatment. Why then do massage therapists not finally get their act together and conduct a few more high quality primary studies? Currently, we have about as many reviews as trials! Doing even more reviews will not answer the question about effectiveness!!!

And it is a damn important question. Back pain is extremely common and extremely expensive for us all. At present, we have no optimal treatment. Chiropractors and osteopaths are claiming to have found a good solution, but many experts are not convinced by their evidence and argue that the risks of spinal manipulation might not outweigh its benefits. Massage, by contrast, is almost risk-free. Considering all this, I believe we need more trials with some urgency.

So, why are such trials not forthcoming? I realise that multiple hurdles have to be taken:

  • Clinical studies of that nature are expensive, and there is no obvious funding source.
  • Massage therapists usually do not have enough research expertise to pull off a sound study.
  • There are multiple methodological problems in conduction a definitive massage trial that might convince us all.

However, none of these obstacles are insurmountable. I suggest massage therapists team up with experts who know how to run clinical trials, hammer out a reasonable study design and approach government or other official funders for support. We need a definitive answers and we need them soon: is massage effective? which type of massage? for which patients? at which stage of non-specific low back pain?

Can one design a clinical study in such a way that it looks highly scientific but, at the same time, has zero chances of generating a finding that the investigators do not want? In other words, can one create false positive findings at will and get away with it? I think it is possible; what is more, I believe that, in alternative medicine, this sort of thing happens all the time. Let me show you how it is done; four main points usually suffice:

  1.  The first rule is that it ought to be an RCT, if not, critics will say the result was due to selection bias. Only RCTs have the reputation of being ‘top notch’.
  2.  Once we are clear about this design feature, we need to define the patient population. Here the trick is to select individuals with an illness that cannot be quantified objectively. Depression, stress, fatigue…the choice is vast. The aim must be to employ an outcome measure that is well-accepted, validated etc. but which nevertheless is entirely subjective.
  3.  Now we need to consider the treatment to be “tested” in our study. Obviously we take the one we are fond of and want to “prove”. It helps tremendously, if this intervention has an exotic name and involves some exotic activity; this raises our patients’ expectations which will affect the result. And it is important that the treatment is a pleasant experience; patients must like it. Finally it should involve not just one but several sessions in which the patient can be persuaded that our treatment is the best thing since sliced bread – even if, in fact, it is entirely bogus.
  4.  We also need to make sure that, for our particular therapy, no universally accepted placebo exists which would allow patient-blinding. That would be fairly disastrous. And we certainly do not want to be innovative and create such a placebo either; we just pretend that controlling for placebo-effects is impossible or undesirable. By far the best solution would be to give the control group no treatment at all. Like this, they are bound to be disappointed for missing out a pleasant experience which, in turn, will contribute to unfavourable outcomes in the control group. This little trick will, of course, make the results in the experimental group look even better.

That’s about it! No matter how ineffective our treatment is, there is no conceivable way our study can generate a negative result; we are in the pink!

Now we only need to run the trial and publish the positive results. It might be advisable to recruit several co-authors for the publication – that looks more serious and is not too difficult: people are only too keen to prolong their publication-list. And we might want to publish our study in one of the many CAM-journals that are not too critical, as long as the result is positive.

Once our article is in print, we can legitimately claim that our bogus treatment is evidence-based. With a bit of luck, other research groups will proceed in the same way and soon we will have not just one but several positive studies. If not, we need to do two or three more trials along the same lines. The aim is to eventually do a meta-analysis that yields a convincingly positive verdict on our phony intervention.

You might think that I am exaggerating beyond measure. Perhaps a bit, I admit, but I am not all that far from the truth, believe me. You want proof? What about this one?

Researchers from the Charite in Berlin just published an RCT to investigate the effectiveness of a mindful walking program in patients with high levels of perceived psychological distress.

To prevent allegations of exaggeration, selective reporting, spin etc. I take the liberty of reproducing the abstract of this study unaltered:

Participants aged between 18 and 65 years with moderate to high levels of perceived psychological distress were randomized to 8 sessions of mindful walking in 4 weeks (each 40 minutes walking, 10 minutes mindful walking, 10 minutes discussion) or to no study intervention (waiting group). Primary outcome parameter was the difference to baseline on Cohen’s Perceived Stress Scale (CPSS) after 4 weeks between intervention and control.

Seventy-four participants were randomized in the study; 36 (32 female, 52.3 ± 8.6 years) were allocated to the intervention and 38 (35 female, 49.5 ± 8.8 years) to the control group. Adjusted CPSS differences after 4 weeks were -8.8 [95% CI: -10.8; -6.8] (mean 24.2 [22.2; 26.2]) in the intervention group and -1.0 [-2.9; 0.9] (mean 32.0 [30.1; 33.9]) in the control group, resulting in a highly significant group difference (P < 0.001).

Conclusion. Patients participating in a mindful walking program showed reduced psychological stress symptoms and improved quality of life compared to no study intervention. Further studies should include an active treatment group and a long-term follow-up

This whole thing could just be a bit of innocent fun, but I am afraid it is neither innocent nor fun, it is, in fact, quite serious. If we accept manipulated trials as evidence, we do a disservice to science, medicine and, most importantly, to patients. If the result of a trial is knowable before the study has even started, it is unethical to run the study. If the trial is not a true test but a simple promotional exercise, research degenerates into a farcical pseudo-science. If we abuse our patients’ willingness to participate in research, we jeopardise more serious investigations for the benefit of us all. If we misuse the scarce funds available for research, we will not have the money to conduct much needed investigations. If we tarnish the reputation of clinical research, we hinder progress.

Swiss chiropractors have just published a clinical trial to investigate outcomes of patients with radiculopathy due to cervical disk herniation (CDH). All patients had neck pain and dermatomal arm pain; sensory, motor, or reflex changes corresponding to the involved nerve root and at least one positive orthopaedic test for cervical radiculopathy were included. CDH was confirmed by magnetic resonance imaging. All patients received regular neck manipulations.

Baseline data included two pain numeric rating scales (NRSs), for neck and arm, and the Neck Disability Index (NDI). At two, four and twelve weeks after the initial consultation, patients were contacted by telephone, and the data for NDI, NRSs, and patient’s global impression of change were collected. High-velocity, low-amplitude thrusts were administered by experienced chiropractors. The proportion of patients reporting to feel “better” or “much better” on the patient’s global impression of change scale was calculated. Pre-treatment and post-treatment NRSs and NDIs were analysed.

Fifty patients were included. At two weeks, 55.3% were “improved,” 68.9% at four and 85.7% at twelve weeks. Statistically significant decreases in neck pain, arm pain, and NDI scores were noted at 1 and 3 months compared with baseline scores. 76.2% of all sub-acute/chronic patients were improved at 3 months.

The authors concluded that most patients in this study, including sub-acute/chronic patients, with symptomatic magnetic resonance imaging-confirmed CDH treated with spinal manipulative therapy, reported significant improvement with no adverse events.

In the presence of disc herniation, chiropractic manipulations have been described to cause serious complications. Some experts therefore believe that CDH is a contra-indication for spinal manipulation. The authors of this study imply, however, that it is not – on the contrary, they think it is an effective intervention for CDH.

One does not need to be a sceptic to notice that the basis for this assumption is less than solid. The study had no control group. This means that the observed effect could have been due to:

a placebo response,

the regression towards the mean,

the natural history of the condition,

concomitant treatments,

social desirability,

or other factors which have nothing to do with the chiropractic intervention per se.

And what about the interesting finding that no adverse-effects were noted? Does that mean that the treatment is safe? Sorry, but it most certainly does not! In order to generate reliable results about possibly rare complications, the study would have needed to include not 50 but well over 50 000 patients.

So what does the study really tell us? I have pondered over this question for some time and arrived at the following answer: NOTHING!

Is that a bit harsh? Well, perhaps yes. And I will revise my verdict slightly: the study does tell us something, after all – chiropractors tend to confuse research with the promotion of very doubtful concepts at the expense of their patients. I think, there is a name for this phenomenon: PSEUDO-SCIENCE.

Gravityscan Badge

Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted.

Click here for a comprehensive list of recent comments.