Since many months, I have noticed a proliferation of so-called pilot studies of alternative therapies. A pilot study (also called feasibility study) is defined as a small scale preliminary study conducted in order to evaluate feasibility, time, cost, adverse events, and improve upon the study design prior to performance of a full-scale research project. Here I submit that most of the pilot studies of alternative therapies are, in fact, bogus.
To qualify as a pilot study, an investigation needs to have an aim that is in line with the above-mentioned definition. Another obvious hallmark must be that its conclusions are in line with this aim. We do not need to conduct much research to find that even these two elementary preconditions are not fulfilled by the plethora of pilot studies that are currently being published, and that proper pilot studies of alternative medicine are very rare.
Three recent examples of dodgy pilot studies will have to suffice (but rest assured, there are many, many more).
The aim of this study was to evaluate the effects of foot reflexotherapy on pain and postural balance in elderly individuals with low back pain. And the conclusions drawn by its authors were that this study demonstrated that foot reflexotherapy induced analgesia but did not affect postural balance in elderly individuals with low back pain.
The aim of this study was to investigate the effect of Tai Chi training on dual-tasking performance that involved stepping down and compared it with that of conventional exercise among stroke survivors. And the conclusions read: These results suggest a beneficial effect of Tai Chi training on cognition among stroke survivors without compromising physical task performance in dual-tasking.
The aim of this study was to evaluate the efficacy [of acupuncture] over 12 weeks of treatment and 12 weeks of follow-up. And the conclusion: Acupuncture decreases WC, HC, HbA1c, TG, and TC values and blood pressure in MetS.
It is almost painfully obvious that these studies are not ‘pilot’ studies as defined above.
So, what are they, and why are they so popular in alternative medicine?
The way I see it, they are the result of amateur researchers conducting pseudo-research for publication in lamentable journals in an attempt to promote their pet therapies (I have yet to find such a study that reports a negative finding). The sequence of events that lead to the publication of such pilot studies is usually as follows:
- An enthusiast or a team of enthusiasts of alternative medicine decide that they will do some research.
- They have no or very little know-how in conducting a clinical trial.
- They nevertheless feel that such a study would be nice as it promotes both their careers and their pet therapy.
- They design some sort of a plan and start recruiting patients for their trial.
- At this point they notice that things are not as easy as they had imagined.
- They have too few funds and too little time to do anything properly.
- This does, however, not stop them to continue.
- The trial progresses slowly, and patient numbers remain low.
- After a while the would-be researchers get fed up and decide that their study has enough patients to stop the trial.
- They improvise some statistical analyses with their results.
- They write up the results the best they can.
- They submit it for publication in a 3rd class journal and, in order to get it accepted, they call it a ‘pilot study’.
- They feel that this title is an excuse for even the most obvious flaws in their work.
- The journal’s reviewers and editors are all proponents of alternative medicine who welcome any study that seems to confirm their belief.
- Thus the study does get published despite the fact that it is worthless.
Some might say ‘so what? no harm done!’
But I beg to differ: these studies pollute the medical literature and misguide people who are unable or unwilling to look behind the smoke-screen. Enthusiasts of alternative medicine popularise these bogus trials, while hiding the fact that their results are unreliable. Journalists report about them, and many consumers assume they are being told the truth – after all it was published in a ‘peer-reviewed’ medical journal!
My conclusions are as simple as they are severe:
- Such pilot studies are the result of gross incompetence on many levels (researchers, funders, ethics committees, reviewers, journal editors).
- They can cause considerable harm, because they mislead many people.
- In more than one way, they represent a violation of medical ethics.
- The could be considered scientific misconduct.
- We should think of stopping this increasingly common form of scientific misconduct.
In the current issue of the Faculty of Homeopathy‘s Simile publication, Dr Peter Fisher, the Queen’s homeopath, re-visits the old story of the ‘Smallwood Report’. To my big surprise, I found the following two paragraphs in his editorial:
A prepublication draft [of the Smallwood report] was circulated for comment with prominent warnings that it was confidential and not to be shared more widely (I can personally vouch for this, since I was one of those asked to comment). Regrettably, Prof Ernst did precisely this, leaking it to The Times who used it as the basis of their lead story. The editor of The Lancet, Richard Horton, certainly no friend of homeopathy, promptly denounced Ernst for having “broken every professional code of scientific behaviour”.
Sir Michael Peat, the Prince of Wales’ Principal Private Secretary, wrote to the vice chancellor of Exeter University protesting at the leak, and the university conducted an investigation. Ernst’s position became untenable, funding for his department dried up and he took early retirement. Thirteen years later he remains sore; in his latest book More Harm than Good? he attacks the Prince of Wales as “foolish and immoral”.
END OF QUOTE
Sadly it is true that Horton wrote these defaming words. Subsequently, I asked him to justify them explaining that they were being used by my university against me. He ignored several of my emails, but eventually he sent a reply. In it, he said that, since the university was investigating the issue, the truth would doubtlessly be disclosed. I remember that I was livid at the arrogance and ignorance of this reply. However, being in the middle of my university’s investigation against me, never did anything about it. Looking back at this part of the episode, I feel that Horton behaved abominably.
But back to Dr Fisher.
Why did his defamatory and false accusation in his new editorial come as a ‘big surprise’ to me?
Should I not have gotten used to the often odd way in which some homeopaths handle the truth?
Yes, I did get used to this phenomenon; but I am nevertheless surprised because I have tried to correct Fisher’s ‘error’ before.
This is from a post about Fisher which I published in 2015:
In this article [available here in archive,org – Admin] which he published as Dr. Peter Fisher, Homeopath to Her Majesty, the Queen, he wrote: There is a serious threat to the future of the Royal London Homoeopathic Hospital (RLHH), and we need your help…Lurking behind all this is an orchestrated campaign, including the ’13 doctors letter’, the front page lead in The Times of 23 May 2006, Ernst’s leak of the Smallwood report (also front page lead in The Times, August 2005), and the deeply flawed, but much publicised Lancet meta-analysis of Shang et al…
If you have read my memoir, you will know that even the hostile 13-months investigation my own university did not find me guilty of the ‘leak’. The Times journalist who interviewed me about the Smallwood report already had the document on his desk when we spoke, and I did not disclose any contents of the report to him…
END OF QUOTE
So, assuming that Dr Peter Fisher has seen my 2015 post, he is knowingly perpetuating a slanderous untruth. However, giving him the benefit of the doubt, he might not have read the post nor my memoir and could be unaware of the truth. Error or lie? I am determined to find out and will send him today’s post with an offer to clarify the situation.
I will keep you posted.
Some commentators on this blog and elsewhere keep on claiming that conventional medicine is dangerous, certainly more dangerous than homeopathy (or other alternative therapies). To test the validity of this assumption, I invite you to a little thought experiment:
Imagine 100 patients suffering from each of the conditions listed below.
- coronary heart disease
- peripheral vascular disease
(this list could be extended ad libitum)
Now imagine all of these patients would receive alternative treatment in the form of homeopathy.
Next ask yourself in how many of these patients would hasten death (i. e. contribute to a fatal outcome earlier than necessary).
Here are my estimates (based on the best available evidence):
- cancer: 100
- AIDS: 100
- Ebola: 100
- sepsis: 100
- TB: 100
- MS: 100
- dementia: 100
- coronary heart disease: 100
- stroke: 100
- diabetes: 100
- peripheral vascular disease: 100
(Please don’t tell me that homeopaths do not regularly claim to be able to treat those conditions; and please don’t say that they do not advocate homeopathy as a truly alternative therapy, because they do – if you don’t believe me, do a simple google search yourself.)
And now imagine these patients are being treated by conventional medicine. It seems obvious that not all lives would be saved and that some would die of their condition. But that was not my question. It was, in how many of these patients would conventional medicine hasten death?
Here are my estimates (based on the best available evidence):
- cancer: 0
- AIDS: 0
- Ebola: 0
- sepsis: 0
- TB: 0
- MS: 0
- dementia: o
- coronary heart disease: 0
- stroke: 0
- diabetes: 0
- peripheral vascular disease: 0
I know this is a bit simplistic (as well as provocative). But I was merely trying to make a point: Homeopathy (and many other alternative treatments) are by no means as safe as its proponents seem to think.
In recent days, journalists across the world had a field day (mis)reporting that doctors practising integrative medicine were doing something positive after all. I think that the paper shows nothing of the kind – but please judge for yourself.
The authors of this article wanted to determine differences in antibiotic prescription rates between conventional General Practice (GP) surgeries and GP surgeries employing general practitioners (GPs) additionally trained in integrative medicine (IM) or complementary and alternative medicine (CAM) (referred to as IM GPs) working within National Health Service (NHS) England.
They conducted a retrospective study on antibiotic prescription rates per STAR-PU (Specific Therapeutic group Age–sex weighting Related Prescribing Unit) using NHS Digital data over 2016. Publicly available data were used on prevalence of relevant comorbidities, demographics of patient populations and deprivation scores. setting Primary Care. Participants were 7283 NHS GP surgeries in England. The association between IM GPs and antibiotic prescribing rates per STAR-PU with the number of antibiotic prescriptions (total, and for respiratory tract infection (RTI) and urinary tract infection (UTI) separately) as outcome. results IM GP surgeries (n=9) were comparable to conventional GP surgeries in terms of list sizes, demographics, deprivation scores and comorbidity prevalence.
Statistically significant fewer total antibiotics were prescribed at NHS IM GP surgeries compared with conventional NHS GP surgeries. In contrast, the number of antibiotics prescribed for UTI were similar between both practices.
The authors concluded that NHS England GP surgeries employing GPs additionally trained in IM/CAM have lower antibiotic prescribing rates. Accessibility of IM/CAM within NHS England primary care is limited. Main study limitation is the lack of consultation data. Future research should include the differences in consultation behaviour of patients self-selecting to consult an IM GP or conventional surgery, and its effect on antibiotic prescription. Additional treatment strategies for common primary care infections used by IM GPs should be explored to see if they could be used to assist in the fight against antimicrobial resistance.
The study was flimsy to say the least:
- It was retrospective and is therefore open to no end of confounders.
- There were only 9 surgeries in the IM group.
Moreover, the results were far from impressive. The differences in antibiotic prescribing between the two groups of GP surgeries were minimal or non-existent. Finally, the study was financed via an unrestricted grant of WALA Heilmittel GmbH, Germany (“approx. 900 different remedies conforming to the anthroposophic understanding of man and nature”) and its senior author has a long track record of publishing papers promotional for anthroposophic medicine.
Such pseudo-research seems to be popular in the realm of CAM, and I have commented before on similarly futile projects. The comparison, I sometimes use is that of a Hamburger restaurant:
Employees by a large Hamburger chain set out to study the association between utilization of Hamburger restaurant services and vegetarianism. The authors used a retrospective cohort design. The study population comprised New Hampshire residents aged 18-99 years, who had entered the premises of a Hamburger restaurant within 90 days for a primary purpose of eating. The authors excluded subjects with a diagnosis of cancer. They measured the likelihood of vegetarianism among recipients of services delivered by Hamburger restaurants compared with a control group of individuals not using meat-dispensing facilities. They also compared the cohorts with regard to the money spent in Hamburger restaurants. The adjusted likelihood of being a vegetarian was 55% lower among the experimental group compared to controls. The average money spent per person in Hamburger restaurants were also significantly lower among the Hamburger group.
To me, it is obvious that such analyses must produce a seemingly favourable result for CAM. In the present case, there are several reasons for this:
- GPs who volunteer to be trained in CAM tend to be in favour of ‘natural’ treatments and oppose synthetic drugs such as antibiotics.
- Education in CAM would only re-inforce this notion.
- Similarly, patients electing to consult IM GPs tend to be in favour of ‘natural’ treatments and oppose synthetic drugs such as antibiotics.
- Such patients might be less severely ill that the rest of the patient population (the data from the present study do in fact imply this to be true).
- These phenomena work in concert to generate less antibiotic prescribing in the IM group.
In the final analysis, all this finding amounts to is a self-fulfilling prophecy: grocery shops sell less meat than butchers! You don’t believe me? Perhaps you need to read a previous post then; it concluded that physicians practicing integrative medicine (the 80% who did not respond to the survey were most likely even worse) not only use and promote much quackery, they also tend to endanger public health by their bizarre, irrational and irresponsible attitudes towards vaccination.
What is upsetting with the present paper, in my view, are the facts that:
- a reputable journal published this junk,
- the international press has a field-day reporting this study implying that CAM is a good thing.
The fact is that it shows nothing of the kind. Imagine we send GPs on a course where they are taught to treat all their patients with blood-letting. This too would result in less prescription of antibiotics, wouldn’t it? But would it be a good thing? Of course not!
True, we prescribe too much antibiotics. Nobody doubts that. And nobody doubts that it is a big problem. The solution to this problem is not more CAM, but less antibiotics. To realise the solution we do not need to teach GPs CAM but we need to remind them of the principles of evidence-based practice. And the two are clearly not the same; in fact, they are opposites.
Did you know that chiropractic is a religion?
Well, not quite but almost.
DD Palmer seriously toyed with the idea of turning chiropractic into a religion.
And rightly so!
In the absence of evidence, belief is everything.
And this is why, to this day, so many chiropractors bank (a most appropriate term in this context!) on belief rather than evidence.
Look, for instance at this lovely advertisement I found on Twitter (there are many more, but this one has to stand for the many).
Seven common benefits of chiropractic care!?!
Beneath the picture of a pathologically straight spine – if that is what chiro does to you, avoid it at all cost! – we see the name of the ‘doctor’ who seems to have designed this impressive picture. ‘Dr’ Schluter is even more versatile than his pretty advertisement implies; he also seems to treat newborn babies! And on his website he also tells us that he is able to treat allergies:
You may be surprised to find that chiropractic can do a great deal to alleviate some allergies. Allergies are very common and we become so used to their effect on us that we tend to ignore their symptoms. And many people are unaware of the gradual decline in health that results. Chiropractic treatment didn’t necessarily set out specifically to provide care for allergies, but due to the nature of the chiropractic care and its effect on the nervous system, it has been shown to help.
If we look at some of the common signs of allergies we find that they include some unexpected examples. Not only do we find the usual ones – asthma, sinus congestion, sneezing, itchy eyes, skin rashes and running nose – but also weight gain, Acne and even fluid retention and heartburn.
Many people approach the problem of allergies as though all allergens affect everyone in the same way; this is not the case. Because we are individuals, different allergens affect each of us in differing ways. Some allergens affect some and not others. Consequently treating the condition must be approached on this basis of individuality.
It may not be the pet fur or dried saliva that is kicked up as your pooch washes and scratches; it may also not be the pollen, grass dust or other one of the many irritants in the air at any one time. It could be that due to a misalignment of the spine (or subluxation), mild though it may be, the nervous system is finding it difficult to help the body adapt to its surroundings and is therefore unable to deal with the necessary adjustments.
As an individual you need to treat your body’s physical and nervous system as such. You could be, without even being aware of it, in a generally stressed condition – this may be through lack of sleep, poor nutrition or any one of a combination of the many other stressors affecting us daily. Suddenly you find that with the first spring pollen dust that comes along you begin to wheeze and sneeze!
You may not have previously connected chiropractors and allergies but, for you or someone you know, the connection could help.
Schluter Chiropractic works on the principle of reducing interference so the nervous system and body can work better. Providing that any symptom or condition (including pain) is occurring as a result of nerve interference from vertebral subluxation, there is a very good chance that it will improve with chiropractic care.
END OF QUOTE
Wise words indeed.
Like most chiros, ‘Dr’ Schulter seems to be a true miracle-worker; and because he can do miracles, he does not need to be rational or concerned about evidence or worried about telling the truth.
For Christ sake, it’s Easter!
We ought to show a bit of belief!!!
Because without it, the benefits of chiropractic would be just an illusion.
Chiropractic for kids? Yes, many chiropractors advocate (and earn good money with) it, yet it has been pointed out ad nauseam that the claim of being able of treating paediatric conditions is bogus (in fact, the BCA even lost a famous court case over this issue). But evidence does rarely seem to stop a chiro on a mission!
This website shows us how UK chiropractors plan to educate colleagues in ‘paediatric chiropractic’.
START OF QUOTE
INSPIRAL PAEDIATRIC SEMINARS ……..KIDS DYNAMIC DEVELOPMENT
- Join us for an exciting weekend of learning and skills development, in a supportive, enjoyable environment
- Learn the latest in Chiropractic Paediatrics from two world class leaders whose seminars receive rave reviews & letters of gratitude
- Increase your confidence and certainty in working with families in your community
- Fri 7th September 2-6pm, Sat 8th 9-6pm, Sun 9th 9-1pm I
- Investment £649 Earlybird ends August 15th Late fee £679
- Inclusions: Notes, Lunch on Saturday, onsite parking Park Inn Hotel, Bath Rd, Sipson, Heathrow UB7 0DU
The seminar offers a neurological approach to healthy development in babies & children. It provides clinically relevant assessment, adjusting & clinical decision making. The focus will be on a Chiropractic wellness paradigm with a collaborative approach to promote healthy outcomes across the infant to adolescent years.
This is a hands-on program with a focus on neuro – developmentally appropriate adjusting of the spine and cranial dural system for health. We address some of the leading challenges with infant health and development, and teach exciting home care plans to facilitate optimum development.
Genevieve & Rosemary Keating are leaders in Chiropractic paediatric health, learning & development.
Both are experienced Chiropractors, Facilitators, Diplomates of the American Chiropractic Neurology Board and Master Practitioners of Neuro Linguistics.
Rosemary holds a Masters in Chiropractic Paediatrics, and Genevieve is completing her PhD in Early Childhood Development.
END OF QUOTE
The event is hosted and organised by the ‘United Chiropractic Association UK’ (UCA), an organisation with a mission to ensure the public has access to vitalistic chiropractic care, which claims that chiropractors provide care that is safe. Because the techniques used by chiropractors are acquired over years of study and experience, chiropractors have an enviable safety record. In fact, in the words of a classic New Zealand study, chiropractic care is “remarkably safe.” Chiropractors use the latest methods. After years of study, licensing examinations and continuing education seminars, chiropractors in the United Kingdom are at the top of their game, using proven techniques and natural methods to help you get well and stay well.
The UCA is firmly rooted in the gospel of the founding fathers (D D Palmer, B J Palmer etc.): Chiropractic is concerned with the preservation and restoration of health, and focuses particular attention on the subluxation. A subluxation is a complex of functional and/or structural and/or pathological articular changes that compromise neural integrity and may influence organ system function and general health. A subluxation is evaluated, diagnosed, and managed through the use of chiropractic procedures based on the best available rational and empirical evidence. Subluxation is a fundamental axiom of the Chiropractic profession. The World Federation of Chiropractors (WFC) policy statement reaffirms the use of the term vertebral subluxation and it is defined succinctly and accepted by the World Health Organisation (WHO).
Thus, the UCA seems to subscribe to both principles of the Palmers. The first is subluxation and the second is profit.
Now, now, now – I must not be so sarcastic.
Try something constructive, Edzard!
You are absolutely correct, Edzard.
Here it is, my constructive contribution to this event:
I herewith offer the UCA to give two lectures during their course; one about the importance of critical thinking in healthcare, and one reviewing the evidence for and against chiropractic for paediatric conditions.
Chiropractors are fast giving up the vitalistic and obsolete concepts of their founding fathers, we are told over and over again. But are these affirmations true? There are good reasons to be sceptical. Take this recent paper, for instance.
The objective of this survey was to investigate the proportion of Australian chiropractic students who hold non-evidence-based beliefs in the first year of study and to determine the extent to which they may be involved in non-musculoskeletal health conditions.
Students from two Australian chiropractic programs were invited to answer a questionnaire on how often they would give advice on 5 common health conditions in their future practices, as well as to provide their opinion on whether chiropractic spinal adjustments could prevent or help seven health-related conditions.
The response rate of this survey was 53%. Students were highly likely to offer advice on a range of non-musculoskeletal conditions. The proportions were lowest in first year and highest the final year. For instance, 64% of students in year 4/5 believed that spinal adjustments improve the health of infants. Also, high numbers of students held non-evidence-based beliefs about ‘chiropractic spinal adjustments’ which tended to occur in gradually decreasing in numbers in sequential years, except for 5th and final year, when a reversal of the pattern occurred.
The authors concluded that new strategies are required for chiropractic educators if they are to produce graduates who understand and deliver evidence-based health care and able to be part of the mainstream health care system.
This is an interesting survey, but I think its conclusion is wrong!
- Educators do not require ‘new strategies’, I would argue; they simply need to take their duty of educating students seriously – educating in this context does not mean brain-washing, it means teaching facts and evidence-based practice. And this is were any concept of true education would run into problems: it would teach students that chiropractic is built on sand.
- Conclusions need to be based on the data presented. Therefore, the most fitting conclusion, in my view, is that chiropractic students are currently being educated such that, once let loose on the unsuspecting and often all too gullible public, they will be a menace and a serious danger to public health.
You might say that this survey is from Australia and that the findings therefore do not necessarily apply to other countries. Correct! However, I very much fear that elsewhere the situation is similar or perhaps even worse. And my fear does not come out of thin air, it is based on things we have discussed before; see for instance these three posts:
But I would be more than willing to change my mind – provided someone can show me good evidence to the contrary.
An article in yesterday’ Times makes the surprising claim that ‘doctors turn to herbal cures when the drugs don’t work’. As the subject is undoubtedly relevant to this blog and as the Times is a highly respected newspaper, I think this might be important and will therefore comment (in normal print) on the full text of the article (in bold print):
GPs are increasingly dissatisfied with doling out pills that do not work for illnesses with social and emotional roots, and a surprising number of them end up turning to alternative medicine.
What a sentence! I would have thought that GPs have always been ‘dissatisfied’ with treatments that are ineffective. But who says they turn to alternative medicine in ‘surprising numbers’ (our own survey does not confirm the notion)? And what is a ‘surprising number’ anyway (zero would be surprising, in my view)?
Charlotte Mendes da Costa is unusual in being both an NHS GP and a registered homeopath. Her frustration with the conventional approach of matching a medicine to a symptom is growing as doctors increasingly see the limits, and the risks, of such a tactic.
Do we get the impression that THE TIMES does not know that homeopathy is not herbal medicine? Do they know that ‘matching a medicine to a symptom’ is what homeopaths believe they are doing? Real doctors try to find the cause of a symptom and, whenever possible, treat it.
She asks patients with sore throats questions that few other GPs pose: “What side is it? Is it easier to swallow solids or liquids? What time of day is it worst?” Dr Mendes da Costa is trying to find out which homeopathic remedy to prescribe. But when NHS guidance for sore throats aims mainly to convince patients that they will get better on their own, her questions are just as important as her prescription.
This section makes no sense. Sore throats do get better on their own, that’s a fact. And empathy is not a monopoly of homeopaths. But Dr Mendes Da Costa might be somewhat detached from reality; she once promoted the nonsensical notion that “up to the end of 2010, 156 randomised controlled trials (RCTs) in homeopathy had been carried out with 41% reporting positive effects, whereas only 7% have been negative. The remainder were non-conclusive.” (see more on this particular issue here)
“It’s very difficult to disentangle the effect of listening to someone properly, in a non-judgmental way, and taking a real rather than a superficial interest,” she says. “With a sore throat [I was trained] really only to be interested in, ‘Do they need antibiotics or not?’ ”
In this case, she should ask her money back; her medical school seems to have been rubbish in training her adequately.
This week a Lancet series on back pain said that millions of patients were getting treatments that did them no good. A government review is looking into how one in 11 people has come to be on potentially addictive drugs such as tranquillisers, opioid painkillers and antidepressants.
Yes, and how is that an argument for homeopathy? It isn’t! It seems to come from the textbook of fallacies.
And this week a BMJ Open study found that GPs with alternative training prescribed a fifth fewer antibiotics.
That study was akin to showing that butchers sell less vegetables than green-grocers. It provided no argument at all for implying that homeopathy is a valuable therapy.
Doctors seem receptive to alternative approaches: in a poll on its website 70 per cent agreed that doctors should recommend acupuncture to patients in pain. The Faculty of Homeopathy now counts 400 doctors among its 700 healthcare professional members.
Wow! Does the Times journalist know that the ‘Faculty of Homeopathy’ is primarily an organisation for doctor homeopaths? If so, why are these figures anything to write home about? And does the author appreciate that the pole was open not just to doctors but to to anyone (particularly those who were motivated, like acupuncturists)?
This horrifies many academics, who say that there is almost no evidence that complementary therapies work.
It horrifies nobody, I’d say. It puzzles some people, and not just academics. And their claim of a lack of sound evidence is evidence-based.
“It’s a false battle”, says Michael Dixon, a GP who chairs the College of Medicine, which is trying to broaden the focus on treatment to patients’ whole lives. “GPs are practical. If a patient gets better that’s all that matters.”
Dr Dixon says there are enormous areas of illness ranging from chronic pain to irritable bowels where few conventional treatments have been shown to be particularly effective, so why not try alternatives with fewer side effects?
Unable to diagnose and treat adequately, let’s all do the next worst thing and apply some outright quackery?!? Logic does not seem to be Dixon’s strong point, does it?
He recommends herbal remedies such as pelargonium — “like a geranium, quite a pretty little flower” — acupressure, and techniques such as self-hypnosis. To those who say these are placebos he replies: so what?
So what indeed! There are over 200 species of pelargonium; only 2 or 3 of them are used in herbal medicine. I don’t suppose Dr Dixon wants to poison us?
“Aromatherapy does work, but only if you believe in it, that’s the way you have to look at it, like a mother kissing knees better.” He continues: “We are healers. That’s what we do as doctors. You can call it theatrical or you can call it a relationship. A lot of patients come in with a metaphor — a headache is actually unhappiness — and the treatment is symbolic.”
It frightens me to know that there are doctors out there who think like this!
What if a patient is seriously ill?
A cancer is a metaphor for what exactly?
As doctors, we have the ethical duty to apply BOTH the science and the art of medicine, BOTH efficacious, evidence-based therapies AND compassion. Can I be so bold as to recommend our book about the ethics of alternative medicine to Dixon?
Such talk makes conventional doctors very nervous. Yet acupuncture illustrates their dilemma. It used to be recommended by the NHS for back pain because patients did improve. Now it is not, after further evidence suggested that patients given placebo “sham acupuncture” did just as well.
No, acupuncture used to be recommended by NICE because there was some evidence; when subsequently more rigorous trials emerged showing that it does NOT work, NICE stopped recommending it. Real medicine develops – it’s only alternative medicine and its proponents that seem to be stuck in the past and resist progress.
Martin Underwood, of the University of Warwick, asks: “So are you going to say, ‘Well, patients get better than they would do otherwise’? Or say it’s all theatrical placebo because it shows no benefit over sham treatment? That’s the question for society.”
Society has long answered it! The answer is called evidence-based medicine. We are not content using quackery for its placebo response; we know that effective treatments do that too, and we want to make progress and improve healthcare of tomorrow.
Although many doctors agree that they need to look at patients more broadly, they insist they do not need to turn to unproven treatments. The magic ingredient, they say, is not an alternative remedy, but time. Helen Stokes-Lampard, chairwoman of the Royal College of GPs, said: “Practices which offer alternative therapies tend to spend longer with patients . . . allowing for more in-depth conversations.”
I am sorry, if this post turned into a bit of a lengthy rant. But it was needed, I think: if there ever was a poorly written, ill focussed, badly researched and badly argued article on alternative medicine, it must be this one.
Did I call the Times a highly respected paper?
I take it back.
As I often said, I find it regrettable that sceptics often say THERE IS NOT A SINGLE STUDY THAT SHOWS HOMEOPATHY TO BE EFFECTIVE (or something to that extent). This is quite simply not true, and it gives homeopathy-fans the occasion to suggest sceptics wrong. The truth is that THE TOTALITY OF THE MOST RELIABLE EVIDENCE FAILS TO SUGGEST THAT HIGHLY DILUTED HOMEOPATHIC REMEDIES ARE EFFECTIVE BEYOND PLACEBO. As a message for consumers, this is a little more complex, but I believe that it’s worth being well-informed and truthful.
And that also means admitting that a few apparently rigorous trials of homeopathy exist and some of them show positive results. Today, I want to focus on this small set of studies.
How can a rigorous trial of a highly diluted homeopathic remedy yield a positive result? As far as I can see, there are several possibilities:
- Homeopathy does work after all, and we have not fully understood the laws of physics, chemistry etc. Homeopaths favour this option, of course, but I find it extremely unlikely, and most rational thinkers would discard this possibility outright. It is not that we don’t quite understand homeopathy’s mechanism; the fact is that we understand that there cannot be a mechanism that is in line with the laws of nature.
- The trial in question is the victim of some undetected error.
- The result has come about by chance. Of 100 trials, 5 would produce a positive result at the 5% probability level purely by chance.
- The researchers have cheated.
When we critically assess any given trial, we attempt, in a way, to determine which of the 4 solutions apply. But unfortunately we always have to contend with what the authors of the trial tell us. Publications never provide all the details we need for this purpose, and we are often left speculating which of the explanations might apply. Whatever it is, we assume the result is false-positive.
Naturally, this assumption is hard to accept for homeopaths; they merely conclude that we are biased against homeopathy and conclude that, however, rigorous a study of homeopathy is, sceptics will not accept its result, if it turns out to be positive.
But there might be a way to settle the argument and get some more objective verdict, I think. We only need to remind ourselves of a crucially important principle in all science: INDEPENDENT REPLICATION. To be convincing, a scientific paper needs to provide evidence that the results are reproducible. In medicine, it unquestionably is wise to accept a new finding only after it has been confirmed by other, independent researchers. Only if we have at least one (better several) independent replications, can we be reasonably sure that the result in question is true and not false-positive due to bias, chance, error or fraud.
And this is, I believe, the extremely odd phenomenon about the ‘positive’ and apparently rigorous studies of homeopathic remedies. Let’s look at the recent meta-analysis of Mathie et al. The authors found several studies that were both positive and fairly rigorous. These trials differ in many respects (e. g. remedies used, conditions treated) but they have, as far as I can see, one important feature in common: THEY HAVE NOT BEEN INDEPENDENTLY REPLICATED.
If that is not astounding, I don’t know what is!
Think of it: faced with a finding that flies in the face of science and would, if true, revolutionise much of medicine, scientists should jump with excitement. Yet, in reality, nobody seems to take the trouble to check whether it is the truth or an error.
To explain this absurdity more fully, let’s take just one of these trials as an example, one related to a common and serious condition: COPD
The study is by Prof Frass and was published in 2005 – surely long enough ago for plenty of independent replications to emerge. Its results showed that potentized (C30) potassium dichromate decreases the amount of tracheal secretions was reduced, extubation could be performed significantly earlier, and the length of stay was significantly shorter. This is a scientific as well as clinical sensation, if there ever was one!
The RCT was published in one of the leading journals on this subject (Chest) which is read by most specialists in the field, and it was at the time widely reported. Even today, there is hardly an interview with Prof Frass in which he does not boast about this trial with truly sensational results (only last week, I saw one). If Frass is correct, his findings would revolutionise the lives of thousands of seriously suffering patients at the very brink of death. In other words, it is inconceivable that Frass’ result has not been replicated!
But it hasn’t; at least there is nothing in Medline.
Why not? A risk-free, cheap, universally available and easy to administer treatment for such a severe, life-threatening condition would normally be picked up instantly. There should not be one, but dozens of independent replications by now. There should be several RCTs testing Frass’ therapy and at least one systematic review of these studies telling us clearly what is what.
But instead there is a deafening silence.
For heaven sakes, why?
The only logical explanation is that many centres around the world did try Frass’ therapy. Most likely they found it does not work and soon dismissed it. Others might even have gone to the trouble of conducting a formal study of Frass’ ‘sensational’ therapy and found it to be ineffective. Subsequently they felt too silly to submit it for publication – who would not laugh at them, if they said they trailed a remedy that was diluted 1: 1000000000000000000000000000000000000000000000000000000000000 and found it to be worthless? Others might have written up their study and submitted it for publication, but got rejected by all reputable journals in the field because the editors felt that comparing one placebo to another placebo is not real science.
And this is roughly, how it went with the other ‘positive’ and seemingly rigorous studies of homeopathy as well, I suspect.
Regardless of whether I am correct or not, the fact is that there are no independent replications (if readers know any, please let me know).
Once a sufficiently long period of time has lapsed and no replications of a ‘sensational’ finding did not emerge, the finding becomes unbelievable or bogus – no rational thinker can possibly believe such a results (I for one have not yet met an intensive care specialist who believes Frass’ findings, for instance). Subsequently, it is quietly dropped into the waste-basket of science where it no longer obstructs progress.
The absence of independent replications is therefore a most useful mechanism by which science rids itself of falsehoods.
It seems that homeopathy is such a falsehood.
The plethora of dodgy meta-analyses in alternative medicine has been the subject of a recent post – so this one is a mere update of a regular lament.
This new meta-analysis was to evaluate evidence for the effectiveness of acupuncture in the treatment of lumbar disc herniation (LDH). (Call me pedantic, but I prefer meta-analyses that evaluate the evidence FOR AND AGAINST a therapy.) Electronic databases were searched to identify RCTs of acupuncture for LDH, and 30 RCTs involving 3503 participants were included; 29 were published in Chinese and one in English, and all trialists were Chinese.
The results showed that acupuncture had a higher total effective rate than lumbar traction, ibuprofen, diclofenac sodium and meloxicam. Acupuncture was also superior to lumbar traction and diclofenac sodium in terms of pain measured with visual analogue scales (VAS). The total effective rate in 5 trials was greater for acupuncture than for mannitol plus dexamethasone and mecobalamin, ibuprofen plus fugui gutong capsule, loxoprofen, mannitol plus dexamethasone and huoxue zhitong decoction, respectively. Two trials showed a superior effect of acupuncture in VAS scores compared with ibuprofen or mannitol plus dexamethasone, respectively.
The authors from the College of Traditional Chinese Medicine, Jinan University, Guangzhou, Guangdong, China, concluded that acupuncture showed a more favourable effect in the treatment of LDH than lumbar traction, ibuprofen, diclofenac sodium, meloxicam, mannitol plus dexamethasone and mecobalamin, fugui gutong capsule plus ibuprofen, mannitol plus dexamethasone, loxoprofen and huoxue zhitong decoction. However, further rigorously designed, large-scale RCTs are needed to confirm these findings.
Why do I call this meta-analysis ‘dodgy’? I have several reasons, 10 to be exact:
- There is no plausible mechanism by which acupuncture might cure LDH.
- The types of acupuncture used in these trials was far from uniform and included manual acupuncture (MA) in 13 studies, electro-acupuncture (EA) in 10 studies, and warm needle acupuncture (WNA) in 7 studies. Arguably, these are different interventions that cannot be lumped together.
- The trials were mostly of very poor quality, as depicted in the table above. For instance, 18 studies failed to mention the methods used for randomisation. I have previously shown that some Chinese studies use the terms ‘randomisation’ and ‘RCT’ even in the absence of a control group.
- None of the trials made any attempt to control for placebo effects.
- None of the trials were conducted against sham acupuncture.
- Only 10 studies 10 trials reported dropouts or withdrawals.
- Only two trials reported adverse reactions.
- None of these shortcomings were critically discussed in the paper.
- Despite their affiliation, the authors state that they have no conflicts of interest.
- All trials were conducted in China, and, on this blog, we have discussed repeatedly that acupuncture trials from China never report negative results.
And why do I find the journal ‘dodgy’?
Because any journal that publishes such a paper is likely to be sub-standard. In the case of ‘Acupuncture in Medicine’, the official journal of the British Medical Acupuncture Society, I see such appalling articles published far too frequently to believe that the present paper is just a regrettable, one-off mistake. What makes this issue particularly embarrassing is, of course, the fact that the journal belongs to the BMJ group.
… but we never really thought that science publishing was about anything other than money, did we?