This is a post that I wanted to write for a while (I had done something similar on acupuncture moths ago); but I had to wait, and wait, and wait…until finally there were the awaited 100 Medline listed articles on homeopathy with a publication date of 2016. It took until the beginning of August to reach the 100 mark. To put this into perspective with other areas of alternative medicine, let me give you the figures for 3 other therapies:
- there are currently 1 413 articles from 2016 on herbal medicine;
- 875 on acupuncture;
- and 256 on chiropractic.
And to give you a flavour of the research activity in some areas of conventional medicine:
- there are currently almost 100 000 articles from 2016 on surgery;
- 1 410 on statins;
- and 33 033 on psychotherapy.
This suggests quite strongly, I think, that the research activity in homeopathy is relatively low (to put it mildly).
So, what do the first 100 Medline articles on homeopathy cover? Here are some of the findings of my mini-survey:
- there were 4 RCTs;
- 3 systematic reviews;
- 8 papers on observational-type data (case series, observational studies etc.);
- 9 animal studies;
- 14 other pre-clinical or basic research studies;
- 1 pilot study;
- 14 investigations of the quality of homeopathic preparations;
- 15 surveys;
- 2 investigations into the adverse effects of homeopathic treatments;
- 49 other papers (e. g. comments, opinion pieces, letters, perspective articles, editorials).
I should mention that, because I assessed 100 papers, the above numbers can be read both as absolute as well as percentage figures.
How should we interpret my findings?
As with my previous evaluation, I must caution not to draw generalizable conclusions from them. What follows should therefore be taken with a pinch of salt (or two):
- The research activity into homeopathy is currently very subdued.
- Arguably the main research question of efficacy does not seem to concern researchers of homeopathy all that much.
- There is an almost irritating abundance of papers that are data-free and thrive on opinion (my category of ‘other papers’).
- Given all this, I find it hard to imagine that this area of investigation is going to generate much relevant new knowledge or clinical progress.
A survey published in 2011 showed that one-third of Danish hospitals offered alternative therapies. In total, 38 hospitals offered acupuncture and one Eye Movement Desensitization and Reprocessing Light Therapy. The most commonly reported reason for offering CAM was “scientific evidence”.
Many readers of this blog might be amazed with both the high level of alternative medicine presence in Danish hospitals and the notion that this was due to ‘scientific evidence’. A recent article provides even more surprises about the Danish alternative medicine scene.
It revealed that 8 out of 10 Danes are interested in using some form of alternative medicine…Some 67 percent of Danes say the national healthcare system should be more open to alternative healing practices, such as homeopathy, acupuncture or chiropractic, and 60 percent would like to see these treatments covered by the public health insurance system. More than half of the 6,000 respondents believe alternative therapies can be just as effective as traditional medicine.
Charlotte Yde, the chairwoman at Sundhedsrådet, which is the umbrella organisation for alternative practitioners in Denmark, contends many Danes feel frustrated because they cannot freely discuss alternative treatment with their doctors. Alternative treatment researcher Helle Johannessen agrees that Danish doctors should openly discuss alternative medicine options with patients. “In other European countries doctors use alternative treatment to a much greater extent than doctors in Denmark,” Johannessen told DR. “[International experience] shows that some forms of alternative therapy can improve quality of life and reduce anxiety and nausea in cancer patients.”
This, it seems to me, is little more than a bonanza of fallacious thinking and misleading information.
- The notion that popularity of a therapy has anything to do with its usefulness is a classical fallacy.
- The notion that belief determines efficacy (More than half of the 6,000 respondents believe alternative therapies can be just as effective as traditional medicine.) or vice versa is complete nonsense.
- The notion that many Danes … cannot freely discuss alternative treatment with their doctors is misleading: patients can discuss what they feel like with whom they feel like.
- The notion that in other European countries doctors use alternative treatment to a much greater extent than doctors in Denmark is also misleading: there are many European countries where LESS alternative therapies are being paid for via the public purse.
- Finally, the notion that that some forms of alternative therapy can improve quality of life and reduce anxiety and nausea in cancer patients – even if it were correct – does not mean that ALL alternative therapies are efficacious, safe, or cost-effective.
Who cares about Denmark?
Why should this be important?
Well, the Danes might care, and it is important because it provides an excellent example of how promoters of bogus treatments tend to argue – not just in Denmark, but everywhere. Unfortunately, politicians all too often fall for such fallacious notions. For them, a popular issue is a potential vote-winner. Within medical systems that are notoriously strapped for money, the looser will inevitably be optimal healthcare.
On this blog, I have discussed the lamentable quality of TCM products before (e. g. here, here and here). In a nutshell, far too high percentages of them are contaminated with toxic substances or adulterated with prescription drugs. It is no question: these deficits put many consumers at risk. Equally, there is no question that the problem has been known for decades.
For the Chinese exporters, such issues are a great embarrassment, not least because TCM-products are amongst the most profitable of all the Chinese exports. In the past, Chinese officials have tried to ignore or suppress the subject as much as possible. I presume they fear that their profits might be endangered by being open about the dubious quality of their TCM-exports.
Recently, however, I came across a website where unusually frank and honest statements of Chinese officials appeared about TCM-products. Here is the quote:
China is to unroll the fourth national survey of traditional Chinese medicine (TCM) resources to ensure a better development of the industry, said a senior health official…
With the public need for TCM therapies growing, the number of medicine resources has decreased and people have turned to the cultivated ones. However, due to a lack of standards, the cultivated TCM resources are sometimes less effective or even unsafe for human use, said Wang Guoqiang, director of the State Administration of TCM, at a TCM seminar held in Kunming, Yunnan Province in southwest China.
There is a pressing need to protect TCM resources, Wang said. “I’ve heard people saying that medicine quality will spell doom for the TCM industry, which I must admit, is no exaggeration,” he said.
The survey has been piloted in 922 counties in 31 provinces in China since 2011. According to its official website, it will draw a clear picture of the variety, distribution, storage and growth trends of TCM resources, including herbs, animals, minerals and synthetic materials.
TCM includes a range of traditional medical practices originating in China. It includes such treatments as herbal medicine, acupuncture, massage (tuina), exercise (qigong) and dietary therapy.
Although well accepted in the mainstream of medical care throughout East Asia, TCM is considered an alternative medical system in much of the western world and has been a source of controversy. A milestone in the recognition of TCM came when Chinese pharmaceutical chemist Tu Youyou won a Nobel Prize in 2015 for her discovery of Artemisinin, a medicinal herb, to help treat malaria.
END OF QUOTE
Surely, these are remarkable, perhaps even unprecedented statements by Chinese officials:
…cultivated TCM resources are sometimes less effective or even unsafe for human use…
…medicine quality will spell doom for the TCM industry…
Let’s hope that, after such words, there will be appropriate actions… finally.
Nobody can doubt that, during the last 200 years, conventional medicine has made monumental progress. Homeopathy, however, has remained more or less like Hahnemann invented it. But now it seems as though homeopathy can celebrate an unprecedented step ahead. As so often in medicine, it originates from a commercial enterprise.
Genexa is a US firm that produces natural health products. On their website, they state that “At Genexa, we believe medicine should be free from unhealthy fillers and toxins”. They recently published a press-release introducing a line of homeopathic medicines certified organic by the U.S. Department of Agriculture and Non-GMO Project verified. They are keen to point out that these products “do not contain any genetically modified ingredients.” In fact, several of their remedies do not contain any active ingredients to speak of: they are homeopathic!
“We are extremely proud of our organic and non-GMO certifications – the seals are prominently featured on all our products and website for easy label reading and patient education,” stated David Johnson, CEO of Genexa, in their press-release. “Our quality standards are among the highest in the over-the-counter medicine industry.”
Genexa’s 11 homeopathic formulations are being advertised for the treatment of common health issues such as flu, cold, allergies, stress, pain, leg cramps, sleeplessness and jet lag. An entire line of products is, according to the press-release, specially formulated for children and includes treatments for cold, allergy and calming.
Genexa’s CMO proudly announced that “It’s important to us that our retail customers feel confident in the products and know they can trust they are purchasing medicines free from unhealthy fillers and toxins and simply focus on healing.” Presumably that trust must include the trust into the efficacy of the homeopathic remedies! Yes, I am pleased to report that, apparently it does; elsewhere they confirm this by stating that “Genexa holds itself to the highest standards in both quality and ethics.” The highest standards of ethics surely include that the remedies in question are demonstrably efficacious.
But how can we be sure? Are any of these homeopathic remedies supported by reasonably strong evidence? Oddly enough, despite all these affirmations, I did get my doubts when I tried to dig a bit deeper.
Take the homeopathic remedy called SLEEPOLOGY, for instance. The website informs us that “This homeopathic formulation consists of nine leading remedies designed to treat sleeplessness, inability to fall asleep, frequent waking, restless sleep and sleeplessness from stress, exhaustion, nervousness, excitability, restlessness, worries, irritability, and pain.” So, it’s a complex homeopathic remedy with 9 different ingredients. But is there any evidence of efficacy for this mixture? I am not aware of any clinical trials that have tested its efficacy. But I must be wrong, because on the website we are being told that “Clinical trials have demonstrated efficacy for treating sleeplessness for piper methysticum, and valeriana officinalis.” That may be so, but the trials were done with herbal extracts, not with homeopathic potencies! Could the statement therefore be more than a little misleading?
On the internet, I found all sorts of fascinating bits about the new homeopathic lines (my compliments to the PR firm that organised the launch!); for instance the revelation that: “The company’s proprietary medicines were created by and are regularly reviewed and enhanced by its chief medical officer, Dr. Todd Rowe*, a nationally respected physician with an expertise in homeopathic medicine formulation. Working with the Genexa team, Dr. Rowe and his team of chemists and pharmacists spent hundreds of hours meticulously formulating and testing the products. The result is a line of effective, potent medicines that are certified organic by the USDA and non-GMO verified by the Non-GMO Project. “Our formulations are based on tried and true principles for miasmatic and energetic balance, so that the remedies potentiate each other and promote the most positive patient outcomes,” said Dr. Rowe. “These powerful medicines work with your body to help it heal itself.”” However, I was unable to find out which potencies are being used for the Genexa homeopathic products. This information might not be that relevant: according to the homeopathic ‘like cures like’ principle, the effects of a substance are reversed through potentiation. This is why coffee, for instance, is potentised by homeopath to generate a sleeping remedy. Does it not follow then that, potentising two or more herbal ingredients that have hypnotic effects (as in SLEEPOLOGY), must generate a remedy for preventing sleep? A similarly puzzling lack of ‘homeopathic logic’ seems to apply to several other products in Genexa’s line of homeopathic remedies.
I have to admit, I am confused.
Could it be that the ‘breakthrough’ turns out to be a breakdown of ‘homeopathic logic’?
Let’s hope someone from Genexa reads these lines and can enlighten us.[*he is the President of the American Medical College of Homeopathy]
I am editor in chief of a journal called FACT. It has a large editorial board, and I am always on the look-out for people who might be a good, productive and colourful addition to it. On 3 June, I sent an invitation to Mel Koppelman, who is by now well known to regular readers of this blog. Here is a copy:
can I ask you a question?
would you consider joining the ed-board of FACT [as you mentioned it in one of your comments, I assume you know this journal – but you are wrong in implying that it has anything to do with the pharmaceutical or any other industry]? if you agree, we would expect you to write 2-3 ‘summaries/commentaries’ per year. in return you get a free subscription and, of course, can submit other articles.
no, this is not a joke or a set-up. I like to have the full spectrum of opinion/expertise on my ed-board, and I do think you understand science quite well. our opinions differ but that’s what I think is good for the journal.
think about it – please.
On 6 June, she replied as follows:
Great to hear from you, I hope you enjoyed your weekend.
Thank you very much for the kind offer, it’s something I would consider. I certainly have no problem with, and in fact embrace, people who have different opinions and views from my own, so long as I feel that they have integrity in their approach.
Just a few questions / comments:
1) Regarding FACT’s affiliations, what I said in my comment was that it was a publication of the Royal Pharmaceutical Society. According to Wiley’s website, Focus on Alternative and Complementary Medicine is copyright by Royal Pharmaceutical Society. It’s also listed on the Pharmaceutical Press website.
Are you telling me that’s incorrect? That I’m “wrong” in saying there’s a relationship? I obviously need to understand the nature of the publication whose editorial board I’m considering joining. Very confusing that you as editor say there’s no relationship to the RPS and yet they claim copyright over your publication. Incidentally, is FACT self-sufficient, earning all of its income from subscriptions? Or does any financial support come from the publishers?
2) As enticing as a free subscription to FACT is, I have access to more high quality peer-reviewed reading material than I could enjoy in many lifetimes. Because my skills seem to be in high demand and because I already spend 10-20 hours per week doing unpaid volunteer work, any additional projects that I take on at this time would need to be financially compensated. I understand that this may be a deal-breaker.
3) While I have no issue with you having different views when it comes to medical research, in order to choose to work with your publication, it’s important to me that it’s run by people with a high level of academic integrity and put patient welfare at the forefront of it’s agenda.
In March, you came out in public support of the NICE draft guidelines. You were quoted in the Guardian as saying: “It is good to see that Nice have now caught up with the evidence. Neither spinal manipulation nor acupuncture are supported by good science when it comes to treating low back pain.”
Following this, it was brought to your attention that the recommendations were contrary to best evidence and that the conclusions were unsupportable. While you have the option of following this up to make sure that the record reflects best evidence, you have indicated that you have no interest in evaluating the situation and possibly admitting an error. This behaviour is concerning from the perspective of academic integrity, particularly when it directly leads to increased human suffering (policy in several countries has already been changed based on the draft), and I would be worried that by joining your board I could be associated with such unethical behaviour.
Perhaps if I understood better your position, which seems to be to ignore the situation, not follow up on the concerns raised, and leave your comments uncorrected even though they may be inaccurate and backing guidelines that cause harm to patients, that might allay certain reservations.
Anyways, these are my initial thoughts. I hope you have an opportunity to enjoy the beautiful weather, and I look forward to hearing from you.
At that stage, I began to fear that I had made a mistake. But, giving her the benefit of the doubt, I swallowed my pride and replied as politely as I could to her concerns which, in my view, were odd, to say the least. This is what I wrote on 7 June:
Thank you for your reply to my invitation. Let me address your points in turn:
- I said that FACT has nothing to do with the pharma industry which is true [when you state that “you as editor say there’s no relationship to the RPS” – it suggests to me that you did not read my email properly]. In their own words, the RPS is “the professional membership body for pharmacists and pharmacy in Great Britain and an internationally renowned publisher of medicines information.” [http://www.rpharms.com/home/about-us.asp] They have a similar status as the Royal Colleges. In the 2 decades that I am running the journal, there has been not a single instance of interference of any kind. We use them simply as an excellent publishing house. And yes, FACT is to the best of my knowledge self-sufficient and survives without funds from 3rd parties.
- I am delighted to hear that your skills are in demand.
- I have stated my position regarding the draft NICE guideline ad nauseam: I prefer to wait until I see their next version of the draft before I make further comments on it. In my view, this is both reasonable and honourable. If you disagree, I can do little about it other than expressing my sincere regrets.
I hope these brief clarifications are helpful for you to arrive at a decision.
On 8 June, I received Mel’s reply:
How interesting! Is he trying to ‘keep his enemies closer’ or am I being too skeptical? He has recognised your talents and dedication and intellect so he is not altogether stupid after all! I eagerly await his response to your reply.
Those who have studied with Ernst say that he’s a genuine chap and misunderstood – which I know is almost unimaginable given his behaviour – but we always have to allow for the possibility that we have misjudged people however remote! Also, people can turn – especially when they get older and near retirement. Alternatively he may just fancy you!…
Mel Koppelman Really enjoying hearing y’all’s thoughts on this. I just want to say that If I had thought that the chances of me being able to create positive change by joining FACT were high, I would have tempered the tone of my reply. But the simple fact that EE can’t even be factual or forthright about whose journal it is suggests an irreparable break with reality. And surely there’s an issue (academic? ethical? legal?) with recruiting someone to your board and denying an industry tie when there is one? Not to mention that if the RSP does fund his journal, he’s been lying about his conflicts of interest. Is that someone I want to spend time adding value to that could be spent with family, patients, time in nature or really making positive change by supporting the ANF? I’ll be interested to read his reply if there is one and especially how he responds regarding the relationship between FACT and RSP. Will keep y’all posted.
John McDonald Healthy skepticism! Healthy journal! And it’s 99% fact-free!
I hope that you find these exchanges as amusing as I did – but are they important? Perhaps not exactly, but revealing certainly. They shed some light on the mind-set of acupuncturists and perhaps other alternative practitioners as well. Let me try to explain.
What struck me first was the degree of suspicion, even outright hostility from the acupuncturists. I had made it quite clear that I was asking Mel to join my Editorial-Board because of her views which vastly differ from mine. In science, differences of opinions and backgrounds can be stimulating and often generate progress. That is not something that seems to be wanted by alternative practitioners; they do not seem to tolerate criticism, different perspectives or views. One cannot help asking to what degree this attitude is immature or even dogmatic.
The next thing that baffled me was the speed with which conclusions are jumped upon. Everyone seemed to be instantly convinced that I was via my journal FACT in the pocket of the pharmaceutical industry. Nobody even bothered to look up what the Royal Pharmaceutical Society truly stand for and to verify that they do NOT represent ‘BIG PHARMA’. This blindness to the possibility of being wrong confirms my fear that alternative therapists are guided by strong beliefs which must not be questioned and are hard to influence, even with facts that take less than a minute to research.
And then there are, of course, the personal attacks which came quick, thick and fast. Its authors might think that such attacks get under my skin. If so, they are mistaken: if anything, they amuse me! I have long been of the opinion that they are important victories of reason. When an acupuncturist went as far as diagnosing me as being borderline psychopathic, I almost fell off my chair laughing! To me, this remark (which has emerged several times before) is emblematic, as it suggests several things at once:
- The author is obviously rude
- He/she is incompetent, even stupid
- He/she lacks empathy – after all, one would expect from a healthcare professional to show some understanding, if I were truly ill! And if not, one would expect more respect towards mentally ill patients.
- But, of course, he/she did not mean it like that; he/she merely meant to insult me. And employing mental health issues for this purpose shows a remarkable lack of professionalism, in my view.
Am I making too much of all this? Perhaps – sorry, I am almost done.
But first I need to briefly address Mel’s doubts about my integrity. She can, of course, question what she likes as often as she wants. My point is that repeating nonsensical arguments ad nauseam does not render then sensical.
Finally, there is Mel’s public claim that I have been lying about my conflict of interests. To me, it suggests a degree of desperation, perhaps even fanaticism, that is only surpassed by her inability to apologize after the truth had become undeniable even to her.
I know that there are some people who would have sued for libel.
For that I find all this far too hilarious.
I have previously reported about the issue of homeopathy on the NHS in Liverpool here. Since then, the NHS Liverpool Clinical Commissioning Group (CCG) has conducted a consultation on whether to continue funding. Personally, I think such polls are a daft waste of resources.
I will explain in a moment; first read the (slightly shortened) summary:
In November 2015, NHS Liverpool CCG Governing Body stated a preference to decommission the homeopathy service and commenced the consultation exercise with the intent to ascertain how the public felt about it. This report was written by the Centre for Public Health, Liverpool John Moores University, and includes independent analysis of the consultation activities.
The consultation ran from 13th November – 22nd December 2015. The two main methods used were 1) a survey available online and in paper format. It was completed by 743 individual respondents and, of those who provided a valid postcode, 68% (323 individuals) lived within the Liverpool CCG area, 2) a small consultation event held on 4th December 2015 facilitated by Liverpool John Moores University. The event was attended by 29 individuals, the majority of whom were patients and staff from the Liverpool Medical Homeopathic Service. Eighteen of the participants at this event resided in Liverpool.
Two thirds of survey respondents (66%; 380 respondents) said they would never use homeopathy services in the future. The reasons for this included the lack of evidence and scientific basis of homeopathy; negative personal experiences of homeopathy; and believing it was an inappropriate use of NHS funding. Those who would be likely to use it in the future (28%) felt they wanted to be able to choose an alternative to conventional medicine; felt it was value for money for the NHS; appreciated the time, care and holistic consultation; and discussed their own positive experiences. Sixty six per cent of survey respondents (111) who had used homeopathy in the past reported an excellent or good experience. Those who reported a positive experience (66%) felt that homeopathy had improved their health where conventional medicine had not, and participants valued that the homeopathic practitioner had treated their emotional as well as their physical needs. Those who reported a below average or poor experience (31%) felt homeopathy had not improved their medical condition and some felt they had been misled and had not been told the remedy contained no active ingredients.
At the consultation event, the majority of the 29 participants were homeopathy service users and they described a positive experience of homeopathy and the ability to choose ‘holistic’ and non-pharmaceutical treatment. Participants also questioned what services they could use if they were unable to access homeopathy on the NHS and were concerned and angry about the service potentially being decommissioned. A small number of participants at this event agreed with the view that there is a lack of evidence regarding efficacy and felt it was an inappropriate use of NHS funds that would be better spent on other, more effective services.
Of the survey respondents, 73% (541 individuals) chose the option to stop funding all homeopathy services; when including only Liverpool residents in the analysis this decreased to 64%. Twenty three per cent of survey respondents (170 individuals) wanted to continue to fund homeopathy services in Liverpool (either at current levels or to increase the budget); when only including Liverpool residents this proportion increased slightly to 30%. At the end of the consultation event the participants in the room (29 individuals) were asked to vote on their preferred funding option; twenty two participants (76%) wanted to continue the service and increase the maximum funding limit; three participants (14%) wanted to stay with the current situation and three participants (10%) wanted to stop funding the service.
There was some tension in what those in the consultation saw as acceptable and appropriate evidence about the effectiveness of homeopathy. Many participants in the survey and at the event reported their positive experience or anecdotal evidence as “proof” that homeopathy is effective. There was a low understanding about how scientific research is conducted or evaluated. The NHS try to base funding decisions on rigorous, high-quality, unbiased, peer-reviewed research, however, the CCG is required to account of all evidence, including patient experience, when funding or discontinuing services.
Across the survey and the consultation event there was some confusion about what types of treatment come under the heading of “homeopathy”, with participants making reference to a range of herbal remedies and supplements. Iscador (a mistletoe extract) may be, in some cases, provided as a complementary treatment for patients with cancer, however, this is not a homeopathic remedy. There was also discussion (in the event and in the survey responses) about other herbal remedies and supplements.
END OF SUMMARY
So, why do I not think highly about exercises of this kind?
In general, surveys are tricky and often very dodgy research tools. Particularly in alternative medicine, they are as popular as they are useless. The potential problems arise from the way the methodology is often applied. For instance, sampling is crucial. If, like in the present case, no rigorous sampling techniques are applied, the results will inevitably be unreliable in reflecting the views of a population.
The findings of the survey above could easily be little more than a reflection of which camp had a better PR. Homeopaths usually are very good on such occasions at persuading others for homeopathy. In this case, the results show that, despite their best efforts, the overall vote was not positive for homeopathy. What we don’t know is whether this is a reflection on the ‘will of the people’. It could be that the public is much more against funding nonsense than this poll suggests.
I would also argue that letting people vote about the availability of medical interventions is nonsensical. The value of healthcare technologies is not determined by such ‘beauty contests’; the value depends on the scientific evidence, and that is not readily evaluated by non-experts. Imagine: next we might vote for or against bone-marrow transplants; who has the expertise to cast such a vote?
Oh yes, and the ‘small consultation’ – what was that supposed to be. Probably just an exercise in political correctness. Nobody in their right mind can have expected any meaningful insight coming from it.
Finally, I dispute that ‘patients’ experience’ is the same as ‘evidence’, as the summary above seems to claim. This is just nonsense. evidence is something entirely different from experience.
But politicians will disregard all this. They will say ‘the public has decided’ and will stop funding homeopathy on the NHS in Liverpool. More by coincidence than by design, this survey went into the right direction. Now one can only hope that the rest of the country will follow suit – on evidence, not on dodgy pseudo-evidence from surveys.
This post is dedicated to Mel Koppelman.
Those who followed the recent discussions about acupuncture on this blog will probably know her; she is an acupuncturist who (thinks she) knows a lot about research because she has several higher qualifications (but was unable to show us any research published by herself). Mel seems very quick in lecturing others about research methodology. Yesterday, she posted this comment in relation to my previous post on a study of aromatherapy and reflexology:
Professor Ernst, This post affirms yet again a rather poor understanding of clinical trial methodology. A pragmatic trial such as this one with a wait-list control makes no attempt to look for specific effects. You say “it is quite simply wrong to assume that this outcome is specifically related to the two treatments.” Where have specific effects been tested or assumed in this study? Your statement in no way, shape or form negates the author’s conclusions that “aromatherapy massage and reflexology are simple and effective non-pharmacologic nursing interventions.” Effectiveness is not a measure of specific effects.
I am most grateful for this comment because it highlights an issue that I had wanted to address for some time: The meanings of the two terms ‘efficacy and effectiveness’ and their differences as seen by scientists and by alternative practitioners/researchers.
Let’s start with the definitions.
I often use the excellent book of Alan Earl-Slater entitled THE HANDBOOK OF CLINICAL TRIALS AND OTHER RESEARCH. In it, EFFICACY is defined as ‘the degree to which an intervention does what it is intended to do under ideal conditions. EFFECTIVENESS is the degree to which a treatment works under real life conditions. An EFFECTIVENESS TRIAL is a trial that ‘is said to approximate reality (i. e. clinical practice). It is sometimes called a pragmatic trial’. An EFFICACY TRIAL ‘is a clinical trial that is said to take place under ideal conditions.’
In other words, an efficacy trial investigates the question, ‘can the therapy work?’, and an effectiveness trial asks, ‘does this therapy work?’ In both cases, the question relate to the therapy per se and not to the plethora of phenomena which are not directly related to it. It seems logical that, where possible, the first question would need to be addressed before the second – it does make little sense to test for effectiveness, if efficacy has not been ascertained, and effectiveness without efficacy does not seem to be possible.
In my 2007 book entitled UNDERSTANDING RESEARCH IN COMPLEMENTARY AND ALTERNATIVE MEDICINE (written especially for alternative therapists like Mel), I adopted these definitions and added: “It is conceivable that a given therapy works only under optimal conditions but not in everyday practice. For instance, in clinical practice patients may not comply with a therapy because it causes adverse effects.” I should have added perhaps that adverse effects are by no means the only reason for non-compliance, and that non-compliance is not the only reason why an efficacious treatment might not be effective.
Most scientists would agree with the above definitions. In fact, I am not aware of a debate about them in scientific circles. But they are not something alternative practitioners tend to like. Why? Because, using these strict definitions, many alternative therapies are neither of proven efficacy nor effectiveness.
What can be done about this unfortunate situation?
Simple! Let’s re-formulate the definitions of efficacy and effectiveness!
Efficacy, according to some alternative medicine proponents, refers to the therapeutic effects of the therapy per se, in other words, its specific effects. (That coincides almost with the scientific definition of this term – except, of course, it fails to tell us anything about the boundary conditions [optimal or real-life conditions].)
Effectiveness, according to the advocates of alternative therapies, refers to its specific effects plus its non-specific effects. Some researchers have even introduced the term ‘real-life effectiveness’ for this.
This is why, the authors of the study discussed in my previous post, could conclude that “aromatherapy massage and reflexology are simple… effective… interventions… to help manage pain and fatigue in patients with rheumatoid arthritis.” Based on their data, neither aromatherapy nor reflexology has been shown to be effective. They might appear to be effective because patients expected to get better, or patients in the no-treatment control group felt worse for not getting the extra care. Based on studies of this nature, giving patients £10 or a box of chocolate might also turn out to be “simple… effective… interventions… to help manage pain and fatigue in patients with rheumatoid arthritis.” Based on these definitions of efficacy and effectiveness, there are hardly any limits to bogus claims for any old quackery.
Such obfuscation suits proponents of alternative therapies fine because, using such definitions, virtually every treatment anyone might ever think of can be shown to be effective! Wishful thinking, it seems, can fulfil almost any dream, it can even turn the truth upside down.
Or can anyone name an alternative treatment that cannot even generate a placebo response when administered with empathy, sympathy and care? Compared to doing nothing, virtually every ineffective therapy might generate outcomes that make the treatment look effective. Even the anticipation of an effect alone might do the trick. How often have you had a tooth-ache, went to the dentist, and discovered sitting in the waiting room that the pain had gone? Does that mean that sitting in a waiting room is an effective treatment for dental pain?
In fact, some enthusiasts of alternative medicine could soon begin to argue that, with their new definition of ‘effectiveness’, we no longer need controlled clinical trials at all, if we want to demonstrate how effective alternative therapies truly are. We can just do observational studies without a control group, note that lots of patients get better, and ‘Bob is your uncle’!!! This is much faster, saves money, time and effort, and has the undeniable advantage of never generating a negative result.
To most outsiders, all this might seem a bit like splitting hair. However, I fear that it is far from that. In fact, it turns out to be a fairly fundamental issue in almost any discussion about the value or otherwise of alternative medicine. And, I think, it is also a matter of principle that reaches far beyond alternative medicine: if we allow various interest groups, lobbyists, sects, cults etc. to use their own definitions of fundamentally important terms, any dialogue, understanding or progress becomes almost impossible.
WHAT IS THE WORST THAT COULD HAPPEN TO HOMEOPATHS?
This might seem like a strange question, but I think it is quite interesting… bear with me.
The worse, you might think, is that the we all agree that highly diluted homeopathic remedies are pure placebos. Apart from the fact that this already is a broad consensus shared by virtually everyone in healthcare (except the homeopaths, of course), I think this is not the worst that could happen to homeopaths. They simply ignore the consensus, continue much as before and carry on earning a living by fooling the public (and often themselves as well).
No, the worse is the opposite of the above. The worse is that we all accept the homeopaths’ view. The worse is to say: Very well, we agree for the moment that your remedies are highly effective. And therefore we need to regulate them just as any other medicine.
Wenn dies für Homöopathen also so eindeutig ist, dann können die zuständigen Institutionen in den Arzneimittel-Gesellschaften (BfArM, AMG) Homöopathika genau so bewerten wie normale Medikamente… die Politik sollte die Homöopathen bei ihrem eigenen Wort nehmen und sie denselben Prüfverfahren unterwerfen wie alle anderen Behandlungsverfahren auch.
And this is my (somewhat liberal) translation:
If homeopathy’s effectiveness is so crystal clear, the regulators should assess homeopathic remedies just like normal drugs… politicians and regulators should take homeopaths by their own word and should apply the same standards as for all other medicines.
In the past, homeopaths have always wanted the cake and eat it; they pleaded that their remedies are so special and therefore they need special regulations and extra considerations. Because of these, they were sheltered and escaped any legal or ethical obligations to demonstrate effectiveness. This introduced an unjustified and regressive double standard with was detrimental to good healthcare, medical ethics and scientific progress.
Now that homeopaths (the Germans are merely an example, other countries’ homeopaths are much the same) have agreed on what they think is solid scientific proof, it is right and necessary to remove the special protection which homeopathy used to enjoy. Let’s for the moment accept the homeopaths’ argument (‘homeopathy is effective just like other medicines) and then force them to deliver the proof of their opinion according to the standards all medicines must be judged by!
That would surely be the end of all this nonsense, and homeopaths would find themselves hoisted by their own petard.
Ad hominem attacks, I have previously pointed out, are victories of reason over unreason. And they are used frequently by supporters of alternative medicine!
If you doubt it, see for yourself.
I recently posted a comment on new Nice guidelines. It generated lots of comments, and mostly they were rational discussions of the issues involved. This changed abruptly when, on 16 May, Mel’s comment started a new, concerted wave of discussion at a time when the usual debate had already subsided. In the course of this new and heated debate, I was repeatedly accused of being rude.
As I have stated repeatedly on this blog, I try to keep rudeness out of the comments as much as I can. Therefore, the claim surprised me and today I reviewed the entire comment section selecting all potential ad hominem attacks. Here are the results:
ACTUAL OR POTENTIAL AD HOMINEM ATTACKS AGAINST ME
Peter Deadman on Tuesday 17 May 2016 at 12:55 Edward Ernst, I always thought you were a bully and a fraud. You’re very macho when it comes to slapping down people who may have experiential reasons for supporting acupuncture and other therapies but don’t have the skill to challenge you on the clinical evidence. Now as soon as somebody does, you back off, cry ‘enough’, say you can’t possibly comment till some undetermined future date and generally act like a wuss. I say put up or shut up. I’d prefer the former because it would be good to see you eat crow but I lean towards the latter because of the substantial harm you are causing and the beautiful silence that would ensue if you did indeed go quiet.
tonto on Tuesday 17 May 2016 at 13:19 You appear as weak in your arguments, as some pendulum swinging, new age dowser, who vainly holds sticks to their guns, not because they can back their position up with scientific evidence, but because it is what they “believe”.
Jill Onyett on Tuesday 17 May 2016 at 14:29 …an unfortunate creature too keen on the sound of his own voice.
Tracey Phillips on Tuesday 17 May 2016 at 13:16 …to date you have been fairly opinionated …
Peter Deadman on Tuesday 17 May 2016 at 16:34 I made an ad hominem response because your blog is all about you as a person. You are constantly rude to others and bypass or ignore responses that you don’t like. It’s you who makes it hominem.
Peter Deadman on Tuesday 17 May 2016 at 16:52 You are hyper-emotional, extremely biased, hostile and contemptuous of anyone you think ‘beneath you’. You gloat over people’s real or imagined inconsistencies and generally come across as a nasty piece of work.
Peter Deadman on Tuesday 17 May 2016 at 19:30 How can such a childish provocateur remain in his post. It demeans the University and it’s time they let him go.
Kylee Junghans on Wednesday 18 May 2016 at 08:42 …you, kind Sir, with your rhetoric and tantrums, are exhibiting a prime example of confirmation bias.
Peter Deadman on Wednesday 18 May 2016 at 08:48 [Ernst] professes a scholarly detachment, a commitment to evidence and an open mind, but in fact is deeply biased… He clearly loves his childish provocative stance and is as far from a disinterested observer as it’s possible to be. I wouldn’t waste my time or breath on him if he didn’t have an influence that far exceeds his worth.
Carol Cooke on Wednesday 18 May 2016 at 09:27 I have followed this discussion with interest. Some of the rudest and most discourteous posts I can see are from Mr Ernst himself. But I get that, I imagine you seek to maintain a bold and authoritative tone simply by dismissing others. Being a bit controversial in your discourse has obviously served you well in that you have built a media profile on it.
ACTUAL OR POTENTIAL AD HOMINEM ATTACKS BY MYSELF
Edzard on Wednesday 18 May 2016 at 09:18 “it is also difficult to get a man to read something, when he is foaming from his mouth”.
I know, this is not really ‘ad hominem’ but I could not find anything more dramatic. Surely, some will disagree this me here, and I do invite them to cite my rudeness from this threat, if they spot it. You are more than welcome!
You may think this is a bit trivial, but I disagree. The main reason I did this little exercise is to demonstrate a point which I think is important and carries a relevant lesson for future comments and discussions:
- WHEN I OR ANYONE ELSE DEFENDING RATIONALITY GET AGGRESSED, WE NATURALLY TEND TO RESPOND SLIGHTLY MORE FORCEFULLY.
- SUBSEQUENTLY, THE OTHER SIDE OFTEN REACTS BY ATTACKING US PERSONALLY.
- THIS OFTEN LEADS TO AN ESCALATION OF TONE.
- EVENTUALLY THE OTHER SIDE CLAIMS WITH INDIGNATION THAT WE ARE THE ONES DOING THE PERSONAL ATTACKS.
- IT IS A TACTIC THAT IS EFFECTIVE BUT DISHONEST, IN MY VIEW.
- THE LESSON IS SIMPLE: DO NOT LET YOURSELF GET PROVOKED INTO ISSUING AD HOMINEM ATTACKS, BE POLITE AND PATIENT.
I know this sounds simpler than it is, and I am far from being immune to the problem, but we owe it to reason to give it a try.
It has been reported in most newspapers that Prince Charles has proposed a solution to the problem of antibiotic over-use in animals and humans. He told an international gathering of scientists and government officials in London that he treats his own cows and sheep with homeopathy. Many people reacted with dismay. I, however, plead for more understanding of this thoroughly good-willed man.
In fact, I intend to go one step further.
We have often heard that he is a considerate and caring man. We ought to give him the benefit of the doubt. I have tried therefore to empathise with his situation, put myself into his shoes and repeat the considerations that made him say what he said. My empathy went so far that I tried to re-live and formulate his thoughts in the first person singular (or should I have used the ‘Royal We’?). The result is the following little monologue where I categorised the considerations under 7 headings.
I wonder why they invited me to give a speech. True, I am a farmer at heart and I know all about husbandry, but I have no real expertise in the field of antibiotics.
Perhaps it is an occasion to tell them a bit about homeopathy. Yes, that subject would surely fascinate the audience!
THE INTELLECTUAL ENVIRONMENT
They tell me that the conference will be packed with very bright people. That sort of thing always makes me a bit self-conscious. Perhaps I should decline the invitation after all? Sometimes, I have the impression that people make fun at me.
No, I must not think like that – after all, I have had a very expensive education too, and I know my stuff.
Homeopathy is such a wonderful subject. I must try to win them over and make them appreciate its beauty. These experts should realize that homeopathy is the future.
I have heard rumours that some blinkered scientists doubt that homeopathy works. But my advisors tell me that it is best to ignore this sort of thing. And my advisors know their stuff even better than I.
This conference is going to have a very high impact. The press will be there. It will be reported across the world. And government’s chief medical officer, Dame Sally Davies (why can we not have more holistic doctors in position like these; I must remember to discuss this with Michael Dixon asap); she once called homeopathy ‘rubbish’ – enough to throw her in the Tower!
Mustn’t think like that! On the contrary, I will make sure they all get the message. I will bowl them over! The press will surely be on my side. This will be a victory for homeopathy.
Mother might be upset; she does not like me to stick my neck out like that…goes on about constitutional role and such trivia…she thinks we should not put our nose into things that are none of our business. And the Royal PR team will not like it either. They do what they can to distract from the image of ‘THE MEDDLING PRINCE’ might think that my speech is a hindrance to their efforts. I better don’t tell anyone in advance about this, they might try to stop me.
But now I feel quite unstoppable.
This is what I will tell them about homeopathy: “It was one of the reasons I converted my farming operation to an organic – or agro-ecological – system over 30 years ago and why we have been successfully using homeopathic – yes, homeopathic – treatments for my cattle and sheep as part of a programme to reduce the use of antibiotics.” I think this will convince everyone. Who needs science when there is powerful rhetoric like this?
What if it does not go well? They will not dare to contradict me, I am the future King, for heaven’s sake! Even if they disagree, they will not show it. They just don’t have the guts. And anyway, I will not take questions, I never do enter into any debate on homeopathy. It is simply too tedious to argue with people who do not understand the issues involved.
It is decided – I’ll do it. I’ll do it for homeopathy and for the good of mankind. If they then chose to misunderstand me, that’s their problem, and my people will issue a statement for the press saying “Homeopathy is used on a case-by-case basis at Home Farm, in combination with more conventional medicine, to minimise dependence on antibiotics.”
*In case you are a lawyer hired by HRH to check out this post: it is pure satire through and through, there is not a factual sentence here; if you want to sue me, please find another reason.