Seasonal allergic rhinitis (hay fever) is a common condition which can considerably reduce the quality of life of sufferers. Homeopathy is often advocated – but does it work?
A new study was meant to be an “assessment of the clinical effectiveness of homeopathic remedies in the alleviation of hay fever symptoms in a typical clinical setting.”
The investigator performed a ‘clinical observational study’ of eight patients from his private practice using Measure Yourself Medical Outcome Profile (MYMOP) self-evaluation questionnaires at baseline and again after two weeks and 4 weeks of individualized homeopathic treatment which was given as an add-on to conventional treatments.
The average MYMOP scores for the eyes, nose, activity and wellbeing had improved significantly after two and 4 weeks of homeopathic treatment. The overall average MYMOP profile score at baseline was 3.83 (standard deviation, SD, 0.78). After 14 and 28 days of treatment the average score had fallen to 1.14 (SD, 0.36; P<0.001) and 1.06 (SD, 0.25; P<0.001) respectively.
The author concluded as follows: Individualized homeopathic treatment was associated with significant alleviation of hay fever symptoms, enabling the reduction in use of conventional treatment. The results presented in this study can be considered as a step towards a pilot pragmatic study that would use more robust outcome measures and include a larger number of patients prescribed a single or a multiple homeopathic prescription on an individualized basis.
It is hard to name the things that are most offensively wrong here; the choice is too large. Let me just list three points:
- The study design is not matched to the research question.
- The implication that homeopathy had anything to do with the observed outcome is unwarranted.
- The conclusion that the results might lend themselves to develop a pilot study is meaningless.
The question whether homeopathy is an effective therapy for hay fever has been tested before, even in RCTs. It seems therefore mysterious why one needs to revert to tiny observational studies in order to plan a pilot, and even less for an assessment of effectiveness.
There are few conditions which are more time-dependent than hay fever. Any attempt of testing the effectiveness of medical interventions without a control group seems therefore not just questionable but wasteful. Clinical studies absorb resources; even if the author was happy to waste his time, he should not assume that he can freely waste the time, effort and availability of his patients.
Two final points, if I may:
- An observational study of homeopathy for hay-fever without a control group might be utterly useless but it is still an investigation that requires certain things. As far as I can see, this study did not even have ethics approval nor is there a mention of informed consent. Strictly speaking, this makes it an unethical study.
- If we allow research of this nature to take place and be published, we give clinical research a bad name and undermine the confidence of the public in science.
I am puzzled how such a paper could pass peer review and how an Elsevier journal could even consider publishing it.
The question whether pharmacists should sell unproven alternative medicines will not go away. On this blog, we have discussed it repeatedly, for instance here, here and here. The Australian Journal of Pharmacy’s latest poll shows that readers have their suspicions about the validity of naturopathic medicines, with a whopping 544 voters choosing the option, “No, there’s no evidence they work” at the time of writing.
This constitutes 65% of readers who took part in the poll. A significant minority – 193 readers, with 23% of the vote – said that pharmacies should stock these medicines as they are legitimate products. Five per cent said that while they questioned their efficacy, pharmacy should stock them; and 3% said they were unsure, but the public wanted them.
Taree pharmacist and member of Friends in Science and Medicine Ian Carr, who has spoken to the AJP several times in the last couple of weeks as debate has continued about the subject of naturopathy in pharmacy, said he was surprised and pleased at the strength of the No vote. “I looked at [the poll] on the first day, and there was definitely a majority saying these things have no evidence, but there was still above 30% saying yes, they were legitimate products,” Carr told the AJP. “That’s been dwarfed by a lot of people who’ve looked in, and it’s interesting to have that many people vote. “I’m glad that it seems to be becoming recognised that there’s a need for the evidence base in these things, and the difference between having a naturopathic product or supplement on the shelf, and having somebody there charging for their time, as a naturopath, dispensing advice without knowing the patient’s background and without an intervention by a registered pharmacist.” He encouraged pharmacists concerned about the validity of naturopathy to consider what products and services they offer.
Where naturopaths are used, they should at least be expected to keep a record of products and advice dispensed, he says, similar to protocols around blood pressure and blood glucose monitoring. “If there’s going to be an insistence that naturopaths remain, that’s the way I’d like to see it: that the pharmacy has good records and oversight of what they’re doing. I think, given our connection to the PBS and the fact that we as pharmacists are looking for a more serious role as part of the health care team generally, and having a more active and integrative role, we would be silly to fritter it away on peripheries like naturopathy. I personally see the opportunities in evidence-based medicine and what flows from that, rather than trying to make up dollars. We’re more likely to lose control of pharmacy if we don’t guard it jealousy.”
One of the suppliers of CAM products to pharmacies responded to the article by stating the following:
“The complementary and alternative medicine (CAMs) sector and its role in healthcare management continues to be hotly debated by the media. Rather than dissuade this debate, we actively encourage this discussion, as it shines a light on many issues which need to be addressed. Of priority is the point that not all complementary and alternative medicine products are equal. As in many media articles, an incredibly wide spectrum of products are grouped under the label of ‘CAMs’. Products with specific clinical evidence, high-quality manufacturing processes and transparency on the sourcing of ingredients are not clearly identified from products without these qualities. Consumers and healthcare professionals are unable to distinguish this difference due to a lack of clear labelling. We agree with calls for CAMs products to be more thoroughly assessed, beyond being simply classified as ‘safe’. Healthcare professionals and consumers deserve this information and are indeed asking for it. Consumers are aware of the impact of their choices and that their demand drives industry change. History is littered with recent examples where consumer awareness has changed the marketplace for the better. Consumer-driven change in the CAMs industry IS possible, it just needs to be supported. The Australian CAMs industry needs to increase healthcare professional and consumer education on the importance of evidence-based CAM products; on what ‘evidence-based’ means and what this difference delivers… Healthcare professionals are key to helping their patients understand that not all CAMs or natural medicine products are equal… It takes time to change the way people see CAMs and natural medicines – but it is of inherent value for the consumer. Something, we believe, is integral to the future of the industry.”
The arguments are clearest, if we focus on a specific type of alternative medicine and spell out what precisely we are talking about. The one that comes to mind is, of course, homeopathy. In my view, there is no good reason why pharmacists should sell homeopathic remedies. It is comforting to know that the Chief Scientist of the UK Royal Pharmaceutical Society, Professor Jayne Lawrence, agrees; she stated about a year ago that “the public have a right to expect pharmacists and other health professionals to be open and honest about the effectiveness and limitations of treatments. Surely it is now the time for pharmacists to cast homeopathy from the shelves and focus on scientifically based treatments backed by clear clinical evidence.”
And what has changed since?
Nothing, as far as I can see – but please correct me, if I am wrong.
I think it is important that we remind the community pharmacists everywhere that they have their very own codes of ethics and that they need to adhere to them. If they don’t, they tacitly agree that they are not really healthcare professionals but mere shop-keepers.
I am editor in chief of a journal called FACT. It has a large editorial board, and I am always on the look-out for people who might be a good, productive and colourful addition to it. On 3 June, I sent an invitation to Mel Koppelman, who is by now well known to regular readers of this blog. Here is a copy:
can I ask you a question?
would you consider joining the ed-board of FACT [as you mentioned it in one of your comments, I assume you know this journal – but you are wrong in implying that it has anything to do with the pharmaceutical or any other industry]? if you agree, we would expect you to write 2-3 ‘summaries/commentaries’ per year. in return you get a free subscription and, of course, can submit other articles.
no, this is not a joke or a set-up. I like to have the full spectrum of opinion/expertise on my ed-board, and I do think you understand science quite well. our opinions differ but that’s what I think is good for the journal.
think about it – please.
On 6 June, she replied as follows:
Great to hear from you, I hope you enjoyed your weekend.
Thank you very much for the kind offer, it’s something I would consider. I certainly have no problem with, and in fact embrace, people who have different opinions and views from my own, so long as I feel that they have integrity in their approach.
Just a few questions / comments:
1) Regarding FACT’s affiliations, what I said in my comment was that it was a publication of the Royal Pharmaceutical Society. According to Wiley’s website, Focus on Alternative and Complementary Medicine is copyright by Royal Pharmaceutical Society. It’s also listed on the Pharmaceutical Press website.
Are you telling me that’s incorrect? That I’m “wrong” in saying there’s a relationship? I obviously need to understand the nature of the publication whose editorial board I’m considering joining. Very confusing that you as editor say there’s no relationship to the RPS and yet they claim copyright over your publication. Incidentally, is FACT self-sufficient, earning all of its income from subscriptions? Or does any financial support come from the publishers?
2) As enticing as a free subscription to FACT is, I have access to more high quality peer-reviewed reading material than I could enjoy in many lifetimes. Because my skills seem to be in high demand and because I already spend 10-20 hours per week doing unpaid volunteer work, any additional projects that I take on at this time would need to be financially compensated. I understand that this may be a deal-breaker.
3) While I have no issue with you having different views when it comes to medical research, in order to choose to work with your publication, it’s important to me that it’s run by people with a high level of academic integrity and put patient welfare at the forefront of it’s agenda.
In March, you came out in public support of the NICE draft guidelines. You were quoted in the Guardian as saying: “It is good to see that Nice have now caught up with the evidence. Neither spinal manipulation nor acupuncture are supported by good science when it comes to treating low back pain.”
Following this, it was brought to your attention that the recommendations were contrary to best evidence and that the conclusions were unsupportable. While you have the option of following this up to make sure that the record reflects best evidence, you have indicated that you have no interest in evaluating the situation and possibly admitting an error. This behaviour is concerning from the perspective of academic integrity, particularly when it directly leads to increased human suffering (policy in several countries has already been changed based on the draft), and I would be worried that by joining your board I could be associated with such unethical behaviour.
Perhaps if I understood better your position, which seems to be to ignore the situation, not follow up on the concerns raised, and leave your comments uncorrected even though they may be inaccurate and backing guidelines that cause harm to patients, that might allay certain reservations.
Anyways, these are my initial thoughts. I hope you have an opportunity to enjoy the beautiful weather, and I look forward to hearing from you.
At that stage, I began to fear that I had made a mistake. But, giving her the benefit of the doubt, I swallowed my pride and replied as politely as I could to her concerns which, in my view, were odd, to say the least. This is what I wrote on 7 June:
Thank you for your reply to my invitation. Let me address your points in turn:
- I said that FACT has nothing to do with the pharma industry which is true [when you state that “you as editor say there’s no relationship to the RPS” – it suggests to me that you did not read my email properly]. In their own words, the RPS is “the professional membership body for pharmacists and pharmacy in Great Britain and an internationally renowned publisher of medicines information.” [http://www.rpharms.com/home/about-us.asp] They have a similar status as the Royal Colleges. In the 2 decades that I am running the journal, there has been not a single instance of interference of any kind. We use them simply as an excellent publishing house. And yes, FACT is to the best of my knowledge self-sufficient and survives without funds from 3rd parties.
- I am delighted to hear that your skills are in demand.
- I have stated my position regarding the draft NICE guideline ad nauseam: I prefer to wait until I see their next version of the draft before I make further comments on it. In my view, this is both reasonable and honourable. If you disagree, I can do little about it other than expressing my sincere regrets.
I hope these brief clarifications are helpful for you to arrive at a decision.
On 8 June, I received Mel’s reply:
How interesting! Is he trying to ‘keep his enemies closer’ or am I being too skeptical? He has recognised your talents and dedication and intellect so he is not altogether stupid after all! I eagerly await his response to your reply.
Those who have studied with Ernst say that he’s a genuine chap and misunderstood – which I know is almost unimaginable given his behaviour – but we always have to allow for the possibility that we have misjudged people however remote! Also, people can turn – especially when they get older and near retirement. Alternatively he may just fancy you!…
Mel Koppelman Really enjoying hearing y’all’s thoughts on this. I just want to say that If I had thought that the chances of me being able to create positive change by joining FACT were high, I would have tempered the tone of my reply. But the simple fact that EE can’t even be factual or forthright about whose journal it is suggests an irreparable break with reality. And surely there’s an issue (academic? ethical? legal?) with recruiting someone to your board and denying an industry tie when there is one? Not to mention that if the RSP does fund his journal, he’s been lying about his conflicts of interest. Is that someone I want to spend time adding value to that could be spent with family, patients, time in nature or really making positive change by supporting the ANF? I’ll be interested to read his reply if there is one and especially how he responds regarding the relationship between FACT and RSP. Will keep y’all posted.
John McDonald Healthy skepticism! Healthy journal! And it’s 99% fact-free!
I hope that you find these exchanges as amusing as I did – but are they important? Perhaps not exactly, but revealing certainly. They shed some light on the mind-set of acupuncturists and perhaps other alternative practitioners as well. Let me try to explain.
What struck me first was the degree of suspicion, even outright hostility from the acupuncturists. I had made it quite clear that I was asking Mel to join my Editorial-Board because of her views which vastly differ from mine. In science, differences of opinions and backgrounds can be stimulating and often generate progress. That is not something that seems to be wanted by alternative practitioners; they do not seem to tolerate criticism, different perspectives or views. One cannot help asking to what degree this attitude is immature or even dogmatic.
The next thing that baffled me was the speed with which conclusions are jumped upon. Everyone seemed to be instantly convinced that I was via my journal FACT in the pocket of the pharmaceutical industry. Nobody even bothered to look up what the Royal Pharmaceutical Society truly stand for and to verify that they do NOT represent ‘BIG PHARMA’. This blindness to the possibility of being wrong confirms my fear that alternative therapists are guided by strong beliefs which must not be questioned and are hard to influence, even with facts that take less than a minute to research.
And then there are, of course, the personal attacks which came quick, thick and fast. Its authors might think that such attacks get under my skin. If so, they are mistaken: if anything, they amuse me! I have long been of the opinion that they are important victories of reason. When an acupuncturist went as far as diagnosing me as being borderline psychopathic, I almost fell off my chair laughing! To me, this remark (which has emerged several times before) is emblematic, as it suggests several things at once:
- The author is obviously rude
- He/she is incompetent, even stupid
- He/she lacks empathy – after all, one would expect from a healthcare professional to show some understanding, if I were truly ill! And if not, one would expect more respect towards mentally ill patients.
- But, of course, he/she did not mean it like that; he/she merely meant to insult me. And employing mental health issues for this purpose shows a remarkable lack of professionalism, in my view.
Am I making too much of all this? Perhaps – sorry, I am almost done.
But first I need to briefly address Mel’s doubts about my integrity. She can, of course, question what she likes as often as she wants. My point is that repeating nonsensical arguments ad nauseam does not render then sensical.
Finally, there is Mel’s public claim that I have been lying about my conflict of interests. To me, it suggests a degree of desperation, perhaps even fanaticism, that is only surpassed by her inability to apologize after the truth had become undeniable even to her.
I know that there are some people who would have sued for libel.
For that I find all this far too hilarious.
I have previously reported about the issue of homeopathy on the NHS in Liverpool here. Since then, the NHS Liverpool Clinical Commissioning Group (CCG) has conducted a consultation on whether to continue funding. Personally, I think such polls are a daft waste of resources.
I will explain in a moment; first read the (slightly shortened) summary:
In November 2015, NHS Liverpool CCG Governing Body stated a preference to decommission the homeopathy service and commenced the consultation exercise with the intent to ascertain how the public felt about it. This report was written by the Centre for Public Health, Liverpool John Moores University, and includes independent analysis of the consultation activities.
The consultation ran from 13th November – 22nd December 2015. The two main methods used were 1) a survey available online and in paper format. It was completed by 743 individual respondents and, of those who provided a valid postcode, 68% (323 individuals) lived within the Liverpool CCG area, 2) a small consultation event held on 4th December 2015 facilitated by Liverpool John Moores University. The event was attended by 29 individuals, the majority of whom were patients and staff from the Liverpool Medical Homeopathic Service. Eighteen of the participants at this event resided in Liverpool.
Two thirds of survey respondents (66%; 380 respondents) said they would never use homeopathy services in the future. The reasons for this included the lack of evidence and scientific basis of homeopathy; negative personal experiences of homeopathy; and believing it was an inappropriate use of NHS funding. Those who would be likely to use it in the future (28%) felt they wanted to be able to choose an alternative to conventional medicine; felt it was value for money for the NHS; appreciated the time, care and holistic consultation; and discussed their own positive experiences. Sixty six per cent of survey respondents (111) who had used homeopathy in the past reported an excellent or good experience. Those who reported a positive experience (66%) felt that homeopathy had improved their health where conventional medicine had not, and participants valued that the homeopathic practitioner had treated their emotional as well as their physical needs. Those who reported a below average or poor experience (31%) felt homeopathy had not improved their medical condition and some felt they had been misled and had not been told the remedy contained no active ingredients.
At the consultation event, the majority of the 29 participants were homeopathy service users and they described a positive experience of homeopathy and the ability to choose ‘holistic’ and non-pharmaceutical treatment. Participants also questioned what services they could use if they were unable to access homeopathy on the NHS and were concerned and angry about the service potentially being decommissioned. A small number of participants at this event agreed with the view that there is a lack of evidence regarding efficacy and felt it was an inappropriate use of NHS funds that would be better spent on other, more effective services.
Of the survey respondents, 73% (541 individuals) chose the option to stop funding all homeopathy services; when including only Liverpool residents in the analysis this decreased to 64%. Twenty three per cent of survey respondents (170 individuals) wanted to continue to fund homeopathy services in Liverpool (either at current levels or to increase the budget); when only including Liverpool residents this proportion increased slightly to 30%. At the end of the consultation event the participants in the room (29 individuals) were asked to vote on their preferred funding option; twenty two participants (76%) wanted to continue the service and increase the maximum funding limit; three participants (14%) wanted to stay with the current situation and three participants (10%) wanted to stop funding the service.
There was some tension in what those in the consultation saw as acceptable and appropriate evidence about the effectiveness of homeopathy. Many participants in the survey and at the event reported their positive experience or anecdotal evidence as “proof” that homeopathy is effective. There was a low understanding about how scientific research is conducted or evaluated. The NHS try to base funding decisions on rigorous, high-quality, unbiased, peer-reviewed research, however, the CCG is required to account of all evidence, including patient experience, when funding or discontinuing services.
Across the survey and the consultation event there was some confusion about what types of treatment come under the heading of “homeopathy”, with participants making reference to a range of herbal remedies and supplements. Iscador (a mistletoe extract) may be, in some cases, provided as a complementary treatment for patients with cancer, however, this is not a homeopathic remedy. There was also discussion (in the event and in the survey responses) about other herbal remedies and supplements.
END OF SUMMARY
So, why do I not think highly about exercises of this kind?
In general, surveys are tricky and often very dodgy research tools. Particularly in alternative medicine, they are as popular as they are useless. The potential problems arise from the way the methodology is often applied. For instance, sampling is crucial. If, like in the present case, no rigorous sampling techniques are applied, the results will inevitably be unreliable in reflecting the views of a population.
The findings of the survey above could easily be little more than a reflection of which camp had a better PR. Homeopaths usually are very good on such occasions at persuading others for homeopathy. In this case, the results show that, despite their best efforts, the overall vote was not positive for homeopathy. What we don’t know is whether this is a reflection on the ‘will of the people’. It could be that the public is much more against funding nonsense than this poll suggests.
I would also argue that letting people vote about the availability of medical interventions is nonsensical. The value of healthcare technologies is not determined by such ‘beauty contests’; the value depends on the scientific evidence, and that is not readily evaluated by non-experts. Imagine: next we might vote for or against bone-marrow transplants; who has the expertise to cast such a vote?
Oh yes, and the ‘small consultation’ – what was that supposed to be. Probably just an exercise in political correctness. Nobody in their right mind can have expected any meaningful insight coming from it.
Finally, I dispute that ‘patients’ experience’ is the same as ‘evidence’, as the summary above seems to claim. This is just nonsense. evidence is something entirely different from experience.
But politicians will disregard all this. They will say ‘the public has decided’ and will stop funding homeopathy on the NHS in Liverpool. More by coincidence than by design, this survey went into the right direction. Now one can only hope that the rest of the country will follow suit – on evidence, not on dodgy pseudo-evidence from surveys.
We tend to trust charities; many of us donate to charities; we think highly of the work they do and the advice they issue. And why shouldn’t we? After all, a ‘charity’ is ‘an institution or organization set up to provide help, money, etc, to those in need’. Not a hint at anything remotely sinister here – charities are good!
Except, of course, those that are not so good!
By ‘not so good’ I mean charities that misinform the public to a point where they might even endanger our health, well-being and savings. Yes, I am speaking of those charities that promote unproven or disproven alternative therapies – and unfortunately, there are many of those around today.
Our recent letter in the SUNDAY TIMES, tried to alert the public to this problem and to the fact that the UK regulator seems to be failing to do much about it. A Charity Commission spokesman, in turn, replied that his organisation had received the letter and would respond formally to it:
“The Commission is required to register organisations as charities which are established for exclusively charitable purposes for the public benefit,” he said. “Charitable purposes for the advancement of health include conventional methods as well as complementary, alternative or holistic methods which are concerned with healing mind, body and spirit in the alleviation of symptoms and the cure of illness. Those organisations dealing with complementary and alternative medicines must be able to demonstrate that they are capable of promoting health otherwise they will not be for the public benefit.
“The Commission is the registrar and regulator of charities however it is not the authority in the efficacy of any and every non-traditional medical treatment. These are issues of substantial debate with a variety of opinions. Each case is considered on its merits based on the evidence available. To be charitable there needs to be sufficient evidence of the efficacy of the method to be used. The Commission must further be assured that any potential harm that might be said to arise does not outweigh the benefit identified by the method.
“The Commission expects charities to provide information that is factually accurate with legitimate evidence.”
But is the information provided by all charities factually accurate?
Take, for instance, YES TO LIFE! Have a good look and then decide for yourself.
On their website they state: “We provide support, information and financial assistance to those with cancer seeking to pursue approaches that are currently unavailable on the NHS. We also run a series of educational seminars and workshops which are aimed at the general public who want to know more and practitioners working with people who have cancer.”
The website informs us about many alternative therapies and directly or indirectly promote them for the curative or supportive treatment of cancer. I have chosen 5 of them and copied the respective summaries as published by YES TO LIFE. My main selection criterion was having done some research myself on the modality in question. Here are the 5 cancer treatments which I selected; the text from YES TO LIFE is in bold, and that of my published research is in normal print with a link to the published paper:
Carctol is a relatively inexpensive product, specifically formulated to assist cells with damaged respiration, it is also a powerful antioxidant that targets free radicals, the cause of much cellular damage. It also acts to detoxify the system.
Often given intravenously as part of a programme of Metabolic Therapy, Laetrile is a non-toxic extract of apricot kernels. The claimed mechanism of action that is broken down by enzymes found in cancer cells. Hydrogen cyanide, one of the products of this reaction then has a local toxic effect on the cells.
The claims that laetrile or amygdalin have beneficial effects for cancer patients are not currently supported by sound clinical data. There is a considerable risk of serious adverse effects from cyanide poisoning after laetrile or amygdalin, especially after oral ingestion. The risk-benefit balance of laetrile or amygdalin as a treatment for cancer is therefore unambiguously negative.
Mistletoe therapy was developed as an adjunct to cancer treatment in Switzerland in 1917-20, in the collaboration between Dr I Wegman MD and Dr Rudolf Steiner PhD (1861-1925). Mistletoe extracts are typically administered by subcutaneous injection, often over many years. Mistletoe treatment improves quality of life, supports patients during recommended conventional cancer treatments and some studies show survival benefit. It is safe and has no adverse interactions with conventional cancer treatments.
None of the methodologically stronger trials exhibited efficacy in terms of quality of life, survival or other outcome measures. Rigorous trials of mistletoe extracts fail to demonstrate efficacy of this therapy.
A type of low toxicity chemotherapy derived from a combination of two known cytotoxic drugs that are of little use individually, as the doses required for effective anticancer action are too high to be tolerated. However the combination is effective at far lower doses, with few side effects.
The data from randomised clinical trials suggest Ukrain to have potential as an anticancer drug. However, numerous caveats prevent a positive conclusion, and independent rigorous studies are urgently needed. [To judge the validity of this last treatment, I also recommend reading a previous post of mine.]
Finally, it might be informative to see who the individuals behind YES TO LIFE are. I invite you to have a look at their list of medical advisors which, I think, speaks for itself. It includes, for instance, Dr Michael Dixon of whom we have heard before on this blog, for instance, here, here and here.
Say no more!
The subject of placebo is a complex but fascinating one, particularly for those interested in alternative medicine. Most sceptics believe that alternative therapies rely heavily, if not entirely, on the placebo effect. Some alternative practitioners, when unable to produce convincing evidence that their treatment is effective, seem to have now settled to admitting that their therapy works (mostly or entirely) via a placebo effect. They the hasten to add that this is perfectly fine, because it is just an explanation as to how it works – a mechanism of action, in other words. Causing benefit via a placebo effect still means, they insist, that their therapy is effective.
In a previous post, I have tried to demonstrate that this belief is erroneous and where the notion comes from. It originates, I believe, from a mistaken definition of ‘effectiveness’: for many alternative practitioners ‘effectiveness’ encompasses the specific plus the non-specific (e. g. placebo) effects of their therapy. In real medicine, ‘effectiveness’ is the degree to which a treatment works under real life conditions.
The ‘alternative’ definition is, of course, incorrect but alternative practitioners stubbornly refuse to acknowledge this fact. Here are just two reasons why it cannot be right:
- If it were correct, it would be hardly conceivable to think of a treatment that is NOT effective. Applied with empathy and compassion, virtually all treatments – however devoid of specific effects – will produce a placebo effect. Thus they will all be effective, and the term would be superfluous because ‘treatment’ would automatically mean ‘effective’. An ineffective treatment would, in other words, be a contradiction in terms.
- If it were correct, any pharmaceutical or devices company could legally market ineffective drugs or gadgets and rightly claim (or even prove) that they are effective. Any such therapy could very easily be shown to generate a placebo-effect under the right circumstances; and as long as this is the case, it would be certifiably effective.
I do sympathise with alt med enthusiasts who find this hard or even impossible to accept. They see almost every day how their placebo-therapy benefits their patients. (It seems worth remembering that not just the placebo phenomenon but several other factors are involved in such outcomes – take, for instance, the natural history of the disease and the regression towards the mean.) And they might think that my arguments are nothing but a devious attempt do away with the beneficial power of the placebo.
The truth, however, is that nobody wants to do anything of the sort; we all want to help patients as much as possible, and that does, of course, include the use of the placebo effect. In clinical practice, we usually want to maximise the placebo effect where possible. But for this goal, we do not require placebo therapies. If we administer a specifically effective therapy with compassion, we undoubtedly also generate a placebo response. In addition, our patients would benefit from the specific effects of the prescribed therapy. Both elements are essential for an optimal therapeutic response, and I don’t know any conventional healthcare professionals who do not aim at this optimal outcome.
Giving just placebos will not normally generate an optimal outcome, and therefore it cannot truly be in the interest of the patient. It is also ethically problematic because it usually entails a degree of deception of the patient. Moreover, placebo effects are unreliable and usually of short duration. Foremost, they do not normally cure a disease; they may alleviate symptoms but they almost never tackle their causes. These characteristics hardly make placebos an acceptable choice for routine clinical practice.
The bottom line is clear and simple: a drug that is not better than placebo can only be classified as being ineffective. The same applies to all non-drug therapies. Double standards are not acceptable in healthcare. And the demonstration of a placebo effect does not turn an ineffective therapy into an effective one.
I know that many alternative practitioners do not agree with this line of thought – so, let’s hear their counter-arguments.
Guest post by Frank Van der Kooy
Some serious flaws in the scientific reporting of two acupuncture clinical trials, for the treatment of infertility and allergic rhinitis, were recently published on this blog. The overly positive way in which the researchers made their mostly negative results public, was also of concern. Both these studies were published by the researcher of the year, Prof Caroline Smith, of the National Institute of Complementary Medicine (NICM), Australia. The stream of comments and discussions that followed made me think of another commonly overlooked aspect when it comes to acupuncture clinical trials. Conflict of interest! In both these studies the authors declared to have no conflicts of interest and in other studies by this author this also seems to be the case. The question can be asked: If you are a practicing acupuncturist who runs a clinical trial of acupuncture, isn’t that, by default, a serious conflict of interest? The intention of this article is not an in-depth discussion of what a conflict of interest is, but rather to compare medical doctors with acupuncturists turned researchers. Let me explain.
Some medical doctors (GPs, surgeons etc.) decide to leave their practice after practicing 10-20 years to become full time researchers (and visa versa). Universities accept these people with open arms because they bring with them a wealth of knowledge regarding the practical side of medicine and healthcare in general. They are thus seen as an asset to any medical research project including clinical trials. Can the same be said about an acupuncturist? They also bring with them years of experience and thus they should also be a major asset to any acupuncture clinical trial. But I am afraid not!
Why? Medical doctors have a multitude of tools (drugs, surgical procedures, diagnostic tools etc.) at their disposal to treat all types of medical conditions. When will their background constitute a conflict of interest? When they publish a positive clinical trial of a specific medical intervention in which they have a vested interest. e.g owning shares in the company producing the medical intervention (financial interest) or if they have been staunch supporters of this intervention during their years of practice (emotional interest). Just imagine that you have prescribed a specific intervention to hundreds of patients over a long period of time, and you swore by it, and now you have to face them with a negative clinical trial result – that will be difficult. The former is easy to declare whilst the latter might be slightly more difficult.
Doctors also tend to focus on a specific disease e.g. cancer and will perform research with the existing tools at their disposal but also try to find new tools in order to improve the risk-benefit profile of the disease treatment. Thus, for a doctor there is the possibility that they might run into a conflict of interest, but due to the multitude of medical interventions out there this is by no means a given.
What about acupuncture practitioners turned researchers? An acupuncturist only has one tool at their disposal to treat all medical conditions. I can hear them say; but we stick needles in different places and depths etc. depending on the medical condition! Yes, but the fact remains that they can only stick needles into people – and that is a single intervention. So is this by default a conflict of interest? I would argue, yes, it is like having only one drug to treat all medical conditions. If you have treated hundreds of patients for various medical conditions with acupuncture and now suddenly you publish a negative clinical trial, you will not only be red faced when you run into your former patients – who paid for your evidence based acupuncture treatment – they might even sue you for misleading them. As an acupuncturist, you cannot allow the single tool that you have to be ineffective, otherwise people might start to question acupuncture. The fact that they have to protect acupuncture means that an acupuncturist will by default have a conflict of interest – no matter what medical condition they aim to treat.
If you have been emotionally and financially invested in acupuncture as a cure-all for 10-20 years, it will be very difficult, if not impossible, to publish a negative result as an acupuncture researcher.
Another aspect is that the acupuncture fraternity is a very tight knit community, where negative results are frowned upon because of everyone’s financial and emotional interests. Surely they will expel you from this community, if you publish negative results?
So how do acupuncture researchers go about running clinical trials? An example: Professors Smith and Bensoussan, both at the NICM, are currently registered as practicing acupuncturists. This means that they can legally practice acupuncture and, because they have been active for decades, they are also well known in the acupuncture fraternity. It is unknown, whether they are still actively practicing in their own practice or part-time in someone else’s practice, or if they have a financial stake in their former or someone else’s practice. Based on the fact that they are still registered as active acupuncturists, I can conclude that they do have an emotional and/or financial interest in the positive outcome of their acupuncture clinical trials.
Because of this inherent conflict of interest, and due to current strict clinical trial regulations, which makes it quite difficult (although not impossible) to fabricate or falsify data, they go for the next best thing – which is the design of their clinical trial e.g. the A+B versus A design. But it doesn’t stop there. As soon as a clinical trial fails to give a positive result, the results will be inflated to make it sound positive.
Why? Because they must prevent themselves from cognitive dissonance, they need to protect the single tool that they have, they must keep the acupuncture fraternity happy and they have to protect themselves against potential lawsuits from former (current) patients or a decrease in patient numbers (and thus financial income). On top of that – how would the media and the public react to an acupuncture clinical trial if the lead researcher declare that they have their own acupuncture clinic? Surely these factors together amount to a conflict of interest and should be declared as such?
So what, in this context, is the main difference between a doctor and an acupuncturist? A doctor has a multitude of medical interventions. He or she might have a conflict of interest, if they work on a specific intervention in which they have a vested interest. An acupuncturist only has one intervention and therefore they have a vested interest by default – which they never seem to declare!
This post is dedicated to Mel Koppelman.
Those who followed the recent discussions about acupuncture on this blog will probably know her; she is an acupuncturist who (thinks she) knows a lot about research because she has several higher qualifications (but was unable to show us any research published by herself). Mel seems very quick in lecturing others about research methodology. Yesterday, she posted this comment in relation to my previous post on a study of aromatherapy and reflexology:
Professor Ernst, This post affirms yet again a rather poor understanding of clinical trial methodology. A pragmatic trial such as this one with a wait-list control makes no attempt to look for specific effects. You say “it is quite simply wrong to assume that this outcome is specifically related to the two treatments.” Where have specific effects been tested or assumed in this study? Your statement in no way, shape or form negates the author’s conclusions that “aromatherapy massage and reflexology are simple and effective non-pharmacologic nursing interventions.” Effectiveness is not a measure of specific effects.
I am most grateful for this comment because it highlights an issue that I had wanted to address for some time: The meanings of the two terms ‘efficacy and effectiveness’ and their differences as seen by scientists and by alternative practitioners/researchers.
Let’s start with the definitions.
I often use the excellent book of Alan Earl-Slater entitled THE HANDBOOK OF CLINICAL TRIALS AND OTHER RESEARCH. In it, EFFICACY is defined as ‘the degree to which an intervention does what it is intended to do under ideal conditions. EFFECTIVENESS is the degree to which a treatment works under real life conditions. An EFFECTIVENESS TRIAL is a trial that ‘is said to approximate reality (i. e. clinical practice). It is sometimes called a pragmatic trial’. An EFFICACY TRIAL ‘is a clinical trial that is said to take place under ideal conditions.’
In other words, an efficacy trial investigates the question, ‘can the therapy work?’, and an effectiveness trial asks, ‘does this therapy work?’ In both cases, the question relate to the therapy per se and not to the plethora of phenomena which are not directly related to it. It seems logical that, where possible, the first question would need to be addressed before the second – it does make little sense to test for effectiveness, if efficacy has not been ascertained, and effectiveness without efficacy does not seem to be possible.
In my 2007 book entitled UNDERSTANDING RESEARCH IN COMPLEMENTARY AND ALTERNATIVE MEDICINE (written especially for alternative therapists like Mel), I adopted these definitions and added: “It is conceivable that a given therapy works only under optimal conditions but not in everyday practice. For instance, in clinical practice patients may not comply with a therapy because it causes adverse effects.” I should have added perhaps that adverse effects are by no means the only reason for non-compliance, and that non-compliance is not the only reason why an efficacious treatment might not be effective.
Most scientists would agree with the above definitions. In fact, I am not aware of a debate about them in scientific circles. But they are not something alternative practitioners tend to like. Why? Because, using these strict definitions, many alternative therapies are neither of proven efficacy nor effectiveness.
What can be done about this unfortunate situation?
Simple! Let’s re-formulate the definitions of efficacy and effectiveness!
Efficacy, according to some alternative medicine proponents, refers to the therapeutic effects of the therapy per se, in other words, its specific effects. (That coincides almost with the scientific definition of this term – except, of course, it fails to tell us anything about the boundary conditions [optimal or real-life conditions].)
Effectiveness, according to the advocates of alternative therapies, refers to its specific effects plus its non-specific effects. Some researchers have even introduced the term ‘real-life effectiveness’ for this.
This is why, the authors of the study discussed in my previous post, could conclude that “aromatherapy massage and reflexology are simple… effective… interventions… to help manage pain and fatigue in patients with rheumatoid arthritis.” Based on their data, neither aromatherapy nor reflexology has been shown to be effective. They might appear to be effective because patients expected to get better, or patients in the no-treatment control group felt worse for not getting the extra care. Based on studies of this nature, giving patients £10 or a box of chocolate might also turn out to be “simple… effective… interventions… to help manage pain and fatigue in patients with rheumatoid arthritis.” Based on these definitions of efficacy and effectiveness, there are hardly any limits to bogus claims for any old quackery.
Such obfuscation suits proponents of alternative therapies fine because, using such definitions, virtually every treatment anyone might ever think of can be shown to be effective! Wishful thinking, it seems, can fulfil almost any dream, it can even turn the truth upside down.
Or can anyone name an alternative treatment that cannot even generate a placebo response when administered with empathy, sympathy and care? Compared to doing nothing, virtually every ineffective therapy might generate outcomes that make the treatment look effective. Even the anticipation of an effect alone might do the trick. How often have you had a tooth-ache, went to the dentist, and discovered sitting in the waiting room that the pain had gone? Does that mean that sitting in a waiting room is an effective treatment for dental pain?
In fact, some enthusiasts of alternative medicine could soon begin to argue that, with their new definition of ‘effectiveness’, we no longer need controlled clinical trials at all, if we want to demonstrate how effective alternative therapies truly are. We can just do observational studies without a control group, note that lots of patients get better, and ‘Bob is your uncle’!!! This is much faster, saves money, time and effort, and has the undeniable advantage of never generating a negative result.
To most outsiders, all this might seem a bit like splitting hair. However, I fear that it is far from that. In fact, it turns out to be a fairly fundamental issue in almost any discussion about the value or otherwise of alternative medicine. And, I think, it is also a matter of principle that reaches far beyond alternative medicine: if we allow various interest groups, lobbyists, sects, cults etc. to use their own definitions of fundamentally important terms, any dialogue, understanding or progress becomes almost impossible.
Alternative medicine has no shortage of research that suggests it to be effective. Almost invariably, however, one finds – when looking a bit more carefully at such investigations – that the positive conclusions are not warranted by the data. Here is an excellent, recent example:
This new study, authored by two Turkish nurses, was an RCT where the patients were randomly assigned to either an aromatherapy massage (n = 17), reflexology (n = 17) or the control group (n = 17). Aromatherapy massage was applied to both knees of subjects in group 1 for 30 minutes. Reflexology was administered to both feet of subjects in group 2 for 40 minutes during weekly home visits. The subjects of group 3, the control group, received no intervention.
Fifty-one subjects with rheumatoid arthritis were recruited from a university hospital rheumatology clinic in Turkey between July 2014 and January 2015 for this trial. Data were collected by personal information form, DAS28 index, Visual Analog Scale and Fatigue Severity Scale. Pain and fatigue scores were measured at baseline and within an hour after each intervention for 6 weeks.
Pain and fatigue scores significantly decreased in the aromatherapy massage and reflexology groups compared with the control group (p < .05). The reflexology intervention started to decrease pain and fatigue scores earlier than aromatherapy massage (week 1 vs week 2 for pain, week 1 vs week 4 for fatigue) (p < .05).
The authors concluded that aromatherapy massage and reflexology are simple and effective non-pharmacologic nursing interventions that can be used to help manage pain and fatigue in patients with rheumatoid arthritis.
I am sure that most readers have spotted the snag: the two interventions generated better outcomes than no therapy. It is quite simply wrong to assume that this outcome is specifically related to the two treatments. Both of these treatments are fairly agreeable and generate expectations, involve touch, attention and care. In my view, it is these latter factors which together have caused the better outcomes. And this is, of course, entirely unrelated to any specific effects of the two therapies.
This might well be trivial, but if such sloppy conclusions pollute the literature to the extend that they currently do in the realm of alternative medicine, it becomes important.
Acupuncture, like most other alternative therapies, is particular popular for indications that are
- associated with a high burden of suffering,
- not easily treatable with conventional therapies,
- are frequently resolved without any intervention.
Infertility or subfertility tick most of these boxes. It is therefore not surprising that acupuncturists the world over claim that acupuncture can cure infertility. But is this claim based on evidence or on wishful thinking?
The objective of this new study was to find out. Specifically, the authors wanted to provide preliminary data to explore whether women with subfertility undergoing a course of acupuncture and lifestyle modification compared with an active control of lifestyle modification alone would demonstrate improved reproductive outcomes, improved menstrual cycles, and increased fertility awareness.
In a pragmatic randomised controlled trial, with the A+B versus B design, sub/infertile women were offered an intervention of acupuncture and lifestyle modification or lifestyle modification only. There was a statistically significant increase in fertility awareness in the acupuncture group (86.4%) compared to 40% of the lifestyle only participants. Changes in menstrual regularity were not statistically significant. There was no statistical difference in the pregnancy rate with seven women achieving pregnancy during the course of the study intervention. Those receiving the acupuncture conceived within an average of 5.5 weeks compared to 10.67 weeks for the lifestyle only group.
The authors concluded that the acupuncture protocol tested influenced women who received it compared to women who used lifestyle modification alone: their fertility awareness and wellbeing increased, and those who conceived did so in half the time.
The first sentences of the authors’ discussion are, I think, revealing: The main findings were that this acupuncture intervention, compared to lifestyle only, resulted in significant increases in fertility awareness and quality of life measures in relation to wellbeing; it increased the ability of the recipients to engage in desired activities, such as exercise or rest, and it shortened the time to conception by half. The findings provide preliminary evidence that the acupuncture intervention is acceptable and is not inert and that acupuncture dose may have a significant influence on outcomes.
In my view, the main findings of this study are entirely different. Let me propose alternatives:
- In alternative medicine, if you did a lousy study, you can just call it a ‘pilot study’ and all is forgiven.
- The infamous A+B vs B design continues to be popular for those who cannot bring themselves to publishing negative findings.
- It works perfectly for subjective parameters but less convincingly for objective ones, such as pregnancy rates.
- Doing such research on infertility is good for the cash flow of acupuncturists.
- Making women aware of fertility increases (surprise, surprise!) fertility awareness.
No need to be so cynical!, some will think. After all, the results showed that women receiving the acupuncture conceived within an average of 5.5 weeks compared to 10.67 weeks for the lifestyle only group. True! But there was no statistically significant difference between these two figures. And that means, the difference was a chance finding (which has no place in an abstract) which probably has no relevance whatsoever.
Or perhaps I am wrong?
I am told to always keep an open mind!
So, let’s keep our minds open to some truly alternative explanations. How about this one: regular acupuncture increases the rate of adultery, which, in turn, decreases the time to conception.
Makes sense, doesn’t it? Has anyone a better idea?