A new study published in JAMA investigated the long-term effects of acupuncture compared with sham acupuncture and being placed in a waiting-list control group for migraine prophylaxis. The trial was a 24-week randomized clinical trial (4 weeks of treatment followed by 20 weeks of follow-up). Participants were randomly assigned to 1) true acupuncture, 2) sham acupuncture, or 3) a waiting-list control group. The trial was conducted from October 2012 to September 2014 in outpatient settings at three clinical sites in China. Participants 18 to 65 years old were enrolled with migraine without aura based on the criteria of the International Headache Society, with migraine occurring 2 to 8 times per month.
Participants in the true acupuncture and sham acupuncture groups received treatment 5 days per week for 4 weeks for a total of 20 sessions. Participants in the waiting-list group did not receive acupuncture but were informed that 20 sessions of acupuncture would be provided free of charge at the end of the trial. Participants used diaries to record migraine attacks. The primary outcome was the change in the frequency of migraine attacks from baseline to week 16. Secondary outcome measures included the migraine days, average headache severity, and medication intake every 4 weeks within 24 weeks.
A total of 249 participants 18 to 65 years old were enrolled, and 245 were included in the intention-to-treat analyses. Baseline characteristics were comparable across the 3 groups. The mean (SD) change in frequency of migraine attacks differed significantly among the 3 groups at 16 weeks after randomization; the mean (SD) frequency of attacks decreased in the true acupuncture group by 3.2 (2.1), in the sham acupuncture group by 2.1 (2.5), and the waiting-list group by 1.4 (2.5); a greater reduction was observed in the true acupuncture than in the sham acupuncture group (difference of 1.1 attacks; 95% CI, 0.4-1.9; P = .002) and in the true acupuncture vs waiting-list group (difference of 1.8 attacks; 95% CI, 1.1-2.5; P < .001). Sham acupuncture was not statistically different from the waiting-list group (difference of 0.7 attacks; 95% CI, −0.1 to 1.4; P = .07).
The authors concluded that among patients with migraine without aura, true acupuncture may be associated with long-term reduction in migraine recurrence compared with sham acupuncture or assigned to a waiting list.
Note the cautious phraseology: “… acupuncture may be associated with long-term reduction …”
The authors were, of course, well advised to be so atypically cautious:
- Comparisons to the waiting list group are meaningless for informing us about the specific effects of acupuncture, as they fail to control for placebo-effects.
- Comparisons between real and sham acupuncture must be taken with a sizable pinch of salt, as the study was not therapist-blind and the acupuncturists may easily have influenced their patients in various ways to report the desired result (the success of patient-blinding was not reported but would have gone some way to solving this problem).
- The effect size of the benefit is tiny and of doubtful clinical relevance.
My biggest concern, however, is the fact that the study originates from China, a country where virtually 100% of all acupuncture studies produce positive (or should that be ‘false-positive’?) findings and data fabrication has been reported to be rife. These facts do not inspire trustworthiness, in my view.
So, does acupuncture work for migraine? The current Cochrane review included 22 studies and its authors concluded that the available evidence suggests that adding acupuncture to symptomatic treatment of attacks reduces the frequency of headaches. Contrary to the previous findings, the updated evidence also suggests that there is an effect over sham, but this effect is small. The available trials also suggest that acupuncture may be at least similarly effective as treatment with prophylactic drugs. Acupuncture can be considered a treatment option for patients willing to undergo this treatment. As for other migraine treatments, long-term studies, more than one year in duration, are lacking.
So, maybe acupuncture is effective. Personally, I am not convinced and certainly do not think that the new JAMA study significantly strengthened the evidence.
Prof Walach has featured on this blog before, for instance here, and here. He is a psychologist by training and a vocal and prominent advocate of several bogus treatments, including homeopathy. He also is the editor in chief of the journal ‘Complementary Medicine Research’ and regularly uses this position to sing the praise of homeopathy. There is a degree of mystery about his affiliation: he informed me about 10 months ago that he has left his post at the Europa Universität Viadrina, Frankfurt/Oder (“Dass ich als “ehemaliger Professor” geführt werde liegt daran, dass ich Ende Januar aufgehört habe. Meine Stelle ist ausgelaufen und ich habe
sie nicht mehr verlängert.”). Yet all, even his recent papers still carry this address.
His latest article is entitled ‘The future of homeopathy’ is no exception. It is remarkable not just because of the mysterious affiliation but also – and mostly – because of its content. Here is my translation of a brief passage from this paper [I added some numbers in square brackets which refer to footnotes below].
START OF MY TRANSLATION
It is entirely undisputed that homeopathy with its therapeutic principles runs against the mainstream of science; and in this, Weymayr  is correct. However, to build on this fact a veritable research prohibition, such as the ‘scientability-concept’ suggests, is not just wrong from a science theoretical perspective, but… also discloses a dogmatic and unscientific stance.
If we see things soberly, homeopathy is – from a science theory point of view – an anomaly: empiric data prove that effects appear regularly and more and more frequently . This is being demonstrated with meta-analyses of placebo-controlled clinical trials. And this also shows with our own provings, which conform well with the newly developed standards as well as with the newer provings. Effects are furthermore noted with such frequency in animal and plant-based studies. Contrary to often voiced statements, there are also models which produce replicated effects – for instance the model of children with ADHD which is currently being replicated. Repeatedly high quality pilot studies emerge, such as the one by Gassmann et al., which show that unexpected effects also appear with higher potencies, documented with objective methods. Homeopathy proves itself as useful in large pragmatic trials of which we, however, have far too few. And let’s not forget: homeopathy is pragmatically useful. Even though aggravations do occur occasionally during homeopathic treatments, the claim that homeopathy is dangerous is a careless interpretation of the data. 
In what way is homeopathy an anomaly? I have already years ago argued that the signature of the data does not suggest that we are dealing with a classical local effect. This would be an effect which would conform with the usual criterion of causality and would thus be stable, regular and more and more evident with improved experimentation. It is unnecessary to repeat this argument  for the purpose of this editorial. But precisely the question of the classic causal effect is the controversy. And exactly this is the issue used by the new wave of critic of homeopathy which is openly aimed at the demise of homeopathy. This situation occurs because also the homeopaths are victims of the misapprehension that homeopathy is based on a classic causal process. But this assumption is most likely wrong, and homeopaths would be well-advised on the one side to point to the empiric evidence, and on the other side to practice theoretical chastity making clear that, for the time being, we have not a clue how homeopathy functions. This is the typical situation when a scientific anomaly occurs…
My prognosis would be: if we stop to misunderstand homeopathy as a classic causal phenomenon and instead view and research it as a non-classical phenomenon, homeopathy would have a chance and science would get richer by a new category of phenomena. This approach will prompt criticism, because it renders the world more complex rather than simpler. But this cannot be changed. Perhaps a new era of therapeutics might even emerge which does not abolish the molecular paradigm but makes it appear as one of several possibilities. 
END OF MY TRANSLATION
For those of you who can read German, here is the original text with references:
Dass die Homöopathie mit ihren therapeutischen Prinzipien dem Hauptstrom der Wissenschaft immer schon zuwiderlief, ist völlig unbestritten, und darin hat Weymayr recht. Aber auf dieser Tatsache ein regelrechtes «Forschungsverbot» aufbauen zu wollen, wie es das Szientabilitätskonzept vorsieht, das ist nicht nur wissenschaftstheoretisch absolut falsch, wie wir in einer Replik gezeigt haben , sondern offenbart auch eine dogmatische und unwissenschaftliche Einstellung.
Wenn man die Sache nüchtern sieht, ist die Homöopathie – wissenschaftstheoretisch betrachtet – eine Anomalie : Empirische Daten belegen, dass immer wieder und insgesamt häufiger als zufällig erwartet Effekte auftreten. Das zeigen Meta-Analysen placebokontrollierter klinischer Studien [4,5,6]. Und das zeigt sich sowohl in unseren eigenen Arzneimittel-Prüfungen , die im Übrigen den erst neuerdings entwickelten Standards gut entsprechen , als auch in neueren Prüfungen . Auch in Tierexperimenten [10,11,12,13] und in Pflanzenstudien [14,15,16] treten Effekte in solcher Häufigkeit auf. Entgegen oft gehörten Äußerungen gibt es durchaus auch Modelle, die replizierte Effekte ergeben – etwa das Modell homöopathischer Behandlung von Kindern mit Aufmerksamkeitsdefizit-/Hyperaktivitätssyndrom [17,18], das gerade repliziert wird . Immer wieder gibt es qualitativ hochwertige Pilotstudien, wie die unlängst publizierte von Gassmann et al. , die zeigen, dass unerwartete Effekte auch unter höheren Potenzen und dokumentiert mit objektiven Methoden zu beobachten sind. Homöopathie erweist sich in großen pragmatischen Studien, von denen es allerdings viel zu wenige gibt, als nützlich [21,22,23]. Und nicht zu vergessen: Homöopathie ist pragmatisch hilfreich [24,25,26,27]. Zwar kommt es bei homöopathischer Behandlung gelegentlich zu einer Erstverschlimmerung [28,29], aber die Behauptung, Homöopathie sei gefährlich , ist eine fahrlässige Interpretation der Daten .
Inwiefern ist die Homöopathie dann eine Anomalie? Ich habe schon vor Jahren argumentiert, dass die Signatur der Daten in der Homöopathie nicht dafür spricht, dass wir es mit einem klassischen, lokalen Effekt zu tun haben . Das wäre ein Effekt, der dem gewöhnlichen Kriterium der Kausalität entspräche und somit stabil, regelmäßig und bei immer besserer Experimentierkunst immer deutlicher hervorträte. Dieses Argument jetzt wieder aufzurollen, ist im Rahmen eines Editorials müßig. Aber genau die Frage nach einem klassisch-kausalen Effekt ist letztlich der Stein des Anstoßes. Und genau diesen Anstoß nimmt nun die neue Welle der Homöopathiekritik, die erklärtermaßen auf die Abschaffung der Homöopathie abzielt, zu ihrem Anlass. Diese Situation ergibt sich, weil auch die Homöopathen dem Selbstmissverständnis aufsitzen, Homöopathie sei ein klassisch-kausaler Prozess. Das ist höchstwahrscheinlich falsch, und die Homöopathie wäre gut beraten, einerseits auf die empirischen Befunde hinzuweisen und auf der anderen Seite theoretische Enthaltsamkeit zu üben und klarzulegen, dass wir vorläufig keinerlei Ahnung haben, wie Homöopathie funktioniert. Das ist die typische Situation, wenn eine wissenschaftliche Anomalie vorliegt…
Meine Prognose wäre: Wenn wir aufhören, die Homöopathie als klassisches Phänomen misszuverstehen, und sie stattdessen als ein mögliches nichtklassisches Phänomen betrachten und beforschen, dann hat die Homöopathie eine Chance und die Wissenschaft wird um eine neue Kategorie von Phänomenen reicher. Dieser Ansatz wird Kritik hervorrufen, denn er macht die Welt eher komplexer als einfacher. Aber das lässt sich nicht ändern. Vielleicht kann sogar eine neue Ära der Therapie beginnen, die das molekulare Paradigma nicht abschafft, aber als eine von mehreren Möglichkeiten erscheinen lässt.
Rather than commenting on this text in full detail, I simply want to provide a few explanations [they refer to the numbers in square brackets inserted by me into my translation] in order to facilitate understanding. I hope, however, that my readers will comment as much as they feel like.
1) Weymayr argued that certain fields lack plausibility to a degree that they do not merit being investigated. Here is an abstract of an article by him:
Evidence-based medicine (EbM) has proved to be very useful in healthcare; thanks to its methodology the reliability of our knowledge of the benefits and harms of interventions can be assessed. This at least applies to interventions which are based on a plausible concept for their mechanism of action and which have already achieved positive effects in experiments and simple studies. However, for interventions whose concepts contradict scientific findings EbM has proved to be unsuitable; it has not been able to prevent that they are still regarded as effective amongst wide parts of the population and medical experts. Particularly homeopathy has managed to even present itself as scientifically justified by using EbM. With the aim of highlighting the speculative character of homeopathy and other procedures and of preventing EbM from getting damaged, the concept of scientability is introduced in this article. This concept only approves of clinical studies if the intervention that is to be tested does not contradict definite scientific findings.
2) A scientific anomaly is “something which cannot be explained by currently accepted scientific theories. Sometimes the new phenomenon leads to new rules or theories, e.g., the discovery of x-rays and radiation.”
3) Even a minimal amount of critical thinking leads to the conclusion that the claims made about homeopathy in this paragraph are mostly not true or exaggerated. On this blog, there is plenty of evidence to contradict Walach on all the points he made here.
4) Walach’s argument is detailed in this article:
Among homeopaths the common idea about a working hypothesis for homeopathic effects seems to be that, during the potentization process, ‘information’ or ‘energy’ is being preserved or even enhanced in homeopathic remedies. The organism is said to be able to pick up this information, which in turn will stimulate the organism into a self-healing response. According to this view the decisive element of homeopathic therapy is the remedy which locally contains and conveys this information. I question this view for empirical and theoretical reasons. Empirical research has shown a repetitive pattern, in fundamental and clinical research alike: there are many anomalies in high-dilution research and clinical homeopathic trials which will set any observing researcher thinking. But no single paradigm has proved stable enough in order to produce repeatable results independent of the researcher. I conclude that the database is too weak and contradictory to substantiate a local interpretation of homeopathy, in which the remedy is endowed with causal-informational content irrespective of the circumstances. I propose a non-local interpretation to understand the anomalies along the lines of Jung’s notion of synchronicity and make some predictions following this analysis.
5) In a nutshell, Walach seems to be saying:
- the empirical evidence for homeopathy is strong;
- nobody understands the mechanisms by which the effects of homeopathy are brought about;
- if we all claim that homeopathy is a ‘scientific anomaly’ which operates according to Jung’s notion of synchronicity, the discrepancy between strong evidence and lack of plausible explanation disappears and everyone can be happy.
This is wrong for the following reasons, in my view:
- the evidence is not strong but negative or extremely weak;
- we understand very well that the effects of homeopathy are due to non-specific effects;
- therefore there is no need for a new paradigm;
- Jung’s notion of synchronicity is pure speculation and not applicable to therapeutics.
In summary, Prof Walach would do well to stop philosophising about homeopathy, read up about critical analysis, fine-tune his BS-detector and familiarise himself with Occam’s razor.
You may recall, we have dealt with the JCAM many times before; for instance here, here, here and here. Now they have come out with another remarkable paper. This study – no, the authors called it a ‘pilot study’ – was to compare the efficacy of Emotional Freedom Techniques (EFT) with that of Cognitive-Behavioral Therapy (CBT) in reducing adolescent anxiety. Sixty-three American high-ability students in grades 6–12, ages 10–18 years, who scored in the moderate to high ranges for anxiety on the Revised Children’s Manifest Anxiety Scale-2 (RCMAS-2) were randomly assigned to one of three groups:
- CBT (n = 21),
- EFT (n = 21),
- or waitlist control (n = 21).
EFT is an alternative therapy that incorporates acupoint stimulation. Students assigned to the CBT or EFT treatment groups received three individual sessions of the identified protocols from trained graduate counseling, psychology, or social work students enrolled at a large northeastern research university. The RCMAS-2 was used to assess preintervention and postintervention anxiety levels in participants.
EFT participants showed significant reduction in anxiety levels compared with the waitlist control group with a moderate to large effect size. CBT participants did not differ significantly from the EFT or control.
The authors concluded that EFT is an efficacious intervention to significantly reduce anxiety for high-ability adolescents.
They also state in their abstract that EFT is an evidence-based treatment for anxiety…
Are you happy with these conclusions?
Are you convinced that this trial lends itself to establish efficacy of anything?
Are you impressed with the trial design, the sample size, etc?
Are you sure that EFT is plausible, credible or evidence-based in any way?
If you look up EFT, you will find that there is a surprising amount of papers on it. Most of them have one thing in common: they were published in highly dubious journals. The field does not inspire trust or competence. The authors of the study state that EFT is an easily implemented strategy that uses such techniques as awareness building, exposure, reframing of interpretation, and systematic desensitization, while teaching the participant to self-stimulate protocol-identified acupoints (i.e., acupuncture points) by tapping. The effectiveness of acupuncture for treating anxiety has been well documented. Rather than using acupuncture needles, EFT relies on the manual stimulation of the acupoints. A recent meta-analysis indicated that interventions using acupoint stimulation had a moderate effect size (Hedge’s g = −0.66 95% CI [−0.99, −0.33]) in reducing symptoms. In EFT, the client stimulates the protocol-identified acupoints by tapping on them. Preliminary studies have suggested that tapping and other alternative ways of stimulating acupuncture points to be as effective as acupuncture needling. The EFT protocol and identified acupoints that were used in this study are the ones recommended for research purposes by the Association for Comprehensive Energy Psychology…
Wikipedia tells us that “Emotional Freedom Techniques (EFT) is a form of counseling intervention that draws on various theories of alternative medicine including acupuncture, neuro-linguistic programming, energy medicine, and Thought Field Therapy (TFT). It is best known through Gary Craig’s EFT Handbook, published in the late 1990s, and related books and workshops by a variety of teachers. EFT and similar techniques are often discussed under the umbrella term “energy psychology”. Advocates claim that the technique may be used to treat a wide variety of physical and psychological disorders, and as a simple form of self-administered therapy. The Skeptical Inquirer describes the foundations of EFT as “a hodgepodge of concepts derived from a variety of sources, [primarily] the ancient Chinese philosophy of chi, which is thought to be the ‘life force’ that flows throughout the body.” The existence of this life force is “not empirically supported”. EFT has no benefit as a therapy beyond the placebo effect or any known-effective psychological techniques that may be provided in addition to the purported “energy” technique. It is generally characterized as pseudoscience and it has not garnered significant support in clinical psychology.”
A recent systematic review of EFT concluded that “there were too few data available comparing EFT to standard-of-care treatments such as cognitive behavioral therapy, and further research is needed to establish the relative efficacy of EFT to established protocols.”
Notwithstanding these and many other verdicts on EFT, we now are asked to agree with the new study that EFT IS EFFICACIOUS.
Is this a joke?
They want us to believe this on the basis of a PILOT STUDY? Such studies are not even supposed to test efficacy! (Yet the authors of the trial state that this study was designed to meet the American Psychological Association (APA) Division 12 quality control criteria and the Consolidated Standards for Reporting Trials (CONSORT) criteria. I have to admit, they could have fooled me!)
No, it is not a joke, it is yet another nonsense from the ‘The Journal of Complementary and Alternative Medicine’ which, in my view, should henceforth be called THE JOURNAL OF ALTERNATIVE FACTS (JAF).
The British Homeopathic Association (BHA) is a registered charity founded in 1902. Their objectives are “to promote and develop the study and practice of homeopathy and to advance education and research in the theory and practice of homeopathy…” and their priority is “to ensure that homeopathy is available to all…” The BHA believes that “homeopathy should be fully integrated into the healthcare system and available as a treatment choice for everyone…”
This does not bode well, in my view. Specifically, it does not seem as though we can expect unbiased information from the BHA. Yet, from a charity we certainly do not expect a packet of outright lies – so, let’s have a look.
The BHA have a website (thank you Greg for reminding me of this source; I have long known about it and used it often for lectures when wanting to highlight the state of homeopathic thinking) where they provide “THE EVIDENCE FOR HOMEOPATHY“. I find the data presented there truly remarkable, so much so that I present a crucial section from it below:
START OF QUOTE
The widely accepted method of proving whether or not a medical intervention works is called a randomised controlled trial (RCT). One group of patients, the control group, receive placebo (a “dummy” pill) or standard treatment, and another group of patients receive the medicine being tested. The trial becomes double-blinded when neither the patient nor the practitioner knows which treatment the patient is getting. RCTs are often referred to as the “gold standard” of clinical research.
Up to the end of 2014, a total of 104 papers reporting good-quality placebo-controlled RCTs in homeopathy (on 61 different medical conditions) have been published in peer-reviewed journals. 41% of these RCTs have reported a balance of positive evidence, 5% a balance of negative evidence, and 54% have not been conclusively positive or negative. For full details of all these RCTs and more in-depth information on the research in general, visit the research section of the Faculty of Homeopathy’s website. Also, see 2-page evidence summary with full references.
END OF QUOTE
But is it true?
Let’s have a closer look at the percentage figures: according to the BHA
- 41% of all RCT are positive,
- 5% are negative,
- 54% are inconclusive.
These numbers are hugely important because they are being cited regularly across the globe as one of the most convincing bit of evidence to date in support of homeopathy. If they were true, many more RCT would be positive than negative. They would, in fact, constitute a strong indicator suggesting that homeopathic remedies are more than placebos.
One does not need to look far to find that these figures are clearly not correct! To disclose the ‘mistake’, we do not even need to study any of the 104 RCTs in question, we only need to straighten out the BHA’s ‘accounting error’ and ask: what on earth is an ‘inconclusive’ RCT?
A positive RCT obviously is one where homeopathy generated better outcomes than the placebo; similarly a negative RCT is one where the opposite was the case; in other words, where the placebo generated better outcomes than homeopathy. But what is an ‘inconclusive’ RCT? It turns out that, according to the BHA, it is one where there was no significant difference between the results obtained with placebo and homeopathy.
Yes, you understood correctly!
Outside homeopathy such RCTs are categorised as negative studies – they fail to show that homeopathy out-performs placebo and therefore confirm the null-hypothesis. An RCT is a test of the null-hypothesis (the experimental treatment is not better than the control) and can only confirm or reject this hypothesis. Certainly finding that the experimental treatment is not better than the control is not inconclusive bit a confirmation of the null-hypothesis. In other words it is a negative result.
So, let’s look at the little BHA – statistic again, and this time let’s do the accounting properly:
- 41% of all RCTs are positive,
- 59% are negative.
This means that, according to this very simplistic method, the majority of RCTs is negative. I say ‘very simplistic’ because, for a proper analysis of the trial evidence, we need to account, of course, for the quality of each trial. If the quality of the positive RCTs is, on average, less rigorous than that of the negative RCTs, the overall result would become yet more clearly negative. Most assessments of homeopathy that consider this essential factor do, in fact, confirm that this is the case.
Once all this has been analysed properly, we still have to account for factors like publication bias. Negative trials get often not published and therefore the overall picture gets easily distorted and generates a false-positive image. At the end of a sound evaluation along these lines, the result would fail to show that homeopathy differs from placebo.
Regardless of all these necessary and important considerations, the BHA website then tells us that the RCT method is problematic when it comes to testing homeopathy: “The RCT model of measuring efficacy of a drug poses some challenges for homeopathic research. In homeopathy, treatment is usually tailored to the individual. A homeopathic prescription is based not only on the symptoms of disease in the patient but also on a host of other factors that are particular to that patient, including lifestyle, emotional health, personality, eating habits and medical history. The “efficacy” of an individualised homeopathic intervention is thus a complex blend of the prescribed medicine together with the other facets of the in-depth consultation and integrated health advice provided by the practitioner; under these circumstances, the specific effect of the homeopathic medicine itself may be difficult to quantify with precision in RCTs.”
What are they trying to say here?
I am not sure.
Are they perhaps claiming that, even if an independent scientist disclosed their ‘accounting error’ and demonstrated that, in fact, the RCT evidence fails to support homeopathy, the BHA would still argue that homeopathy works?
I think so!
It looks to me that the BHA is engaged in the currently popular British past-time: THEY WANT THE CAKE AND EAT IT.
All this is more than a little disturbing, and I think it begs several questions:
- Is this type of behaviour in keeping with the charitable status of the BHA?
- Does it really ‘promote and develop the study and practice of homeopathy and to advance education and research’?
- Is it not rather unethical to mislead the public in such a gross and dishonest fashion?
- Is it not fraudulent to insist on false accounting?
I would be interested to get your views on this.
‘The use of a harmless alternative therapy is not necessarily wrong. Even if the treatment itself is just a placebo, it can help many patients. Some patients feel better with it, and it would be arrogant, high-handed and less than compassionate to reject such therapies simply because they are not supported by sufficient scientific evidence’.
How often have I heard this notion in one or another form?
I hear such words almost every day.
Arguments along these lines are difficult to counter. Any attempt to do so is likely to make us look blinkered, high-handed and less than compassionate.
Yet we all – well almost all – know that the notion is wrong. Not only that, it can be dangerous.
I will try to explain this with a concrete example of a patient employing a harmless alternative remedy with great success… until… well, you’ll see.
The patient is a married women with two kids. She is well known to her doctor because she has suffered from a range of symptoms for years, and the doctor – despite extensive tests – could never find anything really wrong with her. He knows about his patient’s significant psychological problems and has, on occasion, been tempted to prescribe tranquilizers or anti-depressants. Before he does so, however, he tells her to try Rescue Remedies@ (homeopathically diluted placebos from the range of Bach Flower Remedies). The patient is generally ‘alternatively inclined’, seems delighted with this suggestion and only too keen to give it a try.
After a couple of weeks, she reports that the Rescue Remedies (RR) are helping her. She says she can cope much better with stressful situations and has less severe and less frequent headaches or other symptoms. As she embarks on a long period of taking RR more or less regularly, she becomes convinced that the RR are highly effective and uses them whenever needed with apparent success. This goes on for months, and everyone is happy: the patient feels she has finally found a ‘medication that works’, and the doctor (who knows only too well that RR are placebos) is pleased that his patient is suffering less without needing real medication.
Then, a few months later, the patient notices that the RR are becoming less and less effective. Not only that, she also thinks that her headaches have changed and are becoming more intense. As she has been conditioned to believe that the RR are highly effective, she continues to take them. Her doctor too agrees and encourages her to carry on as before. But the pain gets worse and worse. When she develops other symptoms, her doctor initially tries to trivialise them, until they cannot be trivialised any longer. He eventually sends her to a specialist.
The patient has to wait a couple of weeks until an appointment can be arranged. The specialist orders a few tests which take a further two weeks. Finally, he diagnoses a malignant, possibly fast growing brain tumour. The patient has a poor prognosis but nevertheless agrees to an operation. Thereafter, she is paralysed on one side, needs 24-hour care, and dies 4 weeks post-operatively.
The surgeon is certain that, had he seen the patient several months earlier, the prognosis would have been incomparably better and her life could have been saved.
I suspect that most seasoned physicians have encountered stories which are not dissimilar. Fortunately they often do not end as tragically as this one. We tend to put them aside, and the next time the situation arises where a patient reports benefit from a bogus treatment we think: ‘Even if the treatment itself is just a placebo, it might help. Some patients feel better with it, and it would be arrogant, high-handed and less than compassionate to reject this ‘feel-good factor’.
I hope my story might persuade you that this notion is not necessarily correct.
If you are unable to make your patient feel better without resorting to quackery, my advice is to become a pathologist!!!
Homeopathy is never far from my mind, it seems. and this is reflected by the many posts on the subject that I continue to publish. Homeopaths get more than a little irritated by what they see as my ‘obsession’ with their beloved therapy. They thus try anything – yes, I mean anything – to undermine my credibility. One very popular way of doing this is to claim that I am sitting in the ‘ivory tower’ of academia and have no real inkling of the life on the ‘coal face’ of healthcare.
Because this is an argument that I find difficult to counter – I have indeed not routinely seen patients for over 20 years! – I was immensely pleased to read this article by an Australian GP. I take the liberty of quoting a section from it below:
START OF QUOTE
…An intricate web of lies protects the pernicious practice of homeopathy in Australia. Homeopathy is one of the most widespread disciplines of alternative medicines, with an estimated one million Australian consumers. It’s very popular. It also doesn’t work. At all. No better than a sugar pill, anyway. Turns out, vials of homeopathic remedies are chemically indistinguishable from water. Numerous international investigations and a scientific review of over 1800 studies by the National Medical Health Research Council could not be clearer: there is zero evidence that homeopathy is an effective treatment for medical conditions.
And yet the practice of homeopathy in Australia goes largely unchecked. The industry is overwhelmingly self-regulated by its own board, lending it an undue air of legitimacy. Meanwhile homeopaths advertise their ability to treat everything from autism to haemorrhoids with near impunity. Most obscenely, homeopathic therapies attract rebates under private health insurance policies that are funded by public taxes.
The justifications for allowing homeopathy are convoluted. One of the most common defences is that if the remedies truly are ineffective vials of water, then they are harmless. This is perhaps the most toxic myth about these therapies. Giving people a false cure for real symptoms is dangerous, because it delays correct diagnosis and treatment.
As a general practitioner I have observed the consequences of this in practice, seeing patients of homeopaths with conditions ranging from undiagnosed autoimmune disorders to mistreated blood pressure. These experiences mirror more notorious incidents – one West Australian coronial inquest in 2005 revealed a case where a homeopath treated rectal cancer, leading to the patient’s death. In 2009, a nine-month-old child with severe eczema was treated by her homeopath father who was later found guilty of manslaughter by denying her conventional medical care.
These are the kind of horror stories that prompt bureaucracies into symbolic action. Enter the Victorian Health Complaints Commission: a brand new watchdog unveiled last week to reign in, as Premier Daniel Andrews called them, “dodgy health providers”. The idea is that “health service providers” in Victoria, whether officially registered or not, will have to follow a general code of conduct. Included in this category are all homeopaths, and practitioners of other completely debunked practices such as reiki and iridology. The idea seems good on paper. The new code demands practitioners are truthful about their treatments, and act in the patient’s best interest. But here’s the catch – the commission will only take action on complaints lodged against individual practitioners.
This system is clearly geared towards only chasing a handful of rogue practitioners. But the problem isn’t a few rogue practitioners – it’s entirely rogue industries. The discipline of homeopathy, by its very nature, is untruthful.
Perhaps we can begin by following the lead of the United States, where the Federal Trade Council has ruled that homeopathic medicine labels must state that there is no scientific evidence backing homeopathic health claims. You have to admit, it’s bold stuff. It leaves our ACCC looking quite impotent. Real change requires the kind of courage that is in short supply.
That’s what it comes down to – cowardice. Homeopathy, along with an array of debunked complementary and alternative health disciplines, are tolerated by authorities to avoid an inconvenient confrontation. They let it slide to avoid upsetting delusional practitioners, misinformed customers, and anyone profiting from the practice. The presence of disproved medicines has insidiously embedded itself so deeply into our culture that curtailing a false cure is a huge political risk. So the status quo prevails, lest we rock the boat. Never mind that it’s heading straight down a waterfall.
END OF QUOTE
This clearly is a deeply felt and well-expressed article. It reiterates what we have regularly been trying to get across on this blog. But it is much better than anything I could ever contribute to the subject; it comes from someone who encounters the ‘pernicious practice of homeopathy’ on a regular basis and who knows about the harm it can do.
All I need to add is this: WELL DONE DOCTOR VYOM SHARMA!
Yes, it’s a new buzz-word in the realm of alternative medicine – actually, not so new; it’s been around for years and seems to attract charlatans of all imaginable types.
But what precisely is it?
The authors of this paper explain: “While the concept of wellness is still evolving, it is generally recognized that wellness is a holistic concept best represented as a continuum, with sickness, premature death, disability, and reactive approaches to health on one side and high-level wellness, enhanced health, and proactive approaches to health and well-being on the other. It is further acknowledged that wellness is multidimensional and includes physiologic, psychological, social, ecologic, and economic dimensions. These multiple dimensions make wellness difficult to accurately assess as multiple subjective and objective measures are required to account for the different dimensions. Thus, the assessment of wellness in individuals may include a variety of factors, including assessment of physiologic functioning, anthropometry, happiness, depression, anxiety, mood, sleep, health symptoms, toxic load, neurocognitive function, socioeconomic status, social connectivity, and perceived self-efficacy.”
Sounds a bit woolly?
I agree! It sounds like a gimmick for getting at the cash of the gullible public.
Is there money to be made with ‘wellness’?
For instance, with so-called ‘wellness retreats’.
Wellness retreats are all the rage. They use all sorts of bogus therapies within luxurious holiday settings for the ‘well to do’ end of our societies.
But is there any science behind this approach?
Few studies have evaluated the effect of retreat experiences, and no published studies have reported health outcomes. The objective of this new study therefore was to assess the effect of a week-long wellness-retreat experience in wellness tourists. The study was designed as a longitudinal observational study without a control group. Outcomes were assessed upon arrival and departure and 6 weeks after the retreat. The intervention was a ‘holistic, 1-week, residential, retreat experience that included many educational, therapeutic, and leisure activities and an organic, mostly plant-based diet’.
The outcome measures included anthropometric measures, urinary pesticide metabolites, a food and health symptom questionnaire, the Five Factor Wellness Inventory, the General Self Efficacy questionnaire, the Pittsburgh Insomnia Rating Scale, the Depression Anxiety Stress Scale, the Profile of Mood States, and the Cogstate cognitive function test battery.
Statistically significant improvements were seen in almost all measures after 1 week. Many of these improvements were also sustained at 6 weeks. There were statistically significant improvements in all anthropometric measures after 1 week, with reductions in abdominal girth, weight, and average systolic and diastolic pressure. Statistically significant improvements were also noted in psychological and health symptom measures. Urinary pesticide metabolites were detected in pooled urine samples before the retreat and were undetectable after the retreat.
The authors concluded that “the retreat experiences can lead to substantial improvements in multiple dimensions of health and well-being that are maintained for 6 weeks. Further research that includes objective biomarkers and economic measures in different populations is required to determine the mechanisms of these effects and assess the value and relevance of retreat experiences to clinicians and health insurers.”
IS THIS GOOD OR BAD RESEARCH?
Let’s apply my checklist from the previous post:
- published in one of the many dodgy CAM journals? YES
- single author? NO
- authors are known to be proponents of the treatment tested? YES
- author has previously published only positive studies of the therapy in question? YES
- lack of plausible rationale for the study? YES
- lack of plausible rationale for the therapy that is being tested? YES
- stated aim of the study is ‘to demonstrate the effectiveness of…’ ? NO
- stated aim ‘to establish the effectiveness AND SAFETY of…’? NO
- text full of mistakes, e. g. spelling, grammar, etc.? NO
- sample size is tiny? YES
- pilot study reporting anything other than the feasibility of a definitive trial? NO
- methods not described in sufficient detail? YES
- mismatch between aim, method, and conclusions of the study? YES
- results presented only as a graph? NO
- statistical approach inadequate or not sufficiently detailed? NO
- discussion without critical input? NO
- lack of disclosures of ethics, funding or conflicts of interest? NO
- conclusions which are not based on the results? YES
To me, this rough and ready assessment indicates that there are too many warning signals for characterising this as a rigorous study. It looks a lot like pseudo-science, I fear.
But these are at best formal markers. More important is the fact that the whole idea of measuring the effects of a ‘wellness retreat’ makes little sense, particularly in the absence of a control group. If we take a few people out of their usual, stressful work-environment and put them into a nice and luxurious holiday atmosphere where they get papered, eat better food, exercise more, sleep better and relax a lot – what would we expect after one week?
Yes, precisely! We would expect that almost anything measurable has changed for the better!
In fact, this result is so predictable that it is hardly worth documenting. Crucially, the outcome has very little to do with wellness, holism, or alternative medicine.
My conclusion: wellness not only attracts charlatans, entrepreneurs and windbags, it also is firmly steeped in pseudoscience.
One of the questions I hear frequently is ‘HOW CAN I BE SURE THIS STUDY IS SOUND’? Even though I have spent much of my professional life on this issue, I am invariably struggling to provide an answer. Firstly, because a comprehensive reply must inevitably have the size of a book, perhaps even several books. And secondly, to most lay people, the reply would be intensely boring, I am afraid.
Yet many readers of this blog evidently search for some guidance – so, let me try to provide a few indicators – indicators, not more!!! – as to what might signify a good and a poor clinical trial (other types of research would need different criteria).
INDICATORS SUGGESTIVE OF A GOOD CLINICAL TRIAL
- Author from a respected institution.
- Article published in a respected journal.
- A clear research question.
- Full description of the methods used such that an independent researcher could repeat the study.
- Randomisation of study participants into experimental and control groups.
- Use of a placebo in the control group where possible.
- Blinding of patients.
- Blinding of investigators, including clinicians administering the treatments.
- Clear definition of a primary outcome measure.
- Sufficiently large sample size demonstrated with a power calculation.
- Adequate statistical analyses.
- Clear presentation of the data such that an independent assessor can check them.
- Understandable write-up of the entire study.
- A discussion that puts the study into the context of all the important previous work in this area.
- Self-critical analysis of the study design, conduct and interpretation of the results.
- Cautious conclusion which are strictly based on the data presented.
- Full disclosure of ethics approval and informed consent,
- Full disclosure of funding sources.
- Full disclosure of conflicts of interest.
- List of references is up-to-date and includes also studies that contradict the authors’ findings.
I told you this would be boring! Not only that, but each bullet point is far too short to make real sense, and any full explanation would be even more boring to a lay person, I am sure.
What might be a little more fun is to list features of a clinical trial that might signify a poor study. So, let’s try that.
WARNIG SIGNALS INDICATING A POOR CLINICAL TRIAL
- published in one of the many dodgy CAM journals (or in a book, blog or similar),
- single author,
- authors are known to be proponents of the treatment tested,
- author has previously published only positive studies of the therapy in question (or member of my ‘ALT MED HALL OF FAME’),
- lack of plausible rationale for the study,
- lack of plausible rationale for the therapy that is being tested,
- stated aim of the study is ‘to demonstrate the effectiveness of…’ (clinical trials are for testing, not demonstrating effectiveness or efficacy),
- stated aim ‘to establish the effectiveness AND SAFETY of…’ (even large trials are usually far too small for establishing the safety of an intervention),
- text full of mistakes, e. g. spelling, grammar, etc.
- sample size is tiny,
- pilot study reporting anything other than the feasibility of a definitive trial,
- methods not described in sufficient detail,
- mismatch between aim, method, and conclusions of the study,
- results presented only as a graph (rather than figures which others can re-calculate),
- statistical approach inadequate or not sufficiently detailed,
- discussion without critical input,
- lack of disclosures of ethics, funding or conflicts of interest,
- conclusions which are not based on the results.
The problem here (as above) is that one would need to write at least an entire chapter on each point to render it comprehensible. Without further detailed explanations, the issues raised remain rather abstract or nebulous. Another problem is that both of the above lists are, of course, far from complete; they are merely an expression of my own experience in assessing clinical trials.
Despite these caveats, I hope that those readers who are not complete novices to the critical evaluation of clinical trials might be able to use my ‘warning signals’ as a form of check list that helps them to tell the chaff from the wheat.
Nobody really likes to be criticised; it can be painful. Painful but often necessary! Criticism produces progress. Criticism is therefore important. So, let’s think about criticism for a moment.
Obviously I am not talking of criticism such as ‘YOU ARE AN IDIOT’. In fact, that’s not criticism at all; it’s an insult. I am also not thinking about criticism like ‘YOUR ARGUMENT IS IDIOTIC’. I prefer to focus on criticism that is constructive, well-argued and based on evidence.
In healthcare, there is plenty of that type of criticism – luckily, I hasten to add. Its aim is to improve healthcare of the future. We need criticism to make progress. Without it, things come to a standstill or regress. This is why all the major medical journals are full of it, and many medical conferences are entirely or partly focussed on such aspects . For instance, frequently-cited papers in the BMJ, Lancet, NEJM, etc. point out that:
- much of the current medical research is unreliable,
- many therapies in current use have severe adverse effects,
- patients frequently do not get the optimal treatment in a timely fashion,
- modern medicine is too often inhumane.
The hope is that by disclosing these and many other deficits, appropriate actions can be found and taken to improve the situation and make progress. This process is hardly ever straight forward. All too often it is slow, inadequate and impeded by logistic and other obstacles. Therefore, it is crucial that constructive criticism continues to be voiced. Many clinicians, researchers and other experts have dedicated their lives to this very task.
Now, let’s look at the realm of alternative medicine.
There is certainly not less to criticise here than in conventional medicine. So, are all the journals of alternative medicine full of criticism of alternative medicine? Are there regular conferences focussed on criticism? Are alternative practitioners keen to hear about the weaknesses of their beliefs, practice, etc.?
The short answer is, no!
Yet, advocates of alternative medicine are, of course, not adverse to voicing criticism. In fact, they criticise almost non stop – at least this is the impression I get from reading their comments on this blog and from continually discussing with them since 1993.
But there is a fundamental difference: they criticise (often rightly) conventional medicine, and they criticise those (sometimes rightly) who criticise alternative medicine. When it comes to criticising their own practices, however, there is an almost deafening silence.
In my view, these differences between alternative and conventional medicine are far from trivial. In conventional medicine:
- There is a long tradition of criticism.
- Criticism is published and discussed prominently.
- Criticism is usually well-accepted.
- Criticism is often taken on board and appropriate action follows.
- Criticism thus can and often does lead to progress.
By contrast, in alternative medicine almost nothing of the above ever happens. Criticism is directed almost exclusively towards those who are outside the realm. Criticism from the inside is as good as non-existent.
The consequences of this situation are easy to see for everyone, and they can be dramatic:
- The journals of alternative medicine publish nothing that could be perceived to be negative for the practice of alternative medicine.
- Self-critical thinking has no tradition and has remained an almost alien concept.
- The very few people from the ‘inside’ who dare to criticise alternative practices are ousted and/or declared to be incompetent or worse.
- No action is taken to initiate change.
- The assumptions of alternative medicine remain unaltered for centuries.
- Progress is all but absent.
It is time that the world of alternative medicine finally understands that constructive criticism is a necessary step towards progress!
Hyperthyroidism is, so I am told, a frequent veterinary problem, particularly in elderly cats. Homeopathic treatment is sometimes used to treat this condition. One article even provided encouraging details based on 4 case-reports. All 4 cats showed resolution of clinical signs; three attained normal thyroid hormone levels. The authors concluded that homeopathic and complementary therapies avoid the potential side effects of methimazole and surgical thyroidectomy, they are less costly than radioactive iodine treatment, and they provide an option for clients who decline conventional therapies.
Yes, you guessed correctly: such a paper can only be published in the journal ‘HOMEOPATHY‘, respectable journals would not allow such conclusions based on 4 case-reports. They don’t permit inferences as to cause and effect. We have no idea what would have happened to these animals without homeopathy – perhaps they would have fared even better!
What we need is a proper controlled trial. The good news is that such a study has just been published. This double-blinded, placebo-controlled randomised trial was aimed at testing the efficacy of individualised homeopathy in the treatment of feline hyperthyroidism. Cats were randomised into two treatment arms. Either a placebo or a homeopathic treatment was given to each cat blindly.
After 21 days, the T4 levels, weight (Wt) and heart rate (HR) were compared with pre-treatment values. There were no statistically significant differences in the changes seen between the two treatment arms following placebo or homeopathic treatment, or between the means of each parameter for either treatment arm before and after placebo or homeopathic treatment. In a second phase of the study, patients in both treatment arms were given methimazole treatment for 21 days and T4, Wt and HR determined again. Subsequently, statistically significant reductions were noted in T4 (P<0.0001) and HR (P=0.02), and a statistically significant increase was observed in Wt (P=0.004).
The authors concluded that the results of this study failed to provide any evidence of the efficacy of homeopathic treatment of feline hyperthyroidism.
So, homeopathy does not work – not in humans nor in animals. This statement, backed by solid facts, proves all those wrong who cannot resist uttering the notion that HOMEOPATHY CANNOT BE A PLACEBO BECAUSE IT WORKS IN ANIMALS.
And we have seen the evidence for the correctness of this fact so often (for instance here, here, here and here) that I feel embarrassed to say it again: highly diluted homeopathic remedies are placebos. As soon as we adequately control for placebo and other non-specific effects in properly controlled studies, the alleged effects, reported in anecdotes and other uncontrolled studies, simply disappear.