As I am drafting this post, I am in a plane flying back from Finland. The in-flight meal reminded me of the fact that no food is so delicious that it cannot be spoilt by the addition of too many capers. In turn, this made me think about the paper I happened to be reading at the time, and I arrived at the following theory: no trial design is so rigorous that it cannot to be turned into something utterly nonsensical by the addition of a few amateur researchers.
The paper I was reading when this idea occurred to me was a randomised, triple-blind, placebo-controlled cross-over trial of homeopathy. Sounds rigorous and top quality? Yes, but wait!
Essentially, the authors recruited 86 volunteers who all claimed to be suffering from “mental fatigue” and treated them with Kali-Phos 6X or placebo for one week (X-potencies signify dilution steps of 1: 10, and 6X therefore means that the salt had been diluted 1: 1000000 ). Subsequently, the volunteers were crossed-over to receive the other treatment for one week.
The results failed to show that the homeopathic medication had any effect (not even homeopaths can be surprised about this!). The authors concluded that Kali-Phos was not effective but cautioned that, because of the possibility of a type-2-error, they might have missed an effect which, in truth, does exist.
In my view, this article provides an almost classic example of how time, money and other resources can be wasted in a pretence of conducting reasonable research. As we all know, clinical trials usually are for testing hypotheses. But what is the hypothesis tested here?
According to the authors, the aim was to “assess the effectiveness of Kali-Phos 6X for attention problems associated with mental fatigue”. In other words, their hyposesis was that this remedy is effective for treating the symptom of mental fatigue. This notion, I would claim, is not a scientific hypothesis, it is a foolish conjecture!
Arguably any hypothesis about the effectiveness of a highly diluted homeopathic remedy is mere wishful thinking. But, if there were at least some promissing data, some might conclude that a trial was justified. By way of justification for the RCT in question, the authors inform us that one previous trial had suggested an effect; however, this study did not employ just Kali-Phos but a combined homeopathic preparation which contained Kalium-Phos as one of several components. Thus the authors’ “hypothesis” does not even amount to a hunch, not even to a slight incling! To me, it is less than a shot in the dark fired by blind optimists – nobody should be surprised that the bullet failed to hit anything.
It could even be that the investigators themselves dimly realised that something is amiss with the basis of their study; this might be the reason why they called it an “exploratory trial”. But an exploratory study is one whithout a hypothesis, and the trial in question does have a hyposis of sorts – only that it is rubbish. And what exactly did the authos meant to explore anyway?
That self-reported mental fatigue in healthy volunteers is a condition that can be mediatised such that it merits treatment?
That the test they used for quantifying its severity is adequate?
That a homeopathic remedy with virtually no active ingredient generates outcomes which are different from placebo?
That Hahnemann’s teaching of homeopathy was nonsense and can thus be discarded (he would have sharply condemned the approach of treating all volunteers with the same remedy, as it contradicts many of his concepts)?
That funding bodies can be fooled to pay for even the most ridiculous trial?
That ethics-committees might pass applications which are pure nonsense and which are thus unethical?
A scientific hypothesis should be more than a vague hunch; at its simplest, it aims to explain an observation or phenomenon, and it ought to have certain features which many alt med researchers seem to have never heard of. If they test nonsense, the result can only be nonsense.
The issue of conducting research that does not make much sense is far from trivial, particularly as so much (I would say most) of alt med research is of such or even worst calibre (if you do not believe me, please go on Medline and see for yourself how many of the recent articles in the category “complementary alternative medicine” truly contribute to knowledge worth knowing). It would be easy therefore to cite more hypothesis-free trials of homeopathy.
One recent example from Germany will have to suffice: in this trial, the only justification for conducting a full-blown RCT was that the manufacturer of the remedy allegedly knew of a few unpublished case-reports which suggested the treatment to work – and, of course, the results of the RCT eventually showed that it didn’t. Anyone with a background in science might have predicied that outcome – which is why such trials are so deplorably wastefull.
Research-funds are increasingly scarce, and they must not be spent on nonsensical projects! The money and time should be invested more fruitfully elsewhere. Participants of clinical trials give their cooperation willingly; but if they learn that their efforts have been wasted unnecessarily, they might think twice next time they are asked. Thus nonsensical research may have knock-on effects with far-reaching consequences.
Being a researcher is at least as serious a profession as most other occupations; perhaps we should stop allowing total amateurs wasting money while playing at being professioal. If someone driving a car does something seriously wrong, we take away his licence; why is there not a similar mechanism for inadequate researchers, funders, ethics-committees which prevents them doing further damage?
At the very minimum, we should critically evaluate the hypothesis that the applicants for research-funds propose to test. Had someone done this properly in relatiom to the two above-named studies, we would have saved about £150,000 per trial (my estimate). But as it stands, the authors will probably claim that they have produced fascinating findings which urgently need further investigation – and we (normally you and I) will have to spend three times the above-named amount (again, my estimate) to finance a “definitive” trial. Nonsense, I am afraid, tends to beget more nonsense.
Since homeopathy was invented by Samuel Hahnemann about 200 years ago, a steadily growing group of critics have raised their voices more and more loudly. Usually they come from doctors or scientists and only rarely from the legal profession.
Yet, there are exceptions: an Australian barrister and professor of law has published an analysis of “a series of criminal, civil, disciplinary and coronial decisions from difference countries in relation to homeopathic medicine where outcomes have been tragic”. He concludes that “there is an urgent need for reflection and response within the health sector generally, consumer protection authorities, and legal policy-makers about the steps that should be taken to provide community protection from dangerous homeopathic practice”.
He also questions whether homeopathy can ever be registered alongside other health care professionals:
“Until such time as homoeopathy can scientifically justify its fundamental tenets,… it cannot be said that its claims for therapeutic efficacy can be justifiable. This leaves the profession not just exposed to criticisms,… but potentially open to consumer protection actions directed toward whether its representations are false, misleading and deceptive, to civil litigation when its promises have not been fulfilled, and especially when persons have died, and to criminal actions in respect of the financial advantage that is obtained by its practitioners from their representations.
The distressing cases referred to here which led to avoidable deaths and the multiple accusations leveled against homoeopathy require of the profession at least a formal repudiation of the practitioners concerned… In addition, they demand an unequivocal response that homoeopathy will discipline its own in a robust and open way. If the profession is to acquire any scientific credibility, which is difficult to conceive of, the deaths to which homoeopathy has contributed…also require that homoeopathy actively generate a defensible research basis that justifies its claims to efficacy of outcome for its patients. It is only then that the claims of the medical establishment that homoeopathy is a dangerous and too often a lethal form of quackery will be able to be contested rationally. In the meantime, it is timely to consider further the status that homoeopathy has within the general and health care communities and whether that status can be scientifically, ethically or legally justified”.
I believe this legal view to be highly significant. The persistent criticism from skeptics, concerned scientists and doctors has rarely been translated into decisions about health care provision. Homeopaths tended to respond to our criticism by producing anecdotes, unconvincing or cherry-picked data or by producing outright lies, for instance in relation to the “Swiss government’s report” on homeopathy.
In this context, it is worth noting that, in some countries, homeopaths who have no medical qualifications have been accused to practice medicine without a licence. The case of Dana Ullman in the US is probably the most spectecular such incident; this is how one pro-homeopathy site describes it:
Dana is perhaps the person who has done the most for homeopathy since his court case in that he pursues the evangelism of homeopathy through the NCH and his mail order company… He prescribed homeopathic medicine and was arrested for practicing medicine without a license. But he won an important settlement in 1977 in the Oakland Municipal Court in which the court allowed his practice under two stipulations:
- that he did not diagnose or treat disease and that he refers to medical doctors for the diagnosis and treatment of disease;
- that he makes contracts with his patients that clearly define his role as a non-medical homeopathic practitioner and the patient’s role in seeking his care.
But such cases are not the only occasions for lawyers to look at homeopathy. Recently there has been a class action against the Boiron, the world’s largest manufacturer of homeopathic preparations. It was alleged that Boiron made bogus claims for one of its remedies, and there was a settlement worth millions of dollars. Similar cases are likely to follow, e.g.:
- Nelsons Homeopathy (Rescue Remedy, Bach Original Flower Remedies, Pure & Clear, Arnileve, H+Care)
- CVS Homeopathic Products (Flu Relief, Cold Relief, Cold Remedy, Ear Pain Relief)
- Nature’s Innovation (Naturasil Skin Tags, Bed Bug Patrol, Naturasil Scabies)
- Boericke & Tafel Cold/Flu
- Homeolab USA (Kids Relief Cough & Cold)
In June 2003, a British High Court Judge ordered two mothers to ensure that their daughters are appropriately vaccinated. The ruling concerned two separate cases brought by fathers who wanted their daughters immunized despite opposition by the girls’ unwed mothers
The fact that, in the UK and other countries, homeopathic placebos are still being sold as “vaccines” for the prevention of serious, life-threatening infections is, in my view nothing short of a scandal. The fact that a leading figure at Ainsworth actively misleads the public about these products is an outrage. It is high time therefore that the legal profession looks seriously at the full range of issues related to homeopathy with a view of stopping the dangerous nonsense.
Vaccinations are unquestionably amongst the biggest achievements in the history of medicine. They have prevented billions of diseases and saved millions of lives. Despite all this, there has been an irritatingly vocal movement protesting against immunizations and thus jeopardising the progress made. Kata summarized the notions and tactics of these activists and identified the following ‘common anti-vaccination tropes‘ from searching relevant sites on the internet:
1 I am not anti-vaccine, I am pro-safe vaccine.
2 Vaccines are toxic.
3 Vaccines should be 100% safe.
4 You cannot prove that vaccines are safe.
5 Vaccines did not save us.
6 Vaccines are not natural.
7 I am an expert in my own child.
8 Galileo was persecuted too.
9 Science has been wrong before.
10 So many people simply cannot be wrong.
11 You must be in the pocket of BIG PHARMA.
12 I do not believe that the problems after vaccination occur coincidentally.
And what has this to do with alternative medicine, you may well ask?
In my experience, many of the arguments resonate with those of alternative medicine enthusiasts. Moreover, there is a mountain of evidence to show that many practitioners of alternative medicine are an established and important part of the anti-vax movement; in particular, homeopaths, chiropractors, naturopaths and practitioners of anthroposophic medicine are implicated.
The literature on this topic is vast, so I am spoilt for choice in providing an example. The one that I have selected is by Kate Birch, a mother who apparently found homeopathy so effective for her children that she decided to become a homeopath. Her book “Vaccine Free. Prevention & treatment of infectious contagious disease with homeopathy” provides details about the “homeopathic prevention and treatment” of the following diseases:
Rocky Mountain spotted fever
Herpes simplex type 1 and 2
Hepatitis A, B, and C
While copying this list from her book, I became so angry that I was about to write something that I might later regret. It is therefore better to end this post abruptly. I leave it to my readers to comment.
Boiron, the world’s largest manufacturer of homeopathic products, has recently been in the headlines more often and less favourably than their PR-team may have hoped for. Now they have added to this attention by publishing a large and seemingly impressive multi-national study of homeopathy.
Its objective was “to evaluate the effectiveness of homeopathic medicine for the prevention and treatment of migraine in children”. For this purpose, the researchers recruited 59 homeopaths from 12 countries who included into the study a total of 168 children with “definite or probable” migraine. The homeopaths had complete freedom to individualise their treatments according to the distinct characteristics of their patients.
The primary study-endpoints were the frequency, severity and duration of migraine attacks during 3 months of homeopathic treatment compared to the 3 months prior to that period. The secondary outcome measure was the amount of days off school. The results are fairly clear-cut and demonstrated that all of these variables improved in the period of homeopathic care.
This study is remarkable but possibly not in the way Boiron intended it to be. The first thing to notice is that each homeopath in this study barely treated 3 patients. I wonder why anyone would go to the trouble of setting up a multi-national trial with dozens of homeopaths from around the globe when, in the end, the total sample size is not higher than that achievable in one single well-organised, one-centre study. A multitude of countries, cultures and homeopaths is only an asset for a study, if justified through the recruitment of a large patient sample; otherwise, it is just an unwelcome source for confounding and bias.
But the main concern I have with this study lies elsewhere. Its stated objective was “…to evaluate the effectiveness of homeopathic medicines…” This aim cannot possibly be tackled with a study of this nature. As it stands, this study merely investigated what happens in 3 months while children receive 3 months of homeopathic care. The observed findings are not necessarily due to the homeopathic medicines; they might be due to the passage of time, the tender loving care received by their homeopaths, the expectation of the homeopaths and/or the parents, a regression towards the mean, the natural history of the (in some cases only “probable”) migraine, any concomitant treatments administered during the 3 months, a change in life-style, a placebo-effect, a Hawthorne-effect, or the many other factors that I have not thought of.
To put the result of the Boiron-researchers into the right context, we should perhaps remember that even the most outspoken promoters of homeopathy on the planet concluded from an evaluation of the evidence that homeopathy is ineffective as a treatment of migraine. Therefore it seems surprising to publish the opposite result on the basis of such flimsy evidence made to look impressive by its multi-national nature.
I have been accused of going out of my way to comment on bogus evidence in the realm of homeopathy. If this claim were true, I would not be able to do much else. Debunking flawed homeopathy studies is not what I aim for or spend my time on. Yet this study, I thought, does deserve a brief comment.
Why? Because it has exemplary flaws, because it reflects on homeopathy as a whole as well as on the journal it was published in (the top publication in this field), because it is Boiron-authored, because it produced an obviously misleading result, because it could lead many migraine-sufferers up the garden path and – let’s be honest – because Dan Ullman will start foaming from the mouth again, thus proving to the world that homeopathy is ineffective against acute anger and anguish.
Joking apart, the Boiron-authors conclude that “the results of this study demonstrate the interest of homeopathic medicines for this prevention and treatment of migraine attacks in children”. This is an utterly bizarre statement, as it does not follow from the study’s data at all.
But what can possibly be concluded from this article that is relevant to anyone? I did think hard about this question, and here is my considered answer: nothing (other than perhaps the suspicion that homeopathy-research is in a dire state).
Since it was first published, the “Swiss government report” on homeopathy has been celebrated as the most convincing proof so far that homeopathy works. On the back of this news, all sorts of strange stories have emerged. Their aim seems to be that consumers become convinced that homeopathy is based on compelling evidence.
Readers of this blog might therefore benefit from a brief and critical evaluation of this “evidence” in support of homeopathy. Recently, not one, two, three but four independent critiques of this document have become available.
Collectively, these articles [only one of which is mine] suggest that the “Swiss report” is hardly worth the paper it was written on; one of the critiques published in the Swiss Medical Weekly even stated that it amounted to “research misconduct”! Compared to such outspoken language, my own paper concluded much more conservatively: “this report [is] methodologically flawed, inaccurate and biased”.
So what is wrong with it? Why is this document not an accurate summary of the existing evidence? I said this would be a brief post, so I will only mention some of the most striking flaws.
The report is not, as often claimed, a product by the Swiss government; in fact, it was produced by 13 authors who have no connection to any government and who are known proponents of homeopathy. For some unimaginable reason, they decided to invent their very own criteria for what constitutes evidence. For instance, they included case-reports and case-series, re-defined what is meant by effectiveness, were highly selective in choosing the articles they happened to like [presumably because of the direction of the result] while omitting lots of data that did not seem to confirm their prior belief, and assessed only a very narrow range of indications.
The report quotes several of my own reviews of homeopathy but, intriguingly, it omitted others for no conceivable reason. I was baffled to realise that the authors reported my conclusions differently from the original published text in my articles. If this had occurred once or twice, it might have been a forgivable error – but this happened in 10 of 22 instances.
Negative conclusions in my original reviews were thus repeatedly turned into positive verdicts, and evidence against homeopathy suddenly appeared to support it. This is, of course, a serious problem: if someone is too busy to look up my original articles, she is very unlikely to notice this extraordinary attempt to cheat.
To me, this approach seems similar to that of an accountant who produces a balance sheet where debts appear as credits. It is a simple yet dishonest way to generate a positive result where there is none!
The final straw for me came when I realised that the authors of this dubious report had declared that they were free of conflicts of interest. This notion is demonstrably wrong; several of them earn their living through homeopathy!
Knowing all this, sceptics might take any future praise of this “Swiss government report” with more than just a pinch of salt. Once we are aware of the full, embarrassing details, it is not difficult to understand how the final verdict turned out to be in favour of homeopathy: if we convert much of the negative data on any subject into positive evidence, any rubbish will come out smelling of roses – even homeopathy.