Guest post by Björn Leifsson

I stumbled upon a web-article (in Icelandic) in which a local homeopath expresses her concerns over a public statement made by the national medical society suggesting that mention of homeopathic “drugs” should be removed from the legislation because homeopathy was an outdated approach and belief that is “neither based on scientific knowledge nor research”. The indignant homeopath referred to a YouTube video (really!) in support of her claim that many studies exist about the efficacy of homeopathy, as evidenced in the video.

Curiosity led me to browse through it. Towards the end of the video, Rachel Richards, a homeopath described, a seemingly remarkable study on hundreds of piglets that showed homeopathic treatment to significantly reduce infectious diarrhoea in newborn piglets. This, she claimed, is proof that homeopathy works through more than the placebo effect because placebo does not work on animals, right?

Here is the abstract of the paper:

Background: The use of antibiotics in the livestock sector is increasing to such an extent that it threatens negative consequences for human health, animal health and the environment. Homeopathy might be an alternative to antibiotics. It has therefore been tested in a randomised placebo-controlled trial to prevent Escherichia coli diarrhoea in neonatal piglets.

Method: On a commercial pig farm 52 sows of different parities, in their last month of gestation, were treated twice a week with either the homeopathic agent Coli 30K or placebo. The 525 piglets born from these sows were scored for occurrence and duration of diarrhoea.

Results: Piglets of the homeopathic treated group had significantly less E. coli diarrhoea than piglets in the placebo group (P<.0001). Especially piglets from first parity sows gave a good response to treatment with Coli 30K. The diarrhoea seemed to be less severe in the homeopathically treated litters, there was less transmission and duration appeared shorter.

This is a badly written paper. The description of methods and material is unclear and confusing. But I am not going to delve into the curious design and reporting details, such as their vague and conflicting description of the blinding or the rather peculiar description of the treatment procedure. I find the statistical design and analysis incriminating enough.

The study was conducted in four groups (batches). The researchers treated in total 52 sows; 26 were randomly allocated to an arm treated by spraying either dissolved “Coli, 30C” nosode sugar tablets. Another 26 sows were sprayed with the same liquid but without the homeopathic substance. They describe how they sprayed the stuff in[sic] the sow’s vulva’s, twice a week, during four weeks before expected delivery. How they sprayed “in” the vulva of the sows escapes my understanding. Perhaps they just misspelled “in” for “on”? Anyway, the authors curiously postulate that the “homeopathic substance” is absorbed from the vulva.

Ah, I forget… I did not plan to delve into the minor oddities of this eccentric opus.

So, 26 sows were “treated” with the good stuff and 26 with the fake. So far so good.

But this only produces a very small study, nothing more than a pilot – really. Not so good.

Instead of carrying on with this inexpensive study to procure proper power for the analysis, the authors counted the piglets! Approximately ten piglets were produced per sow so the total number of observed piglets became 525! That can certainly seem an impressive number of research subjects. Counting the piglets and not the sows, increased the study material tenfold, making it appear to be well powered study.

This is cheating in my opinion, nothing less, nothing more.

But why not do it this way?

In the week after the piglets were delivered, an observer, blinded to the treatment allocation, regularly noted which piglets got diarrhoea typical for enteropathogenic E. coli infection.

The authors made a big deal about confounding factors such as parity (how many prior litters the sow has produced) and its purported effect on piglet infection risk. They use an elaborate statistical program to perform an advanced (and wrong) analysis called “GLM – Generalised Linear Modelling”, and entered corrections for parity, season and treatment group (batch). But they forgot (omitted?) to correct for the obvious common risk factors to piglets in a litter.

On average ten piglets in each litter shared at least two litter-dependent risk factors, i.e. a sow (mother) and a pen. Thus the risk exposure is not independent between piglets, only between litters/sows.

Sow and pen dependent factors are likely to affect the number of infected piglets in affected litters. If a sow or pen is colonised, a number if not all piglets in that litter may be affected, by a common cause, thereby erroneously multiplying the effect of the studied endpoint. Possible genetic paternal and/or maternal factors may also be at play to affect the vulnerability of the litter as a group rather than individually.

In short, the authors did not have 525 study subjects, only 50.

But they might have tried to make do with this. There was actually an apparent difference between the groups, favouring the nosode arm as shown in the table. To an untrained reader (which most people are) this could easily be interpreted as proof  that the stuff really worked. One might see only the double number of affected litters in the placebo group? But this is far from relevant because the study is severely under-powered. To be able to call this study significant they would at the very least have needed to double this number. You would then still need to individually confirm the results in other separate studies, not the least for an extremely unlikely drug candidate as sugar dissolved in water.

The authors must have somehow realised that their study is abjectly under-powered and therefore resorted to using this veritable trick of mirrors and theatrical smoke to make it look like this was a trustworthy study of over 500 subjects.

Table showing the relative ratios of infected litters.

If we do a statistical test despite it being pointless, the P-value (Chi square) is 0.02176. Any P-value that falls below the 0.05 limit may seem important to the novice reader and probably be super exciting to a believer in homeopathy. But in this very small study such a result suggests absolutely nothing. It may be due to a multitude of factors other than the tested treatment, most likely pure chance perhaps aided by some flaws in the exclusion of bias.

And then there is the big problem with interpreting P-values.

My assessment is that, in compliance with Hanlon’s razor, this charade of scientific method is most likely due to a combination of incompetence and religiously wishful thinking rather than deliberate fraud. It being published in Homeopathy says much about that paper’s lacking editorial qualities.