systematic review

Yoga, it is often claimed, might be a unique method for disease prevention. One website, for instance, states that numerous studies show how yoga can help prevent these diseases: Heart disease, Alzheimer’s, Osteoporosis and Type II Diabetes. 

Cardiovascular diseases (CVD) are responsible for more deaths than any other disease category. Preventing CVD is therefore of prime importance. But are the claims made for yoga really true? What does the reliable evidence tell us?

The aim of our systematic review was to determine the effects of yoga on the primary prevention of CVD. Extensive literature searches were performed to identify all RCTs lasting at least three months, involving healthy adults or people at high risk of CVD. Trials examined any type of yoga and the comparison groups received no intervention or minimal interventions. Outcomes of interest were clinical CVD events and major CVD risk factors. Trials that involved multifactorial lifestyle interventions or weight loss programmes were excluded.

We identified 11 RCTs with a total of just 800 participants. Style and duration of yoga differed between trials. About half of all the trial participants were at high risk of CVD. Most of the studies were at risk of performance bias, with inadequate details reported in many of them to judge the risk of selection bias. None of the studies reported cardiovascular mortality, all-cause mortality or non-fatal events, and most studies were small and short-term.

Yoga was found to produce an average reduction in diastolic blood pressure of 2.90 mmHg. The effect that was small but stable on sensitivity analysis. Triglycerides (-0.27 mmol/l) and high-density lipoprotein (HDL) cholesterol (0.08 mmol/l) were also positively affected. However, these findings were based on small, short-term studies at unclear or high risk of bias. There was no clear evidence of an effect on low-density lipoprotein (LDL) cholesterol. Adverse events, occurrence of type 2 diabetes and costs were not reported in any of the included studies. Quality of life was measured in three trials but the results were inconclusive.

Our conclusion: The limited evidence comes from small, short-term, low-quality studies. There is some evidence that yoga has favourable effects on diastolic blood pressure, HDL cholesterol and triglycerides, and uncertain effects on LDL cholesterol. These results should be considered as exploratory and interpreted with caution.

This systematic review thus offers both good and bad news. The good news is that yoga seems to hold some promise in the prevention of CVD. The bad news, however, is diverse:

  • We cannot be sure what type of yoga is best; yoga can entail anything from regular exercise, to breathing techniques, to a complete and comprehensive change of life style.
  • The effect sizes are far from remarkable.
  • The quality of the research tends to be poor.
  • Once again, we have to note that, by not reporting on adverse effects, alt med researchers are violating fundamental research ethics.

Many systematic reviews conclude by stating that more and better research is required – in the case of yoga, this platitude might actually be true.

The question whether infant colic can be effectively treated with manipulative therapies might seem rather trivial – after all, this is a benign condition which the infant quickly grows out of. However, the issue becomes a little more tricky, if we consider that it was one of the 6 paediatric illnesses which were at the centre of the famous libel case of the BCA against my friend and co-author Simon Singh. At the time, Simon had claimed that there was ‘not a jot of evidence’ for claiming that chiropractic was an effective treatment of infant colic, and my systematic review of the evidence strongly supported his statement. The BCA eventually lost their libel case and with it the reputation of chiropractic. Now a new article on this intriguing topic has become available; do we have to reverse our judgements?

The aim of this new systematic review was to evaluate the efficacy or effectiveness of manipulative therapies for infantile colic. Six RCTs of chiropractic, osteopathy or cranial osteopathy alone or in conjunction with other interventions were included with a total of 325 infants. Of the 6 included studies, 5 were “suggestive of a beneficial effect” and one found no evidence of benefit. Combining all the RCTs suggested that manipulative therapies had a significant effect. The average crying time was reduced by an average of 72 minutes per day. This effect was sustained for studies with a low risk of selection bias and attrition bias. When analysing only those studies with a low risk of performance bias (i.e. parental blinding) the improvement in daily crying hours was no longer statistically significant.

The quality of the studies was variable. There was a generally low risk of selection bias but a high risk of performance bias. Only one of the studies recorded adverse events and none were encountered.

From these data, the authors drew the following conclusion: Parents of infants receiving manipulative therapies reported fewer hours crying per day than parents whose infants did not and this difference was statistically significant. Most studies had a high risk of performance bias due to the fact that the assessors (parents) were not blind to who had received the intervention. When combining only those trials with a low risk of such performance bias the results did not reach statistical significance.

Does that mean that chiropractic does work for infant colic? No, it does not!

The first thing to point out is that the new systematic review included not just RCTs of chiropractic but also osteopathy and cranio-sacral therapy.

The second important issue is that the effects disappear, once performance bias is being accounted for which clearly shows that the result is false positive.

The third relevant fact is that the majority of the RCTs were of poor quality. The methodologically best studies were negative.

And the fourth thing to note is that only one study mentioned adverse effects, which means that the other 5 trials were in breach of one of rather elementary research ethics.

What makes all of this even more fascinating is the fact that the senior author of the new publication, George Lewith, is the very expert who advised the BCA in their libel case against Simon Singh. He seems so fond of his work that he even decided to re-publish it using even more misleading language than before. It is, of course, far from me to suggest that his review was an attempt to white-wash the issue of chiropractic ‘bogus’ claims. However, based on the available evidence, I would have formulated conclusions which are more than just a little different from his; something like this perhaps:

The current best evidence suggests that the small effects that emerge when we pool the data from mostly unreliable studies are due to bias and therefore not real. This systematic review therefore fails to show that manipulative therapies are effective. It furthermore points to a serious breach of research ethics by the majority of researchers in this field.

Imagine an area of therapeutics where 100% of all findings of hypothesis-testing research is positive, i.e. comes to the conclusion that the treatment in question is effective. Theoretically, this could mean that the therapy is a miracle cure which is useful for every single condition in every single setting. But sadly, there are no miracle cures. Therefore something must be badly and worryingly amiss with the research in an area that generates 100% positive results.

Acupuncture is such an area; we and others have shown that Chinese trials of acupuncture hardly ever produce a negative finding. In other words, one does not need to read the paper, one already knows that it is positive – even more extreme: one does not need to conduct the study, one already knows the result before the research has started. But you might not believe my research nor that of others. We might be chauvinist bastards who want to discredit Chinese science. In this case, you might perhaps believe Chinese researchers.

In this systematic review, all randomized controlled trials (RCTs) of acupuncture published in Chinese journals were identified by a team of Chinese scientists. A total of 840 RCTs were found, including 727 RCTs comparing acupuncture with conventional treatment, 51 RCTs with no treatment controls, and 62 RCTs with sham-acupuncture controls. Among theses 840 RCTs, 838 studies (99.8%) reported positive results from primary outcomes and two trials (0.2%) reported negative results. The percentages of RCTs concealment of the information on withdraws or sample size calculations were 43.7%, 5.9%, 4.9%, 9.9%, and 1.7% respectively.

The authors concluded that publication bias might be major issue in RCTs on acupuncture published in Chinese journals reported, which is related to high risk of bias. We suggest that all trials should be prospectively registered in international trial registry in future.

I applaud the authors’ courageous efforts to conduct this analysis, but I do not agree with their conclusion. The question why all Chinese acupuncture trials are positive has puzzled me since many years, and I have quizzed numerous Chinese colleagues why this might be so. The answer I received was uniformly that it would be very offensive for Chinese researchers to conceive a study that does not confirm the views held by their peers. In other words, acupuncture research in China is conducted to confirm the prior assumption that this treatment is effective. It seems obvious that this is an abuse of science which must cause confusion.

Whatever the reasons for the phenomenon, and we can only speculate about them, the fact has been independently confirmed several times and is now quite undeniable: acupuncture trials from China – and these constitute the majority of the evidence-base in this area – cannot be trusted. The only way to adequately deal with this problem that I can think of is to discard them outright.

The mechanisms thorough which spinal manipulative therapy (SMT) exerts its alleged clinical effects are not well established. A new study investigated the effects of subject expectation on clinical outcomes.

Sixty healthy subjects underwent quantitative sensory testing to their legs and low backs. They were randomly assigned to receive a positive, negative, or neutral expectation instructional set regarding the effects of a spe cific SMT technique on pain perception. Following the instructional set, all subjects received SMT and underwent repeat sensory tests.

No inter-group differences in pain response were present in the lower extremity following SMT. However, a main effect for hypoalgesia was present. A significant interaction was present between change in pain perception and group assignment in the low back with participants receiving a negative expectation instructional set demonstrating significant hyperalgesia.

The authors concluded that this study provides preliminary evidence for the influence of a non- specific effect (expectation) on the hypoalgesia associated with a single session of SMT in normal subjects. We replicated our previous findings of hypoalgesia in the lower extremity associated with SMT to the low back. Additionally, the resultant hypoalgesia in the lower extremity was independent of an expectation instructional set directed at the low back. Conversely, participants receiving a negative expectation instructional set demonstrated hyperalgesia in the low back following SMT which was not observed in those receiving a positive or neutral instructional set.

More than 10 years ago, we addressed a similar issue by conducting a systematic review of all sham-controlled trials of SMT. Specifically, we wanted to summarize the evidence from sham-controlled clinical trials of SMT. Eight studies fulfilled our inclusion/exclusion criteria. Three trials (two on back pain and one on enuresis) were judged to be burdened with serious methodological flaws. The results of the three most rigorous studies (two on asthma and one on primary dysmenorrhea) did not suggest that SMT leads to therapeutic responses which differ from an inactive sham-treatment. We concluded that sham-controlled trials of SMT are sparse but feasible. The most rigorous of these studies suggest that SMT is not associated with clinically relevant specific therapeutic effects.

Taken together, these two articles provide intriguing evidence to suggest that SMT is little more than a theatrical placebo. Given the facts that SMT is neither cheap nor devoid of risks, the onus is now on those who promote SMT, e.g. chiropractors, osteopaths and physiotherapists, to show that this is not true.

In China (and increasingly elsewhere too), the gentle, meditative exercise of tai chi is being promoted and used for disease prevention, particularly for the prevention of cardiovascular disease (CVD). But are these exercises effective? We carried out a Cochrane review to find out.

We searched both English language and Asian electronic databases as well as trial registers and reference lists for relevant studies. No language restrictions were applied. We considered randomised clinical trials (RCTs) of tai chi lasting at least three months and involving healthy adults or adults at high risk of CVD. The comparison groups received no or only minimal interventions. Our outcome measures were CVD clinical events and CVD risk factors. We excluded trials involving multifactorial lifestyle interventions or focusing on weight loss. Two reviewers independently selected trials for inclusion, abstracted the data and assessed the risk of bias of each included study.

We identified 13 trials with a total of 1520 participants and three on-going studies. All of them had at least one domain with unclear risk of bias, and some were at high risk of bias. Duration and style of tai chi differed between trials. Seven studies recruited 903 healthy participants, the other studies recruited people with hypertension, elderly people at high risk of falling, and people with ‘liver or kidney yin deficiency syndromes’.

No studies reported on cardiovascular mortality, all-cause mortality or non-fatal events as most studies were short-term. There was also considerable heterogeneity between studies, which meant that it was not possible to combine studies statistically for cardiovascular risk. Nine trials measured systolic blood pressure (SBP), and 6 of them found reductions in SBP. Two trials found no clear evidence of a difference, and one trial found an increase in SBP with tai chi. A similar pattern was seen for diastolic blood pressure (DBP): three trials found a reduction in DBP, while three found no clear evidence of a difference.

Three trials reported lipid levels and two found reductions in total cholesterol, LDL-C and triglycerides, while the third study found no clear evidence of a difference between groups on lipid levels. Quality of life was measured in only one trial: tai chi improved quality of life at three months. None of the included trials reported on adverse events, costs or occurrence of type 2 diabetes.

From these findings, we drew the following conclusions: “There are currently no long-term trials examining tai chi for the primary prevention of CVD. Due to the limited evidence available currently no conclusions can be drawn as to the effectiveness of tai chi on CVD risk factors. There was some suggestion of beneficial effects of tai chi on CVD risk factors but this was not consistent across all studies. There was considerable heterogeneity between the studies included in this review and studies were small and at some risk of bias. Results of the ongoing trials will add to the evidence base but additional longer-term, high-quality trials are needed.”

These findings are somewhat disappointing. Tai chi might convey many health benefits, but whether a reduction of cardiovascular risk is amongst them seems doubtful. Even if a risk reduction were established beyond doubt, one would need to ask whether its effect size is larger than that achievable through regular conventional exercise. In my view, this is unlikely.

Guest Post by Jan Willem Nienhuys

The so-called Swiss government report of 2011 on homeopathy was actually an expanded translation of a 2006 book, which in itself was an expanded version of a document submitted to a Swiss committee (PEK) in charge of evaluation of alternative medicine. It has been severely criticised. A summary of criticisms with links can be found on the RationalWiki item to which we may add the Zeno’s Blog. I present here the results of my scrutiny of chapter 10 (1), although I base my report on the original German edition.

This chapter by itself shows a familiar result: the better the investigation, the less evidence in favor of homeopathy it shows. It shows also how homeopaths systematically distort unfavorable results by mispresenting them. Chapter 10 deals with clinical investigations of homeopathy. The authors restrict their attention to an odd assortment of diseases such as acute rhinitis, allergic rhinitis, allergic asthma, sinusitis, adenoid vegetations, pharyngitis, tonsillitis, influenza-like infection and otitis media, together denoted as ‘upper respiratory tract infections/allergic reactions’ or URTI/A for short.

The number of papers reviewed is very small. The authors looked at much more than randomized clinical trials. Apparently their search did not extend further than 2003, but then they might have found over 150 papers, of which about one third double blind randomized trials that compared how well highly diluted homeopathy and placebo cured one of the indicated diseases. They managed to miss 25 papers mentioned in earlier meta-analyses and about four papers that are summarized in Pubmed.

Among the papers they missed is an extremely strong support for the claim ‘homeopathy works for URTI/A’. For example Riverón-Garrote et al. (2) did a placebo controlled double blind randomized clinical trial of homeopathy (apparently individualised) for asthma. Of about 33 verum patients 32 improved, whereas of about 30 placebo patients only 4 improved. The so-called p-value for such a result is less than 10–11. One wonders why this result wasn’t published in Science or Nature, but only in an obscure Spanish language homeopathic journal. Maybe the paper was excluded because it didn’t state that it was about allergic asthma, but note that in about three quarters of all asthma some kind of allergy is implicated.

Of course this pales in comparison to the paper by Friese and Zabalotnyi (3). Again a double blind randomised clinical trial with 72 sinusitis sufferers for both verum and placebo. But here 71 out of 72 verum patients were free of complaints after three weeks, or at least improved, whereas this was the case for only 8 of the placebo patients. Fisher’s Exact Test gives p = 2.47 times 10-29 (one tailed). A remarkable result, because it is well known that over 80% of sinusitis cases cures spontaneously within two weeks. Maybe placebos are dangerous in the hands of homeopaths. Again one wonders why Friese and Zabalotnyi didn’t share the Nobel prize in, say, 2008, and why it is necessary at all to meticulously analyse papers in which homeopathy shows a marginal advantage.

Instead, Maxion-Bergemann et al. include in their survey a paper by Bahemann (4). We quote the summary of the paper from the internet: ‘In homeopathic practice, Kalium bromatum is known as a remedy in the case of paranoid delusions, e. g. if someone suffers from the delusion of being the object of divine revenge, of being damned, or of being pursued. It is also a very important remedy in the case of nocturnal fears in children as well as in the case of convulsions, when they are hereditary, when they occur in childbed, or during teething. The following case demonstrates the successful treatment of a severe mononucleosis after studying the Materia medica.’ Mononucleosis isn’t even mentioned in the list given that specifies URTI/A. Maybe it was included because one of the symptoms of mononucleosis is a sore throat. Apparently the mononucleosis patient was given Kalium Bromatum (Maxion-Bergemann et al. state that it is Kalium Chromatum 200C, presumably Chromatum and Bromatum don’t differ too much to bother) because of something remarkable the patient said during the anamnesis. The reason for giving Kalium bromatum 200C in cases of paranoia might be that an overdose of bromide can induce psychoses. The homeopathic Materia Medica contains quite a few ‘symptoms’ from accidental poisonings reported in old medical literature; potassium bromide was liberally used in the nineteenth century for the calming of seizure and nervous disorders, according to Wikipedia.

More impressive in the list of 13 RCTs of Maxion-Bergemann are two of the largest ‘homeopathic’ trials known, namely of the remedy Oscillococcinum. These trials cannot be taken seriously. The first one, by Ferley et al. (5), has one glaring fault. They started with 478 ‘influenza’-patients (237 verum), tried to make 149 family physicians note down when the patients recovered, and then elected to restrict their attention to the 63 patients (39 verum) that recovered within 48 hours and therefore probably didn’t have flu at all. Coincidentally this was the only possibility out of 14 that gave a ‘significant’ result: correctly computed, p is just below 0.05. (Ferley et al. based their computation on 462 patients with 228 verum and applied a chi-squared test without continuity correction). It is hardly credible that they set this 48-hour criterion in advance, because even if the remedy worked, the risk of having too few subjects to get a significant result would have been considerable. But if one picks out one result among many possibilities, one should correct for multiple outcome. So the Ferley et al. investigation is at most an exploratory result in need of independent confirmation.

This ‘confirmation’ was undertaken soon afterwards, namely in the beginning of 1991, but the results were only published in 1998 and cannot be found on Pubmed (6). In this paper the definitions are somewhat different, but Papp et al. report that of 334 patients (167 verum) a total of 57 (32 verum) were cured in 48 hours. Now 25 versus 32 is not remarkable at all. One doesn’t need any elaborate computation for this. Calculation gives p=0.4. So one might think that the Ferley hypothesis was soundly refuted. But Papp et al. used something they call ‘the Krauth test’, probably some kind of automated post hoc fishing trip to select the best criteria to distinguish the placebo and verum groups. They claim that this ‘test’ gives p=0.0028. They specifically refer to ‘the null hypothesis (the number of patients free of symptoms after 48 hours is equal in both treatment groups)’, so their computation is wrong. The most remarkable thing about Papp et al. is that nobody seems to have to have noticed the large discrepancy between what the numbers say and the claim of the paper.

Another paper with ‘positive’ results is the 1994 study of Reilly et al. (7), number 28 in Maxion-Bergemann et al. The group of Reilly investigated allergic diseases treated by what they called homeopathy. The typical Reilly experiment consists of administering a highly diluted causative agent such as pollen or house dust mite or cat hairs or bird feathers to persons suffering from pollen allergy (seasonal rhinitis) or allergic asthma. However for true homeopathy one uses a substance that has been the subject of a so-called proving, and the remedy is chosen of the totality of all patient ‘symptoms’ – including things like sleeping position and fear of thunderstorms – sufficiently matches the symptoms of the proving. Let me call Reilly’s method ultra-isopathy. Reilly was already discussing this study on a symposium in 1990, but that paper is not clear. It is about 28 asthma patients, and only 24 were analysed. This small number in itself is already reason enough not to consider it. The main analysis was by comparing a subjective measure of wellbeing, the Visual Analog Scale (VAS). Here we find a significant difference (p=0.003) in favor of ultra-isopathy. However, in the small print we see that change in the very important FEV1-value (Forced Expiratory Volume in 1 second) was non-significant (p=0.08) but this refers only to the 18 patients that took such a test before and after the experiment.

Reilly attracted more attention with his first experiment in this vein (8). He started out with 79 patients in both the verum and the placebo group. The treatment was ultradiluted grass pollen for hay fever. The analysis was only about 56 verum and 52 placebo (in a diagram 53 placebo are shown). Such a large dropout (32%) is not good. On basis of the VAS-scores Reilly found p=0.02. VAS is only an ordinal scale and it is not at all clear that one person’s 60 mm means the same as another person’s 60 mm, and also not that two patients with respectively 40 mm and 80 mm together can be considered as equivalent to two other patients with 60 mm each. If we distinguish only better / equal / worse, then the numbers for the verum group were 34 / 9 / 13 and for the placebo group 27 / 5 / 21. One can analyse this in various ways: as a 3 by 2 contingency table (p=0.15), or as a 2 by 2 table, namely by joining the middle group either to the right (p=0.10) or to the left (p=0.34). In this manner the difference is less impressive.

Maxion-Bergemann et al. collected 29 articles. I take the liberty of removing from these everything that is not a double blind RCT that compares how well highly diluted homeopathy and placebo cures an URTI/A disease. We also remove all research with 50 or less patients. The more or less openly fraudulent or at least grossly mistaken Oscillococcinum trials I also leave out. In order of appearance we have then Wiesenauer 1985 (9) [8] Reilly 1986 (8) [6] Wiesenauer 1989 (10) [10] De Lange-de Klerk 1994 (11) [1] Aabel 2000 (12) [4] Jacobs 2001 (13) [22] Friese 2001 (14) [24] Lewith 2002 (15) [25] White 2003 (16) [29] The square brackets refer to the numbering in Maxion-Bergemann et al. A short review of these nine articles follows.

Wiesenauer 1985: one standard remedy for hayfever. Randomised 213 patients, analysed only 164. “no statistical significance was achieved” says the abstract on Pubmed. Reilly 1986: this we have discussed already. Ultra-isopathy for hayfever. Randomised 158 patients, analysed 108. Statistically significant, but barely so. Wiesenauer 1989: four groups, each with their own standard remedy or placebo for sinusitis, 152 patients. “There was no remarkable difference in the therapeutic success among the investigated homeopathic drug combinations nor between the active drugs and placebo”, according to the abstract in Pubmed De Lange-de Klerk 1994: this research was reported more extensively in the lead author’s dissertation (17). Individualised homeopathy for recurrent URTI in children. 175 children were randomised and 170 analysed after following them for a year. 128 different remedies/potencies were prescribed and all together 1042 different prescriptions were handed out. The result was a non-significant difference between homeopathy and placebo. One striking aspect of this investigation is that only after all computations were done, it was revealed which of the two groups was the placebo group and which the verum group. So the author or her thesis advisors deliberately made it impossible to fall for the temptation to start a fishing expedition in the data after the code was completely broken. See also Pubmed. Aabel 2000: ultra-isopathy for birch pollen allergy. Strictly speaking this investigation shouldn’t be in this short list because it was partly prophylactic. From Pubmed: “Surprisingly, the verum treated patients fared worse than the placebo group”. No measure of statistical significance is mentioned. Remarkably this article is preceded by a similar article (18) that Maxion-Bergemann et al. apparently weren’t able to locate. Jacobs 2001: 75 children with otitis media were treated with individualised homeopathy or placebo. Pubmed: “differences were not statistically significant”. It seems that Jacobs has indulged in a fishing trip because she mentions a “significant decrease in symptoms at 24 and 64 h after treatment in favor of homeopathy”. But that is wrong. Significance only can have a meaning if it refers to a single outcome that was planned before any patients were seen. Just picking out two results out of many and stating they are ‘significant’ betrays a fundamental ignorance of research methodology. Friese 2001: this article is also published elsewhere (19), at least the numbers are exactly the same according to Pubmed. 97 children randomized for either individual homeopathic treatment or placebo treatment of adenoid vegetations, 82 analysed. Apparently these 82 comprised 41 placebo and 41 verum, and of these 12 and 9 respectively required an operation in the end. This allegedly corresponds to p=0.64, “These results show no statistical significance.” Incidentally, this is the same Friese as reference 3. Lewith 2002: again ultra-isopathy, now for asthma, 242 patients randomised, 202 completed all clinical assessments. The full article can be accessed via Pubmed and elsewhere. The main conclusion is “Homoeopathic immunotherapy is not effective in the treatment of patients with asthma.” The authors notice that the averages in both groups behave somewhat erratic, and they have no explanation for this. White 2003: individualised homeopathy compared to placebo for 96 children with asthma, who are followed for 12 months. The conclusion is that there is no evidence that this kind of homeopathy is better than placebo. In other words, out of nine investigations only one (Reilly 1986) obtains a barely significant result.

But the interpretation of Maxion-Bergemann et al. is totally different: “If only the placebo-controlled, randomized trials with the highest EBM evidence are considered, 12 of 16 trials show a positive result for the homeopathically treated group (significantly positive 8/16 and trend 4/16).” Even in the more restricted subset of nine discussed above they are overly optimistic. They mark Wiesenauer (1985), De Lange-de Klerk (1994), Jacobs (2001) as showing a ‘trend for homeopathy’ and Lewith (2002) is even marked ‘significant’. The meticulous and high quality research of De Lange (1993, 1994) is judged ‘trend for homeopathy’.

In case of De Lange it seems clear where this judgement comes from. De Lange had several outcomes (number of sick periods, total duration of sick periods, sum of all dayscores etc., and all these showed roughly the same small non-significant difference in favor of homeopathy. This is not really strange, because these outcomes all measure about the same phenomenon. It is not remarkable that there is a small difference between the averages of the two groups that can only be noticed if the children are followed for a full year. There is not even the beginning of a reason that this has anything to do with the treatment. For example the homeopathy group had ‘significantly’ less pets at home. This might serve as an explanation why they as a group were slightly less sick. One might also speculate that this was retroactively caused by the homeopathic treatment. This is not really more improbable than highly diluted stuff (more than 95% D6 and higher) having an effect.

By convention ‘statistically significant’ is the lower limit where weak conclusions such as ‘worth investigating further’ can be justified, and we repeat: only if it refers to a single outcome measure or endpoint chosen before any data collection has started. De Lange chose recurrent URTI because homeopathy was reputed to be most effective for this type of complaints, especially after investigations such as those of Reilly (1986). If following 170 children for a full year cannot show a clear advantage, then that is simply a negative result. In the case of Lewith the ‘significant for homeopathy’ is probably based on partial results such as that in week 3 ‘homeopathy’ fared better in the asthma VAS. One can just as well point to week 16 where the FEV1 of the placebo group seems much better than in the homeopathy group.

Maxion-Bergemann et al. seem to have been singularly inept in collecting papers on homeopathic trials, and for no apparent reason they decided to look also at a large number of case reports and investigations without control group or blinding, even after investigators as early as 1991 have remarked that henceforth only well designed large double blind RCTs were worth considering. If we restrict our attention to the properly blinded controlled investigations, we see the same thing as in other meta-analyses of homeopathy: there is lots of rubbish in favor of homeopathy, but the good trials say plainly and clearly: homeopathy is ineffective, precisely what can be predicted from the fact that there is nothing in it.

Homeopaths nowadays have a lot to say about RCTs and how they prove homeopathy. RCTs are subtle and complicated scientific tools. It is somewhat strange to see how homeopaths resolutely ignore two centuries of basic science but then argue their cause on the basis of complicated statistics.

Homeopathy is an assortment of wildly different practices and theories. We have seen ultra-isopathy, individualised homeopathy and the practice of giving one standardised remedy for one diagnosis without asking too many personal details from the patient. These standard remedies are often branded mixtures of highly diluted ‘classical’ homeopathy, quite contrary to the opinions of homeopathy’s inventor Hahnemann. There are many more variants of homeopathy and the homeopaths themselves cannot agree which are the correct ones.

Moreover, if a treatment or trial doesn’t work out, then a number of additional hypotheses about homeopathy can be invoked, which is what Maxion-Bergemann et al. do. Homeopathic remedies supposedly are counteracted by lots of regular medications and even by strong tasting or smelling food, such as coffee, parsley, garlic and peppermint. Hahnemann even disapproved of reading in bed and long afternoon naps and prolonged suckling of infants (Organon, section 260). Poor performance of homeopathy can be blamed on something called ‘initial aggravation’ or else on lack of experience of the poorly performing homeopath.

But that these factors are relevant at all is unknown, just like there is no proof at all for the similia principle, nor for the hundred thousands or even millions of ‘symptoms’ associated with highly diluted materials in the homeopathic Materia Medica. If homeopaths really want scientists to share homeopathic beliefs, they should not think up lame excuses for ‘failed’ tests, but for starters they might try to present proofs for all or at least some of their ‘symptoms’. They don’t try very hard and in so far it has been tried, it also has failed (20).

I would like to thank Willem Betz for helpful remarks.

I am a retired mathematician with no other interest than a desire to promote science.


1. Stefanie Maxion-Bergemann, Gudrun Bornhöft, Denise Bloch, Christina Vogt-Frank, Marco Righetti, André Thurneysen. (2011) Clinical Studies on the Effectiveness of Homeopathy for URTI/A (Upper Respiratory Tract Infections and Allergic Reactions) in: Homeopathy in Healthcare – Effectiveness, Appropriateness, Safety, Costs. G. Bornhöft and P.F. Mattheiesen (eds.), Berlin etc., Springer 2011, p. 18-157.

2. Riverón-Garrote, M., Fernandez-Argüelles, R.; Morán-Rodríquez, F.; Campistrou-Labaut, J.L. (1998) Ensayo clínico controlado aleatorízado del tratamiento homeopático del asma bronquial, Boletín Mexicano de Homepatía 1998; 31(2):54-61.

3. Friese, K.-H., Zabalotnyi, D.I. (2007) Homöopathie bei akuter Rhinosinusitis, Eine doppelblinde, placebokontrollierte Studie belegt die Wirksamkeit und Verträglichkeit eines homöopathischen Kombinationsarzneimittels, HNO 55(4):271-277.

4. Bahemann A. (2002) Kalium bromatum bei infektiöser Mononukleose. Zeitschrift für Klassische Homöopathie 46:232–233.

5. Ferley J.P., Zmirou D., D’Adhemar D., Balducci F. (1989). A controlled evaluation of a homoeopathic preparation in the treatment of influenza like syndromes. British Journal of Clinical Pharmacology 27:329-335.

6. Papp R., Schuback G., Beck E., Burkard G., Bengel J., Lehrl S., Belon P. (1998). Oscillococcinum in patients with influenza-like syndromes: a placebo-controlled double-blind evaluation. British Homeopathic Journal 87:69-76.

7. Reilly, D.T., Taylor, M.A., Beattie, N.G.M., Campbell, J.H., McSharry C., Aitchison T.C., Carter R., Stevenson R. (1994) Is evidence for homoeopathy reproducible?, Lancet 1994 344:1601-1606.

8. Reilly, D.T., Taylor, M.A., McSharry, C., Aitchison, T. (1986) Is Homoeopathy a Placebo Response?, Controlled Trial of Homoeopathic Potency – With Pollen in Hayfever as Model, Lancet II.2:881-886.

9. Wiesenauer, M., Gaus, W. (1985) Double-blind Trial Comparing the Effectiveness of Galphimia Potentisation D6 (Homoeopathic Preparation), Galphimia Dilution 10-6 and Placebo on Pollinosis, Arzneimittelforschung 35(11):1745-1747.

10. Wiesenauer M, Gaus W, Bohnacker U, Häussler S (1989) Wirksamkeitsprüfung von homöopathischen Kombinationspräparaten bei Sinusitis: Ergebnisse einer randomisierten Doppelblindstudie unter Praxisbedingungen. Arzneimittelforschung 39:620-625.

11. de Lange-de Klerk E.S.M., Blommers J., Kuik D.J., Bezemer P.D., Feenstra L. (1994). Effects of homoeopathic medicines on daily burden of symptoms in children with recurrent upper respiratory tract infections. BMJ 309:1329-1332.

12. Aabel, S. (2000) No beneficial effect of isopathic prophylactic treatment for birch pollen allergy during a low-pollen season, A double-blind, placebo-controlled clinical trial of homeopathic Betula 30c. British Homeopathic Journal 89(4):169-173.

13. Jacobs, J., Springer, D.A., Crothers, D. (2001) Homeopathic treatment of acute otitis media in children, A preliminary randomized placebo-controlled trial. The Pediatric Infectious Disease Journal 20(2):177-183.

14. Friese K.H., Feuchter U., Lüdtke R., Moeller H. (2001) Results of a randomised prospective double-blind trial on the homeopathic treatment of adenoid vegetations. European Journal of General Practice 7:48-54.

15. Lewith, G.T., Watkins, A.D.; Hyland, M.E.; Shaw, S.; Broomfield, J.A.; Dolan, G.; Holgate, S.T. (2002) Use of ultramolecular potencies of allergen to treat asthmatic people allergic to house dust mite: double blind randomised controlled clinical trial, BMJ 324:520-523.

16. White, A., Slade, P.; Hunt, C.; Hart, A.; Ernst, E. (2003) Individualised homeopathy as an adjunct in the treatment of childhood asthma, A randomised placebo controlled trial. Thorax 58(4):317-321

17. Lange-de Klerk, E.S.M. de, Effects of homoeopathic medicines on children with recurrent upper respiratory tract infections. Vrije Universiteit Amsterdam, 1993 (Dissertation).

18. Aabel, S., Laerum, E.; Dölvik, S.; Djupesland, P. (2000) Is homeopathic ‘immunotherapy’ effective?, A double-blind, placebo-controlled trial with the isopathic remedy Betula 30c for patients with birch pollen allergy. British Homeopathic Journal 89(4):161-168.

19. Friese K.-H., Feuchter U., Möller H. (1997). Die homöopathische Behandling von adenoiden Vegetationen. HNO; 45:618–624.

20. Brien S., Lewith G., Bryant, T. (2003) Ultramolecular homeopathy has no observable clinical effects. A randomized, double-blind, placebo-controlled proving trial of Belladonna 30C.

A recent meta-analysis evaluated the efficacy of acupuncture for treatment of irritable bowel syndrome (IBS) and arrived at bizarrely positive conclusions.

The authors state that they searched 4 electronic databases for double-blind, placebo-controlled trials investigating the efficacy of acupuncture in the management of IBS. Studies were screened for inclusion based on randomization, controls, and measurable outcomes reported.

Six RCTs were included in the meta-analysis, and 5 articles were of high quality.  The pooled relative risk for clinical improvement with acupuncture was 1.75 (95%CI: 1.24-2.46, P = 0.001). Using two different statistical approaches, the authors confirmed the efficacy of acupuncture for treating IBS and concluded that acupuncture exhibits clinically and statistically significant control of IBS symptoms.

As IBS is a common and often difficult to treat condition, this would be great news! But is it true? We do not need to look far to find the embarrassing mistakes and – dare I say it? – lies on which this result was constructed.

The largest RCT included in this meta-analysis was neither placebo-controlled nor double blind; it was a pragmatic trial with the infamous ‘A+B versus B’ design. Here is the key part of its methods section: 116 patients were offered 10 weekly individualised acupuncture sessions plus usual care, 117 patients continued with usual care alone. Intriguingly, this was the ONLY one of the 6 RCTs with a significantly positive result!

The second largest study (as well as all the other trials) showed that acupuncture was no better than sham treatments. Here is the key quote from this trial: there was no statistically significant difference between acupuncture and sham acupuncture.

So, let me re-write the conclusions of this meta-analysis without spin, lies or hype: These results of this meta-analysis seem to indicate that:

  1. currently there are several RCTs testing whether acupuncture is an effective therapy for IBS,
  2. all the RCTs that adequately control for placebo-effects show no effectiveness of acupuncture,
  3. the only RCT that yields a positive result does not make any attempt to control for placebo-effects,
  4. this suggests that acupuncture is a placebo,
  5. it also demonstrates how misleading studies with the infamous ‘A+B versus B’ design can be,
  6. finally, this meta-analysis seems to be a prime example of scientific misconduct with the aim of creating a positive result out of data which are, in fact, negative.

The Australian ‘NATIONAL HEALTH AND MEDICAL RESEARCH COUNCIL’ (NHMRC) has assessed the effectiveness of homeopathy. The evaluation looks like the most comprehensive and most independent in the history of homeopathy. Its draft report has just been released and concludes that “the evidence from research in humans does not show that homeopathy is effective for treating the range of health conditions considered.”

Not for a single health conditions was there reliable evidence that homeopathy was effective. No rigorous studies reported either that homeopathy caused greater health improvements than a placebo, or that homeopathy caused health improvements equal to those of another treatment.

The overview considered a total of 57 systematic reviews that assessed the effectiveness of homeopathy for 61 different health conditions.

The draft report presents the evidence according to 4 different categories:


Homeopathy was reported to be not more effective than placebo in either all the studies found, or in a large majority of the reliable studies for the treatment of the following health conditions:

  • adenoid vegetation in children
  • asthma
  • anxiety or stress-related conditions
  • diarrhoea in children
  • headache and migraine
  • muscle soreness
  • labour
  • pain due to dental work
  • pain due to orthopaedic surgery
  • postoperative ileus
  • premenstrual syndrome
  • upper respiratory tract infections
  • warts.


For the following condition, although some studies reported that homeopathy was more effective than placebo, trials were not reliable and homeopathy was therefore judged to be no more effective than placebo:

  • allergic rhinitis
  • attention deficit/hyperactivity disorder
  • bruising
  • chronic fatigue syndrome
  • diarrhoea in children
  • fibromyalgia
  • hot flushes in women who have had breast cancer
  • human immunodeficiency virus infection
  • influenza-like illness
  • rheumatoid arthritis
  • sinusitis
  • sleep disturbances or circadian rhythm disturbances
  • stomatitis  due to chemotherapy
  • ulcers.


For the following conditions, although some studies reported that homeopathy was as effective as or more effective than another treatment, trials were not reliable:

  • acute otitis media or otitis media with effusion
  • allergic rhinitis
  • anxiety or stress-related conditions
  • depression
  • eczema
  • non-allergic rhinitis
  • osteoarthritis
  • upper respiratory tract infection


There was no reliable evidence on which to draw a conclusion about the effectiveness of homeopathy, compared with placebo, for the treatment of the following health conditions:

  • acne vulgaris
  • acute otitis media in children
  • acute ankle sprain
  • acute trauma
  • amoebiasis and giardiasis
  • ankylosing spondylitis
  • boils and pyoderma
  • Broca’s aphasia after stroke
  • bronchitis
  • cholera
  • cough
  • chronic polyarthritis
  • dystocia
  • eczema
  • heroin addiction
  • knee joint haematoma
  • lower back pain
  • nausea and vomiting associated with chemotherapy
  • oral lichen planus
  • osteoarthritis
  • proctocolitis
  • postoperative pain-agitation syndrome
  • radiodermatitis in women with breast cancer
  • seborrhoeic dermatitis
  • suppression of lactation after childbirth
  • stroke
  • traumatic brain injury
  • uraemic pruritis
  • vein problems due to cannulas in people receiving chemotherapy.


There was no reliable evidence on which to draw a conclusion about the effectiveness of homeopathy compared with other therapies for the treatment of the following health conditions:

  • burns
  • fibromyalgia
  • irritable bowel syndrome
  • malaria
  • proctocolitis
  • recurrent vulvovaginal candidiasis
  • rheumatoid arthritis.

The authors of the report now invite comments from interested parties. This means that homeopaths across the world can submit evidence which they feel has been ignored. It will be fascinating to see whether this changes the conclusion of the NHMRC’s assessment.

Fibromyalgia (FM) is a chronic condition which ruins the quality of life of many patients. It is also a domain of alternative medicine: dozens of different treatments are on offer – this is clearly a paradise for charlatans and bogus claims. So is there a treatment that is demonstrably effective? The purpose of this systematic review is to evaluate the evidence of massage therapy FM.

Electronic databases were searched to identify relevant studies. The main outcome measures were pain, anxiety, depression, and sleep disturbance. Two reviewers independently abstracted data and appraised risk of bias. The risk of bias of eligible studies was assessed based on Cochrane tools.

Nine randomized controlled trials involving 404 patients met the inclusion criteria. A meta-analyses showed that massage therapy with a duration of at least 5 weeks significantly improved pain , anxiety, and depression. Sleep disturbance was not improved by massage therapy.

The authors conclude that massage therapy with duration ≥5 weeks had beneficial immediate effects on improving pain, anxiety, and depression in patients with FM. Massage therapy should be one of the viable complementary and alternative treatments for FM. However, given fewer eligible studies in subgroup meta-analyses and no evidence on follow-up effects, large-scale randomized controlled trials with long follow-up are warrant to confirm the current findings.

To put these results into context, we need to consider the often poor methodological quality of the primary studies. It is, of course, not easy to test massage therapy in rigorous trials. For instance, there is no obvious placebo, and we can therefore not be sure whether the treatment benefits patients through a specific effect or whether non-specific effects are the cause of the improvement.

We also should be aware of the facts that for most other alternative therapies the evidence is not encouraging, and that massage therapy is relatively safe. Therefore the conclusion for those who suffer from FM might well be that massage therapy is worth a try.

Dutch neurologists recently described the case of a 63-year-old female patient presented at their outpatient clinic with a five-week history of severe postural headache, tinnitus and nausea. The onset of these symptoms was concurrent with chiropractic manipulation of the cervical spine which she had tried because of cervical pain.

Cranial MRI showed findings characteristic for intracranial hypotension syndrome. Cervical MRI revealed a large posterior dural tear at the level of C1-2. Following unsuccessful conservative therapy, the patient underwent a lumbar epidural blood patch after which she recovered rapidly.

The authors conclude that manipulation of the cervical spine can cause a dural tear and subsequently an intracranial hypotension syndrome. Postural headaches directly after spinal manipulation should therefore be a reason to suspect this complication. If conservative management fails, an epidural blood patch may be performed.

Quite obviously, this is sound advice that can save lives. The trouble, however, is that the chiropractic profession is, by and large, still in denial. A recent systematic review by a chiropractor included eight cases of intracranial hypotension (IH) and concluded that case reports on IH and spinal manipulative therapy (SMT) have very limited clinical details and therefore cannot exclude other theories or plausible alternatives to explain the IH. To date, the evidence that cervical SMT is not a cause of IH is inconclusive. Further research is required before making any conclusions that cervical SMT is a cause of IH. Chiropractors and other health practitioners should be vigilant in recording established risk factors for IH in all cases. It is possible that the published cases of cervical SMT and IH may have missed important confounding risk factors (e.g. a new headache, or minor neck trauma in young or middle-aged adults).

Instead of distracting us from the fact that chiropractic can lead to serious adverse events, chiropractors would be well-advised to face the music, admit that their treatments are not risk-free and conduct rigorous research with a view of minimizing the harm.

Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted but you must tick the box: “Save my name, email, and website in this browser for the next time I comment.”

The most recent comments from all posts can be seen here.