MD, PhD, FMedSci, FRSB, FRCP, FRCPEd.

study design

1 2 3 19

Turmeric is certainly a plant with fascinating properties; we have therefore discussed it before. Reseach into turmeric continues to be active, and I will continue to report about new studies.

This study was aimed at estimating the effect of turmeric supplementation on quality of life (QoL) and haematological parameters in breast cancer patients who were on Paclitaxel chemotherapy. In this case series with 60 participants, QoL was assessed using a standard questionnaire and haematological parameters were recorded from the patients’ hospital records.

Turmeric supplementation for 21 days resulted in clinically relevant and statistically significant improvement in global health status, symptom scores (fatigue, nausea, vomiting, pain, appetite loss, insomnia), and haematological parameters.

The authors concluded that turmeric supplementation improved QoL, brought about symptom palliation and increased hematological parameters in breast cancer patients.

Really?

The way the conclusions are phrased, they clearly imply that turmeric caused the observed outcomes. How certain can we be that this is true?

On a scale of 0 -10, I would say 0.

Why?

Because there are important other determinants of the outcomes:

  • placebo,
  • concommittant treatments,
  • natural history,
  • etc., etc.

Why does this matter?

  • Because such unwarranted conclusions mislead patients, healthcare professionals and carers.
  • Because such bad science gives a bad name to clinical research.
  • Because this type of nonsense might deter meaningful research into a promising subject.
  • Because no ‘scientific’ journal should be permitted to publish such nonsense.
  • Because it is unethical of ‘scientists’ to make false claims.

But maybe the Indian authors are just a few well-meaning and naive practitioners who merely were doing their unexperienced best? Sadly not! The authors of this paper give the following affiliations:

  • Clinical Pharmacology, Pfizer Healthcare Private Limited, Chennai, Tamil Nadu, India.
  • Department of Radiation Oncology, Faculty of Medicine, Sri Ramachandra Institute of Higher Education and Research, Porur, Chennai, Tamil Nadu, India.
  • Process Development, HCL Technologies, Chennai, Tamil Nadu, India.
  • Department of Pharmacognosy, Faculty of Pharmacy, Sri Ramachandra Institute of Higher Education and Research, Porur, Chennai, Tamil Nadu, India.

Yes, they really should know better!

Dr Jennifer Jacobs is a homeopaths from the US. She is a family physician and a clinical assistant professor in epidemiology at the University of Washington School of Public Health and Community Medicine. She received her MD degree from Wayne State University and a Masters in Public Health from the University of Washington.

Jennifer is foremost famous for the homeopathic childhood diarhoea studies, but does that justify her joining THE ALTERNATIVE MEDICINE HALL OF FAME with its 15 current members who managed the impossible feast of never publishing a negative conclusion about their pet SCAM:

A Medline search generated 25 papers of hers on homeopathy. Here are the key findings of the … that report original data on the effectiveness of homeopathy (clinical trials or reviews):

  1. If and when conventional medicine runs out of options for treating epidemic diseases, homeopathy could be seen as an attractive alternative, but only if there is viable experimental evidence of its success.
  2. The homeopathic syrup appeared to be effective in reducing the severity of cold symptoms in the first day after beginning treatment.
  3. the medicines prescribed in individualised homeopathy may have small, specific, treatment effects.
  4. Homeopathic ear drops may be effective in reducing the use of antibiotics in children with AOM managed with a delayed antibiotic approach.
  5. This study suggests that homeopathic ear drops were moderately effective in treating otalgia in children with AOM and may be most effective in the early period after a diagnosis of AOM. Pediatricians and other primary health care providers should consider homeopathic ear drops a useful adjunct to standard therapy.
  6. The homeopathic combination therapy tested in this study did not significantly reduce the duration or severity of acute diarrhea in Honduran children. Further study is needed to develop affordable and effective methods of using homeopathy to reduce the global burden of childhood diarrhea.
  7. This pilot study provides no evidence to support a therapeutic effect of individually selected homeopathic remedies in children with ADHD. A therapeutic effect of the homeopathic encounter is suggested and warrants further evaluation.
  8. Small sample size precludes definitive answers, but results from this preliminary trial suggest that homeopathy may be of value in the treatment of menopausal symptoms and improving quality of life, especially in those women not on tamoxifen.
  9. The results from these studies confirm that individualized homeopathic treatment decreases the duration of acute childhood diarrhea and suggest that larger sample sizes be used in future homeopathic research to ensure adequate statistical power. Homeopathy should be considered for use as an adjunct to oral rehydration for this illness.
  10. These results suggest that a positive treatment effect of homeopathy when compared with placebo in acute otitis media cannot be excluded and that a larger study is justified.
  11. These results are consistent with the finding from the previous study that individualized homeopathic treatment decreases the duration of diarrhea and number of stools in children with acute childhood diarrhea.
  12. The statistically significant decrease in the duration of diarrhea in the treatment group suggests that homeopathic treatment might be useful in acute childhood diarrhea. Further study of this treatment deserves consideration.

Next to Claudia Witt, Jennifer might be the researcher who has published the most clinical trials of homeopathy with positive conclusions (don’t be jealous Michael Frass, you might be in third place!). Attentive readers have probably noticed, she also published a negative trial with a negative conclusion (No 6) and a negative trial with a not so negative conclusion (No 7). The negative study almost cost her the place in the HALL OF FAME. But let’s be generous, and let’s consider the TRUSTWORTHINESS INDEX which, in her case, is still well and safely in the untrustworthy region. Therefore, I hope we all agree: Jenifer does deserve a place in THE ALTERNATIVE MEDICINE HALL OF FAME.

WELCOME JENNIFER!

This is an analysis that I have long hesitated to conduct. The reason for my hesitation is simple: some people might think it is vindictive, revengeful or ad hominem. After reflecting about it for years, I have now decided to go ahead with it (sorry, it’s a bit lengthy). This case study is not meant to be vindictive, but offers an important insight into the power of conflicts of interest in SCAM that are not financial but ideological. I think it is crucial that people are aware of and consider such conflicts carefully, and I can’t see how else I might demonstrate my point so plainly.

Dr Adrian White was a co-worker of mine for about 10 years. He became a trusted colleague, my ‘right hand’ man and even my deputy at my Exeter department. When I discovered that my trust had been misplaced, I did not prolong his contract (I will not dwell on this episode, those who are interested find it in my memoir). Adrian then got a senior research fellowship with Prof John Campbell (not my favourite colleague at Exeter) at the department of general practice where he continued his research on acupuncture for about 10 more years largely unsupervised.

Adrian had been an acupuncturist body and soul (in fact, I had never before met anyone so utterly convinced of the value of this therapy). When he joined my team, he was scientifically naive, and we spent many month trying to teach him how to think like a scientist. Initially, he found it very difficult to think critically about acupuncture. Later, I thought the problem was under control. Yet, most of his research in my department was guided by me and tightly supervised (i.e. I made sure that out studies were testing rather than promoting SCAM, and that our reviews were critical assessments of the existing evidence).

Thus there exist two separate and well-documented periods of a pro-acupuncture researcher:

  • 10 years guided by me and members of my team;
  • 10 years largely unsupervised.

What could be more tempting than to compare Adrian’s output during these two periods?

To do this, I looked up all of Adrian’s 120 publications on acupuncture and selected those 52 articles that generated factual new data (mostly clinical trials or systematic reviews). As it happens, they are numerically distributed almost equally within the two periods. The endpoints for my analysis were the directions of the conclusions of his papers. I therefore extracted, dated, and rated the 52 articles as follows:

  • P = positive from the point of view of an acupuncture advocate,
  • N = negative from the point of view of an acupuncture advocate.
  • P/N = not clearly pointing in either direction.

To render this exercise transparent (occasionally, I was not entirely sure about my ratings), I copied all the 52 conclusions and provided links to the original papers so that anyone inferested is able to check easily.

Here are my findings. Articles 1 – 27 were published AFTER Adrian had left my department; articles 28 – 52 are his papers from the time while he worked with me.

  1. A definitive three-arm trial is feasible. Further follow-up reminders, minimum data collection and incentives should be considered to improve participant retention in the follow-up processes in the standardised advice and exercise booklet arm. (2016) P/N
  2. The available evidence suggests that adding acupuncture to symptomatic treatment of attacks reduces the frequency of headaches. Contrary to the previous findings, the updated evidence also suggests that there is an effect over sham, but this effect is small. The available trials also suggest that acupuncture may be at least similarly effective as treatment with prophylactic drugs. Acupuncture can be considered a treatment option for patients willing to undergo this treatment. As for other migraine treatments, long-term studies, more than one year in duration, are lacking. (2016) P
  3. The available results suggest that acupuncture is effective for treating frequent episodic or chronic tension-type headaches, but further trials – particularly comparing acupuncture with other treatment options – are needed. (2016) P
  4. Acupuncture during pregnancy appears to be associated with few AEs when correctly applied. (2014) P
  5. Although pooled estimates suggest possible short-term effects there is no consistent, bias-free evidence that acupuncture, acupressure, or laser therapy have a sustained benefit on smoking cessation for six months or more. However, lack of evidence and methodological problems mean that no firm conclusions can be drawn. Electrostimulation is not effective for smoking cessation. Well-designed research into acupuncture, acupressure and laser stimulation is justified since these are popular interventions and safe when correctly applied, though these interventions alone are likely to be less effective than evidence-based interventions. (2014) P
  6. The current evidence suggests that acupuncture may have some effects on drug dependence that have been missed because of choice of outcome in many previous studies, and future studies should use outcomes suggested by clinical experience. Body points and electroacupuncture, used in the original clinical observation, justify further research. (2013) P
  7. Acceptability is very high and may be maximised by taking a number of factors into account: full information should be provided before treatment begins; flexibility should be maintained in the appointment system and different levels of contact between fellow patients should be fostered; sufficient space and staffing should be provided and single-sex groups used wherever possible. (2012) P
  8. This is the first evaluation of nurse-led group (multibed) acupuncture clinics for patients with knee osteoarthritis to include a 2 year follow-up. It shows the practicability of offering a low-cost acupuncture service as an alternative to knee surgery and the service’s success in providing long-term symptom relief in about a third of patients. Using realistic assumptions, the cost consequences for the local commissioning group are an estimated saving of £100 000 a year. Sensitivity analyses are presented using different assumptions. (2012) P
  9. There is no consistent, bias-free evidence that acupuncture, acupressure, laser therapy or electrostimulation are effective for smoking cessation, but lack of evidence and methodological problems mean that no firm conclusions can be drawn. Further, well designed research into acupuncture, acupressure and laser stimulation is justified since these are popular interventions and safe when correctly applied, though these interventions alone are likely to be less effective than evidence-based interventions. (2011) P/N
  10. Eight (8) of 10 international acupuncture experts were able to reach consensus on the syndromes, symptoms, and treatment of postmenopausal women with hot flashes. The syndromes were similar to those used by practitioners in the ACUFLASH clinical trial, but there were considerable differences between the acupuncture points. This difference is likely to be the result of differences in approach of training schools, and whether it is relevant for clinical outcomes is not well understood. (2011) P
  11. 70% of those patients eligible to participate volunteered to do so; all participants had clinically identified MTrPs; a 100% completion rate was achieved for recorded self-assessment data; no serious adverse events were reported as a result of either intervention; and the end of treatment attrition rate was 17%. A phase III study is both feasible and clinically relevant. This study is currently being planned. (2010) P
  12. In conclusion, the results from all studies are in agreement with the hypothesis that acupuncture needling relieves hot flushes. There are few data however supporting the hypothesis that the effect of acupuncture is point specific. Future research should investigate whether there is a biological effect of needling on hot flushes or not, whether tailored treatment is superior to standardised treatment, and ways of delivering treatment that causes least discomfort and least cost. (2010) P
  13. Acupuncture can contribute to a more rapid reduction in vasomotor symptoms and increase in health-related quality of life in postmenopausal women but probably has no long-term effects. (2010) P
  14. within the context of this pilot study, the sham acupuncture intervention was found to be a credible control for acupuncture. This supports its use in a planned, definitive, randomised controlled trial on a similar whiplash injured population. (2009) N/P
  15. factors other than the TCM syndrome diagnoses and the point selection may be of importance regarding the outcome of the treatment. (2009) N/P
  16. Acupuncture plus self-care can contribute to a clinically relevant reduction in hot flashes and increased health-related quality of life in postmenopausal women. (2009) P
  17. the authors conclude that acupuncture could be a valuable non-pharmacological tool in patients with frequent episodic or chronic tension-type headaches. (2009) P
  18. there is consistent evidence that acupuncture provides additional benefit to treatment of acute migraine attacks only or to routine care. There is no evidence for an effect of ‘true’ acupuncture over sham interventions, though this is difficult to interpret, as exact point location could be of limited importance. Available studies suggest that acupuncture is at least as effective as, or possibly more effective than, prophylactic drug treatment, and has fewer adverse effects. Acupuncture should be considered a treatment option for patients willing to undergo this treatment. (2009) P
  19. We have conducted the first survey of the effects of provision of acupuncture in UK general practice, using data provided by the NHS, and uncovered a wide variation in the availability of the service in different areas. We have been unable to demonstrate any consistent differences in the prescribing or referral rates that could be due to the use of acupuncture in these practices. The wide variation in the data means that if such a trend exists, a very large survey would be needed to identify it. However, we discovered inaccuracies and variations in presentation of data by the PCTs which have made the numerical input, and hence our results, unreliable. Thus the practicalities of access to data and the problems with data accuracy would preclude a nationwide survey. (2008) P
  20. In conclusion, there is limited evidence deriving from one study that deep needling directly into myofascial trigger points has an overall treatment effect when compared with standardised care. Whilst the result of the meta-analysis of needling compared with placebo controls does not attain statistically significant, the overall direction could be compatible with a treatment effect of dry needling on myofascial trigger point pain. However, the limited sample size and poor quality of these studies highlights and supports the need for large scale, good quality placebo controlled trials in this area. (2009) P
  21. We conclude that limited evidence supports acupuncture use in treating pregnancy-related pelvic and back pain. Additional high-quality trials are needed to test the existing promising evidence for this relatively safe and popular complementary therapy. (2008) P
  22. Acupuncture appears to offer symptomatic improvement to some patients with fibromyalgia in a tertiary clinic who have failed to respond to other treatments. In view of its safety, further acupuncture research is justified in this population. (2007) P
  23. It is speculated that optimal results from acupuncture treatment for osteoarthritis of the knee may involve: climatic factors, particularly high temperature; high expectations of patients; minimum of four needles; electroacupuncture rather than manual acupuncture, and particularly, strong electrical stimulation to needles placed in muscle; and a course of at least 10 treatments. These factors offer some support to criteria for adequate acupuncture used in the recent review. In addition, ethnic and cultural factors may influence patients’ reporting of their symptoms, and different versions of an outcome measure are likely to differ in their sensitivity – both factors which may lead to apparent rather than real differences between studies. The many variables in a study are likely to be more tightly controlled in a single centre study than in multicentre studies.  (2007) P
  24. Any effects of acupressure on smoking withdrawal, as an adjunct to the use of NRT and behavioural intervention, are unlikely to be detectable by the methods used here and further preliminary studies are required before the hypothesis can be tested. (2007) P
  25. Auricular acupuncture appears to be effective for smoking cessation, but the effect may not depend on point location. This calls into question the somatotopic model underlying auricular acupuncture and suggests a need to re-evaluate sham controlled studies which have used ‘incorrect’ points. Further experiments are necessary to confirm or refute these observational conclusions. (2006) P
  26. Acupuncture that meets criteria for adequate treatment is significantly superior to sham acupuncture and to no additional intervention in improving pain and function in patients with chronic knee pain. Due to the heterogeneity in the results, however, further research is required to confirm these findings and provide more information on long-term effects. (2007) P
  27. There is no consistent evidence that acupuncture, acupressure, laser therapy or electrostimulation are effective for smoking cessation, but methodological problems mean that no firm conclusions can be drawn. Further research using frequent or continuous stimulation is justified. (2006) N/P
  28. Acupuncture is not superior to sham treatment for recovery in activities of daily living and health-related quality of life after stroke, although there may be a limited effect on leg function in more severely affected patients.  (2005) N
  29. The evidence from controlled trials is insufficient to conclude whether acupuncture is an effective treatment for depression, but justifies further trials of electroacupuncture. (2005) N
  30. Acupuncture effectively relieves chronic low back pain. No evidence suggests that acupuncture is more effective than other active therapies. (2005) N/P
  31. In view of the small number of studies and their variable quality, doubt remains about the effectiveness of acupuncture for gynaecological conditions. Acupuncture and acupressure appear promising for dysmenorrhoea, and acupuncture for infertility, and further studies are justified. (2003) N
  32.  In conclusion, the results suggest that the procedure using the new device is indistinguishable from the same procedure using real needles in acupuncture naïve subjects, and is inactive, where the specific needle sensation (de qi) is taken as a surrogate measure of activity. It is therefore a valid control for acupuncture trials. The findings also lend support to the existence of de qi, a major concept underlying traditional Chinese acupuncture. (2002) N/P
  33. There is no clear evidence that acupuncture, acupressure, laser therapy or electrostimulation are effective for smoking cessation. (2002) N
  34. Collectively, these data imply that acupuncture is superior to various control interventions, although there is insufficient evidence to state whether it is superior to placebo. (2002) N/P
  35. In conclusion, the incidence of adverse events following acupuncture performed by doctors and physiotherapists can be classified as minimal; some avoidable events do occur. Acupuncture seems, in skilled hands, one of the safer forms of medical intervention. (2001) N/P
  36. Based on the evidence of rigorous randomised controlled trials, there is no compelling evidence to show that acupuncture is effective in stroke rehabilitation. Further, better-designed studies are warranted. (2001) N
  37. Although it has already been demonstrated that severe adverse events seem to be uncommon in standard practice, many serious cases of negligence have been found in the present review, suggesting that training system for acupuncturists (including medical doctors) should be improved and that unsupervised self-treatment should be discouraged. (2001) N
  38. Direct needling of myofascial trigger points appears to be an effective treatment, but the hypothesis that needling therapies have efficacy beyond placebo is neither supported nor refuted by the evidence from clinical trials. Any effect of these therapies is likely because of the needle or placebo rather than the injection of either saline or active drug. Controlled trials are needed to investigate whether needling has an effect beyond placebo on myofascial trigger point pain. (2001) N/P
  39. Although the incidence of minor adverse events associated with acupuncture may be considerable, serious adverse events are rare. Those responsible for establishing competence in acupuncture should consider how to reduce these risks. (2001) N
  40. In conclusion, this study does not provide evidence that this form of acupuncture is effective in the prevention of episodic tension-type headache. (2000) N
  41. The present study provides no strong evidence to support the hypothesis that the acupuncture point SP6 is more tender in women and in men. Recommendations for further investigations are discussed.  (2000) N
  42. Acupuncture has not been demonstrated to be efficacious as a treatment for tinnitus on the evidence of rigorous randomized controlled trials. (2000) N
  43. We conclude that acupuncture continues to be associated with occasional, serious adverse events and fatalities. These events have no geographical limits. Most of these events are due to negligence. Everyone concerned with setting standards, delivering training, and maintaining competence in acupuncture should familiarise themselves with the lessons to be learnt from these untoward events. (2000) N
  44. Overall, the existing evidence suggests that acupuncture has a role in the treatment of recurrent headaches. However, the quality and amount of evidence is not fully convincing. There is urgent need for well-planned, large-scale studies to assess effectiveness and efficiency of acupuncture under real life conditions. (1999) N/P
  45. While the frequency of adverse effects of acupuncture is unknown and they may be rare, knowledge of normal anatomy and anatomical variations is essential for safe practice and should be reviewed by regulatory bodies and those responsible for training courses. (1999) N
  46.  In conclusion, the hypothesis that acupuncture is efficacious in the treatment of neck pain is not based on the available evidence from sound clinical trials. Further studies are justified. (1999) N
  47. Even though all studies are in accordance with the notion that acupuncture is effective for temporomandibular joint dysfunction, this hypothesis requires confirmation through more rigorous investigations. (1999) N
  48. Acupuncture is not free of risks. All adverse events reported in 1997 would have been avoidable. The absolute number of cases is small, but the degree of underreporting remains unknown. (1999) N
  49. This form of electroacupuncture is no more effective than placebo in reducing nicotine withdrawal symptoms. (1998) N
  50. Acupuncture was shown to be superior to various control interventions, although there is insufficient evidence to state whether it is superior to placebo. (1998) N/P
  51. Considerable variation was observed in the scores awarded by the acupuncture experts. (1998) N
  52. It is therefore concluded that, according to the data published to date, the evidence that acupuncture is a useful adjunct for stroke rehabilitation is encouraging but not compelling. More and better trials are required to clarify this highly relevant issue. (1996) N

The results are remarkable (particularly considering that one would not expect unbiased studies or reviews of acupuncture to generate plenty of positive conclusions):

0 times N, 5 times N/P, 22 times P – after Adrian had left my department,

17 times N, 7 times N/P, 0 times P – while Adrian worked in my department.

From these figures, it is tempting to calculate the ratios for both periods of negative : positive conclusions:

zero versus infinite

If that is not impressive, I don’t know what is!

Looking just at the positive and the negative papers over the years:

One could discuss these papers in more detail, but I think this is hardly necessary. Just a few highlights perhaps: look at articles No 5, 20 and 27 for examples of turning an essentially negative finding into a positive conclusion. Notice that Adrian conducted a clinical trial of acupuncture for smoking cessation (No 49) while working with me and later published uncritical positive reviews on the subject. Does this not indicate that he distrusted his own study because it had not generated the result he had hoped for?

Of course, my analysis is merely a case study and therefore my findings are not generalisable. However, in my personal experience, the described phenomenon is by no means an exception in SCAM research. I have observed similar phenomena over and over again. Just look at the ALTERNATIVE MEDICINE HALL OF FAME that I created for this blog:

But Adrian’s case might be unique because it allows us to make a longitudinal observation over two decades. And it suggests to me that an ideological bias can (and often is) so strong and indistructable that is re-emerges as soon as it is no longer kept under strict control.

I have long suspected that ideological conflicts of interest have a much more powerful influence in SCAM research than financial ones. Such an overpowering influence might even be characteristic to much of SCAM research. And because it can be so dominant, it seems important to know about. People reading research need to be aware that it originates from a biased source, and funders who finance research would be wise to think twice about supporting researchers who are likely to generate findings that are biased and therefore false-positive. In the final analysis, such research is worse than no research at all.

Thirty years ago, I had just been appointed chair of PHYSICAL MEDICINE AND REHABILITATION at the University of Vienna and was about to move – as the first clinical department – into the brand new AKH (General Hospital) of Vienna thus gradually enlarging the team I had taken over from about 20 to 120 co-workers. During this period, I found little time to do original research; however, I did manage to finally write up and publish a study, we had conducted several years before while I was still in Munich. As it is (almost) on the subject of so-called alternative medicine (SCAM), and as it relates to the prevention of a viral infection, I think it might be of interest to give it another outing.

Here is its abstract:

The high morbidity of common colds means that their economic importance is considerable, with colds causing more loss of productivity than any other infection. As no effective prophylaxis is available, this trial was to test the hypothesis that sauna bathing can reduce the incidence of common colds. Twenty-five volunteers were submitted to sauna bathing, with 25 controls abstaining from this or comparable procedures. In both groups the frequency, duration and severity of common colds were recorded for six months. There were significantly fewer episodes of common cold in the sauna group. This was found particularly during the last three months of the study period when the incidence was roughly halved compared to controls. The mean duration and average severity of common colds did not differ significantly between the groups. It is concluded that regular sauna bathing probably reduces the incidence of common colds, but further studies are needed to prove this.

In the discussion section of the paper, we stated the following:

Preventive methods with comparable efficacy have not been described in the literature (2, 12). Vitamin C is of doubtful value (7, 15, 16); vaccination is not feasible since far too many virus types exist (2, 17); virucidal kerchiefs are effective (8, 18) but not available commercially and protect only the environment of a common cold sufferer rather than the sufferer him-/herself.

I believe most of this is still true today (but I might be wrong, as I did not keep up with this particular line of research). Re-reading the paper, I find that our trial was far from optimal:

  • we had to conduct it with zero funding,
  • it was small,
  • it was not randomised,
  • it lacked objective endpoints.

Anyway, sauna bathing is most agreeable, and I can recommend it just for this reason. However, I would doubt that public saunas are a good idea during the present health crisis.

Please, stay safe!

We are all prone to fall victim to the ‘post hoc ergo propter hoc’ fallacy. It describes the erroneous assumption that something that happened after an event was cased by that event. The fallacy is essentially due to confusing correlation with causation:

  • the sun does not rise because the rooster has crowed;
  • yellow colouring of the 2nd and 3rd finger of a smoker is not the cause of lung cancer;
  • some children developing autism after vaccinations does not mean that autism is caused by vaccination.

As I said, we are all prone to this sort of thing, even though we know better. Scientists, journal editors and reviewers of medical papers, however, should not allow themselves to be fooled by overt cases of the ‘post hoc ergo propter hoc’ fallacy. And if they do, they have lost all credibility – just like the individuals involved in a recent paper on animal homeopathy.

Pododermatitis in penguins usually occurs after changes in normal activity that result from being held captive. It is also called ‘bumlefoot’ (which fails to reflect the seriousness of the condition) and amounts to one of most frequent and important clinical complications in penguins kept in captivity or in rehabilitation centres.

This veterinary case study reports the use of oral homeopathic treatment on acute and chronic pododermatitis in five Magellanic penguins in a zoological park setting. During treatment, the patients remained in the penguins’ living area, and the effect of the treatment on the progression of their lesions was assessed visually once weekly. The treatment consisted of a combination of Arnica montana and Calcarea carbonica.

After treatment, the appearance of the lesions had noticeably improved: in the majority of penguins there was no longer evidence of infection or edema in the feet. The rate of recovery depended on the initial severity of the lesion. Those penguins that still showed signs of infection nevertheless exhibited a clear diminution of the size and thickness of the lesions. Homeopathic treatment did not cause any side effects.

The authors concluded that homeopathy offers a useful treatment option for pododermatitis in captive penguins, with easy administration and without side effects.

So, the homeopathic treatment happened before the recovery and, according to the ‘post hoc ergo propter hoc’ fallacy, the recovery must have been caused by the therapy!

I know, this is a tempting conclusion for a lay person, but it is also an unjustified one, and the people responsible for this paper are not lay people. Pododermitis does often disappear by itself, particularly if the hygenic conditions under which the penguins had been kept are improved. In any case, it is a potentially life-threatening condition (a bit like an infected bed sore in an immobilised human patient) that can be treated, and one should certainly not let a homeopath deal with it.

I think that the researchers who wrote the article, the journal editor who accepted it for publication, and the referees who reviewed the paper should all bow their heads in shame and go on a basic science course (perhaps a course in medical ethics as well) before they are let anywhere near research again.

In 2012, we published a systematic review of adverse effects of homeopathy. Here is its abstract:

Aim: The aim of this systematic review was to critically evaluate the evidence regarding the adverse effects (AEs) of homeopathy.

Method: Five electronic databases were searched to identify all relevant case reports and case series.

Results: In total, 38 primary reports met our inclusion criteria. Of those, 30 pertained to direct AEs of homeopathic remedies; and eight were related to AEs caused by the substitution of conventional medicine with homeopathy. The total number of patients who experienced AEs of homeopathy amounted to 1159. Overall, AEs ranged from mild-to-severe and included four fatalities. The most common AEs were allergic reactions and intoxications. Rhus toxidendron was the most frequently implicated homeopathic remedy.

Conclusion: Homeopathy has the potential to harm patients and consumers in both direct and indirect ways. Clinicians should be aware of its risks and advise their patients accordingly.

The paper prompted a number of angry reactions from proponents of homeopathy who claimed, for instance, that homeopathic remedies are highly diluted and thus safe. We responded that homeopaths can nevertheless be dangerous to patients through neglect and bad advice by homeopaths, and that not all homeopathic remedies are highly diluted, and that some might be toxic because of poor quality control of the manufacturing process.

Now, a different group of researchers have looked at the problem from a slightly different angle and with different methodologies. This systematic review and meta-analysis by researchers from NAFKAM focused on observational studies, as a substantial amount of the research base for homeopathy are observational.

Eight electronic databases, central webpages and journals were searched for eligible studies, and a total of 1,169 studies were identified, 41 were included in this review. Eighteen studies were included in a meta-analysis that made an overall comparison between homeopathy and control (conventional medicine and herbs).

Eighty-seven percent (n = 35) of the studies reported adverse effects. They were graded as CTCAE 1, 2 or 3 and equally distributed between the intervention and control groups. Homeopathic aggravations (homeopaths believe that, when the optimal remedy is given, patients will experience an aggravation of their presenting symptoms) were reported in 22,5% (n = 9) of the studies and graded as CTCAE 1 or 2. The frequency of adverse effects for control versus homeopathy was statistically significant (P < 0.0001). Analysis of sub-groups indicated that, compared to homeopathy, the number of adverse effects was significantly higher for conventional medicine (P = 0.0001), as well as other complementary therapies (P = 0.05).

The authors concluded that adverse effects of homeopathic remedies are consistently reported in observational studies, while homeopathic aggravations are less documented. This meta-analysis revealed that the proportion of patients experiencing adverse effects was significantly higher when receiving conventional medicine and herbs, compared to patients receiving homeopathy. Nonetheless, the development and implementation of a standardized reporting system of adverse effects in homeopathic studies is warranted in order to facilitate future risk assessments.

While these results are interesting, they have to be taken with a pinch of salt and beg a number of questions:

  • Is there proof that aggravations exist at all?
  • How can one differentiate them from adverse effects?
  • As even placebos are known to cause adverse effects (nocebo effects), how can one be sure that the adverse effects of homeopathy are not nocebo effects?
  • Is it a good reason to focus on largely inconclusive observational studies, because a substantial amount of the research base for homeopathy are observational?
  • Can one produce conclusive results by meta-analysing inconclusive studies?

For me, the most impressive findings of this review is that in total 86 studies had to be excluded by the authors because they reported no adverse effects or aggravations. I think this renders the interpretation of the evidence from the 41 studies they did include even more flimsy. In fact, I don’t see how any meaningful conclusion can be drawn at all – except of course that many researchers of homeopathy violate the rules of research ethics by not reporting adverse effects in their studies.

As to aggravations, we clearly need to rely on placebo controlled studies, if we want to find out whether they exist at all. This we have done in our 2003 paper:

Homeopathic aggravations have often been described anecdotally. However, few attempts have been made to scientifically verify their existence. This systematic review aimed at comparing the frequency of homeopathic aggravations in the placebo and verum groups of double-blind, randomised clinical trials. Eight independent literature searches were carried out to identify all such trials mentioning either adverse effects or aggravations. All studies thus found were validated and data were extracted by both authors. Twenty-four trials could be included. The average number of aggravations was low. In total, 50 aggravations were attributed to patients treated with placebo and 63 to patients treated with homoeopathically diluted remedies. We conclude that this systematic review does not provide clear evidence that the phenomenon of homeopathic aggravations exists.

What is interesting, from my perspective, is the fact that the NAFKAM authors chose to ignore our 2012 paper completely (even though it is highly relevant to their paper and was not published in an obscure journal) and elected to completely misinterpret the findings of our 2003 paper (stating this about it: Grabia and Ernst reported a total of 103 cases of homeopathic aggravations in 3437 participants (3%), while, in fact, our paper demonstrated that aggravations are a homeopathic figment of imagination).

I wonder why.

In the past, NAFKAM did not have the reputation of doing research that was overtly biased towards homeopathy. Recently, the head of the team retired and was replaced by Miek C. Jong who is a co-author of the present review (plus head of CAMcancer, an organisation of which I am a founding member and which did, I think, some good work in the past). She happens to have a long history as a homeopath or homeopathic researcher and is co-author of many papers in this area. Here are three of her conclusions:

Could it be that, within NAFKAM, the attitude towards homeopathy has changed?

Bloodletting has been used for centuries in many cultures. Its principle, it was assumed, consisted in re-balancing the body’s four humours. Bloodletting had a detrimental effect on most diseases and must have killed millions. It is a good historical example of the harm that ensues, if healthcare adheres to dogma. Today, we know that bloodletting is useful only in rare conditions such as polycythaemia vera or haemochromatosis (and some 30 years ago, a variation of bloodletting, isovolaemic haemodilution, was being discussed as a treatment for circulatory diseases such as intermittent claudication or stroke).

Yet, in so-called alternative medicine (SCAM), there are some practitioners who seem to find it hard to concede that ancient treatments might be not as good as they think. Thus, bloodletting has survived in this realm as a therapy for a wide range of conditions. This study assessed the efficacy of bloodletting therapy (acupoint pricking and cupping) in patients with chronic idiopathic urticaria (CIU) in a randomized, control, parallel-group trial.

A total of 174 patients with CIU were randomized into three groups:

  1. group A was treated with bloodletting therapy and ebastine (an anti-histamine),
  2. group B was treated with placebo treatment (acupoint pseudopricking and cupping) and ebastine,
  3. group C was treated with ebastine only.

The treatment period lasted 4 weeks. An intention-to-treat analysis was conducted, and the primary outcome was the effective rate of UAS7 score being reduced to 7 or below after treatment phase.

The effective rates at the end of treatment phase were different among the three groups, which were

  • 73.7% in group A,
  • 45.6% in group B,
  • and 42.9% in group C.

Multiple analysis indicated differences between groups A and B (P < 0.0125) and groups A and C (P < 0.0125) and no difference between groups B and C (P > 0.0125). No severe bloodletting therapy-related adverse events were observed.

The authors concluded that one month of bloodletting therapy combined with ebastine is clinically beneficial compared with placebo treatment combined with ebastine and treatment with ebastine only. Thus, bloodletting therapy can be an effective complementary treatment in CIU.

Amazed?

Me too!

How on earth might bloodletting help for CIU? Luckily, the authors have an answer to this question:

The clinical feature of urticaria with wheals and pruritus coming and going quickly is the manifestation of wind-evil that lurks in and circulates with blood. Hence, in the treatment of urticaria, dispersing wind is the one of the principle methods, and treating blood before wind is an important procedure because when blood flows fluently , wind-evil will resolve spontaneously. Bloodletting therapy is a direct and effective way of regulating blood.

You see, it’s all perfectly clear!

In this case, the results must be true. And the argument that patients might have known in which treatment group they had ended up (and were thus not blinded) can be discarded.

Acupuncture-moxibustion therapy (AMT) is a so-called alternative medicine (SCAM) that has been used for centuries in treatment of numerous diseases. Some enthusiasts even seem to advocate it for chemotherapy-induced leukopenia (CIL)  The purpose of this review was to evaluate the efficacy and safety of acupuncture-moxibustion therapy in treating CIL.

Relevant studies were searched in 9 databases up to September 19, 2020. Two reviewers independently screened the studies for eligibility, extracted data, and assessed the methodological quality of selected studies. Meta-analysis of the pooled mean difference (MD) and risk ratio (RR) with their respective 95% confidence intervals (CI) were calculated.

Seventeen studies (1206 patients) were included, and the overall quality of the included studies was moderate. In comparison with medical therapy, AMT has a better clinical efficacy for CIL (RR, 1.24; 95% CI, 1.17-1.32; P < 0.00001) and presents advantages in increasing leukocyte count (MD, 1.10; 95% CI, 0.67-1.53; P < 0.00001). Also, the statistical results show that AMT performs better in improving the CIL patients’ Karnofsky performance score (MD, 5.92; 95% CI, 3.03-8.81; P < 0.00001).

The authors concluded that this systematic review and meta-analysis provides updated evidence that AMT is a safe and effective alternative for the patients who suffered from CIL.

A CIL is a serious complication. If I ever were afflicted by it, I would swiftly send any acupuncturist approaching my sickbed packing.

But this is not an evidence-based attitude!!!, I hear some TCM-fans mutter. What more do you want that a systematic review showing it works?

I beg to differ. Why? Because the ‘evidence’ is hardly what critical thinkers can accept as evidence. Have a look at the list of the primary studies included in this review:

  1. Lin Z. T., Wang Q., Yu Y. N., Lu J. S. Clinical observation of post-chemotherapy-leukopenia treated with ShenMai injectionon ST36. World Journal of Integrated Traditional and Western Medicine2010;5(10):873–876. []
  2. Wang H. Clinical Observation of Acupoint Moxibustion on Leukopenia Caused by Chemotherapy. Beijing, China: Beijing University of Chinese Medicine; 2011. []
  3. Fan J. Y. Coupling of Yin and Yang between Ginger Moxibustion Improve the Clinical Effect of the Treatment of Chemotherapy Adverse Reaction. Henan, China: Henan University of Chinese Medicine; 2013. []
  4. Lu D. R., Lu D. X., Wei M., et al. Acupoint injection with addie injection for patients of nausea and vomiting with cisplatin induced by chemotherapy. Journal of Clinical Acupuncture and Moxibustion2013;29(10):33–38. []
  5. Yang J. E. The Clinical Observation on Treatment of Leukopenia after Chemotherapy with Needle Warming Moxibustion. Hubei, China: Hubei University of Chinese Medicine; 2013. []
  6. Fu Y. H., Chi C. Y., Zhang C. Y. Clinical effect of acupuncture and moxibustion on leukopenia after chemotherapy of malignant tumor. Guide of China Medicine2014;12(12) []
  7. Wang J. N., Zhang W. X., Gu Q. H., Jiao J. P., Liu L., Wei P. K. Protection of herb-partitioned moxibustion on bone marrow suppression of gastric cancer patients in chemotherapy period. Chinese Archives of Traditional Chinese Medicine2014;32(12):110–113. []
  8. Zhang J. The Clinical Research on Myelosuppression and Quality of Life after Chemotherapy Treated by Grain-Sized Moxibustion. Nanjing, China: Nanjing University of Chinese Medicine; 2014. []
  9. Tian H., Lin H., Zhang L., Fan Z. N., Zhang Z. L. Effective research on treating leukopenia following chemotherapy by moxibustion. Clinical Journal of Chinese Medicine2015;7(10):35–38. []
  10. Hu G. W., Wang J. D., Zhao C. Y. Effect of acupuncture on the first WBC reduction after chemotherapy for breast cancer. Beijing Journal of Traditional Chinese Medicine2016;35(8):777–779. []
  11. Zhu D. L., Lu H. Y., Lu Y. Y., Wu L. J. Clinical observation of Qi-blood-supplementing needling for leukopenia after chemotherapy for breast cancer. Shanghai Journal of Acupuncture and Moxibustion2016;35(8):964–966. []
  12. Chen L, Xu G. Y. Observation on the prevention and treatment of chemotherapy-induced leukopenia by moxibustion therapy. Zhejiang Journal of Traditional Chinese Medicine2016;51(8):p. 600. []
  13. Mo T., Tian H., Yue S. B., Fan Z. N., Zhang Z. L. Clinical observation of acupoint moxibustion on leukocytopenia caused by tumor chemotherapy. World Chinese Medicine2016;11(10):2120–2122. []
  14. Nie C. M. Nursing observation of acupoint moxibustion in the treatment of leucopenia after chemotherapy. Today Nurse2017;4:93–95. []
  15. Wang D. Y. Clinical Research on Post-chemotherapy-leukopenia with Spleen-Kidney Yang Deficiency in Colorectal Cancer Treated with Point-Injection. Yunnan, China: Yunnan University of Chinese Medicine; 2017. []
  16. Gong Y. Q, Zhang M. Q, Zhang B. C. Prevention and treatment of leucocytopenia after chemotherapy in patients with malignant tumor with ginger partitioned moxibustion. Chinese Medicine Modern Distance Education of China2018;16(21):135–137. []
  17. Li Z. C., Lian M. J., Miao F. G. Clinical observation of fuzheng moxibustion combined with wenyang shengbai decoction in the treatment of 80 cases of leukopenia after chemotherapy. Hunan Journal of Traditional Chinese Medicine2019;35(3):64–66. []

Notice anything peculiar?

  • The studies are all from China where data fabrication was reported to be rife.
  • They are mostly unavailable for checking (why the published adds links that go nowhere is beyond me).
  • Many do not look at all like randomised clinical trials (which, according to the authors, was an inclusion criterion).
  • Many do not look as though their primary endpoint was the leukocyte count (which, according to the authors, was another inclusion criterion).

Intriguingly, the authors conclude that AMT is not just effective but also ‘safe’. How do they know? According to their own data extraction table, most studies failed to mention adverse effects. And how exactly is acupuncture supposed to increase my leukocyte count? Here is what the authors offer as a mode of action:

Based on the theory of traditional Chinese medicine (TCM), CIL belongs to the category of consumptive disease, owing to the exhaustion of genuine qi in the zang-fu viscera and the insufficiency of kidney essence and qi-blood. Researchers believe that there is an intimate association between the occurrence of malignant tumors and the deficiency of genuine qi. During attacking the cancer cells, chemotherapeutics also damaged the function of zang-fu viscera and qi-blood, leading to CIL. According to the theory of TCM and meridian, acupuncture-moxibustion is an ancient therapeutic modality that may be traced back more than 3500 years in China. Through meridian conduction, acupuncture-moxibustion therapy stimulates acupoints to strengthen the condition of zang-fu viscera and immune function, supporting genuine qi to improve symptoms of consumption.

I think it is high time that we stop tolerating that the medical literature gets polluted with such nonsense (helped, of course, by journals that are beyond the pale) – someone might actually believe it, in which case it would surely hasten the death of vulnerable patients.

The amount of different so-called alternative medicines (SCAMs) that are being tried or promoted against COVID-19 is legion. Anything really from vitamins to herbal remedies, homeopathics to chiropractic. In fact, it is hard these days to find a SCAM that is not touted for COVID-19.

This study aimed to evaluate if a dietary supplement of quercetin (a polyphenol contained in many fruit and vegetables), vitamin C and bromelain (a proteolytic enzyme contained in pineapple) could be protective against coronavirus infections.

In the verum group, a supplement containing

  • 500mg of quercetin,
  • 500mg of vitamin C,
  • 50mg of  bromelain (QCB)

was administered daily in 2 divided doses for 71 healthcare workers working in areas with high risk of COVID-19, whereas 42 were the control group who received no supplements. The maximum period of follow-up was 120 days. Termination of QCB use prematurely or having a coronavirus infection was the end of a volunteer’s study participation. A rapid diagnostic test was used to detect immunoglobulin positivity.

Graphic demonstrated survival without COVID-19 during follow up time between groups

Graphic demonstrated survival without COVID-19 during follow up time between groups

A total of 113 persons were included. No significant difference were detected between groups at baseline. Mean age of QCB group was 39.0 ± 8.8 years and control group was 32.9 ± 8.7. Average follow-up period for the QCB group was 113 days, and for the control group, 118 days. During the follow-up period, 1 healthcare worker in the QCB group and 9 out in 42 in control group contracted COVID-19. One case was asymptomatic, while others were not. Transmission risk hazard ratio of participants who did not receive QCB was 12.04 (95% Confidence interval= 1.26-115.06, P = 0.031). No significant effect of gender, smoking, antihypertensive medication exposure and having chronic disease on rate of transmission. The authors concluded that this study revealed that QCB was protective for healthcare workers.

The sudy is so poorly written and reported that I had trouble making sense of it. In fact, I first thought it was a fake. Then I saw this note:

Preprints with The Lancet is part of SSRN´s First Look, a place where journals identify content of interest prior to publication. Authors have opted in at submission to The Lancet family of journals to post their preprints on Preprints with The Lancet. The usual SSRN checks and a Lancet-specific check for appropriateness and transparency have been applied. Preprints available here are not Lancet publications or necessarily under review with a Lancet journal. These preprints are early stage research papers that have not been peer-reviewed. The findings should not be used for clinical or public health decision making and should not be presented to a lay audience without highlighting that they are preliminary and have not been peer-reviewed.

If the results are for real (because of the small sample size, the lack of a placebo-control, dozens of potential confounders, etc., the findings could easily false-positive), they would merit urgent replication in a larger, more rigorous trial.

And meanwhile?

Meanwhile I would be very sceptical about the validity of the results. The paper (it really is just a submission for publication in the Lancet; I am not even sure that it will be officially published and I don’t quite see why it is being made available to the public in this way) is too flimsy for words. Despite these warnings, it is likely that many consumers will fall for the claim that QCB was protective for healthcare workers. 

There is some encouraging evidence regarding the positive influence of vitamin D on COVID-19. But is it convincing? Is it causal? As always, it is worth looking at the totality of the reliable evidence.

In this systematic review and meta-analysis, the researchers analyze the association between vitamin D deficiency and COVID-19 severity. They conducted an analysis of the prevalence of vitamin D deficiency and insufficiency in people with the disease. Five online databases—Embase, PubMed, Scopus, Web of Science, ScienceDirect and pre-print Medrevix were searched. The inclusion criteria were observational studies measuring serum vitamin D in adult and elderly subjects with COVID-19. The main outcome was the prevalence of vitamin D deficiency in severe cases of COVID-19.

The researchers identified 1542 articles and 27 met their inclusion criteria. The results show that

  • vitamin D deficiency was not associated with a higher chance of infection by COVID-19,
  • severe cases of COVID-19 present 64% more vitamin D deficiency compared with mild cases,
  • vitamin D concentration insufficiency increased hospitalization and mortality rates,
  • There was a positive association between vitamin D deficiency and the severity of the disease.

The authors concluded that the results of the meta-analysis confirm the high prevalence of vitamin D deficiency in people with COVID-19, especially the elderly. We should add that vitamin D deficiency was not associated with COVID-19 infection. However, we observed a positive association between vitamin D deficiency and the severity of the disease. From this perspective, evaluating blood vitamin D levels could be considered in the clinical practice of health professionals. Moreover, vitamin D supplementation could be considered in patients with vitamin D deficiency and insufficiency, if they have COVID-19. However, there is no support for supplementation among groups with normal blood vitamin D values with the aim of prevention, prophylaxis or reducing the severity of the disease.

These are interesting findings, no doubt. They relate to associations, as the authors repeatedly stress in the text of the paper. They do not, however, signify cause and effect relationships. The principal outcome of this research should be a hypothesis that subsequently needs testing in clinical trials.

So, why on earth did the authors chose that seriously misleading title of their paper? It clearly implies a causal effect; and this can only be verified by conducting clinical trials. One such study has been published (as discussed here) and it concluded that administration of calcifediol may improve the clinical outcome of subjects requiring hospitalization for COVID-19.

My conclusion: it seems well worth conducting more and more rigorous clinical trials.

1 2 3 19
Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted but you must tick the box: “Save my name, email, and website in this browser for the next time I comment.”

The most recent comments from all posts can be seen here.

Archives
Categories