MD, PhD, FMedSci, FRSB, FRCP, FRCPEd.

study design

1 2 3 12

It is time, I think, to express my gratitude to Dr Jens Behnke, a German homeopath employed by the pro-homeopathy lobby group the ‘Carstens Stiftung’, who diligently tweets trials of homeopathy which he obviously believes prove the value of his convictions.

The primary objective of this new study was to evaluate the efficacy of homoeopathy for women suffering from polycystic ovary syndrome. This condition is characterised by:

  • irregular periods which means your ovaries don’t regularly release eggs,
  • abnormally high levels of male hormones in the body, which may cause physical signs such as excess facial or body hair,
  • polycystic ovaries – ovaries become enlarged and contain many fluid-filled sacs (follicles) which surround the eggs.

There’s no cure for PCOS, but the symptoms can usually be treated. As so often in such situations, homeopaths are happy to step into the fray.

This single-blind, randomised, placebo-controlled pilot study was conducted at two research centres in India. The cases fulfilling the eligibility criteria were enrolled (n = 60) and randomised to either the homoeopathic intervention (HI) (n = 30) or placebo (P) (n = 30) with uniform lifestyle modification (LSM) for 6 months.

The menstrual regularity with improvement in other signs/symptoms was observed in 60% of the cases (n = 18) in HI + LSM group and none (n = 0) in control group. Statistically significant difference was observed in the reduction of intermenstrual duration in HI + LSM in comparison to placebo + LSM group. Significant improvements were also observed in HI+LSM group in domains of weight, fertility, emotions and menstrual problems. No change was observed in respect of improvement in the ultrasound findings. Pulsatilla was the most frequently indicated homeopathic remedy.

The authors concluded that HI along with LSM has shown promising outcome; further comparative study with standard conventional treatment on adequate sample size is desirable.

This trial might convince believers (mostly because they do not even need convincing), but it cannot convince anybody capable of critical thinking. Here is why:

  • According to its authors, this trial was a pilot study; this means it should not report any results and merely focus on the feasibility of a definitive trial.
  • Researchers were not blinded, meaning that they might have influenced the outcome in more than one way.
  • The primary endpoint was subjective and could have been influenced by the non-blinded researchers.
  • 0% success rate in achieving the primary endpoint in the placebo group is not plausible.
  • Compliance to LSM was not checked; as the homeopathy group lost more weight, these patients seemed to have complied better (probably due to being better motivated by the non-blinded researchers).

So?

My conclusion is not very original but all the more true: POORLY DESIGNED STUDIES USUALLY GENERATE UNRELIABLE RESULTS. 

For some researchers, the question whether homeopathy works beyond a placebo effect is not as relevant as the question whether it works as well as an established treatment. To answer it, they must conduct RCTs comparing homeopathy with a therapy that has been shown beyond reasonable doubt to be effective, i.e better than placebo. Such a drug is, for instance, Ibuprofen.

The purpose of this study was to compare the efficacy of Ibuprofen and homeopathic Belladonna for orthodontic pain. 51 females and 21 males, were included in this study. Cases with non-extraction treatment plan having proper contacts’ mesial and distal to permanent first molar and currently not taking any analgesics or antibiotics were included in the study. They were randomly divided into two groups; one group was assigned to ibuprofen 400 mg and second group took Belladonna 6C (that’s a dilution of 1: 1000000000000). Patients were given two doses of medication of their respective remedies one hour before placement of elastomeric separators (Ormco Separators, Ormco Corporation, CA, USA) and one dose 6 h after the placement. Pain scores were recorded on a visual analogue scale (VAS) 2 h after placement, 6 h after placement, bedtime, day 1 morning, day 2 morning, day 3 morning and day 5 morning.

The comparisons showed that there were no differences between the two groups at any time point.

(Mean visual analogue scale pain score at different time intervals after separator placement in Ibuprofen and Belladonna group)

The authors concluded that Ibuprofen and Belladonna 6C are effective and provide adequate analgesia with no statistically significant difference. Lack of adverse effects with Belladonna 6C makes it an effective and viable alternative.

FINALLY, THE PROOF HOMEOPATHS HAVE BEEN WAITING FOR: HOMEOPATHY DOES WORK AFTER ALL!

Not so fast – before we draw any conclusions, let’s have a closer look at this study. Here are a few of its limitations (apart from the fact that it was published in a journal that does not exactly belong to the ‘crème de la crème’ of medical publications):

  • Patients obviously knew which group they were assigned to; thus their expectations would have influenced the outcome.
  • The same applies to the researchers (the study could have been ‘blind’ using a ‘double dummy’ method, but the researchers did not use it).
  • The study was an equivalence trial (it did not test whether homeopathy is superior to placebo, but whether its effects are equivalent to Ibuprofen); such studies need sample sizes that are about one dimension larger than was the case here.

Therefore, all this trial does demonstrate that the sample was too small for an existing group difference in favour of Ibuprofen to show.

So sorry, my homeopathic friends!

Ginkgo biloba is a well-researched herbal medicine which has shown promise for a number of indications. But does this include coronary heart disease?

The aim of this systematic review was to provide information about the effectiveness and safety of Ginkgo Leaf Extract and Dipyridamole Injection (GD) as one adjuvant therapy for treating angina pectoris (AP) and to evaluate the relevant randomized controlled trials (RCTs) with meta-analysis. (Ginkgo Leaf Extract and Dipyridamole Injection is a Chinese compound preparation, which consists of ginkgo flavone glycosides (24%), terpene lactones (ginkgolide about 13%, ginkgolide about 2.9%) and dipyridamole.)

RCTs concerning AP treated by GD were searched and the Cochrane Risk Assessment Tool was adopted to assess the methodological quality of the RCTs. A total of 41 RCTs involving 4,462 patients were included in the meta-analysis. The results indicated that the combined use of GD and Western medicine (WM) against AP was associated with a higher total effective rate [risk ratio (RR)=1.25, 95% confidence interval (CI): 1.21–1.29, P<0.01], total effective rate of electrocardiogram (RR=1.29, 95% CI: 1.21–1.36, P<0.01). Additional, GD combined with WM could decrease the level of plasma viscosity [mean difference (MD)=–0.56, 95% CI:–0,81 to–0.30, P<0.01], fibrinogen [MD=–1.02, 95% CI:–1.50 to–0.54, P<0.01], whole blood low shear viscosity [MD=–2.27, 95% CI:–3.04 to–1.49, P<0.01], and whole blood high shear viscosity (MD=–0.90, 95% CI: 1.37 to–0.44, P<0.01).

The authors concluded that comparing with receiving WM only, the combine use of GD and WM was associated with a better curative effect for patients with AP. Nevertheless, limited by the methodological quality of included RCTs more large-sample, multi-center RCTs were needed to confirm our findings and provide further evidence for the clinical utility of GD.

If one reads this conclusion, one might be tempted to use GD to cure AP. I would, however, strongly warn everyone from doing so. There are many reasons for my caution:

  • All the 41 RCTs originate from China, and we have repeatedly discussed that Chinese TCM trials are highly unreliable.
  • The methodological quality of the primary RCTs was, according to the review authors ‘moderate’. This is not true; it was, in fact, lousy.
  • Dipyridamole is not indicated in angina pectoris.
  • To the best of my knowledge, there is no good evidence from outside China to suggest that Ginkgo biloba is effective for angina pectoris.
  • Angina pectoris is caused by coronary artery disease (a narrowing of one or more coronary arteries due to atherosclerosis), and it seems implausible that this condition can be ‘cured’ with any medication.

So, what we have here is yet another nonsensical paper, published in a dubious journal, employing evidently irresponsible reviewers, run by evidently irresponsible editors, hosted by a seemingly reputable publisher (Springer). This is reminiscent of my previous post (and many posts before). Alarmingly, it is also what I encounter on a daily basis when scanning the new publications in my field.

The effects of this incessant stream of nonsense can only have one of two effects:

  1. People take this ‘evidence’ seriously. In this case, many patients might pay with their lives for this collective incompetence.
  2. People conclude that alt med research cannot be taken seriously. In this case, we are unlikely to ever see anything useful emerging from it.

Either way, the result will be profoundly negative!

It is high time to stop this idiocy; but how?

I wish, I knew the answer.

This systematic review was aimed at evaluating the effects of acupuncture on the quality of life of migraineurs.  Only randomized controlled trials that were published in Chinese and English were included. In total, 62 trials were included for the final analysis; 50 trials were from China, 3 from Brazil, 3 from Germany, 2 from Italy and the rest came from Iran, Israel, Australia and Sweden.

Acupuncture resulted in lower Visual Analog Scale scores than medication at 1 month after treatment and 1-3 months after treatment. Compared with sham acupuncture, acupuncture resulted in lower Visual Analog Scale scores at 1 month after treatment.

The authors concluded that acupuncture exhibits certain efficacy both in the treatment and prevention of migraines, which is superior to no treatment, sham acupuncture and medication. Further, acupuncture enhanced the quality of life more than did medication.

The authors comment in the discussion section that the overall quality of the evidence for most outcomes was of low to moderate quality. Reasons for diminished quality consist of the following: no mentioned or inadequate allocation concealment, great probability of reporting bias, study heterogeneity, sub-standard sample size, and dropout without analysis.

Further worrisome deficits are that only 14 of the 62 studies reported adverse effects (this means that 48 RCTs violated research ethics!) and that there was a high level of publication bias indicating that negative studies had remained unpublished. However, the most serious concern is the fact that 50 of the 62 trials originated from China, in my view. As I have often pointed out, such studies have to be categorised as highly unreliable.

In view of this multitude of serious problems, I feel that the conclusions of this review must be re-formulated:

Despite the fact that many RCTs have been published, the effect of acupuncture on the quality of life of migraineurs remains unproven.

 

I only recently came across this review; it was published a few years ago but is still highly relevant. It summarizes the evidence of controlled clinical studies of TCM for cancer.

The authors searched all the controlled clinical studies of TCM therapies for all kinds of cancers published in Chinese in four main Chinese electronic databases from their inception to November 2011. They found a total of 2964 reports (involving 253,434 cancer patients) including 2385 randomized controlled trials and 579 non-randomized controlled studies.

The top seven cancer types treated were lung cancer, liver cancer, stomach cancer, breast cancer, esophagus cancer, colorectal cancer and nasopharyngeal cancer by both study numbers and case numbers. The majority of studies (72%) applied TCM therapy combined with conventional treatment, whilst fewer (28%) applied only TCM therapy in the experimental groups. Herbal medicine was the most frequently applied TCM therapy (2677 studies, 90.32%). The most frequently reported outcome was clinical symptom improvement (1667 studies, 56.24%) followed by biomarker indices (1270 studies, 42.85%), quality of life (1129 studies, 38.09%), chemo/radiotherapy induced side effects (1094 studies, 36.91%), tumour size (869 studies, 29.32%) and safety (547 studies, 18.45%).

The authors concluded that data from controlled clinical studies of TCM therapies in cancer treatment is substantial, and different therapies are applied either as monotherapy or in combination with conventional medicine. Reporting of controlled clinical studies should be improved based on the CONSORT and TREND Statements in future. Further studies should address the most frequently used TCM therapy for common cancers and outcome measures should address survival, relapse/metastasis and quality of life.

This paper is important, in my view, predominantly because it exemplifies the problem with TCM research from China and with uncritical reviews on this subject. If a cancer patient, who does not know the background, reads this paper, (s)he might think that TCM is worth trying. This conclusion could easily shorten his/her life.

The often-shown fact is that TCM studies from China are not reliable. They are almost invariably positive, their methodological quality is low, and they are frequently based on fabricated data. In my view, it is irresponsible to publish a review that omits discussing these facts in detail and issuing a stark warning.

TCM FOR CANCER IS A VERY BAD CHOICE!

In the latest issue of ‘Simile’ (the Faculty of Homeopathy‘s newsletter), the following short article with the above title has been published. I took the liberty of copying it for you:

Members of the Faculty of Homeopathy practising in the UK have the opportunity to take part in a trial of a new homeopathic remedy for treating infant colic. An American manufacturer of homeopathic remedies has made a registration application for the new remedy to the MHRA (Medicines and Healthcare products Regulatory Agency) under the UK “National Rules” scheme. As part of its application the manufacturer is seeking at least two homeopathic doctors who would be willing to trial the product for about a year, then write a short report about using the remedy and its clinical results. If you would like to take part in the trial, further details can be obtained from …

END OF QUOTE

A homeopathic remedy for infant colic?

Yes, indeed!

The British Homeopathic Association and many similar ‘professional’ organisations recommend homeopathy for infant colic: Infantile colic is a common problem in babies, especially up to around sixteen weeks of age. It is characterised by incessant crying, often inconsolable, usually in the evenings and often through the night. Having excluded underlying pathology, the standard advice given by GPs and health visitors is winding technique, Infacol or Gripe Water. These measures are often ineffective but for­tunately there are a number of homeo­pathic medicines that may be effective. In my experience Colocynth is the most successful; alternatives are Carbo Veg, Chamomilla and Nux vomica.

SO, IT MUST BE GOOD!

But hold on, I cannot find a single clinical trial to suggest that homeopathy is effective for infant colic.

Ahhhhhhhhhhhhhhhhhhh, I see, that’s why they now want to conduct a trial!

They want to do the right thing and do some science to see whether their claims are supported by evidence.

How very laudable!

After all, the members of the Faculty of Homeopathy are doctors; they have certain ethical standards!

After all, the Faculty of Homeopathy aims to provide a high level of service to members and members of the public at all times.

Judging from the short text about the ‘homeopathy for infant colic trial’, it will involve a few (at least two) homeopaths prescribing the homeopathic remedy to patients and then writing a report. These reports will unanimously state that, after the remedy had been administered, the symptoms improved considerably. (I know this because they always do improve – with or without treatment.)

These reports will then be put together – perhaps we should call this a meta-analysis? – and the overall finding will be nice, positive and helpful for the American company.

And now, we all understand what homeopaths, more precisely the Faculty of Homeopathy, consider to be evidence.

 

 

Back pain is a huge problem: it affects many people, causes much suffering and leads to extraordinary high cost for all of us. Considering these facts, it would be excellent to identify a treatment that truly works. However, at present, we do not have found the ideal therapy; instead we have dozens of different approaches that are similarly effective/ineffective. Two of these therapies are massage and acupuncture.

Is one better than the other?

This study compared the efficacy of classical massage (KMT, n = 66) with acupuncture therapy (AKU, n = 66) in patients with chronic back pain. The primary endpoint was the non-inferiority of classical massage compared with the acupuncture treatment in respect of the impairment in everyday life, with the help of the Hannover function questionnaire (HFAQ) and the reduction in pain (“Von Korff”-Questionnaire) at the follow-up after one month.

In the per-protocol analysis during the period between enrollment in the study and follow-up, the responder rate of the KMT was 56.5% and thus tended to be inferior to the responder rate of the AKU with 62.5% (Δ = - 6%; KIΔ: - 23.5 to + 11.4%).

The authors concluded that classical massage therapy is not significantly inferior to acupuncture therapy in the period from admission to follow-up. Thus, the non-inferiority of the KMT to the AKU cannot be proven in the context of the defined irrelevance area.

I find such studies oddly useless.

To conduct a controlled trial, one needs an experimental treatment (the therapy that is not understood) and compare it with an intervention that is understood (such as a placebo that has no specific effects, or a treatment that has been shown to work). In comparative studies like the one above, one compares one unknown with another unknown. I do not see how such a comparison can ever produce a meaningful result.

In a way, it is like an equation with two unknowns: x + 5 = y. It is simply not possible to define either x nor y, and the equation is unsolvable.

For comparative studies of two back-pain treatments to make sense, we would need one of which the effect size is well-established. I do not think that we currently have identified such a therapy. Certainly, we cannot say that we know it for massage or acupuncture.

In other words, comparative studies like the one above are a waste of resources that cannot possibly make a meaningful contribution to our knowledge.

To put it even more bluntly: we ought to stop such pseudo-research.

The Clinic for Complementary Medicine and Diet in Oncology was opened, in collaboration with the oncology department, at the Hospital of Lucca (Italy) in 2013. It uses a range of alternative therapies aimed at reducing the adverse effects of conventional oncology treatments.

Their latest paper presents the results of complementary medicine (CM) treatment targeted toward reducing the adverse effects of anticancer therapy and cancer symptoms, and improving patient quality of life. Dietary advice was aimed at the reduction of foods that promote inflammation in favour of those with antioxidant and anti-inflammatory properties.

This is a retrospective observational study on 357 patients consecutively visited from September 2013 to December 2017. The intensity of symptoms was evaluated according to a grading system from G0 (absent) to G1 (slight), G2 (moderate), and G3 (strong). The severity of radiodermatitis was evaluated with the Radiation Therapy Oncology Group (RTOG) scale. Almost all the patients (91.6%) were receiving or had just finished some form of conventional anticancer therapy.

The main types of cancer were breast (57.1%), colon (7.3%), lung (5.0%), ovary (3.9%), stomach (2.5%), prostate (2.2%), and uterus (2.5%). Comparison of clinical conditions before and after treatment showed a significant amelioration of all symptoms evaluated: nausea, insomnia, depression, anxiety, fatigue, mucositis, hot flashes, joint pain, dysgeusia, neuropathy.

The authors concluded that the integration of evidence-based complementary treatments seems to provide an effective response to cancer patients’ demand for a reduction of the adverse effects of anticancer treatments and the symptoms of cancer itself, thus improving patient’s quality of life and combining safety and equity of access within public healthcare systems. It is, therefore, necessary for physicians (primarily oncologists) and other healthcare professionals in this field to be appropriately informed about the potential benefits of CMs.

Why do I call this ‘wishful thinking’?

I have several reasons:

  1. A retrospective observational study cannot establish cause and effect. It is likely that the findings were due to a range of factors unrelated to the interventions used, including time, extra attention, placebo, social desirability, etc.
  2. Some of the treatments in the therapeutic package were not CM, reasonable and evidence-based. Therefore, it is likely that these interventions had positive effects, while CM might have been totally useless.
  3. To claim that the integration of evidence-based complementary treatments seems to provide an effective response to cancer patients’ is pure fantasy. Firstly, some of the CMs were certainly not evidence-based (the clinic’s prime focus is on homeopathy). Secondly, as already pointed out, the study does not establish cause and effect.
  4. The notion that it is necessary for physicians (primarily oncologists) and other healthcare professionals in this field to be appropriately informed about the potential benefits of CMs is not what follows from the data. The paper shows, however, that the authors of this study are in need to be appropriately informed about EBM as well as CM.

I stumbled across this paper because a homeopath cited it on Twitter claiming that it proves the effectiveness of homeopathy for cancer patients. This fact highlights why such publications are not just annoyingly useless but acutely dangerous. They mislead many cancer patients to opt for bogus treatments. In turn, this demonstrates why it is important to counterbalance such misinformation, critically evaluate it and minimise the risk of patients getting harmed.

On this blog, I have repeatedly discussed chiropractic research that, on closer examination, turns out to be some deplorable caricature of science. Today, I have another example of what I would call pseudo-research.

This RCT compared short-term treatment (12 weeks) versus long-term management (36 weeks) of back and neck related disability in older adults using spinal manipulative therapy (SMT) combined with supervised rehabilitative exercises (SRE).

Eligible participants were aged 65 and older with back and neck disability for more than 12 weeks. Co-primary outcomes were changes in Oswestry and Neck Disability Index after 36 weeks. An intention to treat approach used linear mixed-model analysis to detect between group differences. Secondary analyses included other self-reported outcomes, adverse events and objective functional measures.

A total of 182 participants were randomized. The short-term and long-term groups demonstrated significant improvements in back and neck disability after 36 weeks, with no difference between groups. The long-term management group experienced greater improvement in neck pain at week 36, self-efficacy at week 36 and 52, functional ability and balance.

The authors concluded that for older adults with chronic back and neck disability, extending management with SMT and SRE from 12 to 36 weeks did not result in any additional important reduction in disability.

What renders this paper particularly fascinating is the fact that its authors include some of the foremost researchers in (and most prominent proponents of) chiropractic today. I therefore find it interesting to critically consider the hypothesis on which this seemingly rigorous study is based.

As far as I can see, it essentially is this:

36 weeks of chiropractic therapy plus exercise leads to better results than 12 weeks of the same treatment.

I find this a most remarkable hypothesis.

Imagine any other form of treatment that is, like SMT, not solidly based on evidence of efficacy. Let’s use a new drug as an example, more precisely a drug for which there is no solid evidence for efficacy or safety. Now let’s assume that the company marketing this drug publishes a trial based on the hypothesis that:

36 weeks of therapy with the new drug plus exercise leads to better results than 12 weeks of the same treatment.

Now let’s assume the authors affiliated with the drug manufacturer concluded from their findings that for patients with chronic back and neck disability, extending drug therapy plus exercise from 12 to 36 weeks did not result in any additional important reduction in disability.

WHAT DO YOU THINK SUCH A TRIAL CAN TELL US?

My answer is ‘next to nothing’.

I think, it merely tells us that

  1. daft hypotheses lead to daft research,
  2. even ‘top’ chiropractors have problems with critical thinking,
  3. SMT might not be the solution to neck and back related disability.

I REST MY CASE.

 

Homeopathy for depression? A previous review concluded that the evidence for the effectiveness of homeopathy in depression is limited due to lack of clinical trials of high quality. But that was 13 years ago. Perhaps the evidence has changed?

A new review aimed to assess the efficacy, effectiveness and safety of homeopathy in depression. Eighteen studies assessing homeopathy in depression were included. Two double-blind placebo-controlled trials of homeopathic medicinal products (HMPs) for depression were assessed.

  • The first trial (N = 91) with high risk of bias found HMPs were non-inferior to fluoxetine at 4 and 8 weeks.
  • The second trial (N = 133), with low risk of bias, found HMPs was comparable to fluoxetine and superior to placebo at 6 weeks.

The remaining research had unclear/high risk of bias. A non-placebo-controlled RCT found standardised treatment by homeopaths comparable to fluvoxamine; a cohort study of patients receiving treatment provided by GPs practising homeopathy reported significantly lower consumption of psychotropic drugs and improved depression; and patient-reported outcomes showed at least moderate improvement in 10 of 12 uncontrolled studies. Fourteen trials provided safety data. All adverse events were mild or moderate, and transient. No evidence suggested treatment was unsafe.

The authors concluded that limited evidence from two placebo-controlled double-blinded trials suggests HMPs might be comparable to antidepressants and superior to placebo in depression, and patients treated by homeopaths report improvement in depression. Overall, the evidence gives a potentially promising risk benefit ratio. There is a need for additional high quality studies.

I beg to differ!

What these data really show amounts to far less than the authors imply:

  • The two ‘double-blind’ trials are next to meaningless. As equivalence studies they were far too small to produce meaningful results. Any decent review should discuss this fact in full detail. Moreover, these studies cannot have been double-blind, because the typical adverse-effects of anti-depressants would have ‘de-blinded’ the trial participants. Therefore, these results are almost certainly false-positive.
  • The other studies are even less rigorous and therefore do also not allow positive conclusions.

This review was authored by known proponents of homeopathy. It is, in my view, an exercise in promotion rather than a piece of research. I very much doubt that a decent journal with a responsible peer-review system would have ever published such a biased paper – it had to appear in the infamous EUROPEAN JOURNAL OF INTEGRATIVE MEDICINE.

So what?

Who cares? No harm done!

Again, I beg to differ.

Why?

The conclusion that homeopathy has a ‘promising risk/benefit profile’ is frightfully dangerous and irresponsible. If seriously depressed patients follow it, many lives might be lost.

Yet again, we see that poor research has the potential to kill vulnerable individuals.

1 2 3 12
Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted but you must tick the box: “Save my name, email, and website in this browser for the next time I comment.”


Click here for a comprehensive list of recent comments.

Categories