MD, PhD, MAE, FMedSci, FRSB, FRCP, FRCPEd.

study design

1 2 3 25

In 2007, we published a systematic review summarizing the efficacy of homeopathic remedies used as a sole or additional therapy in cancer care. We have searched the literature using the databases: Amed (from 1985); CINHAL (from 1982); EMBASE (from 1974); Medline (from 1951); and CAMbase (from 1998). Randomized and non-randomized controlled clinical trials including patients with cancer or past experience of cancer receiving single or combined homeopathic interventions were included. The methodological quality of the trials was assessed by Jadad score. Six studies met our inclusion criteria (five were randomized clinical trials and one was a non-randomized study); but the methodological quality was variable including some high-standard studies. Our analysis of published literature on homeopathy thus found insufficient evidence to support the clinical efficacy of homeopathic therapy in cancer care.

Meanwhile, more trials have emerged, not least a dubious study by Frass et al which is currently under investigation. This means that a new evaluation of the totality of the available evidence might be called for. I am pleased to report that such an assessment has just been published.

In this systematic review, the researchers included clinical studies from 1800 until 2020 to evaluate evidence of the effectiveness of homeopathy on physical and mental conditions in patients during oncological treatment.

In February 2021 a systematic search was conducted searching five electronic databases (Embase, Cochrane, PsychInfo, CINAHL, and Medline) to find studies concerning the use, effectiveness, and potential harm of homeopathy in cancer patients.

From all 1352 search results, 18 studies with 2016 patients were included in this SR. The patients treated with homeopathy were mainly diagnosed with breast cancer. The therapy concepts include single and combination homeopathic remedies (used systemically or as mouth rinses) of various dilutions. Outcomes assessed were the influence on toxicity of cancer treatment (mostly hot flashes and menopausal symptoms), time to drain-removal in breast cancer patients after mastectomy, survival, quality of life, global health and subjective well-being, anxiety, and depression as well as safety and tolerance.

The included studies reported heterogeneous results: some studies described significant differences in quality of life or toxicity of cancer treatment favoring homeopathy, whereas others did not find an effect or reported significant differences to the disadvantage of homeopathy or side effects caused by homeopathy. The majority of the studies have low methodological quality.

The authors concluded that, for homeopathy, there is neither a scientifically based hypothesis of its mode of action nor conclusive evidence from clinical studies in cancer care.

I predict that, if we wait another 15 years, we will have even more studies. I also predict that some of them will be less than reliable or even fake. Finally, I predict that the overall result will still be mixed and unconvincing.

Why can I be so sure?

  1. Because homeopathy lacks biological plausibility as a treatment of cancer (or any other condition).
  2. Because highly diluted homeopathic remedies are pure placebos.
  3. Because homeopathy has developed into a cult where one is no longer surprised to see studies emerging that are too good to be true.

Nausea and vomiting are common symptoms of patients with advanced cancer. While there is some evidence for acupuncture point stimulation in the treatment of these symptoms for patients having anticancer treatment, there is little for when they are not related to such treatment.

This study aimed to determine whether acupressure at the pericardium 6 sites can help treat nausea and vomiting suffered by palliative care patients with advanced cancer. The researchers conducted a double-blind randomized controlled trial-active versus placebo acupressure wristbands. In-patients with advanced cancer in two specialist palliative care units who fitted either or both of the following criteria were approached: nausea that was at least of moderate severity; vomiting daily on average for the prior 3 days.

A total of 57 patients were randomized to have either active or placebo acupressure wristbands. There was no difference in any of the outcome measures between the two groups:

  • change from the baseline number of vomits;
  • Visual Analogue Scale for ‘did acupressure wristbands help you to feel better?’;
  • the total number of doses of antiemetic medication;
  • the need for escalation of antiemetics.

The authors concluded that, in contrast to a previously published feasibility study, active acupressure wristbands were no better than placebo for specialist palliative care in patients with advanced cancer and nausea and vomiting.

When the research into acupuncture for nausea and vomiting began some 20 years ago, the evidence turned out to be encouraging. Later, as the studies became more and more rigorous, many trials failed to confirm the initial findings. Today, the totality of the evidence is far less convincing than it seemed years ago.

This is a phenomenon that can be observed not just in acupuncture research but in many types of treatment:

  • Initially, over-enthusiastic researchers become victims of their own optimism.
  • These investigators are less into testing hypotheses than into confirming their own wishful thinking.
  • Thus, several positive trials emerge.
  • These, however, turn out to be methodologically flawed.
  • Eventually, the subject might be picked up by real scientists who truly test hypotheses.
  • More and more negative studies thus emerge.
  • Depending on how many flawed studies were initially published and how critical the authors of systematic reviews are, it can take years until the totality of the evidence depicts the true picture which discloses the initial findings as false-positive.

The message is, I think, clear: poor quality studies have the potential to mislead us for many years. Eventually, however, the self-cleansing ability of science should generate the truth about the value of any treatment. In other words:

poor-quality science is not just useless, it causes long-term harm

and

critical thinking prevents harm

 

As promised, I would like to correct the errors in my previous assessment of this paper. To remind everyone:

This systematic review evaluated individualized homeopathy as a treatment for children with attention deficit and hyperactivity disorder (ADHD) when compared to placebo or usual care alone.

Thirty-seven online sources were searched up to March 2021. Studies investigating the effects of individualized homeopathy against any control in ADHD were eligible. Data were extracted to a predefined excel sheet independently by two reviewers.

Six studies were analyzed:

  • 5 were RCTs
  • 2 were controlled against standard treatments;
  • 4 were placebo-controlled and double-blinded.

The meta-analysis revealed a significant effect size across studies of Hedges’ g = 0.542 (95% CI 0.311-0.772; z = 4,61; p < 0.001) against any control and of g = 0.605 (95% CI 0.05-1.16; z = 2.16, p = 0.03) against placebo. The effect estimations are based on studies with an average sample size of 52 participants.

The authors concluded that individualized homeopathy showed a clinically relevant and statistically robust effect in the treatment of ADHD.

_______________________________

Now that I was able to access the full papers, I would like to offer a thorough analysis.

To get included in the review, primary studies had to be:

  • Published after 1980,
  • Investigating an individualized homeopathic intervention in childhood ADHD,
  • Comparing the intervention to a control condition (placebo, standard care or treatment as usual, both of which are referred to as “active control”) in a randomized or non-randomized parallel-group study
    design with one or more arms.

Six studies were included:

  • Fibert, P., Peasgood, T. & Relton, C. Rethinking ADHD intervention trials: feasibility testing of two treatments and a methodology. Eur. J. Pediatr. 178, 983–993 (2019). – DOI
  • Fibert, P., Relton, C., Heirs, M. & Bowden, D. A comparative consecutive case series of 20 children with a diagnosis of ADHD receiving homeopathic treatment, compared with 10 children receiving usual care. Homeopathy 105, 194–201 (2016). – DOI
  • Jacobs, J., Williams, A. L., Girard, C., Njike, V. Y. & Katz, D. Homeopathy for attention-deficit/hyperactivity disorder: a pilot randomized-controlled trial. J. Altern. Complement. Med. 11, 799–806 (2005). – DOI
  • Jones, M. The efficacy of homoeopathic simillimum in the treatment of attention-deficit/hyperactivity disorder (AD/HD) in schoolgoing children aged 6-11 years. https://openscholar.dut.ac.za/bitstream/10321/534/1/Jones_2009.pdf (2009).
  • Frei, H. et al. Homeopathic treatment of children with attention deficit hyperactivity disorder: a randomised, double blind, placebo controlled crossover trial. Eur. J. Pediatr. 164, 758–767 (2005). – DOI
  • Oberai, P. et al. Homoeopathic management of attention deficit hyperactivity disorder: a randomised placebo-controlled pilot trial. Indian J. Res. Homoeopathy 7, 158–167 (2013).

Exclusion criteria were:

  • Homeopathic intervention not individualized,
  • Serious methodological flaws, such as incidental unblinding, failure to report important data, or insufficient data for meta-analysis.

One study was excluded:

  • Lamont, J. Homoeopathic treatment of attention deficit hyperactivity disorder. Br. Homeopathic J. 86, 196–200 (1997). – DOI

I will first make several points about Walach’s systematic review itself and then have a look at the primary studies that it included. Finally, I will try to draw some conclusions.

The review authors state in their introduction that “beneficial effects of this intervention [homeopathy] have been shown for various kinds of medical conditions, including child diarrhea, supportive care in cancer, fibromyalgia, or ADHD.” In other words, already in the introduction, they disclose their strong pro-homeopathy bias; it would, of course, not be difficult to find investigations that contradict their optimism.

Despite the stated inclusion/exclusion criteria, the authors did include the Frei-study that did not follow a parallel-group design (see also below).

The authors included two active-controlled studies both of which did not report the type of treatment received by the control group. In other words, these trials failed to report important data which was a stated exclusion criterium (see below).

In their discussion section, the authors state that “all included studies employed individualized homeopathy and were of comparable, solid quality, hence a lack of methodological rigor is unlikely the reason for the difference between homeopathy and controls…” This, I think, is grossly misleading; even according to the authors’ own assessments, one study was deemed to have a high risk of bias and in two studies the risk of bias was “unclear”.

The overall positive effect of homeopathy demonstrated by the review was determined almost exclusively by the study of Oberai et al (p-value = 0.000). In fact, the studies by Jones and by Jacobs were negative, and the one by Frei was borderline positive with a p-value of 0.46. The authors address this crucial issue repeatedly and claim that excluding Oberai et al would still generate an overall positive meta-analytic result. Yet, they do not mention that the overall result would no longer be clinically relevant.

Looking at the included primary studies, I should make the following points:

  • The two Filbert studies, as mentioned, failed to report important data and should, according to the stated exclusion criteria, not have been included.
  • The study by Jacobs was a pilot study and generated negative findings.
  • The study by Jones is a non-peer-reviewed thesis. In my view, it should never have been included.
  • The study by Frei was a cross-over trial. According to the exclusion/inclusion criteria of the authors, it should not have been included.
  • The study by Oberai et al is the trial that has by far the largest effect size and thus is the driver of the overall result of the review. It is therefore important to have a closer look at it.

Here is the abstract:

Objective: To evaluate the usefulness of individualised homoeopathic medicines in treatment of Attention Deficit Hyperactivity Disorder (ADHD).
Design: Randomised placebo-controlled single-blind pilot trial.
Setting: Central Research Institute (Homoeopathy), Kottayam, Kerala, India from June 2009 to November 2011.
Participants: Children aged 6-15 years meeting the Diagnostic Statistical Manual of mental disorders (DSM-IV) criteria for ADHD.
Interventions: A total of 61 patients (Homoeopathy = 30, placebo = 31) were randomised to receive either individualised homoeopathic medicine in fifty millesimal (LM) potency or placebo for a period of one year.
Outcome measures: Conner’s Parent Rating Scale-Revised: Short (CPRS-R (S)), Clinical Global Impression-Severity Scale (CGI-SS), Clinical Global Impression- Improvement Scale (CGI-IS) and Academic performance.
Results: A total of 54 patients (homoeopathy = 27, placebo = 27) were analysed under modified intention to treat (ITT). All patients in homoeopathy group showed better outcome in baseline adjusted General Linear Model (GLM) repeated measures ANCOVA for oppositional, cognition problems, hyperactivity and ADHD Index (domains of CPRS-R (S)) and CGI-IS at T3, T6, T9 and T12 (P = 0.0001). The mean baseline-adjusted treatment difference between groups at month 12 from baseline for all individual outcome measures favoured homoeopathy group; Oppositional (−16.4, 95% CI – 20.5 to − 12.2, P = 0.0001), Cognition problems (−15.5, 95% CI − 19.2 to − 11.8, P = 0.0001), Hyperactivity (−20.6, 95% CI − 25.6 to − 15.4, P = 0.0001), ADHD I (−15.6, 95% CI − 19.5 to − 11.6, P = 0.0001), Academic performance 14.4%, 95% CI 8.3 to 20.5, P = 0.0001), CGISS (−1.6, 95% CI − 1.9 to − 1.2, P = 0.0001), CGIIS (−1.6, 95% CI − 2.3 to -0.9, P = 0.0001).
Conclusion: This pilot study provides evidence to support the therapeutic effects of individualised homoeopathic medicines in ADHD children. However, the results need to be validated in multi-center randomised double-blind placebo-controlled clinical trial.

Here are a few points of concern related to the Oberai et al:

  • The trial was a mere pilot study.
  • Despite the fact that it is now 9 years old, the authors never published a definitive trial.
  • The study was published in an obscure journal that is not Medline-listed.
  • The study is very poorly reported.
  • It is unclear how the diagnosis of ADHD for including the patients was verified.
  • The control patients were treated for one year with a placebo and no other therapies. In my view, this is not ethical.
  • The method of randomization is unclear.
  • The authors state that acute symptoms were treated throughout the study period with homeopathy, even in the control group. This seems odd and defies the principle of a placebo-controlled trial.
  • The authors state that only the patients were blind, not the investigators. This opens the door wide for all sorts of biases. It is, for example, likely that it also de-blinded the patients (the verum could be adjusted and changed, while the placebo remained constant).

All in all, this paper is of poor quality, Its findings are far from trustworthy and were not meant to be definitive. According to the following exclusion criteria, it should have been excluded:

  • It had several serious methodological flaws.
  • It did not blind the investigators.
  • It is likely that patients were de-blinded.
  • It failed to report important data.

So, why did Walach and his co-authors include it?

Could it be because, without the Oberai-study, the overall findings of the review would at best have turned out to be borderline significant and not clinically relevant?

Alan Gaby, the assistant editor of the journal Integr Med has written an interesting commentary about widespread fraud in natural health products research. Here is an excerpt of his article:

During the past 49 years, I have reviewed and analyzed more than 50 000 papers from the biomedical literature, most of which were related to the field of nutritional medicine. Doing this work has given me some understanding of how to assess the reliability of a study. Over the past 10 to 15 years, an uncomfortably large and growing number of published papers related to my area of expertise have left me wondering whether the research was fabricated; that is, whether people were writing papers about research that had not actually been conducted. If the studies were not actually conducted, the publishing of this research is an affront to all who value integrity in science, and it has the potential to harm practitioners and patients who rely on its findings.

The studies that have raised concerns have come primarily from Iran and to a lesser extent from Egypt, China, India, Japan, and a few other countries. Characteristics of these concerning studies typically include one or more of the following:

  1. The study comes from an investigator or research group that has published an enormous number of randomized clinical trials in a relatively short period of time.
  2. The number of participants in the trial is unusually large, when considering the resources that appear to be available to the researchers.
  3. The recruitment period for the trial is unusually short.
  4. The paper is submitted to a journal unusually rapidly after the study is completed, or in some cases before it would have been possible to have completed the trial.
  5. A randomized double-blind trial is conducted before there is any preliminary evidence of efficacy in humans (such as case reports or uncontrolled trials). Because double-blind trials are expensive to conduct, such trials are generally reserved for treatments for which there is some evidence of efficacy.
  6. The magnitude of the reported improvement is much larger than is typically seen in trials using just one or two nutrients.
  7. No funding source is listed or the study is listed as self-funded. This is of particular concern when the sample size or study design suggests that the study was expensive.
  8. The design of the study raises ethical issues, such as participants not being permitted to use treatments that are known to be effective.
  9. One or more baseline characteristics of the study group appear to be implausible.
  10. The research was conducted by a student as part of a graduate school thesis, and the magnitude of the project seems to have been beyond the capabilities and resources of a student.

__________________________________

What Gaby alludes to is a problem indeed. I have previously posted about the Chinese aspect of this story. What Gaby does not mention is the fact that even many studies of so-called alternative medicine (SCAM) which seem to be not overtly fraudulent are nevertheless highly suspect. I am referring to trials that are fatally flawed and/or studies that draw unwarranted conclusions. These are, of course, the types of studies that are the main target of this blog. Because they are so numerous, I feel that the damage they do is much bigger than that of the more overtly fraudulent papers.

This systematic review evaluated individualized homeopathy as a treatment for children with attention deficit and hyperactivity disorder (ADHD) when compared to placebo or usual care alone.

Thirty-seven online sources were searched up to March 2021. Studies investigating the effects of individualized homeopathy against any control in ADHD were eligible. Data were extracted to a predefined excel sheet independently by two reviewers.

Six studies were analyzed:

  • 5 were RCTs
  • 2 were controlled against standard treatments;
  • 4 were placebo-controlled and double-blinded.

The meta-analysis revealed a significant effect size across studies of Hedges’ g = 0.542 (95% CI 0.311-0.772; z = 4,61; p < 0.001) against any control and of g = 0.605 (95% CI 0.05-1.16; z = 2.16, p = 0.03) against placebo. The effect estimations are based on studies with an average sample size of 52 participants.

The authors concluded that individualized homeopathy showed a clinically relevant and statistically robust effect in the treatment of ADHD.

This is a counter-intuitive result (to put it mildly), and it is, therefore, wise to have a look at the 6 included studies:

1.Frei, H. et al. Homeopathic treatment of children with attention deficit hyperactivity disorder: a randomised, double blind, placebo controlled crossover trial. Eur. J. Pediatr. 164, 758–767 (2005).

This was a trial with just 62 patients who had previously responded to homeopathy. The study was conducted by known proponents of homeopathy and had a highly unusual design. The results suggested that homeopathy was better than placebo.

2. Oberai, P. et al. Homoeopathic management of attention deficit hyperactivity disorder: a randomised placebo-controlled pilot trial. Indian J. Res. Homoeopathy 7, 158–167 (2013).

This one was published in an obscure journal that I could not access.

3. Jacobs, J., Williams, A. L., Girard, C., Njike, V. Y. & Katz, D. Homeopathy for attention-deficit/hyperactivity disorder: a pilot randomized-controlled trial. J. Altern. Complement. Med. 11, 799–806 (2005)

This study showed that there were no statistically significant differences between homeopathic remedy and placebo groups on the primary or secondary outcome variables.

4. Jones, M. The efficacy of homoeopathic simillimum in the treatment of attention-deficit/hyperactivity disorder (AD/HD) in schoolgoing children aged 6-11 years (2009).

This was a small unpublished (and not peer-reviewed) thesis. Its results showed no statistically significant effect of treatment.

5. Lamont, J. Homoeopathic treatment of attention deficit hyperactivity disorder. Br. Homeopathic J. 86, 196–200 (1997)

This was a small (n=46) trial with an unusual design. Its results suggested that homeopathy was better than placebo.

6. von Ammon, K. et al. Homeopathic RCT embedded in a long-term observational study of children with ADHD—a successful model of whole systems CAM research. Eur. J. Integr. Med. 1, 27 (2008).

Even though the journal is Medline-listed, I was unable to find this paper. I did, however, find a paper by the same authors with the same title. It turned out to be a duplication of the paper by Frei et al listed above.

_________________________

All in all, this brief analysis of the available abstracts (most full papers are behind paywalls) leaves many questions as to the trustworthiness of this systematic review unanswered. The fact that H. Walach (and other apologists of homeopathy) is its senior author does not inspire me with overwhelming confidence. In any case, I very much doubt that the authors’ conclusion is correct. I therefore would encourage someone with access to all full papers to initiate a more thorough analysis; the abstracts obviously leave many questions unanswered. For instance, it would be crucial to know how many of the trials followed an A+B versus B design (I suspect most studies did, and this would completely invalidate the review’s conclusion). I am more than happy to co-operate with such an evaluation.

Two million people in UK are estimated to be currently suffering from long COVID, says the Office for National Statistics. Fatigue continues to be the most common symptom – experienced by 55% of those with self-reported long COVID – followed by 32% with shortness of breath, 23% with a cough, and 23% with muscle ache. The problem is only going to increase in the near future. Thus, many people are frantically looking for an effective therapy. Practitioners of so-called alternative medicine (SCAM) are no exception.

This study aimed to evaluate the potential for inhalation of essential oils to improve energy levels among otherwise healthy female survivors of acute COVID-19 who experience a lack of energy more than five months after recovery.

This was a randomized double-blind, placebo-controlled trial to evaluate the potential for inhalation of Longevity™, a proprietary essential oil blend manufactured by Young Living Essential Oils (Lehi, Utah, USA), on energy levels among female survivors of COVID-19 who continue to experience fatigue more than 5 months recovery from the acute infection. Forty women were randomized to two groups: intervention and placebo. The placebo product contained an inert, odorless fractionated coconut oil. Both groups inhaled the assigned product twice daily for fourteen consecutive days. Fatigue scores were measured using the Multidimensional Fatigue Symptom Inventory (MFSI). Secondary outcomes included scores on each of the MFSI’s ten subscales.

Individuals who inhaled the essential oil blend for 2 weeks had significantly lower fatigue scores after controlling for baseline scores, employment status, BMI, olfactory function, and time since diagnosis, with a large effect size (F (1,39) = 6.15, p = .020, partial eta squared = 0.198). Subscale analysis identified subscales of vigor, as well as global, behavioral, general, and mental fatigue as benefiting from the intervention. This study provides evidence that a proprietary aromatherapy blend can significantly improve energy levels among women who are experiencing fatigue after recovering from COVID-19.

The authors concluded that the use of aromatherapy with Longevity™ essential oil blend to boost energy levels in women who have recovered from COVID-19 provides a novel, non-invasive approach to improving quality of life in this population. This intervention is particularly beneficial for global and mental fatigue, as well as vigor. Other subdomains may experience improvements to energy levels with a smaller effect size; future studies should be conducted to explore this potential.

This trial was funded by Young Living Essential Oils. Perhaps, this explains why there is no mention of the elephant in the room: the trial was not blind! Participants in the verum group knew that they received aromatherapy. Likewise, participants in the placebo group knew that they received the placebo.

Could this fact have influenced the outcome? Certainly!

Could the trial have been designed better? Certainly!

All the investigators needed to do is to use a nice-smelling oil that, according to aromatherapists, does not boost energy, as the placebo.

As it stands, we have no idea whether the authors’ assumption that the verum oil caused the effect is true.

Pity!

Or maybe not?

Perhaps Young Living Essential Oils, the sponsor of the study and producer of the oil never wanted to know the truth. Maybe they are happy to abuse science as a marketing tool?

Osteopathic visceral manipulation (VM) is a bizarre so-called alternative medicine (SCAM) that has been featured on this blog with some regularity, e.g.:

Rigorous trials fail to show that it works for anything. So, the obvious solution to this dilemma is to conduct dodgy trials!

This study tested the effects of VM on dysmenorrhea, irregular, delayed, and/or absent menses, and premenstrual symptoms in PCOS patients.

Thirty Egyptian women with polycystic ovary syndrome (PCOS), with menstruation-related complaints and free from systematic diseases and/or adrenal gland abnormalities, participated in a single-blinded, randomized controlled trial. They were recruited from the women’s health outpatient clinic in the faculty of physical therapy at Cairo University, with an age of 20-34 years, and a body mass index (BMI) ≥25, <30 kg/m2. Patients were randomly allocated into two equal groups (15 patients); the control group received a low-calorie diet for 3 months, and the study group that received the same hypocaloric diet added to VM to the pelvic organs and their related structures for eight sessions over 3 months. Evaluations for body weight, BMI, and menstrual problems were done by weight-height scale, and menstruation-domain of Polycystic Ovary Syndrome Health-Related Quality of Life Questionnaire (PCOSQ), respectively, at baseline and after 3 months from interventions. Data were described as mean, standard deviation, range, and percentage whenever applicable.

Of 60 Egyptian women with PCOS, 30 patients were included, with baseline mean age, weight, BMI, and a menstruation domain score of 27.5 ± 2.2 years, 77.7 ± 4.3 kg, 28.6 ± 0.7 kg/m2, and 3.4 ± 1.0, respectively, for the control group, and 26.2 ± 4.7 years, 74.6 ± 3.5 kg, 28.2 ± 1.1 kg/m2, and 2.9 ± 1.0, respectively, for the study group. Out of the 15 patients in the study group, uterine adhesions were found in 14 patients (93.3%), followed by restricted uterine mobility in 13 patients (86.7%), restricted ovarian/broad ligament mobility (9, 60%), and restricted motility (6, 40%). At baseline, there was no significant difference (p>0.05) in any of the demographics (age, height), or dependent variables (weight, BMI, menstruation domain score) among both groups. Post-study, there was a statistically significant reduction (p=0.000) in weight, and BMI mean values for the diet group (71.2 ± 4.2 kg, and 26.4 ± 0.8 kg/m2, respectively) and the diet + VM group (69.2 ± 3.7 kg; 26.1 ± 0.9 kg/m2, respectively). For the improvement in the menstrual complaints, a significant increase (p<0.05) in the menstruation domain mean score was shown in the diet group (3.9 ± 1.0), and the diet + VM group (4.6 ± 0.5). On comparing both groups post-study, there was a statistically significant improvement (p=0.024) in the severity of menstruation-related problems in favor of the diet + VM group.

The authors concluded that VM yielded greater improvement in menstrual pain, irregularities, and premenstrual symptoms in PCOS patients when added to caloric restriction than utilizing the low-calorie diet alone in treating that condition.

WHERE TO START?

  • Tiny sample size.
  • A trail design (A+B vs B) which will inevitably generate a positive result.
  • Questionable ethics.

VM is a relatively invasive and potentially embarrassing intervention for any woman; I imagine that this is all the more true in Egypt. In such circumstances, it is mandatory to ask whether a planned study is ethically justifiable. I would answer this question related to an implausible treatment like VM with a straight NO!

I realize that there may be people who disagree with me. But even those guys should accept that, at the very minimum, such a study must be designed such that it leads to a clear answer – is VM effective or not? The present trial merely suggests that the placebo effect associated with VM is powerful (which is hardly surprising for a therapy like VM).

Acupuncture is often promoted as a therapeutic option for obesity and weight control. The aim of this study was to investigate the effects of electroacupuncture (EA) on body weight, body mass index (BMI), skin fold thickness, waist circumference and skin temperature of the abdominal region in non-obese women with excessive abdominal subcutaneous fat.

A total of 50 women with excessive abdominal subcutaneous fat (and average BMI of 22) were randomly assigned to one of two groups:

  1. an EA group (n = 25) receiving 10 EA sessions (insertion of needles connected to an electrical stimulator at a frequency of 40 Hz for 40 min),
  2. a control group (n = 25) that received no treatment.

Outcome measures evaluated included waist circumference, supra-iliac and abdominal skinfolds, body composition and superficial skin temperature (measured by cutaneous thermography) before and after treatment.

Compared with the untreated group, women in the EA group exhibited decreased supra-iliac and abdominal skin folds (p < 0.001), waist circumference (p < 0.001), percentage body fat (p = 0.001) and percentage abdominal fat (p < 0.001). In addition, the EA group showed an elevated skin temperature at the site of the treatment. However, EA did not significantly impact body weight (p = 0.01) or BMI (p = 0.2).

The authors concluded that EA promoted a reduction in abdominal waist circumference, supra-iliac and abdominal skin folds, and percentage body and abdominal fat in women of normal BMI with excessive abdominal subcutaneous fat, as well as an increase in the superficial skin temperature of the abdominal region.

If we did not know that acupuncture researchers were all honest investigators testing hypotheses the best they can, we could almost assume that some are trying to fool us. The set-up of this study is ideally suited to introduce a proper placebo treatment. All one has to do is to not switch on the electrical stimulator in the control group. Why did the researchers not do that? Surely not because they wanted to increase the chances of generating a positive result; that would have been dishonest!!!

So, as it stands, what does the study tell us? I think it shows that, compared to patients who receive no treatment, patients who do receive the ritual of EA are better motivated to adhere to calorie restrictions and dietary advice. Thus, I suggest to re-phrase the conclusions of this trial as follows:

The extra attention of the EA treatment motivated obese patients to eat less which caused a reduction in abdominal waist circumference, supra-iliac and abdominal skin folds, and percentage body and abdominal fat in women of normal BMI with excessive abdominal subcutaneous fat.

Ayush-64 is an Ayurvedic formulation, developed by the Central Council for Research in Ayurvedic Sciences (CCRAS), the apex body for research in Ayurveda under the Ministry of Ayush. Originally developed in 1980 for the management of Malaria, this drug has now been repurposed for COVID-19 as its ingredients showed notable antiviral, immune-modulator, and antipyretic properties. Its ingredients are:

Alstonia scholaris R. Br. Aqueous extract of (Saptaparna) Bark-1 part
Picrorhiza Kurroa Royle Aqueous extract of (Kutki) Rhizome-1 part
Swertia chirata Buch-Ham. Aqueous extract of (Chirata) Whole plant-1 part
Caesalphinia crista, Linn. Fine powder of seed (Kuberaksha) Pulp-2 parts

The crucial question, of course, is does AYUSH-64 work?

An open-label randomized controlled parallel-group trial was conducted at a designated COVID care centre in India with 80 patients diagnosed with mild to moderate COVID-19 and randomized into two groups. Participants in the AYUSH-64 add-on group (AG) received AYUSH-64 two tablets (500 mg each) three times a day for 30 days along with standard conventional care. The control group (CG) received standard care alone.

The outcome measures were:

  • the proportion of participants who attained clinical recovery on days 7, 15, 23, and 30,
  • the proportion of participants with negative RT-PCR assay for COVID-19 at each weekly time point,
  • change in pro-inflammatory markers,
  • metabolic functions,
  • HRCT chest (CO-RADS category),
  • the incidence of Adverse Drug Reaction (ADR)/Adverse Event (AE).

Out of 80 participants, 74 (37 in each group) contributed to the final analysis. A significant difference was observed in clinical recovery in the AG (p < 0.001 ) compared to CG. The mean duration for clinical recovery in AG (5.8 ± 2.67 days) was significantly less compared to CG (10.0 ± 4.06 days). Significant improvement in HRCT chest was observed in AG (p = 0.031) unlike in CG (p = 0.210). No ADR/SAE was observed or reported in AG.

The authors concluded that AYUSH-64 as adjunct to standard care is safe and effective in hastening clinical recovery in mild to moderate COVID-19. The efficacy may be further validated by larger multi-center double-blind trials.

I do object to these conclusions for several reasons:

  1. The study cannot possibly determine the safety of AYUSH-64.
  2. Even for assessing its efficacy, it was too small.
  3. The trial design followed the often-discussed A+B vs B concept and is thus prone to generate false-positive results.

I believe that it is highly irresponsible, during a medical crisis like ours, to conduct studies that can only produce unreliable findings. If there is a real possibility that a therapy might work, we do need to test it, but we should take great care that the test is rigorous enough to generate reliable results. This, I think, is all the more true, if – like in the present case – the study was done with governmental support.

This meta-analysis was conducted by researchers affiliated to the Evangelical Clinics Essen-Mitte, Department of Internal and Integrative Medicine, Faculty of Medicine, University of Duisburg-Essen, Germany. (one of its authors is an early member of my ALTERNATIVE MEDICINE HALL OF FAME). The paper assessed the safety of acupuncture in oncological patients.

The PubMed, Cochrane Central Register of Controlled Trials, and Scopus databases were searched from their inception to August 7, 2020. Randomized controlled trials in oncological patients comparing invasive acupuncture with sham acupuncture, treatment as usual (TAU), or any other active control were eligible. Two reviewers independently extracted data on study characteristics and adverse events (AEs). Risk of bias was assessed using the Cochrane Risk of Bias Tool.

Of 4590 screened articles, 65 were included in the analyses. The authors observed that acupuncture was not associated with an increased risk of intervention-related AEs, nonserious AEs, serious AEs, or dropout because of AEs compared with sham acupuncture and active control. Compared with TAU, acupuncture was not associated with an increased risk of intervention-related AEs, serious AEs, or dropout because of AEs but was associated with an increased risk for nonserious AEs (odds ratio, 3.94; 95% confidence interval, 1.16-13.35; P = .03). However, the increased risk of nonserious AEs compared with TAU was not robust against selection bias. The meta-analyses may have been biased because of the insufficient reporting of AEs in the original randomized controlled trials.

The authors concluded that the current review indicates that acupuncture is as safe as sham acupuncture and active controls in oncological patients. The authors recommend researchers heed the CONSORT (Consolidated Standards of Reporting Trials) safety and harm extension for reporting to capture the side effects and better investigate the risk profile of acupuncture in oncology.

You might think this article is not too bad. So, why do I feel that this paper is so bad?

One reason is that the authors included evidence up to August 2020. Since then, there must have been hundreds of further papers on acupuncture. The article was therefore out of date before it was published.

But that is by no means my main reason. We know from numerous investigations that acupuncture studies often fail to report AEs (and thus violate publication ethics). This means that this new analysis is merely an amplification of the under-reporting. It is, in other words, a means of perpetuating a wrong message.

Yes, you might say, but the authors acknowledge this; they even state in the abstract that “The meta-analyses may have been biased because of the insufficient reporting of AEs in the original randomized controlled trials.” True, but this fact does not erase the mistake, it merely concedes it. At the very minimum, the authors should have phrased their conclusion differently, e.g.: the current review confirms that AEs of acupuncture are under-reported in RCTs. Therefore, a meta-analysis of RCTs is unable to verify whether acupuncture is safe. From other types of research, we know that it can cause serious AEs.

An even better solution would have been to abandon or modify the research project when they first came across the mountain of evidence showing that RCTs often fail to mention AEs.

As it stands, the conclusion that acupuncture is as safe as sham acupuncture is simply not true. Since the article probably looks sound to naive readers, I feel that is a particularly good candidate for the WORST PAPER OF 2022 COMPETITION.

 

PS

For those who are interested, here are 4 of my own peer-reviewed articles on the safety of acupuncture (much more can, of course, be found on this blog):

  1. Patient safety incidents from acupuncture treatments: a review of reports to the National Patient Safety Agency – PubMed (nih.gov)
  2. Acupuncture–a critical analysis – PubMed (nih.gov)
  3. Prospective studies of the safety of acupuncture: a systematic review – PubMed (nih.gov)
  4. The risks of acupuncture – PubMed (nih.gov)
1 2 3 25
Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted but you must tick the box: “Save my name, email, and website in this browser for the next time I comment.”

The most recent comments from all posts can be seen here.

Archives
Categories