MD, PhD, FMedSci, FSB, FRCP, FRCPEd

study design

1 2 3 5

After > 200 years of existence, homeopathy still remains unproven – in fact, most rational thinkers would call it disproven. Today only homeopaths doubt this statement; they work hard to find a water-tight proof that might show the doubters to be wrong.

What is better suited for this purpose than a few rigorous animal experiments?

Engystol® is a popular homeopathic product promoted as an anti-viral agent manufactured by Heel GmbH, Baden-Baden, Germany. In several in vivo and in vitro studies, it apparently affected an immune response. This new study was to “evaluate the innate and adaptive immuno-modulatory effects of oral Engystol® (1 or 10 tablets/L water consumed), prior to and post antigenic challenge in a mouse model with a well-characterized and clinically measureable immune system.”

The investigators first evaluated the murine immune response when oral Engystol® was given alone for 28 days. to mice. The animals were then challenged with an antigen-specific H5N1 HA vaccine while on Engystol® for an additional 33 days. Serum and supernatants from cultured splenic lymphocytes were collected and screened with a 32-cytokine panel. Serum vaccine epitope-specific IgG titers plus T cell and B cell phenotypes from splenic tissue were also evaluated.

The results showed that Engystol® alone did not alter immunity. However, upon vaccine challenge, Engystol® decreased CD4+/CD8+ ratios, altered select cytokines/chemokines, and anti-H5N1 HA IgG titers were increased in the group of mice receiving 10 tablet/L.

The authors concluded that “these data suggest that Engystol® can modulate immunity upon antigenic challenge.”

Engystol is being advertised as “a homeopathic preparation which has been scientifically proven to significantly reduce the duration and severity of symptoms during an acute viral infection and help protect from subsequent infections.” I was unable find good evidence for this claim and therefore have to assume that it is bogus. The only human trial I was able to locate was this one:

OBJECTIVE:

To compare the effects of a complex homeopathic preparation (Engystol; Heel GmbH, Baden-Baden, Germany) with those of conventional therapies with antihistamines, antitussives, and nonsteroidal antiinflammatory drugs on upper respiratory symptoms of the common cold in a setting closely related to everyday clinical practice.

DESIGN:

Nonrandomized, observational study over a treatment period of maximally two weeks.

SETTING:

Eighty-five general and homeopathic practices in Germany.

PARTICIPANTS:

Three hundred ninety-seven patients with upper respiratory symptoms of the common cold.

INTERVENTIONS:

Engystol-based therapy or common over-the-counter treatments for the common cold. Patients receiving this homeopathic treatment were allowed other short-term medications, but long-term use of analgesics, antibiotics, and antiinflammatory agents was not permitted. Patients were allowed nonpharmacological therapies such as vitamins, thermotherapies, and others.

MAIN OUTCOME MEASURES:

The effects of treatment were evaluated on the variables fatigue, sensation of illness, chill/tremor, aching joints, overall severity of illness, sum of all clinical variables, temperature, and time to symptomatic improvement.

RESULTS:

Both treatment regimens provided significant symptomatic relief, and this homeopathic treatment was noninferior in a noninferiority analysis. Significantly more patients (P < .05) using Engystol-based therapy reported improvement within 3 days (77.1% vs 61.7% for the control group). No adverse events were reported in any of the treatment groups.

CONCLUSION:

This homeopathic treatment may be a useful component of an integrated symptomatic therapy for the common cold in patients and practitioners choosing an integrative approach to medical care.

Let me comment on the human study first. It is an excellent example of the bias that can be introduced by non-randomization. The patients in the homeopathic group obviously were those who chose to be treated homeopathically. Consequently they had high expectations in this therapy. Consequently they reported better results than the control group. In other words the reported outcomes have nothing to do with the homeopathic remedy.

But what about the animal study? Animals, we hear so often, do not exhibit a placebo response. Does that render this investigation any more reliable?

The answer, I am afraid is no.

The animal study in question had no control group at all. Therefore a myriad of factors could have caused the observed result. This study is very far from a poof of homeopathy!

But even if the findings of the two studies had not been the result of bias and confounding, I would be more than cautious about viewing them as anything near conclusive. The reason lies in the nature of this particular homeopathic remedy.

Engystol® contains Vincetoxicum hirundinaria (D6), Vincetoxicum hirundinaria (D10), Vincetoxicum hirundinaria (D30), sulphur (D4) and sulphur (D10). In other words, it is one of those combination remedies which are not sufficiently dilute to be devoid of active molecules. Sulphur D4, for instance, means that the remedy contains one part of sulphur in 10 000 parts of diluent. It is conceivable, even likely that such a concentration might affect certain immune parameters, I think.

And my conclusion from all this?

The proof of homeopathy – if it ever came – would need to be based on investigations that are more rigorous than these two rather pathetic studies.

In alternative medicine, good evidence is like gold dust and good evidence showing that alternative therapies are efficacious is even rarer. Therefore, I was delighted to come across a brand-new article from an institution that should stand for reliable information: the NIH, no less.

According to its authors, this new article “examines the clinical trial evidence for the efficacy and safety of several specific approaches—acupuncture, manipulation, massage therapy, relaxation techniques including meditation, selected natural product supplements (chondroitin, glucosamine, methylsulfonylmethane, S-adenosylmethionine), tai chi, and yoga—as used to manage chronic pain and related disability associated with back pain, fibromyalgia, osteoarthritis, neck pain, and severe headaches or migraines.”

The results of this huge undertaking are complex, of course, but in a nutshell they are at least partly positive for alternative medicine. Specifically, the authors state that “based on a preponderance of positive trials vs negative trials, current evidence suggests that the following complementary approaches may help some patients manage their painful health conditions: acupuncture and yoga for back pain; acupuncture and tai chi for OA of the knee; massage therapy for neck pain with adequate doses and for short-term benefit; and relaxation techniques for severe headaches and migraine. Weaker evidence suggests that massage therapy, SM, and osteopathic manipulation might also be of some benefit to those with back pain, and relaxation approaches and tai chi might help those with fibromyalgia.”

This is excellent news! Finally, we have data from an authoritative source showing that some alternative treatments can be recommended for common pain conditions.

Hold on, not so fast! Yes, the NIH is a most respectable organisation, but we must not blindly accept anything of importance just because it appears to come form a reputable source. Let’s look a bit closer at the actual evidence provided by the authors of this paper.

Reading the article carefully, it is impossible not to get troubled. Here are a few points that concern me most:

  • the safety of a therapy cannot be evaluated on the basis of data from RCTs (particularly as it has been shown repeatedly that trials of alternative therapies often fail to report adverse effects); much larger samples are needed for that; any statements about safety in the aims of the paper are therefore misplaced;
  • the authors talk about efficacy but seem to mean effectiveness;
  • the authors only included RCTs from the US which must result in a skewed and incomplete picture;
  • the article is from the National Center for Complementary and Integrative Health which is part of the NIH but which has been criticised repeatedly for being biased in favour of alternative medicine;
  • not all of the authors seem to be NIH staff, and I cannot find a declaration of conflicts of interest;
  • the discussion of the paper totally lacks any critical thinking;
  • there is no assessment of the quality of the trials included in this review.

My last point is by far the most important. A summary of this nature that fails to take into account the numerous limitations of the primary data is, I think, as good as worthless. As I know most of the RCTs included in the analyses, I predict that the overall picture generated by this review would have changed substantially, if the risks of bias in the primary studies had been accounted for.

Personally, I find it lamentable that such a potentially worthy exercise ended up employing such lousy methodology. Perhaps even more lamentable is the fact that the NIH (or one of its Centers) can descend that low; to mislead the public in this way borders on scientific misconduct and is, in my view, unethical and unacceptable.

Cranio-sacral therapy has been a subject on this blog before, for instance here, here and here. The authors of this single-blind, randomized trial explain in the introduction of their paper that “cranio-sacral therapy is an alternative and complementary therapy based on the theory that restricted movement at the cranial sutures of the skull negatively affect rhythmic impulses conveyed through the cerebral spinal fluid from the cranium to the sacrum. Restriction within the cranio-sacral system can affect its components: the brain, spinal cord, and protective membranes. The brain is said to produce involuntary, rhythmic movements within the skull. This movement involves dilation and contraction of the ventricles of the brain,  which produce the circulation of the cerebral spinal fluid. The theory states that this fluctuation mechanism causes reciprocal tension within the membranes, transmitting motion to the cranial bones and the sacrum. Cranio-sacral therapy and cranial osteopathic manual therapy originate from the observations made by William G. Sutherland, who said that the bones of the human skeleton have mobility. These techniques are based mainly on the study of anatomic and physiologic mechanisms in the skull and their relation to the body as a whole, which includes a system of diagnostic and therapeutic techniques aimed at treatment and prevention of diseases. These techniques are  based on the so-called primary respiratory movement, which is manifested in the mobility of the cranial bones, sacrum,  dura, central nervous system, and cerebrospinal fluid. The main difference between the two therapies is that cranial osteopathy, in addition to a phase that works in the direction of the lesion (called the functional phase), also uses a phase that worsens the injury, which is called structural phase.”

With this study, the researchers wanted to evaluate the effects of cranio-sacral therapy on disability, pain intensity, quality of life, and mobility in patients with low back pain. Sixty-four patients with chronic non-specific low back pain were assigned to an experimental group receiving 10 sessions of craniosacral therapy, or to the control group receiving 10 sessions of classic massage. Craniosacral therapy took 50 minutes and was conducted as follows: With pelvic diaphragm release, palms are placed in transverse position on the superior aspect of the pubic bone, under  the L5–S1 sacrum, and finger pads are placed on spinal processes.  With respiratory diaphragm release, palms are placed transverse under T12/L1 so that the spine lies along the start of fingers and the border of palm, and the anterior hand is placed on the breastbone. For thoracic inlet release, the thumb and index finger are placed on the opposite sides of the clavicle, with the posterior hand/palm of the hand cupping C7/T1. For the hyoid release, the thumb and index finger are placed on the hyoid, with the index finger on the occiput and the cupping finger pads on the cervical vertebrae. With the sacral technique for stabilizing L5/sacrum, the fingers contact the sulcus and the palm of the hand is in contact with the distal part of the sacral bone. The non-dominant hand of the therapist rested over the pelvis, with one hand on one iliac crest and the elbow/forearm of the other side over the other iliac crest. For CV-4 still point induction, thenar pads are placed under the occipital protuberance, avoiding mastoid sutures.  Classic massage protocol was compounded by the following sequence techniques of soft tissue massage on the  low back: effleurage, petrissage, friction, and kneading. The maneuvers are performed with surface pressure, followed by deep pressure and ending with surface pressure again. The techniques took 30 minutes.

Disability (Roland Morris Disability Questionnaire RMQ, and Oswestry Disability Index) was the primary endpoint. Other outcome measures included the pain intensity (10-point numeric pain rating scale), kinesiophobia (Tampa Scale of Kinesiophobia), isometric endurance of trunk flexor muscles (McQuade test), lumbar mobility in flexion, hemoglobin oxygen saturation, systolic blood pressure, diastolic blood pressure, hemodynamic measures (cardiac index), and biochemical analyses of interstitial fluid. All outcomes were measured at baseline, after treatment, and one-month follow-up.

No statistically significant differences were seen between groups for the main outcome of the study, the RMQ. However, patients receiving craniosacral therapy experienced greater improvement in pain intensity (p ≤ 0.008), hemoglobin oxygen saturation (p ≤ 0.028), and systolic blood pressure (p ≤ 0.029) at immediate- and medium-term and serum potassium (p = 0.023) level and magnesium (p = 0.012) at short-term than those receiving classic massage.

The authors concluded that 10 sessions of cranio-sacral therapy resulted in a statistically greater improvement in pain intensity, hemoglobin oxygen saturation, systolic blood pressure, serum potassium, and magnesium level than did 10 sessions of classic massage in patients with low back pain.

Given the results of this study, the conclusion is surprising. The primary outcome measure failed to show an inter-group difference; in other words, the results of this RCT were essentially negative. To use secondary endpoints – most of which are irrelevant for the study’s aim – in order to draw a positive conclusion seems odd, if not misleading. These positive findings are most likely due to the lack of patient-blinding or to the 200 min longer attention received by the verum patients. They are thus next to meaningless.

In my view, this publication is yet another example of an attempt to turn a negative into a positive result. This phenomenon seems embarrassingly frequent in alternative medicine. It goes without saying that it is not just misleading but also dishonest and unethical.

This post is dedicated to Mel Koppelman.

Those who followed the recent discussions about acupuncture on this blog will probably know her; she is an acupuncturist who (thinks she) knows a lot about research because she has several higher qualifications (but was unable to show us any research published by herself). Mel seems very quick in lecturing others about research methodology. Yesterday, she posted this comment in relation to my previous post on a study of aromatherapy and reflexology:

Professor Ernst, This post affirms yet again a rather poor understanding of clinical trial methodology. A pragmatic trial such as this one with a wait-list control makes no attempt to look for specific effects. You say “it is quite simply wrong to assume that this outcome is specifically related to the two treatments.” Where have specific effects been tested or assumed in this study? Your statement in no way, shape or form negates the author’s conclusions that “aromatherapy massage and reflexology are simple and effective non-pharmacologic nursing interventions.” Effectiveness is not a measure of specific effects.

I am most grateful for this comment because it highlights an issue that I had wanted to address for some time: The meanings of the two terms ‘efficacy and effectiveness’ and their differences as seen by scientists and by alternative practitioners/researchers.

Let’s start with the definitions.

I often use the excellent book of Alan Earl-Slater entitled THE HANDBOOK OF CLINICAL TRIALS AND OTHER RESEARCH. In it, EFFICACY is defined as ‘the degree to which an intervention does what it is intended to do under ideal conditions. EFFECTIVENESS is the degree to which a treatment works under real life conditions. An EFFECTIVENESS TRIAL is a trial that ‘is said to approximate reality (i. e. clinical practice). It is sometimes called a pragmatic trial’. An EFFICACY TRIAL ‘is a clinical trial that is said to take place under ideal conditions.’

In other words, an efficacy trial investigates the question, ‘can the therapy work?’, and an effectiveness trial asks, ‘does this therapy work?’ In both cases, the question relate to the therapy per se and not to the plethora of phenomena which are not directly related to it. It seems logical that, where possible, the first question would need to be addressed before the second – it does make little sense to test for effectiveness, if efficacy has not been ascertained, and effectiveness without efficacy does not seem to be possible.

In my 2007 book entitled UNDERSTANDING RESEARCH IN COMPLEMENTARY AND ALTERNATIVE MEDICINE (written especially for alternative therapists like Mel), I adopted these definitions and added: “It is conceivable that a given therapy works only under optimal conditions but not in everyday practice. For instance, in clinical practice patients may not comply with a therapy because it causes adverse effects.” I should have added perhaps that adverse effects are by no means the only reason for non-compliance, and that non-compliance is not the only reason why an efficacious treatment might not be effective.

Most scientists would agree with the above definitions. In fact, I am not aware of a debate about them in scientific circles. But they are not something alternative practitioners tend to like. Why? Because, using these strict definitions, many alternative therapies are neither of proven efficacy nor effectiveness.

What can be done about this unfortunate situation?

Simple! Let’s re-formulate the definitions of efficacy and effectiveness!

Efficacy, according to some alternative medicine proponents, refers to the therapeutic effects of the therapy per se, in other words, its specific effects. (That coincides almost with the scientific definition of this term – except, of course, it fails to tell us anything about the boundary conditions [optimal or real-life conditions].)

Effectiveness, according to the advocates of alternative therapies, refers to its specific effects plus its non-specific effects. Some researchers have even introduced the term ‘real-life effectiveness’ for this.

This is why, the authors of the study discussed in my previous post, could conclude that “aromatherapy massage and reflexology are simple… effective… interventions… to help manage pain and fatigue in patients with rheumatoid arthritis.” Based on their data, neither aromatherapy nor reflexology has been shown to be effective. They might appear to be effective because patients expected to get better, or patients in the no-treatment control group felt worse for not getting the extra care. Based on studies of this nature, giving patients £10 or a box of chocolate might also turn out to be “simple… effective… interventions… to help manage pain and fatigue in patients with rheumatoid arthritis.” Based on these definitions of efficacy and effectiveness, there are hardly any limits to bogus claims for any old quackery.

Such obfuscation suits proponents of alternative therapies fine because, using such definitions, virtually every treatment anyone might ever think of can be shown to be effective! Wishful thinking, it seems, can fulfil almost any dream, it can even turn the truth upside down.

Or can anyone name an alternative treatment that cannot even generate a placebo response when administered with empathy, sympathy and care? Compared to doing nothing, virtually every ineffective therapy might generate outcomes that make the treatment look effective. Even the anticipation of an effect alone might do the trick. How often have you had a tooth-ache, went to the dentist, and discovered sitting in the waiting room that the pain had gone? Does that mean that sitting in a waiting room is an effective treatment for dental pain?

In fact, some enthusiasts of alternative medicine could soon begin to argue that, with their new definition of ‘effectiveness’, we no longer need controlled clinical trials at all, if we want to demonstrate how effective alternative therapies truly are. We can just do observational studies without a control group, note that lots of patients get better, and ‘Bob is your uncle’!!! This is much faster, saves money, time and effort, and has the undeniable advantage of never generating a negative result.

To most outsiders, all this might seem a bit like splitting hair. However, I fear that it is far from that. In fact, it turns out to be a fairly fundamental issue in almost any discussion about the value or otherwise of alternative medicine. And, I think, it is also a matter of principle that reaches far beyond alternative medicine: if we allow various interest groups, lobbyists, sects, cults etc. to use their own definitions of fundamentally important terms, any dialogue, understanding or progress becomes almost impossible.

Yesterday, I wrote about a new acupuncture trial. Amongst other things, I wanted to find out whether the author who had previously insisted I answer his questions about my view on the new NICE guideline would himself answer a few questions when asked politely. To remind you, this is what I wrote:

This new study was designed as a randomized, sham-controlled trial of acupuncture for persistent allergic rhinitis in adults investigated possible modulation of mucosal immune responses. A total of 151 individuals were randomized into real and sham acupuncture groups (who received twice-weekly treatments for 8 weeks) and a no acupuncture group. Various cytokines, neurotrophins, proinflammatory neuropeptides, and immunoglobulins were measured in saliva or plasma from baseline to 4-week follow-up.

Statistically significant reduction in allergen specific IgE for house dust mite was seen only in the real acupuncture group. A mean (SE) statistically significant down-regulation was also seen in pro-inflammatory neuropeptide substance P (SP) 18 to 24 hours after the first treatment. No significant changes were seen in the other neuropeptides, neurotrophins, or cytokines tested. Nasal obstruction, nasal itch, sneezing, runny nose, eye itch, and unrefreshed sleep improved significantly in the real acupuncture group (post-nasal drip and sinus pain did not) and continued to improve up to 4-week follow-up.

The authors concluded that acupuncture modulated mucosal immune response in the upper airway in adults with persistent allergic rhinitis. This modulation appears to be associated with down-regulation of allergen specific IgE for house dust mite, which this study is the first to report. Improvements in nasal itch, eye itch, and sneezing after acupuncture are suggestive of down-regulation of transient receptor potential vanilloid 1.

…Anyway, the trial itself raises a number of questions – unfortunately I have no access to the full paper – which I will post here in the hope that my acupuncture friend, who are clearly impressed by this paper, might provide the answers in the comments section below:

  1. Which was the primary outcome measure of this trial?
  2. What was the power of the study, and how was it calculated?
  3. For which outcome measures was the power calculated?
  4. How were the subjective endpoints quantified?
  5. Were validated instruments used for the subjective endpoints?
  6. What type of sham was used?
  7. Are the reported results the findings of comparisons between verum and sham, or verum and no acupuncture, or intra-group changes in the verum group?
  8. What other treatments did each group of patients receive?
  9. Does anyone really think that this trial shows that “acupuncture is a safe, effective and cost-effective treatment for allergic rhinitis”?

In the comments section, the author wrote: “after you have read the full text and answered most of your questions for yourself, it might then be a more appropriate time to engage in any meaningful discussion, if that is in fact your intent”, and I asked him to send me his paper. As he does not seem to have the intention to do so, I will answer the questions myself and encourage everyone to have a close look at the full paper [which I can supply on request].

  1. The myriad of lab tests were defined as primary outcome measures.
  2. Two sentences are offered, but they do not allow me to reconstruct how this was done.
  3. No details are provided.
  4. Most were quantified with a 3 point scale.
  5. Mostly not.
  6. Needle insertion at non-acupoints.
  7. The results are a mixture of inter- and intra-group differences.
  8. Patients were allowed to use conventional treatments and the frequency of this use was reported in patient diaries.
  9. I don’t think so.

So, here is my interpretation of this study:

  • It lacked power for many outcome measures, certainly the clinical ones.
  • There were hardly any differences between the real and the sham acupuncture group.
  • Most of the relevant results were based on intra-group changes, rather than comparing sham with real acupuncture, a fact, which is obfuscated in the abstract.
  • In a controlled trial fluctuations within one group must never be interpreted as caused by the treatment.
  • There were dozens of tests for statistical significance, and there seems to be no correction for multiple testing.
  • Thus the few significant results that emerged when comparing sham with real acupuncture might easily be false positives.
  • Patient-blinding seems questionable.
  • McDonald as the only therapist of the study might be suspected to have influenced his patients through verbal and non-verbal communications.

I am sure there are many more flaws, particularly in the stats, and I leave it to others to identify them. The ones I found are, however, already serious enough, in my view, to call for a withdrawal of this paper. Essentially, the authors seem to have presented a study with largely negative findings as a trial with positive results showing that acupuncture is an effective therapy for allergic rhinitis. Subsequently, McDonald went on social media to inflate his findings even more. One might easily ask: is this scientific misconduct or just poor science?

I would be most interested to hear what you think about it [if you want to see the full article, please send me an email].

While looking up an acupuncturist who has recently commented on this blog trying to teach me how to do science and understand research methodology, I was impressed that he, Dr John McDonald, PhD, has just published a clinical trial. Not many acupuncturists do that, you know, and I very much applaud this action, which even seems to have earned him his PhD! McDonald is understandably proud of his achievement – all the more because the study arrived at positive conclusions. This is what he wrote about it:

…So, in a nutshell, acupuncture is a safe, effective and cost-effective treatment for allergic rhinitis which produces lasting changes in the immune system and hence improvements in symptoms and quality of life.    Dr John McDonald

Fascinating! I quickly looked up the paper. Here it is:

This new study was designed as a randomized, sham-controlled trial of acupuncture for persistent allergic rhinitis in adults investigated possible modulation of mucosal immune responses. A total of 151 individuals were randomized into real and sham acupuncture groups (who received twice-weekly treatments for 8 weeks) and a no acupuncture group. Various cytokines, neurotrophins, proinflammatory neuropeptides, and immunoglobulins were measured in saliva or plasma from baseline to 4-week follow-up.

Statistically significant reduction in allergen specific IgE for house dust mite was seen only in the real acupuncture group. A mean (SE) statistically significant down-regulation was also seen in pro-inflammatory neuropeptide substance P (SP) 18 to 24 hours after the first treatment. No significant changes were seen in the other neuropeptides, neurotrophins, or cytokines tested. Nasal obstruction, nasal itch, sneezing, runny nose, eye itch, and unrefreshed sleep improved significantly in the real acupuncture group (post-nasal drip and sinus pain did not) and continued to improve up to 4-week follow-up.

The authors concluded that acupuncture modulated mucosal immune response in the upper airway in adults with persistent allergic rhinitis. This modulation appears to be associated with down-regulation of allergen specific IgE for house dust mite, which this study is the first to report. Improvements in nasal itch, eye itch, and sneezing after acupuncture are suggestive of down-regulation of transient receptor potential vanilloid 1.

These conclusions seem to be based on the data of the study. But they are oddly out of line with the above statement made by McDonald about his trial. What could be the reason for this discrepancy? Could it be that he behaves ‘scientifically’ correct when under the watchful eye of numerous co-authors from the School of Medicine, Menzies Health Institute, Griffith University, Queensland, Australia, the School of Medicine, Menzies Health Institute, Griffith University, Queensland, Australia, the National Institute of Complementary Medicine, Western Sydney University, Sydney, Australia, the Health Innovations Research Institute and School of Health Sciences, RMIT University, Melbourne, Victoria, Australia, and the Stanford University, Palo Alto, California? And could it be that he is a little more ‘liberal’ when on his own? A mere speculation, of course, but it would be nice to know.

Anyway, the trial itself raises a number of questions – unfortunately I have no access to the full paper – which I will post here in the hope that my acupuncture friend, who are clearly impressed by this paper, might provide the answers in the comments section below:

  1. Which was the primary outcome measure of this trial?
  2. What was the power of the study, and how was it calculated?
  3. For which outcome measures was the power calculated?
  4. How were the subjective endpoints quantified?
  5. Were validated instruments used for the subjective endpoints?
  6. What type of sham was used?
  7. Are the reported results the findings of comparisons between verum and sham, or verum and no acupuncture, or intra-group changes in the verum group?
  8. Was the success of patient-blinding checked, quantified and successful?
  9. What other treatments did each group of patients receive?
  10. Does anyone really think that this trial shows that “acupuncture is a safe, effective and cost-effective treatment for allergic rhinitis”?

Homeopathy is not blessed with many geniuses, it seems. Therefore, it is all the more noteworthy that there is one who seems to be so extraordinarily gifted that everything she touches turns to gold.

Her new and remarkable study intended to measure the efficacy of individualized homeopathic treatment for binge eating in adult males.

This case study was a 9-week pilot using an embedded, mixed-methods design. A 3-week baseline period was followed by a 6-week treatment period. The setting was the Homeopathic Health Clinic at the University of Johannesburg in Johannesburg, South Africa. Through purposive sampling, the research team recruited 15 Caucasian, male participants, aged 18-45 y, who were exhibiting binge eating. Individualized homeopathic remedies were prescribed to each participant. Participants were assessed by means of (1) a self-assessment calendar (SAC), recording the frequency and intensity of binging; (2) the Binge Eating Scale (BES), a psychometric evaluation of severity; and (3) case analysis evaluating changes with time.

Ten participants completed the study. The study found a statistically significant improvement with regard to the BES (P = .003) and the SAC (P = .006), with a large effect size, indicating that a decrease occurred in the severity and frequency of binging behaviour during the study period.

The authors concluded that this small study showed the potential benefits of individualized homeopathic treatment of binge eating in males, decreasing both the frequency and severity of binging episodes. Follow-up studies are recommended to explore this treatment modality as a complementary therapeutic option in eating disorders characterized by binge eating.

While two of the three authors have not ventured into trials of homeopathy before, the third and senior author (Janice Pellow from the Department of Homoeopathy, University of Johannesburg, South Africa) already has several homeopathic studies to her name. They seem all quite similar:

Number 1 was a clinical trial that concluded:

The study was too small to be conclusive, but results suggest the homeopathic complex, together with physiotherapy, can significantly improve symptoms associated with chronic low back pain due to osteoarthritis.

Number 2 was an RCT which concluded:

The homeopathic complex used in this study exhibited significant anti-inflammatory and pain-relieving qualities in children with acute viral tonsillitis.

Number 3 was a pilot study concluding:

Findings suggest that daily use of the homeopathic complex does have an effect over a 4-week period on physiological and cognitive arousal at bedtime as well as on sleep onset latency in psychophysiological onset insomnia sufferers.

Number 4 was an RCT that concluded:

The homeopathic medicine reduced the sensitivity reaction of cat allergic adults to cat allergen, according to the skin prick test.

See what I mean? Five studies and 5 positive results!

Considering that they were obtained with different types of homeopathy, with different patients suffering from different conditions, with different trial designs and with different sets of co-workers, this is an even more remarkable achievement. In the hands of Janice Pellow, homeopathy seems to work under all circumstances and for all conditions.

I feel a Noble Prize might be in the air.

Pity that she would not score all that highly on my (self-invented) TI.

 

Mindfulness-based stress reduction (MBSR) has not been rigorously evaluated as a treatment of chronic low back pain. According to its authors, this RCT was aimed at evaluating “the effectiveness for chronic low back pain of MBSR vs cognitive behavioural therapy (CBT) or usual care.”

The investigators randomly assigned patients to receive MBSR (n = 116), CBT (n = 113), or usual care (n = 113). CBT meant training to change pain-related thoughts and behaviours and MBSR meant training in mindfulness meditation and yoga. Both were delivered in 8 weekly 2-hour groups. Usual care included whatever care participants received.

Coprimary outcomes were the percentages of participants with clinically meaningful (≥30%) improvement from baseline in functional limitations (modified Roland Disability Questionnaire [RDQ]; range, 0-23) and in self-reported back pain bothersomeness (scale, 0-10) at 26 weeks. Outcomes were also assessed at 4, 8, and 52 weeks.

There were 342 randomized participants with a mean duration of back pain of 7.3 years. They attended 6 or more of the 8 sessions, 294 patients completed the study at 26 weeks, and 290 completed it at 52 weeks. In intent-to-treat analyses at 26 weeks, the percentage of participants with clinically meaningful improvement on the RDQ was higher for those who received MBSR (60.5%) and CBT (57.7%) than for usual care (44.1%), and RR for CBT vs usual care, 1.31 [95% CI, 1.01-1.69]). The percentage of participants with clinically meaningful improvement in pain bothersomeness at 26 weeks was 43.6% in the MBSR group and 44.9% in the CBT group, vs 26.6% in the usual care group, and RR for CBT vs usual care was 1.69 [95% CI, 1.18-2.41]). Findings for MBSR persisted with little change at 52 weeks for both primary outcomes.

The authors concluded that among adults with chronic low back pain, treatment with MBSR or CBT, compared with usual care, resulted in greater improvement in back pain and functional limitations at 26 weeks, with no significant differences in outcomes between MBSR and CBT. These findings suggest that MBSR may be an effective treatment option for patients with chronic low back pain.

At first glance, this seems like a well-conducted study. It was conducted by one of the leading back pain research team and was published in a top-journal. It will therefore have considerable impact. However, on closer examination, I have serious doubts about certain aspects of this trial. In my view, both the aims and the conclusions of this RCT are quite simply wrong.

The authors state that they aimed at evaluating “the effectiveness for chronic low back pain of MBSR vs cognitive behavioural therapy (CBT) or usual care.” This is not just misleading, it is wrong! The correct aim should have been to evaluate “the effectiveness for chronic low back pain of MBSR plus usual care vs cognitive behavioural therapy plus usual care or usual care alone.” One has to go into the method section to find the crucial statement: “All participants received any medical care they would normally receive.”

Consequently, the conclusions are equally wrong. They should have read as follows: Among adults with chronic low back pain, treatment with MBSR plus usual care or CBT plus usual care, compared with usual care alone, resulted in greater improvement in back pain and functional limitations at 26 weeks, with no significant differences in outcomes between MBSR and CBT.

In other words, this is yet another trial with the dreaded ‘A+B vs B’ design. Because A+B is always more than B (even if A is just a placebo), such a study will never generate a negative result (even if A is just a placebo). The results are therefore entirely compatible with the notion that the two tested treatments are pure placebos. Add to this the disappointment many patients in the ‘usual care group’ might have felt for not receiving an additional therapy for their pain, and you have a most plausible explanation for the observed outcomes.

I am totally puzzled why the authors failed to discuss these possibilities and limitations in full, and I am equally bewildered that JAMA published such questionable research.

 

In recent blogs, I have written much about acupuncture and particularly about the unscientific notions of traditional acupuncturists. I was therefore surprised to see that a UK charity is teaming up with traditional acupuncturists in an exercise that looks as though it is designed to mislead the public.

The website of ‘Anxiety UK’ informs us that this charity and the British Acupuncture Council (BAcC) have launched a ‘pilot project’ which will see members of Anxiety UK being able to access traditional acupuncture through this new partnership. Throughout the pilot project, they proudly proclaim, data will be collected to “determine the effectiveness of traditional acupuncture for treating those living with anxiety and anxiety based depression.”

This, they believe, will enable both parties to continue to build a body of evidence to measure the success rate of this type of treatment. Anxiety UK’s Chief Executive Nicky Lidbetter said: “This is an exciting project and will provide us with valuable data and outcomes for those members who take part in the pilot and allow us to assess the benefits of extending the pilot to a regular service for those living with anxiety. “We know anecdotally that many people find complementary therapies used to support conventional care can provide enormous benefit, although it should be remembered they are used in addition to and not instead of seeking medical advice from a doctor or taking prescribed medication. This supports our strategic aim to ensure that we continue to make therapies and services that are of benefit to those with anxiety and anxiety based depression, accessible.”

And what is wrong with that, you might ask.

What is NOT wrong with it, would be my response.

To start with, traditional acupuncture relies of obsolete assumptions like yin and yang, meridians, energy flow, acupuncture points etc. They have one thing in common: they fly in the face of science and evidence. But this might just be a triviality. More important is, I believe, the fact that a pilot project cannot determine the effectiveness of a therapy. Therefore the whole exercise smells very much like a promotional activity for pure quackery.

And what about the hint in the direction of anecdotal evidence in support of the study? Are they not able to do a simple Medline search? Because, if they had done one, they would have found a plethora of articles on the subject. Most of them show that there are plenty of studies but their majority is too flawed to draw firm conclusions.

A review by someone who certainly cannot be accused of being biased against alternative medicine, for instance, informs us that “trials in depression, anxiety disorders and short-term acute anxiety have been conducted but acupuncture interventions employed in trials vary as do the controls against which these are compared. Many trials also suffer from small sample sizes. Consequently, it has not proved possible to accurately assess the effectiveness of acupuncture for these conditions or the relative effectiveness of different treatment regimens. The results of studies showing similar effects of needling at specific and non-specific points have further complicated the interpretation of results. In addition to measuring clinical response, several clinical studies have assessed changes in levels of neurotransmitters and other biological response modifiers in an attempt to elucidate the specific biological actions of acupuncture. The findings offer some preliminary data requiring further investigation.”

Elsewhere, the same author, together with other pro-acupuncture researchers, wrote this: “Positive findings are reported for acupuncture in the treatment of generalised anxiety disorder or anxiety neurosis but there is currently insufficient research evidence for firm conclusions to be drawn. No trials of acupuncture for other anxiety disorders were located. There is some limited evidence in favour of auricular acupuncture in perioperative anxiety. Overall, the promising findings indicate that further research is warranted in the form of well designed, adequately powered studies.”

What does this mean in the context of the charity’s project?

I think, it tells us that acupuncture for anxiety is not exactly the most promising approach to further investigate. Even in the realm of alternative medicine, there are several interventions which are supported by more encouraging evidence. And even if one disagrees with this statement, one cannot possibly disagree with the fact that more flimsy research is not required. If we do need more studies, they must be rigorous and not promotion thinly disguised as science.

I guess the ultimate question here is one of ethics. Do charities not have an ethical and moral duty to spend our donations wisely and productively? When does such ill-conceived pseudo-research cross the line to become offensive or even fraudulent?

As it is ‘ACUPUNCTURE AWARENESS WEEK’, I thought I make a constructive contribution to this field by assessing what is currently being published on the subject. For this purpose, I looked at the first 100 Medline-listed articles of 2016. This has the advantage, of course, that all the numbers thus generated can be seen as absolute and as percentage figures at the same time. I categorised the articles according to where they were published and what their subject was.

My results show that, of the first 100 articles,

  • 33 were published in CAM journals,
  • 67  were published in mainstream medical journals,
  • 6 were RCTs,
  • 6 were other clinical studies,
  • 30 were pre-clinical investigations,
  • 27 were systematic reviews,
  • 8 were surveys,
  • 23 were other types of papers.

I have to admit, these results are not as bad as I had feared. What seems impressive is foremost the notion that acupuncture research has entered the mainstream journals. But there are issues that might be of concern; in my view these results suggest that:

  • Too little research is focussed on the two big questions: efficacy and safety.
  • In relation to the meagre output in RCTs, there are too many systematic reviews.
  • As long as we cannot be sure that acupuncture is more than a placebo, all these pre-clinical studies seem a bit out of place.
  • The vast majority of the articles were in low or very low impact journals.
  • There was only one paper that I would consider outstanding (my next post will discuss it).

So, what conclusions can one draw from these data?

Not many, I fear.

My little exploration does not lend itself to grand, generalizable or far-reaching conclusions. Acupuncture fans might proudly say: LOOK HOW FAR WE HAVE COME! Less enthusiastic experts, however, might think: LOOK HOW FAR YOU HAVE TO GO!

1 2 3 5
Recent Comments

Note that comments can now be edited for up to five minutes after they are first submitted.


Click here for a comprehensive list of recent comments.

Categories