MD, PhD, FMedSci, FRSB, FRCP, FRCPEd.

systematic review

1 2 3 22

I have often cautioned my readers about the ‘evidence’ supporting acupuncture (and other alternative therapies). Rightly so, I think. Here is yet another warning.

This systematic review assessed the clinical effectiveness of acupuncture in the treatment of postpartum depression (PPD). Nine trials involving 653 women were selected. A meta-analysis demonstrated that the acupuncture group had a significantly greater overall effective rate compared with the control group. Moreover, acupuncture significantly increased oestradiol levels compared with the control group. Regarding the HAMD and EPDS scores, no difference was found between the two groups. The Chinese authors concluded that acupuncture appears to be effective for postpartum depression with respect to certain outcomes. However, the evidence thus far is inconclusive. Further high-quality RCTs following standardised guidelines with a low risk of bias are needed to confirm the effectiveness of acupuncture for postpartum depression.

What a conclusion!

What a review!

What a journal!

What evidence!

Let’s start with the conclusion: if the authors feel that the evidence is ‘inconclusive’, why do they state that ‘acupuncture appears to be effective for postpartum depression‘. To me this does simply not make sense!

Such oddities are abundant in the review. The abstract does not mention the fact that all trials were from China (published in Chinese which means that people who cannot read Chinese are unable to check any of the reported findings), and their majority was of very poor quality – two good reasons to discard the lot without further ado and conclude that there is no reliable evidence at all.

The authors also tell us very little about the treatments used in the control groups. In the paper, they state that “the control group needed to have received a placebo or any type of herb, drug and psychological intervention”. But was acupuncture better than all or any of these treatments? I could not find sufficient data in the paper to answer this question.

Moreover, only three trials seem to have bothered to mention adverse effects. Thus the majority of the studies were in breach of research ethics. No mention is made of this in the discussion.

In the paper, the authors re-state that “this meta-analysis showed that the acupuncture group had a significantly greater overall effective rate compared with the control group. Moreover, acupuncture significantly increased oestradiol levels compared with the control group.” This is, I think, highly misleading (see above).

Finally, let’s have a quick look at the journal ‘Acupuncture in Medicine’ (AiM). Even though it is published by the BMJ group (the reason for this phenomenon can be found here: “AiM is owned by the British Medical Acupuncture Society and published by BMJ“; this means that all BMAS-members automatically receive the journal which thus is a resounding commercial success), it is little more than a cult-newsletter. The editorial board is full of acupuncture enthusiasts, and the journal hardly ever publishes anything that is remotely critical of the wonderous myths of acupuncture.

My conclusion considering all this is as follows: we ought to be very careful before accepting any ‘evidence’ that is currently being published about the benefits of acupuncture, even if it superficially looks ok. More often than not, it turns out to be profoundly misleading, utterly useless and potentially harmful pseudo-evidence.


Reference

Acupunct Med. 2018 Jun 15. pii: acupmed-2017-011530. doi: 10.1136/acupmed-2017-011530. [Epub ahead of print]

Effectiveness of acupuncture in postpartum depression: a systematic review and meta-analysis.

Li S, Zhong W, Peng W, Jiang G.

Most people probably think of acupuncture as being used mainly as a therapy for pain control. But acupuncture is currently being promoted (and has traditionally been used) for all sorts of conditions. One of them is stroke. It is said to speed up recovery and even improve survival rates after such an event. There are plenty of studies on this subject, but their results are far from uniform. What is needed in this situation, is a rigorous summary of the evidence.

The authors of this Cochrane review wanted to assess whether acupuncture could reduce the proportion of people suffering death or dependency after acute ischaemic or haemorrhagic stroke. They included all randomized clinical trials (RCTs) of acupuncture started within 30 days after stroke onset. Acupuncture had to be compared with placebo or sham acupuncture or open control (no placebo) in people with acute ischaemic or haemorrhagic stroke, or both. Comparisons were made versus (1) all controls (open control or sham acupuncture), and (2) sham acupuncture controls.

The investigators included 33 RCTs with 3946 participants. Outcome data were available for up to 22 trials (2865 participants) that compared acupuncture with any control (open control or sham acupuncture) but for only 6 trials (668 participants) comparing acupuncture with sham acupuncture.

When compared with any control (11 trials with 1582 participants), findings of lower odds of death or dependency at the end of follow-up and over the long term (≥ three months) in the acupuncture group were uncertain and were not confirmed by trials comparing acupuncture with sham acupuncture. In trials comparing acupuncture with any control, findings that acupuncture was associated with increases in the global neurological deficit score and in the motor function score were uncertain. These findings were not confirmed in trials comparing acupuncture with sham acupuncture.Trials comparing acupuncture with any control showed little or no difference in death or institutional care or death at the end of follow-up.The incidence of adverse events (eg, pain, dizziness, faint) in the acupuncture arms of open and sham control trials was 6.2% (64/1037 participants), and 1.4% of these patients (14/1037 participants) discontinued acupuncture. When acupuncture was compared with sham acupuncture, findings for adverse events were uncertain.

The authors concluded that this updated review indicates that apparently improved outcomes with acupuncture in acute stroke are confounded by the risk of bias related to use of open controls. Adverse events related to acupuncture were reported to be minor and usually did not result in stopping treatment. Future studies are needed to confirm or refute any effects of acupuncture in acute stroke. Trials should clearly report the method of randomization, concealment of allocation, and whether blinding of participants, personnel, and outcome assessors was achieved, while paying close attention to the effects of acupuncture on long-term functional outcomes.

This Cochrane review seems to be thorough, but it is badly written (Cochrane reviewers: please don’t let this become the norm!). It contains some interesting facts. The majority of the studies came from China. This review confirmed the often very poor methodological quality of acupuncture trials which I have frequently mentioned before.

In particular, the RCTs originating from China were amongst those that most overtly lacked rigor, also a fact that has been discussed regularly on this blog.

For me, by far the most important finding of this review is that studies which at least partly control for placebo effects fail to show positive results. Depending on where you stand in the never-ending debate about acupuncture, this could lead to two dramatically different conclusions:

  1. If you are a believer in or earn your living from acupuncture, you might say that these results suggest that the trials were in some way insufficient and therefore they produced false-negative results.
  2. If you are a more reasonable observer, you might feel that these results show that acupuncture (for acute stroke) is a placebo therapy.

Regardless to which camp you belong, one thing seems to be certain: acupuncture for stroke (and other indications) is not supported by sound evidence. And that means, I think, that it is not responsible to use it in routine care.

An article entitled “Homeopathy in the Age of Antimicrobial Resistance: Is It a Viable Treatment for Upper Respiratory Tract Infections?” cannot possibly be ignored on this blog, particularly if published in the amazing journal ‘Homeopathy‘. The title does not bode well, in my view – but let’s see. Below, I copy the abstract of the paper without any changes; all I have done is include a few numbers in brackets; they refer to my comments that follow.

START OF ABSTRACT

Acute upper respiratory tract infections (URTIs) and their complications are the most frequent cause of antibiotic prescribing in primary care. With multi-resistant organisms proliferating, appropriate alternative treatments to these conditions are urgently required. Homeopathy presents one solution (1); however, there are many methods of homeopathic prescribing. This review of the literature considers firstly whether homeopathy offers a viable alternative therapeutic solution for acute URTIs (2) and their complications, and secondly how such homeopathic intervention might take place.

METHOD:

Critical review of post 1994 (3) clinical studies featuring homeopathic treatment of acute URTIs and their complications. Study design, treatment intervention, cohort group, measurement and outcome were considered. Discussion focused on the extent to which homeopathy is used to treat URTIs, rate of improvement and tolerability of the treatment, complications of URTIs, prophylactic and long-term effects, and the use of combination versus single homeopathic remedies.

RESULTS:

Multiple peer-reviewed (4) studies were found in which homeopathy had been used to treat URTIs and associated symptoms (cough, pharyngitis, tonsillitis, otitis media, acute sinusitis, etc.). Nine randomised controlled trials (RCTs) and 8 observational/cohort studies were analysed, 7 of which were paediatric studies. Seven RCTs used combination remedies with multiple constituents. Results for homeopathy treatment were positive overall (5), with faster resolution, reduced use of antibiotics and possible prophylactic and longer-term benefits.

CONCLUSIONS:

Variations in size, location, cohort and outcome measures make comparisons and generalisations concerning homeopathic clinical trials for URTIs problematic (6). Nevertheless, study findings suggest at least equivalence between homeopathy and conventional treatment for uncomplicated URTI cases (7), with fewer adverse events and potentially broader therapeutic outcomes. The use of non-individualised homeopathic compounds tailored for the paediatric population merits further investigation, including through cohort studies (8). In the light of antimicrobial resistance, homeopathy offers alternative strategies for minor infections and possible prevention of recurring URTIs (9).

END OF ABSTRACT

And here are my comments:

  1. This sounds as though the author already knew the conclusion of her review before she even started.
  2. Did she not just state that homeopathy is a solution?
  3. This is most unusual; why were pre-1994 articles not considered?
  4. This too is unusual; that a study is peer-reviewed is not really possible to affirm, one must take the journal’s word for it. Yet we know that peer-review is farcical in the realm of alternative medicine (see also below). Therefore, this is an odd inclusion criterion to mention in an abstract.
  5. An overall positive result obtained by including uncontrolled observational and cohort studies lacking a control group is meaningless. There is also no assessment of the quality of the RCTs; after a quick glance, I get the impression that the methodologically sound studies do not show homeopathy to be superior to placebo.
  6. Reviewers need to disentangle these complicating factors and arrive at a conclusion. This is almost invariably problematic, but it is the reviewer’s job.
  7. What might be the conventional treatment of uncomplicated URTI?
  8. Why on earth cohort studies? They tell us nothing about equivalence, efficacy etc.
  9. To reach that conclusion seems to have been the aim of this review (see my point number 1). If I am not mistaken, antibiotics are not indicated in the vast majority of cases of uncomplicated URTI. This means that the true alternative in the light of antimicrobial resistance is to not prescribe antibiotics and treat the patient symptomatically. No need at all for homeopathic placebos, and no need for wishful thinking reviews!

In the paper, the author explains her liking of uncontrolled studies: Non-RCTs and patient reported surveys are considered by some to be inferior forms of research evidence, but are important adjuncts to RCTs that can measure key markers such as patient satisfaction, quality of life and functional health. Observational studies such as clinical outcome studies and case reports, monitoring the effects of homeopathy in real-life clinical settings, are a helpful adjunct to RCTs and more closely reflect real-life experiences of patients and physicians than RCTs, and are therefore considered in this study. I would counter that this is not an issue of inferiority but one that depends on the research question; if the research question relates to efficacy/effectiveness, uncontrolled data are next to useless.

She also makes fairly categorical statements in the conclusion section of the paper about the effectiveness of homeopathy: [the] combined evidence from these and other studies suggests that homeopathic treatment can exert biological effects with fewer adverse events and broader therapeutic opportunities than conventional medicine in the treatment of URTIs. Given the cost implications of treating URTIs and their complications in children, and the relative absence of effective alternatives without potential side effects, the use of non-individualised homeopathic compounds tailored for the paediatric population merits further investigation, including through large-scale cohort studies…  the most important evidence still arises from practical clinical experience and from the successful treatment of millions of patients. I would counter that none of these conclusions are warranted by the data presented.

From reading the paper, I get the impression that the author (the paper provides no information about her conflicts of interest, nor funding source) is a novice to conducting reviews (even though the author is a senior lecturer, the paper reads more like a poorly organised essay than a scientific review). I am therefore hesitant to criticise her – but I do nevertheless find the fact deplorable that her article passed the peer-review process of ‘Homeopathy‘ and was published in a seemingly respectable journal. If anything, articles of this nature are counter-productive for everyone concerned; they certainly do not further effective patient care, and they give homeopathy-research a worse name than it already has.

The only time we discussed gua sha, it led to one of the most prolonged discussions we ever had on this blog (536 comments so far). It seems to be a topic that excites many. But what precisely is it?

Gua sha, sometimes referred to as “scraping”, “spooning” or “coining”, is a traditional Chinese treatment that has spread to several other Asian countries. It has long been popular in Vietnam and is now also becoming well-known in the West. The treatment consists of scraping the skin with a smooth edge placed against the pre-oiled skin surface, pressed down firmly, and then moved downwards along muscles or meridians. According to its proponents, gua sha stimulates the flow of the vital energy ‘chi’ and releases unhealthy bodily matter from blood stasis within sore, tired, stiff or injured muscle areas.

The technique is practised by TCM practitioners, acupuncturists, massage therapists, physical therapists, physicians and nurses. Practitioners claim that it stimulates blood flow to the treated areas, thus promoting cell metabolism, regeneration and healing. They also assume that it has anti-inflammatory effects and stimulates the immune system.

These effects are said to last for days or weeks after a single treatment. The treatment causes microvascular injuries which are visible as subcutaneous bleeding and redness. Gua sha practitioners make far-reaching therapeutic claims, including that the therapy alleviates pain, prevents infections, treats asthma, detoxifies the body, cures liver problems, reduces stress, and contributes to overall health.

Gua sha is mildly painful, almost invariably leads to unsightly blemishes on the skin which occasionally can become infected and might even be mistaken for physical abuse.

There is little research of gua sha, and the few trials that exist tend to be published in Chinese. But recently, a new paper has emerged that is written in English. The goal of this systematic review was to evaluate the available evidence from randomized controlled trials (RCTs) of gua sha for the treatment of patients with perimenopausal syndrome.

A total of 6 RCTs met the inclusion criteria. Most were of low methodological quality. When compared with Western medicine therapy alone, meta-analysis of 5 RCTs indicated favorable statistically significant effects of gua sha plus Western medicine. Moreover, study participants who received Gua Sha therapy plus Western medicine therapy showed significantly greater improvements in serum levels of follicle-stimulating hormone (FSH), luteinizing hormone (LH) compared to participants in the Western medicine therapy group.

The authors concluded that preliminary evidence supported the hypothesis that Gua Sha therapy effectively improved the treatment efficacy in patients with perimenopausal syndrome. Additional studies will be required to elucidate optimal frequency and dosage of Gua Sha.

This sounds as though gua sha is a reasonable therapy.

Yet, I think this notion is worth being critically analysed. Here are some caveats that spring into my mind:

  • Gua sha lacks biological plausibility.
  • The reviewed trials are too flawed to allow any firm conclusions.
  • As most are published in Chinese, non-Chinese speakers have no possibility to evaluate them.
  • The studies originate from China where close to 100% of TCM trials report positive results.
  • In my view, this means they are less than trustworthy.
  • The authors of the above-cited review are all from China and might not be willing, able or allowed to publish a critical paper on this subject.
  • The review was published in , a journal not known for its high scientific standards or critical stance towards TCM.

So, is gua sha a reasonable therapy?

I let you make this judgement.

Is homeopathy effective for specific conditions? The FACULTY OF HOMEOPATHY (FoH, the professional organisation of UK doctor homeopaths) say YES. In support of this bold statement, they cite a total of 35 systematic reviews of homeopathy with a focus on specific clinical areas. “Nine of these 35 reviews presented conclusions that were positive for homeopathy”, they claim. Here they are:

Allergies and upper respiratory tract infections 8,9
Childhood diarrhoea 10
Post-operative ileus 11
Rheumatic diseases 12
Seasonal allergic rhinitis (hay fever) 13–15
Vertigo 16

And here are the references (I took the liberty of adding my comments in blod):

8. Bornhöft G, Wolf U, Ammon K, et al. Effectiveness, safety and cost-effectiveness of homeopathy in general practice – summarized health technology assessment. Forschende Komplementärmedizin, 2006; 13 Suppl 2: 19–29.

This is the infamous ‘Swiss report‘ which, nowadays, only homeopaths take seriously.

9. Bellavite P, Ortolani R, Pontarollo F, et al. Immunology and homeopathy. 4. Clinical studies – Part 1. Evidence-based Complementary and Alternative Medicine: eCAM, 2006; 3: 293–301.

This is not a systematic review as it lacks any critical assessment of the primary data and includes observational studies and even case series.

10. Jacobs J, Jonas WB, Jimenez-Perez M, Crothers D. Homeopathy for childhood diarrhea: combined results and metaanalysis from three randomized, controlled clinical trials. Pediatric Infectious Disease Journal, 2003; 22: 229–234.

This is a meta-analysis by Jennifer Jacobs (who recently featured on this blog) of 3 studies by Jennifer Jacobs; hardly convincing I’d say.

11. Barnes J, Resch K-L, Ernst E. Homeopathy for postoperative ileus? A meta-analysis. Journal of Clinical Gastroenterology, 1997; 25: 628–633.

This is my own paper! It concluded that “several caveats preclude a definitive judgment.”

12. Jonas WB, Linde K, Ramirez G. Homeopathy and rheumatic disease. Rheumatic Disease Clinics of North America, 2000; 26: 117–123.

This is not a systematic review; here is the (unabridged) abstract:

Despite a growing interest in uncovering the basic mechanisms of arthritis, medical treatment remains symptomatic. Current medical treatments do not consistently halt the long-term progression of these diseases, and surgery may still be needed to restore mechanical function in large joints. Patients with rheumatic syndromes often seek alternative therapies, with homeopathy being one of the most frequent. Homeopathy is one of the most frequently used complementary therapies worldwide.

Proper systematic reviews fail to show that homeopathy is an effective treatment for rheumatic conditions (see for instance here and here).

13. Wiesenauer M, Lüdtke R. A meta-analysis of the homeopathic treatment of pollinosis with Galphimia glauca. Forschende Komplementärmedizin und Klassische Naturheilkunde, 1996; 3: 230–236.

This is a meta-analysis by Wiesenauer of trials conducted by Wiesenauer.

My own, more recent analysis of these data arrived at a considerably less favourable conclusion: “… three of the four currently available placebo-controlled RCTs of homeopathic Galphimia glauca (GG) suggest this therapy is an effective symptomatic treatment for hay fever. There are, however, important caveats. Most essentially, independent replication would be required before GG can be considered for the routine treatment of hay fever. (Focus on Alternative and Complementary Therapies September 2011 16(3))

14. Taylor MA, Reilly D, Llewellyn-Jones RH, et al. Randomised controlled trials of homoeopathy versus placebo in perennial allergic rhinitis with overview of four trial series. British Medical Journal, 2000; 321: 471–476.

This is a meta-analysis by David Reilly of 4 RCTs which were all conducted by David Reilly. This attracted heavy criticism; see here and here, for instance.

15. Bellavite P, Ortolani R, Pontarollo F, et al. Immunology and homeopathy. 4. Clinical studies – Part 2. Evidence-based Complementary and Alternative Medicine: eCAM, 2006; 3: 397–409.

This is not a systematic review as it lacks any critical assessment of the primary data and includes observational studies and even case series.

16. Schneider B, Klein P, Weiser M. Treatment of vertigo with a homeopathic complex remedy compared with usual treatments: a meta-analysis of clinical trials. Arzneimittelforschung, 2005; 55: 23–29.

This is a meta-analysis of 2 (!) RCTs and 2 observational studies of ‘Vertigoheel’, a preparation which is not a homeopathic but a homotoxicologic remedy (it does not follow the ‘like cures like’ assumption of homeopathy) . Moreover, this product contains pharmacologically active substances (and nobody doubts that active substances can have effects).

________________________________________________________________________________

So, positive evidence from 9 systematic reviews in 6 specific clinical areas?

I let you answer this question.

Forgive me, if this post is long and a bit tedious, but I think it is important.

The claims continue that I am a dishonest falsifier of scientific data, because the renowned Prof R Hahn said so; this, for instance, is from a Tweet that appeared a few days ago

False claims, Edzard Ernst is the worst. Says independent researcher prof Hahn in his blog. His study: https://www.ncbi.nlm.nih.gov/pubmed/24200828 
His blog (German translation) http://www.homeopathy.at/betruegerische-studien-um-homoeopathie-als-wirkungslos-darzustellen…

The source of this permanent flow of defamations is Hahn’s strange article which I have tried to explain several times before. As the matter continues to excite homeopaths around the world, I have decided to give it another go. The following section (in bold) is directly copied from Hahn’s infamous paper where he evaluated several systematic reviews of homeopathy.

_________________________________________________________________________

In 1998, he [Ernst] selected 5 studies using highly diluted remedies from the original 89 and concluded that homeopathy has no effect [5].

In 2000, Ernst and Pittler [6] sought to invalidate the statistically significant superiority of homeopathy over placebo in the 10 studies with the highest Jadad score. The odds ratio, as presented by Linde et al. in 1999 [3], was 2.00 (1.37–2.91). The new argument was that the Jadad score and odds ratio in favor of homeopathy seemed to follow a straight line (in fact, it is asymptotic at both ends). Hence, Ernst and Pittler [6] claimed that the highest Jadad scores should theoretically show zero effect. This reasoning argued that the assumed data are more correct than the real data.

Two years later, Ernst [7] summarized the systematic reviews of homeopathy published in the wake of Linde’s first metaanalysis [2]. To support the view that homeopathy lacks effect, Ernst cited his own publications from 1998 and 2000 [5, 6]. He also presented Linde’s 2 follow-up reports [3, 4] as being further evidence that homeopathy equals placebo. 

_________________________________________________________________________

And that’s it! Except for some snide remarks (copied below) in the discussion section of the article, this is all Hahn has to say about my publications on homeopathy; in other words, he selects 3 of my papers (references are copied below) and (without understanding them, as we will see) vaguely discusses them. In my view, that is remarkable in 3 ways:

  • firstly, there I have published about 100 more papers on homeopathy which Hahn ignores (even though he knows about them as we shall see below);
  • secondly, he does not explain why he selected those 3 and not any others;
  • thirdly, he totally misrepresents all the 3 articles that he has selected.

In the following, I will elaborate on the last point in more detail (anyone capable of running a Medline search and reading Hahn’s article can verify the other points). I will do this by repeating what Hahn states about each of the 3 papers (in bold print), and then explain what each article truly was about.

HERE WE GO

_________________________________________________________________________

FIRST ARTICLE

In 1998, he [Ernst] selected 5 studies using highly diluted remedies from the original 89 and concluded that homeopathy has no effect [5].

This paper [ref 5] was a re-analysis of the Linde Lancet meta-analysis (unfortunately, this paper is not available electronically, but I can send copies to interested parties). For this purpose, I excluded all the studies that did not

  • use homeopathy following the ‘like cures like’ assumption (arguably those studies are not trials of homeopathy at all),
  • use remedies which were not highly diluted and thus contained active molecules (nobody doubts that remedies with pharmacologically active substances can have effects),
  • that did not get the highest rating for methodological quality by Linde et al (flawed trials are known to produce false-positive results).

My methodology was (I think) reasonable, pre-determined and explained in full detail in the article. It left me with 5 placebo-controlled RCTs. A meta-analysis across these 5 trials showed no difference to placebo.

Hahn misrepresents this paper by firstly not explaining what methodology I applied, and secondly by stating that I ‘selected’ the 5 studies from a pool of 89 trials. Yet, I defined my inclusion criteria which were met by just 5 studies.

___________________________________________________________________________

SECOND ARTICLE

In 2000, Ernst and Pittler [6] sought to invalidate the statistically significant superiority of homeopathy over placebo in the 10 studies with the highest Jadad score. The odds ratio, as presented by Linde et al. in 1999 [3], was 2.00 (1.37–2.91). The new argument was that the Jadad score and odds ratio in favor of homeopathy seemed to follow a straight line (in fact, it is asymptotic at both ends). Hence, Ernst and Pittler [6] claimed that the highest Jadad scores should theoretically show zero effect. This reasoning argued that the assumed data are more correct than the real data.

The 1st thing to notice here is that Hahn alleges we had ‘sought to invalidate’. How can he know that? The fact is that we were simply trying to discover something new in the pool of data. The paper he refers to here has been discussed before on this blog. Here is what I stated:

This was a short ‘letter to the editor’ by Ernst and Pittler published in the J Clin Epidemiol commenting on the above-mentioned re-analysis by Linde et al which was published in the same journal. As its text is not available on-line, I re-type parts of it here:

In an interesting re-analysis of their meta-analysis of clinical trials of homeopathy, Linde et al conclude that there is no linear relationship between quality scores and study outcome. We have simply re-plotted their data and arrive at a different conclusion. There is an almost perfect correlation between the odds ratio and the Jadad score between the range of 1-4… [some technical explanations follow which I omit]…Linde et al can be seen as the ultimate epidemiological proof that homeopathy is, in fact, a placebo.

Again Hahn’s interpretation of our paper is incorrect and implies that he has not understood what we actually intended to do here.

_____________________________________________________________________________

THIRD ARTICLE

Two years later, Ernst [7] summarized the systematic reviews of homeopathy published in the wake of Linde’s first metaanalysis [2]. To support the view that homeopathy lacks effect, Ernst cited his own publications from 1998 and 2000 [5, 6]. He also presented Linde’s 2 follow-up reports [3, 4] as being further evidence that homeopathy equals placebo. 

Again, Hahn assumes my aim in publishing this paper (the only one of the 3 papers that is available as full text on-line): ‘to support the view that homeopathy lacks effect’. He does so despite the fact that the paper very clearly states my aim: ‘This article is an attempt to critically evaluate all such papers published since 1997 with a view to defining the clinical effectiveness of homeopathic medicines.‘ This discloses perhaps better than anything else that Hahn’s article is not evidence, but opinion-based and not objective but polemic.

Hahn then seems to resent that I included my own articles. Does he not know that, in a systematic review, one has to include ALL relevant papers? Hahn also seems to imply that I merely included a few papers in my systematic review. In fact, I included all the 17 that were available at the time. It might also be worth mentioning that numerous subsequent and independent analyses that employed similar methodologies as mine arrived at the same conclusions as my review.

_____________________________________________________________________________

Despite Hahn’s overtly misleading statements, he offers little real critique of my work. Certainly Hahn does not state that I made any major mistakes in the 3 papers he cites. For his more vitriolic comments, we need to look at the discussion section of his article where he states:

Ideology Plays a Part

Ernst [7] makes conclusions based on assumed data [6] when the true data are at hand [3]. Ernst [7] invalidates a study by Jonas et al. [18] that shows an odds ratio of 2.19 (1.55–3.11) in favor of homeopathy for rheumatic conditions, using the notion that there are not sufficient data for the treatment of any specific condition [6]. However, his review deals with the overall efficacy of homeopathy and not with specific conditions. Ernst [7] still adds this statistically significant result in favor of homeopathy over placebo to his list of arguments of why homeopathy does not work. Such argumentation must be reviewed carefully before being accepted by the reader.

After re-studying all this in detail, I get the impression that Hahn does not understand (or does not want to understand?) the research questions posed, nor the methodologies employed in my 3 articles. He is remarkably selective in choosing just 3 of my papers (his reference No 7 cites many more of my systematic reviews of homeopathy), and he seems to be determined to get the wrong end of the stick in order to defame me. How he can, based on his ‘analysis’ arrive at the conclusion that ” I have never encountered any scientific writer who is so clearly biased (biased) as this Edzard Ernst“, is totally beyond reason.

In one point, however, Hahn seems to be correct: IDEOLOGY PLAYS A PART (NOT IN MY BUT IN HIS EVALUATION).

_____________________________________________________________________________

REFERENCES AS CITED IN HAHN’S ARTICLE

5 Ernst E: Are highly dilute homeopathic remedies placebos? Perfusion 1998;11:291.

6 Ernst E, Pittler MH: Re-analysis of previous metaanalysis of clinical trials of homeopathy. J Clin Epidemiol 2000;53:1188.

7 Ernst E: A systematic review of systematic reviews of homeopathy. Br J Clin Pharmacol 2002;54:577–582.

______________________________________________________________________________

For more information about Hahn, please see two comments on my previous post (by Björn Geir who understands Hahn’s native language).

This is also where you can find the only comment by Hahn that I am aware of:
Robert Hahn on Saturday 17 September 2016 at 09:50

Somebody alerted me on this website. Dr. Ernst spends most of his effort to reply to my article in Forsch Komplemetmed 2013; 20: 376-381 by discussing who I might be as a person. I hoped to see more effort being put on scientific reasoning.

1. For the scientific part: my experience in scientific reasoning of quite long and extensive. I am the most widely published Swede in the area of anesthesia and intensive care ever. Those who doubt this can look “Hahn RG” on PubMed.

2. For the religious part that, in my mind, has nothing to do with this topic, is that my wife developed a spiritualistic ability in the mid 1990:s which I have explored in four books published in Swedish between 1997 and 2007. I became convinced that much of this is true, but not all. The books reflect interviews with my wife and what happened in our family during that time. Almost half of all Swedes believe in afterlife and in the existence of a spiritual world. Dr. Ernsts reasoning is typical of skeptics, namely that a person with a known religious belief in not to trust – i.e. a person cannot have two sides, a religious and a scientific. I do not agree with that, but the view has led to that almost no scientist dares to tell his religious beliefs to anyone (which Ernst enforces by his reasoning). Besides, I am not very religious person at all, although the years spent writing these books was quite an interesting period of my life. In particular the last book which involved past-life memories that I had been revived during self-hypnotims. I am interested in exploring many sorts of secrets, not only scientific. But all types of evidence must be judged according to its own rules and laws.

3. Why did I write about homeopathy? The reason is a campaign led by skeptics in some summers ago. Teenagers sat in Swedish television and expressed firmly that “there is not a single publication showing that homeopathy works – nothing!”. I wonder how these young boys could know that, and suspected that had simply been instructed to say so by older skeptics . I looked up the topic on PubMed and soon found some positive papers. Not difficult to find. Had they looked? Surely not. I was a frequent blogger at the time, and wrote three blogs summarizing meta-analyses asking the question whether homeopathy was superior to placebo (disregarding the underlying disease). The response for my readers was impressive and I was eventually urged to write it up in English, which I did. That is the background to my article. I have no other involvement in homeopathy.

4. Me and Dr Ernst. I came across his name when scanning articles about homeopathy, and decided to look a bit deeper into what he had written. The typical scenario was to publish meta-analyses but excluding almost all material, leaving very little (of just a scant part of the literature) to summarize. No wonder there were no significant differences. If there were still significant differences the material was typically considered by him to be still too small or too imprecise or whatever to make any conclusion. This was quite systematic, and I lost trust in Ernst´s writings. This was pure scientific reasoning and has nothing to do with religion or anything else.

// Robert Hahn

_________________________________________________________________________

Lastly, if you need more info about Hahn, you might also want to read this.

The HRI is an innovative international charity created to address the need for high quality scientific research in homeopathy… HRI is dedicated to promoting cutting research in homeopathy, using the most rigorous methods available, and communicating the results of such work beyond the usual academic circles… HRI aims to bring academically reliable information to a wide international audience, in an easy to understand form. This audience includes the general public, scientists, healthcare providers, healthcare policy makers, government and the media.

This sounds absolutely brilliant!

I should be a member of the HRI!

For years, I have pursued similar aims!

Hold on, perhaps not?

This article makes me wonder:

START OF QUOTE

… By the end of 2014, 189 randomised controlled trials of homeopathy on 100 different medical conditions had been published in peer-reviewed journals. Of these, 104 papers were placebo-controlled and were eligible for detailed review:
41% were positive (43 trials) – finding that homeopathy was effective
5% were negative (5 trials) – finding that homeopathy was ineffective
54% were inconclusive (56 trials)

How does this compare with evidence for conventional medicine?

An analysis of 1016 systematic reviews of RCTs of conventional medicine had strikingly similar findings2:
44% were positive – the treatments were likely to be beneficial
7% were negative – the treatments were likely to be harmful
49% were inconclusive – the evidence did not support either benefit or harm.

END OF QUOTE

The implication here is that the evidence base for homeopathy is strikingly similar to that of real medicine.

Nice try! But sadly it has nothing to do with ‘reliable information’!!!

In fact, it is grossly (and I suspect deliberately) misleading.

Regular readers of this blog will have spotted the reason, because we discussed (part of) it before. Let me remind you:

_______________________________________________________________________________

A clinical trial is a research tool for testing hypotheses; strictly speaking, it tests the ‘null-hypothesis’: “the experimental treatment generates the same outcomes as the treatment of the control group”. If the trial shows no difference between the outcomes of the two groups, the null-hypothesis is confirmed. In this case, we commonly speak of a negative result. If the experimental treatment was better than the control treatment, the null-hypothesis is rejected, and we commonly speak of a positive result. In other words, clinical trials can only generate positive or negative results, because the null-hypothesis must either be confirmed or rejected – there are no grey tones between the black of a negative and the white of a positive study.

For enthusiasts of alternative medicine, this can create a dilemma, particularly if there are lots of published studies with negative results. In this case, the totality of the available trial evidence is negative which means the treatment in question cannot be characterised as effective. It goes without saying that such an overall conclusion rubs the proponents of that therapy the wrong way. Consequently, they might look for ways to avoid this scenario.

One fairly obvious way of achieving this aim is to simply re-categorise the results. What, if we invented a new category? What, if we called some of the negative studies by a different name? What about INCONCLUSIVE?

That would be brilliant, wouldn’t it. We might end up with a simple statistic where the majority of the evidence is, after all, positive. And this, of course, would give the impression that the ineffective treatment in question is effective!

How exactly do we do this? We continue to call positive studies POSITIVE; we then call studies where the experimental treatment generated worst results than the control treatment (usually a placebo) NEGATIVE; and finally we call those studies where the experimental treatment created outcomes which were not different from placebo INCONCLUSIVE.

In the realm of alternative medicine, this ‘non-conclusive result’ method has recently become incredibly popular . Take homeopathy, for instance. The Faculty of Homeopathy proudly claim the following about clinical trials of homeopathy: Up to the end of 2011, there have been 164 peer-reviewed papers reporting randomised controlled trials (RCTs) in homeopathy. This represents research in 89 different medical conditions. Of those 164 RCT papers, 71 (43%) were positive, 9 (6%) negative and 80 (49%) non-conclusive.

This misleading nonsense was, of course, warmly received by homeopaths. The British Homeopathic Association, like many other organisations and individuals with an axe to grind lapped up the message and promptly repeated it: The body of evidence that exists shows that much more investigation is required – 43% of all the randomised controlled trials carried out have been positive, 6% negative and 49% inconclusive.

Let’s be clear what has happened here: the true percentage figures seem to show that 43% of studies (mostly of poor quality) suggest a positive result for homeopathy, while 57% of them (on average the ones of better quality) were negative. In other words, the majority of this evidence is negative. If we conducted a proper systematic review of this body of evidence, we would, of course, have to account for the quality of each study, and in this case we would have to conclude that homeopathy is not supported by sound evidence of effectiveness.

The little trick of applying the ‘INCONCLUSIVE’ method has thus turned this overall result upside down: black has become white! No wonder that it is so popular with proponents of all sorts of bogus treatments.

__________________________________________________________________________________

But one trick is not enough for the HRI! For thoroughly misinforming the public they have a second one up their sleeve.

And that is ‘comparing apples with pears’  – RCTs with systematic reviews, in their case.

In contrast to RCTs, systematic reviews can be (and often are) INCONCLUSIVE. As they evaluate the totality of all RCTs on a given subject, it is possible that some RCTs are positive, while others are negative. When, for example, the number of high-quality, positive studies included in a systematic review is similar to the number of high-quality, negative trials, the overall result of that review would be INCONCLUSIVE. And this is one of the reasons why the findings of systematic reviews cannot be compared in this way to those of RCTs.

I suspect that the people at the HRI know all this. They are not daft! In fact, they are quite clever. But unfortunately, they seem to employ their cleverness not for informing but for misleading their ‘wide international audience’.

In 2005, I published a systematic review of ophthalmic adverse effects after spinal manipulations. At the time, I found 14 case reports. Clinical symptoms and signs were diverse and included loss of vision, ophthalmoplegia, diplopia and Horner’s syndrome. The underlying mechanism was arterial wall dissection in most cases. The eventual outcome varied and often included permanent deficits. Causality was frequently deemed likely or certain.

I concluded that upper spinal manipulation is associated with ophthalmological adverse effects of unknown frequency. Ophthalmologists should be aware of its risks. Rigorous investigations must be conducted to establish reliable incidence figures.

Now a new article has emerged that throws more light on this issue:

A 46-year-old healthy male with a history of chronic musculoskeletal neck pain presented to the emergency department with left sided weakness after a syncopal episode. The patient had been treated with frequent chiropractic neck manipulations over the past seven years, with his last session one month prior to presentation. One week prior to presentation, the patient developed a new headache, anisocoria, and ptosis of his right upper eyelid. Computed tomography angiography (CTA) of the head and neck showed an internal carotid occlusion with right middle cerebral artery zone of ischemia, and tissue plasminogen activator (tPA) was administered. Subsequently, the patient experienced vision loss in his right eye. MRI and CTA were repeated, revealing a right ICA dissection from below the ophthalmic artery to the posterior communicating artery. On examination, vision in the right eye was no light perception (NLP) and the pupil was amaurotic. Fundus exam showed vascular attenuation, severe pallor of the optic nerve and retina, without a cherry red spot. A diagnosis of ophthalmic artery occlusion was made.

Inpatient workup revealed no stroke risk factors, and he was discharged on aspirin and clopidogrel therapy. Follow up imaging showed re-cannulation of the ICA, although vision remained NLP at outpatient evaluation the following month. Macular spectral domain optical coherence tomography (SDOCT) showed hyperreflectivity of the inner retina diffusely and of the outer retina and retinal pigment epithelium (RPE) centrally. Fluorescein angiography revealed patchy choroidal filling, delayed arterial filling, and macular nonperfusion. Three months after presentation, vision had improved to light perception, and remains stable at one year after the dissection.

Central retinal artery occlusion (CRAO) has been previously described after neck manipulation; however, these cases have been attributed to a dislodged embolic plaque rather than arterial dissection as in this case. Carotid artery dissection after neck manipulation is rare, although the exact incidence is unknown, and may be fatal.

The authors of this case report concluded that internal carotid artery dissection in this case was permanently devastating to the vision of a previously healthy young patient.

What follows is simple:

  1. upper spinal manipulations have no or very little proven benefit;
  2. they are associated with a finite risk;
  3. thus, their risk/benefit balance fails to be positive;
  4. consequently, upper spinal manipulations cannot be recommended as a treatment of any condition.

Personally, I find our good friend Dana Ullman truly priceless. There are several reasons for that; one is that he is often so exemplarily wrong that it helps me to explain fundamental things more clearly. With a bit of luck, this might enable me to better inform people who might be thinking a bit like Dana. In this sense, our good friend Dana has significant educational value.

Recently, he made this comment:

According to present and former editors of THE LANCET and the NEW ENGLAND JOURNAL OF MEDICINE, “evidence based medicine” can no longer be trusted. There is obviously no irony in Ernst and his ilk “banking” on “evidence” that has no firm footing except their personal belief systems: https://medium.com/@drjasonfung/the-corruption-of-evidence-based-medicine-killing-for-profit-41f2812b8704

Ernst is a fundamentalist whose God is reductionistic science, a 20th century model that has little real meaning today…but this won’t stop the new attacks on me personally…

END OF COMMENT

Where to begin?

Let’s start with some definitions.

  • Evidence is the body of facts that leads to a given conclusion. Because the outcomes of treatments such as homeopathy depend on a multitude of factors, the evidence for or against their effectiveness is best based not on experience but on clinical trials and systematic reviews of clinical trials (this is copied from my book).
  • EBM is the integration of best research evidence with clinical expertise and patient values. It thus rests on three pillars: external evidence, ideally from systematic reviews, the clinician’s experience, and the patient’s preferences (and this is from another book).

Few people would argue that EBM, as it is applied currently, is without fault. Certainly I would not suggest that; I even used to give lectures about the limitations of EBM, and many experts (who are much wiser than I) have written about the many problems with EBM. It is important to note that such criticism demonstrates the strength of modern medicine and not its weakness, as Dana seems to think: it is a sign of a healthy debate aimed at generating progress. And it is noteworthy that internal criticism of this nature is largely absent in alternative medicine.

The criticism of EBM is often focussed on the unreliability of the what I called above the ‘best research evidence’. Let me therefore repeat what I wrote about it on this blog in 2012:

… The multifactorial nature of any clinical response requires controlling for all the factors that might determine the outcome other than the treatment per se. Ideally, we would need to create a situation or an experiment where two groups of patients are exposed to the full range of factors, and the only difference is that one group does receive the treatment, while the other one does not. And this is precisely the model of a controlled clinical trial.

Such studies are designed to minimise all possible sources of bias and confounding. By definition, they have a control group which means that we can, at the end of the treatment period, compare the effects of the treatment in question with those of another intervention, a placebo or no treatment at all.

Many different variations of the controlled trial exist so that the exact design can be adapted to the requirements of the particular treatment and the specific research question at hand. The over-riding principle is, however, always the same: we want to make sure that we can reliably determine whether or not the treatment was the cause of the clinical outcome.

Causality is the key in all of this; and here lies the crucial difference between clinical experience and scientific evidence. What clinician witness in their routine practice can have a myriad of causes; what scientists observe in a well-designed efficacy trial is, in all likelihood, caused by the treatment. The latter is evidence, while the former is not.

Don’t get me wrong; clinical trials are not perfect. They can have many flaws and have rightly been criticised for a myriad of inherent limitations. But it is important to realise that, despite all their short-comings, they are far superior than any other method for determining the efficacy of medical interventions.

There are lots of reasons why a trial can generate an incorrect, i.e. a false positive or a false negative result. We therefore should avoid relying on the findings of a single study. Independent replications are usually required before we can be reasonably sure.

Unfortunately, the findings of these replications do not always confirm the results of the previous study. Whenever we are faced with conflicting results, it is tempting to cherry-pick those studies which seem to confirm our prior belief – tempting but very wrong. In order to arrive at the most reliable conclusion about the efficacy of any treatment, we need to consider the totality of the reliable evidence. This goal is best achieved by conducting a systematic review.

In a systematic review, we assess the quality and quantity of the available evidence, try to synthesise the findings and arrive at an overall verdict about the efficacy of the treatment in question. Technically speaking, this process minimises selection and random biases. Systematic reviews and meta-analyses [these are systematic reviews that pool the data of individual studies] therefore constitute, according to a consensus of most experts, the best available evidence for or against the efficacy of any treatment.

END OF QUOTE

Other criticism is aimed at the way EBM is currently used (and abused). This criticism is often justified and necessary, and it is again the expression of our efforts to generate progress. EBM is practised by humans; and humans are far from perfect. They can be corrupt, misguided, dishonest, sloppy, negligent, stupid, etc., etc. Sadly, that means that the practice of EBM can have all of these qualities as well. All we can do is to keep on criticising malpractice, educate people, and hope that this might prevent the worst abuses in future.

Dana and many of his fellow SCAMers have a different strategy; they claim that EBM “can no longer be trusted” (interestingly they never tell us what system might be better; eminence-based medicine? experience-based medicine? random-based medicine? Dana-based medicine?).

The claim that EBM can no longer be trusted is clearly not true, counter-productive and unethical; and I suspect they know it.

Why then do they make it?

Because they feel that it entitles them to argue that homeopathy (or any other form of SCAM) cannot be held to EBM-standards. If EBM is unreliable, surely, nobody can ask the ‘Danas of this world’ to provide anything like sound data!!! And that, of course, would be just dandy for business, wouldn’t it?

So, let’s not be deterred  or misled by these deliberately destructive people. Their motives are transparent and their arguments are nonsensical. EBM is not flawless, but with our continued efforts it will improve. Or, to repeat something that I have said many times before: EBM is the worst form of healthcare, except for all other known options.

My previous post praised the validity and trustworthiness of Cochrane reviews. This post continues in the same line.

Like osteoarthritis, acute stroke has been a condition for which acupuncture-fans prided themselves of being able to produce fairly good evidence. A Cochrane review of 2005, however, was inconclusive and concluded that the number of patients is too small to be certain whether acupuncture is effective for treatment of acute ischaemic or haemorrhagic stroke. Larger, methodologically-sound trials are required.

So, 13 years later, we do have more evidence, and it would be interesting to know what the best evidence tells us today. This new review will tell us because it is the update of the previous Cochrane Review originally published in 2005.

The authors sought randomized clinical trials (RCTs) of acupuncture started within 30 days from stroke onset compared with placebo or sham acupuncture or open control (no placebo) in people with acute ischemic or haemorrhagic stroke, or both. Needling into the skin was required for acupuncture. Comparisons were made versus (1) all controls (open control or sham acupuncture), and (2) sham acupuncture controls.

Two review authors applied the inclusion criteria, assessed trial quality and risk of bias, and extracted data independently. They contacted study authors to ask for missing data and assessed the quality of the evidence by using the GRADE approach. The primary outcome was defined as death or dependency at the end of follow-up.

In total, 33 RCTs with 3946 participants were included. Twenty new trials with 2780 participants had been completed since the previous review. Outcome data were available for up to 22 trials (2865 participants) that compared acupuncture with any control (open control or sham acupuncture) but for only six trials (668 participants) that compared acupuncture with sham acupuncture control. The authors downgraded the evidence to low or very low quality because of risk of bias in included studies, inconsistency in the acupuncture intervention and outcome measures, and imprecision in effect estimates.

When compared with any control (11 trials with 1582 participants), findings of lower odds of death or dependency at the end of follow-up and over the long term (≥ three months) in the acupuncture group were uncertain (odds ratio [OR] 0.61, 95% confidence interval [CI] 0.46 to 0.79; very low-quality evidence; and OR 0.67, 95% CI 0.53 to 0.85; eight trials with 1436 participants; very low-quality evidence, respectively) and were not confirmed by trials comparing acupuncture with sham acupuncture (OR 0.71, 95% CI 0.43 to 1.18; low-quality evidence; and OR 0.67, 95% CI 0.40 to 1.12; low-quality evidence, respectively).In trials comparing acupuncture with any control, findings that acupuncture was associated with increases in the global neurological deficit score and in the motor function score were uncertain (standardized mean difference [SMD] 0.84, 95% CI 0.36 to 1.32; 12 trials with 1086 participants; very low-quality evidence; and SMD 1.08, 95% CI 0.45 to 1.71; 11 trials with 895 participants; very low-quality evidence).

These findings were not confirmed in trials comparing acupuncture with sham acupuncture (SMD 0.01, 95% CI -0.55 to 0.57; low-quality evidence; and SMD 0.10, 95% CI -0.38 to 0.17; low-quality evidence, respectively).Trials comparing acupuncture with any control reported little or no difference in death or institutional care at the end of follow-up (OR 0.78, 95% CI 0.54 to 1.12; five trials with 1120 participants; low-quality evidence), death within the first two weeks (OR 0.91, 95% CI 0.33 to 2.55; 18 trials with 1612 participants; low-quality evidence), or death at the end of follow-up (OR 1.08, 95% CI 0.74 to 1.58; 22 trials with 2865 participants; low-quality evidence).

The incidence of adverse events (eg, pain, dizziness, fainting) in the acupuncture arms of open and sham control trials was 6.2% (64/1037 participants), and 1.4% of these (14/1037 participants) discontinued acupuncture. When acupuncture was compared with sham acupuncture, findings for adverse events were uncertain (OR 0.58, 95% CI 0.29 to 1.16; five trials with 576 participants; low-quality evidence).

The authors concluded that this updated review indicates that apparently improved outcomes with acupuncture in acute stroke are confounded by the risk of bias related to use of open controls. Adverse events related to acupuncture were reported to be minor and usually did not result in stopping treatment. Future studies are needed to confirm or refute any effects of acupuncture in acute stroke. Trials should clearly report the method of randomization, concealment of allocation, and whether blinding of participants, personnel, and outcome assessors was achieved, while paying close attention to the effects of acupuncture on long-term functional outcomes.

These cautious conclusions might be explained by the fact that Chinese researchers are reluctant to state anything overtly negative about any TCM therapy. Recently, one expert who spoke out was even imprisoned for criticising a TCM product! But in truth, this review really shows that acupuncture has no convincing effect in acute stroke.

And for me, this conclusion is fascinating. I have been involved in acupuncture/stroke research since the early 1990s.

Our RCT produced a resounding negative result concluding that acupuncture is not superior to sham treatment for recovery in activities of daily living and health-related quality of life after stroke, although there may be a limited effect on leg function in more severely affected patients.

Our 1996 systematic review concluded that the evidence that acupuncture is a useful adjunct for stroke rehabilitation is encouraging but not compelling.

By 2001, more data had become available but the conclusion became even less encouraging: there is no compelling evidence to show that acupuncture is effective in stroke rehabilitation.

Finally, by 2010, there were 10 RCT and we were able to do a meta-analysis of the data. We concluded that our meta-analyses of data from rigorous randomized sham-controlled trials did not show a positive effect of acupuncture as a treatment for functional recovery after stroke.

Yes, my reviews were on slightly different research questions. Yet, they do reveal how a critical assessment of the slowly emerging evidence had to arrive at more and more negative conclusions about the role of acupuncture in the management of stroke patients. For a long time, this message was in stark contrast to what acupuncture-fans were claiming. I wonder whether they will now finally change their mind.

1 2 3 22
Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted.


Click here for a comprehensive list of recent comments.

Categories