I have said it often, and I say it again: I do like well-conducted systematic reviews; and Cochrane reviews are usually the best, i. e. most transparent, most thorough and least biased. Thus, I was pleased to see a new Cochrane review of acupuncture aimed at assessing the benefits and harms of acupuncture in patients with hip OA.
The authors included randomized controlled trials (RCTs) that compared acupuncture with sham acupuncture, another active treatment, or no specific treatment; and RCTs that evaluated acupuncture as an addition to another treatment. Major outcomes were pain and function at the short term (i.e. < 3 months after randomization) and adverse events.
Six RCTs with 413 participants were included. Four RCTs included only people with OA of the hip, and two included a mix of people with OA of the hip and knee. All RCTs included primarily older participants, with a mean age range from 61 to 67 years, and a mean duration of hip OA pain from two to eight years. Approximately two-thirds of participants were women. Two RCTs compared acupuncture versus sham acupuncture; the other four RCTs were not blinded. All results were evaluated at short-term (i.e. four to nine weeks after randomization).In the two RCTs that compared acupuncture to sham acupuncture, the sham acupuncture control interventions were judged believable, but each sham acupuncture intervention was also judged to have a risk of weak acupuncture-specific effects, due to placement of non-penetrating needles at the correct acupuncture points in one RCT, and the use of penetrating needles not inserted at the correct points in the other RCT. For these two sham-controlled RCTs, the risk of bias was low for all outcomes.
The combined analysis of two sham-controlled RCTs gave moderate quality evidence of little or no effect in reduction in pain for acupuncture relative to sham acupuncture. Due to the small sample sizes in the studies, the confidence interval includes both the possibility of moderate benefit and the possibility of no effect of acupuncture (120 participants; Standardized Mean Difference (SMD) -0.13, (95% Confidence Interval (CI) -0.49 to 0.22); 2.1 points greater improvement with acupuncture compared to sham acupuncture on 100 point scale (i.e., absolute percent change -2.1% (95% CI -7.9% to 3.6%)); relative percent change -4.1% (95% CI -15.6% to 7.0%)). Estimates of effect were similar for function (120 participants; SMD -0.15, (95% CI -0.51 to 0.21)). No pooled estimate, representative of the two sham-controlled RCTs, could be calculated or reported for the quality of life outcome.
The four other RCTs were unblinded comparative effectiveness RCTs, which compared (additional) acupuncture to four different active control treatments. There was low quality evidence that addition of acupuncture to the routine primary care that RCT participants were receiving from their physicians was associated with statistically significant and clinically relevant benefits, compared to the routine primary physician care alone, in pain (1 RCT; 137 participants; mean percent difference -22.9% (95% CI -29.2% to -16.6%); relative percent difference -46.5% (95% CI -59.3% to -33.7%)) and function (mean percent difference -19.0% (95% CI -24.41 to -13.59); relative percent difference -38.6% (95% CI -49.6% to -27.6%)). There was no statistically significant difference for mental quality of life and acupuncture showed a small, significant benefit for physical quality of life.
The effects of acupuncture compared with either advice plus exercise or NSAIDs are uncertain. The authors are also uncertain whether acupuncture plus patient education improves pain, function, and quality of life, when compared to patient education alone.
In general, the overall quality of the evidence for the four comparative effectiveness RCTs was low to very low, mainly due to the potential for biased reporting of patient-assessed outcomes due to lack of blinding and sparse data.
Information on safety was reported in 4 RCTs. Two RCTs reported minor side effects of acupuncture, which were primarily minor bruising, bleeding, or pain at needle insertion sites.
The authors concluded that acupuncture probably has little or no effect in reducing pain or improving function relative to sham acupuncture in people with hip osteoarthritis. Due to the small sample size in the studies, the confidence intervals include both the possibility of moderate benefits and the possibility of no effect of acupuncture. One unblinded trial found that acupuncture as an addition to routine primary physician care was associated with benefits on pain and function. However, these reported benefits are likely due at least partially to RCT participants’ greater expectations of benefit from acupuncture. Possible side effects associated with acupuncture treatment were minor.
This is an excellent review of data that (because of contradictions, methodological limitations, heterogeneity etc.) are not easy to evaluate fairly. The review shows that previous verdicts about acupuncture for osteoarthritis might have been too optimistic. Acupuncture has no or only very small specific therapeutic effects. As we have much better therapeutic options for this condition, it means that acupuncture can no longer be recommended as an effective therapy.
That surely must be big news in the little world of acupuncture!
I have been personally involved in several similar reviews:
In 2006, the balance of evidence seemed to have shifted and more positive data had emerged; consequently our review concluded that sham-controlled RCTs suggest specific effects of acupuncture for pain control in patients with peripheral joint OA. Considering its favourable safety profile acupuncture seems an option worthy of consideration particularly for knee OA. Further studies are required particularly for manual or electro-acupuncture in hip OA.
Now, it seems that my initial conclusion of 1996 was more realistic. To me this is a fascinating highlight on the fact that in EBM, we change our minds based on the current best evidence. By contrast, in alternative medicine, as we have often lamented on this blog, minds do not easily change and all too often dogma seems to reign.
The new Cochrane review is important in several ways. Firstly, it affirms an appropriately high standard for such reviews. Secondly, it originates from a research team that has, in the past, been outspokenly pro-acupuncture; it is therefore unlikely that the largely negative findings were due to an anti-acupuncture bias. Thirdly – and most importantly – osteoarthritis has been THE condition for which even critical reviewers had to admit that there was at least some good, positive evidence.
It seems therefore, that yet again a beautiful theory has been slain by an ugly fact.
Have you ever wondered whether doctors who practice homeopathy are different from those who don’t.
Silly question, of course they are! But how do they differ?
Having practised homeopathy myself during my very early days as a physician, I have often thought about this issue. My personal (and not very flattering) impressions were noted in my memoir where I describe my experience working in a German homeopathic hospital:
… some of my colleagues used homeopathy and other alternative approaches because they could not quite cope with the often exceedingly high demands of conventional medicine. It is almost understandable that, if a physician was having trouble comprehending the multifactorial causes and mechanisms of disease and illness, or for one reason or another could not master the equally complex process of reaching a diagnosis or finding an effective therapy, it might be tempting instead to employ notions such as dowsing, homeopathy or acupuncture, whose theoretical basis, unsullied by the inconvenient absolutes of science, was immeasurably more easy to grasp.
Some of my colleagues in the homeopathic hospital were clearly not cut out to be “real” doctors. Even a very junior doctor like me could not help noticing this somewhat embarrassing fact…
But this is anecdote and not evidence!
So, where is the evidence?
It was published last week and made headlines in many UK daily papers.
Our study was aimed at finding out whether English GP practices that prescribe any homeopathic preparations might differ in their prescribing of other drugs. We identified practices that made any homeopathy prescriptions over six months of data. We measured associations with four prescribing and two practice quality indicators using multivariable logistic regression.
Only 8.5% of practices (644) prescribed homeopathy between December 2016 and May 2017. Practices in the worst-scoring quartile for a composite measure of prescribing quality were 2.1 times more likely to prescribe homeopathy than those in the best category. Aggregate savings from the subset of these measures where a cost saving could be calculated were also strongly associated. Of practices spending the most on medicines identified as ‘low value’ by NHS England, 12.8% prescribed homeopathy, compared to 3.9% for lowest spenders. Of practices in the worst category for aggregated price-per-unit cost savings, 12.7% prescribed homeopathy, compared to 3.5% in the best category. Practice quality outcomes framework scores and patient recommendation rates were not associated with prescribing homeopathy.
We concluded that even infrequent homeopathy prescribing is strongly associated with poor performance on a range of prescribing quality measures, but not with overall patient recommendation or quality outcomes framework score. The association is unlikely to be a direct causal relationship, but may reflect underlying practice features, such as the extent of respect for evidence-based practice, or poorer stewardship of the prescribing budget.
Since our study was reported in almost all of the UK newspapers, it comes as no surprise that, in the interest of ‘journalistic balance’, homeopaths were invited to give their ‘expert’ opinions on our work.
Margaret Wyllie, head of the British Homeopathic Association, was quoted commenting: “This is another example of how real patient experience and health outcomes are so often discounted, when in actuality they should be the primary driver for research to improve our NHS services. This study provides no useful evidence about homeopathy, or about prescribing, and gives absolutely no data that can improve the health of people in the UK.”
The Faculty of Homeopathy was equally unhappy about our study and stated: “The study did not include any measures of patient outcomes, so it doesn’t tell us how the use of homeopathy in English general practice correlates with patients doing well or badly, nor with how many drugs they use.”
Cristal Summer from the Society of Homeopathy said that our research was just a rubbish bit of a study.
Peter Fisher, the Queen’s homeopath and the president of the Faculty of Homeopathy, stated: “We don’t know if these measures correlate with what matters to patients – whether they get better and have side-effects.”
A study aimed at determining whether GP practices that prescribe homeopathic preparations differ in their prescribing habits from those that do not prescribe homeopathics can hardly address these questions, Peter. A test of washing machines can hardly tell us much about the punctuality of trains. And an investigation into the risks of bungee jumping will not inform us about the benefits of regular exercise. Call me biased, but to me these comments indicate mainly one thing: HOMEOPATHS SEEM TO HAVE GREAT DIFFICULTIES UNDERSTANDING SCIENTIFIC PAPERS.
I much prefer the witty remarks of Catherine Bennett in yesterday’s Observer: Homeopath-GPs, naturally, have mustered in response and challenge Goldacre’s findings, with a concern for methodology that could easily give the impression that there is some evidential basis for their parallel system, beyond the fact that the Prince of Wales likes it. In fairness to Charles, his upbringing is to blame. But what is the doctors’ excuse?
I have written about the use of homeopathy in France before (as I now live half of my time in France, this is a subject of considerable interest to me). After decades of deafening silence and uncritical acceptance by the French public, it seems that finally some change to the better might be on its way. Recently, a sizable number of prominent doctors protested publicly against the fact that, despite its implausibility and the lack of proof of efficacy, homeopathy continues to be reimbursed in France and scarce funds are being wasted on it. This action seems to have put pressure on officials to respond.
Yesterday (just in time for the ‘HOMEOPATHIC AWARENESS WEEK’) the French minister of health was quoted making a statement on homeopathy. Here is my translation of what Agnès Buzyn was quoted saying:
“There is a continuous evaluation of the medicines we call complementary. A working group* at the head office of my department checks that all these practices are not dangerous. If a therapy continues to be beneficial without being harmful, it continues to be reimbursed… The French are very attached [to homeopathy]; it’s probably a placebo effect. If it can prevent the use of toxic medicine, I think that we all win. I does not hurt.”
- I would like to know who they are, how they can be contacted, and whether they would consider recruiting my assistance in evaluating alternative therapies.
So, if I understand her correctly, Agnès Buzyn believes that:
- the French people are fond of homeopathy;
- homeopathy is a placebo-therapy;
- homeopathy does no harm;
- homeopathy can even prevent harm from conventional medicine;
- on balance, therefore, homeopathy should continue to be reimbursed in France.
My views of this type of reasoning have been expressed repeatedly. Nevertheless, I will briefly state them again:
- true but not relevant; healthcare is not a popularity contest; and the current popularity is essentially the result of decades of systematic misinformation of consumers;
- wrong: we have, on this blog, discussed ad nauseam how homeopathy can cause serious harm; for instance, whenever it replaces effective treatments, it can cause serious harm and might even kill patients;
- if doctors harm patients by needlessly prescribing harmful treatments, we need to re-train them and stop this abuse; using homeopathy is not the solution to bad medicine;
- wrong: the reimbursement of homeopathy is a waste of money and undermines evidence-based medicine.
So, what’s the conclusion?
Politicians are usually not good at understanding science or scientific evidence. They (have to?) think in time spans from one election to the next. And they are, of course, keenly aware that, in order to stay in power, they rely on the vote of the people. Therefore, the popularity of homeopathy (even though it is scientifically irrelevant) is a very real factor for them. This means that, on a political level, homeopathy is sadly much more secure than it should be. In turn, this means we need to:
- use different arguments when arguing with politicians (for instance, the economic impact of wasting money on placebo-therapies, or the fact that systematically misinforming the public is highly unethical and counter-productive),
- and make politicians understand science better than they do at present, perhaps even insist that ministers are experts in their respective areas (i. e. a minister of health fully understands the fundamental issues of healthcare).
Does that mean the new developments in the realm of French homeopathy are all doomed to failure?
No, I don’t think so – at least (and at last) we have a vocal group of doctors protesting against wasteful nonsense, and a fairly sound and accurate statement from a French minister of health:
HOMEOPATHY, IT’S PROBABLY A PLACEBO EFFECT!
In recent days, journalists across the world had a field day (mis)reporting that doctors practising integrative medicine were doing something positive after all. I think that the paper shows nothing of the kind – but please judge for yourself.
The authors of this article wanted to determine differences in antibiotic prescription rates between conventional General Practice (GP) surgeries and GP surgeries employing general practitioners (GPs) additionally trained in integrative medicine (IM) or complementary and alternative medicine (CAM) (referred to as IM GPs) working within National Health Service (NHS) England.
They conducted a retrospective study on antibiotic prescription rates per STAR-PU (Specific Therapeutic group Age–sex weighting Related Prescribing Unit) using NHS Digital data over 2016. Publicly available data were used on prevalence of relevant comorbidities, demographics of patient populations and deprivation scores. setting Primary Care. Participants were 7283 NHS GP surgeries in England. The association between IM GPs and antibiotic prescribing rates per STAR-PU with the number of antibiotic prescriptions (total, and for respiratory tract infection (RTI) and urinary tract infection (UTI) separately) as outcome. results IM GP surgeries (n=9) were comparable to conventional GP surgeries in terms of list sizes, demographics, deprivation scores and comorbidity prevalence.
Statistically significant fewer total antibiotics were prescribed at NHS IM GP surgeries compared with conventional NHS GP surgeries. In contrast, the number of antibiotics prescribed for UTI were similar between both practices.
The authors concluded that NHS England GP surgeries employing GPs additionally trained in IM/CAM have lower antibiotic prescribing rates. Accessibility of IM/CAM within NHS England primary care is limited. Main study limitation is the lack of consultation data. Future research should include the differences in consultation behaviour of patients self-selecting to consult an IM GP or conventional surgery, and its effect on antibiotic prescription. Additional treatment strategies for common primary care infections used by IM GPs should be explored to see if they could be used to assist in the fight against antimicrobial resistance.
The study was flimsy to say the least:
- It was retrospective and is therefore open to no end of confounders.
- There were only 9 surgeries in the IM group.
Moreover, the results were far from impressive. The differences in antibiotic prescribing between the two groups of GP surgeries were minimal or non-existent. Finally, the study was financed via an unrestricted grant of WALA Heilmittel GmbH, Germany (“approx. 900 different remedies conforming to the anthroposophic understanding of man and nature”) and its senior author has a long track record of publishing papers promotional for anthroposophic medicine.
Such pseudo-research seems to be popular in the realm of CAM, and I have commented before on similarly futile projects. The comparison, I sometimes use is that of a Hamburger restaurant:
Employees by a large Hamburger chain set out to study the association between utilization of Hamburger restaurant services and vegetarianism. The authors used a retrospective cohort design. The study population comprised New Hampshire residents aged 18-99 years, who had entered the premises of a Hamburger restaurant within 90 days for a primary purpose of eating. The authors excluded subjects with a diagnosis of cancer. They measured the likelihood of vegetarianism among recipients of services delivered by Hamburger restaurants compared with a control group of individuals not using meat-dispensing facilities. They also compared the cohorts with regard to the money spent in Hamburger restaurants. The adjusted likelihood of being a vegetarian was 55% lower among the experimental group compared to controls. The average money spent per person in Hamburger restaurants were also significantly lower among the Hamburger group.
To me, it is obvious that such analyses must produce a seemingly favourable result for CAM. In the present case, there are several reasons for this:
- GPs who volunteer to be trained in CAM tend to be in favour of ‘natural’ treatments and oppose synthetic drugs such as antibiotics.
- Education in CAM would only re-inforce this notion.
- Similarly, patients electing to consult IM GPs tend to be in favour of ‘natural’ treatments and oppose synthetic drugs such as antibiotics.
- Such patients might be less severely ill that the rest of the patient population (the data from the present study do in fact imply this to be true).
- These phenomena work in concert to generate less antibiotic prescribing in the IM group.
In the final analysis, all this finding amounts to is a self-fulfilling prophecy: grocery shops sell less meat than butchers! You don’t believe me? Perhaps you need to read a previous post then; it concluded that physicians practicing integrative medicine (the 80% who did not respond to the survey were most likely even worse) not only use and promote much quackery, they also tend to endanger public health by their bizarre, irrational and irresponsible attitudes towards vaccination.
What is upsetting with the present paper, in my view, are the facts that:
- a reputable journal published this junk,
- the international press has a field-day reporting this study implying that CAM is a good thing.
The fact is that it shows nothing of the kind. Imagine we send GPs on a course where they are taught to treat all their patients with blood-letting. This too would result in less prescription of antibiotics, wouldn’t it? But would it be a good thing? Of course not!
True, we prescribe too much antibiotics. Nobody doubts that. And nobody doubts that it is a big problem. The solution to this problem is not more CAM, but less antibiotics. To realise the solution we do not need to teach GPs CAM but we need to remind them of the principles of evidence-based practice. And the two are clearly not the same; in fact, they are opposites.
The media have (rightly) paid much attention to the three Lancet-articles on low back pain (LBP) which were published this week. LBP is such a common condition that its prevalence alone renders it an important subject for us all. One of the three papers covers the treatment and prevention of LBP. Specifically, it lists various therapies according to their effectiveness for both acute and persistent LBP. The authors of the article base their judgements mainly on published guidelines from Denmark, UK and the US; as these guidelines differ, they attempt a synthesis of the three.
Several alternative therapist organisations and individuals have consequently jumped on the LBP bandwagon and seem to feel encouraged by the attention given to the Lancet-papers to promote their treatments. Others have claimed that my often critical verdicts of alternative therapies for LBP are out of line with this evidence and asked ‘who should we believe the international team of experts writing in one of the best medical journals, or Edzard Ernst writing on his blog?’ They are trying to create a division where none exists,
The thing is that I am broadly in agreement with the evidence presented in Lancet-paper! But I also know that things are a bit more complex.
Below, I have copied the non-pharmacological, non-operative treatments listed in the Lancet-paper together with the authors’ verdicts regarding their effectiveness for both acute and persistent LBP. I find no glaring contradictions with what I regard as the best current evidence and with my posts on the subject. But I feel compelled to point out that the Lancet-paper merely lists the effectiveness of several therapeutic options, and that the value of a treatment is not only determined by its effectiveness. Crucial further elements are a therapy’s cost and its risks, the latter of which also determines the most important criterion: the risk/benefit balance. In my version of the Lancet table, I have therefore added these three variables for non-pharmacological and non-surgical options:
|EFFECTIVENESS ACUTE LBP||EFFECTIVENESS PERSISTENT LBP||RISKS||COSTS||RISK/BENEFIT BALANCE|
|Advice to stay active||+, routine||+, routine||None||Low||Positive|
|Education||+, routine||+, routine||None||Low||Positive|
|Superficial heat||+/-||Ie||Very minor||Low to medium||Positive (aLBP)|
|Exercise||Limited||+/-, routine||Very minor||Low||Positive (pLBP)|
|CBT||Limited||+/-, routine||None||Low to medium||Positive (pLBP)|
|Rehab||Ie||+/-||Minor||Medium to high||Questionable|
Routine = consider for routine use
+/- = second line or adjunctive treatment
Ie = insufficient evidence
Limited = limited use in selected patients
vfbmae = very frequent, minor adverse effects
sae = serious adverse effects, including deaths, are on record
aLBP = acute low back pain
The reason why my stance, as expressed on this blog and elsewhere, is often critical about certain alternative therapies is thus obvious and transparent. For none of them (except for massage) is the risk/benefit balance positive. And for spinal manipulation, it even turns out to be negative. It goes almost without saying that responsible advice must be to avoid treatments for which the benefits do not demonstrably outweigh the risks.
I imagine that chiropractors, osteopaths and acupuncturists will strongly disagree with my interpretation of the evidence (they might even feel that their cash-flow is endangered) – and I am looking forward to the discussions around their objections.
Homeopathy has always enjoyed a special status in Germany, its country of origin. Germans use homeopathy more often than the citizens of most other countries, they spend more money on it, and they even have elevated it to some kind of medical speciality. In 2003, the German medical profession re-considered the requirements for carrying the title of ‘Doctor of Homeopathy’. It was decided that only physicians who already were specialists in one medical field were allowed to be certified with this title after a post-graduate education and training programme of 6 months, or 100 hours of case studies under supervision plus 160 hours of course work. Many German physicians seem to find this rigorously regulated programme attractive, opted for it, and earn good money with it; the number of ‘doctors of homeopathy’ has risen from 2212 to 6712 between 1993 and 2009.
Personally, I find much of this surprising, even laughable, and have repeatedly stated that even the most rigorously regulated education in nonsense can only result in nonsense.
Luckily, I am not alone. A multidisciplinary group of experts (Muensteraner Kreis) has just filed an official application with the current 121st General Assembly of the German medical profession to completely abolish the title ‘Doctor of Homeopathy’. Our application itself is a lengthy document outlining in some detail the nature of our arguments. Here, I will merely translate its conclusion:
Even though present in science-business, homeopathy is not scientifically founded. Its basis – potentisation and the simile principle – contradicts scientific facts; homeopathy therefore must be categorised as esoteric. The international scientific community does not interpret the clinical studies of homeopathy as a sufficient proof for its efficacy. Giving an esoteric approach to medicine the veneer of credibility by officially establishing the title ‘Doctor of Homeopathy’ contradicts the physicians’ claim of a scientifically-based medicine and weakens the status of the science-based medicine through blurring the boundaries between science and belief. Problems within science-based medicine must be solved internally and cannot be unburdened onto an unscientific approach to medicine. We consider the abolishment of the ‘Doctor of Homeopathy’ to be urgently indicated.
END OF MY TRANSLATION
I think it would be more than a little over-optimistic to assume that the Assembly will swiftly adopt our suggestion. Perhaps this is also not the intention of our application. In Germany (I learnt my homeopathy in this country), homeopathy is still very much protected by powerful lobby groups and financial interests, as well as loaded with heavy emotional baggage. Yet I do hope that our application will start a discussion which, eventually, will bring a rational resolution to the embarrassing anachronism of the ‘Doctor of Homeopathy’ (Arzt fuer Homoeopathie).
The German medical profession might even have the opportunity to be internationally at the forefront of reason and progress.
Doctor Jonas is an important figure head of US ‘Integrative Medicine’. As we discussed in a recent post, he pointed out that many US hospital doctors fail to answer the following questions relating to their chronically ill patients:
- “What matters most for this patient?
- What is the person’s lifestyle like – their nutrition, movement and sleep?
- How does that patient manage their stress?
- Does that patient have a good support system at home?
- What supplements does that patient take? Has your patient seen any CAM practitioners to cope with their condition?
- Why do they want to get well?”
In my previous post, I tried to explain that this is embarrassing – embarrassing for doctor Jonas, I meant.
But Jonas also claims that most US hospital doctors he addressed during his lecture tour, were unable to answer these questions. And that might be embarrassing not for Jonas, but for those physicians. Let’s consider this possibility for a moment.
The way I see it, the doctors in question might not have answered to Jonas for the following reasons:
- They felt that the questions were simply too daft to bother.
- They were too polite to tell Jonas what they think of him.
- They were truly unable to answer the questions.
Here I want to briefly deal with the last category.
I do not doubt for a minute that this category of physician exists. They have little interest in what matters to their patients, don’t ask the right questions, have no time and even less empathy and compassion. Yet nobody can deny that medical school teaches all of these qualities, skills and attitudes. And there is no doubt that good doctors practice them; it is not a choice but an ethical and moral imperative.
So, what went wrong with these doctors?
Probably lots, and I cannot begin to tell you what exactly. However, I can easily tell you that those doctors are not practicing good medicine. Similarly, I can tell you what these doctors ought to do: re-train and be reminded of what medical school has once taught them.
And what about those physicians who advocate ‘integrated medicine’ reminding everyone of the core values of healthcare?
Aren’t they fabulous?
No, they aren’t!
Because they too have evidently forgotten what they should have learnt at medical school. If not, they would not be able to pretend that ‘integrative medicine’ has a monopoly on core values of all healthcare. Their messages are akin to a new ‘school’ of ship-building insisting that it is beneficial to build ships that do not leak.
What I am trying to say in my clumsy way is this:
DOCTORS WHO PRACTICE BAD MEDICINE SHOULD RE-TRAIN – TOGETHER WITH THOSE PHYSICIANS WHO ADVOCATE ‘INTEGRATIVE MEDICINE‘, BECAUSE THEY BOTH HAVE FORGOTTEN WHAT THEY LEARNT AT MEDICAL SCHOOL.
The website of BMJ Clinical Evidence seems to be popular with fans of alternative medicine (FAMs). That sounds like good news: it’s an excellent source, and one can learn a lot about EBM when studying it. But there is a problem: FAMs don’t seem to really study it (alternatively they do not have the power of comprehension to understand the data); they merely pounce on this figure and cite it endlessly:
They interpret it to mean that only 11% of what conventional clinicians do is based on sound evidence. This is water on their mills, because now they feel able to claim:
THE MAJORITY OF WHAT CONVENTIONAL CLINICIANS DO IS NOT EVIDENCE-BASED. SO, WHY DO SO-CALLED RATIONAL THINKERS EXPECT ALTERNATIVE THERAPIES TO BE EVIDENCE-BASED? IF WE NEEDED PROOF THAT THEY ARE HYPOCRITES, HERE IT IS!!!
The question is: are these FAMs correct?
The answer is: no!
They are merely using a logical fallacy (tu quoque); what is worse, they use it based on misunderstanding the actual data summarised in the above figure.
Let’s look at this in a little more detail.
The first thing we need to understand the methodologies used by ‘Clinical Evidence’ and what the different categories in the graph mean. Here is the explanation:
So, arguably the top three categories amounting to 42% signify some evidential support (if we decided to be more rigorous and merely included the two top categories, we would still arrive at 35%). This is not great, but we must remember two things here:
- EBM is fairly new;
- lots of people are working hard to improve the evidence base of medicine so that, in future, these figures will be better (by contrast, in alternative medicine, no similar progress is noticeable).
The second thing that strikes me is that, in alternative medicine, these figures would surely be much, much worse. I am not aware of reliable estimates, but I guess that the percentages might be one dimension smaller.
The third thing to mention is that the figures do not cover the entire spectrum of treatments available today but are based on ~ 3000 selected therapies. It is unclear how they were chosen, presumably the choice is pragmatic and based on the information available. If an up-to date systematic review has been published and provided the necessary information, the therapy was included. This means that the figures include not just mainstream but also plenty of alternative treatments (to the best of my knowledge ‘Clinical Evidence’ makes no distinction between the two). It is thus nonsensical to claim that the data highlight the weakness of the evidence in conventional medicine. It is even possible that the figures would be better, if alternative treatments had been excluded (I estimate that around 2 000 systematic reviews of alternative therapies have been published [I am the author of ~400 of them!]).
The fourth and possibly the most important thing to mention is that the percentage figures in the graph are certainly NOT a reflection of what percentage of treatments used in routine care are based on good evidence. In conventional practice, clinicians would, of course, select where possible those treatments with the best evidence base, while leaving the less well documented ones aside. In other words, they will use the ones in the two top categories much more frequently than those from the other categories.
At this stage, I hear some FAMs say: how does he know that?
Because several studies have been published that investigated this issue in some detail. They have monitored what percentage of interventions used by conventional clinicians in their daily practice are based on good evidence. In 2004, I reviewed these studies; here is the crucial passage from my paper:
“The most conclusive answer comes from a UK survey by Gill et al who retrospectively reviewed 122 consecutive general practice consultations. They found that 81% of the prescribed treatments were based on evidence and 30% were based on randomised controlled trials (RCTs). A similar study conducted in a UK university hospital outpatient department of general medicine arrived at comparable figures; 82% of the interventions were based on evidence, 53% on RCTs. Other relevant data originate from abroad. In Sweden, 84% of internal medicine interventions were based on evidence and 50% on RCTs. In Spain these percentages were 55 and 38%, respectively. Imrie and Ramey pooled a total of 15 studies across all medical disciplines, and found that, on average, 76% of medical treatments are supported by some form of compelling evidence — the lowest was that mentioned above (55%),6 and the highest (97%) was achieved in anaesthesia in Britain. Collectively these data suggest that, in terms of evidence-base, general practice is much better than its reputation.”
My conclusions from all this:
FAMs should study the BMJ Clinical Evidence more thoroughly. If they did, they might comprehend that the claims they tend to make about the data shown there are, in fact, bogus. In addition, they might even learn a thing or two about EBM which might eventually improve the quality of the debate.
The new guidelines by the American College of Physicians entitled ‘Noninvasive Treatments for Acute, Subacute, and Chronic Low Back Pain: A Clinical Practice Guideline From the American College of Physicians’ have already been the subject of the previous post. Today, I want to have a closer look at a small section of these guidelines which, I think, is crucial. It is entitled ‘HARMS OF NONPHARMACOLOGIC THERAPIES’. I have taken the liberty of copying it below:
“Evidence on adverse events from the included RCTs and systematic reviews was limited, and the quality of evidence for all available harms data is low. Harms were poorly reported (if they were reported at all) for most of the interventions.
Low-quality evidence showed no reported harms or serious adverse events associated with tai chi, psychological interventions, multidisciplinary rehabilitation, ultrasound, acupuncture, lumbar support, or traction (9,95,150,170–174). Low-quality evidence showed that when harms were reported for exercise, they were often related to muscle soreness and increased pain, and no serious harms were reported. All reported harms associated with yoga were mild to moderate (119). Low-quality evidence showed that none of the RCTs reported any serious adverse events with massage, although 2 RCTs reported soreness during or after massage therapy (175,176). Adverse events associated with spinal manipulation included muscle soreness or transient increases in pain (134). There were few adverse events reported and no clear differences between MCE and controls. Transcutaneous electrical nerve stimulation was associated with an increased risk for skin site reaction but not serious adverse events (177). Two RCTs (178,179) showed an increased risk for skin flushing with heat compared with no heat or placebo, and no serious adverse events were reported. There were no data on cold therapy. Evidence was insufficient to determine harms of electrical muscle stimulation, LLLT, percutaneous electrical nerve stimulation, interferential therapy, short-wave diathermy, and taping.”
The first thing that strikes me is the brevity of the section. Surely, guidelines of this nature must include a full discussion of the risks of the treatments in question!
The second thing that is noteworthy is the fact that the authors confirm the fact I have been banging on about for years: clinical trials of alternative therapies far too often fail to mention adverse effects. I have often pointed out that the failure to report adverse effects in clinical trials is an unacceptable violation of medical ethics. By contrast, the guideline authors seem not to feel strongly about this omission.
The third thing that is noteworthy is that the guidelines evaluate the harms of the treatments purely on the basis of the adverse effects reported in the clinical trials and systematic reviews included in their efficacy assessments. This is nonsensical for at least two reasons:
- The guideline authors themselves are aware that the trials very often fail to mention adverse effects.
- For any assessment of harm, one has to go far beyond the evidence of clinical trials, because trials tend to be too small to pick up rare adverse effects, and because they are always conducted under optimally controlled conditions where adverse effects are less likely to occur than in real life.
Together, these features of the assessment of harms explain why the guideline authors arrive at conclusions which are oddly misguided; I would even feel that they resemble a white-wash. Here are two of the most overt misjudgements:
- no harms associated with acupuncture,
- only trivial harm associated with spinal manipulations.
The best evidence we have today shows that acupuncture leads to mild adverse effects in about 10% of all cases and is also associated with very severe complications (e.g. pneumothorax, cardiac tamponade, infections, deaths) in an unknown number of patients. More details can be found for instance here, here, here and here.
And the best evidence available shows that spinal manipulation leads to moderately severe adverse effects in ~50% of all cases. In addition, we know of hundreds of cases of very severe complications resulting in stroke, permanent neurological deficits or deaths. More details can be found for instance here, here, here and here.
In the introduction, I stated that this small section of the guidelines is crucial.
The reason is simple: any responsible therapeutic decision has to be based not just on the efficacy of the treatment in question but on its risk/benefit balance. The evidence shows that the risks of some alternative therapies can be considerable, a fact that is almost totally neglected in the guidelines. Therefore, the recommendations of the new guidelines by the American College of Physicians entitled ‘Noninvasive Treatments for Acute, Subacute, and Chronic Low Back Pain: A Clinical Practice Guideline From the American College of Physicians’ are in several aspects not entirely correct and need to be reconsidered.
The website of the HOMEOPATHY HUB gives us intriguing access to the brain of a homeopath. It tells us that “protecting patient choice is at the heart of everything we do. Homeopathy, which is the second largest system of medicine in the world, is a form of treatment which plays a vital role in the lives of hundreds of thousands of people across the UK. There is, however, a movement to try and withdraw homeopathy from the public and make homeopathic medicines difficult to secure. Our intention is to be a central “hub” for accurate information on current campaigns to retain access to homeopathy and details on how you can get involved and make your voice heard. Without public and patient support we will not be successful.”
Here are a few of the above statements that I find doubtful:
- protecting patient choice – choice requires reliable information; as we will see, this is not provided here;
- second largest system of medicine in the world – really?
- plays a vital role – where is the evidence for that claim?
- movement to try and withdraw homeopathy from the public and make homeopathic medicines difficult to secure – nobody works towards this aim, some people are trying to stop wasting public funds on useless therapies, but that’s quite different, I find;
The HOMEOPATHY HUB recently alerted its readers to the fact that the Charity Commission (CC) is currently conducting a public consultation on whether organisations promoting the use of complementary and alternative medicines (CAM) should have charitable status (https://www.gov.uk/government/consultations/consultation-on-complementary-and-alternative-medicines) and urged its readers to defend homeopathy by responding to the CC offering a “few helpful points” to raise. These 7 points give, I think, a good insight into the thinking of homeopaths. I therefore copy them here and add a few of my own comments below:
- there are many types of evidence that should be considered when evaluating the effectiveness of a therapy. These include scientific studies, patient feedback and the clinical experience of doctors who have trained in a CAM discipline. Within Homeopathy there is considerable evidence which can be found (https://www.hri-research.org)
- many conventional therapies and drugs have inconclusive evidence or prove to be useful in only some cases, for example SSRIs (anti-depressants). Inconsistent evidence is often the result of the complexity of both the medical condition being treated and the therapy being used. It is not indicative of a therapy that doesn’t work
- removing all therapies or interventions that have inconsistent or inconclusive evidence would seriously limit the public and the medical profession’s ability to help treat and ease patients suffering.
- all over the world there are doctors, nurses, midwives, vets and other healthcare professional who integrate CAM therapies into their daily practice because they see effectiveness. They would not use these therapies if they did not see their patients benefitting from them. For example in the UK, within the NHS hospital setting, outcome studies demonstrate effectiveness of homeopathy. (http://www.britishhomeopathic.org/evidence/results-from-the-homeopathic-hospitals/)
- practitioners of many CAM therapies belong to registering bodies which expect their members to comply to the highest professional standards in regards to training and practice
- In the UK the producers and suppliers of CAM treatments (homeopathy, herbal medicine etc) are strictly regulated
- as well as providing valuable information to the growing number of people seeking to use CAM as part of their healthcare, CAM charities frequently fund treatment for those people, particularly the elderly and those on a low income, whose health has benefitted from these therapies but who cannot afford them. This meets the charity’s criterion of providing a public benefit.
- “Patient feedback and the clinical experience of doctors” may be important but is not what can be considered evidence of therapeutic effectiveness.
- Yes, in medicine evidence is often inconsistent; this is why we need to rely on proper assessments of the totality of the reliable data. If that fails to be positive (as is the case for homeopathy and several other forms of alternative medicine), we are well advised not to employ the treatment in question in routine healthcare.
- Removing all treatments for which the best evidence fails to show effectiveness – such as homeopathy – would greatly improve healthcare and reduces cost. It is one of the aims of EBM and an ethical imperative.
- Yes, some healthcare professionals do use useless therapies. They urgently need to be educated in the principles of EBM. Outcome studies have normally no control groups and therefore are no adequate tools for testing the effectiveness of medical interventions.
- The highest professional standards in regards to training and practice of nonsense will still result in nonsense.
- The proper regulation of nonsense can only generate proper nonsense.
- Yes, CAM charities frequently fund bogus treatments; hopefully (and with the help of readers of this blog), the CC will put an end to this soon.
I think these 7 points by the HOMEOPATHY HUB are a very poor defence of homeopathy. In fact, they are so bad that it is not worth analysing more closely than I did above. Yet, they do provide us with an insight into the homeopathic mind-set and show how illogical, misguided and wrong the arguments of homeopathy enthusiasts really are.
I do encourage you to give your response to the CC – it wound be hard to use better arguments than the homeopaths!!!