Personally, I find our good friend Dana Ullman truly priceless. There are several reasons for that; one is that he is often so exemplarily wrong that it helps me to explain fundamental things more clearly. With a bit of luck, this might enable me to better inform people who might be thinking a bit like Dana. In this sense, our good friend Dana has significant educational value.
Recently, he made this comment:
According to present and former editors of THE LANCET and the NEW ENGLAND JOURNAL OF MEDICINE, “evidence based medicine” can no longer be trusted. There is obviously no irony in Ernst and his ilk “banking” on “evidence” that has no firm footing except their personal belief systems: https://medium.com/@drjasonfung/the-corruption-of-evidence-based-medicine-killing-for-profit-41f2812b8704
Ernst is a fundamentalist whose God is reductionistic science, a 20th century model that has little real meaning today…but this won’t stop the new attacks on me personally…
END OF COMMENT
Where to begin?
Let’s start with some definitions.
- Evidence is the body of facts that leads to a given conclusion. Because the outcomes of treatments such as homeopathy depend on a multitude of factors, the evidence for or against their effectiveness is best based not on experience but on clinical trials and systematic reviews of clinical trials (this is copied from my book).
- EBM is the integration of best research evidence with clinical expertise and patient values. It thus rests on three pillars: external evidence, ideally from systematic reviews, the clinician’s experience, and the patient’s preferences (and this is from another book).
Few people would argue that EBM, as it is applied currently, is without fault. Certainly I would not suggest that; I even used to give lectures about the limitations of EBM, and many experts (who are much wiser than I) have written about the many problems with EBM. It is important to note that such criticism demonstrates the strength of modern medicine and not its weakness, as Dana seems to think: it is a sign of a healthy debate aimed at generating progress. And it is noteworthy that internal criticism of this nature is largely absent in alternative medicine.
The criticism of EBM is often focussed on the unreliability of the what I called above the ‘best research evidence’. Let me therefore repeat what I wrote about it on this blog in 2012:
… The multifactorial nature of any clinical response requires controlling for all the factors that might determine the outcome other than the treatment per se. Ideally, we would need to create a situation or an experiment where two groups of patients are exposed to the full range of factors, and the only difference is that one group does receive the treatment, while the other one does not. And this is precisely the model of a controlled clinical trial.
Such studies are designed to minimise all possible sources of bias and confounding. By definition, they have a control group which means that we can, at the end of the treatment period, compare the effects of the treatment in question with those of another intervention, a placebo or no treatment at all.
Many different variations of the controlled trial exist so that the exact design can be adapted to the requirements of the particular treatment and the specific research question at hand. The over-riding principle is, however, always the same: we want to make sure that we can reliably determine whether or not the treatment was the cause of the clinical outcome.
Causality is the key in all of this; and here lies the crucial difference between clinical experience and scientific evidence. What clinician witness in their routine practice can have a myriad of causes; what scientists observe in a well-designed efficacy trial is, in all likelihood, caused by the treatment. The latter is evidence, while the former is not.
Don’t get me wrong; clinical trials are not perfect. They can have many flaws and have rightly been criticised for a myriad of inherent limitations. But it is important to realise that, despite all their short-comings, they are far superior than any other method for determining the efficacy of medical interventions.
There are lots of reasons why a trial can generate an incorrect, i.e. a false positive or a false negative result. We therefore should avoid relying on the findings of a single study. Independent replications are usually required before we can be reasonably sure.
Unfortunately, the findings of these replications do not always confirm the results of the previous study. Whenever we are faced with conflicting results, it is tempting to cherry-pick those studies which seem to confirm our prior belief – tempting but very wrong. In order to arrive at the most reliable conclusion about the efficacy of any treatment, we need to consider the totality of the reliable evidence. This goal is best achieved by conducting a systematic review.
In a systematic review, we assess the quality and quantity of the available evidence, try to synthesise the findings and arrive at an overall verdict about the efficacy of the treatment in question. Technically speaking, this process minimises selection and random biases. Systematic reviews and meta-analyses [these are systematic reviews that pool the data of individual studies] therefore constitute, according to a consensus of most experts, the best available evidence for or against the efficacy of any treatment.
END OF QUOTE
Other criticism is aimed at the way EBM is currently used (and abused). This criticism is often justified and necessary, and it is again the expression of our efforts to generate progress. EBM is practised by humans; and humans are far from perfect. They can be corrupt, misguided, dishonest, sloppy, negligent, stupid, etc., etc. Sadly, that means that the practice of EBM can have all of these qualities as well. All we can do is to keep on criticising malpractice, educate people, and hope that this might prevent the worst abuses in future.
Dana and many of his fellow SCAMers have a different strategy; they claim that EBM “can no longer be trusted” (interestingly they never tell us what system might be better; eminence-based medicine? experience-based medicine? random-based medicine? Dana-based medicine?).
The claim that EBM can no longer be trusted is clearly not true, counter-productive and unethical; and I suspect they know it.
Why then do they make it?
Because they feel that it entitles them to argue that homeopathy (or any other form of SCAM) cannot be held to EBM-standards. If EBM is unreliable, surely, nobody can ask the ‘Danas of this world’ to provide anything like sound data!!! And that, of course, would be just dandy for business, wouldn’t it?
So, let’s not be deterred or misled by these deliberately destructive people. Their motives are transparent and their arguments are nonsensical. EBM is not flawless, but with our continued efforts it will improve. Or, to repeat something that I have said many times before: EBM is the worst form of healthcare, except for all other known options.
19 Responses to Dana Ullman: “evidence-based medicine can no longer be trusted” !?!?
Leave a Reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
They DO tell us what system would be better. Clinical experience and what they see in their own clinics (Not a Doctor Paul Theirault wibbles to this effect often). The concepts of regression to the mean, placebo effect and observer bias are, of course, of no relevance.
We should bear in mind that many have been critical of EBM because of its flaws where prior plausibility is concerned, hence the rise of the concept of Science-Based Medicine. Which, of course, bangs more and larger metaphorical nails into coffin of homeopathy and the like using a much bigger and heavier logical hammer.
Lenny, Professor Ernst’s article above is correct in its entirety and science based medicine is it, except that one needs to know the science in order to evaluate it.
My issue is that I believe that Professor Ernst and the medical science experts investigating homeopathy do not understand the science of homeopathy. The very fact that 99% of studies even mention the ‘treatment’ of some ‘medical condition’ as part of the study RULES THEM OUT as being homeopathy right from the outset.
I look forward to reading on commenting on scientific evaluations of Homeopathy when medical science finally gets to the point of investigating it (and not something else under the name of ‘homeopathy’.
“I believe that Professor Ernst and the medical science experts investigating homeopathy do not understand the science of homeopathy”
BELIEF IS NOT EVIDENCE
do you have any evidence for this allegation that you repeat like a broken record?
Edzard, I know it is a difficult point for you to accept because you are not trained in Homeopathy.
If you do your Homeopathic Practitioner training then you might understand.
Sorry, but I can offer you more at this stage. I will comment on posts or comments that relate to homeopathy when they occur.
1) I AM trained as a homeopath
2) You cannot offer more because you have no more.
@Greg In the past Ernst admitted he never took homeopathy courses and was never licensed to independently prescribe as an MD/homeopath. He could only prescribe as a ‘homeopath’ under the umbrella of the hospital where he was first exposed to homeopathic prescribing. He also wrote that he was impressed by the results of homeopathic treatment there.
for Christ sake Sandra, pay attention when you read stuff.
what you are stating here is demonstrably wrong.
For those who have followed Sandra’s Don Quixote -esque campaign (here and on Twitter in my case) against the antagonists of homeopathy, it is in my opinion not impolite to conclude that she and truth/realism do not sing in harmony.
I realize that accuracy and compatibility with reality are not the prime focus of quacks and quack-lovers, but outright lying should be beneath even their flexible morals.
Edzard said:
She’s expert at being wrong.
Ignorance is no excuse for foolishness. And I’m not talking about Prof. Dr. Ernst, but about you.
Had you done your homework even in a most superficial manner, then you would have known that
– Prof. Dr. Ernst started out his career with a comprehensive training as a homeopath, and
– homeopathy is a system of belief, based on sympathetic magic and several well-known fallacies in human reasoning, not on science.
Homeopathy is obsolete rubbish, and those who still believe in it are fools who should know better, given the amount and quality of information at their disposal.
Well, Greg, first, homeopathy is not science. Second, there are several studies on individualized homeopathy, analyzed by Mathie 2014. Although Mathie found a small effect [note: small !] the analysis is of dubious quality (in my opinion) since Mathie includes positive studies that may have confounding factors and excludes negative studies that are of high quality. A prime example for an included study with confounding factors is the study of diarrhea in honduran children where almost twice as many children in the homeopathy arm have been pretreated with real medicine. Although the p-value for this factor is not significant, it is quite low and the factor likely has an influence on the end result.
Ah yes, that is an interesting link that Dana Ullman supplies.
Dr. Jason Fung, the author is the founder of the Intensive Dietary Management Program https://idmprogram.com/
The costs don’t look to bad! https://idmprogram.com/join/
The line form on the right.
I agree, but I am afraid that he is such a caricature that it may lead people to dismiss any comments critical of him as cheap shots, ridiculing a pitiful special-needs human being challenged by mental health issues. I don’t know if this actually the case, but I think the danger is certainly there.
sure, but that would be the danger with anyone holding such nonsensical views.
Indeed so.
The matter is further complicated by the fact that we have, as far as I know, no system in place to help these people.
In principle, education should be able prevent this condition to a large extent, but since that would obviously present a threat to the privileged situation religions of all stripes enjoy in most societies, it seems unlikely we will live to see such education to any meaningful extent.
good point, I think.
It is always fun to see naturopaths, homeopaths, etc. come up with the argument that 70% of studies in medicine are false. They do not seem to realize that this fraction applies to the studies they cite *also*, thinning the ice they stand on considerably.
Dana conflates the type I benefit which probably will be gained from a consultation with a homeopath (resulting from TLC and placebo responses) with the type II effects of injesting a homeopathic remedy.
The remedies have no effect – as EBM, SBM and even DBM have shown.
Move on from fake remedies, save on the expense, but carry on caring. Please.