A team of 42 authors from various disciplines (mostly medicine and philosophy) have published an appeal to broaden the definition of evidence. They reached several overlapping conclusions with implications for policy and practice in research and clinical care, which they summarised as follows:
1. ‘Evidence’ is typically evidence of causation. Common terms used in EBM, such as ‘intervention’, ‘outcome’ or ‘increased risk’, are relevant to evidence-based decision making only insofar as they point to causal matters: causal interventions and their effects. Although there is growing reluctance to make causal claims in areas of uncertainty, the correct response to such uncertainty is not to avoid talking about causation but instead to improve our methods of understanding it.
2. Establishing causation often requires the use of multiple methods since no single method will be universally applicable or perfect for this purpose. This means that statistical approaches, in particular randomised controlled trials and systematic reviews, cannot uncover all causally relevant information, contrary to their widespread assumed status as the universal gold standards of EBM.
3. An understanding of causal mechanisms can help to determine whether an intervention works (ie, its efficacy shown in experiment or effectiveness in clinical practice). In addition, we should strive to understand how an intervention works (ie, its mechanism) and how it can be made to work (ie, the conditions under which it works best). Understanding mechanisms is essential for both of these. For instance, a medical intervention that works experimentally might not do so when combined with a negatively interacting substance.
4. Although animal experiments can shed light on causal mechanisms, other types of evidence can add to our understanding. This is because causal mechanisms are complex, involving multiple causal interactions of various factors. These factors play roles in the effectiveness of the treatment and in interactions between the treatment and the individual patient.
5. Given the multiplicity of methods (cf 2) and a wide interpretation of what counts as a mechanism (cf 3 and 4), causation should be understood in non-reductionist terms. That is, the scope of relevant causal interactions extends beyond the molecular, pharmacological and physiological levels of interaction. Any thorough causal account should also include higher-level factors, such as the behaviour of tissues, whole organs and individuals, including psychological and social factors.
6. ‘Causal evidence’ should be extended to include different types of evidence, including case studies and case reports, which can in some cases provide valuable information for understanding causation and causal mechanisms. This is particularly important when dealing with rare disorders, marginal groups or outliers.
7. Patient narratives and phenomenological approaches are useful tools for looking beyond evidence such as symptoms and outcomes, and to elucidate the core causes or sources for chronic and unexplained conditions.
8. Causation has a non-negligible temporal aspect. Whether of long or short duration, a causal interaction cannot be fully understood from a ‘snapshot’, but requires both backwardlooking perspectives (towards the origin) and forward-looking perspectives (towards the outcome).
These points are well worth considering, in my view. As we have often discussed on this blog, causation is the key. The authors see their paper as a philosophical analysis that ought to have a direct impact on the practice of medicine. If we are to understand what is meant by ‘evidence’, what is the ‘best available evidence’ and how to apply it in the context of medicine, they write, we need to tackle the problem of causation head on. In practice, this means understanding the context in which evidence is obtained, as well as how the evidence might be interpreted and applied when making practical clinical decisions. It also means being explicit about what kind of causal knowledge can be gained through various research methods. The possibility that mechanistic and other types of evidence can be used to add value or initiate a causal claim should not be ignored.
Their plea has much scope for being misunderstood by enthusiasts of so-called alternative medicine (SCAM). And I am keen to hear what you think about the 8 points raised here.
19 Responses to An appeal to broaden the definition of evidence
Leave a Reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
This is a noble perspective, and these points are necessary to evolve the field of medicine, but also very difficult to put into practice in my opinion. One reason is that more complexity brings more possible biases and confounders: maybe with the aid of machine learning and great computing power, letting us to study with multiple regressions a large amount of variables and interactions from large datasets simultaneously, we could take into account some of these points effectively. The risk of a simplistic interpretation of this new definition of EBM and its exploitation to justify pseudoscience is, in fact, around the corner.
very true!
EE asked
“And I am keen to hear what you think about the 8 points raised here.”
Any chance of getting the pharmaceutical companies on board with this thinking ? I don’t see where they had any part in this thinking.
On second thought, never mind, pharma is not interested in integrity.
if you say so, it must be true.
The intellectual dishonesty of some people is disconcerting! Firstly because they see only dishonesty where the most resounding successes of therapy have occurred (anesthetics, pain killers, vaccines, antibiotics, antiarrhythmics, vasodilators, bronchodilators, antineoplastics, antihistamines, corticosteroids, etc.). Forgetting all this because of some acts of dishonesty cannot be sensible. Second, because they use dishonesty, an exception, as an (obsolete) way of making it appear to justify their pseudo-scientific beliefs. It seems a joke to upset or malicious people to try to criticize the stupendous advances of modern medicine by appealing to such foolish arguments. Scam, falsehood, unscrupulous financial interest have always been part of the history of human societies, but mechanisms derived from human nature itself and its most spectacular creations have always acted in the opposite direction. The result is that (genuine) science is the only human activity to experience progress (Karl Popper: Conjectures and Refutations).
I find your comment very unclear: you are accusing people of dishonesty, but I have no clue who they are. can you be a little more clear and specific, please?
I was very clear and referred to people who act the way I indicated. You understood very well. What do you intend? I referred to people who accuse Medicina and doctors of subservience to an industry that they say is dishonest. This is a claim repeated hundreds of times on the Internet. In the face of all the advances of modern therapy, this is a baseless and therefore dishonest statement, and who utters it is intellectually dishonest.
sorry, but you make no sense to me whatsoever.
I believe MM is saying that the likes of RG are intellectually dishonest because they throw the baby out with the bath water. Because Big Pharma and EBM are imperfect they are completely worthless, is false.
Many good things have emerged from them. To deny that is to be dishonest. The honest science that they do is good because it is honest science. Dishonest science does not falsify honest science.
I think that is the gist.
@Munir Massud
I sympathize with your difficulties in communicating in written English. But the words you use and the way you put them together makes it very difficult to understand your point. You’re wrong to accuse Edzard Ernst of arrogance: he is saying what many readers must think, viz. “I think this person Munir Massud probably has a good point to make, but the way he writes it is almost impossible to comprehend.
Now Leigh Jackson has done us all a favour: he has summed up with clarity what I think you’re intending to say. And it matches a thought I have had very often, typically when I watch a TV programme about some recent, stunning advance in medical treatment. I ask myself “how can the people who promote archaic, nonsensical “medicine” watch something like this without feeling ashamed of themselves.
You have accused them of intellectual dishonesty and one can only say you are absolutely right. I wonder how they react to something like Horizon, 2019, The 250 Million Pound Cancer Cure shown recently on BBC2 in the UK (and, sadly, currently unavailable on catch-up TV). The programme describes the construction of an entire new and very complex new building at the Christie Hospital in Manchester for provision of proton beam therapy just to treat a small subset of cancers with high-precision proton beams. Can a homeopath, a chiropractor, an acupuncturist, or any of the other sadly deluded adherents to so-called alternative medicine (SCAM) really not feel ashamed that they promote mediæval-minded nonsense that should have gone into the trash can of history long ago?
Thank you, Munir Massud. You have pointed out that SCAM proponents and practitioners are being intellectually dishonest. I fully agree.
I think these arguments are full of possibilities for fraud. They open windows with them so that only observations (in Popper’s sense) will serve to verify relevant issues in medicine. It seems to me that these arguments provide a certain kind of deliberate return to eighteenth-century rationalism where theories and medical systems that killed more than the Napoleonic wars proliferated. It is not strange that taking into account social and “spiritual” aspects without proper elaborate scientific scrutiny corroborates this return to the dark past. MBE provides the elements for verifying all variables claimed in these claims and saying that they do not apply to possible variants is not true. No other scientific method has provided mankind with the ability to distinguish interpretive error in medicine. No causation (understanding of pathogenesis or pathophysiology) in THERAPY can replace methodologically sound clinical trials. These arguments seem to me to be a deliberate return in order to transform medicine into a schizophrenic hybrid of science and pseudoscience. Moreover, I still cannot understand how to include the notion of “spirit” in the pathogenesis of disease.
Guyat et al. (Guidelines for the Use of Medical Literature) warn of huge flaws that limit our observations to a sample size and deficiencies in human inferencing processes. “Predictions about intervention effects on important patient outcomes based on physiological experiments may be right, but occasionally they are disastrously wrong.” Which methods, so abundant, do the authors of the document refer to and which are strictly scientific and allow evidence of a relation of accusation and effect? What variable controls does MBE fail to collate? I think the terms “causation”, “multiple methods” are confusing there. It seems clear that in the face of rare medical cases, isolated descriptions may or may not provide help. But nowhere can generalizations be made, unless other cases are described, the disease best understood and the tests performed. What is not scientific in medicine is not ethical.
I think you probably mean well. but you ought to learn how to express your thoughts more clearly.
Munir Massud
The pharma industry is made up of corporations, where the vast bulk of medicinal drugs are developed. By their very nature, corporations are greedy, dishonest and corrupt. Corporations are pushed to make money to pay investor dividends, and increase stock share price value. They have no other reason to exist. The sooner you learn and accept this reality of life… the better off you will be.
I have been a stock market investor for 30 years, most of that time investing heavily in the Pharma sector, until more recently.
When I add my experience as a patient to my knowledge of the investing world, my opinion becomes even more qualified. I can assure you that most advances in modern medicine are incentivised by for profit gains. This is the rule, NOT the exception.
Once the financial carrot is offered and realized, the greed factor is the driver, and not easily replaced.
“By their very nature, corporations are greedy, dishonest and corrupt.”
this is a crazy statement!
Big bad Boiron not been doing so well on the greed front recently – see one year graph.
https://www.google.com/search?q=boiron+homeopathy+stock+market&oq=boiron+homeopathy+stock+market&aqs=chrome..69i57.24559j0j8&sourceid=chrome&ie=UTF-8
Nonsense based greed.
I accuse dishonest intellectuals who, even though they know that there is clear evidence of a reality, deny it for the sake of a subordinate interest. I was very clear, yes and you understood. For example, in the face of all advances in modern therapy, to deny them for accusing modern medicine and doctors as subservient to an industry they accuse of dishonest is to be intellectual dishonest. Using unproven knowledge to justify therapies that ache human suffering is intellectual dishonesty. The expression I use is intellectual dishonesty. I generalized an attitude that seems common and improper to me and did not refer to anyone personally.
I make the best effort to express myself well in English. In my youth I was almost compelled to arrest the Frenchman. Do you speak Portuguese or Spanish? I am Brazilian and would make me understand these languages better. You should have the sensitivity to realize that. But you prefer arrogance.
What I said about intellectual dishonesty, using the pharmaceutical industry example, was inspired by the opinion of a previous participant of mine, although I did not refer to him. It makes sense, yes. Even if I did not do it in this context, there is nothing there that is not an expression of reality. I stop going to your site.
sorry you see my difficulties in understanding your comments as arrogance.
1-4 are pretty much what the “science based medicine” say.
5 is effectively a truism – things have psychological and social causes as well as tissue and biochemical. The “non-reductionist” phrase is a warning sign because it can lead to accepting meaningless causes like miasmas, evil spirits, and nonsense “energies”.
6 and 7 are the quackery approach. Case studies and reports shed very limited light on causality and patient experience is a very limited guide to therapeutic causation because the individual cannot discriminate between the possible underlying causes of their experiences.
This is the second time in a week I’ve heard of “phenomenological approaches”. It seems to just mean subjective personal mental experiences. Those are largely evidence only of personal mental experiences, and very affected by distortions in what people say vs their actual experience.