Nobody really likes criticism, I suppose. Yet everyone with a functional brain agrees that criticism is a precondition to making progress. So most of us do listen to it, introspect and try to learn a lesson.
Not so in alternative medicine! The last post by Preston Long was a summary of constructive criticism of his own profession; it brought that message home to me much clearer than previous discussions on this blog (probably because it did not directly concern me) and, after some reflection, I realised that apologists of alternative medicine have developed five distinct strategies to avoid progress that otherwise might develop from criticism (alright, these strategies do exist in other fields too, but I think that many of the comments on this blog demonstrate that they are particularly evident in alternative medicine).
We could also call this method ‘The Prince of Wales Technique of Avoiding Progress’ because HRH is famous for making statements ‘ex cathedra’ without ever defending them or facing his critics or allowing others to directly challenge him. When he advocated the Gerson diet for cancer, for instance, Prof Baum challenged him in an open letter asking him to use his influence more wisely. Like with all other criticism directed to him, he decided to ignore it. This strategy is a safe bet for stalling progress and it has the added advantage that it does not require anything other than ignorance.
As it requires some basic understanding of the issues at hand, this method is a little more demanding. You need to look closely at the criticism and subsequently shoot holes in it. If you cannot find any, invent some. For instance, you might state that your critic misquoted the evidence. Very few people will bother to read up the original data, and you are likely to get away even with fairly obvious lies. To beef your response up a bit, pretend that there is plenty of good evidence demonstrating exactly the opposite of what your critic has said. If asked to provide actual references or sources for your claims, don’t listen. An extreme example of the bluff-method is to sue your critic for libel – but be careful, this can backfire in a major way!
A very popular method is to claim that the critic is not actually competent to criticise. The discussion of Long’s post demonstrated that technique in a classic fashion. His detractors argued that he was a failed chiropractor who had an axe to grind and thus had no right to criticise chiropractic (“Preston H Long you are a disgrace to the chiropractic profession…take off your chiropractic hat, you dont deserve to wear it. YOU sir are a shame and a folly!!”). Of course, you need to be a bit simple in order to agree with this type of logic, but lots of people seem to be just that!
Even more popular is the blame-game. It involves arguing that, ok not all is rosy on your side of the fence, but the other side is so, so much worse. Before they dare to challenge you, they should look at their own mess; and while it is not sorted, they must simply shut up. For instance, if the criticism is that chiropractors have put hundreds of their patients into wheelchairs with their neck-manipulations, you must point out that doctors with their nasty drugs are much, much worse (“Long discounts the multitudes that chiropractic has… saved from dangerous drugs and surgery. As far as risks of injury from seeing a chiropractor vs. medicine, all one needs to do is compare malpractice insurance rates to see that insurance carriers rate medicine as an exponentially more dangerous undertaking”). Few people will realise that this is a fallacy and that the risks of any therapy must be seen in relation to its potential benefits.
When criticised, you are understandably annoyed; most people will therefore forgive you calling your critic names which are not normally used in polite circles (“who is this idiot, who wouldnt know the first thing about chiropractic”). Ad hominem attacks are the last resort of apologists of alternative medicine which emerges with depressing regularity when they have run out of rational arguments; they are signs of victories of reason over unreason. In the case of those chiropractors who were unable to stomach Long’s critique, the insults were coming thick and fast. The reason for only very few being visible is quite simple: I often delete the worst excesses of such primitive reactions.
As I write these words, I am travelling back from a medical conference. The organisers had invited me to give a lecture which I concluded saying: “anyone in medicine not believing in evidence-based health care is in the wrong business”. This statement was meant to stimulate the discussion and provoke the audience who were perhaps just a little on the side of those who are not all that taken by science.
I may well have been right, because, in the coffee break, several doctors disputed my point; to paraphrase their arguments: “You don’t believe in the value of experience, you think that science is the way to know everything. But you are wrong! Philosophers and other people, who are a lot cleverer than you, tell us that science is not the way to real knowledge; and in some forms of medicine we have a wealth of experience which we cannot ignore. This is at least as important as scientific knowledge. Take TCM, for instance, thousands of years of tradition must mean something; in fact it tells us more than science will ever be able to. Qi-energy, for instance, is a concept based on experience, and science is useless at verifying it.”
I disagreed, of course. But I am afraid that I did not convince my colleagues. The appeal to tradition is amazingly powerful, so much so that even well-seasoned physicians fall for it. Yet it nevertheless is a fallacy, I am sure.
So what does experience tell us, how is it generated and why should it be unreliable?
On the level of the individual, experience emerges when a clinician makes similar observations several times in a row. This is so persuasive that few doctors are immune to the phenomenon. Let’s assume the experience is about acupuncture, more precisely about acupuncture for smoking cessation. The acupuncturist presumably has learnt during his training that his therapy works for that indication via stimulating the flow of Qi, and promptly tries it on several patients. Some of them come back for more and report that they find it easier to give up cigarettes after consulting him. This happens repeatedly, and our clinician forthwith is convinced – in fact, he knows – that acupuncture is effective for smoking cessation.
If we critically analyse this scenario, what does it tell us? It tells us very little of relevance, I am afraid. The scenario is entirely compatible with a whole host of explanations which have nothing to do with the effects of acupuncture per se:
- Those patients who did not manage to stop smoking might not have returned. Only seeing his successes without his failures, the acupuncturist would have got the wrong end of the stick.
- Human memory is selective such that the few patients who did come back and reported failure might easily get forgotten by the clinician. We all remember the good things and forget the disappointments, particularly if we are clinicians.
- The placebo-effect might have played a dirty trick on the experience of our acupuncturist.
- Some patients might have used nicotine patches that helped him to stop smoking without disclosing this fact to the acupuncturist who then, of course, attributed the benefit to his needling.
- The acupuncturist – being a very kind and empathetic clinician – might have involuntarily induced some of his patients to show kindness in return and thus tell porkies about their smoking habits which would have created a false positive impression about the effectiveness of his treatment.
- Being so empathetic, the acupuncturists would have provided lots of encouragement to stop smoking which, in some patients, might have been sufficient to kick the habit.
The long and short of all this is that our acupuncturist gradually got convinced by this interplay of factors that Qi exists and that acupuncture is an ineffective treatment. Hence forth he would bet his last shirt that he is right about this – after all, he has seen it with his own eyes, not just once but many times. And he will doubt anyone who shows him evidence that says otherwise. In fact, he is likely become very sceptical about scientific evidence in general – just like the doctors who talked to me after my lecture.
On a population level, such experience will be prevalent in not just one but most acupuncturists. Our clinician’s experience is certainly not unique; others will have made it too. In fact, as an acupuncturist, it is hard not to make it. Acupuncturists would have told everyone else about it, perhaps reported it on conferences or published it in articles or books. Experience of this nature is passed on from generation to generation, and soon someone will be able to demonstrate that acupuncture has been used ’effectively’ for smoking cessation since decades or centuries. The creation of a myth out of unreliable experience is thus complete.
Am I saying that experience of this nature is always and necessarily wrong or useless? No, I am not. It can be and often is correct. But, at the same time, it is frequently incorrect. It can serve as a valuable indicator but not more. Experience is not a tool for reliably informing us about the effectiveness of medical interventions. Experience based-medicine is an obsolete pseudo-medicine burdened with concepts that are counter-productive to optimal health care.
Philosophers and other people who are much cleverer than I am have been trying for some time to separate good from bad science and evidence from experience. Most recently, two philosophers, MASSIMO PIGLIUCCI and MAARTEN BOUDRY, commented specifically on this problem in relation to TCM. I leave you with some extensive quotes from what they wrote.
… pointing out that some traditional Chinese remedies (like drinking fresh turtle blood to alleviate cold symptoms) may in fact work, and therefore should not be dismissed as pseudoscience… risks confusing the possible effectiveness of folk remedies with the arbitrary theoretical-metaphysical baggage attached to it. There is no question that some folk remedies do work. The active ingredient of aspirin, for example, is derived from willow bark…
… claims about the existence of “Qi” energy, channeled through the human body by way of “meridians,” though, is a different matter. This sounds scientific, because it uses arcane jargon that gives the impression of articulating explanatory principles. But there is no way to test the existence of Qi and associated meridians, or to establish a viable research program based on those concepts, for the simple reason that talk of Qi and meridians only looks substantive, but it isn’t even in the ballpark of an empirically verifiable theory.
…the notion of Qi only mimics scientific notions such as enzyme actions on lipid compounds. This is a standard modus operandi of pseudoscience: it adopts the external trappings of science, but without the substance.
…The notion of Qi, again, is not really a theory in any meaningful sense of the word. It is just an evocative word to label a mysterious force of which we do not know and we are not told how to find out anything at all.
Still, one may reasonably object, what’s the harm in believing in Qi and related notions, if in fact the proposed remedies seem to help? Well, setting aside the obvious objections that the slaughtering of turtles might raise on ethical grounds, there are several issues to consider. To begin with, we can incorporate whatever serendipitous discoveries from folk medicine into modern scientific practice, as in the case of the willow bark turned aspirin. In this sense, there is no such thing as “alternative” medicine, there’s only stuff that works and stuff that doesn’t.
Second, if we are positing Qi and similar concepts, we are attempting to provide explanations for why some things work and others don’t. If these explanations are wrong, or unfounded as in the case of vacuous concepts like Qi, then we ought to correct or abandon them. Most importantly, pseudo-medical treatments often do not work, or are even positively harmful. If you take folk herbal “remedies,” for instance, while your body is fighting a serious infection, you may suffer severe, even fatal, consequences.
…Indulging in a bit of pseudoscience in some instances may be relatively innocuous, but the problem is that doing so lowers your defenses against more dangerous delusions that are based on similar confusions and fallacies. For instance, you may expose yourself and your loved ones to harm because your pseudoscientific proclivities lead you to accept notions that have been scientifically disproved, like the increasingly (and worryingly) popular idea that vaccines cause autism.
Philosophers nowadays recognize that there is no sharp line dividing sense from nonsense, and moreover that doctrines starting out in one camp may over time evolve into the other. For example, alchemy was a (somewhat) legitimate science in the times of Newton and Boyle, but it is now firmly pseudoscientific (movements in the opposite direction, from full-blown pseudoscience to genuine science, are notably rare)….
The borderlines between genuine science and pseudoscience may be fuzzy, but this should be even more of a call for careful distinctions, based on systematic facts and sound reasoning. To try a modicum of turtle blood here and a little aspirin there is not the hallmark of wisdom and even-mindedness. It is a dangerous gateway to superstition and irrationality
It was 20 years ago today that I started my job as ‘Professor of Complementary Medicine’ at the University of Exeter and became a full-time researcher of all matters related to alternative medicine. One issue that was discussed endlessly during these early days was the question whether alternative medicine can be investigated scientifically. There were many vociferous proponents of the view that it was too subtle, too individualised, too special for that and that it defied science in principle. Alternative medicine, they claimed, needed an alternative to science to be validated. I spent my time arguing the opposite, of course, and today there finally seems to be a consensus that alternative medicine can and should be submitted to scientific tests much like any other branch of health care.
Looking back at those debates, I think it is rather obvious why apologists of alternative medicine were so vehement about opposing scientific investigations: they suspected, perhaps even knew, that the results of such research would be mostly negative. Once the anti-scientists saw that they were fighting a lost battle, they changed their tune and adopted science – well sort of: they became pseudo-scientists (‘if you cannot beat them, join them’). Their aim was to prevent disaster, namely the documentation of alternative medicine’s uselessness by scientists. Meanwhile many of these ‘anti-scientists turned pseudo-scientists’ have made rather surprising careers out of their cunning role-change; professorships at respectable universities have mushroomed. Yes, pseudo-scientists have splendid prospects these days in the realm of alternative medicine.
The term ‘pseudo-scientist’ as I understand it describes a person who thinks he/she knows the truth about his/her subject well before he/she has done the actual research. A pseudo-scientist is keen to understand the rules of science in order to corrupt science; he/she aims at using the tools of science not to test his/her assumptions and hypotheses, but to prove that his/her preconceived ideas were correct.
So, how does one become a top pseudo-scientist? During the last 20 years, I have observed some of the careers with interest and think I know how it is done. Here are nine lessons which, if followed rigorously, will lead to success (… oh yes, in case I again have someone thick enough to complain about me misleading my readers: THIS POST IS SLIGHTLY TONGUE IN CHEEK).
- Throw yourself into qualitative research. For instance, focus groups are a safe bet. This type of pseudo-research is not really difficult to do: you assemble about 5 -10 people, let them express their opinions, record them, extract from the diversity of views what you recognise as your own opinion and call it a ‘common theme’, write the whole thing up, and – BINGO! – you have a publication. The beauty of this approach is manifold: 1) you can repeat this exercise ad nauseam until your publication list is of respectable length; there are plenty of alternative medicine journals who will hurry to publish your pseudo-research; 2) you can manipulate your findings at will, for instance, by selecting your sample (if you recruit people outside a health food shop, for instance, and direct your group wisely, you will find everything alternative medicine journals love to print); 3) you will never produce a paper that displeases the likes of Prince Charles (this is more important than you may think: even pseudo-science needs a sponsor [or would that be a pseudo-sponsor?]).
- Conduct surveys. These are very popular and highly respected/publishable projects in alternative medicine – and they are almost as quick and easy as focus groups. Do not get deterred by the fact that thousands of very similar investigations are already available. If, for instance, there already is one describing the alternative medicine usage by leg-amputated police-men in North Devon, and you nevertheless feel the urge of going into this area, you can safely follow your instinct: do a survey of leg-amputated police men in North Devon with a medical history of diabetes. There are no limits, and as long as you conclude that your participants used a lot of alternative medicine, were very satisfied with it, did not experience any adverse effects, thought it was value for money, and would recommend it to their neighbour, you have secured another publication in an alternative medicine journal.
- If, for some reason, this should not appeal to you, how about taking a sociological, anthropological or psychological approach? How about studying, for example, the differences in worldviews, the different belief systems, the different ways of knowing, the different concepts about illness, the different expectations, the unique spiritual dimensions, the amazing views on holism – all in different cultures, settings or countries? Invariably, you will, of course, conclude that one truth is at least as good as the next. This will make you popular with all the post-modernists who use alternative medicine as a playground for getting a few publications out. This approach will allow you to travel extensively and generally have a good time. Your papers might not win you a Nobel prize, but one cannot have everything.
- It could well be that, at one stage, your boss has a serious talk with you demanding that you start doing what (in his narrow mind) constitutes ‘real science’. He might be keen to get some brownie-points at the next RAE and could thus want you to actually test alternative treatments in terms of their safety and efficacy. Do not despair! Even then, there are plenty of possibilities to remain true to your pseudo-scientific principles. By now you are good at running surveys, and you could, for instance, take up your boss’ suggestion of studying the safety of your favourite alternative medicine with a survey of its users. You simply evaluate their experiences and opinions regarding adverse effects. But be careful, you are on somewhat thinner ice here; you don’t want to upset anyone by generating alarming findings. Make sure your sample is small enough for a false negative result, and that all participants are well-pleased with their alternative medicine. This might be merely a question of selecting your patients cleverly. The main thing is that your conclusion is positive. If you want to go the extra pseudo-scientific mile, mention in the discussion of your paper that your participants all felt that conventional drugs were very harmful.
- If your boss insists you tackle the daunting issue of therapeutic efficacy, there is no reason to give up pseudo-science either. You can always find patients who happened to have recovered spectacularly well from a life-threatening disease after receiving your favourite form of alternative medicine. Once you have identified such a person, you write up her experience in much detail and call it a ‘case report’. It requires a little skill to brush over the fact that the patient also had lots of conventional treatments, or that her diagnosis was assumed but never properly verified. As a pseudo-scientist, you will have to learn how to discretely make such irritating details vanish so that, in the final paper, they are no longer recognisable. Once you are familiar with this methodology, you can try to find a couple more such cases and publish them as a ‘best case series’ – I can guarantee that you will be all other pseudo-scientists’ hero!
- Your boss might point out, after you have published half a dozen such articles, that single cases are not really very conclusive. The antidote to this argument is simple: you do a large case series along the same lines. Here you can even show off your excellent statistical skills by calculating the statistical significance of the difference between the severity of the condition before the treatment and the one after it. As long as you show marked improvements, ignore all the many other factors involved in the outcome and conclude that these changes are undeniably the result of the treatment, you will be able to publish your paper without problems.
- As your boss seems to be obsessed with the RAE and all that, he might one day insist you conduct what he narrow-mindedly calls a ‘proper’ study; in other words, you might be forced to bite the bullet and learn how to plan and run an RCT. As your particular alternative therapy is not really effective, this could lead to serious embarrassment in form of a negative result, something that must be avoided at all cost. I therefore recommend you join for a few months a research group that has a proven track record in doing RCTs of utterly useless treatments without ever failing to conclude that it is highly effective. There are several of those units both in the UK and elsewhere, and their expertise is remarkable. They will teach you how to incorporate all the right design features into your study without there being the slightest risk of generating a negative result. A particularly popular solution is to conduct what they call a ‘pragmatic’ trial, I suggest you focus on this splendid innovation that never fails to produce anything but cheerfully positive findings.
- It is hardly possible that this strategy fails – but once every blue moon, all precautions turn out to be in vain, and even the most cunningly designed study of your bogus therapy might deliver a negative result. This is a challenge to any pseudo-scientist, but you can master it, provided you don’t lose your head. In such a rare case I recommend to run as many different statistical tests as you can find; chances are that one of them will nevertheless produce something vaguely positive. If even this method fails (and it hardly ever does), you can always home in on the fact that, in your efficacy study of your bogus treatment, not a single patient died. Who would be able to doubt that this is a positive outcome? Stress it clearly, select it as the main feature of your conclusions, and thus make the more disappointing findings disappear.
- Now that you are a fully-fledged pseudo-scientist who has produced one misleading or false positive result after the next, you may want a ‘proper’ confirmatory study of your pet-therapy. For this purpose run the same RCT over again, and again, and again. Eventually you want a meta-analysis of all RCTs ever published. As you are the only person who ever conducted studies on the bogus treatment in question, this should be quite easy: you pool the data of all your trials and, bob’s your uncle: a nice little summary of the totality of the data that shows beyond doubt that your therapy works. Now even your narrow-minded boss will be impressed.
These nine lessons can and should be modified to suit your particular situation, of course. Nothing here is written in stone. The one skill any pseudo-scientist must have is flexibility.
Every now and then, some smart arse is bound to attack you and claim that this is not rigorous science, that independent replications are required, that you are biased etc. etc. blah, blah, blah. Do not panic: either you ignore that person completely, or (in case there is a whole gang of nasty sceptics after you) you might just point out that:
- your work follows a new paradigm; the one of your critics is now obsolete,
- your detractors fail to understand the complexity of the subject and their comments merely reveal their ridiculous incompetence,
- your critics are less than impartial, in fact, most are bought by BIG PHARMA,
- you have a paper ‘in press’ that fully deals with all the criticism and explains how inappropriate it really is.
In closing, allow me a final word about publishing. There are hundreds of alternative medicine journals out there to chose from. They will love your papers because they are uncompromising promotional. These journals all have one thing in common: they are run by apologists of alternative medicine who abhor to read anything negative about alternative medicine. Consequently hardly a critical word about alternative medicine will ever appear in these journals. If you want to make double sure that your paper does not get criticised during the peer-review process (this would require a revision, and you don’t need extra work of that nature), you can suggest a friend for peer-reviewing it. In turn, you can offer to him/her that you do the same to him/her the next time he/she has an article to submit. This is how pseudo-scientists make sure that the body of pseudo-evidence for their pseudo-treatments is growing at a steady pace.
It is time, I think, to call an end to this series of articles on ‘drowning in a sea of misinformation’. Not that I have covered every contributor to and aspect of it. On the contrary, I could have carried on for another couple of weeks writing a post every day as I did during the last 15 days. But it was getting a bit boring – at least for me. So, for the last post, I have decided to briefly discuss politicians. In my view, they are crucially important in this context, as they create the general atmosphere and framework in which all the other mis-informers can thrive.
Peter Hain (Labour) is a campaigner for homeopathy and wants to see it widely used on the NHS. He was quoted as saying: “I first came to know about homeopathy through my son who as a baby suffered from eczema. He had it a couple of years but with conventional treatment the eczema was getting progressively worse and at the age of four he also developed asthma. We turned to homeopathy out of desperation and were stunned with the positive results. Since then I have used homeopathy for a wide variety of illnesses, but I rely on arnica as it’s excellent for treating the everyday bruises and shocks to the system we face. My view is that homeopathy and conventional medicines must remain side by side under the NHS to offer the best to patients”
Politicians who put anecdote before evidence do worry me quite a bit, I have to admit; by doing this, they provide us with strong evidence that they would be wise to keep their mouth shut when it comes to matters of science and medicine. But Hain is in good company: Jeremy Hunt (conservative), the current Secretary for Health, signed the following Early Day Motion in 2007: That this House welcomes the positive contribution made to the health of the nation by the NHS homeopathic hospitals; notes that some six million people use complementary treatments each year; believes that complementary medicine has the potential to offer clinically-effective and cost-effective solutions to common health problems faced by NHS patients, including chronic difficult to treat conditions such as musculoskeletal and other chronic pain, eczema, depression, anxiety and insomnia, allergy, chronic fatigue and irritable bowel syndrome; expresses concern that NHS cuts are threatening the future of these hospitals; and calls on the Government actively to support these valuable national assets.
The wording here is remarkable, I think: “…believes that complementary medicine has the potential to offer clinically-effective and cost-effective solutions to common health problems faced by NHS patients…” What is this supposed to mean? Health politics based on believe??? What it, in fact, implies is that there is merely belief but no evidence. Bravo! This looks like an own-goal to me.
And there are many, many more politicians who seem to prefer belief over evidence – not just in the UK but in virtually every country; our US friends would probably want me to mention Senator Tom Harkin who is responsible for spending billions of tax-payers’ dollars on researching implausible concepts with flawed studies. To make things worse, it is not just individual politicians who promote woo, as far as I can see, most political parties have a group of members promoting pseudo-science.
But why? Why do so many politicians misinform their voters about the values of unproven and disproven treatments? And I do not mean those members of parliament who nobody seems to be able to take seriously, like David Tredenick; I mean otherwise respectable politicians with real influence. Should they not be the first to insist on reliable evidence? Do they not have a mandate and an ethical/moral obligation to do so?
Call me cynical, but I have come to the conclusion that the answer is actually quite simple. Politicians need to be (re-)elected, and therefore they have to run with whatever subject is popular – and, like it or not, alternative medicine is popular. Politicians rarely take a reasonably long view on health care (in fact, very few understand the first thing about science or medicine); their perspective has exactly the same length as the current legislative period. They usually do not even attach much importance to alternative medicine; after all, it only amounts to a tiny fraction of the total health care budget.
Tony Blair (Labour) is as good an example as any other politician; in relation to homeopathy, he is quoted saying: I think that most people today have a rational view about science and my advice to the scientific community would be fight the battles you need to fight. I wouldn’t bother fighting a great battle over homeopathy – there are people who use it, people who don’t use it, it is not going to determine the future of the world, frankly. What will determine the future of the world however, is the scientific community explaining for example the science of genetics and how it develops, or the issue to do with climate change and so on.
Sounds reasonable? Almost, but not quite. Firstly, if people employ homeopathy to protect themselves from infectious diseases like malaria, typhus, TB, AIDS etc., or if people believe those charlatans who promote it as an effective cure for life-threatening conditions, we do have a serious public health issue at hand. Secondly, why should the vast majority of health care professionals bend over backwards to do their very best implementing the concepts of EBM, if homeopathy is being given a free ride to continue existing in a virtual universe of belief-based medicine? Thirdly, how on earth can scientists possibly explain “the science of genetics and how it develops, or the issue to do with climate change”, if they lack the skill, courage, power or honesty to adequately respond to harmful quackery masquerading as medicine?
It is not difficult to criticise politicians but what might be the way forward and out of this mess? Because of the central role they play in all this, I think that it would be important that those politicians who take up posts in science-based areas be adequately educated and trained in science. I know this may sound naïve, but I think it would be an essential step towards avoiding politicians regularly making fools of themselves, misinforming the public and misguiding important decisions which might affect all of us.
I have said it so often that I hesitate to state it again: an uncritical researcher is a contradiction in terms. This begs the question as to how critical the researchers of alternative medicine truly are. In my experience, most tend to be uncritical in the extreme. But how would one go about providing evidence for this view? In a previous blog-post, I have suggested a fairly simple method: to calculate an index of negative conclusions drawn in the articles published by a specific researcher. This is what I wrote:
If we calculated the percentage of a researcher’s papers arriving at positive conclusions and divided this by the percentage of his papers drawing negative conclusions, we might have a useful measure. A realistic example might be the case of a clinical researcher who has published a total of 100 original articles. If 50% had positive and 50% negative conclusions about the efficacy of the therapy tested, his trustworthiness index (TI) would be 1.
Depending on what area of clinical medicine this person is working in, 1 might be a figure that is just about acceptable in terms of the trustworthiness of the author. If the TI goes beyond 1, we might get concerned; if it reaches 4 or more, we should get worried.
An example would be a researcher who has published 100 papers of which 80 are positive and 20 arrive at negative conclusions. His TI would consequently amount to 4. Most of us equipped with a healthy scepticism would consider this figure highly suspect.
So how would alternative medicine researchers do, if we applied this method for assessing their trustworthiness? Very poorly, I fear – but that is speculation! Let’s see some data. Let’s look at one prominent alternative medicine researcher and see. As an example, I have chosen Professor George Lewith (because his name is unique which avoids confusion with researchers), did a quick Medline search to identify the abstracts of his articles on alternative medicine, and extracted the crucial sentence from the conclusions of the most recent ones:
- The study design of registered TCM trials has improved in estimating sample size, use of blinding and placebos
- Real treatment was significantly different from sham demonstrating a moderate specific effect of PKP
- These findings highlight the importance of helping patients develop coherent illness representations about their LBP before trying to engage them in treatment-decisions, uptake, or adherence
- Existing theories of how context influences health outcomes could be expanded to better reflect the psychological components identified here, such as hope, desire, optimism and open-mindedness
- …mainstream science has moved on from the intellectual sterility and ad hominem attacks that characterise the sceptics’ movement
- Trustworthy and appropriate information about practitioners (e.g. from professional regulatory bodies) could empower patients to make confident choices when seeking individual complementary practitioners to consult
- Comparative effectiveness research is an emerging field and its development and impact must be reflected in future research strategies within complementary and integrative medicine
- The I-CAM-Q has low face validity and low acceptability, and is likely to produce biased estimates of CAM use if applied in England, Romania, Italy, The Netherlands or Spain
- Our main finding was of beta power decreases in primary somatosensory cortex and SFG, which opens up a line of future investigation regarding whether this contributes toward an underlying mechanism of acupuncture.
- …physiotherapy was appraised more negatively in the National Health Service than the private sector but osteopathy was appraised similarly within both health-care sectors
This is a bit tedious, I agree, so I stop after just 10 articles. But even this short list does clearly indicate the absence of negative conclusions. In fact, I see none at all – arguably a few neutral ones, but nothing negative. All is positive in the realm of alternative medicine research then? In case you don’t agree with that assumption, you might prefer to postulate that this particular alternative medicine researcher somehow avoids negative conclusions. And if you believe that, you are not far from considering that we are being misinformed.
Alternative medicine is not really a field where one might reasonably expect that rigorous research generates nothing but positive results; even to expect 50 or 40% of such findings would be quite optimistic. It follows, I think, that if researchers only find positives, something must be amiss. I have recently demonstrated that the most active research homeopathic group (Professor Witt from the Charite in Berlin) has published nothing but positive findings; even if the results were not quite positive, they managed to formulate a positive conclusion. Does anyone doubt that this amounts to misinformation?
So, I do have produced at least some tentative evidence for my suspicion that some alternative medicine researchers misinform us. But how precisely do they do it? I can think of several methods for avoiding publishing a negative result or conclusion, and I fear that all of them are popular with alternative medicine researchers:
- design the study in such a way that it cannot possibly give a negative result
- manipulate the data
- be inventive when it comes to statistics
- home in on to the one positive aspect your generally negative data might show
- do not write up your study; like this nobody will ever see your negative results
And why do they do it? My impression is that they use science not for testing their interventions but for proving them. Critical thinking is a skill that alternative medicine researchers do not seem to cultivate. Often they manage to hide this fact quite cleverly and for good reasons: no respectable funding body would give money for such an abuse of science! Nevertheless, the end-result is plain to see: no negative conclusions are being published!
There are at least two further implications of the fact that alternative medicine researchers misinform the public. The first concerns the academic centres in which these researchers are organised. If a prestigious university accommodates a research unit of alternative medicine, it gives considerable credence to alternative medicine itself. If the research that comes out of the unit is promotional pseudo-science, the result, in my view, amounts to misleading the public about the value of alternative medicine.
The second implication relates to the journals in which researchers of alternative medicine prefer to publish their articles. Today, there are several hundred journals specialised in alternative medicine. We have shown over and over again that these journals publish next to nothing in terms of negative results. In my view, this too amounts to systematic misinformation.
My conclusion from all this is depressing: the type of research that currently dominates alternative medicine is, in fact, pseudo-research aimed not at rigorously falsifying hypotheses but at promoting bogus treatments. In other words alternative medicine researchers crucially contribute to the ‘sea of misinformation’ in this area.
It almost goes without saying that alternative practitioners contribute importantly to the ‘sea of misinformation’ about alternative medicine. Again, I could write books about this subject but have to refrain myself and therefore will merely put quick spotlights on several types of practitioners, mostly drawing from my own research on these subjects.
A survey of more than 9000 patients of U.K. non-medically trained acupuncturists showed that a considerable number had received advice from their therapists about prescribed medicines. Since these acupuncturists hold no medical qualifications, they are not qualified to issue such advice. It is therefore clear to me that the advice given is likely to be misleading. In 2000, we directly asked the U.K. acupuncturists’ advice about electro-acupuncture treatment for smoking cessation, a treatment which we previously had identified to be ineffective. The advice we received was frequently not based on current best evidence and some of it also raised serious safety concerns (Schmidt, K., & Ernst, E. Internet advice by acupuncturists—a risk factor for cardiovascular patients? Perfusion,2002, 15: 44-50. Article not Medline-listed).
Many chiropractors from the UK and other countries make unsustainable therapeutic claims on their websites. In 2002, at the height of the ‘‘MMR scare’’ in Britain, we conducted a study revealing that a sizable proportion of U.K. chiropractors advised mothers against having the measles-mumps-rubella (MMR) jab for their children. A survey of the U.K. chiropractors demonstrated that an alarming percentage of the U.K. chiropractors fail to provide advice about the risks of spinal manipulation before commencing treatment. As these risks are, in fact, considerable, this behaviour amounts to misinformation and is an obvious violation of medical ethics.
With osteopaths, it is a very similar story; the main difference is that there are far less investigations than for chiropractors. This may be due to the fact that, in the US, osteopaths are not alternative but conventional clinicians with much the same training and skills as proper doctors. But in Europe, they are strictly alternative and make as many bogus claims as chiropractors. Systematic investigations are rare, but I only need to remind us of my recent blog-post where I pointed out that:
Most osteopaths treat children for a wide range of conditions and claim that their interventions are helpful. They believe that children are prone to structural problems which can be corrected by their interventions. Here is an example from just one of the numerous promotional websites on this topic:
STRUCTURAL PROBLEMS, such as those affecting the proper mobility and function of the body’s framework, can lead to a range of problems. These may include:
- Postural – such as scoliosis
- Respiratory – such as asthma
- Manifestations of brain injury – such as cerebral palsy and spasticity
- Developmental – with delayed physical or intellectual progress, perhaps triggering learning behaviour difficulties
- Infections – such as ear and throat infections or urinary disturbances, which may be recurrent.
OSTEOPATHY can assist in the prevention of health problems, helping children to make a smooth transition into normal, healthy adult life.
Encouraging evidence exists for some specific herbs in the treatment of some specific conditions. Yet, virtually no good evidence exists to suggest that the prescriptions of individualized herbal mixtures by traditional herbalists across the globe generate more good than harm. Despite this lack of evidence, herbalists do not seem to offer this information voluntarily to his or her patients. When we directly asked the UK herbalists for advice on a clinical case, we found that it was ‘‘misleading at best and dangerous at worst’’ . In other words, herbalists misinform their patients and the public about the value of their treatments.
Many non-medically trained homeopaths advise their clients against the immunization of children. Instead, these practitioners often recommend using ‘‘homeopathic vaccinations’’ for which no good evidence exists. For instance, the vice-chair of the board of directors of ‘‘The Society of Homeopaths’’ had a site with the following statements: ‘‘Homeopathic alternatives to children’s immunisation are now available.’’ ‘‘Our clinic offers alternative immunisation programmes for the whole family.’’ Such statements amounts to misinformation which puts children’s health at risk.
Other alternative practitioners
I have chosen the above-listed professions almost at random and could have selected any other type as well. Arguably, all alternative practitioners who employ unproven treatments – and that must be the vast majority – misinform their patients to some extend. The only way to avoid this is to say: ‘look, I am going to give you a therapy for which there is no good evidence – I hope you don’t mind’. If they did that, they would be out of business in a flash. It follows, I think, that being in business is tantamount to misleading patients.
And there is, of course, another way of misinforming patients which is often forgotten yet very important: withholding essential information. In all of health care, informed consent is a ‘sine qua non’. Alternative practitioners very rarely obtain informed consent from their patients. The reason seems obvious (see above). I would argue that not informing people when they should be informed is a form of misinformation.
In this context, it is worth mentioning an investigation we did in 2009: We obtained the ethical codes of the following bodies: Association of Naturopathic Practitioners, Association of Traditional Chinese Medicine (UK), Ayurvedic Practitioners Association, British Acupuncture Council, Complementary and Natural Healthcare Council, European Herbal Practitioners Association, General Chiropractic Council, General Osteopathic Council, General Regulatory Council for Complementary Therapies, National Institute of Medical Herbalists, Register of Chinese Herbal Medicine, Society of Homeopaths, UK Healers, Unified Register of Herbal Practitioners. We then extracted the statements from these codes referring to evidence-based practice (EBP). The results showed that only the General Chiropractic Council, the General Osteopathic Council and the General Regulatory Council for Complementary Therapies oblige their members to adopt EBP.
It seems that misinformation is an alternative practitioner’s daily bread. Without it, alternative therapists would need to confine their practice to the few treatments/conditions for which the evidence is positive. If they ever followed this strategy, they would hardly be able to earn a living.
A total of 256 newspaper articles were evaluated. We identified 80 articles in the German papers and 176 in the British; thus, the British reported on medical topics more than twice as often as German broadsheets. Articles in German papers were on average considerably longer and took a positive attitude more often than British ones. We identified 4 articles on alternative medicine in the German and 26 in the UK newspapers. The tone of the UK articles was unanimously positive (100%) whereas most 75% of the German articles on alternative medicine were critical.
This analysis, we concluded, suggested that, compared with German newspapers, British newspapers report more frequently on medical matters and generally have a more critical attitude. The proportion of articles on alternative medicine seems to be considerably larger in the UK (15% v 5%), and, in contrast to articles on medical matters in general, reporting on alternative medicine in the UK was overwhelmingly positive.
That was 13 years ago, and things may well have changed since then. My impression is that more critical coverage of alternative medicine has finally and thankfully begun to emerge. But, even if this is true, we still cannot be sure how misleading it is. In 2006, we therefore conducted another investigation aimed at assessing UK newspapers’ coverage of alternative medicine, this time specifically for cancer.
We searched the “Lexis Nexis” database for 3-month periods in 2002, 2003 and 2004 to retrieve all relevant articles. A total of 310 articles were thus found: 117 came from national and 193 originated from local newspapers. The UK press showed an increasing interest towards alternative medicine for cancer (in 2002, 81 articles; in 2003, 82 articles and in 2004, 147 articles). The most frequently mentioned alternative therapies were diets and supplements (17.7%). Articles mainly focused on alternative medicine as possible cancer treatments (44.8%), and 53.4% of all treatments mentioned were not backed up by evidence. The tone of the articles was generally positive towards alternative medicine. Promotional articles increased over the years, especially for cancer centres and clinics.
Our conclusion: UK national newspapers frequently publish articles on alternative medicine for cancer. Much of this information seems to be uncritical with a potential for misleading patients.
There is no doubt that, in recent months, some journalists have produced excellent articles on alternative medicine. Let me use this occasion to congratulate them for this achievement. Yet, at the same time, it is indisputably true that misleading journalism continues to cause harm on a daily basis. Vulnerable people are thus led to make wrong therapeutic decisions; in some cases, this will only cost money, in other instances, it may well cost lives. Today, I would therefore formulate a much more constructive conclusion: it is time, I think that, when writing about health and medicine, journalists constantly remind themselves that they have a responsibility towards public health and stop giving bogus treatments a free ride.
And finally, to make this more fun, I invite you the reader of this post to report the most misleading newspaper article about alternative medicine you have come across (and, if possible provide a link to it). I will try to be a shining example and start with my choice: I’ve seen herbal remedy make tumours disappear, says respected cancer doctor (THE DAILY TELEGRAPH 20 SEPT 2004) The sub-headline that followed was: Since I have been putting people on Carctol I have seen miracles. Carctol, it turns out, once we do some research, is a herbal mixture heavily promoted as an alternative cancer cure, which is not supported by any reliable evidence at all. I do wonder how many lives have been shortened by this article!
Numerous charities in the UK, US and elsewhere abuse their charitable status to misinform the pubic about alternative medicine. As the BMJ today published an article on one this organisation, I have chosen HOMEOPATHS WITHOUT BORDERS as an example – from a disturbingly vast choice, I hasten to add.
Sounds good, doesn’t it? HOMEOPATHS WITHOUT BORDERS (HWB). Unless, of course, you happen to know that this organisation has nothing whatsoever to do with the much-admired ‘Medicine without Borders’. HWB and its numerous national branches promote the use of homeopathic remedies worldwide, particularly in disaster-stricken and extremely poor areas. On their website, they state: When disaster strikes or in times of crisis, homeopathy can provide effective treatment for acute anxiety and the after effects of shock and trauma. No, no, no! Homeopathy is a placebo-therapy; it is not effective for anxiety or anything else, crisis or no crisis.
To get an impression about their activities, here are HWB’s projects for 2013:
- We plan to train as many as 40 additional Homeopathe Communautaires in 2013.
- We’ll support the Homeopathe Communautaires as they grow with study groups and ongoing clinical support provided by our volunteer homeopaths.
- The 2012 graduates of the Fundamentals program will become teachers, moving HWB toward achieving our vision of Haitians teaching Haitians.
- We hope to bring continuing homeopathic medical care to the people of Haiti, reaching nearly three times as many people as we did in 2012.
- We plan to initiate a training program in 2013 for Haitian midwives and birth attendants for homeopathic therapeutics in pregnancy, delivery and postpartum care.
All of this looks to me as though HWB should be re-named into HOMEOPATHS WITHOUT SCRUPLES! Under the guise of some humanitarian activity, they seem to promote misinformation about a disproven treatment for some of the most vulnerable people in the world. I cannot imagine many things that are more despicable than that.
David Shaw, senior research fellow, Institute for Biomedical Ethics, University of Basel, Switzerland, has just published the above-mentioned BMJ-article on HWB. He discloses their activities as deeply unethical and concludes: Despite Homeopaths Without Borders’ claims to the contrary, “homeopathic humanitarian help” is a contradiction in terms. Although providing food, water, and solace to people in areas affected by wars and natural disasters certainly constitutes valuable humanitarian work, any homeopathic treatment deceives patients into thinking they are receiving real treatment when they are not. Furthermore, training local people as homeopaths in affected areas amounts to exploiting vulnerable people to increase the reach of homeopathy. Much as an opportunistic infection can take hold when a person’s immune system is weakened, so Homeopaths Without Borders strikes when a country is weakened by a disaster. However, infections are expunged once the immune system recovers but Homeopaths Without Borders’ methods ensure that homeopathy persists in these countries long after the initial catastrophe has passed. Homeopathy is neither helpful nor humanitarian, and to claim otherwise to the victims of disasters amounts to exploitation of those in need of genuine aid.
I strongly recommend reading the article in full.
And lastly: can I encourage readers to post their experience with and knowledge of other woo-infested charities, please?
Where can someone turn to who wants reliable information on alternative medicine? Many consumers and patients who ask themselves this question might already be somewhat weary of the Internet; everybody should by now know that websites can be dangerously misleading and usually commercially driven. What about books then? People still tend to trust books; they are written by experts, published by responsible enterprises, and sold through respectable outlets. Surely we can trust books, or can’t we?
The first thing that strikes you when you look into the subject is the fact that there are thousands of books on alternative medicine. You only need to visit a major book shop in your high street and admire the rows and rows of these volumes. Since many years, I have been evaluating such volumes, for instance, for our journal FACT where we regularly publish reviews of new material. Through this and other work, I have gained the impression, that most of these books are not worth the paper they are printed on and constitute a major contributor to the misinformation bombarding the consumer in this area. But that was just an impression, hard data would be better.
In 1998, we assessed for the first time the quality of books on alternative medicine ( Int J Risk Safety Med 1998, 11: 209-215. [For some reason, this article is not Medline-listed]). We chose a random sample of 6 such books all published in 1997, and we assessed their contents according to pre-defined criteria. The findings were sobering: the advice given in these volumes was frequently misleading, not based on good evidence and often inaccurate. If followed, it would have caused significant harm to patients.
In 2006, we conducted a similar investigation which we then reported in the first and second editions of our book THE DESKTOP GUIDE TO COMPLEMENTARY AND ALTERNATIVE MEDICINE. This time, we selected 7 best-sellers in alternative medicine and scrutinised them in much the same way. What we found was revealing. Almost every treatment seemed to be recommended for almost every condition. There was no agreement between the different books which therapy might be effective for which condition. Some treatments were even named as indications for a certain condition, while, in other books, they were listed as contra-indications for the same problem. A bewildering plethora of treatments was recommended for most conditions, for instance:
- addictions: 120 different treatments
- arthritis: 131 different treatments
- asthma: 119 different treatments
- cancer: 133 different treatments
- etc. etc.
This experience, which we published as a chapter in our book entitled AN EPITAPH TO OPINION-BASED MEDICINE, confirmed our suspicion that books on alternative medicine are a major contributor to the ‘sea of misinformation’ in this area – and, as they are read by many people, a significant risk factor to public health.
Why do publishers allow such rubbish to be printed? Why do so many authors spend their time misleading the public with their dangerous half-knowledge? Why do consumers buy such overtly uncritical nonsense? I do not know the answers, I must admit. But I know that books of this nature do a disservice to everyone involved, including the few respectable aspect of alternative medicine which might actually exist.
Has the situation changed since 2006? We cannot be sure; there is, to the best of my knowledge, no hard data; and nobody has repeated our investigations. But my impression from regularly reviewing new books for FACT and other journals is far from encouraging. I fear that our ‘epitaph to opinion-based medicine’ might have been a little premature.
The WHO is one of the most respected organisations in all of health care. It therefore might come as a surprise that it features in my series of institutions contributing to the ‘sea of misinformation’ in the area of alternative medicine. I have deliberately selected the WHO from many other organisations engaging in similarly misleading activities in order to show that even the most respectable bodies can have little enclaves of quackery hidden in their midst.
In 2006, the WHO invited Prince Charles to elaborate on his most bizarre concepts in relation to ‘integrated medicine’. He told the World Health Assembly in Geneva: “The proper mix of proven complementary, traditional and modern remedies, which emphasises the active participation of the patient, can help to create a powerful healing force in the world…Many of today’s complementary therapies are rooted in ancient traditions that intuitively understood the need to maintain balance and harmony with our minds, bodies and the natural world…Much of this knowledge, often based on oral traditions, is sadly being lost, yet orthodox medicine has so much to learn from it.” He urged countries across the globe to improve the health of their populations through a more integrated approach to health care. What he failed to mention is the fact that integrating disproven therapies into our clinical routine, as proponents of ‘integrated medicine’ demonstrably do, will not render medicine better or more compassionate but worse and less evidence-based. Or as my more brash US friends often point out: adding cow pie to apple pie is no improvement.
For many years during the early 2000s, the WHO had also been working on a document that would have promoted homeopathy worldwide. They had convened a panel of ‘experts’ including the Queen’s homeopath Peter Fisher. They advocated using this disproven treatment for potentially deadly diseases such as malaria, childhood diarrhoea, or TB as an alternative to conventional medicine. I had been invited to comment on a draft version of this document, but judging from the second draft, my criticism had been totally ignored. Fortunately, the publication of this disastrous advice could be stopped through a concerted initiative of concerned scientists who protested and pointed out that the implementation of this nonsense would kill millions.
In 2003, the WHO had already published a very similar report: a long consensus document on acupuncture. It includes the following list of diseases, symptoms or conditions for which acupuncture has been proved-through controlled trials-to be an effective treatment:
Adverse reactions to radiotherapy and/or chemotherapy
Allergic rhinitis (including hay fever)
Depression (including depressive neurosis and depression following stroke)
Dysentery, acute bacillary
Epigastralgia, acute (in peptic ulcer, acute and chronic gastritis, and gastrospasm)
Facial pain (including craniomandibular disorders)
Induction of labour
Low back pain
Malposition of fetus, correction of
Nausea and vomiting
Pain in dentistry (including dental pain and temporomandibular dysfunction)
Periarthritis of shoulder
If we compare these claims to the reliable evidence on the subject, we find that the vast majority of these indications is not supported by sound data (a fuller discussion on the WHO report and its history can be found in our book TRICK OR TREATMENT…). So, how can any organisation as well-respected globally as the WHO arrive at such outrageously misleading conclusions? The recipe for achieving this is relatively simple and time-tested by many similarly reputable institutions:
- One convenes a panel of ‘experts’ all or most of whom have a known preconceived opinion in the direction on has decided to go.
- One allows this panel to work out their own methodology for arriving at the conclusion they desire.
- One encourages cherry-picking of the data.
- One omits a meaningful evaluation of the quality of the reviewed studies.
- One prevents any type of critical assessment of the report such as peer-review by sceptics.
- If criticism does emerge nevertheless, one ignores it.
I should stress again that the WHO is, on the whole, a very good and useful organisation. This is precisely why I chose it for this post. As long as it is big enough, ANY such institution is likely to contain a little niche where woo and anti-science flourishes. There are far too many examples to mention, e.g. NICE, the NIH, UK and other governments. And this is the reason we must be watchful. It is all to human to assume that information is reliable simply because it originates from an authoritative source; the appeal to authority is appealing, of course, but it also is fallacious!