The volume of medical research, as listed on Medline, is huge and increases steadily each year. This phenomenon can easily be observed with simple Medline searches. If we use search terms related to conventional medicine, we find near linear increases in the number of articles (here I do not make a distinction between types of articles) published in each area over time, invariably with a peak in 2013, the last year for which Medline listing is currently complete. Three examples will suffice:
PHARMACOTHERAPY 117 414 articles in 2013
PHARMACOLOGY 210 228 articles in 2013
ADVERSE EFFECTS 86 067 articles in 2013
Some of the above subjects are obviously heavily industry-dependent and thus perhaps not typical of the volume of research in health care generally. Let’s therefore look up three fields where there is no such powerful industry to support research:
PSYCHOTHERAPY 7 208 articles in 2013
PHYSIOTHERAPY 7 713 articles in 2013
SURGERY 154 417 articles in 2013
Now, if we conduct similar searches for topics related to alternative medicine, the picture changes in at least three remarkable ways: 1) there is no linear increase of the volume per year; instead the curves look flat and shapeless (the only exception is ‘herbal medicine’ where the increase even looks exponential). 2) The absolute volume does not necessarily peak in 2013 (exceptions are ‘acupuncture’ and ‘herbal medicine’). 3) The number of articles in the year with the most articles (as listed below) is small or even tiny:
ACUPUNCTURE 1 491 articles in 2013
CHIROPRACTIC 283 articles in 2011
HERBAL MEDICINE 2 503 articles in 2013
HOMEOPATHY 233 articles in 2005
NATUROPATHY 69 articles in 2010
You may think: so what? But I find these figures intriguing. They demonstrate that the research output in alternative medicine is minimal compared to that in conventional medicine. Moreover, they imply that this output is not only not increasing steadily, as it is in conventional medicine, but in the case of chiropractic, homeopathy and naturopathy, it has recently been decreasing.
To put this into context, we need to know that:
- there is a plethora of journals dedicated to alternative medicine which are keen to publish all sorts of articles,
- the peer-review process of most of these journals seems farcically poor,
- as a result, the quality of the research into alternative medicine is often dismal, as regularly disclosed on this blog,
- enthusiasts of alternative medicine often see rigorous research into their subject as a dangerous threat: it might disprove their prior beliefs.
In their defence, proponents of alternative medicine would probably claim that the low volume of research is due to a severe and unfair lack of funding. However, I fail to see how this can be the sole or even the main explanation: areas of conventional medicine that do not have industry support seem to manage a much higher output than alternative medicine (and I should stress that I have chosen 5 sections within alternative medicine that are associated with the highest number of articles per year). Research in these areas is usually sponsored by charitable and government sources, and it needs to be stressed that these are open to any researcher who submits good science.
What follows, I think, is simple: in general, alternative medicine advocates have little interest in research and even less expertise to conduct it.
The chiropractic profession have been reminded time and times again that their claim to be able to effectively treat paediatric conditions is bogus. Many experts have asked them to produce some compelling evidence or stop this dangerous nonsense. Yet most of them seem to remain in denial, famously documented by the British Chiropractic Association suing Simon Singh for libel after he disclosed that they happily promote bogus treatments.
Some chiropractors now say that things have changed and that chiropractors are finally getting their act together. If that is true, progress must be painfully slow – so slow, in fact, that it is hard to see it at all. There are still far too many chiropractors who carry on just as before. There are hundreds, if not thousands of articles promoting chiropractic for childhood conditions; a very basic Google search for ‘chiropractic for children’ returns more than 7 million hits many of which advertise this sort of approach. Take this website, for instance; it makes its bogus claims entirely unabashed:
Even as an infant your child may have spinal nerve stress, known as subluxations. Although subluxations may not be painful, they can pose serious threats to your child’s development. If your baby was in a difficult position in the womb, or experienced a traumatic birth they may have developed subluxations. A common condition attributed to subluxations in children is known as Blocked Atlantal Nerve Syndrome. This condition may be the primary cause of ear and upper respiratory infections, and chronic tonsillitis.
Even regular childhood activities such as tumbles taken while learning to walk and run, bike riding, and participation in sports can also cause stress on your child’s body. Emotional stress and trauma may also be a cause of subluxations. Unless they are corrected they can affect future nerve function and the development of your child’s nervous system. They can also cause problems as your child grows and develops into adulthood.
With regular chiropractic care your child may be at less risk for common childhood disease such as colds and fevers. Some children show a marked improvement in asthma symptoms with regular chiropractic care and nutritional counselling. While chiropractors do no treat disease or sickness, they can identify and remove subluxations which interfere with your child’s natural ability to heal. By removing this stress from your child’s spinal system their immune system may function more efficiently and your child may have a better defense to disease. Their overall health may improve as their natural healing power is released. Children who receive regular chiropractic care may also be able to handle emotional and physical stress better and this care may contribute to their natural development.
Your child is never too young to start chiropractic care. Well-child care starts are early as the first month of life. Doctors use a very gentle pressure to treat children (no more pressure than picking up a tomato in the grocery store) and their treatments are very soothing to your child. After their first visit it is recommended that they receive treatments every three months up to age three, and then every six to 12 month after that. You may also want to visit your chiropractor after major milestones in your child’s life such as learning to sit up, crawl, and walk. They should also be seen if they experience any falls or trauma, and if you notice any balance issues they may be experiencing. These may include head tilting and limping.
Pediatric chiropractic care has many benefits. Children as young as infants may see an improvement in their development and overall health with regular care. Doctors of chiropractic take a proactive approach to health by striving to return and maintain your body’s natural balance. If you are looking for an alternative or supplement to traditional medical care, look into chiropractic care for your entire family.
Just a few rotten apples!… the apologists would probably say. But this is clearly not true. I find it even hard to locate the non-rotten apples in this decomposing and disgusting mess. More importantly, if it were true that things were now changing, one would expect that the progressive sections of the chiropractic profession protest regularly, sharply and effectively to shame the many charlatans amongst their midst. Crucially, one would expect the chiropractic professional organisations to oust their bogus members systematically and swiftly.
The sad truth, however, is that none of this is really happening – certainly not in the US or the UK. On the contrary, organisations like THE INTERNATIONAL PEDIATRIC ASSOCIATION, books entitled CHIROPRACTIC PEDIATRICS, and periodicals like the JOURNAL OF PEDIATRIC, MATERNAL AND FAMILY CHIROPRACTIC remain popular and respected within the chiropractic profession. A few lip-services here and there, yes. But truly effective action? No!
The tolerance of quackery, I would argue, must be one of the most important hallmarks of a quack profession.
Today, there are several dozens of journals publishing articles on alternative medicine. ‘The Journal of Alternative and Complementary Medicine’ is one of the best known, and it has one of the highest impact factors of them all. The current issue holds a few ‘gems’ which might be worthy of a comment or two. Here I have selected three articles reporting clinical studies, and I reproduce their abstracts (almost) in full (in italics) and add my comments (for clarity in bold). All the articles are available electronically, and I have provided the links for those who want to investigate beyond the abstracts.
STUDY No 1
The first ‘pilot study‘ was aimed to demonstrate the potential of auricular acupuncture (AAT) for insomnia in maintenance haemodialysis (MHD) patients and to prepare for a future randomized controlled trial.
Eligible patients were enrolled into this descriptive pilot study and received AAT designed to manage insomnia for 4 weeks. Questionnaires that used the Pittsburgh sleep quality index (PSQI) were completed at baseline, after a 4-week intervention, and 1 month after completion of treatment. Sleep quality and other clinical characteristics, including sleeping pills taken, were statistically compared between different time points.
A total of 22 patients were selected as eligible participants and completed the treatment and questionnaires. The mean global PSQI score was significantly decreased after AAT intervention (p<0.05). Participants reported improved sleep quality (p<0.01), shorter sleep latency (p<0.05), less sleep disturbance (p<0.01), and less daytime dysfunction (p=0.01). They also exhibited less dependency on sleep medications, indicated by the reduction in weekly estazolam consumption from 6.98±4.44 pills to 4.23±2.66 pills (p<0.01). However, these improvements were not preserved 1 month after treatment.
Conclusions: In this single-center pilot study, complementary AAT for MHD patients with severe insomnia was feasible and well tolerated and showed encouraging results for sleep quality.
In alternative medicine research, it has become far too common (almost generally accepted) to call a flimsy trial a ‘pilot study’. The authors give their game away by stating that, by conducting this trial, they want to ‘demonstrate the potential of AAT’. This is not a legitimate aim of research; science is for TESTING hypotheses, not for PROVING them!
The results of this trial show that patients experienced improvements after receiving AAT which, however, did not last. As there was no placebo control group, the most likely explanation for these outcomes would be that AAT generated a short-lasting placebo effect.
A sample size of 22 is, of course, far to small to allow any conclusions about the safety of the intervention. Despite these obvious facts, the authors seem convinced that AAT is both safe and effective.
STUDY No 2
The aim of the second study was to compare the therapeutic effect of Yamamoto new scalp acupuncture (YNSA), a recently developed microcupuncture system, with traditional acupuncture (TCA) for the prophylaxis and treatment of migraine headache.
In a randomized clinical trial, 80 patients with migraine headache were assigned to receive YNSA or TCA. A pain visual analogue scale (VAS) and migraine therapy assessment questionnaire (MTAQ) were completed before treatment, after 6 and 18 sections of treatment, and 1 month after completion of therapy.
All the recruited patients completed the study. Baseline characteristics were similar between the two groups. Frequency and severity of migraine attacks, nausea, the need for rescue treatment, and work absence rate decreased similarly in both groups. Recovery from headache and ability to continue daily activities 2 hours after medical treatment showed similar improvement in both groups (p>0.05).
Conclusions: Classic acupuncture and YNSA are similarly effective in the prophylaxis and treatment of migraine headache and may be considered as alternatives to pharmacotherapy.
This is what is technically called an ‘equivalence trial’, i.e. a study that compares an experimental treatment (YNSA) to one that is (assumed to be) effective. To demonstrate equivalence, such trials need to have large sample sizes, and this study is woefully underpowered. As it stands, the results show nothing meaningful at all; if anything, they suggest that both interventions were similarly useless.
STUDY No 3
The third study determined whether injection with hypertonic dextrose and morrhuate sodium (prolotherapy) using a pragmatic, clinically determined injection schedule for knee osteoarthritis (KOA) results in improved knee pain, function, and stiffness compared to baseline status.
The participants were 38 adults who had at least 3 months of symptomatic KOA and who were in the control groups of a prior prolotherapy randomized controlled trial (RCT) (Prior-Control), were ineligible for the RCT (Prior-Ineligible), or were eligible but declined the RCT (Prior-Declined).
The injection sessions at occurred at 1, 5, and 9 weeks with as-needed treatment at weeks 13 and 17. Extra-articular injections of 15% dextrose and 5% morrhuate sodium were done at peri-articular tendon and ligament insertions. A single intra-articular injection of 6 mL 25% dextrose was performed through an inferomedial approach.
The primary outcome measure was the validated Western Ontario McMaster University Osteoarthritis Index (WOMAC). The secondary outcome measure was the Knee Pain Scale and postprocedure opioid medication use and participant satisfaction.
The Prior-Declined group reported the most severe baseline WOMAC score (p=0.02). Compared to baseline status, participants in the Prior-Control group reported a score change of 12.4±3.5 points (19.5%, p=0.002). Prior-Decline and Prior-Ineligible groups improved by 19.4±7.0 (42.9%, p=0.05) and 17.8±3.9 (28.4%, p=0.008) points, respectively; 55.6% of Prior-Control, 75% of Prior-Decline, and 50% of Prior-Ineligible participants reported score improvement in excess of the 12-point minimal clinical important difference on the WOMAC measure. Postprocedure opioid medication resulted in rapid diminution of prolotherapy injection pain. Satisfaction was high and there were no adverse events.
Conclusions: Prolotherapy using dextrose and morrhuate sodium injections for participants with mild-to-severe KOA resulted in safe, significant, sustained improvement of WOMAC-based knee pain, function, and stiffness scores compared to baseline status.
This study had nothing that one might call a proper control group: all the three groups mentioned were treated with the experimental treatment. No attempt was made to control for even the most obvious biases: the observed effects could have been due to placebo or any other non-specific effects. The authors conclusions imply a causal relationship between the treatment and the outcome which is wrong. The notion that the experimental treatment is ‘safe’ is based on just 38 patients and therefore not reasonable.
All of this might seem rather trivial, and my comments could be viewed as a deliberate and vicious attempt to discredit one of the most respected journals of alternative medicine. Yet, considering that articles of this nature are more the rule than the exception in alternative medicine, I do think that this flagrant lack of scientific rigour is a relevant issue and has important implications.
As long as research in this area continues to be deeply flawed, as long as reviewers turn a blind eye to (or are not smart enough to detect) even the most obvious mistakes, as long as journal editors accept any rubbish in order to fill their pages, there is a great danger that we are being continuously being mislead about the supposed therapeutic value of alternative therapies.
Many who read this blog will, of course, have the capacity to think critically and might therefore not fall into the trap of accepting the conclusions of fatally flawed research. But many other people, including politicians, journalists and consumers, might not have the necessary appraisal skills and will thus not be able to tell that such studies can serve only one purpose: to popularise bogus treatments and thereby render health care less effective and more dangerous. Enthusiasts of alternative medicine are usually fully convinced that such studies amount to evidence and ram this pseudo-information down the throat of health care decision makers – the effects of such lobbying on public health can be disastrous.
And there is another downside to the publication of such dismal drivel: assuming (as I do) that not all of alternative medicine is completely useless, such embarrassingly poor research will inevitably have detrimental effects on the discipline of alternative medicine. After being exposed to a seemingly endless stream of pseudo-research, critics will eventually give up taking any of it seriously and might claim that none of it is worth the bother. In other words, those who conduct, accept or publish such nonsensical papers are not only endangering medical progress in general, they are also harming the very cause they try so desperately hard to advance.
I have often asked myself whether it is right/necessary to scientifically test things which are entirely implausible. Should we, for instance test the effectiveness of treatments which have a very low prior probability of generating a positive effect such as paranormal healing, homeopathy or Bach flower remedies? If you believe in the principles of evidence-based medicine you might focus on the clinical evidence and see biological plausibility as secondary. If you are a basic scientist, you are likely to do the reverse.
A recent article addressed this issue. The author points out that evaluating the absurd is absurd. Specifically, he noted that the empirical evaluation of a therapy would normally assume a plausible rationale regarding the mechanism of action. However, examination of the historical background and underlying principles for reflexology, iridology, acupuncture, auricular acupuncture, and some herbal medicines, reveals a rationale founded on the principle of analogical correspondences, which is a common basis for magical thinking and pseudoscientific beliefs such as astrology and chiromancy. Where this is the case, it is suggested that subjecting these therapies to empirical evaluation may be tantamount to evaluating the absurd.
This makes a lot of sense – but is it really entirely true? Are there no legitimate reasons at all for testing alternative treatments that lack biological plausibility? Ten or twenty years ago, I would have disagreed with the notion that plausibility is an essential prerequisite for scientific testing; today, I have changed my mind a little, but not as much as to agree completely with the assumption. In other words, I still see more than one good reason why evaluating the absurd might be reasonable or even advisable.
- Using plausibility as the only arbiter of scientific ‘evaluability’, assumes that we understand everything about plausibility there is to know. Yet it might just be possible that we mis-categorise something as implausible simply because we are not yet fully aware of all the facts.
- Declaring something as plausible and another thing as implausible are not hard and fast verdicts but judgements which, at least to some degree, are subjective. Sceptics find the axioms of homeopathy utterly implausible, for instance – but ask a homeopath, and you will hear all sorts of explanations which, at least to them, sound plausible.
- If an implausible alternative treatment is in wide-spread use, we arguably have a responsibility to test it scientifically in order to demonstrate the truth about it (to those proponents of that therapy who are willing to accept that rigorous science can find the truth). If we fail to do this, it will be the enthusiasts of that therapy who conduct less than rigorous science and produce false positive results. In turn, this will give the impression that the treatment is effective and mislead consumers, politicians, journalists etc. Seen from this perspective, it might even be unethical to not do the science.
So, I am in two minds about this (which might be a reflection of the fact that, during different periods of my life, I have been a clinician, a basic scientist and a clinical researcher). I realise that plausibility and prior probability are important – much more so than I appreciated years ago. But I think they should not be the only criteria. The clinical evidence should not be pushed aside completely.
I’d be interested to learn your views on this tricky issue.
Homeopathy is a deeply puzzling subject for many observers. Perhaps it gets a little easier to understand, if we consider the three main perspectives on homeopathy. For the purpose of this post, I take the liberty of exaggerating, almost caricaturizing, these perspectives in order to contrast them as clearly as possible.
THE SCEPTICS’ PERSPECTIVE
Sceptics take a brief look at the two main assumptions which underpin homeopathy (like cures like and potentiation/dilution/water memory) and henceforward are convinced that homeopathic remedies are pure placebos. Homeopathy flies in the face of science; if homeopathy is right, several laws of nature must be wrong, they love to point out. As this is most unlikely, they reject homeopathy outright, usually even without looking in any detail at what homeopaths consider to be evidence in support of their trade. If sceptics are forced to consider a positive study of homeopathy, they know before they have seen it that its results are wrong – due to an error caused by chance, faulty study design or fabrication. The sceptics’ conclusion on homeopathy: it is a placebo-therapy, no doubt about it; and further investment into research is a waste of scarce resources which must be stopped.
THE BELIEVERS’ PERSPECTIVE
The believers in homeopathy know from experience that homeopathy works. They therefore feel that they have no choice but to reject almost every word the sceptics might tell them. They cling on to the gospel of Hahnemann and elaborate on the modern but vague theories that might support the theoretical assumptions of homeopathy. They point to positive clinical trials and outcome studies, to 200 years of experience, and to the endorsement of homeopathy by VIPs. When confronted with the weaknesses of their arguments, they find even weaker ones, such as ‘much of conventional medicine is also not based on good evidence, and the mechanism of action of many mainstream drugs is also not fully understood’. Alternatively, they employ the phoniest argument of them all: ‘even if it works via a placebo effect, it still helps patients and therefore is a useful therapy’. When even this fails, they tend to resort to ad hominem attacks against their opponents. The believers’ conclusion on homeopathy: it is unquestionably a valuable type of therapy regardless of what anyone else might say; research is merely needed to confirm their belief.
THE PERSPECTIVE OF THE ADVOCATES OF EVIDENCE-BASED MEDICINE (EBM)
The perspective of EBM-advocates is pragmatic; they simply say: “show me the evidence!” If the majority of the most reliable clinical trials of homeopathic remedies (or anything else) suggests an effect beyond placebo, they conclude that they are effective. If that is not the case, they doubt the effectiveness. If the evidence is highly contradictory or incomplete, they are likely to advocate more rigorous research. Advocates of EBM are usually not all that concerned by the lack of plausibility of the interventions they evaluate. If it works, it works, they think – and if a plausible mechanism is currently not available, it might be found in due course. The advocates of EBM have no preconceived ideas about homeopathy. Their conclusion on homeopathy goes exactly where the available best evidence leads them.
The arguments and counter-arguments originating from the various perspectives would surely continue for another 200 years – unless, of course, two of the three perspectives merge and arrive at the same or very similar conclusions. And this is precisely what has now happened. As I have pointed out in a recent post, the most thorough and independent evaluation of homeopathy according to rigorous EBM-standards has concluded that “the evidence from research in humans does not show that homeopathy is effective for treating the range of health conditions considered.”
In other words, two of the three principal perspectives have now drawn conclusions which are virtually identical: there is a consensus between the EBM-advocates and the sceptics. This isolates the believers and renders their position no longer tenable. If we furthermore consider that the believers are heavily burdened with obvious conflicts of interest, while the other two groups are by definition much more independent and objective, it appears more and more as though homeopathy is fast degenerating into a cult characterised by the unquestioning commitment and unconditional submission of its members who are too heavily brain-washed to realize that their fervour has isolated them from the rational sections of society. And a cult is hardly what we need in heath care, I should think.
It seems to me therefore that these intriguing developments might finally end the error that homeopathy represented for nearly 200 years.
Progress at last?
Advocates of alternative medicine are incredibly fond of supporting their claims with anecdotes, or ‘case-reports’ as they are officially called. There is no question, case-reports can be informative and important, but we need to be aware of their limitations.
A recent case-report from the US might illustrated this nicely. It described a 65-year-old male patient who had had MS for 20 years when he decided to get treated with Chinese scalp acupuncture. The motor area, sensory area, foot motor and sensory area, balance area, hearing and dizziness area, and tremor area were stimulated once a week for 10 weeks, then once a month for 6 further sessions.
After the 16 treatments, the patient showed remarkable improvements. He was able to stand and walk without any problems. The numbness and tingling in his limbs did not bother him anymore. He had more energy and had not experienced incontinence of urine or dizziness after the first treatment. He was able to return to work full time. Now the patient has been in remission for 26 months.
The authors of this case-report conclude that Chinese scalp acupuncture can be a very effective treatment for patients with MS. Chinese scalp acupuncture holds the potential to expand treatment options for MS in both conventional and complementary or integrative therapies. It can not only relieve symptoms, increase the patient’s quality of life, and slow and reverse the progression of physical disability but also reduce the number of relapses and help patients.
There is absolutely nothing wrong with case-reports; on the contrary, they can provide extremely valuable pointers for further research. If they relate to adverse effects, they can give us crucial information about the risks associated with treatments. Nobody would ever argue that case-reports are useless, and that is why most medical journals regularly publish such papers. But they are valuable only, if one is aware of their limitations. Medicine finally started to make swift progress, ~150 years ago, when we gave up attributing undue importance to anecdotes, began to doubt established wisdom and started testing it scientifically.
Conclusions such as the ones drawn above are not just odd, they are misleading to the point of being dangerous. A reasonable conclusion might have been that this case of a MS-patient is interesting and should be followed-up through further observations. If these then seem to confirm the positive outcome, one might consider conducting a clinical trial. If this study proves to yield encouraging findings, one might eventually draw the conclusions which the present authors drew from their single case.
To jump at conclusions in the way the authors did, is neither justified nor responsible. It is unjustified because case-reports never lend themselves to such generalisations. And it is irresponsible because desperate patients, who often fail to understand the limitations of case-reports and tend to believe things that have been published in medical journals, might act on these words. This, in turn, would raise false hopes or might even lead to patients forfeiting those treatments that are evidence-based.
It is high time, I think, that proponents of alternative medicine give up their love-affair with anecdotes and join the rest of the health care professions in the 21st century.
Nobody really likes criticism, I suppose. Yet everyone with a functional brain agrees that criticism is a precondition to making progress. So most of us do listen to it, introspect and try to learn a lesson.
Not so in alternative medicine! The last post by Preston Long was a summary of constructive criticism of his own profession; it brought that message home to me much clearer than previous discussions on this blog (probably because it did not directly concern me) and, after some reflection, I realised that apologists of alternative medicine have developed five distinct strategies to avoid progress that otherwise might develop from criticism (alright, these strategies do exist in other fields too, but I think that many of the comments on this blog demonstrate that they are particularly evident in alternative medicine).
We could also call this method ‘The Prince of Wales Technique of Avoiding Progress’ because HRH is famous for making statements ‘ex cathedra’ without ever defending them or facing his critics or allowing others to directly challenge him. When he advocated the Gerson diet for cancer, for instance, Prof Baum challenged him in an open letter asking him to use his influence more wisely. Like with all other criticism directed to him, he decided to ignore it. This strategy is a safe bet for stalling progress and it has the added advantage that it does not require anything other than ignorance.
As it requires some basic understanding of the issues at hand, this method is a little more demanding. You need to look closely at the criticism and subsequently shoot holes in it. If you cannot find any, invent some. For instance, you might state that your critic misquoted the evidence. Very few people will bother to read up the original data, and you are likely to get away even with fairly obvious lies. To beef your response up a bit, pretend that there is plenty of good evidence demonstrating exactly the opposite of what your critic has said. If asked to provide actual references or sources for your claims, don’t listen. An extreme example of the bluff-method is to sue your critic for libel – but be careful, this can backfire in a major way!
A very popular method is to claim that the critic is not actually competent to criticise. The discussion of Long’s post demonstrated that technique in a classic fashion. His detractors argued that he was a failed chiropractor who had an axe to grind and thus had no right to criticise chiropractic (“Preston H Long you are a disgrace to the chiropractic profession…take off your chiropractic hat, you dont deserve to wear it. YOU sir are a shame and a folly!!”). Of course, you need to be a bit simple in order to agree with this type of logic, but lots of people seem to be just that!
Even more popular is the blame-game. It involves arguing that, ok not all is rosy on your side of the fence, but the other side is so, so much worse. Before they dare to challenge you, they should look at their own mess; and while it is not sorted, they must simply shut up. For instance, if the criticism is that chiropractors have put hundreds of their patients into wheelchairs with their neck-manipulations, you must point out that doctors with their nasty drugs are much, much worse (“Long discounts the multitudes that chiropractic has… saved from dangerous drugs and surgery. As far as risks of injury from seeing a chiropractor vs. medicine, all one needs to do is compare malpractice insurance rates to see that insurance carriers rate medicine as an exponentially more dangerous undertaking”). Few people will realise that this is a fallacy and that the risks of any therapy must be seen in relation to its potential benefits.
When criticised, you are understandably annoyed; most people will therefore forgive you calling your critic names which are not normally used in polite circles (“who is this idiot, who wouldnt know the first thing about chiropractic”). Ad hominem attacks are the last resort of apologists of alternative medicine which emerges with depressing regularity when they have run out of rational arguments; they are signs of victories of reason over unreason. In the case of those chiropractors who were unable to stomach Long’s critique, the insults were coming thick and fast. The reason for only very few being visible is quite simple: I often delete the worst excesses of such primitive reactions.
It is time, I think, to call an end to this series of articles on ‘drowning in a sea of misinformation’. Not that I have covered every contributor to and aspect of it. On the contrary, I could have carried on for another couple of weeks writing a post every day as I did during the last 15 days. But it was getting a bit boring – at least for me. So, for the last post, I have decided to briefly discuss politicians. In my view, they are crucially important in this context, as they create the general atmosphere and framework in which all the other mis-informers can thrive.
Peter Hain (Labour) is a campaigner for homeopathy and wants to see it widely used on the NHS. He was quoted as saying: “I first came to know about homeopathy through my son who as a baby suffered from eczema. He had it a couple of years but with conventional treatment the eczema was getting progressively worse and at the age of four he also developed asthma. We turned to homeopathy out of desperation and were stunned with the positive results. Since then I have used homeopathy for a wide variety of illnesses, but I rely on arnica as it’s excellent for treating the everyday bruises and shocks to the system we face. My view is that homeopathy and conventional medicines must remain side by side under the NHS to offer the best to patients”
Politicians who put anecdote before evidence do worry me quite a bit, I have to admit; by doing this, they provide us with strong evidence that they would be wise to keep their mouth shut when it comes to matters of science and medicine. But Hain is in good company: Jeremy Hunt (conservative), the current Secretary for Health, signed the following Early Day Motion in 2007: That this House welcomes the positive contribution made to the health of the nation by the NHS homeopathic hospitals; notes that some six million people use complementary treatments each year; believes that complementary medicine has the potential to offer clinically-effective and cost-effective solutions to common health problems faced by NHS patients, including chronic difficult to treat conditions such as musculoskeletal and other chronic pain, eczema, depression, anxiety and insomnia, allergy, chronic fatigue and irritable bowel syndrome; expresses concern that NHS cuts are threatening the future of these hospitals; and calls on the Government actively to support these valuable national assets.
The wording here is remarkable, I think: “…believes that complementary medicine has the potential to offer clinically-effective and cost-effective solutions to common health problems faced by NHS patients…” What is this supposed to mean? Health politics based on believe??? What it, in fact, implies is that there is merely belief but no evidence. Bravo! This looks like an own-goal to me.
And there are many, many more politicians who seem to prefer belief over evidence – not just in the UK but in virtually every country; our US friends would probably want me to mention Senator Tom Harkin who is responsible for spending billions of tax-payers’ dollars on researching implausible concepts with flawed studies. To make things worse, it is not just individual politicians who promote woo, as far as I can see, most political parties have a group of members promoting pseudo-science.
But why? Why do so many politicians misinform their voters about the values of unproven and disproven treatments? And I do not mean those members of parliament who nobody seems to be able to take seriously, like David Tredenick; I mean otherwise respectable politicians with real influence. Should they not be the first to insist on reliable evidence? Do they not have a mandate and an ethical/moral obligation to do so?
Call me cynical, but I have come to the conclusion that the answer is actually quite simple. Politicians need to be (re-)elected, and therefore they have to run with whatever subject is popular – and, like it or not, alternative medicine is popular. Politicians rarely take a reasonably long view on health care (in fact, very few understand the first thing about science or medicine); their perspective has exactly the same length as the current legislative period. They usually do not even attach much importance to alternative medicine; after all, it only amounts to a tiny fraction of the total health care budget.
Tony Blair (Labour) is as good an example as any other politician; in relation to homeopathy, he is quoted saying: I think that most people today have a rational view about science and my advice to the scientific community would be fight the battles you need to fight. I wouldn’t bother fighting a great battle over homeopathy – there are people who use it, people who don’t use it, it is not going to determine the future of the world, frankly. What will determine the future of the world however, is the scientific community explaining for example the science of genetics and how it develops, or the issue to do with climate change and so on.
Sounds reasonable? Almost, but not quite. Firstly, if people employ homeopathy to protect themselves from infectious diseases like malaria, typhus, TB, AIDS etc., or if people believe those charlatans who promote it as an effective cure for life-threatening conditions, we do have a serious public health issue at hand. Secondly, why should the vast majority of health care professionals bend over backwards to do their very best implementing the concepts of EBM, if homeopathy is being given a free ride to continue existing in a virtual universe of belief-based medicine? Thirdly, how on earth can scientists possibly explain “the science of genetics and how it develops, or the issue to do with climate change”, if they lack the skill, courage, power or honesty to adequately respond to harmful quackery masquerading as medicine?
It is not difficult to criticise politicians but what might be the way forward and out of this mess? Because of the central role they play in all this, I think that it would be important that those politicians who take up posts in science-based areas be adequately educated and trained in science. I know this may sound naïve, but I think it would be an essential step towards avoiding politicians regularly making fools of themselves, misinforming the public and misguiding important decisions which might affect all of us.
Can one design a clinical study in such a way that it looks highly scientific but, at the same time, has zero chances of generating a finding that the investigators do not want? In other words, can one create false positive findings at will and get away with it? I think it is possible; what is more, I believe that, in alternative medicine, this sort of thing happens all the time. Let me show you how it is done; four main points usually suffice:
- The first rule is that it ought to be an RCT, if not, critics will say the result was due to selection bias. Only RCTs have the reputation of being ‘top notch’.
- Once we are clear about this design feature, we need to define the patient population. Here the trick is to select individuals with an illness that cannot be quantified objectively. Depression, stress, fatigue…the choice is vast. The aim must be to employ an outcome measure that is well-accepted, validated etc. but which nevertheless is entirely subjective.
- Now we need to consider the treatment to be “tested” in our study. Obviously we take the one we are fond of and want to “prove”. It helps tremendously, if this intervention has an exotic name and involves some exotic activity; this raises our patients’ expectations which will affect the result. And it is important that the treatment is a pleasant experience; patients must like it. Finally it should involve not just one but several sessions in which the patient can be persuaded that our treatment is the best thing since sliced bread – even if, in fact, it is entirely bogus.
- We also need to make sure that, for our particular therapy, no universally accepted placebo exists which would allow patient-blinding. That would be fairly disastrous. And we certainly do not want to be innovative and create such a placebo either; we just pretend that controlling for placebo-effects is impossible or undesirable. By far the best solution would be to give the control group no treatment at all. Like this, they are bound to be disappointed for missing out a pleasant experience which, in turn, will contribute to unfavourable outcomes in the control group. This little trick will, of course, make the results in the experimental group look even better.
That’s about it! No matter how ineffective our treatment is, there is no conceivable way our study can generate a negative result; we are in the pink!
Now we only need to run the trial and publish the positive results. It might be advisable to recruit several co-authors for the publication – that looks more serious and is not too difficult: people are only too keen to prolong their publication-list. And we might want to publish our study in one of the many CAM-journals that are not too critical, as long as the result is positive.
Once our article is in print, we can legitimately claim that our bogus treatment is evidence-based. With a bit of luck, other research groups will proceed in the same way and soon we will have not just one but several positive studies. If not, we need to do two or three more trials along the same lines. The aim is to eventually do a meta-analysis that yields a convincingly positive verdict on our phony intervention.
You might think that I am exaggerating beyond measure. Perhaps a bit, I admit, but I am not all that far from the truth, believe me. You want proof? What about this one?
Researchers from the Charite in Berlin just published an RCT to investigate the effectiveness of a mindful walking program in patients with high levels of perceived psychological distress.
To prevent allegations of exaggeration, selective reporting, spin etc. I take the liberty of reproducing the abstract of this study unaltered:
Participants aged between 18 and 65 years with moderate to high levels of perceived psychological distress were randomized to 8 sessions of mindful walking in 4 weeks (each 40 minutes walking, 10 minutes mindful walking, 10 minutes discussion) or to no study intervention (waiting group). Primary outcome parameter was the difference to baseline on Cohen’s Perceived Stress Scale (CPSS) after 4 weeks between intervention and control.
Seventy-four participants were randomized in the study; 36 (32 female, 52.3 ± 8.6 years) were allocated to the intervention and 38 (35 female, 49.5 ± 8.8 years) to the control group. Adjusted CPSS differences after 4 weeks were -8.8 [95% CI: -10.8; -6.8] (mean 24.2 [22.2; 26.2]) in the intervention group and -1.0 [-2.9; 0.9] (mean 32.0 [30.1; 33.9]) in the control group, resulting in a highly significant group difference (P < 0.001).
Conclusion. Patients participating in a mindful walking program showed reduced psychological stress symptoms and improved quality of life compared to no study intervention. Further studies should include an active treatment group and a long-term follow-up
This whole thing could just be a bit of innocent fun, but I am afraid it is neither innocent nor fun, it is, in fact, quite serious. If we accept manipulated trials as evidence, we do a disservice to science, medicine and, most importantly, to patients. If the result of a trial is knowable before the study has even started, it is unethical to run the study. If the trial is not a true test but a simple promotional exercise, research degenerates into a farcical pseudo-science. If we abuse our patients’ willingness to participate in research, we jeopardise more serious investigations for the benefit of us all. If we misuse the scarce funds available for research, we will not have the money to conduct much needed investigations. If we tarnish the reputation of clinical research, we hinder progress.
If one spends a lot of time, as I presently do, sorting out old files, books, journals etc., one is bound to come across plenty of weird and unusual things. I for one, am slow at making progress with this task, mainly because I often start reading the material that is in front of me. It was one of those occasions that I had begun studying a book written by one of the more fanatic proponent of alternative medicine and stumbled over the term THE PROOF OF EXPERIENCE. It made me think, and I began to realise that the notion behind these four words is quite characteristic of the field of alternative health care.
When I studied medicine, in the 1970s, we were told by our peers what to do, which treatments worked for which conditions and why. They had all the experience and we, by definition, had none. Experience seemed synonymous with proof. Nobody dared to doubt the word of ‘the boss’. We were educated, I now realise, in the age of EMINENCE-BASED MEDICINE.
All of this gradually changed when the concepts of EVIDENCE-BASED MEDICINE became appreciated and generally adopted by responsible health care professionals. If now the woman or man on top of the medical ‘pecking order’ claims something that is doubtful in view of the published evidence, it is possible (sometimes even desirable) to say so – no matter how junior the doubter happened to be. As a result, medicine has thus changed for ever: progress is no longer made funeral by funeral [of the bosses] but new evidence is much more swiftly translated into clinical practice.
Don’t get me wrong, EVIDENCE-BASED MEDICINE does not does not imply disrespect EXPERIENCE; it merely takes it for what it is. And when EVIDENCE and EXPERIENCE fail to agree with each other, we have to take a deep breath, think hard and try to do something about it. Depending on the specific situation, this might involve further study or at least an acknowledgement of a degree of uncertainty. The tension between EXPERIENCE and EVIDENCE often is the impetus for making progress. The winner in this often complex story is the patient: she will receive a therapy which, according to the best available EVIDENCE and careful consideration of the EXPERIENCE, is best for her.
NOT SO IN ALTERNATIVE MEDICINE!!! Here EXPERIENCE still trumps EVIDENCE any time, and there is no need for acknowledging uncertainty: EXPERIENCE = proof!!!
In case you think I am exaggerating, I recommend thumbing through a few books on the subject. As I already stated, I have done this quite a bit in recent months, and I can assure you that there is very little evidence in these volumes to suggest that data, research, science, etc.. matter a hoot. No critical thinking is required, as long as we have EXPERIENCE on our side!
‘THE PROOF OF EXPERIENCE’ is still a motto that seems to be everywhere in alternative medicine. In many ways, it seems to me, this motto symbolises much of what is wrong with alternative medicine and the mind-set of its proponents. Often, the EXPERIENCE is in sharp contrast to the EVIDENCE. But this little detail does not seem to irritate anyone. Apologists of alternative medicine stubbornly ignore such contradictions. In the rare case where they do comment at all, the gist of their response normally is that EXPERIENCE is much more relevant than EVIDENCE. After all, EXPERIENCE is based on hundreds of years and thousands of ‘real-life’ cases, while EVIDENCE is artificial and based on just a few patients.
As far as I can see, nobody in alternative medicine pays more than a lip service to the fact that EXPERIENCE can be [and often is] grossly misleading. Little or no acknowledgement exists of the fact that, in clinical routine, there are simply far too many factors that interfere with our memories, impressions, observations and conclusions. If a patient gets better after receiving a therapy, she might have improved for a dozen reasons which are unrelated to the treatment per se. And if a patient does not get better, she might not come back at all, and the practitioner’s memory will therefore fail register such events as therapeutic failures. Whatever EXPERIENCE is, in health care, it rarely constitutes proof!
The notion of THE PROOF OF EXPERIENCE, it thus turns out, is little more than self-serving, wishful thinking which characterises the backward attitude that seems to be so remarkably prevalent in alternative medicine. No tension between EXPERIENCE and EVIDENCE is noticeable because the EVIDENCE is being ignored; as a result, there is no progress. The looser is, of course, the patient: she will receive a treatment based on criteria which are less than reliable.
Isn’t it time to burry the fallacy of THE PROOF OF EXPERIENCE once and for all?