MD, PhD, FMedSci, FSB, FRCP, FRCPEd

bias

As promised in the last post, I will try to briefly address the issues which make me uncomfortable about the quotes by Anthony Campbell. Readers will recall that Campbell, an ex-director of what was arguably the most influential homeopathic hospital in the world and a long-time editor of the journal HOMEOPATHY, freely admitted that homeopathy was unproven and its effects were most likely not due to any specific properties of the homeopathic remedies [which are, in fact, pure placebos] but largely rely on non-specific effects.

I agree with much that Campbell wrote but I disagree with one particular implication of his conclusions:Homeopathy has not been proved to work but neither has it been conclusively disproven….” and “…it is impossible to say categorically that all the remedies are without objective effect…”

This is an argument, we hear from proponents of alternative medicine with unfailing regularity:MY TREATMENT MAY NOT BE SUPPORTED BY GOOD SCIENCE [BECAUSE GOOD SCIENCE IS EXPENSIVE, AND WE CANNOT AFFORD IT] BUT IT HAS NOT BEEN DISPROVEN EITHER – AND, AS LONG AS IT IS NOT DISPROVEN, NOBODY SHOULD STOP US USING IT”

Campbell does not explicitly draw this latter conclusion but he certainly implies it. In his book, he explains that, even though homeopathic remedies probably are placebos, homeopathy does a lot of good through the placebo effect and through its spiritual aspects. And that is, in his view, sufficient reason to employ it for healing the sick. The very last sentence of his book reads: “Love it or loathe it, homeopathy is here to stay”

So the implication is there: alternative therapies can be as bizarre, nonsensical, implausible, unscientific or idiotic as they like, if we scientists cannot disprove them, they must be legitimate for general use. But there are, of course, two obvious errors in this line of reasoning:

  1. Why on earth should scientists waste their time and resources on testing notions which are clearly bonkers? It is hard to imagine research that is less fruitful than such an endeavour.
  2. Disproving homeopathy [or similarly ridiculous treatments] is a near impossibility. Proving a negative is rarely feasible in science.

In the best interest of patients, responsible health care has to follow an entirely different logic: we must consider any treatment to be unproven, while it is not supported with reasonably sound evidence for effectiveness; and in clinical routine, we employ mostly such treatments which are backed by sound evidence, and we avoid those that are unproven. In other words, whether homeopathy or any other medicine is unproven or disproven is of little practical consequence: we try not to use either category.

While I applaud Campbell’s candid judgement regarding the lack of effectiveness of homeopathic remedies, I feel the need to finish his conclusion for him giving it a dramatically different meaning: Homeopathy has not been proved to work but neither has it been conclusively disproven; this means that, until new evidence unambiguously demonstrates otherwise, we should classify homeopathy as ineffective – and this, of course, applies not just to homeopathy but to ALL unproven interventions.

Indian researchers published a survey aimed at determining the practice of prescription by homeopathic undergraduate students. A cross-sectional study was carried out involving all the students from 4 government homeopathic schools of West Bengal, India. Data were collected using self-administered questionnaires.

A total of 328 forms were completed. 80.5% of all homeopathic undergraduate students admitted prescribing homeopathic medicines independently and 40.5% said that they did this 2-3 times a year. The most common reasons for this activity were ‘urgency of the problem’ (35.2%), ‘previous experience with same kind of illness’ (31.8%), and ‘the problem too trivial to go to a doctor’ (25.8%). About 63.4% of the students thought that it was alright to independently diagnose an illness, while 51.2% thought that it was alright for them to prescribe medicines to others. Common conditions encountered were fever, indigestion, and injury. Prescription by students gradually increased with academic years of homeopathic schools. Many students thought it was alright for students to diagnose and treat illnesses.

The authors conclude that prescription of medicines by homeopathic undergraduate students is quite rampant and corrective measures are warranted.

It’s hard to know whether to laugh or cry about these findings:

  • If you are a homeopath, you ought to be upset to hear that students who are obviously neither fully trained, qualified or licensed already prescribe medicines.
  • If you are aware of the fact that homeopathic remedies are pure placebos, you might laugh about all this thinking “who cares?”
  • If you are into public health, you will worry that homeopaths are obviously being taught that homeopathic remedies can treat conditions which are considered to be urgent.
  • If you are someone who believes that sick people need evidence-based treatments, you might want to change the authors’ conclusion into something like: prescription of medicines by homeopaths is quite rampant and, in the interest of patients, corrective measures are required to stop them.

Informed consent is generally considered to be an essential precondition for any health care practice. It requires the clinician giving the patient full information about the condition and the possible treatments. Amongst other things, the following information may be needed:

  • the nature and prognosis of the condition,
  • the evidence regarding the efficacy and risks of the proposed treatment,
  • the evidence regarding alternative options.

Depending on the precise circumstances of the clinical situation, patient’s consent can be given either in writing or orally. Not obtaining any form of informed consent is a violation of the most fundamental ethics of health care.

In alternative medicine, informed consent seems often to be woefully neglected. This may have more than one reason:

  • practitioners have frequently no adequate training in medical ethics,
  • there is no adequate regulation and control of alternative practitioners,
  • practitioners have conflicts of interest and might view informed consent as commercially counter-productive

In order to render this discussion less theoretical, I will outline several scenarios from the realm of chiropractic. Specifically, I will discuss the virtual case of an asthma patient consulting a chiropractor for alleviation of his symptoms. I should stress that I have chosen chiropractic merely as an example – the issues outlines below apply to chiropractic as much as they apply to most other forms of alternative medicine.

SCENARIO 1

Our patient has experienced breathing problems and has heard that chiropractors are able to help this kind of condition. He consults a ‘straight’ chiropractor who adheres to Palmer’s gospel of ‘subluxation’. She explains to the patient that chiropractors use a holistic approach. By adjusting subluxations in the spine, she is confident to stimulate healing which will naturally ease the patient’s breathing problems. No conventional diagnosis is discussed, nor is there any mention of the prognosis, likelihood of benefit, risks of treatment and alternative therapeutic options.

SCENARIO 2

Our patient consults a chiropractor who does not fully believe in the ‘subluxation’ theory of chiropractic. She conducts a thorough examination of our patient’s spine and diagnoses several spinal segments that are blocked. She tells our patient that he might be suffering from asthma and that spinal manipulation might remove the blockages and thus increase the mobility of the spine which, in turn, would alleviate his breathing problems. She does not mention risks of the proposed interventions nor other therapeutic options.

SCENARIO 3

Our patient visits a chiropractor who considers herself a back pain specialist. She takes a medical history and conducts a physical examination. Subsequently she informs the patient that her breathing problems could be due to asthma and that she is neither qualified nor equipped to ascertain this diagnosis. She tells out patient that chiropractic is not an effective treatment for asthma but that his GP would be able to firstly make a proper diagnosis and secondly prescribe the optimal treatment for her condition. She writes a short note summarizing her thoughts and hands it to our patient to give it to his GP.

One could think of many more scenarios but the three above seem to cover a realistic spectrum of what a patient might encounter in real life. It seems clear, that the chiropractor in scenario 1 and 2 failed dismally regarding informed consent. In other words, only scenario 3 describes a behaviour that is ethically acceptable.

But how likely is scenario 3? I fear that it is an extremely rare turn of events. Even if well-versed in both medical ethics and scientific evidence, a chiropractor might think twice about providing all the information required for informed consent – because, as scenario 3 demonstrates, full informed consent in chiropractic essentially discourages a patient from agreeing to be treated. In other words, chiropractors have a powerful conflict of interest which prevents them to adhere to the rules of informed consent.

AND, AS POINTED OUT ALREADY, THAT DOES NOT JUST APPLY TO CHIROPRACTIC, IT APPLIES TO MOST OF ALTERNATIVE MEDICINE! IT SEEMS TO FOLLOW, I FEAR, THAT MUCH OF ALTERNATIVE MEDICINE IS UNETHICAL.

Dutch neurologists recently described the case of a 63-year-old female patient presented at their outpatient clinic with a five-week history of severe postural headache, tinnitus and nausea. The onset of these symptoms was concurrent with chiropractic manipulation of the cervical spine which she had tried because of cervical pain.

Cranial MRI showed findings characteristic for intracranial hypotension syndrome. Cervical MRI revealed a large posterior dural tear at the level of C1-2. Following unsuccessful conservative therapy, the patient underwent a lumbar epidural blood patch after which she recovered rapidly.

The authors conclude that manipulation of the cervical spine can cause a dural tear and subsequently an intracranial hypotension syndrome. Postural headaches directly after spinal manipulation should therefore be a reason to suspect this complication. If conservative management fails, an epidural blood patch may be performed.

Quite obviously, this is sound advice that can save lives. The trouble, however, is that the chiropractic profession is, by and large, still in denial. A recent systematic review by a chiropractor included eight cases of intracranial hypotension (IH) and concluded that case reports on IH and spinal manipulative therapy (SMT) have very limited clinical details and therefore cannot exclude other theories or plausible alternatives to explain the IH. To date, the evidence that cervical SMT is not a cause of IH is inconclusive. Further research is required before making any conclusions that cervical SMT is a cause of IH. Chiropractors and other health practitioners should be vigilant in recording established risk factors for IH in all cases. It is possible that the published cases of cervical SMT and IH may have missed important confounding risk factors (e.g. a new headache, or minor neck trauma in young or middle-aged adults).

Instead of distracting us from the fact that chiropractic can lead to serious adverse events, chiropractors would be well-advised to face the music, admit that their treatments are not risk-free and conduct rigorous research with a view of minimizing the harm.

Last week, it was announced that Claus Fritzsche had killed himself on 14 January 2014 at the age of 49. He was an industrious blogger and evangelic promoter of alternative medicine who seemed to spend much of his time and energy to defame those who disagreed with him. In this capacity, he certainly did tirelessly direct ‘ad hominem’ attacks in my direction. When it was revealed, about two years ago, that several German homeopathic firms paid him generously for this activity, his sponsors withdrew with plenty of egg on their faces, and subsequently Fritzsche’s insults became less frequent.

I never met Fritzsche in person but, over the years, I had many email exchanges with him. Invariably, these were unpleasant, to put it mildly. One might admire his tenacity but, from my perspective, it was hard to like Fritzsche. During the last months of his life, I refused to have contact with him, even via email – not because I failed to find our correspondence interesting or amusing, but because our exchanges always ended with some sort of escalation of aggression from his side.

Why then does his death sadden me so deeply?

Any death is a sad event but, if a death is so unnecessary and wasteful, it is particularly depressing. Fritzsche clearly had many skills and a lot of talents. He was young, intelligent and probably was a pleasant person to know personally, at least that is what some people who knew him have said. Alright, we did not agree on many things, but that does not mean that he was a bad person. He just seemed extremely irrational and tragically delivered the ultimate proof for his irrationality through his suicide.

Nobody knows what motivated Fritzsche to kill himself [when Walach speculated that his financial situation following the disclosure of the nature of his sponsorship had anything to do with it, he finds himself yet again way beyond the established facts]. Presumably, he suffered from depression, and presumably he was deeply insecure, and perhaps he was also desperately lonely.

Suspecting that this was the case, I now wish I had continued writing emails to him. Having argument after argument, even at the risk of getting yet again insulted and attacked, might have just been what was required to prevent him sliding into the abyss. I feel sorry for breaking off email contact when, in a strange sense, he might have needed me and the type of irritation people like me seemed to cause him.

The news that the use of Traditional Chinese Medicine (TCM) positively affects cancer survival might come as a surprise to many readers of this blog; but this is exactly what recent research has suggested. As it was published in one of the leading cancer journals, we should be able to trust the findings – or shouldn’t we?

The authors of this new study used the Taiwan National Health Insurance Research Database to conduct a retrospective population-based cohort study of patients with advanced breast cancer between 2001 and 2010. The patients were separated into TCM users and non-users, and the association between the use of TCM and patient survival was determined.

A total of 729 patients with advanced breast cancer receiving taxanes were included. Their mean age was 52.0 years; 115 patients were TCM users (15.8%) and 614 patients were TCM non-users. The mean follow-up was 2.8 years, with 277 deaths reported to occur during the 10-year period. Multivariate analysis demonstrated that, compared with non-users, the use of TCM was associated with a significantly decreased risk of all-cause mortality (adjusted hazards ratio [HR], 0.55 [95% confidence interval, 0.33-0.90] for TCM use of 30-180 days; adjusted HR, 0.46 [95% confidence interval, 0.27-0.78] for TCM use of > 180 days). Among the frequently used TCMs, those found to be most effective (lowest HRs) in reducing mortality were Bai Hua She She Cao, Ban Zhi Lian, and Huang Qi.

The authors of this paper are initially quite cautious and use adequate terminology when they write that TCM-use was associated with increased survival. But then they seem to get carried away by their enthusiasm and even name the TCM drugs which they thought were most effective in prolonging cancer survival. It is obvious that such causal extrapolations are well out of line with the evidence they produced (oh, how I wished that journal editors would finally wake up to such misleading language!) .

Of course, it is possible that some TCM drugs are effective cancer cures – but the data presented here certainly do NOT demonstrate anything like such an effect. And before such a far-reaching claim is being made, much more and much better research would be necessary.

The thing is, there are many alternative and plausible explanations for the observed phenomenon. For instance, it is conceivable that users and non-users of TCM in this study differed in many ways other than their medication, e.g. severity of cancer, adherence to conventional therapies, life-style, etc. And even if the researchers have used clever statistical methods to control for some of these variables, residual confounding can never be ruled out in such case-control studies.

Correlation is not causation, they say. Neglect of this elementary axiom makes for very poor science – in fact, it produces dangerous pseudoscience which could, like in the present case, lead a cancer patient straight up the garden path towards a premature death.

When we talk about conflicts of interest, we usually think of financial concerns. But conflicts of interests also extend to non-financial matters, such as strong beliefs. These are important in alternative medicine – I would even go as far as to claim that they dominate this field.

My detractors have often claimed that this is where my problem lies. They are convinced that, in 1993, I came into the job as PROFESSOR OF COMPLEMENTARY MEDICINE with an axe to grind; I was determined or perhaps even paid to show that all alternative medicine is utter hocus-pocus, they say. The truth is that, if anything, I was on the side of alternative medicine – and I can prove it. Using the example of homeopathy, I have dedicated an entire article to demonstrate that the myth is untrue – I was not closed-minded or out to ditch homeopathy (or any other form of alternative medicine for that matter).

What then could constitute my ‘conflict of interest’? Surely, he was bribed, I hear them say. Just look at the funds he took from industry. Some of those people have even gone to the trouble of running freedom of information requests to obtain the precise figures for my research-funding. Subsequently they triumphantly publish them and say: Look he got £x from this company and £y from that firm. And they are, of course, correct: I did receive support from commercially interested parties on several occasions. But what my detractors forget is that these were all pro-alternative medicine institutions. More importantly, I always made very sure that no strings were attached with any funds we accepted.

Our core funds came from ‘The Laing Foundation’ which endowed Exeter University with £ 1.5 million. This was done with the understanding that Exeter would put the same amount again into the kitty (which they never did). Anyone who can do simple arithmetic can tell that, to sustain up to 20 staff for almost 20 years, £1.5 million is not nearly enough. There must have been other sources. Who exactly gave money?

Despite utterly useless fundraising by the University, we did manage to obtain additional funds. I managed to receive support in the form of multiple research fellowships, for instance. It came from various sources; for instance, manufacturers of herbal medicines, Boots, the Pilkington Family Trust (yes, the glass manufacturers).

A hugely helpful contributor to our work was the sizable number (I estimate around 30) of visitors from abroad who came on their own money simply because they wanted to learn from and with us. They stayed between 3 months and 4 years, and importantly contributed to our research, knowledge and fun.

In addition, we soon devised ways to generate our own money. For instance, we started an annual conference for researchers in our field which ran for 14 successful years. As we managed everything on a shoestring and did all the organisation ourselves, we made a tidy profit each year which, of course, went straight back into our research. We also published several books which generated some revenue for the same purpose.

And then we received research funding for specific projects, for instance, from THE PRINCE OF WALES’ FOUNDATION FOR INTEGRATED HEALTH, a Japanese organisation supporting Jorhei Healing, THE WELCOME TRUST, the NHS, and even a homeopathic company.

So, do I have a conflict of interest? Did I take money from anyone who might have wanted to ditch alternative medicine? I don’t think so! And if I tell you that, when I came to Exeter in 1993, I donated ~£120 000 of my own funds towards the research of my unit, even my detractors might, for once, be embarrassed to have thought otherwise.

The most widely used definition of EVIDENCE-BASED MEDICINE (EBM) is probably this one: The judicious use of the best current available scientific research in making decisions about the care of patients. Evidence-based medicine (EBM) is intended to integrate clinical expertise with the research evidence and patient values.

David Sackett’s own definition is a little different: Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research.

Even though the principles of EBM are now widely accepted, there are those who point out that EBM has its limitations. The major criticisms of EBM relate to five themes: reliance on empiricism, narrow definition of evidence, lack of evidence of efficacy, limited usefulness for individual patients, and threats to the autonomy of the doctor/patient relationship.

Advocates of alternative medicine have been particularly vocal in pointing out that EBM is not really applicable to their area. However, as their arguments were less than convincing, a new strategy for dealing with EBM seemed necessary. Some proponents of alternative medicine therefore are now trying to hoist EBM-advocates by their own petard.

In doing so they refer directly to the definitions of EBM and argue that EBM has to fulfil at least three criteria: 1) external best evidence, 2) clinical expertise and 3) patient values or preferences.

Using this argument, they thrive to demonstrate that almost everything in alternative medicine is evidence-based. Let me explain this with two deliberately extreme examples.

CRYSTAL THERAPY FOR CURING CANCER

There is, of course, not a jot of evidence for this. But there may well be the opinion held by crystal therapist that some cancer patients respond to their treatment. Thus the ‘best’ available evidence is clearly positive, they argue. Certainly the clinical expertise of these crystal therapists is positive. So, if a cancer patient wants crystal therapy, all three preconditions are fulfilled and CRYSTAL THERAPY IS ENTIRELY EVIDENCE-BASED.

CHIROPRACTIC FOR ASTHMA

Even the most optimistic chiropractor would find it hard to deny that the best evidence does not demonstrate the effectiveness of chiropractic for asthma. But never mind, the clinical expertise of the chiropractor may well be positive. If the patient has a preference for chiropractic, at least two of the three conditions are fulfilled. Therefore – on balance – chiropractic for asthma is [fairly] evidence-based.

The ‘HOISTING ON THE PETARD OF EBM’-method is thus a perfect technique for turning the principles of EBM upside down. Its application leads us straight back into the dark ages of medicine when anything was legitimate as long as some charlatan could convince his patients to endure his quackery and pay for it – if necessary with his life.

Do you think that chiropractic is effective for asthma? I don’t – in fact, I know it isn’t because, in 2009, I have published a systematic review of the available RCTs which showed quite clearly that the best evidence suggested chiropractic was ineffective for that condition.

But this is clearly not true, might some enthusiasts reply. What is more, they can even refer to a 2010 systematic review which indicates that chiropractic is effective; its conclusions speak a very clear language: …the eight retrieved studies indicated that chiropractic care showed improvements in subjective measures and, to a lesser degree objective measures… How on earth can this be?

I would not be surprised, if chiropractors claimed the discrepancy is due to the fact that Prof Ernst is biased. Others might point out that the more recent review includes more studies and thus ought to be more reliable. The newer review does, in fact, have about twice the number of studies than mine.

How come? Were plenty of new RCTs published during the 12 months that lay between the two publications? The answer is NO. But why then the discrepant conclusions?

The answer is much less puzzling than you might think. The ‘alchemists of alternative medicine’ regularly succeed in smuggling non-evidence into such reviews in order to beautify the overall picture and confirm their wishful thinking. The case of chiropractic for asthma does by no means stand alone, but it is a classic example of how we are being misled by charlatans.

Anyone who reads the full text of the two reviews mentioned above will find that they do, in fact, include exactly the same amount of RCTs. The reason why they arrive at different conclusions is simple: the enthusiasts’ review added NON-EVIDENCE to the existing RCTs. To be precise, the authors included one case series, one case study, one survey, two randomized controlled trials (RCTs), one randomized patient and observer blinded cross-over trial, one single blind cross study design, and one self-reported impairment questionnaire.

Now, there is nothing wrong with case reports, case series, or surveys – except THEY TELL US NOTHING ABOUT EFFECTIVENESS. I would bet my last shirt that the authors know all of that; yet they make fairly firm and positive conclusions about effectiveness. As the RCT-results collectively happen to be negative, they even pretend that case reports etc. outweigh the findings of RCTs.

And why do they do that? Because they are interested in the truth, or because they don’t mind using alchemy in order to mislead us? Your guess is as good as mine.

Systematic reviews are widely considered to be the most reliable type of evidence for judging the effectiveness of therapeutic interventions. Such reviews should be focused on a well-defined research question and identify, critically appraise and synthesize the totality of the high quality research evidence relevant to that question. Often it is possible to pool the data from individual studies and thus create a new numerical result of the existing evidence; in this case, we speak of a meta-analysis, a sub-category of systematic reviews.

One strength of systematic review is that they minimise selection and random biases by considering at the totality of the evidence of a pre-defined nature and quality. A crucial precondition, however, is that the quality of the primary studies is critically assessed. If this is done well, the researchers will usually be able to determine how robust any given result is, and whether high quality trials generate similar findings as those of lower quality. If there is a discrepancy between findings from rigorous and flimsy studies, it is obviously advisable to trust the former and discard the latter.

And this is where systematic reviews of alternative treatments can run into difficulties. For any given research question in this area we usually have a paucity of primary studies. Equally important is the fact that many of the available trials tend to be of low quality. Consequently, there often is a lack of high quality studies, and this makes it all the more important to include a robust critical evaluation of the primary data. Not doing so would render the overall result of the review less than reliable – in fact, such a paper would not qualify as a systematic review at all; it would be a pseudo-systematic review, i.e. a review which pretends to be systematic but, in fact, is not. Such papers are a menace in that they can seriously mislead us, particularly if we are not familiar with the essential requirements for a reliable review.

This is precisely where some promoters of bogus treatments seem to see their opportunity of making their unproven therapy look as though it was evidence-based. Pseudo-systematic reviews can be manipulated to yield a desired outcome. In my last post, I have shown that this can be done by including treatments which are effective so that an ineffective therapy appears effective (“chiropractic is so much more than just spinal manipulation”). An even simpler method is to exclude some of the studies that contradict one’s belief from the review. Obviously, the review would then not comprise the totality of the available evidence. But, unless the reader bothers to do a considerable amount of research, he/she would be highly unlikely to notice. All one needs to do is to smuggle the paper past the peer-review process – hardly a difficult task, given the plethora of alternative medicine journals that bend over backwards to publish any rubbish as long as it promotes alternative medicine.

Alternatively (or in addition) one can save oneself a lot of work and omit the process of critically evaluating the primary studies. This method is increasingly popular in alternative medicine. It is a fool-proof method of generating a false-positive overall result. As poor quality trials have a tendency to deliver false-positive results, it is obvious that a predominance of flimsy studies must create a false-positive result.

A particularly notorious example of a pseudo-systematic review that used this as well as most of the other tricks for misleading the reader is the famous ‘systematic’ review by Bronfort et al. It was commissioned by the UK GENERAL CHIROPRACTIC COUNCIL after the chiropractic profession got into trouble and was keen to defend those bogus treatments disclosed by Simon Singh. Bronfort and his colleagues thus swiftly published (of course, in a chiro-journal) an all-encompassing review attempting to show that, at least for some conditions, chiropractic was effective. Its lengthy conclusions seemed encouraging: Spinal manipulation/mobilization is effective in adults for: acute, subacute, and chronic low back pain; migraine and cervicogenic headache; cervicogenic dizziness; manipulation/mobilization is effective for several extremity joint conditions; and thoracic manipulation/mobilization is effective for acute/subacute neck pain. The evidence is inconclusive for cervical manipulation/mobilization alone for neck pain of any duration, and for manipulation/mobilization for mid back pain, sciatica, tension-type headache, coccydynia, temporomandibular joint disorders, fibromyalgia, premenstrual syndrome, and pneumonia in older adults. Spinal manipulation is not effective for asthma and dysmenorrhea when compared to sham manipulation, or for Stage 1 hypertension when added to an antihypertensive diet. In children, the evidence is inconclusive regarding the effectiveness for otitis media and enuresis, and it is not effective for infantile colic and asthma when compared to sham manipulation. Massage is effective in adults for chronic low back pain and chronic neck pain. The evidence is inconclusive for knee osteoarthritis, fibromyalgia, myofascial pain syndrome, migraine headache, and premenstrual syndrome. In children, the evidence is inconclusive for asthma and infantile colic. 

Chiropractors across the world cite this paper as evidence that chiropractic has at least some evidence base. What they omit to tell us (perhaps because they do not appreciate it themselves) is the fact that Bronfort et al

  • failed to formulate a focussed research question,
  • invented his own categories of inconclusive findings,
  • included all sorts of studies which had nothing to do with chiropractic,
  • and did not to make an assessment of the quality of the included primary studies they included in their review.

If, for a certain condition, three trials were included, for instance, two of which were positive but of poor quality and one was negative but of good quality, the authors would conclude that, overall, there is sound evidence.

Bronfort himself is, of course, more than likely to know all that (he has learnt his trade with an excellent Dutch research team and published several high quality reviews) - but his readers mostly don’t. And for chiropractors, this ‘systematic’ review is now considered to be the most reliable evidence in their field.

Categories