MD, PhD, FMedSci, FSB, FRCP, FRCPEd

bias

1 2 3 12

Much has been written on this blog about progress in the area of chiropractic practice and research. But where is the evidence for progress? I did a little search and one of the first sites I stumbled across was this one which is full to bursting with bogus claims. This cannot be what chiropractors call ‘progress’, I thought.

Determined to find real progress, I continued searching and found THE FOUNDATION FOR CHIROPRACTIC PROGRESS. Great, I thought, an organisation and a website entirely devoted to the very subject I was looking for. Consequently, I studied the information provided here in some detail. What follows are excerpts from the site:

Chiropractic care is a health option that has proven beneficial for a multitude of health conditions, along with in the practice of achieving optimal wellness. It is essential for those unaware of chiropractic care to be adequately informed, so they too can experience the benefits that over 60,000 practicing doctors of chiropractic in the U.S. provide to their patients daily. Established in 2003, the not-for-profit Foundation for Chiropractic Progress (F4CP) aims to educate the public about the many benefits associated with chiropractic care.On behalf of the F4CP, I invite you to tour this site and learn more about this effective form of treatment.
Kind regards,
Kent Greenawalt
Chairman | Foundation for Chiropractic Progress
THIS WAS A STRANGE INTRODUCTION, I THOUGHT; BUT UNDETERRED I READ ON:
Parents of Colicky Infants Turn to Chiropractic Care

For those parents who never imagined their ailing babies and toddlers could be helped by chiropractic care, it may be time for some rethinking.New mom Jean, a 31-year-old speech therapist from New Jersey, became an advocate after enlisting the help of her own chiropractor to treat her colicky infant girl, Emma. After having had what she says was “no luck” with the usual ways of alleviating colic symptoms – including giving Emma children’s probiotics daily – one appointment with board-certified in chiropractic pediatrics Dr. Lora Tanis produced an immediate difference.

Concussions Among Athletes

A concussion is a type of traumatic brain injury caused by a bump, blow or jolt to the head that can change the way the brain functions. Symptoms include dizziness, instability and confusion.

Using methods that rely on brain-based, non-invasive, drugfree approaches — like chiropractic
care and physical rehab — can help re-establish balance and maximal brain and nervous system functionality.

News of Health – Improving Military Health Care

Retired U.S. Army Brigadier General Becky Halstead—the first woman in U.S. history to command in combat at the strategic level—is speaking out on the value of chiropractic care for the nation’s military men and women.

Good Health

With the epidemic now estimated to be costing the nation $147 billion annually, it’s a question that’s very much on the minds of health experts. And many, including lifestyle guru Shea Vaughn, are citing chiropractic care as a crucial part of overall wellness programs.

FEELING A LITTLE DISAPPOINTED, I STOPPED READING AND THOUGHT

PROGRESS INDEED !!!

Homeopathy has many critics who claim that there is no good evidence for this type of therapy. Homeopaths invariably find this most unfair and point to a plethora of studies that show an effect. They are, of course, correct! There are plenty of trials that suggest that homeopathic remedies do work. The question, however, is HOW RELIABLE ARE THESE STUDIES?

Here is a brand new one which might stand for dozens of others.

In this study, homeopaths treated 50 multimorbid patients with homeopathic remedies identifies by a method called ‘polarity analysis’ (PA) and prospectively followed them over one year (PA enables homeopaths to calculate a relative healing probability, based on Boenninghausen’s grading of polar symptoms).

The 43 patients (86%) who completed the observation period experienced an average improvement of 91% in their initial symptoms. Six patients dropped out, and one did not achieve an improvement of 80%, and was therefore also counted as a treatment failure. The cost of homeopathic treatment was 41% of projected equivalent conventional treatment.

Good news then for enthusiasts of homeopathy? 91% improvement!

Yet, I am afraid that critics might not be bowled over. They might smell a whiff of selection bias, lament the lack of a control group or regret the absence of objective outcome measures. But I was prepared to go as far as stating that such results might be quite interesting… until I read the authors’ conclusions that is:

Polarity Analysis is an effective method for treating multimorbidity. The multitude of symptoms does not prevent the method from achieving good results. Homeopathy may be capable of taking over a considerable proportion of the treatment of multimorbid patients, at lower costs than conventional medicine.

Virtually nothing in these conclusions is based on the data provided. They are pure extrapolation and wild assumptions. Two questions seem to emerge from this:

  1. How on earth can we take this and so many other articles on homeopathy seriously?
  2. When does this sort of article cross the line between wishful thinking and scientific misconduct?

Few subjects lead to such heated debate as the risk of stroke after chiropractic manipulations (if you think this is an exaggeration, look at the comment sections of previous posts on this subject). Almost invariably, one comes to the conclusion that more evidence would be helpful for arriving at firmer conclusions. Before this background, this new publication by researchers (mostly chiropractors) from the US ‘Dartmouth Institute for Health Policy & Clinical Practice’ is noteworthy.

The purpose of this study was to quantify the risk of stroke after chiropractic spinal manipulation, as compared to evaluation by a primary care physician, for Medicare beneficiaries aged 66 to 99 years with neck pain.

The researchers conducted a retrospective cohort analysis of a 100% sample of annualized Medicare claims data on 1 157 475 beneficiaries aged 66 to 99 years with an office visit to either a chiropractor or to a primary care physician for neck pain. They compared hazard of vertebrobasilar stroke and any stroke at 7 and 30 days after office visit using a Cox proportional hazards model. We used direct adjusted survival curves to estimate cumulative probability of stroke up to 30 days for the 2 cohorts.

The findings indicate that the proportion of subjects with a stroke of any type in the chiropractic cohort was 1.2 per 1000 at 7 days and 5.1 per 1000 at 30 days. In the primary care cohort, the proportion of subjects with a stroke of any type was 1.4 per 1000 at 7 days and 2.8 per 1000 at 30 days. In the chiropractic cohort, the adjusted risk of stroke was significantly lower at 7 days as compared to the primary care cohort (hazard ratio, 0.39; 95% confidence interval, 0.33-0.45), but at 30 days, a slight elevation in risk was observed for the chiropractic cohort (hazard ratio, 1.10; 95% confidence interval, 1.01-1.19).

The authors conclude that, among Medicare B beneficiaries aged 66 to 99 years with neck pain, incidence of vertebrobasilar stroke was extremely low. Small differences in risk between patients who saw a chiropractor and those who saw a primary care physician are probably not clinically significant.

I do, of course, applaud any new evidence on this rather ‘hot’ topic – but is it just me, or are the above conclusions a bit odd? Five strokes per 1000 patients is definitely not “extremely low” in my book; and furthermore I do wonder whether all experts would agree that a doubling of risk at 30 days in the chiropractic cohort is “probably not clinically significant” – particularly, if we consider that chiropractic spinal manipulation has so very little proven benefit.

My message to (chiropractic) researchers is simple: PLEASE REMEMBER THAT SCIENCE IS NOT A TOOL FOR CONFIRMING BUT FOR TESTING HYPOTHESES.

On 1/12/2014 I published a post in which I offered to give lectures to students of alternative medicine:

Getting good and experienced lecturers for courses is not easy. Having someone who has done more research than most working in the field and who is internationally known, might therefore be a thrill for students and an image-boosting experience of colleges. In the true Christmas spirit, I am today making the offer of being of assistance to the many struggling educational institutions of alternative medicine .

A few days ago, I tweeted about my willingness to give free lectures to homeopathic colleges (so far without response). Having thought about it a bit, I would now like to extend this offer. I would be happy to give a free lecture to the students of any educational institution of alternative medicine.

I did not think that this would create much interest – and I was right: only the ANGLO-EUROPEAN COLLEGE OF CHIROPRACTIC has so far hoisted me on my own petard and, after some discussion (see comment section of the original post) hosted me for a lecture. Several people seem keen on knowing how this went; so here is a brief report.

I was received, on 14/1/2015, with the utmost kindness by my host David Newell. We has a coffee and a chat and then it was time to start the lecture. The hall was packed with ~150 students and the same number was listening in a second lecture hall to which my talk was being transmitted.

We had agreed on the title CHIROPRACTIC: FALLACIES AND FACTS. So, after telling the audience about my professional background, I elaborated on 7 fallacies:

  1. Appeal to tradition
  2. Appeal to authority
  3. Appeal to popularity
  4. Subluxation exists
  5. Spinal manipulation is effective
  6. Spinal manipulation is safe
  7. Ad hominem attack

Numbers 3, 5 and 6 were dealt with in more detail than the rest. The organisers had asked me to finish by elaborating on what I perceive as the future challenges of chiropractic; so I did:

  1. Stop happily promoting bogus treatments
  2. Denounce obsolete concepts like ‘subluxation’
  3. Clarify differences between chiros, osteos and physios
  4. Start a culture of critical thinking
  5. Take action against charlatans in your ranks
  6. Stop attacking everyone who voices criticism

I ended by pointing out that the biggest challenge, in my view, was to “demonstrate with rigorous science which chiropractic treatments demonstrably generate more good than harm for which condition”.

We had agreed that my lecture would be followed by half an hour of discussion; this period turned out to be lively and had to be extended to a full hour. Most questions initially came from the tutors rather than the students, and most were polite – I had expected much more aggression.

In his email thanking me for coming to Bournemouth, David Newell wrote about the event: The general feedback from staff and students was one of relief that you possessed only one head, :-). I hope you may have felt the same about us. You came over as someone who had strong views, a fair amount of which we disagreed with, but that presented them in a calm, informative and courteous manner as we did in listening and discussing issues after your talk. I think everyone enjoyed the questions and debate and felt that some of the points you made were indeed fair critique of what the profession may need to do, to secure a more inclusive role in the health care arena.

 
As you may have garnered from your visit here, the AECC is committed to this task as we continue to provide the highest quality of education for the 21st C representatives of such a profession. We believe centrally that it is to our society at large and our communities within which we live and work that we are accountable. It is them that we serve, not ourselves, and we need to do that as best we can, with the best tools we have or can develop and that have as much evidence as we can find or generate. In this aim, your talk was important in shining a more ‘up close and personal’ torchlight on our profession and the tasks ahead whilst also providing us with a chance to debate the veracity or otherwise of yours and ours differing positions on interpretation of the evidence.

My own impression of the day is that some of my messages were not really understood, that some of the questions, including some from the tutors, seemed like coming from a different planet, and that people were more out to teach me than to learn from my talk. One overall impression that I took home from that day is that, even in this college which prides itself of being open to scientific evidence and unimpressed by chiropractic fundamentalism, students are strangely different from other health care professionals. The most tangible aspect of this is the openly hostile attitude against drug therapies voiced during the discussion by some students.

The question I always ask myself after having invested a lot of time in preparing and delivering a lecture is: WAS IT WORTH IT? In the case of this lecture, I think the answer is YES. With 300 students present, I am fairly confident that I did manage to stimulate a tiny bit of critical thinking in a tiny percentage of them. The chiropractic profession needs this badly!

 

According to the ‘General Osteopathic Council’ (GOC), osteopathy is a primary care profession, focusing on the diagnosis, treatment, prevention and rehabilitation of musculoskeletal disorders, and the effects of these conditions on patients’ general health.

Using many of the diagnostic procedures applied in conventional medical assessment, osteopaths seek to restore the optimal functioning of the body, where possible without the use of drugs or surgery. Osteopathy is based on the principle that the body has the ability to heal, and osteopathic care focuses on strengthening the musculoskeletal systems to treat existing conditions and to prevent illness. 

Osteopaths’ patient-centred approach to health and well-being means they consider symptoms in the context of the patient’s full medical history, as well as their lifestyle and personal circumstances. This holistic approach ensures that all treatment is tailored to the individual patient.

On a good day, such definitions make me smile; on a bad day, they make me angry. I can think of quite a few professions which would fit this definition just as well or better than osteopathy. What are we supposed to think about a profession that is not even able to provide an adequate definition of itself?

Perhaps I try a different angle: what conditions do osteopaths treat? The GOC informs us that commonly treated conditions include back and neck pain, postural problems, sporting injuries, muscle and joint deterioration, restricted mobility and occupational ill-health.

This statement seems not much better than the previous one. What on earth is ‘muscle and joint deterioration’? It is not a condition that I find in any medical dictionary or textbook. Can anyone think of a broader term than ‘occupational ill health’? This could be anything from tennis elbow to allergies or depression. Do osteopaths treat all of those?

One gets the impression that osteopaths and their GOC are deliberately vague – perhaps because this would diminish the risk of being held to account on any specific issue?

The more one looks into the subject of osteopathy, the more confused one gets. The profession goes back to Andrew Still ((August 6, 1828 – December 12, 1917) Palmer, the founder of chiropractic is said to have been one of Still’s pupils and seems to have ‘borrowed’ most of his concepts from him – even though he always denied this) who defined osteopathy as a science which consists of such exact exhaustive and verifiable knowledge of the structure and functions of the human mechanism, anatomy and physiology & psychology including the chemistry and physics of its known elements as is made discernable certain organic laws and resources within the body itself by which nature under scientific treatment peculiar to osteopathic practice apart from all ordinary methods of extraneous, artificial & medicinal stimulation and in harmonious accord with its own mechanical principles, molecular activities and metabolic processes may recover from displacements, derangements, disorganizations and consequent diseases and regain its normal equilibrium of form and function in health and strength.

This and many other of his statements seem to indicate that the art of using language for obfuscation has a long tradition in osteopathy and goes back directly to its founding father.

What makes the subject of osteopathy particularly confusing is not just the oddity that, in conventional medicine, the term means ‘disease of the bone’ (which renders any literature searches in this area a nightmare) but also the fact that, in different countries, osteopaths are entirely different professionals. In the US, osteopathy has long been fully absorbed by mainstream medicine and there is hardly any difference between MDs and ODs. In the UK, osteopaths are alternative practitioners regulated by statute but are, compared to chiropractors, of minor importance. In Germany, osteopaths are not regulated and fairly ‘low key’, while in France, they are numerous and like to see themselves as primary care physicians.

And what about the evidence base of osteopathy? Well, that’s even more confusing, in my view. Evidence for which treatment? As US osteopaths might use any therapy from drugs to surgery, it could get rather complicated. So let’s just focus on the manual treatment as used by osteopaths outside the US.

Anyone who attempts to critically evaluate the published trial evidence in this area will be struck by at least two phenomena:

  1. the wide range of conditions treated with osteopathic manual therapy (OMT)
  2. the fact that there are several groups of researchers that produce one positive result after the next.

The best example is probably the exceedingly productive research team of J. C. Licciardone from the Osteopathic Research Center, University of North Texas. Here are a few conclusions from their clinical studies:

  1. The large effect size for OMT in providing substantial pain reduction in patients with chronic LBP of high severity was associated with clinically important improvement in back-specific functioning. Thus, OMT may be an attractive option in such patients before proceeding to more invasive and costly treatments.
  2. The large effect size for short-term efficacy of OMT was driven by stable responders who did not relapse.
  3. Osteopathic manual treatment has medium to large treatment effects in preventing progressive back-specific dysfunction during the third trimester of pregnancy. The findings are potentially important with respect to direct health care expenditures and indirect costs of work disability during pregnancy.
  4. Severe somatic dysfunction was present significantly more often in patients with diabetes mellitus than in patients without diabetes mellitus. Patients with diabetes mellitus who received OMT had significant reductions in LBP severity during the 12-week period. Decreased circulating levels of TNF-α may represent a possible mechanism for OMT effects in patients with diabetes mellitus. A larger clinical trial of patients with diabetes mellitus and comorbid chronic LBP is warranted to more definitively assess the efficacy and mechanisms of action of OMT in this population.
  5. The OMT regimen met or exceeded the Cochrane Back Review Group criterion for a medium effect size in relieving chronic low back pain. It was safe, parsimonious, and well accepted by patients.
  6. Osteopathic manipulative treatment slows or halts the deterioration of back-specific functioning during the third trimester of pregnancy.
  7. The only consistent finding in this study was an association between type 2 diabetes mellitus and tissue changes at T11-L2 on the right side. Potential explanations for this finding include reflex viscerosomatic changes directly related to the progression of type 2 diabetes mellitus, a spurious association attributable to confounding visceral diseases, or a chance observation unrelated to type 2 diabetes mellitus. Larger prospective studies are needed to better study osteopathic palpatory findings in type 2 diabetes mellitus.
  8. OMT significantly reduces low back pain. The level of pain reduction is greater than expected from placebo effects alone and persists for at least three months. Additional research is warranted to elucidate mechanistically how OMT exerts its effects, to determine if OMT benefits are long lasting, and to assess the cost-effectiveness of OMT as a complementary treatment for low back pain.

Based on this brief review of the evidence origination from one of the most active research team, one could be forgiven to think that osteopathy is a panacea. But such an assumption is, of course, nonsensical; a more reasonable conclusion might be the following: osteopathy is one of the most confusing and confused subject under the already confused umbrella of alternative medicine.

If we go on the internet, we find no end of positive claims for TM. The official TM website, for instance, claims that more than 350 peer-reviewed research studies on the TM technique have been published in over 160 scientific journals. These studies were conducted at many US and international universities and research centers, including Harvard Medical School, Stanford Medical School, Yale Medical School, and UCLA Medical School.

This may well be true – but do those studies amount to more than a heap of beans? Let’s find out.

The objective of our Cochrane review was to determine the effectiveness of TM for the primary prevention of cardiovascular disease (CVD). We searched the following electronic databases: the Cochrane Central Register of Controlled Trials (CENTRAL) (2013, Issue 10); MEDLINE (Ovid) (1946 to week three November 2013); EMBASE Classic and EMBASE (Ovid) (1947 to week 48 2013); ISI Web of Science (1970 to 28 November 2013); and Database of Abstracts of Reviews of Effects (DARE) and Health Technology Assessment Database and Health Economics Evaluations Database (November 2013). We also searched the Allied and complementary Medicine Database (AMED) (inception to January 2014) and IndMed (inception to January 2014). We hand searched trial registers and reference lists of reviews and articles and contacted experts in the field. We applied no language restrictions.

We included randomised controlled trials (RCTs) of at least three months’ duration involving healthy adults or adults at high risk of CVD. Trials examined TM only and the comparison group was no intervention or minimal intervention. We excluded trials that involved multi-factorial interventions. Outcomes of interest were clinical CVD events (cardiovascular mortality, all-cause mortality and non-fatal events) and major CVD risk factors (e.g. blood pressure and blood lipids, occurrence of type 2 diabetes, quality of life, adverse events and costs). Two authors independently selected trials for inclusion, extracted data and assessed the risk of bias.

We identified 4 RCTs with a total of 430 participants for inclusion in this review. The included trials were small, short term (three months) and at risk of bias. In all studies, TM was practised for 15 to 20 minutes twice a day. None of the included studies reported all-cause mortality, cardiovascular mortality or non-fatal endpoints as trials were short term, but one study reported survival rate three years after the trial was completed. In view of the considerable statistical heterogeneity between the results of the studies for the only outcomes reported, systolic blood pressure (I2 = 72%) and diastolic blood pressure (I2 = 66%), we decided not to undertake a meta-analysis. None of the four trials reported blood lipids, occurrence of type 2 diabetes, adverse events, costs or quality of life.

We concluded that there are few trials with limited outcomes examining the effectiveness of TM for the primary prevention of CVD. Due to the limited evidence to date, we could draw no conclusions as to the effectiveness of TM for the primary prevention of CVD. There was considerable heterogeneity between trials and the included studies were small, short term and at overall serious risk of bias. More and larger long-term, high-quality trials are needed.

Even though I am a co-author of this review, I am not entirely sure that the last sentence of our conclusion is totally correct. The TM movement has, in my view, all the characteristics of a cult with all its the dangers that cults entail. This means, I think, we ought to be cautious about TM and sceptical about their research and results. At the risk of provoking harsh criticism, I would even say we should be distrustful of their aims and methods.

I would have never thought that someone would be able to identify the author of the text I quoted in the previous post:

It is known that not just novel therapies but also traditional ones, such as homeopathy, suffer opposition and rejection by some doctors without having ever been subjected to serious tests. The doctor is in charge of medical treatment; he is thus responsible foremost for making sure all knowledge and all methods are employed for the benefit of public health…I ask the medical profession to consider even previously excluded therapies with an open mind. It is necessary that an unbiased evaluation takes place, not just of the theories but also of the clinical effectiveness of alternative medicine.

More often than once has science, when it relied on theory alone, arrived at verdicts which later had to be overturned – frequently this occurred only after long periods of time, after progress had been hindered and most acclaimed pioneers had suffered serious injustice. I do not need to remind you of the doctor who, more than 100 years ago, in fighting puerperal fever, discovered sepsis and asepsis but was laughed at and ousted by his colleagues throughout his lifetime. Yet nobody would today deny that this knowledge is most relevant to medicine and that it belongs to the basis of medicine. Insightful doctors, some of whom famous, have, during the recent years, spoken openly about the crisis in medicine and the dead end that health care has maneuvered itself into. It seems obvious that the solution is going in directions which embrace nature. Hardly any other form of science is so tightly bound to nature as is the science occupied with healing living creatures. The demand for holism is getting stronger and stronger, a general demand which has already been fruitful on the political level. For medicine, the challenge is to treat more than previously by influencing the whole organism when we aim to heal a diseased organ.

It is from the opening speech by Rudolf Hess on the occasion of the WORLD CONFERENCE ON HOMEOPATHY 1937, in Berlin. Hess, at the time Hitler’s deputy, was not the only Nazi-leader. I knew of the opening speech because, a few years ago, DER SPIEGEL published a theme issue on homeopathy, and they published a photo of the opening ceremony of this meeting. It shows many men in SS-uniform and, in the first row of the auditorium, we see Hess (as well as Himmler) ready to spring into action.

Hess in particular was besotted with alternative medicine which the Nazis elected to call NEUE DEUTSCHE HEILKUNDE. Somewhat to the dismay of today’s alternative medicine enthusiasts, I have repeatedly published on this aspect of alternative medicine’s past, and it also is an important part of my new book A SCIENTIST IN WONDERLAND which the lucky winner (my congratulations!) of my little competition to identify the author has won. The abstract of an 2001 article explains this history succinctly:

The aim of this article is to discuss complementary/alternative medicine (CAM) in the Third Reich. Based on a general movement towards all things natural, a powerful trend towards natural ways of healing had developed in the 19(th)century. By 1930 this had led to a situation where roughly as many lay practitioners of CAM existed in Germany as doctors. To re-unify German medicine under the banner of ‘Neue Deutsche Heilkunde’, the Nazi officials created the ‘Heilpraktiker’ – a profession which was meant to become extinct within one generation. The ‘flag ship’ of the ‘Neue Deutsche Heilkunde’ was the ‘Rudolf Hess Krankenhaus’ in Dresden. It represented a full integration of CAM and orthodox medicine. An example of systematic research into CAM is the Nazi government’s project to validate homoeopathy. Even though the data are now lost, the results of this research seem to have been negative. Even though there are some striking similarities between today’s CAM and yesterday’s ‘Neue Deutsche Heilkunde’ there are important differences. Most importantly, perhaps, today’s CAM is concerned with the welfare of the individual, whereas the ‘Neue Deutsche Heilkunde’ was aimed at ensuring the dominance of the Aryan race.

One fascinating aspect of this past is the fact that the NEUE DEUTSCHE HEILKUNDE was de facto the invention of what we today call ‘integrated medicine’. Then it was more like a ‘shot-gun marriage’, while today it seems to be driven more by political correctness and sloppy thinking. It did not work 70 years ago for the same reason that it will fail today: the integration of bogus (non-evidence based) treatments into conventional medicine must inevitably render health care not better but worse!

One does not need to be a rocket scientist to understand that, and Hess as well as other proponents of alternative medicine of his time had certainly got the idea. So they initiated the largest ever series of scientific tests of homeopathy. This research program was not just left to the homeopaths, who never had a reputation of being either rigorous or unbiased, but some of the best scientists of the era were recruited for it. The results vanished in the hands of the homeopaths during the turmoil of the war. But one eye-witness report of a homeopaths, Fritz Donner, makes it very clear: as it turned out, there was not a jot of evidence in favour of homeopathy.

And this, I think, is the other fascinating aspect of the story: homeopaths did not give up their plight to popularise homeopathy. On the contrary, they re-doubled their efforts to fool us all and to convince us with dodgy results (see recent posts on this blog) that homeopathy somehow does defy the laws of nature and is, in effect, very effective for all sorts of diseases.

My readers suggested all sorts of potential authors for the Hess speech; and they are right! It could have been written by any proponent of alternative medicine. This fact is amusing and depressing at the same time. Amusing because it discloses the lack of new ideas and arguments (even the same fallacies are being used). Depressing because it suggests that progress in alternative medicine is almost totally absent.

The very first article on a subject related to alternative medicine with a 2015 date that I came across is a case-report. I am afraid it will not delight our chiropractic friends who tend to deny that their main therapy can cause serious problems.

In this paper, US doctors tell the story of a young woman who developed headache, vomiting, diplopia, dizziness, and ataxia following a neck manipulation by her chiropractor. A computed tomography scan of the head was ordered and it revealed an infarct in the inferior half of the left cerebellar hemisphere and compression of the fourth ventricle causing moderately severe, acute obstructive hydrocephalus. Magnetic resonance angiography showed severe narrowing and low flow in the intracranial segment of the left distal vertebral artery. The patient was treated with mannitol and a ventriculostomy. Following these interventions, she made an excellent functional recovery.

The authors of the case-report draw the following conclusions: This report illustrates the potential hazards associated with neck trauma, including chiropractic manipulation. The vertebral arteries are at risk for aneurysm formation and/or dissection, which can cause acute stroke.

I can already hear the counter-arguments: this is not evidence, it’s an anecdote; the evidence from the Cassidy study shows there is no such risk!

Indeed the Cassidy study concluded that vertebral artery accident (VBA) stroke is a very rare event in the population. The increased risks of VBA stroke associated with chiropractic and primary care physician visits is likely due to patients with headache and neck pain from VBA dissection seeking care before their stroke. We found no evidence of excess risk of VBA stroke associated chiropractic care compared to primary care. That, of course, was what chiropractors longed to hear (and it is the main basis for their denial of risk) – so much so that Cassidy et al published the same results a second time (most experts feel that this is a violation of publication ethics).

But repeating arguments does not make them more true. What we should not forget is that the Cassidy study was but one of several case-control studies investigating this subject. And the totality of all such studies does not deny an association between neck manipulation and stroke.

Much more important is the fact that a re-analysis of the Cassidy data found that prior studies grossly misclassified cases of cervical dissection and mistakenly dismissed a causal association with manipulation. The authors of this new paper found a classification error of cases by Cassidy et al and they re-analysed the Cassidy data, which reported no association between spinal manipulation and cervical artery dissection (odds ratio [OR] 5 1.12, 95% CI .77-1.63). These re-calculated results reveal an odds ratio of 2.15 (95% CI.98-4.69). For patients less than 45 years of age, the OR was 6.91 (95% CI 2.59-13.74). The authors of the re-analysis conclude as follows: If our estimates of case misclassification are applicable outside the VA population, ORs for the association between SMT exposure and CAD are likely to be higher than those reported using the Rothwell/Cassidy strategy, particularly among younger populations. Future epidemiologic studies of this association should prioritize the accurate classification of cases and SMT exposure.
I think they are correct; but my conclusion of all this would be more pragmatic and much simpler: UNTIL WE HAVE CONVINCING EVIDENCE TO THE CONTRARY, WE HAVE TO ASSUME THAT CHIROPRACTIC NECK MANIPULATION CAN CAUSE A STROKE.

Moxibustion is an ancient variation of acupuncture using  moxa made from dried mugwort (Artemisia argyi). It has long played an important role in the traditional heath care systems of China and other Asian countries. More recently, it has become popular also in the West. Practitioners use moxa sticks indirectly to warm acupuncture needles, or burn it close to the patient’s skin. Essentially, moxibustion is a treatment where acupuncture points are stimulated mainly or exclusively by the heat of burning moxa.

Because of moxibustion’s long history of usage and the fact that it is employed in many countries for a very wide range of conditions, some might argue that it has stood the ‘test of time’ and should be considered to be a well-established therapy. More critical thinkers would, however, point out that this is not an argument but a classical fallacy.

My team at Exeter regularly had research fellows from Korea and other Asian countries, and we managed to develop a truly productive cooperation. It enabled us to conduct systematic reviews including the Asian literature – and this is how we got involved in an unusual amount of research into moxibustion which, after all, is a fairly exotic alternative therapy. In 2010, we began a series of systematic reviews of moxibustion.

One of the first such articles included 9 RCTs testing the effectiveness of this treatment for stroke rehabilitation. Three RCTs reported favorable effects of moxibustion plus standard care on motor function versus standard care alone Three randomized clinical trials compared the effects of moxibustion on activities of daily living alone but failed to show favorable effects of moxibustion.

Also in 2010, our systematic review of RCTs of moxibustion as a treatment of ulcerative colitis (UC) concluded that current evidence is insufficient to show that moxibustion is an effective treatment of UC. Most of included trials had high risk of bias. More rigorous studies seem warranted.

Our (2010) systematic review od RCTs of moxibustion as a therapy in cancer care found that the evidence was limited to suggest moxibustion is an effective supportive cancer care in nausea and vomiting. However, all studies had a high risk of bias so effectively there was not enough evidence to draw any conclusion.

Our (2010) systematic review of RCTs of moxibustion for treating hypertension concluded that there was insufficient evidence to suggest that moxibustion is an effective treatment for hypertension.

Our (2010) systematic review of RCTs of moxibustion for constipation concluded as follows: Given that the methodological quality of all RCTs was poor, the results from the present review are insufficient to suggest that moxibustion is an effective treatment for constipation. More rigorous studies are warranted.

Our (2010) systematic review found few RCTs were available that test the effectiveness of moxibustion in the management of pain, and most of the existing trials had a high risk of bias. Therefore, more rigorous studies are required before the effectiveness of moxibustion for the treatment of pain can be determined.

Our (2011) systematic review of 14 RCTs of moxibustion for rheumatic conditions failed to provide conclusive evidence for the effectiveness of moxibustion compared with drug therapy in rheumatic conditions.

The, so far, last article in this series has only just been published. The purpose of this systematic review was to assess the efficacy of moxibustion as a treatment of chemotherapy-induced leukopenia. Twelve databases were searched from their inception through June 2014, without a language restriction. Randomized clinical trials (RCTs) were included, if moxibustion was used as the sole treatment or as a part of a combination therapy with conventional drugs for leukopenia induced by chemotherapy. Cochrane criteria were used to assess the risk of bias.

Six RCTs with a total of 681 patients met our inclusion criteria. All of the included RCTs were associated with a high risk of bias. The trials included patients with various types of cancer receiving ongoing chemotherapy or after chemotherapy. The results of two RCTs suggested the effectiveness of moxibustion combined with chemotherapy vs. chemotherapy alone. In four RCTs, moxibustion was more effective than conventional drug therapy. Six RCTs showed that moxibustion was more effective than various types of control interventions in increasing white blood cell counts.

Our conclusion: there is low level of evidence based on these six trials that demonstrates the superiority of moxibustion over drug therapies in the treatment of chemotherapy-induced leukopenia. However, the number of trials, the total sample size, and the methodological quality are too low to draw firm conclusions. Future RCTs appear to be warranted.

Was all this research for nothing?

I know many people who would think so. However, I disagree. If nothing else, these articles demonstrated several facts quite clearly:

  • There is quite a bit of research even on the most exotic alternative therapy; sometimes one needs to look hard and include languages other than English.
  • Studies from China and other Asian counties very rarely report negative results; this fact casts a dark shadow on the credibility of such data.
  • The poor quality of trials in most areas of alternative medicine is lamentable and must be stimulus for researchers in this field to improve their act.
  • Authors of systematic reviews must resist the temptation to draw positive conclusions based on flawed primary data.
  • Moxibustion is a perfect example for demonstrating that the ‘test of time’ is no substitute for evidence.
  • As for moxibustion, it cannot currently be considered an evidence-based treatment for any condition.

As promised, I will try with this post to explain my reservations regarding the new meta-analysis suggesting that individualised homeopathic remedies are superior to placebos. Before I start, however, I want to thank all those who have commented on various issues; it is well worth reading the numerous and diverse comments.

To remind us of the actual meta-analysis, it might be useful to re-publish its abstract (the full article is also available online):

BACKGROUND:

A rigorous and focused systematic review and meta-analysis of randomised controlled trials (RCTs) of individualised homeopathic treatment has not previously been undertaken. We tested the hypothesis that the outcome of an individualised homeopathic treatment approach using homeopathic medicines is distinguishable from that of placebos.

METHODS:

The review’s methods, including literature search strategy, data extraction, assessment of risk of bias and statistical analysis, were strictly protocol-based. Judgment in seven assessment domains enabled a trial’s risk of bias to be designated as low, unclear or high. A trial was judged to comprise ‘reliable evidence’ if its risk of bias was low or was unclear in one specified domain. ‘Effect size’ was reported as odds ratio (OR), with arithmetic transformation for continuous data carried out as required; OR > 1 signified an effect favouring homeopathy.

RESULTS:

Thirty-two eligible RCTs studied 24 different medical conditions in total. Twelve trials were classed ‘uncertain risk of bias’, three of which displayed relatively minor uncertainty and were designated reliable evidence; 20 trials were classed ‘high risk of bias’. Twenty-two trials had extractable data and were subjected to meta-analysis; OR = 1.53 (95% confidence interval (CI) 1.22 to 1.91). For the three trials with reliable evidence, sensitivity analysis revealed OR = 1.98 (95% CI 1.16 to 3.38).

CONCLUSIONS:

Medicines prescribed in individualised homeopathy may have small, specific treatment effects. Findings are consistent with sub-group data available in a previous ‘global’ systematic review. The low or unclear overall quality of the evidence prompts caution in interpreting the findings. New high-quality RCT research is necessary to enable more decisive interpretation.

Since my team had published an RCTs of individualised homeopathy, it seems only natural that my interest focussed on why the study (even though identified by Mathie et al) had not been included in the meta-analysis. Our study had provided no evidence that adjunctive homeopathic remedies, as prescribed by experienced homeopathic practitioners, are superior to placebo in improving the quality of life of children with mild to moderate asthma in addition to conventional treatment in primary care.

I was convinced that this trial had been rigorous and thus puzzled why, despite receiving ‘full marks’ from the reviewers, they had not included it in their meta-analysis. I thus wrote to Mathie, the lead author of the meta-analysis, and he explained: For your trial (White et al. 2003), under domain V of assessment, we were unable to extract data for meta-analysis, and so it was attributed high risk of bias, as specified by the Cochrane judgmental criteria. Our designated main outcome was the CAQ, for which we needed to know (or could at least estimate) a mean and SD for both the baseline and the end-point of the study. Since your paper reported only the change from baseline in Table 3 or in the main text, it is not possible to derive the necessary end-point for analysis.

It took a while and several further emails until I understood: our study did report both the primary (Table 2 quality of life) and secondary outcome measure (Table 3 severity of symptoms). The primary outcome measure was reported in full detail such that a meta-analysis would have been possible. The secondary outcome measure was also reported but not in full detail, and the data provided by us would not lend themselves to meta-analyses. By electing not our primary but our secondary outcome measure for their meta-analysis, Mathie et al were able to claim that they were unable to use our study and reject it for their meta-analysis.

Why did they do that?

The answer is simple: in their methods section, they specify that they used outcome measures “based on a pre-specified hierarchical list in order of greatest to least importance, recommended by the WHO“. This, I would argue is deeply flawed: the most important outcome measure of a study is usually the one for which the study was designed, not the one that some guys at the WHO feel might be important (incidentally, the WHO list was never meant to be applied to meta-analyses in that way).

By following rigidly their published protocol, the authors of the meta-analysis managed to exclude our negative trial. Thus they did everything right – or did they?

Well, I think they committed several serious mistakes.

  • Firstly, they wrote the protocol, which forced them to exclude our study. Following a protocol is not a virtue in itself; if the protocol is nonsensical it even is the opposite. Had they proceeded as is normal in such cases and used our primary outcome measure in their meta-analyses, it is most likely that their overall results would not have been in favour of homeopathy.
  • Secondly, they awarded our study a malus point for the criterium ‘selective outcome reporting’. This is clearly a wrong decision: we did report the severity-outcome, albeit not in sufficient detail for their meta-analysis. Had they not committed this misjudgment, our RCT would have been the only one with an ‘A’ rating. This would have very clearly highlighted the nonsense of excluding the best-rated trial from meta-analysis.

There are several other oddities as well. For instance, Mathie et al judge our study to be NOT free of vested interest. I asked Mathie why they had done this and was told it is because we accepted free trial medication from a homeopathic pharmacy. I would argue that my team was far less plagued by vested interest than the authors of their three best (and of course positive) trials who, as I happen to know, are consultants for homeopathic manufacturers.

And all of this is just in relation to our own study. Norbert Aust has uncovered similar irregularities with other trials and I take the liberty of quoting his comments posted previously again here:

I have reason to believe that this review and metaanalysis in biased in favor of homeopathy. To check this, I compared two studies (1) Jacobs 1994 about the treatment of childhood diarrhea in Nicaragua, (2) Walach 1997 about homeopathic threatment of headaches. The Jacobs study is one of the three that provided ‘reliable evidence’, Walach’s study earned a poor C2.2 rating and was not included in the meta-analyses. Jacobs’ results were in favour of homeopathy, Walach’s not.

For the domains where the rating of Walach’s study was less than that of the Jacobs study, please find citations from the original studies or my short summaries for the point in question.

Domain I: Sequence generation:
Walach:
“The remedy selected was then mailed to a notary public who held a stock of placebos. The notary threw a dice and mailed either the homeopathic remedy or an appropriate placebo. The notary was provided with a blank randomisation list.”
Rating: UNCLEAR (Medium risk of bias)

Jacobs:
“For each of these medications, there was a box of tubes in sequentially numbered order which had been previously randomized into treatment or control medication using a random numbers table in blocks of four”
Rating: YES (Low risk of bias)

Domain IIIb: Blinding of outcome assessor
Walach:
“The notary was provided with a blank randomization list which was an absolutely unique document. It was only handed out after the biometrician (WG) had deposited all coded original data as a printout at the notary’s office. (…) Data entry was performed blindly by personnel not involved in the study. ”
Rating: UNCLEAR (Medium risk of bias)

Jacobs:
“All statistical analyses were done before breaking the randomisation code, using the program …”
Rating: YES (Low risk of bias)

Domain V: Selective outcome reporting

Walach:
Study protocol was published in 1991 prior to enrollment of participants, all primary outcome variables were reported with respect to all participants and the endpoints.
Rating: NO (high risk of bias)

Jacobs:
No prior publication of protocol, but a pilot study exists. However this was published in 1993 only after the trial was performed in 1991. Primary outcome defined (duration of diarrhea), reported but table and graph do not match, secondary outcome (number of unformed stools on day 3) seems defined post hoc, for this is the only one point in time, this outcome yielded a significant result.
Rating: YES (low risk of bias)

Domain VI: Other sources of bias:

Walach:
Rating: NO (high risk of bias), no details given

Jacobs:
Imbalance of group properties (size, weight and age of children), that might have some impact on course of disease, high impact of parallel therapy (rehydration) by far exceeding effect size of homeopathic treatment
Rating: YES (low risk of bias), no details given

In a nutshell: I fail to see the basis for the different ratings in the studies themselves. I assume bias of the authors of the review.

Conclusion

So, what about the question posed in the title of this article? The meta-analysis is clearly not a ‘proof of concept’. But is it proof for misconduct? I asked Mathie and he answered as follows: No, your statement does not reflect the situation at all. As for each and every paper, we selected the main outcome measure for your trial using the objective WHO classification approach (in which quality of life is clearly of lower rank than severity). This is all clearly described in our prospective protocol. Under no circumstances did we approach this matter retrospectively, in the way you are implying. 

Some nasty sceptics might have assumed that the handful of rigorous studies with negative results were well-known to most researchers of homeopathy. In this situation, it would have been hugely tempting to write the protocol such that these studies must be excluded. I am thrilled to be told that the authors of the current new meta-analysis (who declared all sorts of vested interests at the end of the article) resisted this temptation.

1 2 3 12
Recent Comments
Click here for a comprehensive list of recent comments.
Categories