MD, PhD, FMedSci, FSB, FRCP, FRCPEd

scientific misconduct

A remarkable article about homeopathy and immunisation entitled THE IMMUNISATION DILEMMA came to my attention recently. Its abstract promised: “evidence quantifying the effectiveness of vaccination and HP (homeoprophylaxis) will be examined. New international research describing and analysing HP interventions will be reported. An evidence-based conclusion will be reached.”

Sounds interesting? Let’s see what the article really offers. Here is the relevant text:

…evidence does exist to support claims regarding the effectiveness of homeopathic immunisation is undeniable.

I was first invited to visit Cuba in December 2008 to present at an international conference hosted by the Finlay Institute, which is a W. H. O.-accredited vaccine manufacturer. The Cubans described their use of HP to control an outbreak of leptospirosis (Weilʼs syndrome – a potentially fatal, water-born bacterial disease) in 2007 among the residents of the three eastern provinces which were most severely damaged by a severe hurricane – over 2.2 million people [7]. 2008 was an even worse year involving three hurricanes, and the countryʼs food production was only just recovering at the time of the conference. The HP program had been repeated in 2008, but data was not available at the conference regarding that intervention.

I revisited Cuba in 2010 and 2012, each time to work with the leader of the HP interventions, Dr. Bracho, to analyse the data available. Dr. Bracho is not a homeopath; he is a published and internationally recognised expert in the manufacture of vaccine adjuvants. He worked in Australia at Flinders University during 2004 with a team trying to develop an antimalarial vaccine.

In 2012 we accessed the raw leptospirosis surveillance data, comprising weekly reports from 15 provinces over 9 years (2000 to 2008) reporting 21 variables. This yielded a matrix with 147 420 possible entries. This included data concerning possible confounders, such as vaccination and chemoprophylaxis, which allowed a careful examination of possible distorting effects. With the permission of the Cubans, I brought this data back to Australia and it is being examined by mathematicians at an Australian university to see what other information can be extracted. Clearly, there is objective data supporting claims regarding the effectiveness of HP.

The 2008 result was remarkable, and could only be explained by the effectiveness of the HP intervention. Whilst the three hurricanes caused immense damage throughout the country it was again worse in the east, yet the three homeopathically immunised provinces experienced a negligible increase in cases whilst the rest of the country showed significant increases until the dry season in January 2009 [8].

This is but one example – there are many more. It is cited to show that there is significant data available, and that orthodox scientists and doctors have driven the HP interventions, in the Cuban case. Many people internationally now know this, so once again claims by orthodox authorities that there is no evidence merely serves to show that either the authorities are making uninformed/unscientific statements, or that they are aware but are intentionally withholding information. Either way, confidence is destroyed and leads to groups of people questioning what they are told…

Final Conclusions

The attacks against homeopathy in general and HP in particular will almost certainly continue. If we can achieve a significant level of agreement then we would be able to answer challenges to HP with a single, cohesive, evidence-based, and generally united response. This would be a significant improvement to the existing situation.

 

Reference 7 is the following article: Bracho G, Varela E, Fernández R et al. Large-scale application of highly-diluted bacteria for Leptospirosis epidemic control. Homeopathy 2010; 99: 156-166. The crucial bit if this paper are as follows:

A homeoprophylactic formulation was prepared from dilutions of four circulating strains of Leptospirosis. This formulation was administered orally to 2.3 million persons at high risk in an epidemic in a region affected by natural disasters. The data from surveillance were used to measure the impact of the intervention by comparing with historical trends and non-intervention regions.

After the homeoprophylactic intervention a significant decrease of the disease incidence was observed in the intervention regions. No such modifications were observed in non-intervention regions. In the intervention region the incidence of Leptospirosis fell below the historic median. This observation was independent of rainfall.

The homeoprophylactic approach was associated with a large reduction of disease incidence and control of the epidemic. The results suggest the use of HP as a feasible tool for epidemic control, further research is warranted.

The paper thus describes little more than an observational study. It shows that one region was less affected than another. I think it is quite clear that this could have many reasons which are unrelated to the homeopathic immunisation. Even the authors are cautious and speak in their conclusions not of a causal effect but of an “association”.

The 2012 data cited in the text remains unpublished; until it is available for public scrutiny, it is impossible to confirm that it is sound and meaningful.

Reference 8 refers to this article: Golden I, Bracho G. Adaptability of homœoprophylaxis in endemic, epidemic and stable background conditions. Homœopathic Links 2009; 22: 211-213. I have no access to this paper (if someone does, please fill us in) but, judging from both its title and the way it is described in the text, it does not seem to show reliable data about the efficacy of homeopathic immunisation.

So, is it true that “evidence does exist to support claims regarding the effectiveness of homeopathic immunisation”?

I do not think so!

Immunisation is by no means a trivial matter; wrong decisions in this area have the potential to cost the lives of millions. Therefore proofs of efficacy need to be published in peer-reviewed journals of high standing. These findings need then be criticised, replicated and re-criticised and re-replicated. Only when there is a wide consensus about the efficacy/safety or lack of efficacy/safety of a new form of immunisation, can it be generally accepted and implemented into clinical practice.

The current consensus about homeopathic immunisation is that it is nothing less than dangerous phantasy. Those who promote this quackery should be publicly exposed as charlatans of the worst kind.

Some practitioners of alternative medicine (doctors, naturopaths, chiropractors and others) earn a lot of money with the claim that chelation therapy (an effective mainstream treatment for acute heavy metal poisoning) is an effective means to  treat cardiovascular disease. However, the notion is controversial and implausible. Several systematic reviews of the best evidence concluded less than optimistically:

…more controlled studies are required to determine the efficacy of chelation therapy in cardiovascular disease before it can be used broadly in the clinical setting.

The best available evidence does not support the therapeutic use of EDTA chelation therapy in the treatment of cardiovascular disease.

Given the potential of chelation therapy to cause severe adverse effects, this treatment should now be considered obsolete.

The available data do not support the use of chelation in cardiovascular diseases.

More recently, important new evidence has emerged. The largest study of chelation therapy (TACT) ever conducted cost ~ $ 30 million and concluded that among stable patients with a history of MI, use of an intravenous chelation regimen with disodium EDTA, compared with placebo, modestly reduced the risk of adverse cardiovascular outcomes, many of which were revascularization procedures. These results provide evidence to guide further research but are not sufficient to support the routine use of chelation therapy for treatment of patients who have had an MI.

At the time, the TACT trial was heavily and rightly criticised for a whole host of reasons. For instance, because of the result of the FDA inspection of the highest accruing TACT site:

  • The investigators didn’t conduct the investigation in accordance with the signed statement and investigational plan. Several examples were given of shoddy procedures, prefilled forms, and failure to train personnel.
  • Failure to report promptly to the IRB all unanticipated problems involving risk to human subjects or others. Examples are given, including failure to report the deaths of patients on the study in a timely fashion (in one case the death wasn’t reported to the IRB until four months later; in another case it was never reported at all). In other cases, adverse event reports were not submitted to the IRB.
  • Failure to prepare or maintain adequate case histories with respect to observations and data pertinent to the investigation.
  • Investigational drug disposition records are not adequate with respect to dates, quantity, and use by subjects.

Despite these problems, the study was published in JAMA, albeit with a very critical editorial:

Differential dropout in TACT suggests unmasking, but the problem of intentional unblinding is more concerning. The sponsors of the trial, the National Heart, Lung, and Blood Institute (NHLBI) and the National Center for Complementary and Alternative Medicine (NCCAM), were unblinded throughout the trial. The National Institutes of Health policy unwisely allows the sponsor access to unblinded trial data, and both organizations sent observers to the closed sessions of the data monitoring committee. This gave them access to confidential data during each of the 11 interim analyses. The unblinding of the study sponsor represents a serious deviation from acceptable standards of conduct for supervision of clinical trials. If a pharmaceutical company sponsoring a trial were allowed access to actual outcome data during the study, there would be major objections. Like any sponsor, the NHLBI and NCCAM cannot be considered unbiased observers. These agencies made major financial commitments to the trial and may intentionally or inadvertently influence study conduct if inappropriately unblinded during the study…

Given the numerous concerns with this expensive, federally funded clinical trial, including missing data, potential investigator or patient unmasking, use of subjective end points, and intentional unblinding of the sponsor, the results cannot be accepted as reliable and do not demonstrate a benefit of chelation therapy. The findings of TACT should not be used as a justification for increased use of this controversial therapy.

Orac, makes several further critical points about the published trial:

First, the primary endpoint (i.e., the aggregated serious cardiovascular events) did indeed show a modest difference, namely 30% of placebo subjects versus 26.5% of the EDTA chelation subjects (hazard ratio 0.82 for chelation). However, one notes that the result is just barely statistically significant, p = 0.035, with the 99% confidence interval for the hazard ratio ranging from 0.69 to 0.99. (The predetermined level for statistical significance for purposes of this study was 0.036; so this is statistically significant by the barest margin.) More importantly, if you look at the individual endpoints that make up that aggregate, there was no statistically significant difference in death, myocardial infarction, stroke, coronary revascularization, and hospitalization for angina. Subgroup analysis (always a questionable analysis that requires replication, even when preplanned, as in TACT) purported to show a much greater benefit for diabetics, with a hazard ratio of 0.61 (p=0.002), while patients without diabetes showed no statistically significant difference in any of the outcome measures, including the aggregated total bad outcomes.

Now a paper that has just emerged describes the intent-to-treat comparison of this trial in patients with diabetes.

This was a double-blind, placebo-controlled, 2 × 2 factorial multicenter randomized trial of 1,708 post-myocardial infarction (MI) patients ≥50 years of age and with creatinine ≤2.0 mg/dL randomized to receive 40 EDTA chelation or placebo infusions plus 6 caplets daily of a 28-component multivitamin-multimineral mixture or placebo. The primary end point was a composite of total mortality, MI, stroke, coronary revascularization, or hospitalization for angina.

Median age was 65 years, 18% were female, 94% were Caucasian, 37% were diabetic, 83% had prior coronary revascularization, and 73% were on statins. Five-year Kaplan-Meier estimates for the primary end point was 31.9% in the chelation + high-dose vitamin group, 33.7% in the chelation + placebo vitamin group, 36.6% in the placebo infusion + active vitamin group, and 40.2% in the placebo infusions + placebo vitamin group. The reduction in primary end point by double active treatment compared with double placebo was significant (hazard ratio 0.74, 95% CI 0.57-0.95, P = .016). In patients with diabetes, the primary end point reduction of double active compared with double placebo was more pronounced (hazard ratio 0.49, 95% CI 0.33-0.75, P < .001).

The authors conclude that in stable post-MI patients on evidence-based medical therapy, the combination of oral high-dose vitamins and chelation therapy compared with double placebo reduced clinically important cardiovascular events to an extent that was both statistically significant and of potential clinical relevance.

I fear that these conclusions are erroneous and misleading: the marginally positive finding might have nothing to do with chelation per se; most likely they are due to the fact that the ‘vitamin’ mixture administered along with chelation contained ingredients like heparin and procaine which are potentially beneficial for cardiovascular conditions. Moreover, the placebo contained a considerable amount of glucose which could easily explain the better outcome of the diabetic subgroup receiving the verum – in other words, the verum generated better results not because it was effective but because the ‘placebo’ had detrimental effects.

Yesterday, BBC NEWS published the following interesting text about a BBC4 broadcast entitled ‘THE ROYAL ACTIVIST’ aired on the same day:

Prince Charles has been a well-known supporter of complementary medicine. According to a… former Labour cabinet minister, Peter Hain, it was a topic they shared an interest in.

“He had been constantly frustrated at his inability to persuade any health ministers anywhere that that was a good idea, and so he, as he once described it to me, found me unique from this point of view, in being somebody that actually agreed with him on this, and might want to deliver it.”
Mr Hain added: “When I was Secretary of State for Northern Ireland in 2005-7, he was delighted when I told him that since I was running the place I could more or less do what I wanted to do.***
“I was able to introduce a trial for complementary medicine on the NHS, and it had spectacularly good results, that people’s well-being and health was vastly improved.

“And when he learnt about this he was really enthusiastic and tried to persuade the Welsh government to do the same thing and the government in Whitehall to do the same thing for England, but not successfully,” added Mr Hain.

*** obviously there is no homeopathic remedy for megalomania (but that’s a different story)

Oh really?

A TRIAL?

SPECTACULARLY GOOD RESULTS?

NO KIDDING?

Let’s have a look at the ‘trial’ and its results. An easily accessible report provides the following details about it:

From February 2007 to February 2008, Get Well UK ran the UK’s first government-backed complementary therapy pilot. Sixteen practitioners provided treatments including acupuncture, osteopathy and aromatherapy, to more than 700 patients at two GP practices in Belfast and Derry.   

The BBC made an hour long documentary following our trials and tribulations, which was broadcast on BBC1 NI on 5 May 2008.

Following the successful completion of the pilot, the results were analysed by Social and Market Research and recommendations were made to the Health Minister

Aims and Objectives 

The aim of the project was to pilot services integrating complementary medicine into existing primary care services in Northern Ireland. Get Well UK provided this pilot project for the Department for Health, Social Services and Public Safety (DHSSPS) during 2007.

The objectives were:

  • To measure the health outcomes of the service and monitor health improvements.
  • To redress inequalities in access to complementary medicine by providing therapies through the NHS, allowing access regardless of income.
  • To contribute to best practise in the field of delivering complementary therapies through primary care.
  • To provide work for suitably skilled and qualified practitioners.
  • To increase patient satisfaction with quick access to expert care.
  • To help patients learn skills to improve and retain their health.
  • To free up GP time to work with other patients.
  • To deliver the programme for 700 patients.

Results 

The results of the pilot were analysed by Social and Market Research, who produced this report.

The findings can be summarised as follows: 

Following the pilot, 80% of patients reported an improvement in their symptoms, 64% took less time off work and 55% reduced their use of painkillers.

In the pilot, 713 patients with a range of ages and demographic backgrounds and either physical or mental health conditions were referred to various complementary and alternative medicine (CAM) therapies via nine GP practices in Belfast and Londonderry. Patients assessed their own health and wellbeing pre and post therapy and GPs and CAM practitioners also rated patients’ responses to treatment and the overall effectiveness of the scheme.

Health improvement
• 81% of patients reported an improvement in their physical health
• 79% reported an improvement in their mental health
• 84% of patients linked an improvement in their health and wellbeing directly to their CAM treatment
• In 65% of patient cases, GPs documented a health improvement, correlating closely to patient-reported improvements
• 94% of patients said they would recommend CAM to another patient with their condition
• 87% of patient indicated a desire to continue with their CAM treatment

Painkillers and medication
• Half of GPs reported prescribing less medication and all reported that patients had indicated to them that they needed less
• 62% of patients reported suffering from less pain
• 55% reported using less painkillers following treatment
• Patients using medication reduced from 75% before treatment to 61% after treatment
• 44% of those taking medication before treatment had reduced their use afterwards

Health service and social benefits
• 24% of patients who used health services prior to treatment (i.e. primary and secondary care, accident and emergency) reported using the services less after treatment
• 65% of GPs reported seeing the patient less following the CAM referral
• Half of GPs said the scheme had reduced their workload and 17% reported a financial saving for their practice
• Half of GPs said their patients were using secondary care services less.

Impressed? Well, in case you are, please consider this:

  • there was no control group
  • therefore it is not possible to attribute any of the outcomes to the alternative therapies offered
  • they could have been due to placebo-effects
  • or to the natural history of the disease
  • or to regression towards the mean
  • or to social desirability
  • or to many other factors which are unrelated to the alternative treatments provided
  • most outcome measures were not objectively verified
  • the patients were self-selected
  • they would all have had conventional treatments in parallel
  • this ‘trial’ was of such poor quality that its findings were never published in a peer-reviewed journal
  • this was not a ‘trial’ but a ‘pilot study’
  • pilot studies are not normally for measuring outcomes but for testing the feasibility of a proper trial
  • the research expertise of the investigators was close to zero
  • the scientific community merely had pitiful smiles for this ‘trial’ when it was published
  • neither Northern Ireland nor any other region implemented the programme despite its “spectacularly good results”.

So, is the whole ‘trial’ story an utterly irrelevant old hat?

Certainly not! Its true significance does not lie in the fact that a few amateurs are trying to push bogus treatments into the NHS via the flimsiest pseudo-research of the century. The true significance, I think, is that it shows how Prince Charles, once again, oversteps the boundaries of his constitutional role.

Some time ago, I published a post entitled HOW TO BECOME A CHARLATAN. This prompted ‘THE NORWEGIAN ACADEMY OF SCIENCE AND LETTERS’ to invite me to give a lecture on the subject, a great honour, I am sure. Consequently, I have thought about this somewhat unusual subject quite a lot.

Obviously, my thoughts come from the perspective of someone who has researched alternative medicine for many years. Pseudoscientists seem to love alternative medicine and proponents of alternative medicine love pseudoscience. As a result, alternative medicine is densely populated by pseudoscientists.

But what is the characteristic of pseudoscience? Reflecting on this question, I found not one but several hallmarks (and for each of them, there are many posts on this blog which provide further explanations):

Based on these 12 hallmarks, one could create a simple score which indicates the likelihood of the presence of pseudoscience. In other words, it might be useful to consider pseudoscience in terms of a sliding scale. Some things in alternative medicine can be just a bit pseudoscientific, others quite a lot, while others again are hopelessly so.

The issue of pseudoscience is by no means just academic; it is very real problem and has many important, practical implications. The most important one probably is that, in health care (and other areas as well), pseudoscience can be harmful, even to the point that it costs lives of vulnerable patients who believe that everything masquerading as science can be relied upon.

Dr. Oz, famous through his TV show promoting all types of quackery, recently testified before a US Senate subcommittee hearing on protecting consumers from false and deceptive advertising of weight loss products. This event turned out to be less than flattering for Dr Oz. One journalist commented that he “might as well be a cowardly lion — sent home with his tail between his legs after being accused at a congressional hearing of lying on his show about weight-loss claims.”

“I don’t get why you need to say this stuff, because you know it’s not true,” said Senator Claire McCaskill, who led the commerce subcommittee hearing. “The scientific community is almost monolithically against you in terms of the efficacy of the products you called ‘miracles,’ ” the Democratic senator from Missouri told Oz. “It’s a major problem when people are spending more and more money and they’re gaining more and more weight,” said Senator Amy Klobuchar.“Either you don’t talk about these things at all, or you’re going to have to be more specific because right now . . . this is not working.”

A source close to Dr Oz said he was perplexed: “We were invited down to Washington to testify at a hearing about scams and instead it became all about how much we hate your show.” Oz himself testified that he “heard the message…I do personally believe in the items that I talk about.”

“I intensively study them. I have given my family these products. . . . If you can lose a pound a week more than you would have lost by using them, it jump-starts you and gets you going. I think it makes sense.” “I’m surprised you’re defending this,” McCaskill replied. “It’s something that gives people false hope. I don’t see why you need to go there.”

Another journalist commented that the Senators repeatedly placed him on the defense over his weight loss products: “I know you know how much power you have. I know you know that. You are very powerful and [with] power comes a great deal of responsibility,” Senator Claire McCaskill , who led the Senate’s consumer protection hearing titled “Protecting Consumers from False and Deceptive Advertising of Weight-Loss Products…You are being made an example of today because of the power you have in this space…We didn’t call this hearing to beat up on you but we did call this hearing to talk about a real crisis in consumer protection. You can either be part of the police here or you can be part of the problem.”

Oz insisted he was no huckster but admitted the products promoted on his show don’t always have “the scientific muster” to present their benefits as “fact…I actually do personally believe in the items that I talk about in the show. I passionately studied them. I recognize that oftentimes they don’t have the scientific muster to present as fact but nevertheless I would give my audience the advice I give my family all the time. And I have given my family these products,” he said.

Dr Oz also said that some alternative treatments, such as prayer, cannot be tested scientifically. “I don’t think this ought to be a referendum on the use of alternative medical therapies. Because if that’s the case, listen, I’ve been criticized for having folks coming on my show talking about the power of prayer,” he said. “I can’t prove that prayer helps people survive an illness.”

No, Dr Oz! I know you are mistaken! I have done the research – both on alternative slimming aids and on spiritual healing. The results quite clearly show that these methods are not more effective than a placebo.

If we search on ‘Medline’ for ‘complementary alternative medicine’ (CAM), we currently get about 13000 hits. A little graph on the side of the page demonstrates that, during the last 4 years, the number of articles on this subject has grown exponentially.

Surely, this must be very good news: such intense research activity will soon tell us exactly which alternative treatments work for which conditions and which don’t.

I beg to differ. Let me explain why.

The same ‘Medline’ search informs us that the majority of the recent articles were published in an open access journal called ‘Evidence-Based Complementary and Alternative Medicine’ (eCAM). For example, of the 80 most recent articles listed in Medline (on 26/5/2014), 53 came from that journal. The publication frequency of eCAM and its increase in recent years beggars belief: in 2011, they published just over 500 articles which is already a high number, but, in 2012, the figure had risen to >800, and in 2013 it was >1300 (the equivalent 2013 figure for the BMJ/BMJ Open by comparison is 4, and that for another alt med journal, e.g. Forsch Komplement, is 10)

How do they do it? How can eCAM be so dominant in publishing alt med research? The trick seems to be fairly simple.

Let’s assume you are an alt med researcher and you have an article that you would like to see published. Once you submit it to eCAM, your paper is sent to one of the ~150 members of the editorial board. These people are almost all strong proponents of alternative medicine; critics are a true rarity in this group. At this stage, you are able to suggest the peer reviewers for your submission (all who ever accepted this task are listed on the website; they amount to several thousand!), and it seems that, with the vast majority of submissions, the authors’ suggestions are being followed.

It goes without saying that most researchers suggest colleagues for peer reviewing who are not going to reject their work (the motto seems to be “if you pass my paper, I will pass yours). Therefore even faily flimsy bits of research pass this peer review process and get quickly published online in eCAM.

This process explains a lot, I think: 1) the extraordinarily high number of articles published 2) why currently more than 50% of all alt med research originate from eCAM 3) why so much of it is utter rubbish.

Even the mere titles of some of the articles might demonstrate my point. A few examples have to suffice:

  • Color distribution differences in the tongue in sleep disorder
  • Wen-dan decoction improves negative emotions in sleep-deprived rats by regulating orexin-a and leptin expression.
  • Yiqi Huoxue Recipe Improves Heart Function through Inhibiting Apoptosis Related to Endoplasmic Reticulum Stress in Myocardial Infarction Model of Rats.
  • Protective Effects of Bu-Shen-Huo-Xue Formula against 5/6 Nephrectomy-Induced Chronic Renal Failure in Rats
  • Effects and Mechanisms of Complementary and Alternative Medicine during the Reproductive Process
  • Evidence-based medicinal plants for modern chronic diseases
  • Transforming Pain into Beauty: On Art, Healing, and Care for the Spirit

This system of uncritical peer review and fast online publication seems to suit many of the people involved in this process: the journal’s owners are laughing all the way to the bank; there is a publication charge of US$ 2000 per article, and, in 2013, the income of eCAM must therefore have been well over US$2 000 000. The researchers are equally delighted; they get even their flimsiest papers published (remember: ‘publish or perish’!). And the evangelic believers in alternative medicine are pleased because they can now claim that their field is highly research-active and that there is plenty of evidence to support the use of this or that therapy.

But there are others who are not served well by eCAM habit of publishing irrelevant, low quality articles:

  • professionals who would like to advance health care and want to see reliable evidence as to which treatments work and which don’t,
  • the public who, in one way or another, pay for all this and might assume that published research tends to be relevant and reliable,
  • the patients who have given their time to researchers in the hope that their gift will improve health care,
  • ill individuals who hope that alternative treatments might relieve their suffering,
  • politicians who rely on research to be reliable in order to arrive at the right decisions.

Come to think of it, the vast majority of people should be less than enchanted with eCAM and similar journals.

Auricular acupuncture (AA), according to the ‘COLLEGE OF AURICULAR ACUPUNCTURE’, has its origins in Modern Europe. In 1957 Dr. Paul Nogier, a neurologist from Lyons in France, observed a locum doctor treating sciatica by cauterizing an area of the ear. This prompted extensive research culminating in the development of the somatopic correspondence of specific parts of the body to the ear based upon the concept of an inverted foetus. Dr. Nogier believed that pain and other symptoms in the body could be alleviated by needling, massaging or electronically stimulating the corresponding region of the ear. Auricular Acupuncture is a specialized complementary therapy where acupuncture points on the outer ear are treated, using either needles or acupunctoscopes (electrical location and stimulation machines) to help relieve many chronic complaints. There are over 200 acupuncture points on the ear, each point named after an area of our anatomy. The outer ear acts like a switchboard to the brain. Each acupuncture point being treated, triggers electrical impulses from the ear via the brain, to the specific part of the body being treated.

Sounds odd? Well, that’s because it is odd!

But just because something is odd does not mean it is ineffective – so, what does the reliable evidence tell us? Here are some conclusions from systematic reviews:

The evidence that auricular acupuncture reduces postoperative pain is promising but not compelling.

The evidence for the effectiveness of AA for the symptomatic treatment of insomnia is limited.

The benefit of ear-acupressure for symptomatic relief of allergic rhinitis is unknown…

All of these analyses point out that the quality of the studies is usually very poor, and stress that more and better research is required. It is therefore interesting to note that a new study has just been published. Perhaps it could settle the question about the effectiveness of AA?

The aim of this study was 1) to evaluate whether auricular acupuncture effective for reducing health care provider stress and anxiety and 2) to determine, if auricular acupuncture impacts provider capacity for developing caring relationships with patients. Pre-intervention and post-intervention surveys were evaluated to see, if auricular acupuncture was associated with changes in State-Trait Anxiety Inventory (STAI), Professional Quality of Life, and Caring Ability Inventory scores. The results indicate that, compared with baseline, participants had a significant reduction in state anxiety (STAI), trait anxiety (STAI), burnout, and secondary traumatic stress scores (Professional Quality of Life). Significant increases were noted in courage and patience, two dimensions of the Caring Ability Inventory.

From these findings, the authors conclude that auricular acupuncture is an effective intervention for the relief of stress/anxiety in providers and supports heightened capacity for caring.

Sounds odd again? Yes, because it is odd!

I would argue that a study of any controversial therapy that has already been tested repeatedly in poor quality trials must have sufficient scientific rigor to advance the field of inquiry. If it does not fulfil this criterion, it is quite simply not ethical. The new study does not even have a control group; we can therefore not begin to tell whether the observed outcomes were due to non-specific effects, the natural history of the condition or regression towards the mean (to mention but a few of the possible sources of bias). To conclude that AA is ‘an effective intervention’ is therefore utterly barmy.

All of this could be entirely trivial and inconsequential. I am afraid, however, that it is not. Alternative medicine is littered with such unethically flawed research conducted by naïve and clueless pseudo-scientists who arrive at outrageous conclusions. This relentless flow of false-positive findings misleads consumers, health care professionals, decision makers and everyone else to draw the wrong conclusions about bogus therapies. And, in the end, this sort of thing even does a grave disfavour to any branch of alternative medicine that might have some degree of respectability.

IT IS HIGH TIME THAT THIS NONSENSE STOPS! IT BORDERS ON SCIENTIFIC MISCONDUCT.

The question whether infant colic can be effectively treated with manipulative therapies might seem rather trivial – after all, this is a benign condition which the infant quickly grows out of. However, the issue becomes a little more tricky, if we consider that it was one of the 6 paediatric illnesses which were at the centre of the famous libel case of the BCA against my friend and co-author Simon Singh. At the time, Simon had claimed that there was ‘not a jot of evidence’ for claiming that chiropractic was an effective treatment of infant colic, and my systematic review of the evidence strongly supported his statement. The BCA eventually lost their libel case and with it the reputation of chiropractic. Now a new article on this intriguing topic has become available; do we have to reverse our judgements?

The aim of this new systematic review was to evaluate the efficacy or effectiveness of manipulative therapies for infantile colic. Six RCTs of chiropractic, osteopathy or cranial osteopathy alone or in conjunction with other interventions were included with a total of 325 infants. Of the 6 included studies, 5 were “suggestive of a beneficial effect” and one found no evidence of benefit. Combining all the RCTs suggested that manipulative therapies had a significant effect. The average crying time was reduced by an average of 72 minutes per day. This effect was sustained for studies with a low risk of selection bias and attrition bias. When analysing only those studies with a low risk of performance bias (i.e. parental blinding) the improvement in daily crying hours was no longer statistically significant.

The quality of the studies was variable. There was a generally low risk of selection bias but a high risk of performance bias. Only one of the studies recorded adverse events and none were encountered.

From these data, the authors drew the following conclusion: Parents of infants receiving manipulative therapies reported fewer hours crying per day than parents whose infants did not and this difference was statistically significant. Most studies had a high risk of performance bias due to the fact that the assessors (parents) were not blind to who had received the intervention. When combining only those trials with a low risk of such performance bias the results did not reach statistical significance.

Does that mean that chiropractic does work for infant colic? No, it does not!

The first thing to point out is that the new systematic review included not just RCTs of chiropractic but also osteopathy and cranio-sacral therapy.

The second important issue is that the effects disappear, once performance bias is being accounted for which clearly shows that the result is false positive.

The third relevant fact is that the majority of the RCTs were of poor quality. The methodologically best studies were negative.

And the fourth thing to note is that only one study mentioned adverse effects, which means that the other 5 trials were in breach of one of rather elementary research ethics.

What makes all of this even more fascinating is the fact that the senior author of the new publication, George Lewith, is the very expert who advised the BCA in their libel case against Simon Singh. He seems so fond of his work that he even decided to re-publish it using even more misleading language than before. It is, of course, far from me to suggest that his review was an attempt to white-wash the issue of chiropractic ‘bogus’ claims. However, based on the available evidence, I would have formulated conclusions which are more than just a little different from his; something like this perhaps:

The current best evidence suggests that the small effects that emerge when we pool the data from mostly unreliable studies are due to bias and therefore not real. This systematic review therefore fails to show that manipulative therapies are effective. It furthermore points to a serious breach of research ethics by the majority of researchers in this field.

A new book is currently being promoted. It specifically targets cancer patients and misleads them into thinking that alternative therapies offer hope for this vulnerable group of patients. Here is what the press release says:

Endeavoring to provide the 1.2 million Americans diagnosed with cancer annually with alternative treatments co-authors Johanna C. Schipper and Frank J. Vanderlugt announce the launch of “The Natural Cancer Handbook”. The useful book explores how more than fifty alternative treatments work, their price, and where they can be obtained…. Contributing to the war on cancer with a bevy of scientific and anecdotal evidence to support the effectiveness of the treatments the handbook is a respite from the mixed messages patients often endure.

With more than fifty of the most effective alternative cancer treatments listed The Natural Cancer Handbook is the work of two years of research. Used successfully over the last century, the remedies found in the handbook are significantly cheaper than standard cancer treatments and in most cases can be used alongside them.

…The handbook discusses the successful alternative treatments Budwig Diet, Beta 1, 3D Glucan, and the readily available green food supplements such as barley grass, chlorella and spirulina. The Natural Cancer Handbook also explores the benefits of Melatonin, Noni, Resveratrol and the Canadian Resonant Light and the Hulda Clark generators.

Vanderlugt is a Chartered Accountant with a Bachelor of Science in Biology and Schipper has researched cancer extensively and has five years training in medicine.

Let’s just take the first treatment mentioned above; this is what a reliable source like CANCER RESEARCH UK have to say about it:

The Budwig diet was developed by a German biochemist called Johanna Budwig in the 1950s. It involves eating flaxseed mixed with cottage cheese or milk. Flax is a plant grown in many parts of the world. Pressing its seeds produces linseed oil to use in cooking or as a food supplement. The seeds contain high levels of fibre and many vitamins and minerals. You grind the flaxseed, usually in a coffee grinder. As well as flaxseed and cottage cheese, the Budwig diet is rich in fruit, vegetables and fibre. You also have to avoid sugar, meat, and fats such as butter, margarine and salad oil.

There is no reliable scientific evidence to show that the Budwig diet (or any highly specific diet) helps people with cancer. It is important to make sure that you have a well balanced diet when you are ill, especially if you are undernourished. We know from research that a healthy, well balanced diet can reduce the risk of cancer. You can find information about diet, healthy eating and cancer on our News and Resources website.

This is a polite way of telling us that diets such as this one is not balanced and not what cancer patients need; in fact, such diets are not just ineffective, they can be dangerous to cancer patients.

Texts like the Natural Cancer Handbook tend to make me quite angry. I find it deeply immoral to mislead cancer patients in this way, simply to make a profit. The truth could not be simpler: There is and never will be such a thing as an alternative cancer ‘cure’.

The concept assumes that there exists an effective cure which is being suppressed only because it originates from alternative medicine circles. But this assumption is idiotic. As soon as a treatment shows promise, it will be picked up by the scientific and oncologic communities and researched until its therapeutic value is known. At the end of this process, we might have a new option to treat cancer effectively. Many examples exist where a new drug was developed from a plant; taxol is but one of many examples.

Those who deny these simple facts in order to make a fast buck from the desperation of some of the most vulnerable patients are, in my view, charlatans of the worst kind.

Today, there are several dozens of journals publishing articles on alternative medicine. ‘The Journal of Alternative and Complementary Medicine’ is one of the best known, and it has one of the highest impact factors of them all. The current issue holds a few ‘gems’ which might be worthy of a comment or two. Here I have selected three articles reporting clinical studies, and I reproduce their abstracts (almost) in full (in italics) and add my comments (for clarity in bold). All the articles are available electronically, and I have provided the links for those who want to investigate beyond the abstracts.

STUDY No 1

The first ‘pilot study‘ was aimed to demonstrate the potential of auricular acupuncture (AAT) for insomnia in maintenance haemodialysis (MHD) patients and to prepare for a future randomized controlled trial.

Eligible patients were enrolled into this descriptive pilot study and received AAT designed to manage insomnia for 4 weeks. Questionnaires that used the Pittsburgh sleep quality index (PSQI) were completed at baseline, after a 4-week intervention, and 1 month after completion of treatment. Sleep quality and other clinical characteristics, including sleeping pills taken, were statistically compared between different time points.

A total of 22 patients were selected as eligible participants and completed the treatment and questionnaires. The mean global PSQI score was significantly decreased after AAT intervention (p<0.05). Participants reported improved sleep quality (p<0.01), shorter sleep latency (p<0.05), less sleep disturbance (p<0.01), and less daytime dysfunction (p=0.01). They also exhibited less dependency on sleep medications, indicated by the reduction in weekly estazolam consumption from 6.98±4.44 pills to 4.23±2.66 pills (p<0.01). However, these improvements were not preserved 1 month after treatment.

Conclusions: In this single-center pilot study, complementary AAT for MHD patients with severe insomnia was feasible and well tolerated and showed encouraging results for sleep quality.

My comments:

In alternative medicine research, it has become far too common (almost generally accepted) to call a flimsy trial a ‘pilot study’. The authors give their game away by stating that, by conducting this trial, they want to ‘demonstrate the potential of AAT’. This is not a legitimate aim of research; science is for TESTING hypotheses, not for PROVING them!

The results of this trial show that patients experienced improvements after receiving AAT which, however, did not last. As there was no placebo control group, the most likely explanation for these outcomes would be that AAT generated a short-lasting placebo effect.

A sample size of 22 is, of course, far to small to allow any conclusions about the safety of the intervention. Despite these obvious facts, the authors seem convinced that AAT is both safe and effective.

STUDY No 2

The aim of the second study was to compare the therapeutic effect of Yamamoto new scalp acupuncture (YNSA), a recently developed microcupuncture system, with traditional acupuncture (TCA) for the prophylaxis and treatment of migraine headache.

In a randomized clinical trial, 80 patients with migraine headache were assigned to receive YNSA or TCA. A pain visual analogue scale (VAS) and migraine therapy assessment questionnaire (MTAQ) were completed before treatment, after 6 and 18 sections of treatment, and 1 month after completion of therapy.

All the recruited patients completed the study. Baseline characteristics were similar between the two groups. Frequency and severity of migraine attacks, nausea, the need for rescue treatment, and work absence rate decreased similarly in both groups. Recovery from headache and ability to continue daily activities 2 hours after medical treatment showed similar improvement in both groups (p>0.05).

Conclusions: Classic acupuncture and YNSA are similarly effective in the prophylaxis and treatment of migraine headache and may be considered as alternatives to pharmacotherapy.

My comments:

This is what is technically called an ‘equivalence trial’, i.e. a study that compares an experimental treatment (YNSA) to one that is (assumed to be) effective. To demonstrate equivalence, such trials need to have large sample sizes, and this study is woefully underpowered. As it stands, the results show nothing meaningful at all; if anything, they suggest that both interventions were similarly useless.

STUDY No 3

The third study determined whether injection with hypertonic dextrose and morrhuate sodium (prolotherapy) using a pragmatic, clinically determined injection schedule for knee osteoarthritis (KOA) results in improved knee pain, function, and stiffness compared to baseline status.

The participants were 38 adults who had at least 3 months of symptomatic KOA and who were in the control groups of a prior prolotherapy randomized controlled trial (RCT) (Prior-Control), were ineligible for the RCT (Prior-Ineligible), or were eligible but declined the RCT (Prior-Declined).

The injection sessions at occurred at 1, 5, and 9 weeks with as-needed treatment at weeks 13 and 17. Extra-articular injections of 15% dextrose and 5% morrhuate sodium were done at peri-articular tendon and ligament insertions. A single intra-articular injection of 6 mL 25% dextrose was performed through an inferomedial approach.

The primary outcome measure was the validated Western Ontario McMaster University Osteoarthritis Index (WOMAC). The secondary outcome measure was the Knee Pain Scale and postprocedure opioid medication use and participant satisfaction.

The Prior-Declined group reported the most severe baseline WOMAC score (p=0.02). Compared to baseline status, participants in the Prior-Control group reported a score change of 12.4±3.5 points (19.5%, p=0.002). Prior-Decline and Prior-Ineligible groups improved by 19.4±7.0 (42.9%, p=0.05) and 17.8±3.9 (28.4%, p=0.008) points, respectively; 55.6% of Prior-Control, 75% of Prior-Decline, and 50% of Prior-Ineligible participants reported score improvement in excess of the 12-point minimal clinical important difference on the WOMAC measure. Postprocedure opioid medication resulted in rapid diminution of prolotherapy injection pain. Satisfaction was high and there were no adverse events.

Conclusions: Prolotherapy using dextrose and morrhuate sodium injections for participants with mild-to-severe KOA resulted in safe, significant, sustained improvement of WOMAC-based knee pain, function, and stiffness scores compared to baseline status.

My Comments:

This study had nothing that one might call a proper control group: all the three groups mentioned were treated with the experimental treatment. No attempt was made to control for even the most obvious biases: the observed effects could have been due to placebo or any other non-specific effects. The authors conclusions imply a causal relationship between the treatment and the outcome which is wrong. The notion that the experimental treatment is ‘safe’ is based on just 38 patients and therefore not reasonable.

IMPLICATION

All of this might seem rather trivial, and my comments could be viewed as a deliberate and vicious attempt to discredit one of the most respected journals of alternative medicine. Yet, considering that articles of this nature are more the rule than the exception in alternative medicine, I do think that this flagrant lack of scientific rigour is a relevant issue and has important implications.

As long as research in this area continues to be deeply flawed, as long as reviewers turn a blind eye to (or are not smart enough to detect) even the most obvious mistakes, as long as journal editors accept any rubbish in order to fill their pages, there is a great danger that we are being continuously being mislead about the supposed therapeutic value of alternative therapies.

Many who read this blog will, of course, have the capacity to think critically and might therefore not fall into the trap of accepting the conclusions of fatally flawed research. But many other people, including politicians, journalists and consumers, might not have the necessary appraisal skills and will thus not be able to tell that such studies can serve only one purpose: to popularise bogus treatments and thereby render health care less effective and more dangerous. Enthusiasts of alternative medicine are usually fully convinced that such studies amount to evidence and ram this pseudo-information down the throat of health care decision makers – the effects of such lobbying on public health can be disastrous.

And there is another downside to the publication of such dismal drivel: assuming (as I do) that not all of alternative medicine is completely useless, such embarrassingly poor research will inevitably have detrimental effects on the discipline of alternative medicine. After being exposed to a seemingly endless stream of pseudo-research, critics will eventually give up taking any of it seriously and might claim that none of it is worth the bother. In other words, those who conduct, accept or publish such nonsensical papers are not only endangering medical progress in general, they are also harming the very cause they try so desperately hard to advance.

Categories