MD, PhD, FMedSci, FSB, FRCP, FRCPEd

bias

Yesterday, BBC NEWS published the following interesting text about a BBC4 broadcast entitled ‘THE ROYAL ACTIVIST’ aired on the same day:

Prince Charles has been a well-known supporter of complementary medicine. According to a… former Labour cabinet minister, Peter Hain, it was a topic they shared an interest in.

“He had been constantly frustrated at his inability to persuade any health ministers anywhere that that was a good idea, and so he, as he once described it to me, found me unique from this point of view, in being somebody that actually agreed with him on this, and might want to deliver it.”
Mr Hain added: “When I was Secretary of State for Northern Ireland in 2005-7, he was delighted when I told him that since I was running the place I could more or less do what I wanted to do.***
“I was able to introduce a trial for complementary medicine on the NHS, and it had spectacularly good results, that people’s well-being and health was vastly improved.

“And when he learnt about this he was really enthusiastic and tried to persuade the Welsh government to do the same thing and the government in Whitehall to do the same thing for England, but not successfully,” added Mr Hain.

*** obviously there is no homeopathic remedy for megalomania (but that’s a different story)

Oh really?

A TRIAL?

SPECTACULARLY GOOD RESULTS?

NO KIDDING?

Let’s have a look at the ‘trial’ and its results. An easily accessible report provides the following details about it:

From February 2007 to February 2008, Get Well UK ran the UK’s first government-backed complementary therapy pilot. Sixteen practitioners provided treatments including acupuncture, osteopathy and aromatherapy, to more than 700 patients at two GP practices in Belfast and Derry.   

The BBC made an hour long documentary following our trials and tribulations, which was broadcast on BBC1 NI on 5 May 2008.

Following the successful completion of the pilot, the results were analysed by Social and Market Research and recommendations were made to the Health Minister

Aims and Objectives 

The aim of the project was to pilot services integrating complementary medicine into existing primary care services in Northern Ireland. Get Well UK provided this pilot project for the Department for Health, Social Services and Public Safety (DHSSPS) during 2007.

The objectives were:

  • To measure the health outcomes of the service and monitor health improvements.
  • To redress inequalities in access to complementary medicine by providing therapies through the NHS, allowing access regardless of income.
  • To contribute to best practise in the field of delivering complementary therapies through primary care.
  • To provide work for suitably skilled and qualified practitioners.
  • To increase patient satisfaction with quick access to expert care.
  • To help patients learn skills to improve and retain their health.
  • To free up GP time to work with other patients.
  • To deliver the programme for 700 patients.

Results 

The results of the pilot were analysed by Social and Market Research, who produced this report.

The findings can be summarised as follows: 

Following the pilot, 80% of patients reported an improvement in their symptoms, 64% took less time off work and 55% reduced their use of painkillers.

In the pilot, 713 patients with a range of ages and demographic backgrounds and either physical or mental health conditions were referred to various complementary and alternative medicine (CAM) therapies via nine GP practices in Belfast and Londonderry. Patients assessed their own health and wellbeing pre and post therapy and GPs and CAM practitioners also rated patients’ responses to treatment and the overall effectiveness of the scheme.

Health improvement
• 81% of patients reported an improvement in their physical health
• 79% reported an improvement in their mental health
• 84% of patients linked an improvement in their health and wellbeing directly to their CAM treatment
• In 65% of patient cases, GPs documented a health improvement, correlating closely to patient-reported improvements
• 94% of patients said they would recommend CAM to another patient with their condition
• 87% of patient indicated a desire to continue with their CAM treatment

Painkillers and medication
• Half of GPs reported prescribing less medication and all reported that patients had indicated to them that they needed less
• 62% of patients reported suffering from less pain
• 55% reported using less painkillers following treatment
• Patients using medication reduced from 75% before treatment to 61% after treatment
• 44% of those taking medication before treatment had reduced their use afterwards

Health service and social benefits
• 24% of patients who used health services prior to treatment (i.e. primary and secondary care, accident and emergency) reported using the services less after treatment
• 65% of GPs reported seeing the patient less following the CAM referral
• Half of GPs said the scheme had reduced their workload and 17% reported a financial saving for their practice
• Half of GPs said their patients were using secondary care services less.

Impressed? Well, in case you are, please consider this:

  • there was no control group
  • therefore it is not possible to attribute any of the outcomes to the alternative therapies offered
  • they could have been due to placebo-effects
  • or to the natural history of the disease
  • or to regression towards the mean
  • or to social desirability
  • or to many other factors which are unrelated to the alternative treatments provided
  • most outcome measures were not objectively verified
  • the patients were self-selected
  • they would all have had conventional treatments in parallel
  • this ‘trial’ was of such poor quality that its findings were never published in a peer-reviewed journal
  • this was not a ‘trial’ but a ‘pilot study’
  • pilot studies are not normally for measuring outcomes but for testing the feasibility of a proper trial
  • the research expertise of the investigators was close to zero
  • the scientific community merely had pitiful smiles for this ‘trial’ when it was published
  • neither Northern Ireland nor any other region implemented the programme despite its “spectacularly good results”.

So, is the whole ‘trial’ story an utterly irrelevant old hat?

Certainly not! Its true significance does not lie in the fact that a few amateurs are trying to push bogus treatments into the NHS via the flimsiest pseudo-research of the century. The true significance, I think, is that it shows how Prince Charles, once again, oversteps the boundaries of his constitutional role.

If you are pregnant, a ‘breech presentation’ is not good news. It occurs when the fetus presents ‘bottom-down’ in the uterus. There are three types:

  • Breech with extended legs (frank) – 85% of cases
  • Breech with fully flexed legs (complete)
  • Footling (incomplete) with one or both thighs extended

The significance of breech presentation is its association with higher perinatal mortality and morbidity when compared to cephalic presentations. This is due both to pre-existing congenital malformation, increased incidence of breech in premature deliveries and increased risk of intrapartum trauma or asphyxia. Caesarean section has been adopted as the ‘normal’ mode of delivery for term breech presentations in Europe and the USA, as the consensus is that this reduces the risk of birth-related complications.

But Caesarian section is also not a desirable procedure. Something far less invasive would be much more preferable, of course. This is where the TCM-practitioners come in. They claim they have the solution: moxibustion, i.e. the stimulation of acupuncture points by heat. But does it really work? Can it turn the fetus into the correct position?

This new study aimed to assess the efficacy of moxibustion (heating of the acupuncture needle with an igniting charcoal moxa stick) with acupuncture for version of breech presentations to reduce their rate at 37 weeks of gestation and at delivery. It was a randomized, placebo-controlled, single-blind trial including 328 pregnant women recruited in a university hospital center between 33 4/7 and 35 4/7 weeks of gestation. Moxibustion with acupuncture or inactivated laser (placebo) treatment was applied to point BL 67 for 6 sessions. The principal endpoint was the percentage of fetuses in breech presentation at 37 2/7 weeks of gestation.

The results show that the percentage of fetuses in breech presentation at 37 2/7 weeks of gestation was not significantly different in both groups (72.0 in the moxibustion with acupuncture group compared with 63.4% in the placebo group).

The authors concluded that treatment by moxibustion with acupuncture was not effective in correcting breech presentation in the third trimester of pregnancy.

You might well ask why on earth anyone expected that stimulating an acupuncture point would turn a fetus in the mother’s uterus into the optimal position that carries the least risk during the process of giving birth. This is what proponents of this technique say about this approach:

During a TCM consultation to turn a breech baby the practitioner will take a comprehensive case history, make a diagnosis and apply the appropriate acupuncture treatment.  They will assess if moxibustion might be helpful. Practitioners will then instruct women on how to locate the appropriate acupuncture points and demonstrate how to safely apply moxa at home. The acupuncture point UB 67 is the primary point selected for use because it is the most dynamic point to activate the uterus.  Its forte is in turning malpositioned babies.  It is located on the outer, lower edge of both little toenails. According to TCM theory, moxa has a tonifying and warming effect which promotes movement and activity.  The nature of heat is also rising.  This warming and raising effect is utilised to encourage the baby to become more active and lift its bottom up in order to gain adequate momentum to summersault into the head down position. This technique can also be used to reposition transverse presentation, a situation where the baby’s has its shoulder or back pointing down, or is lying sideways across the abdomen.

Not convinced? I can’t say I blame you!

Clearly, we need to know what the totality of the most reliable evidence shows; and what better than a Cochrane review to inform us about it? Here is what it tells us:

Moxibustion was not found to reduce the number of non-cephalic presentations at birth compared with no treatment (P = 0.45). Moxibustion resulted in decreased use of oxytocin before or during labour for women who had vaginal deliveries compared with no treatment (risk ratio (RR) 0.28, 95% confidence interval (CI) 0.13 to 0.60). Moxibustion was found to result in fewer non-cephalic presentations at birth compared with acupuncture (RR 0.25, 95% CI 0.09 to 0.72). When combined with acupuncture, moxibustion resulted in fewer non-cephalic presentations at birth (RR 0.73, 95% CI 0.57 to 0.94), and fewer births by caesarean section (RR 0.79, 95% CI 0.64 to 0.98) compared with no treatment. When combined with a postural technique, moxibustion was found to result in fewer non-cephalic presentations at birth compared with the postural technique alone (RR 0.26, 95% CI 0.12 to 0.56).

In other words, there is indeed some encouraging albeit not convincing evidence! How can this be? There is no plausible explanation why this treatment should work!

But there is a highly plausible explanation why the results of many of the relevant trials are false-positive thus rendering a meta-analysis false-positive as well. I have repeatedly pointed out on this blog that practically all Chinese TCM-studies report (false) positive results; and many of the studies included in this review were done in China. The Cochrane review provides a strong hint about the lack of rigor in its ‘plain language summary’:

The included trials were of moderate methodological quality, sample sizes in some of the studies were small, how the treatment was applied differed and reporting was limited. While the results were combined they should be interpreted with caution due to the differences in the included studies. More evidence is needed concerning the benefits and safety of moxibustion.

So, would I recommend moxibustion for breech conversion? I don’t think so!

Traditional Chinese medicine (TCM) is often promoted as an effective therapy for cancer, and are numerous controlled clinical studies published in Chinese literature, yet no systematic analysis has been done of this body of evidence. This systematic review summarizes the evidence from controlled clinical studies published in Chinese on this subject.

The researchers looked for controlled clinical studies of TCM therapies for all kinds of cancers published in Chinese in four main Chinese electronic databases and found 2964 reports including 2385 randomized clinical trials and 579 non-randomized controlled studies.

The top seven cancer types treated were lung cancer, liver cancer, stomach cancer, breast cancer, esophagus cancer, colorectal cancer and nasopharyngeal cancer. The majority of studies (72%) applied TCM therapy combined with conventional treatments, whilst fewer (28%) applied only TCM therapy in the experimental groups. Herbal medicine was the most frequently tested TCM therapy (2677 studies, 90.32%).

The most frequently reported outcome was clinical symptom improvement (1667 studies, 56.24%) followed by biomarker indices (1270 studies, 42.85%), quality of life (1129 studies, 38.09%), chemo/radiotherapy induced side effects (1094 studies, 36.91%), tumor size (869 studies, 29.32%) and safety (547 studies, 18.45%). Completeness and adequacy of reporting appeared to improve with time.

The authors of this paper drew the following conclusion: data from controlled clinical studies of TCM therapies in cancer treatment is substantial, and different therapies are applied either as monotherapy or in combination with conventional medicine. Reporting of controlled clinical studies should be improved based on the CONSORT and TREND Statements in future. Further studies should address the most frequently used TCM therapy for common cancers and outcome measures should address survival, relapse/metastasis and quality of life.

Almost 3000 controlled clinical trials! This number is likely to impress many people – unless, of course, one knows that the quality of these studies is dismal. Interestingly, no formal assessment of study quality was included in this analysis. But it was mentioned that only 63 of these trials reported patient-blinding, and only 5 were deemed to be “relatively well designed” by the authors of this paper (who, incidentally, are strong proponents of TCM).

What I find the most interesting aspect of this article is the fact that the authors fail to mention how many of the studies reported a positive result – in a way, they don’t need to: there is plenty of evidence to show that virtually all of the Chinese studies of TCM are positive. In my view, this invalidates this body of evidence completely.

Analysis like the present one tend to lead us up the garden path. They suggest that there is a realistic hope for effective new treatments hidden in this difficult to access, large amount of data. This might lead other researchers to try to replicate some of the original studies. I fear that they would be wasting their time. From all I know, they are irreproducible.

If we search on ‘Medline’ for ‘complementary alternative medicine’ (CAM), we currently get about 13000 hits. A little graph on the side of the page demonstrates that, during the last 4 years, the number of articles on this subject has grown exponentially.

Surely, this must be very good news: such intense research activity will soon tell us exactly which alternative treatments work for which conditions and which don’t.

I beg to differ. Let me explain why.

The same ‘Medline’ search informs us that the majority of the recent articles were published in an open access journal called ‘Evidence-Based Complementary and Alternative Medicine’ (eCAM). For example, of the 80 most recent articles listed in Medline (on 26/5/2014), 53 came from that journal. The publication frequency of eCAM and its increase in recent years beggars belief: in 2011, they published just over 500 articles which is already a high number, but, in 2012, the figure had risen to >800, and in 2013 it was >1300 (the equivalent 2013 figure for the BMJ/BMJ Open by comparison is 4, and that for another alt med journal, e.g. Forsch Komplement, is 10)

How do they do it? How can eCAM be so dominant in publishing alt med research? The trick seems to be fairly simple.

Let’s assume you are an alt med researcher and you have an article that you would like to see published. Once you submit it to eCAM, your paper is sent to one of the ~150 members of the editorial board. These people are almost all strong proponents of alternative medicine; critics are a true rarity in this group. At this stage, you are able to suggest the peer reviewers for your submission (all who ever accepted this task are listed on the website; they amount to several thousand!), and it seems that, with the vast majority of submissions, the authors’ suggestions are being followed.

It goes without saying that most researchers suggest colleagues for peer reviewing who are not going to reject their work (the motto seems to be “if you pass my paper, I will pass yours). Therefore even faily flimsy bits of research pass this peer review process and get quickly published online in eCAM.

This process explains a lot, I think: 1) the extraordinarily high number of articles published 2) why currently more than 50% of all alt med research originate from eCAM 3) why so much of it is utter rubbish.

Even the mere titles of some of the articles might demonstrate my point. A few examples have to suffice:

  • Color distribution differences in the tongue in sleep disorder
  • Wen-dan decoction improves negative emotions in sleep-deprived rats by regulating orexin-a and leptin expression.
  • Yiqi Huoxue Recipe Improves Heart Function through Inhibiting Apoptosis Related to Endoplasmic Reticulum Stress in Myocardial Infarction Model of Rats.
  • Protective Effects of Bu-Shen-Huo-Xue Formula against 5/6 Nephrectomy-Induced Chronic Renal Failure in Rats
  • Effects and Mechanisms of Complementary and Alternative Medicine during the Reproductive Process
  • Evidence-based medicinal plants for modern chronic diseases
  • Transforming Pain into Beauty: On Art, Healing, and Care for the Spirit

This system of uncritical peer review and fast online publication seems to suit many of the people involved in this process: the journal’s owners are laughing all the way to the bank; there is a publication charge of US$ 2000 per article, and, in 2013, the income of eCAM must therefore have been well over US$2 000 000. The researchers are equally delighted; they get even their flimsiest papers published (remember: ‘publish or perish’!). And the evangelic believers in alternative medicine are pleased because they can now claim that their field is highly research-active and that there is plenty of evidence to support the use of this or that therapy.

But there are others who are not served well by eCAM habit of publishing irrelevant, low quality articles:

  • professionals who would like to advance health care and want to see reliable evidence as to which treatments work and which don’t,
  • the public who, in one way or another, pay for all this and might assume that published research tends to be relevant and reliable,
  • the patients who have given their time to researchers in the hope that their gift will improve health care,
  • ill individuals who hope that alternative treatments might relieve their suffering,
  • politicians who rely on research to be reliable in order to arrive at the right decisions.

Come to think of it, the vast majority of people should be less than enchanted with eCAM and similar journals.

Boiron is the world’s biggest producer of homeopathic remedies. It also is a firm that is relatively active in research into homeopathy. Here is one of their investigations which I find most remarkable.

This study was designed to describe the sociodemographic and clinical characteristics of patients recommended allopathic and/or homeopathic medicines for influenza-like illness (ILI) or ear nose and throat ENT disorders by pharmacists in France and to investigate the effectiveness of these treatments.

The introduction of the article includes interesting information; it informs us that, although homeopathy is more popular in Europe than in the Unites States, sales of homeopathic medicines in the United States grew by more than 1,000% in the late 1970s and early 1980s and continue to grow. In parallel, the number of physicians specializing in homeopathy doubled between 1980 and 1982. In 2003, sales of homeopathic medicines in the United States were estimated to be between $300 and $450 million, with an average growth rate of approximately 8% per year. Homeopathic drugs are among the top 10 nonprescription products sold in the category of analgesics to treat coughs, colds, and flu. The sale of homeopathic medicines in the United States is controlled by the Food, Drug, and Cosmetic Act and regulations issued by the U.S Food and Drug Administration (FDA). Homeopathic medicines in the United States are subject to well-controlled regulatory processes that closely resemble those used for allopathic medicines. FDA regulations for the sale of homeopathic medicines in the United States state that they can only be sold without prescription if they are for self-limiting conditions such as the common cold…

Am I mistaken, or does that paragraph read a bit like a text written by the marketing team of Boiron wanting to establish their products in the US?

Anyway, the methodology and results of the study are described in the abstract as follows:

A prospective, observational, multicenter study was carried out in randomly selected pharmacies across the 8 IDREM medical regions of France. Pharmacies that agreed to participate recruited male or female patients who responded to the following inclusion criteria: age ≥ 12 years presenting with the first symptoms of an ILI or ENT disorder that were 
present for less than 36 hours prior to the pharmacy visit. All medicines recorded in the study were recommended by the pharmacists. The following data were recorded at inclusion and after 3 days of treatment: the intensity of 13 symptoms, global symptom score, and disease impact on daily activities and sleep. Two groups of patients were compared: those recommended allopathic medicine only (AT group) and those recommended homeopathic medicine with or without allopathic medicine (HAT group). The number and severity of symptoms, change in global symptom score, and disease impact on daily activities and sleep were compared in the 2 treatment groups after 3 days of treatment. Independent predictors of recommendations for homeopathic medicine were identified by multi-
factorial logistic regression analysis.

A total of 242 pharmacies out of 4,809 (5.0%) contacted agreed to participate in the study, and 133 (2.8%) included at least 1 patient; 573 patients were analyzed (mean age: 42.5 ± 16.2 years; 61.9% female). Of these, 428 received allopathic medicines only (74.7%; AT group), and 145 (25.3%) received homeopathic medicines (HAT group) alone (9/145, 1.6%) or associated with allopathy (136/145, 23.7%). At inclusion, HAT patients were significantly younger (39.6 ± 14.8 vs. 43.4 ± 16.1 years; P  less than  0.05), had a higher mean number of symptoms (5.2 ± 2.5 vs. 4.4 ± 2.5; P  less than  0.01), and more severe symptoms (mean global symptom score: 24.3 ± 5.5 vs. 22.3 ± 5.8; P = 0.0019) than AT patients. After 3 days, the improvement in symptoms and disease impact on daily activities and sleep was comparable in both groups of patients.

From these findings, the authors draw the following conclusions: Patients recommended homeopathic medicine by pharmacists were younger and had more severe symptoms than those recommended allopathic medicine. After 3 days of treatment, clinical improvement was comparable in both treatment groups. Pharmacists have an important role to play in the effective management of ILI and ENT disorders.

And, to make perfectly clear what all this is about, the first sentence of the ‘discussion’ puts it to the point by stating that homeopathic medicine, with or without allopathic medicine, appears to be effective at alleviating the symptoms of ILI or ENT disorders.

Oh really?

As I have heard it said that Boiron seems to have the nasty habit of threatening their critics with legal action, I ought to be quite cautious in my assessment of this ‘masterpiece of promotion’. Yet a few comments must surely be permitted.

‘To describe the sociodemographic and clinical characteristics of patients recommended allopathic and/or homeopathic medicines’ is not what I personally find an interesting subject of research, nor is it anything that will affect health care meaningfully, I think. Yet ‘to investigate the effectiveness of these treatments’ is certainly interesting and important. I will therefore focus on this second aim of the study.

Hold on, was this really a ‘study’? On closer inspection, it seemed much more like a survey. People who felt that they were suffering from ILI and ENT disorders and thus went to a pharmacy to buy something for their problem were offered either homeopathic or conventional medicines. Those who accepted either of the recommendations were asked to fill out some self-assessment forms and received a phone call three days later to check their symptoms. 94% of all patients in the homeopathy group took homeopathic medicine in combination with ‘allopathic’ medicine (it is interesting, perhaps even telling, that this term used by the authors was invented by Hahnemann as an insult to conventional medicine!). There was no examination by a doctor to verify what condition the survey-participants were truly suffering from, and there was no verification that the information provided during the follow-up telephone call was in any way real. The most frequently recommended homeopathic medicine was Anas barbariae 200C (Oscillococcinum) which is Boiron’s famous homeopathically diluted (about one molecule per universe, I guess) duck-liver heavily promoted in France against colds and similar conditions.

As it turns out, those survey-participants who accepted the homeopathic recommendation were significantly younger than those who accepted the recommendation for a conventional treatment (many surveys confirm that younger people are more prone to trying alternative medicine than older ones). It stands to reason, that the younger (and therefore fitter) patients were in better general health and therefore might recover quicker than the older ones. But, in fact, they did not!

Could this be due to the homeopathic remedies actually delaying recovery? Of course not! Who would be silly enough to claim that homeopathy could have this (or any other) effect? According to the authors, it is due to the fact that this group ‘had more severe symptoms than those recommended allopathic medicine’. But, as I said, we have to take their word for it; there is no independent verification of this. It would, of course, be quite ridiculous to postulate that those survey-participants accepting homeopathy were also a little more introspective or concerned about their own health (perhaps even more gullible) and thus claimed more severe symptoms!

And what about the authors’ conclusion that clinical improvement was comparable in both treatment groups? Well, this is more than a little problematic, in my view: first, we have no independent verification of the ‘improvement’ in either group. Second, we don’t know that the conventional treatments actually worked, and it could well be that both approaches were similarly ineffective, and that the observed outcomes are merely a reflection of the natural history of the condition. And third, one might expect the homeopathic (younger) group to do not similarly well but slightly better, simply because the natural history of the illness would tend to be more benign in younger people.

Before I finish,I should make a brief comment about the authors’ courageous statement that  homeopathic medicine, with or without allopathic medicine, appears to be effective at alleviating the symptoms of ILI or ENT disorders. I think, for the reasons I already provided, this is extremely doubtful. In my view, more critical scientists would have phrased the conclusions differently:

THIS SURVEY SHOWS THAT EVALUATING THE EFFECTIVENESS OF MEDICAL INTERVENTIONS REQUIRES A MORE RIGOROUS METHODOLOGY THAN THAT OF A SURVEY.

But perhaps this would be asking a little too much of the authors; after all, at the end of the article, we find this telling footnote: Laboratoires Boiron provided financial support for the study. Cognet-Dementhon, Thevenard, Duru, and Allaert received consulting fees from Laboratoires Boiron for this study. Danno and Bordet are employees of Laboratoires Boiron.

The question whether infant colic can be effectively treated with manipulative therapies might seem rather trivial – after all, this is a benign condition which the infant quickly grows out of. However, the issue becomes a little more tricky, if we consider that it was one of the 6 paediatric illnesses which were at the centre of the famous libel case of the BCA against my friend and co-author Simon Singh. At the time, Simon had claimed that there was ‘not a jot of evidence’ for claiming that chiropractic was an effective treatment of infant colic, and my systematic review of the evidence strongly supported his statement. The BCA eventually lost their libel case and with it the reputation of chiropractic. Now a new article on this intriguing topic has become available; do we have to reverse our judgements?

The aim of this new systematic review was to evaluate the efficacy or effectiveness of manipulative therapies for infantile colic. Six RCTs of chiropractic, osteopathy or cranial osteopathy alone or in conjunction with other interventions were included with a total of 325 infants. Of the 6 included studies, 5 were “suggestive of a beneficial effect” and one found no evidence of benefit. Combining all the RCTs suggested that manipulative therapies had a significant effect. The average crying time was reduced by an average of 72 minutes per day. This effect was sustained for studies with a low risk of selection bias and attrition bias. When analysing only those studies with a low risk of performance bias (i.e. parental blinding) the improvement in daily crying hours was no longer statistically significant.

The quality of the studies was variable. There was a generally low risk of selection bias but a high risk of performance bias. Only one of the studies recorded adverse events and none were encountered.

From these data, the authors drew the following conclusion: Parents of infants receiving manipulative therapies reported fewer hours crying per day than parents whose infants did not and this difference was statistically significant. Most studies had a high risk of performance bias due to the fact that the assessors (parents) were not blind to who had received the intervention. When combining only those trials with a low risk of such performance bias the results did not reach statistical significance.

Does that mean that chiropractic does work for infant colic? No, it does not!

The first thing to point out is that the new systematic review included not just RCTs of chiropractic but also osteopathy and cranio-sacral therapy.

The second important issue is that the effects disappear, once performance bias is being accounted for which clearly shows that the result is false positive.

The third relevant fact is that the majority of the RCTs were of poor quality. The methodologically best studies were negative.

And the fourth thing to note is that only one study mentioned adverse effects, which means that the other 5 trials were in breach of one of rather elementary research ethics.

What makes all of this even more fascinating is the fact that the senior author of the new publication, George Lewith, is the very expert who advised the BCA in their libel case against Simon Singh. He seems so fond of his work that he even decided to re-publish it using even more misleading language than before. It is, of course, far from me to suggest that his review was an attempt to white-wash the issue of chiropractic ‘bogus’ claims. However, based on the available evidence, I would have formulated conclusions which are more than just a little different from his; something like this perhaps:

The current best evidence suggests that the small effects that emerge when we pool the data from mostly unreliable studies are due to bias and therefore not real. This systematic review therefore fails to show that manipulative therapies are effective. It furthermore points to a serious breach of research ethics by the majority of researchers in this field.

Imagine an area of therapeutics where 100% of all findings of hypothesis-testing research is positive, i.e. comes to the conclusion that the treatment in question is effective. Theoretically, this could mean that the therapy is a miracle cure which is useful for every single condition in every single setting. But sadly, there are no miracle cures. Therefore something must be badly and worryingly amiss with the research in an area that generates 100% positive results.

Acupuncture is such an area; we and others have shown that Chinese trials of acupuncture hardly ever produce a negative finding. In other words, one does not need to read the paper, one already knows that it is positive – even more extreme: one does not need to conduct the study, one already knows the result before the research has started. But you might not believe my research nor that of others. We might be chauvinist bastards who want to discredit Chinese science. In this case, you might perhaps believe Chinese researchers.

In this systematic review, all randomized controlled trials (RCTs) of acupuncture published in Chinese journals were identified by a team of Chinese scientists. A total of 840 RCTs were found, including 727 RCTs comparing acupuncture with conventional treatment, 51 RCTs with no treatment controls, and 62 RCTs with sham-acupuncture controls. Among theses 840 RCTs, 838 studies (99.8%) reported positive results from primary outcomes and two trials (0.2%) reported negative results. The percentages of RCTs concealment of the information on withdraws or sample size calculations were 43.7%, 5.9%, 4.9%, 9.9%, and 1.7% respectively.

The authors concluded that publication bias might be major issue in RCTs on acupuncture published in Chinese journals reported, which is related to high risk of bias. We suggest that all trials should be prospectively registered in international trial registry in future.

I applaud the authors’ courageous efforts to conduct this analysis, but I do not agree with their conclusion. The question why all Chinese acupuncture trials are positive has puzzled me since many years, and I have quizzed numerous Chinese colleagues why this might be so. The answer I received was uniformly that it would be very offensive for Chinese researchers to conceive a study that does not confirm the views held by their peers. In other words, acupuncture research in China is conducted to confirm the prior assumption that this treatment is effective. It seems obvious that this is an abuse of science which must cause confusion.

Whatever the reasons for the phenomenon, and we can only speculate about them, the fact has been independently confirmed several times and is now quite undeniable: acupuncture trials from China – and these constitute the majority of the evidence-base in this area – cannot be trusted. The only way to adequately deal with this problem that I can think of is to discard them outright.

The fact that practitioners of alternative medicine frequently advise their patients against immunising their children has been documented repeatedly. In particular, doctors of anthroposophy, chiropractors and homeopaths are implicated in thus endangering public health. Less is known about naturopaths attitude in this respect. Now new data have emerged which confirm some of our worst fears.

This survey aimed at assessing the attitudes, education, and sources of knowledge surrounding childhood vaccinations of 560 students at National College of Natural Medicine in Portland, US. Students were asked about demographics, sources of information about childhood vaccines, differences between mainstream and CAM education on childhood vaccines, alternative vaccine schedules, adverse effects, perceived efficacy, and credibility of information sources.

A total of 109 students provided responses (19.4% response rate). All students surveyed learned about vaccinations in multiple courses and through independent study. The information sources employed had varying levels of credibility. Only 26% of the responding students planned on regularly prescribing or recommending vaccinations for their patients; 82% supported the general concept of vaccinations for prevention of infectious diseases.

The vast majority (96%) of those who might recommend vaccinations reported that they would only recommend a schedule that differed from the standard CDC-ACIP schedule.

Many respondents were concerned about vaccines being given too early (73%), too many vaccines administered simultaneously (70%), too many vaccines overall (59%), and about preservatives and adjuvants in vaccines (72%). About 40% believed that a healthy diet and lifestyle was more important for prevention of infectious diseases than vaccines. 90% admitted that they were more critical of vaccines than mainstream pediatricians, medical doctors, and medical students.

These results speak for themselves and leave me (almost) speechless. The response rate was truly dismal, and it is fair to assume that the non-responding students held even more offensive views on vaccination than their responding colleagues. The findings seem to indicate that naturopaths are systematically trained to become anti-vaxers who believe that their naturopathic treatments offer better protection than vaccines. They are thus depriving many of their patients of arguably the most successful means of disease prevention that exists today. To put it bluntly: naturopaths seem to be brain-washed into becoming a danger to public health.

Today, there are several dozens of journals publishing articles on alternative medicine. ‘The Journal of Alternative and Complementary Medicine’ is one of the best known, and it has one of the highest impact factors of them all. The current issue holds a few ‘gems’ which might be worthy of a comment or two. Here I have selected three articles reporting clinical studies, and I reproduce their abstracts (almost) in full (in italics) and add my comments (for clarity in bold). All the articles are available electronically, and I have provided the links for those who want to investigate beyond the abstracts.

STUDY No 1

The first ‘pilot study‘ was aimed to demonstrate the potential of auricular acupuncture (AAT) for insomnia in maintenance haemodialysis (MHD) patients and to prepare for a future randomized controlled trial.

Eligible patients were enrolled into this descriptive pilot study and received AAT designed to manage insomnia for 4 weeks. Questionnaires that used the Pittsburgh sleep quality index (PSQI) were completed at baseline, after a 4-week intervention, and 1 month after completion of treatment. Sleep quality and other clinical characteristics, including sleeping pills taken, were statistically compared between different time points.

A total of 22 patients were selected as eligible participants and completed the treatment and questionnaires. The mean global PSQI score was significantly decreased after AAT intervention (p<0.05). Participants reported improved sleep quality (p<0.01), shorter sleep latency (p<0.05), less sleep disturbance (p<0.01), and less daytime dysfunction (p=0.01). They also exhibited less dependency on sleep medications, indicated by the reduction in weekly estazolam consumption from 6.98±4.44 pills to 4.23±2.66 pills (p<0.01). However, these improvements were not preserved 1 month after treatment.

Conclusions: In this single-center pilot study, complementary AAT for MHD patients with severe insomnia was feasible and well tolerated and showed encouraging results for sleep quality.

My comments:

In alternative medicine research, it has become far too common (almost generally accepted) to call a flimsy trial a ‘pilot study’. The authors give their game away by stating that, by conducting this trial, they want to ‘demonstrate the potential of AAT’. This is not a legitimate aim of research; science is for TESTING hypotheses, not for PROVING them!

The results of this trial show that patients experienced improvements after receiving AAT which, however, did not last. As there was no placebo control group, the most likely explanation for these outcomes would be that AAT generated a short-lasting placebo effect.

A sample size of 22 is, of course, far to small to allow any conclusions about the safety of the intervention. Despite these obvious facts, the authors seem convinced that AAT is both safe and effective.

STUDY No 2

The aim of the second study was to compare the therapeutic effect of Yamamoto new scalp acupuncture (YNSA), a recently developed microcupuncture system, with traditional acupuncture (TCA) for the prophylaxis and treatment of migraine headache.

In a randomized clinical trial, 80 patients with migraine headache were assigned to receive YNSA or TCA. A pain visual analogue scale (VAS) and migraine therapy assessment questionnaire (MTAQ) were completed before treatment, after 6 and 18 sections of treatment, and 1 month after completion of therapy.

All the recruited patients completed the study. Baseline characteristics were similar between the two groups. Frequency and severity of migraine attacks, nausea, the need for rescue treatment, and work absence rate decreased similarly in both groups. Recovery from headache and ability to continue daily activities 2 hours after medical treatment showed similar improvement in both groups (p>0.05).

Conclusions: Classic acupuncture and YNSA are similarly effective in the prophylaxis and treatment of migraine headache and may be considered as alternatives to pharmacotherapy.

My comments:

This is what is technically called an ‘equivalence trial’, i.e. a study that compares an experimental treatment (YNSA) to one that is (assumed to be) effective. To demonstrate equivalence, such trials need to have large sample sizes, and this study is woefully underpowered. As it stands, the results show nothing meaningful at all; if anything, they suggest that both interventions were similarly useless.

STUDY No 3

The third study determined whether injection with hypertonic dextrose and morrhuate sodium (prolotherapy) using a pragmatic, clinically determined injection schedule for knee osteoarthritis (KOA) results in improved knee pain, function, and stiffness compared to baseline status.

The participants were 38 adults who had at least 3 months of symptomatic KOA and who were in the control groups of a prior prolotherapy randomized controlled trial (RCT) (Prior-Control), were ineligible for the RCT (Prior-Ineligible), or were eligible but declined the RCT (Prior-Declined).

The injection sessions at occurred at 1, 5, and 9 weeks with as-needed treatment at weeks 13 and 17. Extra-articular injections of 15% dextrose and 5% morrhuate sodium were done at peri-articular tendon and ligament insertions. A single intra-articular injection of 6 mL 25% dextrose was performed through an inferomedial approach.

The primary outcome measure was the validated Western Ontario McMaster University Osteoarthritis Index (WOMAC). The secondary outcome measure was the Knee Pain Scale and postprocedure opioid medication use and participant satisfaction.

The Prior-Declined group reported the most severe baseline WOMAC score (p=0.02). Compared to baseline status, participants in the Prior-Control group reported a score change of 12.4±3.5 points (19.5%, p=0.002). Prior-Decline and Prior-Ineligible groups improved by 19.4±7.0 (42.9%, p=0.05) and 17.8±3.9 (28.4%, p=0.008) points, respectively; 55.6% of Prior-Control, 75% of Prior-Decline, and 50% of Prior-Ineligible participants reported score improvement in excess of the 12-point minimal clinical important difference on the WOMAC measure. Postprocedure opioid medication resulted in rapid diminution of prolotherapy injection pain. Satisfaction was high and there were no adverse events.

Conclusions: Prolotherapy using dextrose and morrhuate sodium injections for participants with mild-to-severe KOA resulted in safe, significant, sustained improvement of WOMAC-based knee pain, function, and stiffness scores compared to baseline status.

My Comments:

This study had nothing that one might call a proper control group: all the three groups mentioned were treated with the experimental treatment. No attempt was made to control for even the most obvious biases: the observed effects could have been due to placebo or any other non-specific effects. The authors conclusions imply a causal relationship between the treatment and the outcome which is wrong. The notion that the experimental treatment is ‘safe’ is based on just 38 patients and therefore not reasonable.

IMPLICATION

All of this might seem rather trivial, and my comments could be viewed as a deliberate and vicious attempt to discredit one of the most respected journals of alternative medicine. Yet, considering that articles of this nature are more the rule than the exception in alternative medicine, I do think that this flagrant lack of scientific rigour is a relevant issue and has important implications.

As long as research in this area continues to be deeply flawed, as long as reviewers turn a blind eye to (or are not smart enough to detect) even the most obvious mistakes, as long as journal editors accept any rubbish in order to fill their pages, there is a great danger that we are being continuously being mislead about the supposed therapeutic value of alternative therapies.

Many who read this blog will, of course, have the capacity to think critically and might therefore not fall into the trap of accepting the conclusions of fatally flawed research. But many other people, including politicians, journalists and consumers, might not have the necessary appraisal skills and will thus not be able to tell that such studies can serve only one purpose: to popularise bogus treatments and thereby render health care less effective and more dangerous. Enthusiasts of alternative medicine are usually fully convinced that such studies amount to evidence and ram this pseudo-information down the throat of health care decision makers – the effects of such lobbying on public health can be disastrous.

And there is another downside to the publication of such dismal drivel: assuming (as I do) that not all of alternative medicine is completely useless, such embarrassingly poor research will inevitably have detrimental effects on the discipline of alternative medicine. After being exposed to a seemingly endless stream of pseudo-research, critics will eventually give up taking any of it seriously and might claim that none of it is worth the bother. In other words, those who conduct, accept or publish such nonsensical papers are not only endangering medical progress in general, they are also harming the very cause they try so desperately hard to advance.

A recent US study found that belief in conspiracy theories is rife in health care. The investigators presented people with 6 different conspiracy theories, and the one that was most widely believed was the following:

THE FOOD AND DRUG ADMINISTRATION IS DELIBERATELY PREVENTING THE PUBLIC FROM GETTING NATURAL CURES FOR CANCER AND OTHER DISEASES BECAUSE OF PRESSURE FROM DRUG COMPANIES.

A total of 37% agreed with this statement, 31% had no opinion on the matter, and 32% disagreed. What is more, the belief in this particular conspiracy correlated positively with the usage of alternative medicine.

Essentially, this implies that the current popularity of alternative medicine is at least partly driven by the conviction that there is a sinister plot by the FDA or more generally speaking ‘the establishment’ that prevents people from benefitting from the wonders of alternative treatments.

I think it was Woody Allen who noted that, just because you are paranoid does not mean that they are not following you. So, let’s look for evidence suggesting that the FDA or any similar organisation is suppressing alternative medicine.

A prime candidate is, of course, the often implicated, thoroughly evil ‘BIG PHARMA‘. I am not a fan of the pharmaceutical industry and I know few people who are. But where is the evidence for BIG PHARMA’s conspiracy against alternative medicine? In the many years of researching this sector, I have never come across a jot of evidence to support this notion. On the contrary, BIG PHARMA seems all to keen to jump on to the alternative bandwagon and make a few quick bucks from the gullibility of the consumer.

What about the rest of the medical establishment? All I see is that universities, hospitals, charities and other organisations in health care currently bend over backwards in order to accommodate as much alternative medicine as they possibly can get away with in view of the often embarrassing lack of convincing evidence for the treatments in question. Conspiracy against alternative medicine? I don’t think so.

The closer we look, the more we arrive at the conclusion that the conspiracy against alternative medicine is a myth and a figment of the imagination of those who religiously believe in alternative medicine. They seem to long for an explanation why their favourite therapy is not in even more wide-spread use. Cognitive dissonance seems to prevent them to consider that the lack of evidence has anything to do with this situation. Consequently, they prefer to invent a conspiracy theory.

And this is where an interesting question emerges, in my view: do people who believe that the FDA or other organisations prevent the public from getting more alternative medicine really need more alternative medicine, or do they perhaps just need an effective treatment for their paranoia?

Categories