Today, there are several dozens of journals publishing articles on alternative medicine. ‘The Journal of Alternative and Complementary Medicine’ is one of the best known, and it has one of the highest impact factors of them all. The current issue holds a few ‘gems’ which might be worthy of a comment or two. Here I have selected three articles reporting clinical studies, and I reproduce their abstracts (almost) in full (in italics) and add my comments (for clarity in bold). All the articles are available electronically, and I have provided the links for those who want to investigate beyond the abstracts.
STUDY No 1
The first ‘pilot study‘ was aimed to demonstrate the potential of auricular acupuncture (AAT) for insomnia in maintenance haemodialysis (MHD) patients and to prepare for a future randomized controlled trial.
Eligible patients were enrolled into this descriptive pilot study and received AAT designed to manage insomnia for 4 weeks. Questionnaires that used the Pittsburgh sleep quality index (PSQI) were completed at baseline, after a 4-week intervention, and 1 month after completion of treatment. Sleep quality and other clinical characteristics, including sleeping pills taken, were statistically compared between different time points.
A total of 22 patients were selected as eligible participants and completed the treatment and questionnaires. The mean global PSQI score was significantly decreased after AAT intervention (p<0.05). Participants reported improved sleep quality (p<0.01), shorter sleep latency (p<0.05), less sleep disturbance (p<0.01), and less daytime dysfunction (p=0.01). They also exhibited less dependency on sleep medications, indicated by the reduction in weekly estazolam consumption from 6.98±4.44 pills to 4.23±2.66 pills (p<0.01). However, these improvements were not preserved 1 month after treatment.
Conclusions: In this single-center pilot study, complementary AAT for MHD patients with severe insomnia was feasible and well tolerated and showed encouraging results for sleep quality.
In alternative medicine research, it has become far too common (almost generally accepted) to call a flimsy trial a ‘pilot study’. The authors give their game away by stating that, by conducting this trial, they want to ‘demonstrate the potential of AAT’. This is not a legitimate aim of research; science is for TESTING hypotheses, not for PROVING them!
The results of this trial show that patients experienced improvements after receiving AAT which, however, did not last. As there was no placebo control group, the most likely explanation for these outcomes would be that AAT generated a short-lasting placebo effect.
A sample size of 22 is, of course, far to small to allow any conclusions about the safety of the intervention. Despite these obvious facts, the authors seem convinced that AAT is both safe and effective.
STUDY No 2
The aim of the second study was to compare the therapeutic effect of Yamamoto new scalp acupuncture (YNSA), a recently developed microcupuncture system, with traditional acupuncture (TCA) for the prophylaxis and treatment of migraine headache.
In a randomized clinical trial, 80 patients with migraine headache were assigned to receive YNSA or TCA. A pain visual analogue scale (VAS) and migraine therapy assessment questionnaire (MTAQ) were completed before treatment, after 6 and 18 sections of treatment, and 1 month after completion of therapy.
All the recruited patients completed the study. Baseline characteristics were similar between the two groups. Frequency and severity of migraine attacks, nausea, the need for rescue treatment, and work absence rate decreased similarly in both groups. Recovery from headache and ability to continue daily activities 2 hours after medical treatment showed similar improvement in both groups (p>0.05).
Conclusions: Classic acupuncture and YNSA are similarly effective in the prophylaxis and treatment of migraine headache and may be considered as alternatives to pharmacotherapy.
This is what is technically called an ‘equivalence trial’, i.e. a study that compares an experimental treatment (YNSA) to one that is (assumed to be) effective. To demonstrate equivalence, such trials need to have large sample sizes, and this study is woefully underpowered. As it stands, the results show nothing meaningful at all; if anything, they suggest that both interventions were similarly useless.
STUDY No 3
The third study determined whether injection with hypertonic dextrose and morrhuate sodium (prolotherapy) using a pragmatic, clinically determined injection schedule for knee osteoarthritis (KOA) results in improved knee pain, function, and stiffness compared to baseline status.
The participants were 38 adults who had at least 3 months of symptomatic KOA and who were in the control groups of a prior prolotherapy randomized controlled trial (RCT) (Prior-Control), were ineligible for the RCT (Prior-Ineligible), or were eligible but declined the RCT (Prior-Declined).
The injection sessions at occurred at 1, 5, and 9 weeks with as-needed treatment at weeks 13 and 17. Extra-articular injections of 15% dextrose and 5% morrhuate sodium were done at peri-articular tendon and ligament insertions. A single intra-articular injection of 6 mL 25% dextrose was performed through an inferomedial approach.
The primary outcome measure was the validated Western Ontario McMaster University Osteoarthritis Index (WOMAC). The secondary outcome measure was the Knee Pain Scale and postprocedure opioid medication use and participant satisfaction.
The Prior-Declined group reported the most severe baseline WOMAC score (p=0.02). Compared to baseline status, participants in the Prior-Control group reported a score change of 12.4±3.5 points (19.5%, p=0.002). Prior-Decline and Prior-Ineligible groups improved by 19.4±7.0 (42.9%, p=0.05) and 17.8±3.9 (28.4%, p=0.008) points, respectively; 55.6% of Prior-Control, 75% of Prior-Decline, and 50% of Prior-Ineligible participants reported score improvement in excess of the 12-point minimal clinical important difference on the WOMAC measure. Postprocedure opioid medication resulted in rapid diminution of prolotherapy injection pain. Satisfaction was high and there were no adverse events.
Conclusions: Prolotherapy using dextrose and morrhuate sodium injections for participants with mild-to-severe KOA resulted in safe, significant, sustained improvement of WOMAC-based knee pain, function, and stiffness scores compared to baseline status.
This study had nothing that one might call a proper control group: all the three groups mentioned were treated with the experimental treatment. No attempt was made to control for even the most obvious biases: the observed effects could have been due to placebo or any other non-specific effects. The authors conclusions imply a causal relationship between the treatment and the outcome which is wrong. The notion that the experimental treatment is ‘safe’ is based on just 38 patients and therefore not reasonable.
All of this might seem rather trivial, and my comments could be viewed as a deliberate and vicious attempt to discredit one of the most respected journals of alternative medicine. Yet, considering that articles of this nature are more the rule than the exception in alternative medicine, I do think that this flagrant lack of scientific rigour is a relevant issue and has important implications.
As long as research in this area continues to be deeply flawed, as long as reviewers turn a blind eye to (or are not smart enough to detect) even the most obvious mistakes, as long as journal editors accept any rubbish in order to fill their pages, there is a great danger that we are being continuously being mislead about the supposed therapeutic value of alternative therapies.
Many who read this blog will, of course, have the capacity to think critically and might therefore not fall into the trap of accepting the conclusions of fatally flawed research. But many other people, including politicians, journalists and consumers, might not have the necessary appraisal skills and will thus not be able to tell that such studies can serve only one purpose: to popularise bogus treatments and thereby render health care less effective and more dangerous. Enthusiasts of alternative medicine are usually fully convinced that such studies amount to evidence and ram this pseudo-information down the throat of health care decision makers – the effects of such lobbying on public health can be disastrous.
And there is another downside to the publication of such dismal drivel: assuming (as I do) that not all of alternative medicine is completely useless, such embarrassingly poor research will inevitably have detrimental effects on the discipline of alternative medicine. After being exposed to a seemingly endless stream of pseudo-research, critics will eventually give up taking any of it seriously and might claim that none of it is worth the bother. In other words, those who conduct, accept or publish such nonsensical papers are not only endangering medical progress in general, they are also harming the very cause they try so desperately hard to advance.
It has been reported that Belgium has just officially recognised homeopathy. The government had given the green light already in July last year, but the Royal Decree has only now become official. This means that, from now on, Belgian doctors, dentists and midwives can only call themselves homeopaths, if they have attended recognised courses in homeopathy and are officially certified. While much of the new regulation is as yet unclear (at least to me), it seems that, in future, only doctors, dentists and midwives are allowed to practice homeopathy, according to one source.
However, the new law also seems to provide that those clinicians with a Bachelor degree in health care who have already been practicing as homeopaths can continue their activities under a temporary measure.
Moreover, the official recognition as a homeopath does not automatically imply that the services will be refunded from a health insurance.
It is said that, in general, homeopaths are happy with the new regulation; they are delighted to have been up-graded in this way and argue that the changes will result in higher quality standards: “This is a very important step and it can only be to the benefit of the patients’ safety. Patients will know whether or not they are dealing with someone who correctly applies homeopathic medicine”, Leon Schepers of the Unio Homeopathica Belgica was quoted saying.
The delight of homeopaths is in sharp contrast to the dismay of rational thinkers. The NHMRC recently assessed the effectiveness of homeopathy. The evaluation is both comprehensive and independent; it concluded that “the evidence from research in humans does not show that homeopathy is effective for treating the range of health conditions considered.” In other words, homeopathic remedies are implausible, over-priced placebos.
Granting an official status to homeopaths cannot possibly benefit patients. On the contrary, it will only render health care less effective and charlatans more assertive.
It is not often that we see an article of the great George Vithoulkas, the ‘über-guru‘ of homeopathy, in a medical journal. In fact, this paper, which he co-authored with several colleagues, seems to be a rare exception: in his entire career, he seems to have published just 15 Medline- listed articles most of which are letters to the editor.
According to Wikipedia, Vithoulkas has been described as “the maestro of classical homeopathy” by Robin Shohet; Lyle Morgan says he is “widely considered to be the greatest living homeopathic theorist”; and Scott Shannon calls him a “contemporary master of homeopathy.” Paul Ekins credited Vithoulkas with the revival of the credibility of homeopathy.
In his brand new paper, Vithoulkas provides evidence for the notion that homeopathy can treat infertility. More specifically, the authors present 5 cases of female infertility treated successfully with the use of homeopathic remedies.
Yes, really! The American Medical College of Homeopathy informs us that homeopathy has an absolute solution that can augment your probability of conception. Homeopathic treatment of Infertility addresses both physical and emotional imbalances in a person. Homeopathy plays a role in treating Infertility by strengthening the reproductive organs in both men and women, by regulating hormonal balance, menstruation and ovulation in women, by escalating blood flow into the pelvic region, by mounting the thickness of the uterine lining and preventing the uterus from contracting hence abating chances of a miscarriage, and by increasing quality and quantity of sperm count in men. It can also be advantageous in reducing anxiety so that the embryo implantation can take place in a favourable environment. Homoeopathy is a system of medicine directed at assisting the body’s own healing process.
Imagine: the 5 women in Vithoulkas ‘study’ wanted to have children; they consulted homeopaths because they did not get pregnant in a timely fashion. The homeopaths prescribed individualised homeopathy and treated them for prolonged periods of time. Eventually, BINGO!, all of the 5 women got pregnant.
What a hoot!
It beggars belief that this result is being credited to the administration of homeopathic remedies. Do the authors not know that, in many cases, it can take many months until a pregnancy occurs? Do they not think that the many women they treated unsuccessfully for the same problem should raise some doubts about homeopathy? Do they really believe that their remedies had any causal relationship to the 5 pregnancies?
Vithoulkas was a recipient of the Right Livelihood Award in 1996. I hope they did not give it to him in recognition of his scientific achievements!
There is much debate about the usefulness of chiropractic. Specifically, many people doubt that their chiropractic spinal manipulations generate more good than harm, particularly for conditions which are not related to the spine. But do chiropractors treat such conditions frequently and, if yes, what techniques do they employ?
This investigation was aimed at describing the clinical practices of chiropractors in Victoria, Australia. It was a cross-sectional survey of 180 chiropractors in active clinical practice in Victoria who had been randomly selected from the list of 1298 chiropractors registered on Chiropractors Registration Board of Victoria. Twenty-four chiropractors were ineligible, 72 agreed to participate, and 52 completed the study.
Each participating chiropractor documented encounters with up to 100 consecutive patients. For each chiropractor-patient encounter, information collected included patient health profile, patient reasons for encounter, problems and diagnoses, and chiropractic care.
Data were collected on 4464 chiropractor-patient encounters between 11 December 2010 and 28 September 2012. In most (71%) cases, patients were aged 25-64 years; 1% of encounters were with infants. Musculoskeletal reasons for the consultation were described by patients at a rate of 60 per 100 encounters, while maintenance and wellness or check-up reasons were described at a rate of 39 per 100 encounters. Back problems were managed at a rate of 62 per 100 encounters.
The most frequent care provided by the chiropractors was spinal manipulative therapy and massage. The table shows the precise conditions treated
|Problem group||No. (%) of recorded diagnoses* (n = 5985)||Rate per 100 encounters (n = 4417)||95% CI||ICC|
|Back problem||2757 (46.07%)||62.42||(55.24–70.53)||0.312|
|Neck problem||683 (11.41%)||15.46||(11.23–21.30)||0.233|
|Muscle problem||434 (7.25%)||9.83||(6.64–14.55)||0.207|
|Health maintenance or preventive care||254 (4.24%)||5.75||(3.24–10.22)||0.251|
|Back syndrome with radiating pain||215 (3.59%)||4.87||(2.91–8.14)||0.165|
|Musculoskeletal symptom or complaint, or other||219 (3.66%)||4.96||(2.39–10.28)||0.350|
|Sprain or strain of joint||167 (2.79%)||3.78||(2.30–6.22)||0.115|
|Shoulder problem||87 (1.45%)||1.97||(1.37–2.83)||0.022|
|Nerve-related problem||62 (1.04%)||1.40||(0.72–2.75)||0.072|
|General symptom or complaint, other||51 (0.85%)||1.15||(0.22–6.06)||0.407|
|Bursitis, tendinitis or synovitis||47 (0.79%)||1.06||(0.71–1.60)||0.011|
|Kyphosis and scoliosis||47 (0.79%)||1.06||(0.65–1.75)||0.023|
|Foot or toe symptom or complaint||48 (0.80%)||1.09||(0.41–2.87)||0.123|
|Ankle problem||46 (0.77%)||1.04||(0.40–2.69)||0.112|
|Osteoarthrosis, other (not spine)||39 (0.65%)||0.88||(0.51–1.53)||0.023|
|Hip symptom or complaint||35 (0.58%)||0.79||(0.53–1.19)||0.006|
|Leg or thigh symptom or complaint||35 (0.58%)||0.79||(0.49–1.28)||0.012|
|Musculoskeletal injury||33 (0.55%)||0.75||(0.45–1.24)||0.013|
These findings are impressive in that they suggest that most Australian chiropractors treat non-spinal conditions for which there is no evidence that the most frequently used interventions are effective. The treatments employed are depicted in this graph:
Distribution of techniques and care provided by chiropractors, with 95% CI
[Activator = hand-held spring-loaded device that delivers an impulse to the spine. Drop piece = chiropractic treatment table with a segmented drop system which quickly lowers the section of the patient’s body corresponding with the spinal region being treated. Blocks = wedge-shaped blocks placed under the pelvis.
Chiro system = chiropractic system of care, eg, Applied Kinesiology, Sacro-Occipital Technique, Neuroemotional Technique. Flexion distraction = chiropractic treatment table that flexes in the middle to provide traction and mobilisation to the lumbar spine.]
There is no good evidence I know of demonstrating these techniques to be effective for the majority of the conditions listed in the above table.
A similar bone of contention is the frequent use of ‘maintenance’ and ‘wellness’ care. The authors of the article comment: The common use of maintenance and wellness-related terms reflects current debate in the chiropractic profession. “Chiropractic wellness care” is considered by an indeterminate proportion of the profession as an integral part of chiropractic practice, with the belief that regular chiropractic care may have value in maintaining and promoting health, as well as preventing disease. The definition of wellness chiropractic care is controversial, with some chiropractors promoting only spine care as a form of wellness, and others promoting evidence-based health promotion, eg, smoking cessation and weight reduction, alongside spine care. A 2011 consensus process in the chiropractic profession in the United States emphasised that wellness practice must include health promotion and education, and active strategies to foster positive changes in health behaviours. My own systematic review of regular chiropractic care, however, shows that the claimed effects are totally unproven.
One does not need to be overly critical to conclude from all this that the chiropractors surveyed in this investigation earn their daily bread mostly by being economical with the truth regarding the lack of evidence for their actions.
These days, there is so much hype about alternative cancer treatments that it is hard to find a cancer patient who is not tempted to try this or that alternative medicine. Often it is employed without the knowledge of the oncology team, solely on the advice of non-medically qualified practitioners (NMPs). But is that wise? The aim of this survey was to find out.
Members of several German NMP-associations were invited to complete an online questionnaire. The questionnaire explored areas such as the diagnosis and treatment, goals for using complementary/alternative medicine (CAM), communication with the oncologist, and sources of information.
Of a total of 1,500 members of the NMP associations, 299 took part in this survey. The results show that the treatments employed by NMPs were heterogeneous. Homeopathy was used by 45% of the NMPs, and 10% believed it to be a treatment directly against cancer. Herbal therapy, vitamins, orthomolecular medicine, ordinal therapy, mistletoe preparations, acupuncture, and cancer diets were used by more than 10% of the NMPs. None of the treatments were discussed with the respective physician on a regular basis.
The authors concluded from these findings that many therapies provided by NMPs are biologically based and therefore may interfere with conventional cancer therapy. Thus, patients are at risk of interactions, especially as most NMPs do not adjust their therapies to those of the oncologist. Moreover, risks may arise from these CAM methods as NMPs partly believe them to be useful anticancer treatments. This may lead to the delay or even omission of effective therapies.
Anyone faced with a diagnosis of CANCER is understandably keen to leave no stone unturned to bring about a cure of the disease. Many patients thus go on to the Internet and look what alternative options are on offer. There they find virtually millions of sites advertising thousands of bogus cancer ‘cures’. Others consult their alternative practitioners and seek help. This new survey shows yet again that the advice they receive is dangerous. In fact, it might well be even more dangerous than the results imply: the response rate of the survey was dismal, and I fear that the less responsible NMPs tended not to reply.
None of the treatments listed above can cure cancer. For instance, homeopathy, the most popular alternative cancer treatment in Germany, will have no effect whatsoever on the natural history of the disease. To claim otherwise is criminally irresponsible.
But far too many patients are unaware of the evidence and of the dangers of being misled by bogus claims. What we need, I think, is a major campaign to get the word out. It would be a campaign that saves lives!
Fibromyalgia (FM) is a chronic condition which ruins the quality of life of many patients. It is also a domain of alternative medicine: dozens of different treatments are on offer – this is clearly a paradise for charlatans and bogus claims. So is there a treatment that is demonstrably effective? The purpose of this systematic review is to evaluate the evidence of massage therapy FM.
Electronic databases were searched to identify relevant studies. The main outcome measures were pain, anxiety, depression, and sleep disturbance. Two reviewers independently abstracted data and appraised risk of bias. The risk of bias of eligible studies was assessed based on Cochrane tools.
Nine randomized controlled trials involving 404 patients met the inclusion criteria. A meta-analyses showed that massage therapy with a duration of at least 5 weeks significantly improved pain , anxiety, and depression. Sleep disturbance was not improved by massage therapy.
The authors conclude that massage therapy with duration ≥5 weeks had beneficial immediate effects on improving pain, anxiety, and depression in patients with FM. Massage therapy should be one of the viable complementary and alternative treatments for FM. However, given fewer eligible studies in subgroup meta-analyses and no evidence on follow-up effects, large-scale randomized controlled trials with long follow-up are warrant to confirm the current findings.
To put these results into context, we need to consider the often poor methodological quality of the primary studies. It is, of course, not easy to test massage therapy in rigorous trials. For instance, there is no obvious placebo, and we can therefore not be sure whether the treatment benefits patients through a specific effect or whether non-specific effects are the cause of the improvement.
We also should be aware of the facts that for most other alternative therapies the evidence is not encouraging, and that massage therapy is relatively safe. Therefore the conclusion for those who suffer from FM might well be that massage therapy is worth a try.
Indian researchers published a survey aimed at determining the practice of prescription by homeopathic undergraduate students. A cross-sectional study was carried out involving all the students from 4 government homeopathic schools of West Bengal, India. Data were collected using self-administered questionnaires.
A total of 328 forms were completed. 80.5% of all homeopathic undergraduate students admitted prescribing homeopathic medicines independently and 40.5% said that they did this 2-3 times a year. The most common reasons for this activity were ‘urgency of the problem’ (35.2%), ‘previous experience with same kind of illness’ (31.8%), and ‘the problem too trivial to go to a doctor’ (25.8%). About 63.4% of the students thought that it was alright to independently diagnose an illness, while 51.2% thought that it was alright for them to prescribe medicines to others. Common conditions encountered were fever, indigestion, and injury. Prescription by students gradually increased with academic years of homeopathic schools. Many students thought it was alright for students to diagnose and treat illnesses.
The authors conclude that prescription of medicines by homeopathic undergraduate students is quite rampant and corrective measures are warranted.
It’s hard to know whether to laugh or cry about these findings:
- If you are a homeopath, you ought to be upset to hear that students who are obviously neither fully trained, qualified or licensed already prescribe medicines.
- If you are aware of the fact that homeopathic remedies are pure placebos, you might laugh about all this thinking “who cares?”
- If you are into public health, you will worry that homeopaths are obviously being taught that homeopathic remedies can treat conditions which are considered to be urgent.
- If you are someone who believes that sick people need evidence-based treatments, you might want to change the authors’ conclusion into something like: prescription of medicines by homeopaths is quite rampant and, in the interest of patients, corrective measures are required to stop them.
The aim of this survey was to investigate the use of alternative medicines (AMs) by Scottish healthcare professionals involved in the care of pregnant women, and to identify predictors of usage.
135 professionals (midwives, obstetricians, anaesthetists) involved in the care of pregnant women filled a questionnaire. A response rate of 87% was achieved. A third of respondents (32.5%) had recommended (prescribed, referred, or advised) the use of AMs to pregnant women. The most frequently recommended AMs modalities were: vitamins and minerals (excluding folic acid) (55%); massage (53%); homeopathy (50%); acupuncture (32%); yoga (32%); reflexology (26%); aromatherapy (24%); and herbal medicine (21%). Univariate analysis identified that those who recommended AMs were significantly more likely to be midwives who had been in post for more than 5 years, had received training in AMs, were interested in AMs, and were themselves users of AMs. However, the only variable retained in bivariate logistic regression was ‘personal use of AM’ (odds ratio of 8.2).
The authors draw the following conclusion: Despite the lack of safety or efficacy data, a wide variety of AM therapies are recommended to pregnant women by approximately a third of healthcare professionals, with those recommending the use of AMs being eight times more likely to be personal AM users.
There are virtually thousands of websites which recommend unproven treatments to pregnant women. This one may stand for the rest:
Chamomile, lemon balm, peppermint, and raspberry leaf are also effective in treating morning sickness. Other helpful herbs for pregnancy discomforts include:
- dandelion leaf for water retention
- lavender, mint, and slippery elm for heartburn
- butcher’s broom, hawthorn, and yarrow, applied externally to varicose veins
- garlic for high blood pressure
- witch hazel, applied externally to haemorrhoids.
Our research has shown that midwives are particularly keen to recommend and often sell AMs to their patients. In fact, it would be difficult to find a midwife in the UK or elsewhere who is not involved in this sort of thing. Similarly, we have demonstrated that the advice given by herbalists is frequently not based on evidence and prone to harm the unborn child, the mother or both. Finally, we have pointed out that many of the AMs in question are by no means free of risks.
The most serious risk, I think, is that advice to use AM for health problems during pregnancy might delay adequate care for potentially serious conditions. For instance, the site quoted above advocates garlic for a pregnant women who develops high blood pressure during pregnancy and dandelion for water retention. These two abnormalities happen to be early signs that a pregnant women might be starting to develop eclampsia. Treating such serious conditions with a few unproven herbal remedies is dangerous and recommendations to do so are irresponsible.
I think the new survey discussed above suggests a worrying degree of sympathy amongst conventional healthcare professionals for unproven treatments. This is likely to render healthcare less effective and less safe and is not in the interest of patients.
Do you think that chiropractic is effective for asthma? I don’t – in fact, I know it isn’t because, in 2009, I have published a systematic review of the available RCTs which showed quite clearly that the best evidence suggested chiropractic was ineffective for that condition.
But this is clearly not true, might some enthusiasts reply. What is more, they can even refer to a 2010 systematic review which indicates that chiropractic is effective; its conclusions speak a very clear language: …the eight retrieved studies indicated that chiropractic care showed improvements in subjective measures and, to a lesser degree objective measures… How on earth can this be?
I would not be surprised, if chiropractors claimed the discrepancy is due to the fact that Prof Ernst is biased. Others might point out that the more recent review includes more studies and thus ought to be more reliable. The newer review does, in fact, have about twice the number of studies than mine.
How come? Were plenty of new RCTs published during the 12 months that lay between the two publications? The answer is NO. But why then the discrepant conclusions?
The answer is much less puzzling than you might think. The ‘alchemists of alternative medicine’ regularly succeed in smuggling non-evidence into such reviews in order to beautify the overall picture and confirm their wishful thinking. The case of chiropractic for asthma does by no means stand alone, but it is a classic example of how we are being misled by charlatans.
Anyone who reads the full text of the two reviews mentioned above will find that they do, in fact, include exactly the same amount of RCTs. The reason why they arrive at different conclusions is simple: the enthusiasts’ review added NON-EVIDENCE to the existing RCTs. To be precise, the authors included one case series, one case study, one survey, two randomized controlled trials (RCTs), one randomized patient and observer blinded cross-over trial, one single blind cross study design, and one self-reported impairment questionnaire.
Now, there is nothing wrong with case reports, case series, or surveys – except THEY TELL US NOTHING ABOUT EFFECTIVENESS. I would bet my last shirt that the authors know all of that; yet they make fairly firm and positive conclusions about effectiveness. As the RCT-results collectively happen to be negative, they even pretend that case reports etc. outweigh the findings of RCTs.
And why do they do that? Because they are interested in the truth, or because they don’t mind using alchemy in order to mislead us? Your guess is as good as mine.
Systematic reviews are widely considered to be the most reliable type of evidence for judging the effectiveness of therapeutic interventions. Such reviews should be focused on a well-defined research question and identify, critically appraise and synthesize the totality of the high quality research evidence relevant to that question. Often it is possible to pool the data from individual studies and thus create a new numerical result of the existing evidence; in this case, we speak of a meta-analysis, a sub-category of systematic reviews.
One strength of systematic review is that they minimise selection and random biases by considering at the totality of the evidence of a pre-defined nature and quality. A crucial precondition, however, is that the quality of the primary studies is critically assessed. If this is done well, the researchers will usually be able to determine how robust any given result is, and whether high quality trials generate similar findings as those of lower quality. If there is a discrepancy between findings from rigorous and flimsy studies, it is obviously advisable to trust the former and discard the latter.
And this is where systematic reviews of alternative treatments can run into difficulties. For any given research question in this area we usually have a paucity of primary studies. Equally important is the fact that many of the available trials tend to be of low quality. Consequently, there often is a lack of high quality studies, and this makes it all the more important to include a robust critical evaluation of the primary data. Not doing so would render the overall result of the review less than reliable – in fact, such a paper would not qualify as a systematic review at all; it would be a pseudo-systematic review, i.e. a review which pretends to be systematic but, in fact, is not. Such papers are a menace in that they can seriously mislead us, particularly if we are not familiar with the essential requirements for a reliable review.
This is precisely where some promoters of bogus treatments seem to see their opportunity of making their unproven therapy look as though it was evidence-based. Pseudo-systematic reviews can be manipulated to yield a desired outcome. In my last post, I have shown that this can be done by including treatments which are effective so that an ineffective therapy appears effective (“chiropractic is so much more than just spinal manipulation”). An even simpler method is to exclude some of the studies that contradict one’s belief from the review. Obviously, the review would then not comprise the totality of the available evidence. But, unless the reader bothers to do a considerable amount of research, he/she would be highly unlikely to notice. All one needs to do is to smuggle the paper past the peer-review process – hardly a difficult task, given the plethora of alternative medicine journals that bend over backwards to publish any rubbish as long as it promotes alternative medicine.
Alternatively (or in addition) one can save oneself a lot of work and omit the process of critically evaluating the primary studies. This method is increasingly popular in alternative medicine. It is a fool-proof method of generating a false-positive overall result. As poor quality trials have a tendency to deliver false-positive results, it is obvious that a predominance of flimsy studies must create a false-positive result.
A particularly notorious example of a pseudo-systematic review that used this as well as most of the other tricks for misleading the reader is the famous ‘systematic’ review by Bronfort et al. It was commissioned by the UK GENERAL CHIROPRACTIC COUNCIL after the chiropractic profession got into trouble and was keen to defend those bogus treatments disclosed by Simon Singh. Bronfort and his colleagues thus swiftly published (of course, in a chiro-journal) an all-encompassing review attempting to show that, at least for some conditions, chiropractic was effective. Its lengthy conclusions seemed encouraging: Spinal manipulation/mobilization is effective in adults for: acute, subacute, and chronic low back pain; migraine and cervicogenic headache; cervicogenic dizziness; manipulation/mobilization is effective for several extremity joint conditions; and thoracic manipulation/mobilization is effective for acute/subacute neck pain. The evidence is inconclusive for cervical manipulation/mobilization alone for neck pain of any duration, and for manipulation/mobilization for mid back pain, sciatica, tension-type headache, coccydynia, temporomandibular joint disorders, fibromyalgia, premenstrual syndrome, and pneumonia in older adults. Spinal manipulation is not effective for asthma and dysmenorrhea when compared to sham manipulation, or for Stage 1 hypertension when added to an antihypertensive diet. In children, the evidence is inconclusive regarding the effectiveness for otitis media and enuresis, and it is not effective for infantile colic and asthma when compared to sham manipulation. Massage is effective in adults for chronic low back pain and chronic neck pain. The evidence is inconclusive for knee osteoarthritis, fibromyalgia, myofascial pain syndrome, migraine headache, and premenstrual syndrome. In children, the evidence is inconclusive for asthma and infantile colic.
Chiropractors across the world cite this paper as evidence that chiropractic has at least some evidence base. What they omit to tell us (perhaps because they do not appreciate it themselves) is the fact that Bronfort et al
- failed to formulate a focussed research question,
- invented his own categories of inconclusive findings,
- included all sorts of studies which had nothing to do with chiropractic,
- and did not to make an assessment of the quality of the included primary studies they included in their review.
If, for a certain condition, three trials were included, for instance, two of which were positive but of poor quality and one was negative but of good quality, the authors would conclude that, overall, there is sound evidence.
Bronfort himself is, of course, more than likely to know all that (he has learnt his trade with an excellent Dutch research team and published several high quality reviews) – but his readers mostly don’t. And for chiropractors, this ‘systematic’ review is now considered to be the most reliable evidence in their field.