Some practitioners of alternative medicine (doctors, naturopaths, chiropractors and others) earn a lot of money with the claim that chelation therapy (an effective mainstream treatment for acute heavy metal poisoning) is an effective means to treat cardiovascular disease. However, the notion is controversial and implausible. Several systematic reviews of the best evidence concluded less than optimistically:
More recently, important new evidence has emerged. The largest study of chelation therapy (TACT) ever conducted cost ~ $ 30 million and concluded that among stable patients with a history of MI, use of an intravenous chelation regimen with disodium EDTA, compared with placebo, modestly reduced the risk of adverse cardiovascular outcomes, many of which were revascularization procedures. These results provide evidence to guide further research but are not sufficient to support the routine use of chelation therapy for treatment of patients who have had an MI.
At the time, the TACT trial was heavily and rightly criticised for a whole host of reasons. For instance, because of the result of the FDA inspection of the highest accruing TACT site:
- The investigators didn’t conduct the investigation in accordance with the signed statement and investigational plan. Several examples were given of shoddy procedures, prefilled forms, and failure to train personnel.
- Failure to report promptly to the IRB all unanticipated problems involving risk to human subjects or others. Examples are given, including failure to report the deaths of patients on the study in a timely fashion (in one case the death wasn’t reported to the IRB until four months later; in another case it was never reported at all). In other cases, adverse event reports were not submitted to the IRB.
- Failure to prepare or maintain adequate case histories with respect to observations and data pertinent to the investigation.
- Investigational drug disposition records are not adequate with respect to dates, quantity, and use by subjects.
Despite these problems, the study was published in JAMA, albeit with a very critical editorial:
Differential dropout in TACT suggests unmasking, but the problem of intentional unblinding is more concerning. The sponsors of the trial, the National Heart, Lung, and Blood Institute (NHLBI) and the National Center for Complementary and Alternative Medicine (NCCAM), were unblinded throughout the trial. The National Institutes of Health policy unwisely allows the sponsor access to unblinded trial data, and both organizations sent observers to the closed sessions of the data monitoring committee. This gave them access to confidential data during each of the 11 interim analyses. The unblinding of the study sponsor represents a serious deviation from acceptable standards of conduct for supervision of clinical trials. If a pharmaceutical company sponsoring a trial were allowed access to actual outcome data during the study, there would be major objections. Like any sponsor, the NHLBI and NCCAM cannot be considered unbiased observers. These agencies made major financial commitments to the trial and may intentionally or inadvertently influence study conduct if inappropriately unblinded during the study…
Given the numerous concerns with this expensive, federally funded clinical trial, including missing data, potential investigator or patient unmasking, use of subjective end points, and intentional unblinding of the sponsor, the results cannot be accepted as reliable and do not demonstrate a benefit of chelation therapy. The findings of TACT should not be used as a justification for increased use of this controversial therapy.
Orac, makes several further critical points about the published trial:
First, the primary endpoint (i.e., the aggregated serious cardiovascular events) did indeed show a modest difference, namely 30% of placebo subjects versus 26.5% of the EDTA chelation subjects (hazard ratio 0.82 for chelation). However, one notes that the result is just barely statistically significant, p = 0.035, with the 99% confidence interval for the hazard ratio ranging from 0.69 to 0.99. (The predetermined level for statistical significance for purposes of this study was 0.036; so this is statistically significant by the barest margin.) More importantly, if you look at the individual endpoints that make up that aggregate, there was no statistically significant difference in death, myocardial infarction, stroke, coronary revascularization, and hospitalization for angina. Subgroup analysis (always a questionable analysis that requires replication, even when preplanned, as in TACT) purported to show a much greater benefit for diabetics, with a hazard ratio of 0.61 (p=0.002), while patients without diabetes showed no statistically significant difference in any of the outcome measures, including the aggregated total bad outcomes.
Now a paper that has just emerged describes the intent-to-treat comparison of this trial in patients with diabetes.
This was a double-blind, placebo-controlled, 2 × 2 factorial multicenter randomized trial of 1,708 post-myocardial infarction (MI) patients ≥50 years of age and with creatinine ≤2.0 mg/dL randomized to receive 40 EDTA chelation or placebo infusions plus 6 caplets daily of a 28-component multivitamin-multimineral mixture or placebo. The primary end point was a composite of total mortality, MI, stroke, coronary revascularization, or hospitalization for angina.
Median age was 65 years, 18% were female, 94% were Caucasian, 37% were diabetic, 83% had prior coronary revascularization, and 73% were on statins. Five-year Kaplan-Meier estimates for the primary end point was 31.9% in the chelation + high-dose vitamin group, 33.7% in the chelation + placebo vitamin group, 36.6% in the placebo infusion + active vitamin group, and 40.2% in the placebo infusions + placebo vitamin group. The reduction in primary end point by double active treatment compared with double placebo was significant (hazard ratio 0.74, 95% CI 0.57-0.95, P = .016). In patients with diabetes, the primary end point reduction of double active compared with double placebo was more pronounced (hazard ratio 0.49, 95% CI 0.33-0.75, P < .001).
The authors conclude that in stable post-MI patients on evidence-based medical therapy, the combination of oral high-dose vitamins and chelation therapy compared with double placebo reduced clinically important cardiovascular events to an extent that was both statistically significant and of potential clinical relevance.
I fear that these conclusions are erroneous and misleading: the marginally positive finding might have nothing to do with chelation per se; most likely they are due to the fact that the ‘vitamin’ mixture administered along with chelation contained ingredients like heparin and procaine which are potentially beneficial for cardiovascular conditions. Moreover, the placebo contained a considerable amount of glucose which could easily explain the better outcome of the diabetic subgroup receiving the verum – in other words, the verum generated better results not because it was effective but because the ‘placebo’ had detrimental effects.
If you are pregnant, a ‘breech presentation’ is not good news. It occurs when the fetus presents ‘bottom-down’ in the uterus. There are three types:
- Breech with extended legs (frank) – 85% of cases
- Breech with fully flexed legs (complete)
- Footling (incomplete) with one or both thighs extended
The significance of breech presentation is its association with higher perinatal mortality and morbidity when compared to cephalic presentations. This is due both to pre-existing congenital malformation, increased incidence of breech in premature deliveries and increased risk of intrapartum trauma or asphyxia. Caesarean section has been adopted as the ‘normal’ mode of delivery for term breech presentations in Europe and the USA, as the consensus is that this reduces the risk of birth-related complications.
But Caesarian section is also not a desirable procedure. Something far less invasive would be much more preferable, of course. This is where the TCM-practitioners come in. They claim they have the solution: moxibustion, i.e. the stimulation of acupuncture points by heat. But does it really work? Can it turn the fetus into the correct position?
This new study aimed to assess the efficacy of moxibustion (heating of the acupuncture needle with an igniting charcoal moxa stick) with acupuncture for version of breech presentations to reduce their rate at 37 weeks of gestation and at delivery. It was a randomized, placebo-controlled, single-blind trial including 328 pregnant women recruited in a university hospital center between 33 4/7 and 35 4/7 weeks of gestation. Moxibustion with acupuncture or inactivated laser (placebo) treatment was applied to point BL 67 for 6 sessions. The principal endpoint was the percentage of fetuses in breech presentation at 37 2/7 weeks of gestation.
The results show that the percentage of fetuses in breech presentation at 37 2/7 weeks of gestation was not significantly different in both groups (72.0 in the moxibustion with acupuncture group compared with 63.4% in the placebo group).
The authors concluded that treatment by moxibustion with acupuncture was not effective in correcting breech presentation in the third trimester of pregnancy.
You might well ask why on earth anyone expected that stimulating an acupuncture point would turn a fetus in the mother’s uterus into the optimal position that carries the least risk during the process of giving birth. This is what proponents of this technique say about this approach:
During a TCM consultation to turn a breech baby the practitioner will take a comprehensive case history, make a diagnosis and apply the appropriate acupuncture treatment. They will assess if moxibustion might be helpful. Practitioners will then instruct women on how to locate the appropriate acupuncture points and demonstrate how to safely apply moxa at home. The acupuncture point UB 67 is the primary point selected for use because it is the most dynamic point to activate the uterus. Its forte is in turning malpositioned babies. It is located on the outer, lower edge of both little toenails. According to TCM theory, moxa has a tonifying and warming effect which promotes movement and activity. The nature of heat is also rising. This warming and raising effect is utilised to encourage the baby to become more active and lift its bottom up in order to gain adequate momentum to summersault into the head down position. This technique can also be used to reposition transverse presentation, a situation where the baby’s has its shoulder or back pointing down, or is lying sideways across the abdomen.
Not convinced? I can’t say I blame you!
Clearly, we need to know what the totality of the most reliable evidence shows; and what better than a Cochrane review to inform us about it? Here is what it tells us:
Moxibustion was not found to reduce the number of non-cephalic presentations at birth compared with no treatment (P = 0.45). Moxibustion resulted in decreased use of oxytocin before or during labour for women who had vaginal deliveries compared with no treatment (risk ratio (RR) 0.28, 95% confidence interval (CI) 0.13 to 0.60). Moxibustion was found to result in fewer non-cephalic presentations at birth compared with acupuncture (RR 0.25, 95% CI 0.09 to 0.72). When combined with acupuncture, moxibustion resulted in fewer non-cephalic presentations at birth (RR 0.73, 95% CI 0.57 to 0.94), and fewer births by caesarean section (RR 0.79, 95% CI 0.64 to 0.98) compared with no treatment. When combined with a postural technique, moxibustion was found to result in fewer non-cephalic presentations at birth compared with the postural technique alone (RR 0.26, 95% CI 0.12 to 0.56).
In other words, there is indeed some encouraging albeit not convincing evidence! How can this be? There is no plausible explanation why this treatment should work!
But there is a highly plausible explanation why the results of many of the relevant trials are false-positive thus rendering a meta-analysis false-positive as well. I have repeatedly pointed out on this blog that practically all Chinese TCM-studies report (false) positive results; and many of the studies included in this review were done in China. The Cochrane review provides a strong hint about the lack of rigor in its ‘plain language summary’:
The included trials were of moderate methodological quality, sample sizes in some of the studies were small, how the treatment was applied differed and reporting was limited. While the results were combined they should be interpreted with caution due to the differences in the included studies. More evidence is needed concerning the benefits and safety of moxibustion.
So, would I recommend moxibustion for breech conversion? I don’t think so!
Many proponents of chiropractic claim that chiropractic spinal manipulation therapy (SMT) for chronic low back pain (LBP) might save health care cost. As LBP is a hugely expensive condition, this is a mighty important question. The evidence on this issue is, however, flimsy to say the least. Most experts seem to conclude that more reliable data are needed. On this background, it seems relevant to note that a new relevant study has just become available.
The purpose of this analysis was to report the incremental costs and benefits of different doses of SMT in patients with LBP.
The researchers randomized 400 patients with chronic LBP to receive doses of 0, 6, 12, or 18 sessions of SMT. Patients were scheduled for 18 visits for 6 weeks and received SMT or light massage control from a chiropractor. Societal costs in the year after study enrollment were estimated using patient reports of health care use and lost productivity. The main health outcomes were the number of pain-free days and disability-free days.
The results show that costs for treatment and lost productivity ranged from $3398 for 12 SMT sessions to $3815 for 0 SMT sessions with no statistically significant differences between groups. Baseline patient characteristics related to increase in costs were greater age, greater disability, lower quality-adjusted life year scores, and higher costs in the period preceding enrolment. Pain-free and disability-free days were greater for all SMT doses compared with control, but only SMT 12 yielded a statistically significant benefit of 22.9 pain-free days and 19.8 disability-free days. No statistically significant group differences in quality-adjusted life years were noted.
The authors drew the following conclusions from these data: a dose of 12 SMT sessions yielded a modest benefit in pain-free and disability-free days. Care of chronic LBP with SMT did not increase the costs of treatment plus lost productivity.
So, is chiropractic SMT for LBP cost-effective? I leave it to my readers to answer this question.
There are many terms for this type of treatment: energy healing, Therapeutic Touch, Reiki, spiritual healing and para-normal healing are just some of the better-known ones. These interventions are based of the belief that some sort of ‘energy’ can be channelled by the healer into the body of the patient to assist its capacity for self-healing. Needless to say that their biological plausibility is suspiciously close to zero.
This new study was aimed at testing the effectiveness of energy healing on the well-being of patients and at assessing the influence on the results of participating in a randomized controlled trial. A total of 247 colorectal cancer patients were included in the trial. One half of them were randomized to either:
- healing (RH) or
- control (RC)
The other half of the patients was not randomized and had either:
- self-selected healing (SH) or
- self-selected control condition (SC)
All patients completed questionnaires assessing well-being Quality of Life (QoL), depressive symptoms, mood, and sleep quality), attitude toward complementary and alternative medicine (CAM), and faith/spirituality at baseline, 1 week, and 2 months post-intervention. Patients were also asked to indicate, at baseline, whether they considered QoL, depressive symptoms, mood, and sleep quality as important outcomes.
Compared with controls, no overall effect of healing were noted on QoL, depressive symptoms, mood, or sleep quality in the intervention groups (RH, SH). Effects of healing on mood were only found for patients who initially had a positive attitude toward CAM and considered the outcome in question as important.
The authors of this study arrived at the following conclusions: Whereas it is generally assumed that CAMs such as healing have beneficial effects on well-being, our results indicated no overall effectiveness of energy healing on QoL, depressive symptoms, mood, and sleep quality in colorectal cancer patients. Effectiveness of healing on well-being was, however, related to factors such as self-selection and a positive attitude toward the treatment.
Survey after survey shows that ‘energy healing’ is popular amongst cancer patients. But medicine is no popularity contest, and the existing clinical trials have mostly failed to show that these treatments work beyond a sometimes remarkably strong placebo-effect. Consequently, several systematic reviews have arrived at conclusions that were far from positive:
Since the publication of our previous systematic review in 2000, several rigorous new studies have emerged. Collectively they shift the weight of the evidence against the notion that distant healing is more than a placebo
This new and fairly rigorous trial clearly points in the same direction. Thus we a faced with the fact that these treatments are:
- utterly implausible
- not supported by good clinical evidence
What follows seems as simple as it is indisputable: energy healing is nonsense and does not merit further research.
Traditional Chinese medicine (TCM) is often promoted as an effective therapy for cancer, and are numerous controlled clinical studies published in Chinese literature, yet no systematic analysis has been done of this body of evidence. This systematic review summarizes the evidence from controlled clinical studies published in Chinese on this subject.
The researchers looked for controlled clinical studies of TCM therapies for all kinds of cancers published in Chinese in four main Chinese electronic databases and found 2964 reports including 2385 randomized clinical trials and 579 non-randomized controlled studies.
The top seven cancer types treated were lung cancer, liver cancer, stomach cancer, breast cancer, esophagus cancer, colorectal cancer and nasopharyngeal cancer. The majority of studies (72%) applied TCM therapy combined with conventional treatments, whilst fewer (28%) applied only TCM therapy in the experimental groups. Herbal medicine was the most frequently tested TCM therapy (2677 studies, 90.32%).
The most frequently reported outcome was clinical symptom improvement (1667 studies, 56.24%) followed by biomarker indices (1270 studies, 42.85%), quality of life (1129 studies, 38.09%), chemo/radiotherapy induced side effects (1094 studies, 36.91%), tumor size (869 studies, 29.32%) and safety (547 studies, 18.45%). Completeness and adequacy of reporting appeared to improve with time.
The authors of this paper drew the following conclusion: data from controlled clinical studies of TCM therapies in cancer treatment is substantial, and different therapies are applied either as monotherapy or in combination with conventional medicine. Reporting of controlled clinical studies should be improved based on the CONSORT and TREND Statements in future. Further studies should address the most frequently used TCM therapy for common cancers and outcome measures should address survival, relapse/metastasis and quality of life.
Almost 3000 controlled clinical trials! This number is likely to impress many people – unless, of course, one knows that the quality of these studies is dismal. Interestingly, no formal assessment of study quality was included in this analysis. But it was mentioned that only 63 of these trials reported patient-blinding, and only 5 were deemed to be “relatively well designed” by the authors of this paper (who, incidentally, are strong proponents of TCM).
What I find the most interesting aspect of this article is the fact that the authors fail to mention how many of the studies reported a positive result – in a way, they don’t need to: there is plenty of evidence to show that virtually all of the Chinese studies of TCM are positive. In my view, this invalidates this body of evidence completely.
Analysis like the present one tend to lead us up the garden path. They suggest that there is a realistic hope for effective new treatments hidden in this difficult to access, large amount of data. This might lead other researchers to try to replicate some of the original studies. I fear that they would be wasting their time. From all I know, they are irreproducible.
I find it always nice to see that people appreciate my work. Yet sometimes I am a little surprised to realise what some commercially interested firms make of it. Recently I came across a website that proudly used my research for advertising the use of magnetic bracelets against pain. Here is the text in question:
The extra strong magnets make this magnetic bracelet the fastest acting pain reliever. While wearing this magnetic bracelet customers with wrist and hand pain report significant pain relief….
What is a magnetic bracelet and what are the benefits? Magnetic bracelets are a piece of jewelry, worn for the therapeutic benefits of the magnetic field. Magnetic bracelets has been used successfully by many people for pain relief of inflammatory conditions such as arthritis, tendinitis and bursitis.
A randomized, placebo controlled trial with three parallel groups, came to the conclusion : Pain from osteoarthritis of the hip and knee decreases when wearing magnetic bracelets. It is uncertain whether this response is due to specific or non-specific (placebo) effects. Tim Harlow, general practitioner, Colin Greaves, research fellow, Adrian White, senior research fellow, Liz Brown, research assistant, Anna Hart, statistician, Edzard Ernst, professor of complementary medicine.
The entrepreneurs seem to have forgotten a few things which we tried to make clear in our paper:
- this article was published in the Christmas issue of the BMJ which specialises in publishing unusual and odd findings with a high entertainment value,
- in our paper, we point out that “the contamination of group B with stronger magnets prevented a more objective estimation of any-placebo effect”,
- and stressed that “there were problems with the weak magnets”,
- and that “a per-specification analysis suggested (but could not confirm) a specific effect of magnetic bracelets over and above placebo”.
Most importantly, this was just one trial, and surely one swallow does not make a summer! We should always consider the totality of the reliable evidence. Being conscientious researchers, at the time, we did exactly that and conducted a systematic review. Here is the abstract in its full beauty:
Static magnets are marketed with claims of effectiveness for reducing pain, although evidence of scientific principles or biological mechanisms to support such claims is limited. We performed a systematic review and meta-analysis to assess the clinical evidence from randomized trials of static magnets for treating pain.
Systematic literature searches were conducted from inception to March 2007 for the following data sources: MEDLINE, EMBASE, AMED (Allied and Complementary Medicine Database), CINAHL, Scopus, the Cochrane Library and the UK National Research Register. All randomized clinical trials of static magnets for treating pain from any cause were considered. Trials were included only if they involved a placebo control or a weak magnet as the control, with pain as an outcome measure. The mean change in pain, as measured on a 100-mm visual analogue scale, was defined as the primary outcome and was used to assess the difference between static magnets and placebo.
Twenty-nine potentially relevant trials were identified. Nine randomized placebo-controlled trials assessing pain with a visual analogue scale were included in the main meta-analysis; analysis of these trials suggested no significant difference in pain reduction (weighted mean difference [on a 100-mm visual analogue scale] 2.1 mm, 95% confidence interval -1.8 to 5.9 mm, p = 0.29). This result was corroborated by sensitivity analyses excluding trials of acute effects and conditions other than musculoskeletal conditions. Analysis of trials that assessed pain with different scales suggested significant heterogeneity among the trials, which means that pooling these data is unreliable.
The evidence does not support the use of static magnets for pain relief, and therefore magnets cannot be recommended as an effective treatment. For osteoarthritis, the evidence is insufficient to exclude a clinically important benefit, which creates an opportunity for further investigation.
So, would I, on the basis of the current best evidence, recommend magnetic bracelets to people who suffer from pain? No! In my view, only charlatans would do such a thing.
Auricular acupuncture (AA), according to the ‘COLLEGE OF AURICULAR ACUPUNCTURE’, has its origins in Modern Europe. In 1957 Dr. Paul Nogier, a neurologist from Lyons in France, observed a locum doctor treating sciatica by cauterizing an area of the ear. This prompted extensive research culminating in the development of the somatopic correspondence of specific parts of the body to the ear based upon the concept of an inverted foetus. Dr. Nogier believed that pain and other symptoms in the body could be alleviated by needling, massaging or electronically stimulating the corresponding region of the ear. Auricular Acupuncture is a specialized complementary therapy where acupuncture points on the outer ear are treated, using either needles or acupunctoscopes (electrical location and stimulation machines) to help relieve many chronic complaints. There are over 200 acupuncture points on the ear, each point named after an area of our anatomy. The outer ear acts like a switchboard to the brain. Each acupuncture point being treated, triggers electrical impulses from the ear via the brain, to the specific part of the body being treated.
Sounds odd? Well, that’s because it is odd!
But just because something is odd does not mean it is ineffective – so, what does the reliable evidence tell us? Here are some conclusions from systematic reviews:
All of these analyses point out that the quality of the studies is usually very poor, and stress that more and better research is required. It is therefore interesting to note that a new study has just been published. Perhaps it could settle the question about the effectiveness of AA?
The aim of this study was 1) to evaluate whether auricular acupuncture effective for reducing health care provider stress and anxiety and 2) to determine, if auricular acupuncture impacts provider capacity for developing caring relationships with patients. Pre-intervention and post-intervention surveys were evaluated to see, if auricular acupuncture was associated with changes in State-Trait Anxiety Inventory (STAI), Professional Quality of Life, and Caring Ability Inventory scores. The results indicate that, compared with baseline, participants had a significant reduction in state anxiety (STAI), trait anxiety (STAI), burnout, and secondary traumatic stress scores (Professional Quality of Life). Significant increases were noted in courage and patience, two dimensions of the Caring Ability Inventory.
From these findings, the authors conclude that auricular acupuncture is an effective intervention for the relief of stress/anxiety in providers and supports heightened capacity for caring.
Sounds odd again? Yes, because it is odd!
I would argue that a study of any controversial therapy that has already been tested repeatedly in poor quality trials must have sufficient scientific rigor to advance the field of inquiry. If it does not fulfil this criterion, it is quite simply not ethical. The new study does not even have a control group; we can therefore not begin to tell whether the observed outcomes were due to non-specific effects, the natural history of the condition or regression towards the mean (to mention but a few of the possible sources of bias). To conclude that AA is ‘an effective intervention’ is therefore utterly barmy.
All of this could be entirely trivial and inconsequential. I am afraid, however, that it is not. Alternative medicine is littered with such unethically flawed research conducted by naïve and clueless pseudo-scientists who arrive at outrageous conclusions. This relentless flow of false-positive findings misleads consumers, health care professionals, decision makers and everyone else to draw the wrong conclusions about bogus therapies. And, in the end, this sort of thing even does a grave disfavour to any branch of alternative medicine that might have some degree of respectability.
IT IS HIGH TIME THAT THIS NONSENSE STOPS! IT BORDERS ON SCIENTIFIC MISCONDUCT.
One alternative therapy that I have so far neglected on this blog is the Alexander Technique (AT). Actually, it was not really meant to become an alternative therapy when it was first discovered.
F.M. Alexander (1869-1955), an Australian actor, often experienced chronic laryngitis while performing. As his doctors could not help him, he developed a solution on his own. He found that excess muscular tension in his neck and body were causing his problems, and began to experiment on new ways to speak and move with greater ease. His health improved to such an extent that he decided to teach others what he had learned. Over a career span of more than fifty years, he refined his methods. After teaching for over 35 years, he began to train teachers of the ‘Alexander Technique’.
As used in alternative medicine, AT is an educational method aimed at increased sensory awareness and kinesthetic control to modify postural and movement patterns which might be associated with musculoskeletal problems. Proponents claim AT works for a range of conditions, including:
It has been suggested that AT is effective for chronic low back pain; however, so far only one recent study has determined its effects on chronic non-specific neck pain.
In this randomized controlled trial, patients were randomly allocated to either 5 weekly sessions of AT, heat pack application (HEAT) or guided imagery (GI). The primary outcome measure was the neck pain intensity on a 100-mm visual analogue scale at week 5. Secondary outcomes included neck disability, quality of life, satisfaction and adverse effects. Analyses of covariance were applied on an intention-to-treat population testing ordered hypotheses AT vs. HEAT and AT vs. GI.
A total of 72 patients were included, and 52 of them received all 5 interventions. No significant group difference was found for neck pain intensity when AT was compared to HEAT. However, an exploratory analysis revealed superiority of AT over GI. Significant group differences were also found for physical quality of life in favor of AT vs. HEAT or GI. Adverse events mainly related to slightly increased pain and muscle soreness. AT patients reported increased body awareness and control over the body, relaxing or stimulating effects and mood changes after the sessions.
The authors conclude that 5 sessions of AT were no better than a heat pack application for relieving chronic non-specific neck pain. Therefore it cannot be recommended as routine intervention at this time. Since exploratory analysis revealed some improvements of AT further trials are warranted for conclusive judgment.
One of the most irritating things with alternative medicine research, in my view, is the phenomenon that researchers tend to be quasi-religious advocates of the treatment they investigate. This seems to compel them all too often to extrapolate beyond reason and to drawing conclusions which are way too optimistic, frequently to an extend that borders on scientific misconduct. It is therefore a real pleasure to find an article that does not fall into this trap. I commend the authors for reporting this RCT and for their wisdom of being adequately cautious when formulating their conclusions.
I only wished it would happen more often!
Imagine an area of therapeutics where 100% of all findings of hypothesis-testing research is positive, i.e. comes to the conclusion that the treatment in question is effective. Theoretically, this could mean that the therapy is a miracle cure which is useful for every single condition in every single setting. But sadly, there are no miracle cures. Therefore something must be badly and worryingly amiss with the research in an area that generates 100% positive results.
Acupuncture is such an area; we and others have shown that Chinese trials of acupuncture hardly ever produce a negative finding. In other words, one does not need to read the paper, one already knows that it is positive – even more extreme: one does not need to conduct the study, one already knows the result before the research has started. But you might not believe my research nor that of others. We might be chauvinist bastards who want to discredit Chinese science. In this case, you might perhaps believe Chinese researchers.
In this systematic review, all randomized controlled trials (RCTs) of acupuncture published in Chinese journals were identified by a team of Chinese scientists. A total of 840 RCTs were found, including 727 RCTs comparing acupuncture with conventional treatment, 51 RCTs with no treatment controls, and 62 RCTs with sham-acupuncture controls. Among theses 840 RCTs, 838 studies (99.8%) reported positive results from primary outcomes and two trials (0.2%) reported negative results. The percentages of RCTs concealment of the information on withdraws or sample size calculations were 43.7%, 5.9%, 4.9%, 9.9%, and 1.7% respectively.
The authors concluded that publication bias might be major issue in RCTs on acupuncture published in Chinese journals reported, which is related to high risk of bias. We suggest that all trials should be prospectively registered in international trial registry in future.
I applaud the authors’ courageous efforts to conduct this analysis, but I do not agree with their conclusion. The question why all Chinese acupuncture trials are positive has puzzled me since many years, and I have quizzed numerous Chinese colleagues why this might be so. The answer I received was uniformly that it would be very offensive for Chinese researchers to conceive a study that does not confirm the views held by their peers. In other words, acupuncture research in China is conducted to confirm the prior assumption that this treatment is effective. It seems obvious that this is an abuse of science which must cause confusion.
Whatever the reasons for the phenomenon, and we can only speculate about them, the fact has been independently confirmed several times and is now quite undeniable: acupuncture trials from China – and these constitute the majority of the evidence-base in this area – cannot be trusted. The only way to adequately deal with this problem that I can think of is to discard them outright.
Today, there are several dozens of journals publishing articles on alternative medicine. ‘The Journal of Alternative and Complementary Medicine’ is one of the best known, and it has one of the highest impact factors of them all. The current issue holds a few ‘gems’ which might be worthy of a comment or two. Here I have selected three articles reporting clinical studies, and I reproduce their abstracts (almost) in full (in italics) and add my comments (for clarity in bold). All the articles are available electronically, and I have provided the links for those who want to investigate beyond the abstracts.
STUDY No 1
The first ‘pilot study‘ was aimed to demonstrate the potential of auricular acupuncture (AAT) for insomnia in maintenance haemodialysis (MHD) patients and to prepare for a future randomized controlled trial.
Eligible patients were enrolled into this descriptive pilot study and received AAT designed to manage insomnia for 4 weeks. Questionnaires that used the Pittsburgh sleep quality index (PSQI) were completed at baseline, after a 4-week intervention, and 1 month after completion of treatment. Sleep quality and other clinical characteristics, including sleeping pills taken, were statistically compared between different time points.
A total of 22 patients were selected as eligible participants and completed the treatment and questionnaires. The mean global PSQI score was significantly decreased after AAT intervention (p<0.05). Participants reported improved sleep quality (p<0.01), shorter sleep latency (p<0.05), less sleep disturbance (p<0.01), and less daytime dysfunction (p=0.01). They also exhibited less dependency on sleep medications, indicated by the reduction in weekly estazolam consumption from 6.98±4.44 pills to 4.23±2.66 pills (p<0.01). However, these improvements were not preserved 1 month after treatment.
Conclusions: In this single-center pilot study, complementary AAT for MHD patients with severe insomnia was feasible and well tolerated and showed encouraging results for sleep quality.
In alternative medicine research, it has become far too common (almost generally accepted) to call a flimsy trial a ‘pilot study’. The authors give their game away by stating that, by conducting this trial, they want to ‘demonstrate the potential of AAT’. This is not a legitimate aim of research; science is for TESTING hypotheses, not for PROVING them!
The results of this trial show that patients experienced improvements after receiving AAT which, however, did not last. As there was no placebo control group, the most likely explanation for these outcomes would be that AAT generated a short-lasting placebo effect.
A sample size of 22 is, of course, far to small to allow any conclusions about the safety of the intervention. Despite these obvious facts, the authors seem convinced that AAT is both safe and effective.
STUDY No 2
The aim of the second study was to compare the therapeutic effect of Yamamoto new scalp acupuncture (YNSA), a recently developed microcupuncture system, with traditional acupuncture (TCA) for the prophylaxis and treatment of migraine headache.
In a randomized clinical trial, 80 patients with migraine headache were assigned to receive YNSA or TCA. A pain visual analogue scale (VAS) and migraine therapy assessment questionnaire (MTAQ) were completed before treatment, after 6 and 18 sections of treatment, and 1 month after completion of therapy.
All the recruited patients completed the study. Baseline characteristics were similar between the two groups. Frequency and severity of migraine attacks, nausea, the need for rescue treatment, and work absence rate decreased similarly in both groups. Recovery from headache and ability to continue daily activities 2 hours after medical treatment showed similar improvement in both groups (p>0.05).
Conclusions: Classic acupuncture and YNSA are similarly effective in the prophylaxis and treatment of migraine headache and may be considered as alternatives to pharmacotherapy.
This is what is technically called an ‘equivalence trial’, i.e. a study that compares an experimental treatment (YNSA) to one that is (assumed to be) effective. To demonstrate equivalence, such trials need to have large sample sizes, and this study is woefully underpowered. As it stands, the results show nothing meaningful at all; if anything, they suggest that both interventions were similarly useless.
STUDY No 3
The third study determined whether injection with hypertonic dextrose and morrhuate sodium (prolotherapy) using a pragmatic, clinically determined injection schedule for knee osteoarthritis (KOA) results in improved knee pain, function, and stiffness compared to baseline status.
The participants were 38 adults who had at least 3 months of symptomatic KOA and who were in the control groups of a prior prolotherapy randomized controlled trial (RCT) (Prior-Control), were ineligible for the RCT (Prior-Ineligible), or were eligible but declined the RCT (Prior-Declined).
The injection sessions at occurred at 1, 5, and 9 weeks with as-needed treatment at weeks 13 and 17. Extra-articular injections of 15% dextrose and 5% morrhuate sodium were done at peri-articular tendon and ligament insertions. A single intra-articular injection of 6 mL 25% dextrose was performed through an inferomedial approach.
The primary outcome measure was the validated Western Ontario McMaster University Osteoarthritis Index (WOMAC). The secondary outcome measure was the Knee Pain Scale and postprocedure opioid medication use and participant satisfaction.
The Prior-Declined group reported the most severe baseline WOMAC score (p=0.02). Compared to baseline status, participants in the Prior-Control group reported a score change of 12.4±3.5 points (19.5%, p=0.002). Prior-Decline and Prior-Ineligible groups improved by 19.4±7.0 (42.9%, p=0.05) and 17.8±3.9 (28.4%, p=0.008) points, respectively; 55.6% of Prior-Control, 75% of Prior-Decline, and 50% of Prior-Ineligible participants reported score improvement in excess of the 12-point minimal clinical important difference on the WOMAC measure. Postprocedure opioid medication resulted in rapid diminution of prolotherapy injection pain. Satisfaction was high and there were no adverse events.
Conclusions: Prolotherapy using dextrose and morrhuate sodium injections for participants with mild-to-severe KOA resulted in safe, significant, sustained improvement of WOMAC-based knee pain, function, and stiffness scores compared to baseline status.
This study had nothing that one might call a proper control group: all the three groups mentioned were treated with the experimental treatment. No attempt was made to control for even the most obvious biases: the observed effects could have been due to placebo or any other non-specific effects. The authors conclusions imply a causal relationship between the treatment and the outcome which is wrong. The notion that the experimental treatment is ‘safe’ is based on just 38 patients and therefore not reasonable.
All of this might seem rather trivial, and my comments could be viewed as a deliberate and vicious attempt to discredit one of the most respected journals of alternative medicine. Yet, considering that articles of this nature are more the rule than the exception in alternative medicine, I do think that this flagrant lack of scientific rigour is a relevant issue and has important implications.
As long as research in this area continues to be deeply flawed, as long as reviewers turn a blind eye to (or are not smart enough to detect) even the most obvious mistakes, as long as journal editors accept any rubbish in order to fill their pages, there is a great danger that we are being continuously being mislead about the supposed therapeutic value of alternative therapies.
Many who read this blog will, of course, have the capacity to think critically and might therefore not fall into the trap of accepting the conclusions of fatally flawed research. But many other people, including politicians, journalists and consumers, might not have the necessary appraisal skills and will thus not be able to tell that such studies can serve only one purpose: to popularise bogus treatments and thereby render health care less effective and more dangerous. Enthusiasts of alternative medicine are usually fully convinced that such studies amount to evidence and ram this pseudo-information down the throat of health care decision makers – the effects of such lobbying on public health can be disastrous.
And there is another downside to the publication of such dismal drivel: assuming (as I do) that not all of alternative medicine is completely useless, such embarrassingly poor research will inevitably have detrimental effects on the discipline of alternative medicine. After being exposed to a seemingly endless stream of pseudo-research, critics will eventually give up taking any of it seriously and might claim that none of it is worth the bother. In other words, those who conduct, accept or publish such nonsensical papers are not only endangering medical progress in general, they are also harming the very cause they try so desperately hard to advance.