Some practitioners of alternative medicine (doctors, naturopaths, chiropractors and others) earn a lot of money with the claim that chelation therapy (an effective mainstream treatment for acute heavy metal poisoning) is an effective means to treat cardiovascular disease. However, the notion is controversial and implausible. Several systematic reviews of the best evidence concluded less than optimistically:
More recently, important new evidence has emerged. The largest study of chelation therapy (TACT) ever conducted cost ~ $ 30 million and concluded that among stable patients with a history of MI, use of an intravenous chelation regimen with disodium EDTA, compared with placebo, modestly reduced the risk of adverse cardiovascular outcomes, many of which were revascularization procedures. These results provide evidence to guide further research but are not sufficient to support the routine use of chelation therapy for treatment of patients who have had an MI.
At the time, the TACT trial was heavily and rightly criticised for a whole host of reasons. For instance, because of the result of the FDA inspection of the highest accruing TACT site:
- The investigators didn’t conduct the investigation in accordance with the signed statement and investigational plan. Several examples were given of shoddy procedures, prefilled forms, and failure to train personnel.
- Failure to report promptly to the IRB all unanticipated problems involving risk to human subjects or others. Examples are given, including failure to report the deaths of patients on the study in a timely fashion (in one case the death wasn’t reported to the IRB until four months later; in another case it was never reported at all). In other cases, adverse event reports were not submitted to the IRB.
- Failure to prepare or maintain adequate case histories with respect to observations and data pertinent to the investigation.
- Investigational drug disposition records are not adequate with respect to dates, quantity, and use by subjects.
Despite these problems, the study was published in JAMA, albeit with a very critical editorial:
Differential dropout in TACT suggests unmasking, but the problem of intentional unblinding is more concerning. The sponsors of the trial, the National Heart, Lung, and Blood Institute (NHLBI) and the National Center for Complementary and Alternative Medicine (NCCAM), were unblinded throughout the trial. The National Institutes of Health policy unwisely allows the sponsor access to unblinded trial data, and both organizations sent observers to the closed sessions of the data monitoring committee. This gave them access to confidential data during each of the 11 interim analyses. The unblinding of the study sponsor represents a serious deviation from acceptable standards of conduct for supervision of clinical trials. If a pharmaceutical company sponsoring a trial were allowed access to actual outcome data during the study, there would be major objections. Like any sponsor, the NHLBI and NCCAM cannot be considered unbiased observers. These agencies made major financial commitments to the trial and may intentionally or inadvertently influence study conduct if inappropriately unblinded during the study…
Given the numerous concerns with this expensive, federally funded clinical trial, including missing data, potential investigator or patient unmasking, use of subjective end points, and intentional unblinding of the sponsor, the results cannot be accepted as reliable and do not demonstrate a benefit of chelation therapy. The findings of TACT should not be used as a justification for increased use of this controversial therapy.
Orac, makes several further critical points about the published trial:
First, the primary endpoint (i.e., the aggregated serious cardiovascular events) did indeed show a modest difference, namely 30% of placebo subjects versus 26.5% of the EDTA chelation subjects (hazard ratio 0.82 for chelation). However, one notes that the result is just barely statistically significant, p = 0.035, with the 99% confidence interval for the hazard ratio ranging from 0.69 to 0.99. (The predetermined level for statistical significance for purposes of this study was 0.036; so this is statistically significant by the barest margin.) More importantly, if you look at the individual endpoints that make up that aggregate, there was no statistically significant difference in death, myocardial infarction, stroke, coronary revascularization, and hospitalization for angina. Subgroup analysis (always a questionable analysis that requires replication, even when preplanned, as in TACT) purported to show a much greater benefit for diabetics, with a hazard ratio of 0.61 (p=0.002), while patients without diabetes showed no statistically significant difference in any of the outcome measures, including the aggregated total bad outcomes.
Now a paper that has just emerged describes the intent-to-treat comparison of this trial in patients with diabetes.
This was a double-blind, placebo-controlled, 2 × 2 factorial multicenter randomized trial of 1,708 post-myocardial infarction (MI) patients ≥50 years of age and with creatinine ≤2.0 mg/dL randomized to receive 40 EDTA chelation or placebo infusions plus 6 caplets daily of a 28-component multivitamin-multimineral mixture or placebo. The primary end point was a composite of total mortality, MI, stroke, coronary revascularization, or hospitalization for angina.
Median age was 65 years, 18% were female, 94% were Caucasian, 37% were diabetic, 83% had prior coronary revascularization, and 73% were on statins. Five-year Kaplan-Meier estimates for the primary end point was 31.9% in the chelation + high-dose vitamin group, 33.7% in the chelation + placebo vitamin group, 36.6% in the placebo infusion + active vitamin group, and 40.2% in the placebo infusions + placebo vitamin group. The reduction in primary end point by double active treatment compared with double placebo was significant (hazard ratio 0.74, 95% CI 0.57-0.95, P = .016). In patients with diabetes, the primary end point reduction of double active compared with double placebo was more pronounced (hazard ratio 0.49, 95% CI 0.33-0.75, P < .001).
The authors conclude that in stable post-MI patients on evidence-based medical therapy, the combination of oral high-dose vitamins and chelation therapy compared with double placebo reduced clinically important cardiovascular events to an extent that was both statistically significant and of potential clinical relevance.
I fear that these conclusions are erroneous and misleading: the marginally positive finding might have nothing to do with chelation per se; most likely they are due to the fact that the ‘vitamin’ mixture administered along with chelation contained ingredients like heparin and procaine which are potentially beneficial for cardiovascular conditions. Moreover, the placebo contained a considerable amount of glucose which could easily explain the better outcome of the diabetic subgroup receiving the verum – in other words, the verum generated better results not because it was effective but because the ‘placebo’ had detrimental effects.
Arnold Relman has died aged 91. He was a great personality, served for many years as editor-in-chief of ‘The New England Journal of Medicine’ and was professor of medicine and social medicine at Harvard Medical School. He also was an brilliantly outspoken critic of alternative medicine, and I therefore believe that he deserves to be remembered here. The following excerpts are from an article he wrote in 1998 about Andrew Weil, America’s foremost guru of alternative medicine; I have taken the liberty of extracting a few paragraphs which deal with alternative medicine in general terms.
Until now, alternative medicine has generally been rejected by medical scientists and educators, and by most practicing physicians. The reasons are many, but the most important reason is the difference in mentality between the alternative practitioners and the medical establishment. The leaders of the establishment believe in the scientific method, and in the rule of evidence, and in the laws of physics, chemistry, and biology upon which the modern view of nature is based. Alternative practitioners either do not seem to care about science or explicitly reject its premises. Their methods are often based on notions totally at odds with science, common sense, and modern conceptions of the structure and the function of the human body. In advancing their claims, they do not appear to recognize the need for objective evidence, asserting that the intuitions and the personal beliefs of patients and healers are all that is needed to validate their methods. One might have expected such thinking to alienate most people in a technologically advanced society such as ours; but the alternative medicine movement, and the popularity of gurus such as Weil, are growing rapidly…
That people usually “get better,” that most relatively minor diseases heal spontaneously or seem to improve with simple common remedies, is hardly news. Every physician, indeed every grandmother, knows that. Yet before we accept Weil’s contention that serious illnesses such as “bone cancer,” “Parkinson’s disease,” or “scleroderma” are similarly curable, or respond to alternative healing methods, we need at least to have some convincing medical evidence that the patients whom he reports in these testimonials did indeed suffer from these diseases, and that they were really improved or healed. The perplexity is not that Weil is using “anecdotes” as proof, but that we don’t know whether the anecdotes are true.
Anecdotal evidence is often used in the conventional medical literature to suggest the effectiveness of treatment that has not yet been tested by formal clinical trials. In fact, much of the mainstream professional literature in medicine consists of case reports — “anecdotes,” of a kind. The crucial difference between those case reports and the testimonials that abound in Weil’s books (and throughout the literature of alternative medicine) is that the case reports in the mainstream literature are almost always meticulously documented with objective data to establish the diagnosis and to verify what happened, whereas the testimonials cited by alternative medicine practitioners usually are not. Weil almost never gives any objective data to support his claims. Almost everything is simply hearsay and personal opinion.
To the best of my knowledge, Weil himself has published nothing in the peer-reviewed medical literature to document objectively his personal experiences with allegedly cured patients or to verify his claims for the effectiveness of any of the unorthodox remedies he uses. He is not alone in this respect. Few proponents of alternative medicine have so far published clinical reports that would stand the rigorous scientific scrutiny given to studies of traditional medical treatments published in the serious medical journals. Alternative medicine is still a field rich in undocumented claims and anecdotes and relatively lacking in credible scientific reports…
… Thus Weil can believe in miraculous cures even while claiming to be rational and scientific, because he thinks that quantum theory supports his views.
Yet the leading physicists of our time do not accept such an interpretation of quantum theory. They do not believe quantum theory says anything about the role of human consciousness in the physical world. They see quantum laws as simply a useful mathematical formulation for describing subatomic phenomena that are not adequately handled by classical physical theory, although the latter remains quite satisfactory for the analysis of physical events at the macro-level. Steven Weinberg has observed that “quantum mechanics has been overwhelmingly important to physics, but I cannot find any messages for human life in quantum mechanics that are different in any important way from those of Newtonian physics.” And overriding all discussions of the meaning of quantum physics is the fundamental fact that quantum theory, like all other scientific law, is only valid to the extent that it predicts and accords with the evidence provided by observation and objective measurement. Richard Feynman said it quite simply: “Observation is the ultimate and final judge of the truth of an idea.” Feynman also pointed out that scientific observations need to be objective, reproducible, and, in a sense, public — that is, available to all interested scientists who wish to check the observations for themselves.
Surely almost all scientists would agree with Feynman that, regardless of what theory of nature we wish to espouse, we cannot escape the obligation to support our claims with objective evidence. All theories must conform to the facts or be discarded. So, if Weil cannot produce credible evidence to validate the miraculous cures that he claims for the healing powers of the mind, and if he does not support with objective data the claims he and others make for the effectiveness of alternative healing methods, he cannot presume to wear the mantle of science, and his appeal to quantum theory cannot help him.
Some apologists for alternative medicine have argued that since their healing methods are based on a “paradigm” different from that of traditional medicine, traditional standards of evidence do not apply. Weil sometimes seems to agree with that view, as when he talks about “stoned thinking” and the “ambivalent” nature of reality, but more recently — as he seeks to integrate alternative with allopathic medicine — he seems to acknowledge the need for objective evidence. This, at least, is how I would interpret one of his most recent and ambitious publishing ventures, the editorship of the new quarterly journal Integrative Medicine***.
Integrative Medicine describes itself as a “peer-reviewed journal … committed to gathering evidence for the safety and efficacy of all approaches to health according to the highest standards of scientific research, while remaining open to new paradigms and honoring the healing power of nature.” The Associate Editors and Editorial Board include prominent names in both alternative medicine and allopathic medicine, who presumably support that mission. Yet the first two issues will disappoint those who were looking for original clinical research based on new, objective data. Perhaps subsequent issues will be different, but in any case it is hard to understand the need for Weil’s new journal if he truly intends to hold manuscripts to accepted scientific standards: there already exist many leading peer-reviewed medical journals that will review research studies of alternative healing methods on their merits. During the past decade or so, only a few such studies have passed rigorous review and have been published in first-rate journals. Recently, more studies have been published, but very few of them report significant clinical effects. And that is pretty much where matters now stand. Despite much avowed interest in research on alternative medicine and increased investment in support of such research, the evidentiary underpinnings of unconventional healing methods are still largely lacking…
The alternative medicine movement has been around for a long time, but it was eclipsed during most of this century by the success of medical science. Now there is growing public disenchantment with the cost and the impersonality of modern medical care, as well as concern about medical mistakes and the complications and side-effects of pharmaceuticals and other forms of medical treatment. For their part, physicians have allowed the public to perceive them as uninterested in personal problems, as inaccessible to their patients except when carrying out technical procedures and surgical operations. The “doctor knows best” attitude, which dominated patient-doctor relations during most of the century, has in recent decades given way to a more activist, consumer-oriented view of the patient’s role. Moreover, many other licensed health-care professionals, such as nurse-practitioners, psychotherapists, pharmacists, and chiropractors, are providing services once exclusively reserved to allopathic physicians.
The net result of all these developments has been a weakening of the hegemony that allopathic medicine once exercised over the health care system, and a growing interest by the public in exploring other healing approaches. The authority of allopathic medicine is also being challenged by a swelling current of mysticism and anti-scientism that runs deep through our culture. Even as the number and the complexity of urgent technological and scientific issues facing contemporary society increase, there seems to be a growing public distrust of the scientific outlook and a reawakening of interest in mysticism and spiritualism.
All this obscurantism has given powerful impetus to the alternative medicine movement, with its emphasis on the power of mind over matter. And so consumer demand for alternative remedies is rising, as is public and private financial support for their study and clinical use. It is no wonder that practicing physicians, the academic medical establishment, and the National Institutes of Health are all finding reasons to pay more attention to the alternative medicine movement. Indeed, it is becoming politically incorrect for the movement’s critics to express their skepticism too strongly in public…
There is no doubt that modern medicine as it is now practiced needs to improve its relations with patients, and that some of the criticisms leveled against it by people such as Weil — and by many more within the medical establishment itself — are valid. There also can be no doubt that a few of the “natural” medicines and healing methods now being used by practitioners of alternative medicine will prove, after testing, to be safe and effective. This, after all, has been the way in which many important therapeutic agents and treatments have found their way into standard medical practice in the past. Mainstream medicine should continue to be open to the testing of selected unconventional treatments. In keeping an open mind, however, the medical establishment in this country must not lose its scientific compass or weaken its commitment to rational thought and the rule of evidence.
There are not two kinds of medicine, one conventional and the other unconventional, that can be practiced jointly in a new kind of “integrative medicine.” Nor, as Andrew Weil and his friends also would have us believe, are there two kinds of thinking, or two ways to find out which treatments work and which do not. In the best kind of medical practice, all proposed treatments must be tested objectively. In the end, there will only be treatments that pass that test and those that do not, those that are proven worthwhile and those that are not. Can there be any reasonable “alternative”?
*** the journal only existed for a short period of time
Tai Chi has been suggested to have many health benefits. Might it even prolong life? There are many enthusiasts who claim just that, but is there any evidence?
This study is a retrospective cross-sectional investigation to compare the rejuvenating and anti-ageing effects among a Tai Chi group (TCC) and a brisk walking group (BW) and a no exercise habit group (NEH) of volunteers. Thirty-two participants were separated into three groups: the TCC group (practicing TC for more than 1 year), the BW group (practicing BW for more than 1 year), and the NEH group. The CD34+ cell counts in peripheral blood of the participants was determined, and the Kruskal‐Wallis test was used to evaluate and compare the antiaging effects of the three groups. The results show that the participants in the TCC group (N = 10) outperformed the NEH group (N = 12) with respect to the number of CD34+ progenitor cells. No significant difference was found between the TCC group and the BW group. The authors of this study conclude that TCC practice sustained for more than 1 year may be an intervention against aging as effective as BW in terms of its benefits on the improvement of CD34+ number.
I was alerted to this new paper by several rather sensational headlines in the daily press which stated that Tai chi (TC) had anti-aging effects. So I searched for the press release about the article where I found the following quotes:
“It is possible that Tai Chi may prompt vasodilation and increase blood flow,” said Lin. “Considering that BW may require a larger space or more equipment, Tai Chi seems to be an easier and more convenient choice of anti-aging exercise.” “This study provides the first step into providing scientific evidence for the possible health benefits of Tai Chi.” said Dr. Paul R. Sanberg, distinguished professor at the Center of Excellence for Aging and Brain Repair, Morsani College of Medicine, University of South Florida, Tampa, FL. “Further study of how Tai Chi can elicit benefit in different populations and on different parameters of aging are necessary to determine its full impact.”
Personally, I find both the press release and the original conclusions of the authors quite amazing. If anyone wanted to write a textbook on how not to do such things, he/she could use them as excellent examples.
Seen with just a tinge of critical thinking the paper reports a flimsy case-control study comparing three obviously self-selected groups of people who had chosen to follow different exercise regimen for several months. In all likelihood they also differed in terms of life-style, nutrition, sleeping pattern, alcohol intake, smoking habits and a million other things. These rather tiny groups were then compared according to a surrogate measure for ageing and some differences were identified.
To conclude from this, or even to imply, that TC has anti-ageing effects is as far-fetched as claiming the tooth fairy has money problems.
This story could be just funny or trivial or boring – however, I think, it is also a bit worrying. It shows, I fear, how uncritical researchers in conjunction with some naïve press officer are able to induce silly journalists and headline-writers to mislead the public.
Boiron is the world’s biggest producer of homeopathic remedies. It also is a firm that is relatively active in research into homeopathy. Here is one of their investigations which I find most remarkable.
This study was designed to describe the sociodemographic and clinical characteristics of patients recommended allopathic and/or homeopathic medicines for influenza-like illness (ILI) or ear nose and throat ENT disorders by pharmacists in France and to investigate the effectiveness of these treatments.
The introduction of the article includes interesting information; it informs us that, although homeopathy is more popular in Europe than in the Unites States, sales of homeopathic medicines in the United States grew by more than 1,000% in the late 1970s and early 1980s and continue to grow. In parallel, the number of physicians specializing in homeopathy doubled between 1980 and 1982. In 2003, sales of homeopathic medicines in the United States were estimated to be between $300 and $450 million, with an average growth rate of approximately 8% per year. Homeopathic drugs are among the top 10 nonprescription products sold in the category of analgesics to treat coughs, colds, and flu. The sale of homeopathic medicines in the United States is controlled by the Food, Drug, and Cosmetic Act and regulations issued by the U.S Food and Drug Administration (FDA). Homeopathic medicines in the United States are subject to well-controlled regulatory processes that closely resemble those used for allopathic medicines. FDA regulations for the sale of homeopathic medicines in the United States state that they can only be sold without prescription if they are for self-limiting conditions such as the common cold…
Am I mistaken, or does that paragraph read a bit like a text written by the marketing team of Boiron wanting to establish their products in the US?
Anyway, the methodology and results of the study are described in the abstract as follows:
A prospective, observational, multicenter study was carried out in randomly selected pharmacies across the 8 IDREM medical regions of France. Pharmacies that agreed to participate recruited male or female patients who responded to the following inclusion criteria: age ≥ 12 years presenting with the first symptoms of an ILI or ENT disorder that were present for less than 36 hours prior to the pharmacy visit. All medicines recorded in the study were recommended by the pharmacists. The following data were recorded at inclusion and after 3 days of treatment: the intensity of 13 symptoms, global symptom score, and disease impact on daily activities and sleep. Two groups of patients were compared: those recommended allopathic medicine only (AT group) and those recommended homeopathic medicine with or without allopathic medicine (HAT group). The number and severity of symptoms, change in global symptom score, and disease impact on daily activities and sleep were compared in the 2 treatment groups after 3 days of treatment. Independent predictors of recommendations for homeopathic medicine were identified by multi- factorial logistic regression analysis.
A total of 242 pharmacies out of 4,809 (5.0%) contacted agreed to participate in the study, and 133 (2.8%) included at least 1 patient; 573 patients were analyzed (mean age: 42.5 ± 16.2 years; 61.9% female). Of these, 428 received allopathic medicines only (74.7%; AT group), and 145 (25.3%) received homeopathic medicines (HAT group) alone (9/145, 1.6%) or associated with allopathy (136/145, 23.7%). At inclusion, HAT patients were significantly younger (39.6 ± 14.8 vs. 43.4 ± 16.1 years; P less than 0.05), had a higher mean number of symptoms (5.2 ± 2.5 vs. 4.4 ± 2.5; P less than 0.01), and more severe symptoms (mean global symptom score: 24.3 ± 5.5 vs. 22.3 ± 5.8; P = 0.0019) than AT patients. After 3 days, the improvement in symptoms and disease impact on daily activities and sleep was comparable in both groups of patients.
From these findings, the authors draw the following conclusions: Patients recommended homeopathic medicine by pharmacists were younger and had more severe symptoms than those recommended allopathic medicine. After 3 days of treatment, clinical improvement was comparable in both treatment groups. Pharmacists have an important role to play in the effective management of ILI and ENT disorders.
And, to make perfectly clear what all this is about, the first sentence of the ‘discussion’ puts it to the point by stating that homeopathic medicine, with or without allopathic medicine, appears to be effective at alleviating the symptoms of ILI or ENT disorders.
As I have heard it said that Boiron seems to have the nasty habit of threatening their critics with legal action, I ought to be quite cautious in my assessment of this ‘masterpiece of promotion’. Yet a few comments must surely be permitted.
‘To describe the sociodemographic and clinical characteristics of patients recommended allopathic and/or homeopathic medicines’ is not what I personally find an interesting subject of research, nor is it anything that will affect health care meaningfully, I think. Yet ‘to investigate the effectiveness of these treatments’ is certainly interesting and important. I will therefore focus on this second aim of the study.
Hold on, was this really a ‘study’? On closer inspection, it seemed much more like a survey. People who felt that they were suffering from ILI and ENT disorders and thus went to a pharmacy to buy something for their problem were offered either homeopathic or conventional medicines. Those who accepted either of the recommendations were asked to fill out some self-assessment forms and received a phone call three days later to check their symptoms. 94% of all patients in the homeopathy group took homeopathic medicine in combination with ‘allopathic’ medicine (it is interesting, perhaps even telling, that this term used by the authors was invented by Hahnemann as an insult to conventional medicine!). There was no examination by a doctor to verify what condition the survey-participants were truly suffering from, and there was no verification that the information provided during the follow-up telephone call was in any way real. The most frequently recommended homeopathic medicine was Anas barbariae 200C (Oscillococcinum) which is Boiron’s famous homeopathically diluted (about one molecule per universe, I guess) duck-liver heavily promoted in France against colds and similar conditions.
As it turns out, those survey-participants who accepted the homeopathic recommendation were significantly younger than those who accepted the recommendation for a conventional treatment (many surveys confirm that younger people are more prone to trying alternative medicine than older ones). It stands to reason, that the younger (and therefore fitter) patients were in better general health and therefore might recover quicker than the older ones. But, in fact, they did not!
Could this be due to the homeopathic remedies actually delaying recovery? Of course not! Who would be silly enough to claim that homeopathy could have this (or any other) effect? According to the authors, it is due to the fact that this group ‘had more severe symptoms than those recommended allopathic medicine’. But, as I said, we have to take their word for it; there is no independent verification of this. It would, of course, be quite ridiculous to postulate that those survey-participants accepting homeopathy were also a little more introspective or concerned about their own health (perhaps even more gullible) and thus claimed more severe symptoms!
And what about the authors’ conclusion that clinical improvement was comparable in both treatment groups? Well, this is more than a little problematic, in my view: first, we have no independent verification of the ‘improvement’ in either group. Second, we don’t know that the conventional treatments actually worked, and it could well be that both approaches were similarly ineffective, and that the observed outcomes are merely a reflection of the natural history of the condition. And third, one might expect the homeopathic (younger) group to do not similarly well but slightly better, simply because the natural history of the illness would tend to be more benign in younger people.
Before I finish,I should make a brief comment about the authors’ courageous statement that homeopathic medicine, with or without allopathic medicine, appears to be effective at alleviating the symptoms of ILI or ENT disorders. I think, for the reasons I already provided, this is extremely doubtful. In my view, more critical scientists would have phrased the conclusions differently:
THIS SURVEY SHOWS THAT EVALUATING THE EFFECTIVENESS OF MEDICAL INTERVENTIONS REQUIRES A MORE RIGOROUS METHODOLOGY THAN THAT OF A SURVEY.
But perhaps this would be asking a little too much of the authors; after all, at the end of the article, we find this telling footnote: Laboratoires Boiron provided financial support for the study. Cognet-Dementhon, Thevenard, Duru, and Allaert received consulting fees from Laboratoires Boiron for this study. Danno and Bordet are employees of Laboratoires Boiron.
I have often asked myself whether it is right/necessary to scientifically test things which are entirely implausible. Should we, for instance test the effectiveness of treatments which have a very low prior probability of generating a positive effect such as paranormal healing, homeopathy or Bach flower remedies? If you believe in the principles of evidence-based medicine you might focus on the clinical evidence and see biological plausibility as secondary. If you are a basic scientist, you are likely to do the reverse.
A recent article addressed this issue. The author points out that evaluating the absurd is absurd. Specifically, he noted that the empirical evaluation of a therapy would normally assume a plausible rationale regarding the mechanism of action. However, examination of the historical background and underlying principles for reflexology, iridology, acupuncture, auricular acupuncture, and some herbal medicines, reveals a rationale founded on the principle of analogical correspondences, which is a common basis for magical thinking and pseudoscientific beliefs such as astrology and chiromancy. Where this is the case, it is suggested that subjecting these therapies to empirical evaluation may be tantamount to evaluating the absurd.
This makes a lot of sense – but is it really entirely true? Are there no legitimate reasons at all for testing alternative treatments that lack biological plausibility? Ten or twenty years ago, I would have disagreed with the notion that plausibility is an essential prerequisite for scientific testing; today, I have changed my mind a little, but not as much as to agree completely with the assumption. In other words, I still see more than one good reason why evaluating the absurd might be reasonable or even advisable.
- Using plausibility as the only arbiter of scientific ‘evaluability’, assumes that we understand everything about plausibility there is to know. Yet it might just be possible that we mis-categorise something as implausible simply because we are not yet fully aware of all the facts.
- Declaring something as plausible and another thing as implausible are not hard and fast verdicts but judgements which, at least to some degree, are subjective. Sceptics find the axioms of homeopathy utterly implausible, for instance – but ask a homeopath, and you will hear all sorts of explanations which, at least to them, sound plausible.
- If an implausible alternative treatment is in wide-spread use, we arguably have a responsibility to test it scientifically in order to demonstrate the truth about it (to those proponents of that therapy who are willing to accept that rigorous science can find the truth). If we fail to do this, it will be the enthusiasts of that therapy who conduct less than rigorous science and produce false positive results. In turn, this will give the impression that the treatment is effective and mislead consumers, politicians, journalists etc. Seen from this perspective, it might even be unethical to not do the science.
So, I am in two minds about this (which might be a reflection of the fact that, during different periods of my life, I have been a clinician, a basic scientist and a clinical researcher). I realise that plausibility and prior probability are important – much more so than I appreciated years ago. But I think they should not be the only criteria. The clinical evidence should not be pushed aside completely.
I’d be interested to learn your views on this tricky issue.
The mechanisms thorough which spinal manipulative therapy (SMT) exerts its alleged clinical effects are not well established. A new study investigated the effects of subject expectation on clinical outcomes.
Sixty healthy subjects underwent quantitative sensory testing to their legs and low backs. They were randomly assigned to receive a positive, negative, or neutral expectation instructional set regarding the effects of a spe cific SMT technique on pain perception. Following the instructional set, all subjects received SMT and underwent repeat sensory tests.
No inter-group differences in pain response were present in the lower extremity following SMT. However, a main effect for hypoalgesia was present. A significant interaction was present between change in pain perception and group assignment in the low back with participants receiving a negative expectation instructional set demonstrating significant hyperalgesia.
The authors concluded that this study provides preliminary evidence for the influence of a non- specific effect (expectation) on the hypoalgesia associated with a single session of SMT in normal subjects. We replicated our previous findings of hypoalgesia in the lower extremity associated with SMT to the low back. Additionally, the resultant hypoalgesia in the lower extremity was independent of an expectation instructional set directed at the low back. Conversely, participants receiving a negative expectation instructional set demonstrated hyperalgesia in the low back following SMT which was not observed in those receiving a positive or neutral instructional set.
More than 10 years ago, we addressed a similar issue by conducting a systematic review of all sham-controlled trials of SMT. Specifically, we wanted to summarize the evidence from sham-controlled clinical trials of SMT. Eight studies fulfilled our inclusion/exclusion criteria. Three trials (two on back pain and one on enuresis) were judged to be burdened with serious methodological flaws. The results of the three most rigorous studies (two on asthma and one on primary dysmenorrhea) did not suggest that SMT leads to therapeutic responses which differ from an inactive sham-treatment. We concluded that sham-controlled trials of SMT are sparse but feasible. The most rigorous of these studies suggest that SMT is not associated with clinically relevant specific therapeutic effects.
Taken together, these two articles provide intriguing evidence to suggest that SMT is little more than a theatrical placebo. Given the facts that SMT is neither cheap nor devoid of risks, the onus is now on those who promote SMT, e.g. chiropractors, osteopaths and physiotherapists, to show that this is not true.
It is not often that we see an article of the great George Vithoulkas, the ‘über-guru‘ of homeopathy, in a medical journal. In fact, this paper, which he co-authored with several colleagues, seems to be a rare exception: in his entire career, he seems to have published just 15 Medline- listed articles most of which are letters to the editor.
According to Wikipedia, Vithoulkas has been described as “the maestro of classical homeopathy” by Robin Shohet; Lyle Morgan says he is “widely considered to be the greatest living homeopathic theorist”; and Scott Shannon calls him a “contemporary master of homeopathy.” Paul Ekins credited Vithoulkas with the revival of the credibility of homeopathy.
In his brand new paper, Vithoulkas provides evidence for the notion that homeopathy can treat infertility. More specifically, the authors present 5 cases of female infertility treated successfully with the use of homeopathic remedies.
Yes, really! The American Medical College of Homeopathy informs us that homeopathy has an absolute solution that can augment your probability of conception. Homeopathic treatment of Infertility addresses both physical and emotional imbalances in a person. Homeopathy plays a role in treating Infertility by strengthening the reproductive organs in both men and women, by regulating hormonal balance, menstruation and ovulation in women, by escalating blood flow into the pelvic region, by mounting the thickness of the uterine lining and preventing the uterus from contracting hence abating chances of a miscarriage, and by increasing quality and quantity of sperm count in men. It can also be advantageous in reducing anxiety so that the embryo implantation can take place in a favourable environment. Homoeopathy is a system of medicine directed at assisting the body’s own healing process.
Imagine: the 5 women in Vithoulkas ‘study’ wanted to have children; they consulted homeopaths because they did not get pregnant in a timely fashion. The homeopaths prescribed individualised homeopathy and treated them for prolonged periods of time. Eventually, BINGO!, all of the 5 women got pregnant.
What a hoot!
It beggars belief that this result is being credited to the administration of homeopathic remedies. Do the authors not know that, in many cases, it can take many months until a pregnancy occurs? Do they not think that the many women they treated unsuccessfully for the same problem should raise some doubts about homeopathy? Do they really believe that their remedies had any causal relationship to the 5 pregnancies?
Vithoulkas was a recipient of the Right Livelihood Award in 1996. I hope they did not give it to him in recognition of his scientific achievements!
After a traumatic brain injury (TBI) the risk of stroke is significantly increased. Taiwanese researchers conducted a study to find out whether acupuncture can help to protect TBI patients from stroke. They used Taiwan’s National Health Insurance Research Database to conduct a retrospective cohort study of 7409 TBI patients receiving acupuncture treatment and 29,636 propensity-score-matched TBI patients without acupuncture treatment as controls. Both TBI cohorts were followed for up to two years and adjusted for immortal time to measure the incidence and adjusted hazard ratios (HRs) of new-onset stroke.
TBI patients with acupuncture treatment (4.9 per 1000 person-years) had a lower incidence of stroke compared with those without acupuncture treatment (7.5 per 1000 person-years), with a HR of 0.59 (95% CI = 0.50-0.69) after adjustment for sociodemographics, coexisting medical conditions and medications. The association between acupuncture treatment and stroke risk was investigated by sex and age group (20-44, 45-64, and ≥65 years). The probability curve with log-rank test showed that TBI patients receiving acupuncture treatment had a lower probability of stroke than those without acupuncture treatment during the follow-up period (p<0.0001).
The authors conclude that patients with TBI receiving acupuncture treatment show decreased risk of stroke compared with those without acupuncture treatment. However, this study was limited by lack of information regarding lifestyles, biochemical profiles, TBI severity, and acupuncture points used in treatments.
I want to congratulate the authors for adding the last sentence to their conclusions. There is no plausible mechanism that I can think of by which acupuncture might bring about the observed effect. This does not mean that an effect does not exist; it means, however, that it is wise to be cautious and to not jump to conclusions which later need to be revised. The simplest interpretation, by far, of the observed phenomenon is that those patients opting to have acupuncture were, on average, less ill and therefore had a lower risk of stroke.
Having said that, the findings are, I think, intriguing enough to conduct further investigations – provided they are rigorous and eliminate the confounders that prevented this study from arriving at more definitive conclusions.
The news that the use of Traditional Chinese Medicine (TCM) positively affects cancer survival might come as a surprise to many readers of this blog; but this is exactly what recent research has suggested. As it was published in one of the leading cancer journals, we should be able to trust the findings – or shouldn’t we?
The authors of this new study used the Taiwan National Health Insurance Research Database to conduct a retrospective population-based cohort study of patients with advanced breast cancer between 2001 and 2010. The patients were separated into TCM users and non-users, and the association between the use of TCM and patient survival was determined.
A total of 729 patients with advanced breast cancer receiving taxanes were included. Their mean age was 52.0 years; 115 patients were TCM users (15.8%) and 614 patients were TCM non-users. The mean follow-up was 2.8 years, with 277 deaths reported to occur during the 10-year period. Multivariate analysis demonstrated that, compared with non-users, the use of TCM was associated with a significantly decreased risk of all-cause mortality (adjusted hazards ratio [HR], 0.55 [95% confidence interval, 0.33-0.90] for TCM use of 30-180 days; adjusted HR, 0.46 [95% confidence interval, 0.27-0.78] for TCM use of > 180 days). Among the frequently used TCMs, those found to be most effective (lowest HRs) in reducing mortality were Bai Hua She She Cao, Ban Zhi Lian, and Huang Qi.
The authors of this paper are initially quite cautious and use adequate terminology when they write that TCM-use was associated with increased survival. But then they seem to get carried away by their enthusiasm and even name the TCM drugs which they thought were most effective in prolonging cancer survival. It is obvious that such causal extrapolations are well out of line with the evidence they produced (oh, how I wished that journal editors would finally wake up to such misleading language!) .
Of course, it is possible that some TCM drugs are effective cancer cures – but the data presented here certainly do NOT demonstrate anything like such an effect. And before such a far-reaching claim is being made, much more and much better research would be necessary.
The thing is, there are many alternative and plausible explanations for the observed phenomenon. For instance, it is conceivable that users and non-users of TCM in this study differed in many ways other than their medication, e.g. severity of cancer, adherence to conventional therapies, life-style, etc. And even if the researchers have used clever statistical methods to control for some of these variables, residual confounding can never be ruled out in such case-control studies.
Correlation is not causation, they say. Neglect of this elementary axiom makes for very poor science – in fact, it produces dangerous pseudoscience which could, like in the present case, lead a cancer patient straight up the garden path towards a premature death.
There are dozens of observational studies of homeopathy which seem to suggest – at least to homeopaths – that homeopathic treatments generate health benefits. As these investigations lack a control group, their results can be all to easily invalidated by pointing out that factors like ‘regression towards the mean‘ (RTM, a statistical artefact caused by the phenomenon that a variable that is extreme on its first measurement tends to be closer to the average on its second measurement) might be the cause of the observed change. Thus the debate whether such observational data are reliable or not has been raging for decades. Now, German (pro-homeopathy) investigators have published a paper which potentially could resolve this dispute.
With this re-analysis of an observational study, the investigators wanted to evaluate whether the observed changes in previous cohort studies are due to RTM and to estimate RTM adjusted effects. SF-36 quality-of-life (QoL) data from a cohort of 2827 chronically diseased adults treated with homeopathy were reanalysed using a method described in 1991 by Mee and Chua’s. RTM adjusted effects, standardized by the respective standard deviation at baseline, were 0.12 (95% CI: 0.06-0.19, P < 0.001) in the mental and 0.25 (0.22-0.28, P < 0.001) in the physical summary score of the SF-36. Small-to-moderate effects were confirmed for most individual diagnoses in physical, but not in mental component scores. Under the assumption that the true population mean equals the mean of all actually diseased patients, RTM adjusted effects were confirmed for both scores in most diagnoses.
The authors reached the following conclusion: “In our paper we showed that the effects on quality of life observed in patients receiving homeopathic care in a usual care setting are small or moderate at maximum, but cannot be explained by RTM alone. Due to the uncontrolled study design they may, however, completely be due to nonspecific effects. All our analyses made a restrictive and conservative assumption, so the true treatment effects might be larger than shown.”
Of course, the analysis heavily relies on the validity of Mee and Chua’s modified t-test. It requires the true mean in the target population to be known, a requirement that seldom can be fulfilled. The authors therefore took the SF-36 mean summary scores from the 1998 German health survey as proxies. I am not a statistician and therefore unable to tell how reliable this method might be (- if there is someone out there who can give us some guidance here, please post your comment).
In order to make sense of these data, we need to consider that, during the study period, about half of the patients admitted to have had additional visits to non-homeopathic doctors, and 27% also received conventional drugs. In addition, they would have benefitted from:
- the benign history of the conditions they were suffering from,
- a placebo-effect,
- the care and attention they received
- and all sorts of other non-specific effects.
So, considering these factors, what does this interesting re-analysis really tell us? My interpretation is as follows: the type of observational study that homeopaths are so fond of yields false-positive results. If we correct them – as the authors have done here for just one single factor, the RTM – the effect size gets significantly smaller. If we were able to correct them for some of the other factors mentioned above, the effect size would shrink more and more. And if we were able to correct them for all confounders, their results would almost certainly concur with those of rigorously controlled trials which demonstrate that homeopathic remedies are pure placebos.
I am quite sure that this interpretation is unpopular with homeopaths, but I am equally certain that it is correct.