The new guidelines by the American College of Physicians entitled ‘Noninvasive Treatments for Acute, Subacute, and Chronic Low Back Pain: A Clinical Practice Guideline From the American College of Physicians’ have already been the subject of the previous post. Today, I want to have a closer look at a small section of these guidelines which, I think, is crucial. It is entitled ‘HARMS OF NONPHARMACOLOGIC THERAPIES’. I have taken the liberty of copying it below:
“Evidence on adverse events from the included RCTs and systematic reviews was limited, and the quality of evidence for all available harms data is low. Harms were poorly reported (if they were reported at all) for most of the interventions.
Low-quality evidence showed no reported harms or serious adverse events associated with tai chi, psychological interventions, multidisciplinary rehabilitation, ultrasound, acupuncture, lumbar support, or traction (9,95,150,170–174). Low-quality evidence showed that when harms were reported for exercise, they were often related to muscle soreness and increased pain, and no serious harms were reported. All reported harms associated with yoga were mild to moderate (119). Low-quality evidence showed that none of the RCTs reported any serious adverse events with massage, although 2 RCTs reported soreness during or after massage therapy (175,176). Adverse events associated with spinal manipulation included muscle soreness or transient increases in pain (134). There were few adverse events reported and no clear differences between MCE and controls. Transcutaneous electrical nerve stimulation was associated with an increased risk for skin site reaction but not serious adverse events (177). Two RCTs (178,179) showed an increased risk for skin flushing with heat compared with no heat or placebo, and no serious adverse events were reported. There were no data on cold therapy. Evidence was insufficient to determine harms of electrical muscle stimulation, LLLT, percutaneous electrical nerve stimulation, interferential therapy, short-wave diathermy, and taping.”
The first thing that strikes me is the brevity of the section. Surely, guidelines of this nature must include a full discussion of the risks of the treatments in question!
The second thing that is noteworthy is the fact that the authors confirm the fact I have been banging on about for years: clinical trials of alternative therapies far too often fail to mention adverse effects. I have often pointed out that the failure to report adverse effects in clinical trials is an unacceptable violation of medical ethics. By contrast, the guideline authors seem not to feel strongly about this omission.
The third thing that is noteworthy is that the guidelines evaluate the harms of the treatments purely on the basis of the adverse effects reported in the clinical trials and systematic reviews included in their efficacy assessments. This is nonsensical for at least two reasons:
- The guideline authors themselves are aware that the trials very often fail to mention adverse effects.
- For any assessment of harm, one has to go far beyond the evidence of clinical trials, because trials tend to be too small to pick up rare adverse effects, and because they are always conducted under optimally controlled conditions where adverse effects are less likely to occur than in real life.
Together, these features of the assessment of harms explain why the guideline authors arrive at conclusions which are oddly misguided; I would even feel that they resemble a white-wash. Here are two of the most overt misjudgements:
- no harms associated with acupuncture,
- only trivial harm associated with spinal manipulations.
The best evidence we have today shows that acupuncture leads to mild adverse effects in about 10% of all cases and is also associated with very severe complications (e.g. pneumothorax, cardiac tamponade, infections, deaths) in an unknown number of patients. More details can be found for instance here, here, here and here.
And the best evidence available shows that spinal manipulation leads to moderately severe adverse effects in ~50% of all cases. In addition, we know of hundreds of cases of very severe complications resulting in stroke, permanent neurological deficits or deaths. More details can be found for instance here, here, here and here.
In the introduction, I stated that this small section of the guidelines is crucial.
The reason is simple: any responsible therapeutic decision has to be based not just on the efficacy of the treatment in question but on its risk/benefit balance. The evidence shows that the risks of some alternative therapies can be considerable, a fact that is almost totally neglected in the guidelines. Therefore, the recommendations of the new guidelines by the American College of Physicians entitled ‘Noninvasive Treatments for Acute, Subacute, and Chronic Low Back Pain: A Clinical Practice Guideline From the American College of Physicians’ are in several aspects not entirely correct and need to be reconsidered.
Concerned about the new ACP guidelines on ‘Noninvasive Treatments for Acute, Subacute, and Chronic Low Back Pain: A Clinical Practice Guideline From the American College of Physicians’, Andrea MacGregor asked me to publish her ‘open letter’:
I am a student about to graduate and register as a massage therapist in Canada, and I am writing to express my concern with your recommendation of the use of acupuncture in your new guideline for low-back pain management.
Leading medical and health research experts from around the world, including many who are highly familiar with the use of complementary and alternative therapies, have contributed to a highly informed commentary (attached) assembled by the Friends of Science in Medicine association (Aus.), which supports a strong conclusion that acupuncture is not effective for any specific condition, and that the evidence for it being an effective intervention for low-back pain is not convincing. Another review of acupuncture by FSM concluding that there is a lack of evidence of a therapeutic effect has been endorsed by the Royal Australian and New Zealand College of Obstetricians and Gynaecologists. Respected American medical science writers also maintain that claims of acupuncture’s efficacy are not science-based (examples here and here).
Additionally, previous acupuncture recommendations are being reconsidered by prominent institutions in other parts of the world. The National Institute for Health and Care Excellence guideline for NHS patients in the United Kingdom now recommends against the use of acupuncture for low-back pain, following a high-quality review that critically examined the existing evidence regarding the use of acupuncture and found it to be no more effective than a placebo. The Toronto Hospital for Sick Children has also recently removed references on their website that suggested the efficacy of acupuncture in managing specific chronic pain conditions. The World Health Organization has done the same, no longer suggesting that acupuncture is effective for low-back pain and sciatica.
As someone about to enter a field that is frequently associated with, or considered a part of, complementary healthcare, I know how tempting it can be for us, as professionals and as researchers, to exaggerate claims of efficacy and pin some very high hopes on “new possibilities” in physical therapies.
I also know first-hand how misguided and overblown some of these claims and hopes can be. Many of my own peers and instructors are proponents of acupuncture, and it is common for Canadian massage therapists to become licensed acupuncturists (a similar connection between massage and acupuncture communities, of course, also exists in the United States). I have often seen my own mentors and comrades pushing for the use of acupuncture treatments for many chronic and serious conditions for which there is no basis of evidence at all of acupuncture’s efficacy, including systemic, neurological, and developmental conditions. When questioned, they will usually refer to authorities perceived as “legitimate”, including the American College of Physicians, to say that claims of acupuncture “working” are backed by experts— whether their claims are even pain-related or not.
We see a similar situation with advertisers and media using the guise of “expert-backed” legitimization to recommend acupuncture in misleading ways, often to vulnerable people who could be making better-informed and more effective treatment and management choices for their conditions. Many of these advertising and media entities specifically mention the American College of Physicians as lending credence to their claims, sometimes somewhat out of context.
As someone with a chronic neurological disorder, I find it troubling to see untrue or exaggerated claims of benefit for incurable or serious conditions when we could be focusing on more accurate ideas and having more honest, realistic discussions of our options. This is also important when it comes to deciding how to best allocate our limited health funding resources. Quite a lot of our insurance and out-of-pocket funds are spent on alternative therapies, and it’s important to see things going to use in a way that’s proportionate and appropriate to the evidence we have.
I hope that you will reconsider your recommendation of a practice that is simply not supported by the majority of the research evidence that exists to date. Patients with complex conditions, including low-back pain, deserve accurate and realistic information regarding their treatment options, especially from such trusted and reputable sources as the American College of Physicians. Thank you for your time and attention.
Below are informed conclusions on acupuncture from 28 international experts from 10 countries, including Australia, Canada, Denmark, France, Greece, Italy, Netherlands, New Zealand, United Kingdom and United States of America.
– Sir Richard John Roberts, English biochemist and molecular biologist, 1993 Nobel Prize in Physiology or Medicine – Prof Nikolai Bogduk AM, Emeritus Professor of Pain Medicine, University of Newcastle, Australia – Prof Timothy Caulfield, LLM, FRSC, FCAHS, Canada Research Chair in Health Law & Policy, Trudeau Fellow & Professor, Faculty of Law and School of Public Health, Research Director, Health Law Institute, University of Alberta, Canada – Prof. Assimakis Kanellopoulos, PhD MSc.Prof. Applied Physiotherapy, TEI Lamia, Greece – Prof Lesley Campbell AM, MBBS, FRACP FRCP(UK), Senior Endocrinologist, Diabetes Services, St Vincent’s Hospital, Professor of Medicine, UNSW. Laboratory Co-Head, Clinical Diabetes, Appetite and Metabolism, Garvan Institute of Medical Research, SVH, NSW, Australia – Emeritus Prof Donald M. Marcus, MD, Professor of Medicine and Immunology, Emeritus, Baylor College of Medicine, Houston, United States of America (USA) – Dr Michael Vagg, MBBS(Hons) FAFRM(RACP) FFPMANZCA, Consultant in Rehabilitation and Pain Medicine, Barwon Health. Clinical Senior Lecturer, Deakin University School of Medicine. Fellow, Institute for Science in Medicine, Victoria, Australia – Prof Bernie Garrett, The University of British Columbia, School of Nursing, Vancouver, BC, Canada – A/Prof David H Gorski, MD PhD FACS, surgical oncologist, Barbara Ann Karmanos Cancer Institute, Team Leader, Breast Cancer Multidisciplinary Team, Co-Leader, Breast Cancer Biology Program, Co-Director, Alexander J Walt Comprehensive Breast Center, Chief, Section of Breast Surgery, A/Professor, Surgery, Wayne State University School of Medicine, , and Professor (Honorary) Hanoi Medical University, USA – Prof Carl Bartecchi, MD, MACP, Distinguished Professor of Clinical Medicine, University of Colorado School of Medicine, USA – Prof David Colquhoun, FRS, Dept of Pharmacology, UCL United Kingdom (UK) – Prof Edzard Ernst, MD PhD FMEdSci FSB FRCP FRCP(Edin), Complementary Medicine, Peninsula Medical School, UK – Prof Marcello Costa FAAS. Matthew Flinders Distinguished Professor and Professor of Neurophysiology (2012), Professor of Neurophysiology, Flinders University, Australia. – Emeritus Prof Alastair H MacLennan AO MB CHb MD FRCOG FRANZCOG. The Robinson Research Institute, The University of Adelaide, Australia – Prof John M Dwyer AO PhD FRACP FRCPI Doc Uni(Hon) ACU. Emeritus Professor of Medicine, University of New South Wales. Founder of the Australian Health Care Reform Alliance. Clinical consultant to the NSW Government’s Inter-Agency committee on Health Care Fraud, Australia – A/Prof Steven M Novella, clinical neurologist Yale University School of Medicine, Connecticut, USA – Prof William M London, EdD, MPH, Department of Public Health, California State University, Los Angeles, USA – Dr Steven Barrett, MD, retired psychiatrist, author, co-founder of the National Council Against Health Fraud (NCAHF), USA – Prof. Steven L. Salzberg, Ph.D., Bloomberg Distinguished Professor of Biomedical Engineering, Computer Science, and Biostatistics, Johns Hopkins University School of Medicine, USA – Prof Christopher C French, Head of the Anomalistic Psychology Research Unit, Department of Psychology, Goldsmiths, University of London, UK – Dr Cees Renckens MD PhD, gynaecologist, past president of the Dutch Society against Quackery, Netherlands – Dr Alain Braillon. MD PhD. Senior consultant. University hospital, France – Dr John McLennan, MBBS FRACP, Paediatrician, Vic – Prof Shaun Holt, BPharm(hons), MBChB(hons), Medical Researcher, Victoria University of Wellington, New Zealand – Dr Lloyd B Oppel, MD, MHSc, Canada – Professor Asbjørn Hróbjartsson, Centre for Evidence-Based Medicine, University of Southern Denmark & Odense University Hospital, Denmark – Prof Maurizio Pandolfi MD, Florence, former Professor of Clinical Ophthalmology, The University of Lund, Sweden, Italy – Professor Mark Baker, Centre for Clinical Practice Director, National Institute for Health and Care Excellence (NICE), UK
According to Sir Richard: “From everything I have read about acupuncture I have to conclude that the evidence for efficacy is just not there. I can believe it has a very strong and effective placebo effect, but if it really worked as advertised why are the numbers of successful outcomes so small when compared to treatments such as drugs that really do work. As a scientist, who likes to see proper experiments carried out so that the results can be judged with a rational analysis, the experiments I have read about just don’t meet even a low bar of acceptability. I certainly do not believe it should be endorsed as an effective treatment by any professional scientific or medical body that values its reputation.”
According to Professor Bogduk: “Although studies have shown that acupuncture “works”, the definition of “works” is generous. Most studies show minimal to no effect greater than that of sham therapy. Needles do not need to be placed at specific points; so, learning about meridians is not required. Effectiveness is marginally greater in those patients who believe in acupuncture or expect it to work. However, no studies have shown that acupuncture stops pain, while also restoring normal function and removing the need for other health care.”
According to Professor Caulfield: “In popular culture, acupuncture is often portrayed as being effective for a range of conditions. It is held up as an alternative medicine success story. In fact, the relevant data are, at best, equivocal. The most rigorous studies, such as those that are well controlled and use sham comparators, have found that in most situations acupuncture is little better than placebo. More importantly, the supernatural foundations of the practice – that illness can be attributed to an imbalance in a life force energy – has absolutely no scientific basis. Given this reality, public representations of acupuncture that present it as science-based and effective can be deeply misleading. Policies are needed to counter this noise, including, inter alia, the more aggressive deployment of truth-in-advertising regulations, the enforcement of a conceptually consistent science-based informed consent standard, and the oversight of healthcare professionals by the relevant regulatory entities.”
According to Professor Kanellopoulos: “According to the systematic reviews in the field of acupuncture, the benefits of the method, if any, are nothing more than a temporary placebo effect. From a scientific point of view, acupuncture is based on a theory, which has nothing to do with modern physiology and medicine. From a researcher’s point of view, any presented acupuncture effectiveness is due to methodological errors, data manipulation, statistical artefacts and (purposely?) poorly designed clinical trials in general. Finally, regarding the patient, any symptom’s relief comes from despair and post hoc fallacy. After decades of research and over 3000 clinical trials, any continuation of practicing, advertising, and research in the field of acupuncture is a waste of resources and puts the patients at risk, raising ethical issues for both science and society.”
According to Professor Campbell: “Acupuncture holds great theatrical appeal through its dramatic and historical aspects, particularly to those who feel that conventional medicine has failed to offer pain relief or sufficient improvement in symptoms. However an extensive body of data now exists from rigorous approaches to testing the validity of its claims of benefit actually related to the placement of the needles and not to placebo effect. For example, most recently the beneficial effect achieved in relieving fatigue in Parkinsons Disease (and there was one) was identical in a randomised controlled trial to that of placebo.”
According to Professor Donald M. Marcus: “When trials of acupuncture for relief of pain of osteoarthritis of the knee or back pain include a sham acupuncture control, there is no clinically relevant difference in efficacy between the conventional and sham procedures. A number of sham procedures have been used, including toothpicks in a plastic guide tube in a study of back pain. It’s evident that relief of pain, and probably other complaints, by acupuncture is mediated by a placebo mechanism. Since there is no scientific evidence supporting its efficacy, medical insurance should not pay for acupuncture treatments. Moreover, it is unethical to deceive patients by providing a placebo treatment without disclosure.”
According to pain specialist Dr Vagg: “Due to the lack of a scientifically plausible mechanism, and the poor quality of the bulk of the research concerning acupuncture in its many and varied forms, no credible body of pain medicine researchers or clinicians has endorsed any type of acupuncture as a recommended treatment for any identifiable group of patients with persistent pain. Moreover, there is no reason to suppose that further research of high quality will change this conclusion, given that high-quality, randomized and double-blinded studies have uniformly shown that any form of acupuncture is indistinguishable from placebo, making further research unwarranted.”
According to Professor Garrett: “Current levels of evidence on acupuncture as a therapeutic intervention for any condition is very poor. Most studies reported are of very poor quality and are not reliable. Unfortunately, there is a strong element of propaganda in the dissemination of support for acupuncture in China, as it is a part of the Traditional Chinese Medicine supported by the government there. As such, much research has been demonstrated to involve data fabrication and extreme levels of confirmation bias. There are also strong ethical concerns about research involving acupuncture in China for anesthesia or other conditions where there is no established clinical theoretical basis for its use, and far better established therapeutics are available. Overall the current state of evidence on acupuncture is that the effectiveness of acupuncture as a treatment of any health condition remains unproven, and the only good quality trials have identified it has no better outcomes than placebo. Therefore, any claims of efficacy made against specific medical conditions are deceptive.”
According to Professor Gorski: “Acupuncture seems to garner more belief because it seems more plausible. The reason is that, unlike many other alternative therapies, acupuncture actually involves a physical act, namely inserting needles into the skin. However, it is also the case that the more acupuncture has been studied, the more it has become clear that it is, as David Colquhoun and Steve Novella put it, nothing more than a theatrical placebo. Indeed, as acupuncture is more rigorously studied in randomized clinical trials with proper controls and proper blinding, the more its seeming effects disappear, so that it becomes indistinguishable from placebo. Nor is it without risk, either. Recommending acupuncture to treat any condition is, from an ethical and scientific view, indefensible.”
According to Professor Bartecchi: “Acupuncture has no medical value other than that of a placebo. Acupuncture as viewed by many of us in academic medicine is merely an elaborate, theatrical placebo, a pre-scientific superstition which lacks a plausible mechanism. It really fits the bill as an alternative medicine hoax.”
According to Professor Colquhoun: “After over 3000 trials, some of them very well designed, there is still argument about the effectiveness of acupuncture. If that were the case for a new drug, it would long since have been abandoned. The literature suggests that acupuncture has only a small and variable placebo effect: too small to be of noticeable benefit to patients. Most of its apparent effects result from a statistical artefact, regression to the mean. The continued use of acupuncture probably arises from the lack of effective treatments for conditions like non-specific low back pain. That cannot be justified, Neither is it worth spending yet more money on further research. The research has been done and it failed to produce convincing evidence.”
According to Professor Ernst: “The current evidence on acupuncture is mixed. Many trials are less than rigorous and thus not reliable. Much of the research comes from China where data fabrication has been disclosed to be at epidemic levels; it would therefore be a mistake to rely on studies from China which almost invariably report positive results. If we account for such caveats and critically review the literature, we arrive at the following conclusions: – Acupuncture is clearly not free of risks, some of which are serious; – The effectiveness of acupuncture as a treatment of any condition remains unproven, and – The current research in this area is mostly pseudo-research aimed at promoting rather than testing acupuncture”.
According to Professor Costa: “Acupuncture as a part of Traditional Chinese Medicine is not based on science simply because, as for all pre-scientific medicines, whether Greco-Roman-European, Indian or any other, none are founded on any evidence. As a Neuroscientist, I teach medical and non medical students the very foundations of how the nervous system works and how sensory stimulation affects the brain. There simply is no evidence that twigging the skin with needles or, for that matter with toothpicks, does any more than create an expectation to feel better. This is the well-known placebo effect. Selling placebos under the disguise of medicine is totally unethical.”
According to Professor MacLennan: “Acupuncture is elaborate quackery and like many placebos sold by those without responsibility for or knowledge of the wide range of health disorders and disease it can be dangerous. Dangerous because acupuncture may delay correct diagnosis and therapy, dangerous because it may delay possible evidence-based therapies and allow progression of disorders present and dangerous because it sucks limited health resources from the community. Acupuncturists derive their income from elaborate subterfuge, taking advantage of the gullible unwell who are desperate, uneducated and seek a magic cure. If there is a placebo effect it is usually temporary, and eventually disappointment from lack of long term effect may lead to secondary depression in the patient. According to Professor Dwyer: “Modern understanding of human anatomy and the distribution and function of the components of the human nervous system make a nonsense of theories that suggest there are invisible meridians criss-crossing the body wherein there are trigger spots which, when stimulated, can produce an array of benefits remote from that site. Scientists however, while dismissing the prescientific explanations offered by traditional Chinese medicine, have sought other reasons why acupuncture might provide clinical benefits particularly the relief of pain. Numerous theories have been addressed by numerous studies with many being conducted using disciplined scientific methods. The conclusions leave us with no doubt that acupuncture provides the scenario for a superb theatrical placebo; no more.”
According to Dr Novella: “Pain is a big problem. If you read about pain management centers, you might think it had been solved. It has not. And when no effective treatment exists for a medical problem, it leads to a tendency to clutch at straws. Research has shown that acupuncture is little more than such a straw. It is clear from meta-analyses that results of acupuncture trials are variable and inconsistent, even for single conditions. After thousands of trials of acupuncture and hundreds of systematic reviews, arguments continue unabated. In 2011, Pain published an editorial that summed up the present situation well.”
According to Professor London & Dr Barrett: “The optimistic article by Vickers et al did not consider an important point. Research studies may not reflect what takes place in most acupuncturist offices. Most acupuncturists are graduates of “oriental medical schools,” where they learn about 5element theory, “energy” flow through meridians, and other fanciful traditional Chinese medicine (TCM) concepts that do not correspond with scientific knowledge of anatomy, physiology, or pathology. Practitioners of TCM typically rely on inappropriate diagnostic procedures (pulse and tongue diagnosis) and prescribe herbal mixtures that have not been sufficiently studied. Diagnoses based on TCM such as “Qi stagnation,” “blood stagnation,” “kidney Qi deficiency,” and “yin deficiency” may not jeopardize patients who are treated in an academic setting, where they have received a medical diagnosed before entering the study. But what about people with conditions that TCM-trained acupuncturists are not qualified or inclined to diagnose? Real-world evaluations of acupuncture should also consider the cost of unnecessary treatment.”
According to Professor Salzberg: “Acupuncture is a pre-scientific practice that persists only because of relentless and often very clever marketing by its proponents. The claimed mechanisms by which acupuncture works are clearly and obviously false: modern physiology, neurology, cell biology, and other scientific disciplines explain how pain signals are transmitted in the body, and none of them support the supposed “qi” or energy fields flowing along “meridians,” as acupuncturists describe them. Hundreds of scientific studies have shown that acupuncture doesn’t work for any medical condition. Acupuncture proponents ignore the evidence and persist, primarily because they profit from their practices. There are also documented risks of complications from acupuncture, ranging from infections to punctured lungs. For these and other reasons, recommending acupuncture for any patient is simply unethical. Acupuncturists make profits by putting patients at risk.”
According to Professor French: “Acupuncture has been extensively evaluated with respect to its possible therapeutic effectiveness for a wide range of disorders. The overall conclusion from meta-analyses of such studies is that any beneficial effects reported are small in terms of effect size and probably best accounted for in terms of statistical artefacts and placebo effects, etc. In general, the higher the quality of the study, the less likely are any beneficial effects to be reported. In light of this, it would be unwise and unethical to recommend acupuncture as the treatment of choice for any condition.”
According to Dr Renckens: “In 1683 the Dutch physician Willem ten Rhijne published the first book in the western world in which the word ‘acupuncture’ was mentioned, which referred to – as the Dutch title of the book was – ‘The Chinese and Japanese way of curing all diseases and especially the podagra by burning moxa and stabbing the Golden Needle’. This exotic treatment did not gain any popularity in the Netherlands and was mainly ridiculed. This heavenly situation remained unchanged until Nixon’s trip to China (1972) and the ‘successful’ acupuncture-treatment of the journalist James Reston of the New York Times. His story in that influential newspaper caused worldwide interest in acupunctures possible benefits. Also in the Netherlands and as early as 1989 a series of systematic reviews on the efficacy of acupuncture in a number of diseases was published in the Huisarts & Wetenschap, a journal of GP’s in the Dutch language (Ter Riet et al. H&W,1989;32:308-312).Their final conclusion was: ‘the main achievement of Chinese acupuncture is to have discovered a number of spots on the human body into which needles can be safely inserted’. The huge amount of scientific research into acupuncture has since been unable to undermine this right conclusion.”
According to Dr Braillon: “No discrimination! The US Federal Trade Commission announced that homeopathic drugs should “be held to the same truthin-advertising standards as other products claiming health benefits”; very soon, homeopathic products will include statements indicating: “There is no scientific evidence backing homeopathic health claims” and “Homeopathic claims are based only on theories from the 1700s that are not accepted by modern medical experts.” In Australia, the Royal Australian College of General Practitioners formally recommended GPs to ban homeopathic products from their prescriptions and pharmacists to ban them from their shelves. The same should be required for acupuncture.”
According to Dr McLennon: “Despite claims for effectiveness, there have been very few studies of acupuncture on children that have confirmed significant benefits. Conditions such as headache, abdominal pain, bed wetting and fibromyalgia and behaviour problems such as ADHD have been investigated. More trials with better structure have universally been recommended. A double blinded trial on the treatment of headaches with laser acupuncture illustrates the problems. The number of patients was quite small (21 in each arm), the diagnoses were reasonable medically but required rediagnosis to fit Traditional Chinese Medicine criteria and treatments were individualised based on these diagnoses. It was not made clear whether the patients were completely blinded i.e. unaware they received active treatment or placebo. Until blinding can be guaranteed, trials of acupuncture will remain inconclusive.”
According to Professor Holt: “Unlike some alternative therapies, acupuncture has been extensively studied for many medical conditions and a summary would be that the higher the quality of the study, the less likely it is that a benefit other than a placebo effect is found. Studies have shown conclusively that a key aspect of acupuncture, putting needles into energy lines for medical benefits, is not true, and the same effect is elicited wherever the needles are placed. Acupuncture is not a science-based practice, can cause side effects and is not recommended for any medical condition.”
According to Dr Oppel: “It is extremely concerning that there remains no plausible rationale for a mechanism of action of acupuncture. It is noteworthy that different schools of acupuncture offer contradictory patterns of treatment. It should not go without notice that acupuncture has been so well-researched that there are hundreds , if not thousands, of clinical trials now available Unfortunately, although there is no compelling evidence of effectiveness for any of the myriad of conditions where acupuncture is claimed to be of benefit, poor quality unreplicated trials continue to be put forward by proponents as proof of acupuncture’s effectiveness. Critical thinkers will also take note that while the large majority of acupuncture trials are positive, the vast majority of properly controlled trials are not. We are in a situation now where we have excellent evidence that acupuncture is not effective.”
According to Professor Hróbjartsson: “While there have been many trials done with acupuncture, most of them are small pilot studies and large scale high quality trials are rare. Some studies have reported measurable effects, but the mechanism is not yet understood, the size of the effect is small and it is possible that a large part of the effect or all of the effect is placebo. It is obvious that you would see a physiological effect when you stick a needle into your body, the question is whether that has a measurable clinical effect. There is insufficient evidence to say that electro acupuncture is any more or any less effective.”
According to Professor Pandolfi: “With a rationale completely disconnected from the basic principles of science acupuncture cannot be considered as belonging to modern evidence–based medicine.”
According to Professor Baker: “Millions of people are affected every year by these often debilitating and distressing conditions. For most their symptoms improve in days or weeks. However for some, the pain can be distressing and persist for a long time. Regrettably there is a lack of convincing evidence of effectiveness for some widely used treatments. For example acupuncture is no longer recommended for managing low back pain with or without sciatica. This is because there is not enough evidence to show that it is more effective than sham treatment.”
The BMJ has always been my favourite Medical journal. (Need any proof for this statement? A quick Medline search tells me that I have over 60 publications in the BMJ.) But occasionally, the BMJ also disappoints me a great deal.
One of the most significant disappointments was recently published under the heading of STATE OF THE ART REVIEW. A review that is ‘state of the art’ must fulfil certain criteria; foremost it should be informative, unbiased and correct. The paper I am discussing here has, I think, neither of these qualities. It is entitled ‘Management of chronic pain using complementary and integrative medicine’, and here is its abstract:
Complementary and integrative medicine (CIM) encompasses both Western-style medicine and complementary health approaches as a new combined approach to treat a variety of clinical conditions. Chronic pain is the leading indication for use of CIM, and about 33% of adults and 12% of children in the US have used it in this context. Although advances have been made in treatments for chronic pain, it remains inadequately controlled for many people. Adverse effects and complications of analgesic drugs, such as addiction, kidney failure, and gastrointestinal bleeding, also limit their use. CIM offers a multimodality treatment approach that can tackle the multidimensional nature of pain with fewer or no serious adverse effects. This review focuses on the use of CIM in three conditions with a high incidence of chronic pain: back pain, neck pain, and rheumatoid arthritis. It summarizes research on the mechanisms of action and clinical studies on the efficacy of commonly used CIM modalities such as acupuncture, mind-body system, dietary interventions and fasting, and herbal medicine and nutrients.
The full text of this article is such that I could take issue with almost every second statement in it. Obviously, this would be too long and too boring for this blog. So, to keep it crisp and entertaining, let me copy the (tongue in cheek) ‘letter to the editor’ some of us published in the BMJ as a response to the review:
“Alternative facts are fashionable in politics these days, so why not also in healthcare? The article by Chen and Michalsen on thebmj.com provides a handy set of five instructions for smuggling alternative facts into medicine.
1. Create your own terminology: the term ‘complementary and integrated medicine’ (CIM) is nonsensical. Integrated medicine (a hotly disputed field) already covers complementary and conventional medicine.
2. Pretend to be objective: Chen and Michalsen elaborate on the systematic searches they conducted. But they omit hundreds of sources which do not support their message, which cherry-picks only evidence for the efficacy of the treatments they promote.
3. Avoid negativity: they bypass any material that might challenge what they include. For instance, when discussing therapeutic risks, they omit the disturbing lack of post-marketing surveillance: the reason we lack information on adverse events. They even omit to mention the many fatalities caused by their ‘CIM’.
4. Create an impression of thoroughness: Chen and Michalsen cite a total of 225 references. This apparent scholarly attention to detail masks their misuse of many of they list. Reference 82, for example, is employed to back up the claim that “satisfaction was lowest among complementary medicine users with rheumatoid arthritis, vasculitis, or connective tissue diseases”. In fact, it shows nothing of the sort.
5. Back up your message with broad generalisations: Chen and Michalsen conclude that “Taken together, CIM has an increasing role in the management of chronic pain, but high quality research is needed”. The implication is that all the CIMs mentioned in their figure 1 are candidates for pain control – even discredited treatments such as homeopathy.
In our view, these authors render us a service: they demonstrate to the novice how alternative facts may be used in medicine.”
James May, Edzard Ernst, Nick Ross, on behalf of HealthWatch UK
END OF QUOTE
I am sure you have your own comments and opinions, and I encourage you to post them here or (better) submit them to the BMJ or (best) both.
As the data suggesting that homeopathy is effective for improving health is – to put it mildly – less than convincing, a frantic search is currently on amongst homeopaths and their followers to identify a specific condition for which the evidence is stronger than for all conditions pooled into one big analysis. If they could show that it works for just one disease, they could celebrate this finding and henceforth use it for refuting doubters stating that highly diluted homeopathic remedies are pure placebos. One such condition is allergic rhinitis; there have been several trials suggesting that homeopathy might be effective for it, and therefore it is only logical that homeopathy-promoters want to summarise these data in order to silence sceptics once and for all.
A new paper ought to be seen in this vein. It is systematic review by the Mathie group with the stated aim “to evaluate the efficacy and effectiveness of homeopathic intervention in the treatment of seasonal or perennial allergic rhinitis (AR).”
Randomized controlled trials evaluating all forms of homeopathic treatment for AR were included in a systematic review (SR) of studies published up to and including December 2015. Two authors independently screened potential studies, extracted data, and assessed risk of bias. Primary outcomes included symptom improvement and total quality-of-life score. Treatment effect size was quantified as mean difference (continuous data), or by risk ratio (RR) and odds ratio (dichotomous data), with 95% confidence intervals (CI). Meta-analysis was performed after assessing heterogeneity and risk of bias.
Eleven studies were eligible for SR. All trials were placebo-controlled except one. Six trials used the treatment approach known as isopathy, but they were unsuitable for meta-analysis due to problems of heterogeneity and data extraction. The overall standard of methods and reporting was poor: 8/11 trials were assessed as “high risk of bias”; only one trial, on isopathy for seasonal AR, possessed reliable evidence. Three trials of variable quality (all using Galphimia glauca for seasonal AR) were included in the meta-analysis: nasal symptom relief at 2 and 4 weeks (RR = 1.48 [95% CI 1.24-1.77] and 1.27 [95% CI 1.10-1.46], respectively) favoured homeopathy compared with placebo; ocular symptom relief at 2 and 4 weeks also favoured homeopathy (RR = 1.55 [95% CI 1.33-1.80] and 1.37 [95% CI 1.21-1.56], respectively). The single trial with reliable evidence had a small positive treatment effect without statistical significance. A homeopathic and a conventional nasal spray produced equivalent improvements in nasal and ocular symptoms.
The authors concluded that the low or uncertain overall quality of the evidence warrants caution in drawing firm conclusions about intervention effects. Use of either Galphimia glauca or a homeopathic nasal spray may have small beneficial effects on the nasal and ocular symptoms of AR. The efficacy of isopathic treatment of AR is unclear.
Extracts of Galphimia glauca (GG) have been used traditionally in South America for the treatment of allergic conditions, with some reports suggesting effectiveness. A 1997 meta-analysis of 11 clinical trials (most of them of very poor quality) of homeopathic GG suggested this therapy to be effective in the treatment of AR. In 2011, I published a review (FACT 2011, 16 200-203) focussed exclusively on the remarkable set of RCTs of homeopathic Galphimia glauca (GG). My conclusions were as follows: three of the four currently available placebo-controlled RCTs of homeopathic GG suggest this therapy is an effective symptomatic treatment for hay fever. There are, however, important caveats. Most essentially, independent replication would be required before GG can be considered for the routine treatment of hay fever. Since then, no new studies have emerged.
I am citing this for two main reasons:
- There is nothing homeopathic about the principle of using GG for allergic conditions; according to homeopathic theory GG extracts would need to cause allergies for GG to have potential as a homeopathic allergy remedy. Arguably, the GG trials should therefore have been excluded from this meta-analysis for not following the homeopathic principal of ‘like cures like’.
- All the RCTs of GG were done by the same German research group. There is not a single independent replication of their findings!
Seen from this perspective, the conclusion by Mathie et al, that the use of either Galphimia glauca … may have small beneficial effects on the nasal and ocular symptoms of AR, seems more than a little over-optimistic.
The question whether spinal manipulative therapy (SMT) is effective for acute low back pain is still discussed controversially. Chiropractors (they use SMT more regularly than other professionals) try everything to make us believe it does work, while the evidence is far less certain. Therefore, it is worth considering the best and most up-to-date data.
The aim of this paper was to systematically review studies of the effectiveness and harms of SMT for acute (≤6 weeks) low back pain. The research question was straight forward: Is the use of SMT in the management of acute (≤6 weeks) low back pain associated with improvements in pain or function?
A through literature search was conducted to locate all relevant papers. Study quality was assessed using the Cochrane Back and Neck (CBN) Risk of Bias tool. The evidence was assessed using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) criteria. The main outcome measures were pain (measured by either the 100-mm visual analog scale, 11-point numeric rating scale, or other numeric pain scale), function (measured by the 24-point Roland Morris Disability Questionnaire or Oswestry Disability Index [range, 0-100]), or any harms measured within 6 weeks.
Of 26 eligible RCTs identified, 15 RCTs (1711 patients) provided moderate-quality evidence that SMT has a statistically significant association with improvements in pain (pooled mean improvement in the 100-mm visual analog pain scale, −9.95 [95% CI, −15.6 to −4.3]). Twelve RCTs (1381 patients) produced moderate-quality evidence that SMT has a statistically significant association with improvements in function (pooled mean effect size, −0.39 [95% CI, −0.71 to −0.07]). Heterogeneity was not explained by type of clinician performing SMT, type of manipulation, study quality, or whether SMT was given alone or as part of a package of therapies. No RCT reported any serious adverse event. Minor transient adverse events such as increased pain, muscle stiffness, and headache were reported 50% to 67% of the time in large case series of patients treated with SMT.
The authors concluded that among patients with acute low back pain, spinal manipulative therapy was associated with modest improvements in pain and function at up to 6 weeks, with transient minor musculoskeletal harms. However, heterogeneity in study results was large.
This meta-analysis has been celebrated by chiropractors around the world as a triumph for their hallmark therapy, SMT. But there have also been more cautionary voices – not least from the lead author of the paper. Patients undergoing spinal manipulation experienced a decline of 1 point in their pain rating, says Dr. Paul Shekelle, an internist with the West Los Angeles Veterans Affairs Medical Center and the Rand Corporation who headed the study. That’s about the same amount of pain relief as from NSAIDs, over-the-counter nonsteroidal anti-inflammatory medication, such as ibuprofen. The study also found spinal manipulation modestly improved function. On average, patients reported greater ease and comfort engaging in two day-to-day activities — such as finding they could walk more quickly, were having less difficulty turning over in bed or were sleeping more soundly.
It’s not clear exactly how spinal manipulation relieves back pain. But it may reposition the small joints in the spine in a way that causes less pain, according to Dr. Richard Deyo, an internist and professor of evidence-based medicine at the Oregon Health and Science University. Deyo wrote an editorial published along with the study. Another possibility, Deyo says, is that spinal manipulation may restore some material in the disk between the vertebrae, or it may simply relax muscles, which could be important. There may also be mind-body interaction that comes from the “laying of hands” or a trusting relationship between patients and their health care provider, he says.
Deyo notes that there are many possible treatments for lower back pain, including oral medicine, injected medicine, corsets, traction, surgery, acupuncture and massage therapy. But of about 200 treatment options, “no single treatment is clearly superior,” he says.
In another comment by Paul Ingraham the critical tone was much clearer: “Claiming it as a victory is one of the best examples I’ve ever seen of making lemonade out of science lemons! But I can understand the mistake, because the review itself does seem positive at first glance: the benefits of SMT are disingenuously summarized as “statistically significant” in the abstract, with no mention of clinical significance (effect size; see Statistical Significance Abuse). So the abstract sounds like good news to anyone but the most wary readers, while deep in the main text the same results are eventually conceded to be “clinically modest.” But even even that seems excessively generous: personally, I need at least a 2-point improvement in pain on a scale of 10 to consider it a “modest” improvement! This is not a clearly positive review: it shows weak evidence of minor efficacy, based on “significant unexplained heterogeneity” in the results. That is, the results were all over the place — but without any impressive benefits reported by any study — and the mixture can’t be explained by any obvious, measurable factor. This probably means there’s just a lot of noise in the data, too many things that are at least as influential as the treatment itself. Or — more optimistically — it could mean that SMT is “just” disappointingly mediocre on average, but might have more potent benefits in a minority of cases (that no one seems to be able to reliably identify). Far from being good news, this review continues a strong trend (eg Rubinstein 2012) of damning SMT with faint praise, and also adds evidence of backfiring to mix. Although fortunately “no RCT reported any serious adverse event,” it seems that minor harms were legion: “increased pain, muscle stiffness, and headache were reported 50% to 67% of the time in large case series of patients treated with SMT.” That’s a lot of undesirable outcomes. So the average patient has a roughly fifty-fifty chance of up to roughly maybe a 20% improvement… or feeling worse to some unknown degree! That does not sound like a good deal to me. It certainly doesn’t sound like good medicine.”
END OF QUOTE
As I have made clear in many previous posts, I do fully agree with these latter statements and would add just three points:
- We know that many of the SMT studies completely neglect reporting adverse effects. Therefore it is hardly surprising that no serious complications were on record. Yet, we know that they do occur with sad regularity.
- None of the studies controlled for placebo effects. It is therefore possible – I would say even likely – that a large chunk of the observed benefit is not due to SMT per se but to a placebo response.
- It seems more than questionable whether the benefits of SMT outweigh its risks.
‘What Doctors Don’t Tell You’ (WDDTY) have been shown to be strangely economical with the truth many times before (for instance here, here and here). Now they have published an article entitled ‘Ombudsman investigates ‘flawed’ homeopathic study that claimed it doesn’t work’ It attacks in no uncertain terms the ‘NHMRC Statement on Homeopathy and NHMRC Information Paper – Evidence on the effectiveness of homeopathy for treating health conditions’ which I believe to be a sound evaluation of homeopathy and therefore have mentioned repeatedly on this blog. Here is what WDDTY stated:
START OF QUOTE
A major and influential review of homeopathy concluded that the controversial therapy doesn’t work—but it was so riddled with error and bad science that it’s sparked an official ombudsman investigation.
The world’s media announced that homeopathy was a scam after the Australia government’s National Health and Medical Research Council (NHMRC) published its findings in 2015 that “there are no health conditions for which there is reliable evidence that homeopathy is effective.”
But now the Commonwealth Ombudsman is investigating the review’s procedures after receiving reports of inaccuracies, mishandling of evidence and conflicts of interest.
The review has been triggered by the Australian Homeopathic Association (AHA), supported by the Homeopathic Research Institute (HRI), which began questioning the review’s processes after several solid studies that demonstrated homeopathy’s benefits had been overlooked.
The NHMRC review team set arbitrary parameters that only studies that involved more than 150 people—and which met standards that even drug trials rarely achieve—would be considered. Those requirements reduced the number of qualifying studies to just five—from an initial pool of more than 1,800 trials—and none of these showed that homeopathy was effective.
One of the NHMRC’s own reviewers produced a mysterious first report that has never been published, and hasn’t been released despite Freedom of Information requests.
And the AHA has discovered that Prof Peter Brooks, chair of the NHMRC committee that carried out the homeopathy review, never declared that he was a member of the anti-homeopathy lobby group, Friends of Science in Medicine.
There are solid studies that demonstrate homeopathy is effective against childhood diarrhea, sinusitis and hay fever—but they all involve fewer than 150 people, said HRI chief executive Rachel Roberts. “The public has a right to know that there are high quality studies showing homeopathy works for some medical conditions—information that was lost only due to NHMRC’s mishandling of the evidence.”
The homeopaths aren’t alone in challenging the NHMRC review: Australia’s independent Cochrane Centre said its conclusions are not an accurate reflection of the evidence, and a second expert also said he felt “uncertain of the definitive nature of the report’s conclusions.”
END OF QUOTE
As it happens, I am in contact with the lead author of this report, Paul Glasziou, not least because he very kindly wrote the foreword for my book HOMEOPATHY, THE UNDILUTED FACTS. So, we corresponded and discussed the latest WDDTY diatribe. Thus I am now in a position to put a few things straight (I hope Paul does not mind).
ISSUE 1. – The NHMRC review team set arbitrary parameters that only studies involving more than 150 people—and which met standards that even drug trials rarely achieve—would be considered.
The truth is that report focused on systematic reviews of trials, not individual trials. The 57 included systematic reviews found 176 individual trials which covered 61 conditions: an average of about 3 trials per condition. But some conditions only had 1 trial, and one small trial would, of course, not be considered a reasonable basis for reliable conclusions. GRADE – the international standard for assessing evidence – downgrades reviews for “imprecision” – the GRADE Handbook suggests “whenever there are sample sizes that are less than 400, review authors and guideline developers should certainly consider rating down for imprecision.” Hence the criterion of 150 which the Australians decided to use is considerably more lenient than the current GRADE guideline.
ISSUE 2 – Those requirements reduced the number of qualifying studies to just five—from an initial pool of more than 1,800 trials—and none of these showed that homeopathy was effective.
This is simply not correct. The report found 57 systematic reviews that contained 176 individual trials, not 5. These 176 trials, which covered 61 conditions, formed the body of evidence for the NHMRC report’s conclusions.
ISSUE 3 – There are solid studies that demonstrate homeopathy is effective against childhood diarrhoea, sinusitis and hay fever—but they all involve fewer than 150 people, said HRI chief executive Rachel Roberts.
The NHMRC report focused on systematic reviews that covered all trials for individual conditions. Given the conventional p-value of 0.05, one would expect 1 in 20 single trials to be “false positives”. So with 176 trials, we expect about 9 “false positive” trials. But using systematic reviews that combine all trials for individual conditions, reduces this risk of false positives. Most national evidence review bodies require more than 1 trial, e.g, the FDA requires 2 positive trials, whereas many others require a systematic review which has at least 2 trials. Replication of findings is obviously a cornerstone of science.
ISSUE 4 The homeopaths aren’t alone in challenging the NHMRC review: Australia’s independent Cochrane Centre said its conclusions are not an accurate reflection of the evidence, and a second expert also said he felt “uncertain of the definitive nature of the report’s conclusions.”
The truth is that the Cochrane Centre, which provided an independent check during the processes of the NHMRC review, concluded that “Overall, the conclusions arising from the review appear justified based on the evidence presented.”
I REST MY CASE.
This new systematic review by proponents of homeopathy (and supported by a grant from the Manchester Homeopathic Clinic) tested the null hypothesis that “the main outcome of treatment using a non-individualised (standardised) homeopathic medicine is indistinguishable from that of placebo“. An additional aim was to quantify any condition-specific effects of non-individualised homeopathic treatment. In reporting this paper, I will stay very close to the published text hoping that this avoids both misunderstandings and accusations of bias on my side:
Literature search strategy, data extraction and statistical analysis followed the methods described in a pre-published protocol. A trial comprised ‘reliable evidence’ if its risk of bias was low or it was unclear in one specified domain of assessment. ‘Effect size’ was reported as standardised mean difference (SMD), with arithmetic transformation for dichotomous data carried out as required; a negative SMD indicated an effect favouring homeopathy.
The authors excluded the following types of trials: studies of crossover design; of radionically prepared homeopathic medicines; of homeopathic prophylaxis; of homeopathy combined with other (complementary or conventional) intervention; for other specified reasons. The final explicit exclusion criterion was that there was obviously no blinding of participants and practitioners to the assigned intervention.
Forty-eight different clinical conditions were represented in 75 eligible RCTs; 49 were classed as ‘high risk of bias’ and 23 as ‘uncertain risk of bias’; the remaining three trials displayed sufficiently low risk of bias to be designated reliable evidence. Fifty-four trials had extractable data: pooled SMD was -0.33 (95% confidence interval (CI) -0.44, -0.21), which was attenuated to -0.16 (95% CI -0.31, -0.02) after adjustment for publication bias. The three trials with reliable evidence yielded a non-significant pooled SMD: -0.18 (95% CI -0.46, 0.09). There was no single clinical condition for which meta-analysis produced reliable evidence.
A meta-regression was performed to test specifically for within-group differences for each sub-group. The results showed that there were no significant differences between studies that were and were not:
- included in previous meta-analyses (p = 0.447);
- pilot studies (p = 0.316);
- greater than the median sample (p = 0.298);
- potency ≥ 12C (p = 0.221);
- imputed for meta-analysis (p = 0.384);
- free from vested interest (p = 0.391);
- acute/chronic (p = 0.796);
- different types of homeopathy (p = 0.217).
After removal of ‘C’-rated trials, the pooled SMD still favoured homeopathy for all sub-groups, but was statistically non-significant for 10 of the 18 (included in previous meta-analysis; pilot study; sample size > median; potency ≥12C; data imputed; free of vested interest; not free of vested interest; combination medicine; single medicine; chronic condition). There remained no significant differences between sub-groups—with the exception of the analysis for sample size > median (p = 0.028).
Meta-analyses were possible for eight clinical conditions, each analysis comprising two to 5 trials. A statistically significant pooled SMD, favouring homeopathy, was observed for influenza (N = 2), irritable bowel syndrome (N = 2), and seasonal allergic rhinitis (N = 5). Each of the other five clinical conditions (allergic asthma, arsenic toxicity, infertility due to amenorrhoea, muscle soreness, post-operative pain) showed non-significant findings. Removal of ‘C’-rated trials negated the statistically significant effect for seasonal allergic rhinitis and left the non-significant effect for post-operative pain unchanged; no higher-rated trials were available for additional analysis of arsenic toxicity, infertility due to amenorrhoea or irritable bowel syndrome. There were no ‘C’-rated trials to remove for allergic asthma, influenza, or muscle soreness. Thus, influenza was the only clinical condition for which higher-rated trials indicated a statistically significant effect; neither of its contributing trials, however, comprised reliable evidence.
The authors concluded that the quality of the body of evidence is low. A meta-analysis of all extractable data leads to rejection of our null hypothesis, but analysis of a small sub-group of reliable evidence does not support that rejection. Reliable evidence is lacking in condition-specific meta-analyses, precluding relevant conclusions. Better designed and more rigorous RCTs are needed in order to develop an evidence base that can decisively provide reliable effect estimates of non-individualised homeopathic treatment.
I am sure that this paper will lead to lively discussions in the comments section of this blog. I will therefore restrict my comments to a bare minimum.
In my view, this new meta-analysis essentially yield a negative result and confirms most previous, similar reviews.
- It confirms Linde’s conclusion that “insufficient evidence from these studies that homeopathy is clearly efficacious for any single clinical condition”.
- It confirms Linde’s conclusion that “there was clear evidence that studies with better methodological quality tended to yield less positive results”.
- It confirms Kleinjen’s conclusion that “most trials are of low methodological quality”.
- It also confirms the results of the meta-analysis by Shang et al (much-maligned by homeopaths) than “finding is compatible with the notion that the clinical effects of homoeopathy are placebo effects.”
- Finally, it confirms the conclusion of the analysis of the Australian National Health and Medical Research Council: “Homeopathy should not be used to treat health conditions that are chronic, serious, or could become serious. People who choose homeopathy may put their health at risk if they reject or delay treatments for which there is good evidence for safety and effectiveness. People who are considering whether to use homeopathy should first get advice from a registered health practitioner. Those who use homeopathy should tell their health practitioner and should keep taking any prescribed treatments.”
Another not entirely unimportant point that often gets missed in these discussions is this: even if we believe (which I do not) the most optimistic interpretation of these (and similar data) by homeopaths, we ought to point out that there is no evidence whatsoever that homeopathy cures anything. At the very best it provides marginal symptomatic relief. Yet, the claim of homeopaths that we hear constantly is that homeopathy is a causal and curative therapy.
The first author of the new meta-analysis is an employee of the Homeopathy Research Institute. We might therefore forgive him that he he repeatedly insists on dwelling on largely irrelevant (i. e. based on unreliable primary studies) findings. It seems obvious that firm conclusions can only be based on reliable data. I therefore disregard those analyses and conclusions that include such studies.
In the discussion, the authors of the new meta-analysis confirm my interpretation this by stating that they “reject the null hypothesis (non-individualised homeopathy is indistinguishable from placebo) on the basis of pooling all studies, but fail to reject the null hypothesis on the basis of the reliable evidence only.” And, in the long version of their conclusions, we find this remarkable statement: “Our meta-analysis of the current reliable evidence base therefore fails to reject the null hypothesis that the outcome of treatment using a non-individualised homeopathic medicine is not distinguishable from that using placebo.” A most torturous way of stating the obvious: the more reliable data show no difference between homeopathy and placebo.
Tui Na is a massage technique that is based on the Taoist principles of TCM. It involves a range of manipulations usually performed by an operator’s finger, hand, elbow, knee, or foot applied to muscle or soft tissue at specific parts of the body. According to one website of TCM-proponents “Tui Na makes use of various hand techniques in combination with acupuncture and other manipulation techniques. To enhance the healing process, the practitioner may recommend the use of Chinese herbs. Many of the techniques used in this massage resemble that of a western massage like gliding, kneading, vibration, tapping, friction, pulling, rolling, pressing and shaking. In Tui Na massage, the muscles and tendons are massaged with the help of hands, and an acupressure technique is applied to directly affect the flow of Qi at different acupressure points of the body, thus facilitating the healing process. It removes the blockages and keeps the energy moving through the meridians as well as the muscles. A typical session of Tui Na massage may vary from thirty minutes to an hour. The session timings may vary depending on the patient’s needs and condition. The best part of the therapy is that it relaxes as well as energizes the person. The main benefit of Tui Na massage is that it focuses on the specific problem, whether it is an acute or a chronic pain associated with the joints, muscles or a skeletal system. This technique is very beneficial in reducing the pain of neck, shoulders, hips, back, arms, highs, legs and ankle disorders. It is a very effective therapy for arthritis, pain, sciatica and muscle spasms. Other benefits of this massage therapy include alleviation of the stress related disorders like insomnia, constipation, headaches and other disorders related to digestive, respiratory and reproductive systems. The greatest advantage of Tui Na is that it focuses on maintaining overall balance with both physical and mental health. Any one who wants to avoid the side effects of drugs or a chemical based treatment can adopt this effective massage technique to alleviate their pain. Tui Na massage therapy is now becoming a more common therapy method due to its focus on specific problems rather than providing a general treatment.”
This clearly begs the question IS IT EFFECTIVE?
This systematic review assessed the evidence of Tui Na for cervical radiculopathy. Seven databases were searched. Randomised controlled trials (RCTs) incorporating Tui Na alone or Tui Na combined with conventional treatment were included. Five studies involving 448 patients were found. The pooled analysis from the 3 trials indicated that Tui Na alone showed a significant lowering immediate effects on pain score with moderate heterogeneity compared to cervical traction. The meta-analysis from 2 trials revealed significant immediate effects of Tui Na plus cervical traction in improving pain score with no heterogeneity compared to cervical traction alone. None of the RCTs mentioned adverse effects. There was very low quality or low quality evidence to support the results.
The authors concluded that “Tui Na alone or Tui Na plus cervical traction may be helpful to cervical radiculopathy patients, but supportive evidence seems generally weak. Future clinical studies with low risk of bias and adequate follow-up design are recommended.”
In my view, this is a misleading conclusion. A correct one would have been: THE CURRENT EVIDENCE IS INSUFFICIENT TO DRAW ANY CONCLUSIONS ABOUT THE EFFECTIVENESS OF TUI NA.
Here are some of the most obvious reasons:
- there are far too few studies for a firm conclusion,
- the included RCTs lack scientific rigour,
- all trials originate from China where reliability seems to be a serious problem,
- traction is not a useful therapy for radiculopathy,
- the primary studies violate research ethics by not reporting adverse effects.
Personally, I am getting very tired of conclusions stating ‘…XY MAY BE EFFECTIVE/HELPFUL/USEFUL/WORTH A TRY…’ It is obvious that the therapy in question MAY be effective, otherwise one would surely not conduct a systematic review. If a review fails to produce good evidence, it is the authors’ ethical, moral and scientific obligation to state this clearly. If they don’t, they simply misuse science for promotion and mislead the public. Strictly speaking, this amounts to scientific misconduct.
Is spinal manipulative therapy (SMT) dangerous? This question has kept us on this blog busy for quite some time now. To me, there is little doubt that SMT can cause adverse effects some of which are serious. But many chiropractors seem totally unconvinced. Perhaps this new overview of reviews might help to clarify the issue. Its aim was to elucidate and quantify the risk of serious adverse events (SAEs) associated with SMT.
The authors searched five electronic databases from inception to December 8, 2015 and included reviews on any type of studies, patients, and SMT technique. The primary outcome was SAEs. The quality of the included reviews was assessed using a measurement tool to assess systematic reviews (AMSTAR). Since there were insufficient data for calculating incidence rates of SAEs, they used an alternative approach; the conclusions regarding safety of SMT were extracted for each review, and the communicated opinion were judged by two reviewers independently as safe, harmful, or neutral/unclear. Risk ratios (RRs) of a review communicating that SMT is safe and meeting the requirements for each AMSTAR item, were calculated.
A total of 283 eligible reviews were identified, but only 118 provided data for synthesis. The most frequently described adverse events (AEs) were stroke, headache, and vertebral artery dissection. Fifty-four reviews (46%) expressed that SMT is safe, 15 (13%) expressed that SMT is harmful, and 49 reviews (42%) were neutral or unclear. Thirteen reviews reported incidence estimates for SAEs, roughly ranging from 1 in 20,000 to 1 in 250,000,000 manipulations. Low methodological quality was present, with a median of 4 of 11 AMSTAR items met (interquartile range, 3 to 6). Reviews meeting the requirements for each of the AMSTAR items (i.e. good internal validity) had a higher chance of expressing that SMT is safe.
The authors concluded that it is currently not possible to provide an overall conclusion about the safety of SMT; however, the types of SAEs reported can indeed be significant, sustaining that some risk is present. High quality research and consistent reporting of AEs and SAEs are needed.
This article is valuable, if only for the wealth of information one can extract from it. There are, however, numerous problems. One is that the overview included mostly reviews of the effectiveness of SMT for various conditions. We know that studies of SMT often do not even mention AEs. If such studies are then pooled in a review, they inevitably generate an impression of safety. But this would, of course, be a false-positive result!
The authors of the overview are aware of this problem and address it in the following paragraph: “When only considering the subset of reviews, where the objective was to investigate AEs (37 reviews), then 8 reviews (22%) expressed that SMT is safe, 13 reviews (35%) expressed that SMT is harmful and 16 reviews (43%) were neutral or unclear regarding the safety of SMT. Hence, there is a tendency that a bigger proportion of these reviews are expressing that SMT is harmful compared to the full sample of reviews…”
To my surprise, I found several of my own reviews in the ‘neutral or unclear’ category. Here are the verbatim conclusions of three of them:
- It is concluded that serious cerebrovascular complications of spinal manipulation continue to be reported.
- The most common serious adverse events are vertebrobasilar accidents, disk herniation, and cauda equina syndrome.
- These data indicate that mild and transient adverse events seem to be frequent. Serious adverse events are probably rare but their incidence can only be estimated at present.
I find it puzzling how this could be classified as neutral or unclear. The solution of the puzzle might lie in the methodology used: “we appraised the communicated opinions of each review concerning the safety of SMT based on their conclusions regarding the AEs and SAEs. This was done by two reviewers independently (SMN, LK), who judged the communicated opinions as either ‘safe’, ‘neutral/unclear’ or ‘harmful’, based on the qualitative impression the reviewers had when reading the conclusions. The reviewers had no opinion about the safety/harmfulness of SMT before commencing the judgements. Cohen’s weighted Kappa was calculated for the agreement between the reviewers, with a value of 0.40–0.59 indicating ‘fair agreement’, 0.60–0.74 indicating ‘good agreement’ and ≥0.75 indicating ‘excellent agreement’. Disagreements were resolved by a third reviewer (MH).”
In other words, the categorisation was done on the basis of subjective judgements of two researchers. It seems obvious that, if their attitude was favourable towards SMT, their judgements would be influenced. The three examples from my own work cited above indicates to me that their verdicts were indeed far from objective.
So what is the main message here? In my view, it can be summarized in the following quote from the overview: “a bigger proportion of these reviews are expressing that SMT is harmful …”
Yes, yes, yes – I know that, if you are a chiropractor (or other practitioner using mostly SMT), you are unlikely to agree with this!
Perhaps you can agree with this statement then:
As long as there is reasonable doubt about the safety of SMT, and as long as we cannot be sure that SMT generates more good than harm, we should be very cautious using it for routine healthcare and do rigorous research to determine the truth (it’s called the precautionary principle and applies to all types of healthcare).
Therapeutic Touch is a therapy mostly popular with nurses. We have discussed it before, for instance here, here, here and here. To call it implausible would be an understatement. But what does the clinical evidence tell us? Does it work?
This literature review by Iranian authors was aimed at critically evaluating the data from clinical trials examining the clinical efficacy of therapeutic touch as a supportive care modality in adult patients with cancer.
Four electronic databases were searched from the year 1990 to 2015 to locate potentially relevant peer-reviewed articles using the key words therapeutic touch, touch therapy, neoplasm, cancer, and CAM. Additionally, relevant journals and references of all the located articles were manually searched for other potentially relevant studies.
The number of 334 articles was found on the basis of the key words, of which 17 articles related to the clinical trial were examined in accordance with the objectives of the study. A total of 6 articles were in the final dataset in which several examples of the positive effects of healing touch on pain, nausea, anxiety and fatigue, and life quality and also on biochemical parameters were observed.
The authors concluded that, based on the results of this study, an affirmation can be made regarding the use of TT, as a non-invasive intervention for improving the health status in patients with cancer. Moreover, therapeutic touch was proved to be a useful strategy for adult patients with cancer.
This review is badly designed and poorly reported. Crucially, its conclusions are not credible. Contrary to what the authors stated when formulating their aims, the methods lack any attempt of critically evaluating the primary data.
A systematic review is more than a process of ‘pea counting’. It requires a rigorous assessment of the risk of bias of the included studies. If that crucial step is absent, the article is next to worthless and the review degenerates into a promotional excercise. Sadly, this is the case with the present review.
You may think that this is relatively trivial (“Who cares what a few feeble-minded nurses do?”), but I would disagree: if the medical literature continues to be polluted by such irresponsible trash, many people (nurses, journalists, healthcare decision makers, researchers) who may not be in a position to see the fatal flaws of such pseudo-reviews will arrive at the wrong conclusions and make wrong decisions. This will inevitably contribute to a hindrance of progress and, in certain circumstances, must endanger the well-being or even the life of vulnerable patients.