Systematic reviews are aimed at summarising and critically evaluating the evidence on a specific research question. They are the highest level of evidence and are more reliable than anything else we have. Therefore, they represent a most useful tool for both clinicians and researchers.
But there are, of course, exceptions. Take, for instance, this recent systematic review by researchers from the
- Texas Chiropractic College, Pasadena, the Palmer Center for Chiropractic Research, Palmer College of Chiropractic, Davenport,
- Department of Planning, Policy and Design, University of California, Irvine,
- VA Puget Sound Health Care System, Tacoma,
- New York Chiropractic College, Seneca Falls,
- Logan University College of Chiropractic, Chesterfield,
- University of Western States, Portland.
The ‘Daily Mail’ is not a paper famed for its objective reporting. In politics, this can influence elections; in medicine, it can endanger public health.
A recent article is a case in point, I think.
START OF QUOTE
Traditional Chinese medicines could help prevent heart disease and the progression of pre-diabetes, according to research. Some herbal treatments proved as effective in lowering blood pressure as Western drugs and improved heart health by lowering cholesterol, scientists found. Certain alternative medicines could lower blood sugar and insulin levels, too.
Chinese medicines could be used alongside conventional treatments, say researchers from Shandong University Qilu Hospital in China. Or they can be beneficial as an alternative for patients intolerant of Western drugs, they said in their review of medical studies over a ten-year period. Senior review author from the university’s department of traditional Chinese medicine said: ‘The pharmacological effects and the underlying mechanisms of some active ingredients of traditional Chinese medications have been elucidated. Thus, some medications might be used as a complementary and alternative approach for primary and secondary prevention of cardiovascular disease.”
It’s potentially good news for people living with diabetes, which is now a global epidemic and has proved a tricky condition to manage for many people. High blood pressure is very common too, affecting more than one in four adults in the UK, although many won’t show symptoms and realise it. If untreated, it increases your risk of serious problems including heart disease, the number one killer globally.
The Chinese have used herbs for treating diseases for thousands of years and have become increasingly popular in Europe and North America, mainly as complement to Western medicine. But the researchers also warn that much of the research conducted have limitations and so their long-term effects are not proven.
Herbs for high blood pressure
The blood pressure-lowering effect of herb zhongfujiangya was found to be similar to that of oral anti-hypertension medication benazeprilm, which goes by the brand name Lotensin. Similarly, patients treated for eight weeks with herbal tiankuijiangya had a lower reading than those given a placebo. Herbal Jiangya tablets were found to ‘significantly lower’ systolic blood pressure, that is the amount of pressure in your arteries during contraction of your heart muscle compared to a fake treatment. The herb Jiangyabao also had a significant effect compared to a placebo, but just at night. But overall, compared to the drug Nimodipine, a calcium channel blocker, it worked just as well. Qiqilian capsules also proved more effective compared to a placebo.
Herbs for diabetes
The team report some Chinese medicines medications – such as xiaoke, tangminling, jinlida, and jianyutangkang – have a ‘potent’ effect on lowering blood sugar levels and b-cell function, which controls the release of insulin. Some remedies – such as tangzhiping and tianqi – might prevent the progression of pre-diabetes to diabetes, they note.
Herbs for cholesterol
The researchers looked at research on dyslipidemia, the term for unbalanced or unhealthy cholesterol levels. They found that jiangzhitongluo, salviamiltiorrhiza and pueraria lobata, and zhibitai capsule all have a ‘potent lipid-lowing effect’.
Herbs for heart disease
Some traditional Chinese medicines such as qiliqiangxin, nuanxin, shencaotongmai, and yangxinkang, might be effective in improving function in patients with chronic heart failure, they wrote.
Limitations with trials
But Western scientists often reject Chinese medicine for specific reasons, warned Dr Zhao’s team. Chinese medicines are frowned upon because they do not go through the same exhaustive approval process as trials conducted domestically, they pointed out. Plus, one treatment can be made of many different ingredients with various chemical compounds, making it hard to pinpoint how their benefits work. ‘One should bear in mind that traditional Chinese medicine medications are usually prescribed as complex formulae, which are often further manipulated by the practitioner on a personalized basis,’ said Dr Zhao.
END OF QUOTE
Apart from the fact that this article is badly written, it is also misleading to the point of being outright dangerous. Regular readers of my blog will be aware that Chinese research is everything but reliable; there are practically no Chinese TCM-trials that report negative results. Furthermore, the safety of Chinese herbal preparations is as good as unknown and they are often contaminated with toxic substances as well as adulterated with synthetic drugs. Most of these preparations are also unavailable outside China. Moreover, Chinese herbal treatments are usually individualised (mixtures are tailor-made for each individual patient), and there is no good evidence that this approach is effective. Crucially, the trial evidence is often of such poor quality that it would be a dangerous mistake to trust these findings.
None of these important caveats, it seems, are important enough to get a mention in the Daily Mail.
Don’t let the truth get in the way of a sensational story!
Let’s just for a moment imagine what would happen if people took the Mail article seriously (is there anyone out there who does take the Mail seriously?). In a best case scenario, they would take Chinese herbs in addition to their prescribed medication. This might case plenty of unwanted side-effects and herb-drug interactions. In addition, people would lose a lot of their hard-earned cash. In a worst case scenario, they would abandon their prescribed medication for dubious Chinese herbal mixtures. This could cause thousands of premature deaths.
With just a little research, I managed to find the original article on which the Mail’s report was based. Here is its abstract:
Traditional Chinese medicine (TCM) has more than 2,000 years of history and has gained widespread clinical applications. However, the explicit role of TCM in preventing and treating cardiovascular disease remains unclear due to a lack of sound scientific evidence. Currently available randomized controlled trials on TCM are flawed, with small sample sizes and diverse outcomes, making it difficult to draw definite conclusions about the actual benefits and harms of TCM. Here, we systematically assessed the efficacy and safety of TCM for cardiovascular disease, as well as the pharmacological effects of active TCM ingredients on the cardiovascular system and potential mechanisms. Results indicate that TCM might be used as a complementary and alternative approach to the primary and secondary prevention of cardiovascular disease. However, further rigorously designed randomized controlled trials are warranted to assess the effect of TCM on long-term hard endpoints in patients with cardiovascular disease.
In my view, the authors of this review are grossly over-optimistic in their conclusions (but nowhere near as bad as the Mail journalist). If the trials are of poor quality, as the review-authors admit, no firm conclusions should be permissible about the usefulness of the therapies in question.
As the Mail article is obviously based on a press release (several other papers worldwide reported about the review as well), it seems interesting to note what the editor of the Journal of the American College of Cardiology (the journal that published the review) recently had to say about the responsibility of journalists and researchers:
START OF QUOTE
…I would like to suggest that journalists and researchers must share equally in shouldering the burden of responsibility to improve appropriate communication about basic and clinical research.
First, there is an obligation on the part of the researchers not to inflate the importance of their findings. This has been widely recognized as damaging, especially if bias is introduced in the paper…
Second, researchers should take some responsibility for the creation of the press release about their research, which is written by the media or press relations department at their hospital or society. Press releases are often how members of the media get introduced to a particular study, and these releases can often introduce errors or exaggerations. In fact, British researchers evaluated 462 press releases on biomedical and health-related science issued by 20 leading U.K. universities in 2011, alongside their associated peer-reviewed research papers and the news stories that followed (n = 668). They found that 40% of the press releases contained exaggerated advice, 33% contained exaggerated causal claims, and 36% contained exaggerated inference to humans from animal research. When press releases contained such exaggeration, 58%, 81%, and 86% of news stories, respectively, contained further exaggeration, compared with rates of 17%, 18%, and 10% in the news when the press releases were not exaggerated. Researchers should not be excused from being part of the press release process, as the author(s) should at least review the release before it gets disseminated to the media. I would even encourage researchers to engage in the process at the writing stage and to not allow their hospital’s or society’s public relations department to extrapolate their study’s results. Ultimately, the authors and the journals in which the studies are published will be held accountable for the information that trickles into the headlines, not the public relations departments, so we must make sure that the information is accurate and representative of the study’s actual findings.
END OF QUOTE
Sound advice indeed.
Now we only need to ALL follow it!!!
The current volume of the ‘Allgemeinen Homöopathischen Zeitung’ contains all the abstracts of the ‘Homeopathic World Congress 2017’ which will be hosted in Leipzig, 14-17 July this year by the ‘Deutschen Zentralvereins Homöopathischer Ärzte’ under the patronage of the German Health Secretary, Annette Widmann-Mauz. As not many readers of this blog are likely to be regular readers of this important journal, I have copied six of the more amusing abstracts below:
A male patient with bilateral solid renal mass was investigated and given an individualized homeopathic remedy. Antimonium crudum in 50000 potency was selected after proper case taking and evaluation. Investigations were done before and after treatment. Follow ups took place monthly. Results The patient had symptomatic relief from pain in flanks, acute retention and hematuria. The ultrasonography suggests a reduction in size of both lesions over a period of two years. A small number of lymph nodes of the para-aortic group are still visible. There is a normal level of urea and creatinine, no anemia or hypertention. The patient is surviving since 2014. Conclusion In the present day when malignancies are treated with surgeries, chemo and radiotherapies, homeopathy has a significant role to play as seen in the above case. This case with bilateral solid renal mass, probably a renal cell carcinoma, received an individualized homeopathic remedy-treatment compliant with the totality of symptoms, and permitted the patient to live longer without anemia, hypertension, anorexia or weight loss. The quality of life was maintained without the side effects of surgery, radiotherapy and chemotherapy. Acute retentions, which he used to suffer also remained absent, thereafter. The result of this case suggests to take up further studies on individualized homeopathic treatment in malignant diseases.
Urinary tract infections (UTI) are often a complaint in the homeopathic practice, mainly as uncomplicated infections in the form of a one time event. Some patients, however, have a tendency to develop recurrent or complicated urinary tract infections. Methods It is shown on the basis of case documentation that UTI should be treated homeopathic, variably. The issue of prophylaxis will be discussed. Results If there is a tendency to complicated UTI, chronic treatment after case taking of the symptom-totality of the affected must take place during a free interval. In contrast, the chronically recurring and flaming up of UTI, as well as the uniquely occurring of uncomplicated UTI, are handled as an acute illness. The treatment is based on the striking, characteristic symptoms of the infected. Conclusion The homeopathic treatment of UTI in the acute case of uncomplicated forms is usually very successful, The chronic treatment of complicated UTI shows certain difficulties. A safe homeopathic prophylaxis, in terms of conventional medicine, is problematical.
The homeopathic clinic of the Municipal Public Servant Hospital of São Paulo (HSPM – Brazil) has among patient records some cases of thyroid gland diseases (hypothyroidism or hyperthyroidism), which were treated whith the systemic homeopathic method of Carillo. This study evaluates patients with diseases of thyroid gland, analyzing improvements using a Iodium-like equalizer, adjacent to the systemic medication. The reviewed 21 cases using Iodium equalizer for the disease, adjacent to the systemic medication, in the homeopathic clinic of the HSPM, from 2000 to 2013. In four cases, it was possible to reduce the dose of allopathic medicine and finally terminate it due to normalization of the thyroid gland function. There was one case of hyperthyroidism and it was possible to terminate the use of methimazole. There were four cases, in which the function of the thyroid gland was normalized without the associated use of hormone. In three cases it was possible to reduce the dose of hormone. There were nine cases, in which it was not possible to reduce the dose of the hormone. In cases where there was an improvement applying homeopathic treatment, TSH and free T4 returned to the normal reference value. In cases that were not effective, TSH and free T4 had not normalized. Therefore, the effectiveness of Iodium depends on the ability and stability of the gland thyroid to increase or decrease hormone production, in addition to the treatment of a chronic disease, that affects the thyroid gland.
Cystitis composes infections in the urinary system, especially bladder and urethra. It has multiple causes, but the most common is infection due to microorganisms such as E. coli, streptococcus, staphylococcus etc. If the system is attacked by pathogenetic agents, the defense must include more powerful noxious agents which can fight and destroy the attacking organisms, here is the role of nosodes. Nosodes are the potentised remedies made up from dangerous noxious materials. The use of nosodes in cystitis is based on the aphorism 26– Therapeutic Law of Nature: A weaker one is always distinguished by the stronger one! Colibacillinum, streptococcinum, staphylococcinum, lyssinum, medorrhinum, psorinum and tuberculinum are useful in handling cystitis relating to the organism involved [as found in urine test] and symptom similarity. Method An observational prospective study on a group of 30 people proves the immediate, stronger defensive action of nosodes. Result Amazing! Nosodes given in low potency provided instant relief to patients. Repetition of the same, over several months offered immunity for further attacks of cystitis, as Hering had already testified nosodes have prophylactic action. Conclusion According to law of similia – as per the pathology, as per the defense! By inducing a strong artificial disease, homeopathy can eliminate the natural disease from the body. Usually nosodes are used as intercurrent drugs which play the role of catalysts, on the journey to recovery, but they are also very effective in cystitis as an acute remedy. Acute cystitis is a very troublesome state for the patients, to cure it homeopathy has an arsenal of nosodes.
In 1991, no antiretroviral therapy (ART) treatment was available. The Central Council for Research in Homeopathy had established a clinical research unit at Mumbai for undertaking investigations in HIV/AIDS. So far 2502 cases have been enrolled for homeopathic treatment and three studies have been published since then. In this paper we will highlight the impact of long term homeopathic management of cases, which have been followed up for more than 15 years. Method The HIV positive cases enrolled in different studies are continuously being managed in this unit and even after study conclusion. All the cases are being treated solely with individualised homeopathy. The cases are assessed clinically (body weight, opportunistic infections, etc.) as well as in respect to CD4 counts and CD4/CD8 ratio. Results The CD4 count was maintained in all patients, except in one case. Three patients had the CD4 level in the range of 500–1200, four in the range of 300–500, one had a 272 CD4 count. There has been a decline of CD4/ CD8 ratio since baseline, but the patients have maintained their body weights and remained free from major HIV related illnesses and opportunistic infections. The frequently indicated remedies were pulsatilla pratensis, lycopodium clavatum, nux vomica,tuberculinum bovinum, natrum muriaticum, rhus toxicodendron, medorrhinum, arsenicum album, mercurius solubilis, thuja occidentalis, nitic acid, sulphur, bryonia alba and hepar sulph. Conclusion In the emergent scenario of drug resistance and adverse reactions of ART in HIV infections, there may be a possibility of employing homeopathy as an adjuvant therapy to existing standard ART treatment. Further studies are desirable.
In the last 20 years we have treated in the Clinica St. Croce many patients with cancer. We often deal with palliative states and we aim at pain relief and improvement of life-quality, and if possible a prolongation of life. Is this possible by prescribing a homeopathic therapy? Methodology The exact application and the knowledge of the responses to the Q-potencies often give indications for the correct choice of remedy. Acute conditions of pain often need a more frequent repetition of the C-potencies needed for pain relief. Results Even with severe pain or in so-called final stages homeopathy can offer great assistance. On the basis of case reports from Clinica St. Croce, the procedure for the homeopathic treatment of cancer, and the treatment of pain and final states will be illustrated and clarified. In addition, some clinically proven homeopathic remedies will be presented for the optimal palliation in the treatment of end-states and accompanying the dying. Conclusions With the precise application and knowledge of the responses to the Q- and C-potencies, the homeopathic doctor is given a wonderful helper to treat even the most serious palliative states and can accomplish, sometimes, a miraculous healing.
MY BRIEF COMMENT
These abstracts are truly hilarious and show how totally unaware some homeopaths are of the scientific method. I say ‘some’, but perhaps it is most or even all? How can a scientific committee reviewing these abstracts let them pass and allow the material to be presented at the ‘World Congress’? How can a Health Secretary accept the patronage of such a farce?
These abstracts are therefore not just hilarious but also truly depressing. If we had needed proof that homeopathy has no place in real healthcare of today, these abstracts would go a long way in providing it. To realise that politicians, physicians, patients, consumers, journalists etc. take such infantile nonsense seriously is not just depressing but at the same time worrying, I find.
A recent post discussed a ‘STATE OF THE ART REVIEW’ from the BMJ. When I wrote it, I did not know that there was more to come. It seems that the BMJ is planning an entire series on the state of the art of BS! The new paper certainly looks like it:
Headaches, including primary headaches such as migraine and tension-type headache, are a common clinical problem. Complementary and integrative medicine (CIM), formerly known as complementary and alternative medicine (CAM), uses evidence informed modalities to assist in the health and healing of patients. CIM commonly includes the use of nutrition, movement practices, manual therapy, traditional Chinese medicine, and mind-body strategies. This review summarizes the literature on the use of CIM for primary headache and is based on five meta-analyses, seven systematic reviews, and 34 randomized controlled trials (RCTs). The overall quality of the evidence for CIM in headache management is generally low and occasionally moderate. Available evidence suggests that traditional Chinese medicine including acupuncture, massage, yoga, biofeedback, and meditation have a positive effect on migraine and tension headaches. Spinal manipulation, chiropractic care, some supplements and botanicals, diet alteration, and hydrotherapy may also be beneficial in migraine headache. CIM has not been studied or it is not effective for cluster headache. Further research is needed to determine the most effective role for CIM in patients with headache.
My BS-detector struggled with the following statements:
- integrative medicine (CIM), formerly known as complementary and alternative medicine (CAM) – the fact that CIM is a nonsensical new term has been already mentioned in the previous post;
- evidence informed modalities – another new term! evidence-BASED would be too much? because it would require using standards that do not apply to CIM? double standards promoted by the BMJ, what next?
- CIM commonly includes the use of nutrition – yes, so does any healthcare or indeed life!
- the overall quality of the evidence for CIM in headache management is generally low and occasionally moderate – in this case, no conclusions should be drawn from it (see below);
- evidence suggests that traditional Chinese medicine including acupuncture, massage, yoga, biofeedback, and meditation have a positive effect on migraine and tension headaches – no, it doesn’t (see above)!
- further research is needed to determine the most effective role for CIM in patients with headache – this sentence does not even make the slightest sense to me; have the reviewers of this article been asleep?
And this is just the abstract!
The full text provides enough BS to fertilise many acres of farmland!
Moreover, the article is badly researched, cherry-picked, poorly constructed, devoid of critical input, and poorly written. Is there anything good about it? You tell me – I did not find much!
My BS-detector finally broke when we came to the conclusions:
The use of CIM therapies has the potential to empower patients and help them take an active role in their care. Many CIM modalities, including mind-body therapies, are both self selected and self administered after an education period. This, coupled with patients’ increased desire to incorporate integrative medicine, should prompt healthcare providers to consider and discuss its inclusion in the overall management strategy. Low to moderate quality evidence exists for the effectiveness of some CIM therapies in the management of primary headache. The evidence for and use of CIM is continuously changing so healthcare professionals should direct their patients to reliable and updated resources, such as NCCIH.
WHAT IS HAPPENING TO THE BMJ?
IT USED TO BE A GOOD JOURNAL!
The website of BMJ Clinical Evidence seems to be popular with fans of alternative medicine (FAMs). That sounds like good news: it’s an excellent source, and one can learn a lot about EBM when studying it. But there is a problem: FAMs don’t seem to really study it (alternatively they do not have the power of comprehension to understand the data); they merely pounce on this figure and cite it endlessly:
They interpret it to mean that only 11% of what conventional clinicians do is based on sound evidence. This is water on their mills, because now they feel able to claim:
THE MAJORITY OF WHAT CONVENTIONAL CLINICIANS DO IS NOT EVIDENCE-BASED. SO, WHY DO SO-CALLED RATIONAL THINKERS EXPECT ALTERNATIVE THERAPIES TO BE EVIDENCE-BASED? IF WE NEEDED PROOF THAT THEY ARE HYPOCRITES, HERE IT IS!!!
The question is: are these FAMs correct?
The answer is: no!
They are merely using a logical fallacy (tu quoque); what is worse, they use it based on misunderstanding the actual data summarised in the above figure.
Let’s look at this in a little more detail.
The first thing we need to understand the methodologies used by ‘Clinical Evidence’ and what the different categories in the graph mean. Here is the explanation:
So, arguably the top three categories amounting to 42% signify some evidential support (if we decided to be more rigorous and merely included the two top categories, we would still arrive at 35%). This is not great, but we must remember two things here:
- EBM is fairly new;
- lots of people are working hard to improve the evidence base of medicine so that, in future, these figures will be better (by contrast, in alternative medicine, no similar progress is noticeable).
The second thing that strikes me is that, in alternative medicine, these figures would surely be much, much worse. I am not aware of reliable estimates, but I guess that the percentages might be one dimension smaller.
The third thing to mention is that the figures do not cover the entire spectrum of treatments available today but are based on ~ 3000 selected therapies. It is unclear how they were chosen, presumably the choice is pragmatic and based on the information available. If an up-to date systematic review has been published and provided the necessary information, the therapy was included. This means that the figures include not just mainstream but also plenty of alternative treatments (to the best of my knowledge ‘Clinical Evidence’ makes no distinction between the two). It is thus nonsensical to claim that the data highlight the weakness of the evidence in conventional medicine. It is even possible that the figures would be better, if alternative treatments had been excluded (I estimate that around 2 000 systematic reviews of alternative therapies have been published [I am the author of ~400 of them!]).
The fourth and possibly the most important thing to mention is that the percentage figures in the graph are certainly NOT a reflection of what percentage of treatments used in routine care are based on good evidence. In conventional practice, clinicians would, of course, select where possible those treatments with the best evidence base, while leaving the less well documented ones aside. In other words, they will use the ones in the two top categories much more frequently than those from the other categories.
At this stage, I hear some FAMs say: how does he know that?
Because several studies have been published that investigated this issue in some detail. They have monitored what percentage of interventions used by conventional clinicians in their daily practice are based on good evidence. In 2004, I reviewed these studies; here is the crucial passage from my paper:
“The most conclusive answer comes from a UK survey by Gill et al who retrospectively reviewed 122 consecutive general practice consultations. They found that 81% of the prescribed treatments were based on evidence and 30% were based on randomised controlled trials (RCTs). A similar study conducted in a UK university hospital outpatient department of general medicine arrived at comparable figures; 82% of the interventions were based on evidence, 53% on RCTs. Other relevant data originate from abroad. In Sweden, 84% of internal medicine interventions were based on evidence and 50% on RCTs. In Spain these percentages were 55 and 38%, respectively. Imrie and Ramey pooled a total of 15 studies across all medical disciplines, and found that, on average, 76% of medical treatments are supported by some form of compelling evidence — the lowest was that mentioned above (55%),6 and the highest (97%) was achieved in anaesthesia in Britain. Collectively these data suggest that, in terms of evidence-base, general practice is much better than its reputation.”
My conclusions from all this:
FAMs should study the BMJ Clinical Evidence more thoroughly. If they did, they might comprehend that the claims they tend to make about the data shown there are, in fact, bogus. In addition, they might even learn a thing or two about EBM which might eventually improve the quality of the debate.
The new guidelines by the American College of Physicians entitled ‘Noninvasive Treatments for Acute, Subacute, and Chronic Low Back Pain: A Clinical Practice Guideline From the American College of Physicians’ have already been the subject of the previous post. Today, I want to have a closer look at a small section of these guidelines which, I think, is crucial. It is entitled ‘HARMS OF NONPHARMACOLOGIC THERAPIES’. I have taken the liberty of copying it below:
“Evidence on adverse events from the included RCTs and systematic reviews was limited, and the quality of evidence for all available harms data is low. Harms were poorly reported (if they were reported at all) for most of the interventions.
Low-quality evidence showed no reported harms or serious adverse events associated with tai chi, psychological interventions, multidisciplinary rehabilitation, ultrasound, acupuncture, lumbar support, or traction (9,95,150,170–174). Low-quality evidence showed that when harms were reported for exercise, they were often related to muscle soreness and increased pain, and no serious harms were reported. All reported harms associated with yoga were mild to moderate (119). Low-quality evidence showed that none of the RCTs reported any serious adverse events with massage, although 2 RCTs reported soreness during or after massage therapy (175,176). Adverse events associated with spinal manipulation included muscle soreness or transient increases in pain (134). There were few adverse events reported and no clear differences between MCE and controls. Transcutaneous electrical nerve stimulation was associated with an increased risk for skin site reaction but not serious adverse events (177). Two RCTs (178,179) showed an increased risk for skin flushing with heat compared with no heat or placebo, and no serious adverse events were reported. There were no data on cold therapy. Evidence was insufficient to determine harms of electrical muscle stimulation, LLLT, percutaneous electrical nerve stimulation, interferential therapy, short-wave diathermy, and taping.”
The first thing that strikes me is the brevity of the section. Surely, guidelines of this nature must include a full discussion of the risks of the treatments in question!
The second thing that is noteworthy is the fact that the authors confirm the fact I have been banging on about for years: clinical trials of alternative therapies far too often fail to mention adverse effects. I have often pointed out that the failure to report adverse effects in clinical trials is an unacceptable violation of medical ethics. By contrast, the guideline authors seem not to feel strongly about this omission.
The third thing that is noteworthy is that the guidelines evaluate the harms of the treatments purely on the basis of the adverse effects reported in the clinical trials and systematic reviews included in their efficacy assessments. This is nonsensical for at least two reasons:
- The guideline authors themselves are aware that the trials very often fail to mention adverse effects.
- For any assessment of harm, one has to go far beyond the evidence of clinical trials, because trials tend to be too small to pick up rare adverse effects, and because they are always conducted under optimally controlled conditions where adverse effects are less likely to occur than in real life.
Together, these features of the assessment of harms explain why the guideline authors arrive at conclusions which are oddly misguided; I would even feel that they resemble a white-wash. Here are two of the most overt misjudgements:
- no harms associated with acupuncture,
- only trivial harm associated with spinal manipulations.
The best evidence we have today shows that acupuncture leads to mild adverse effects in about 10% of all cases and is also associated with very severe complications (e.g. pneumothorax, cardiac tamponade, infections, deaths) in an unknown number of patients. More details can be found for instance here, here, here and here.
And the best evidence available shows that spinal manipulation leads to moderately severe adverse effects in ~50% of all cases. In addition, we know of hundreds of cases of very severe complications resulting in stroke, permanent neurological deficits or deaths. More details can be found for instance here, here, here and here.
In the introduction, I stated that this small section of the guidelines is crucial.
The reason is simple: any responsible therapeutic decision has to be based not just on the efficacy of the treatment in question but on its risk/benefit balance. The evidence shows that the risks of some alternative therapies can be considerable, a fact that is almost totally neglected in the guidelines. Therefore, the recommendations of the new guidelines by the American College of Physicians entitled ‘Noninvasive Treatments for Acute, Subacute, and Chronic Low Back Pain: A Clinical Practice Guideline From the American College of Physicians’ are in several aspects not entirely correct and need to be reconsidered.
The BMJ has always been my favourite Medical journal. (Need any proof for this statement? A quick Medline search tells me that I have over 60 publications in the BMJ.) But occasionally, the BMJ also disappoints me a great deal.
One of the most significant disappointments was recently published under the heading of STATE OF THE ART REVIEW. A review that is ‘state of the art’ must fulfil certain criteria; foremost it should be informative, unbiased and correct. The paper I am discussing here has, I think, neither of these qualities. It is entitled ‘Management of chronic pain using complementary and integrative medicine’, and here is its abstract:
Complementary and integrative medicine (CIM) encompasses both Western-style medicine and complementary health approaches as a new combined approach to treat a variety of clinical conditions. Chronic pain is the leading indication for use of CIM, and about 33% of adults and 12% of children in the US have used it in this context. Although advances have been made in treatments for chronic pain, it remains inadequately controlled for many people. Adverse effects and complications of analgesic drugs, such as addiction, kidney failure, and gastrointestinal bleeding, also limit their use. CIM offers a multimodality treatment approach that can tackle the multidimensional nature of pain with fewer or no serious adverse effects. This review focuses on the use of CIM in three conditions with a high incidence of chronic pain: back pain, neck pain, and rheumatoid arthritis. It summarizes research on the mechanisms of action and clinical studies on the efficacy of commonly used CIM modalities such as acupuncture, mind-body system, dietary interventions and fasting, and herbal medicine and nutrients.
The full text of this article is such that I could take issue with almost every second statement in it. Obviously, this would be too long and too boring for this blog. So, to keep it crisp and entertaining, let me copy the (tongue in cheek) ‘letter to the editor’ some of us published in the BMJ as a response to the review:
“Alternative facts are fashionable in politics these days, so why not also in healthcare? The article by Chen and Michalsen on thebmj.com provides a handy set of five instructions for smuggling alternative facts into medicine.
1. Create your own terminology: the term ‘complementary and integrated medicine’ (CIM) is nonsensical. Integrated medicine (a hotly disputed field) already covers complementary and conventional medicine.
2. Pretend to be objective: Chen and Michalsen elaborate on the systematic searches they conducted. But they omit hundreds of sources which do not support their message, which cherry-picks only evidence for the efficacy of the treatments they promote.
3. Avoid negativity: they bypass any material that might challenge what they include. For instance, when discussing therapeutic risks, they omit the disturbing lack of post-marketing surveillance: the reason we lack information on adverse events. They even omit to mention the many fatalities caused by their ‘CIM’.
4. Create an impression of thoroughness: Chen and Michalsen cite a total of 225 references. This apparent scholarly attention to detail masks their misuse of many of they list. Reference 82, for example, is employed to back up the claim that “satisfaction was lowest among complementary medicine users with rheumatoid arthritis, vasculitis, or connective tissue diseases”. In fact, it shows nothing of the sort.
5. Back up your message with broad generalisations: Chen and Michalsen conclude that “Taken together, CIM has an increasing role in the management of chronic pain, but high quality research is needed”. The implication is that all the CIMs mentioned in their figure 1 are candidates for pain control – even discredited treatments such as homeopathy.
In our view, these authors render us a service: they demonstrate to the novice how alternative facts may be used in medicine.”
James May, Edzard Ernst, Nick Ross, on behalf of HealthWatch UK
END OF QUOTE
I am sure you have your own comments and opinions, and I encourage you to post them here or (better) submit them to the BMJ or (best) both.
Shiatsu is one of those alternative therapies where there is almost no research. Therefore, every new study is of interest, and I was delighted to find this new trial.
Italian researchers tested the efficacy and safety of combining shiatsu and amitriptyline to treat refractory primary headaches in a single-blind, randomized, pilot study. Subjects with a diagnosis of primary headache and who experienced lack of response to ≥2 different prophylactic drugs were randomized in a 1:1:1 ratio to receive one of the following treatments:
- shiatsu plus amitriptyline,
- shiatsu alone,
- amitriptyline alone
The treatment period lasted 3 months and the primary endpoint was the proportion of patients experiencing ≥50%-reduction in headache days. Secondary endpoints were days with headache per month, visual analogue scale, and number of pain killers taken per month.
After randomization, 37 subjects were allocated to shiatsu plus amitriptyline (n = 11), shiatsu alone (n = 13), and amitriptyline alone (n = 13). Randomization ensured well-balanced demographic and clinical characteristics at baseline.
The results show that all the three groups improved in terms of headache frequency, visual analogue scale score, and number of pain killers and there was no between-group difference in the primary endpoint. Shiatsu (alone or in combination) was superior to amitriptyline in reducing the number of pain killers taken per month. Seven (19%) subjects reported adverse events, all attributable to amitriptyline, while no side effects were related with shiatsu treatment.
The authors concluded that shiatsu is a safe and potentially useful alternative approach for refractory headache. However, there is no evidence of an additive or synergistic effect of combining shiatsu and amitriptyline. These findings are only preliminary and should be interpreted cautiously due to the small sample size of the population included in our study.
Yes, I would advocate great caution indeed!
The results could easily be said to demonstrate that shiatsu is NOT effective. There is NO difference between the groups when looking at the primary endpoint. This plus the lack of a placebo-group renders the findings uninterpretable:
- If we take the comparison 2 versus 3, this might indicate efficacy of shiatsu.
- If we take the comparison 1 versus 3, it would indicate the opposite.
- If we finally take the comparison 1 versus 2, it would suggest that the drug was ineffective.
So, we can take our pick!
Moreover, I do object to the authors’ conclusion that “shiatsu is a safe”. For such a statement, we would need sample sizes that are about two dimensions greater that those of this study.
So, what might be an acceptable conclusion from this trial? I see only one that is in accordance with the design and the results of this study:
POORLY DESIGNED RESEARCH CANNOT LEAD TO ANY CONCLUSIONS ABOUT THERAPEUTIC EFFICACY OR SAFETY. IT IS A WASTE OF RESOURCES AND A VIOLATION OF RESEARCH ETHICAL.
On this blog, we have had (mostly unproductive) discussions with homeopath so often that sometimes they sound like a broken disk. I don’t want to add to this kerfuffle; what I hope to do today is to summarise a certain line of argument which, from the homeopaths’ point of view, seems entirely logical. I do this in the form of a fictitious conversation between a scientist (S) and a classical homeopath (H). My aim is to make the reader understand homeopaths better so that, future debates might be better informed.
HERE WE GO:
S: I have studied the evidence from studies of homeopathy in some detail, and I have to tell you, it fails to show that homeopathy works.
H: This is not true! We have plenty of evidence to prove that patients get better after seeing a homeopath.
S: Yes, but this is not because of the remedy; it is due to non-specific effect like the empathetic consultation with a homeopath. If one controls for these factors in adequately designed trials, the result usually is negative.
I will re-phrase my claim: the evidence fails to show that highly diluted homeopathic remedies are more effective than placebos.
H: I disagree, there are positive studies as well.
S: Let’s not cherry pick. We must always consider the totality of the reliable evidence. We now have a meta-analysis published by homeopaths that demonstrates the ineffectiveness of homeopathy quite clearly.
H: This is because homeopathy was not used correctly in the primary trials. Homeopathy must be individualised for each unique patient; no two cases are alike! Remember: homeopathy is based on the principle that like cures like!!!
S: Are you saying that all other forms of using homeopathy are wrong?
H: They are certainly not adhering to what Hahnemann told us to do; therefore you cannot take their ineffectiveness as proof that homeopathy does not work.
S: This means that much, if not most of homeopathy as it is used today is to be condemned as fake.
H: I would not go that far, but it is definitely not the real thing; it does not obey the law of similars.
S: Let’s leave this to one side for the moment. If you insist on individualised homeopathy, I must tell you that this approach can also be tested in clinical trials.
H: I know; and there is a meta-analysis which proves that it is effective.
S: Not quite; it concluded that medicines prescribed in individualised homeopathy may have small, specific treatment effects. Findings are consistent with sub-group data available in a previous ‘global’ systematic review. The low or unclear overall quality of the evidence prompts caution in interpreting the findings. New high-quality RCT research is necessary to enable more decisive interpretation.
If you call this a proof of efficacy, I would have to disagree with you. The effect was tiny and at least two of the best studies relevant to the subject were left out. If anything, this paper is yet another proof that homeopathy is useless!
H: You simply don’t understand homeopathy enough to say that. I tried to tell you that the remedy must be carefully chosen to fit each unique patient. This is a very difficult task, and sometimes it is not successful – mainly because the homeopaths employed in clinical trials are not skilled enough to find it. This means that, in these studies, we will always have a certain failure rate which, in turn, is responsible for the small average effect size.
S: But these studies are always conducted by experienced homeopaths, and only the very best, most experienced homeopaths were chosen to cooperate in them. Your argument that the trials are negative because of the ineffectiveness of the homeopaths – rather than the ineffectiveness of homeopathy – is therefore nonsense.
H: This is what you say because you don’t understand homeopathy!
S: No, it is what you say because you don’t understand science. How else would you prove that your hypothesis is correct?
H: Simple! Just look at individual cases from the primary studies within this meta-analysis . You will see that there are always patients who did improve. These cases are the proof we need. The method of the RCT is only good for defining average effects; this is not what we should be looking at, and it is certainly not what homeopaths are interested in.
S: Are you saying that the method of the RCT is wrong?
H: It is not always wrong. Some RCTs of homeopathy are positive and do very clearly prove that homeopathy works. These are obviously the studies where homeopathy has been applied correctly. We have to make a meta-analysis of such trials, and you will see that the result turns out to be positive.
S: So, you claim that all the positive studies have used the correct method, while all the negative ones have used homeopathy incorrectly.
H: If you insist to put it like that, yes.
S: I see, you define a trial to have used homeopathy correctly by its result. Essentially you accept science only if it generates the outcome you like.
H: Yes, that sounds odd to you – because you don’t understand enough of homeopathy.
S: No, what you seem to insist on is nothing short of double standards. Or would you accept a drug company claiming: some patients did feel better after taking our new drug, and this is proof that it works?
H: You see, not understanding homeopathy leads to serious errors.
S: I give up.
The aim of this pragmatic study was “to investigate the effectiveness of acupuncture in addition to routine care in patients with allergic asthma compared to treatment with routine care alone.”
Patients with allergic asthma were included in a controlled trial and randomized to receive up to 15 acupuncture sessions over 3 months plus routine care, or to a control group receiving routine care alone. Patients who did not consent to randomization received acupuncture treatment for the first 3 months and were followed as a cohort. All trial patients were allowed to receive routine care in addition to study treatment. The primary endpoint was the asthma quality of life questionnaire (AQLQ, range: 1–7) at 3 months. Secondary endpoints included general health related to quality of life (Short-Form-36, SF-36, range 0–100). Outcome parameters were assessed at baseline and at 3 and 6 months.
A total of 1,445 patients were randomized and included in the analysis (184 patients randomized to acupuncture plus routine care and 173 to routine care alone, and 1,088 in the nonrandomized acupuncture plus routine care group). In the randomized part, acupuncture was associated with an improvement in the AQLQ score compared to the control group (difference acupuncture vs. control group 0.7 [95% confidence interval (CI) 0.5–1.0]) as well as in the physical component scale and the mental component scale of the SF-36 (physical: 2.5 [1.0–4.0]; mental 4.0 [2.1–6.0]) after 3 months. Treatment success was maintained throughout 6 months. Patients not consenting to randomization showed similar improvements as the randomized acupuncture group.
The authors concluded that in patients with allergic asthma, additional acupuncture treatment to routine care was associated with increased disease-specific and health-related quality of life compared to treatment with routine care alone.
We have been over this so many times (see for instance here, here and here) that I am almost a little embarrassed to explain it again: it is fairly easy to design an RCT such that it can only produce a positive result. The currently most popular way to achieve this aim in alternative medicine research is to do a ‘A+B versus B’ study, where A = the experimental treatment, and B = routine care. As A always amounts to more than nothing – in the above trial acupuncture would have placebo effects and the extra attention would also amount to something – A+B must always be more than B alone. The easiest way of thinking of this is to imagine that A and B are both finite amounts of money; everyone can understand that A+B must always be more than B!
Why then do acupuncture researchers not get the point? Are they that stupid? I happen to know some of the authors of the above paper personally, and I can assure you, they are not stupid!
I am afraid there is only one reason I can think of: they know perfectly well that such an RCT can only produce a positive finding, and precisely that is their reason for conducting such a study. In other words, they are not using science to test a hypothesis, they deliberately abuse it to promote their pet therapy or hypothesis.
As I stated above, it is fairly easy to design an RCT such that it can only produce a positive result. Yet, it is arguably also unethical, perhaps even fraudulent, to do this. In my view, such RCTs amount to pseudoscience and scientific misconduct.