MD, PhD, FMedSci, FSB, FRCP, FRCPEd

systematic review

1 2 3 17

An article by Rabbi Yair Hoffman for the Five Towns Jewish Times caught my eye. Here are a few excerpts:

“I am sorry, Mrs. Ploni, but the muscle testing we performed on you indicates that your compatibility with your spouse is a 1 out of a possible 10 on the scale.”

“Your son being around his father is bad for his energy levels. You should seek to minimize it.”

“Your husband was born normal, but something happened to his energy levels on account of the vaccinations he received as a child. It is not really his fault, but he is not good for you.”

Welcome to the world of Applied Kinesiology (AK) or health Kinesiology… Incredibly, there are people who now base most of their life decisions on something called “muscle testing.” Practitioners believe or state that the body’s energy levels can reveal remarkable information, from when a bride should get married to whether the next Kinesiology appointment should be in one week or two weeks. Prices for a 45 minute appointment can range from $125 to $250 a session. One doctor who is familiar with people who engage in such pursuits remarked, “You have no idea how many inroads this craziness has made in our community.”

… AK (applied kinesiology) is system that evaluates structural, chemical, and mental aspects of health by using “manual muscle testing (MMT)” along with other conventional diagnostic methods. The belief of AK adherents is that every organ dysfunction is accompanied by a weakness in a specific corresponding muscle… Treatments include joint manipulation and mobilization, myofascial, cranial and meridian therapies, clinical nutrition, and dietary counselling. A manual muscle test is conducted by having the patient resist using the target muscle or muscle group while the practitioner applies force the other way. A smooth response is called a “strong muscle” and a response that was not appropriate is called a “weak response.” Like some Ouiji board out of the 1970’s, Applied Kinesiology is used to ask “Yes or No” questions about issues ranging from what type of Parnassa courses one should be taking, to what Torah music tapes one should listen to, to whether a therapist is worthwhile to see or not.

“They take everything with such seriousness – they look at it as if it is Torah from Sinai,” remarked one person familiar with such patients. One spouse of an AK patient was shocked to hear that a diagnosis was made concerning himself through the muscle testing of his wife – without the practitioner having ever met him… And the lines at the office of the AK practitioner are long. One husband holds a crying baby for three hours, while his wife attends a 45 minute session. Why so long? The AK practitioner let other patients ahead – because of emergency needs…

END OF EXCERPTS

The article  is a reminder how much nonsense happens in the name of alternative medicine. AK is one of the modalities that is exemplary:

  • it is utterly implausible;
  • there is no good evidence that it works.

The only systematic review of AK was published in 2008 by a team known to be strongly in favour of alternative medicine. It included 22 relevant studies. Their methodology was poor. The authors concluded that there is insufficient evidence for diagnostic accuracy within kinesiology, the validity of muscle response and the effectiveness of kinesiology for any condition. 

Some AK fans might now say: absence of evidence is not evidence of absence!!! There is no evidence that AK does not work, and therefore we should give it the benefit of the doubt and use it.

This, of course, is absolute BS! Firstly, the onus is on those who claim that AK works to prove their assumption. Secondly, in responsible healthcare, we are obliged to employ those modalities for which the evidence is positive, while avoiding those for which the evidence fails to be positive.

 

It used to be called ‘good bedside manners’. The term is an umbrella for a range of attitudes and behaviours including compassion, empathy and conveying positive messages. What could be more obvious than the assumption that good bedside manners are better than bad ones?

But as sceptics, we need to doubt obvious assumptions! Where is the evidence? we need to ask. So, where is the evidence that positive messages have any clinical effects? A meta-analysis has tackled the issue, and the results are noteworthy.

The researchers aimed to estimate the efficacy of positive messages for pain reduction. They included RCTs of the effects of positive messages. Their primary outcome measures were differences in patient- or observer reported pain between groups who were given positive messages and those who were not. Of the 16 RCTs (1703 patients) that met the inclusion criteria, 12 trials had sufficient data for meta-analysis. The pooled standardized effect size was −0.31 (95% CI −0.61 to −0.01, P = 0.04, I² = 82%). The effect size remained positive but not statistically significant after we excluded studies considered to have a high risk of bias (standard effect size −0.17, 95% CI −0.54 to 0.19, P = 0.36, I² = 84%). The authors concluded that care of patients with chronic or acute pain may be enhanced when clinicians deliver positive messages about possible clinical outcomes. However, we have identified several limitations of the present study that suggest caution when interpreting the results. We recommend further high quality studies to confirm (or falsify) our result.

The 1st author of this paper published a comment in which he stated that our recent mega-study with 12 randomized trials confirmed that doctors who use positive language reduce patient pain by a similar amount to drugs. Other trials show that positive messages can:

• help Parkinson’s patients move their hands faster,
• increase ‘peak flow’ (a measure of how much air is breathed) in asthma patients,
• improve the diameter of arteries in heart surgery patients, and
• reduce the amount of pain medication patients use.

The way a positive message seems to help is biological. When a patient anticipates a good thing happening (for example that their pain will go away), this activates parts of the brain that help the body make its own drugs like endorphins. A positive doctor may also help a patient relax which can also improve health.

I am not sure that this is entirely correct. When the authors excluded the methodologically weak and therefore unreliable studies, the effect was no longer significant. That is to say, it was likely due to chance.

And what about the other papers cited above? I am not sure about them either. Firstly, they do not necessarily show that positive messages are effective. Secondly, there is just one study for each claim, and one swallow does not make a summer; we would need independent replications.

So, am I saying that being positive as a clinician is ineffective? No! I am saying that the evidence is too flimsy to be sure. And possibly, this means that the effect of positive messages is smaller than we all thought.

This recently published report provides updated clinical practice guidelines from the Society for Integrative Oncology on the use of integrative therapies for specific clinical indications during and after breast cancer treatment, including anxiety/stress, depression/mood disorders, fatigue, quality of life/physical functioning, chemotherapy-induced nausea and vomiting, lymphedema, chemotherapy-induced peripheral neuropathy, pain, and sleep disturbance.

The practice guidelines are based on a systematic literature review from 1990 through 2015. The recommendations are as follows:

  • Music therapy, meditation, stress management, and yoga are recommended for anxiety/stress reduction.
  • Meditation, relaxation, yoga, massage, and music therapy are recommended for depression/mood disorders.
  • Meditation and yoga are recommended to improve quality of life.
  • Acupressure and acupuncture are recommended for reducing chemotherapy-induced nausea and vomiting.
  • Acetyl-L-carnitine is not recommended to prevent chemotherapy-induced peripheral neuropathy due to a possibility of harm.
  • No strong evidence supports the use of ingested dietary supplements to manage breast cancer treatment-related side effects.

The authors conclude that there is a growing body of evidence supporting the use of integrative therapies, especially mind-body therapies, as effective supportive care strategies during breast cancer treatment. Many integrative practices, however, remain understudied, with insufficient evidence to be definitively recommended or avoided.

I have to admit that I am puzzled by this paper.

The first obvious point to make is that these treatments are not ‘integrative therapies’; they are alternative or complementary and I fail to see what is integrative about them.

The second point is that the positive recommendations are based on often poor-quality studies which did not control for placebo effects.

The third point is that the negative recommendations are woefully incomplete. There are many more alternative therapies for which there is no strong evidence.

The forth point is the conclusion implying that treatment supported by insufficient evidence should be avoided. I would not claim that any of the mentioned treatments is backed by SUFFICIENT evidence. Therefore, we should avoid them all, one might argue.

But these concerns are perhaps relatively trivial or far-fetched. More important is the fact that a very similar article been published in 2014. Here is the abstract:

Background

The majority of breast cancer patients use complementary and/or integrative therapies during and beyond cancer treatment to manage symptoms, prevent toxicities, and improve quality of life. Practice guidelines are needed to inform clinicians and patients about safe and effective therapies.

Methods

Following the Institute of Medicine’s guideline development process, a systematic review identified randomized controlled trials testing the use of integrative therapies for supportive care in patients receiving breast cancer treatment. Trials were included if the majority of participants had breast cancer and/or breast cancer patient results were reported separately, and outcomes were clinically relevant. Recommendations were organized by outcome and graded based upon a modified version of the US Preventive Services Task Force grading system.

Results

The search (January 1, 1990–December 31, 2013) identified 4900 articles, of which 203 were eligible for analysis. Meditation, yoga, and relaxation with imagery are recommended for routine use for common conditions, including anxiety and mood disorders (Grade A). Stress management, yoga, massage, music therapy, energy conservation, and meditation are recommended for stress reduction, anxiety, depression, fatigue, and quality of life (Grade B). Many interventions (n = 32) had weaker evidence of benefit (Grade C). Some interventions (n = 7) were deemed unlikely to provide any benefit (Grade D). Notably, only one intervention, acetyl-l-carnitine for the prevention of taxane-induced neuropathy, was identified as likely harmful (Grade H) as it was found to increase neuropathy. The majority of intervention/modality combinations (n = 138) did not have sufficient evidence to form specific recommendations (Grade I).

Conclusions

Specific integrative therapies can be recommended as evidence-based supportive care options during breast cancer treatment. Most integrative therapies require further investigation via well-designed controlled trials with meaningful outcomes.

I have harshly criticised this review on this blog in 2016. For instance, I voiced concern about the authors declaration of conflicts of interest and stated:

 

I know none of the authors (Heather Greenlee, Lynda G. Balneaves, Linda E. Carlson, Misha Cohen, Gary Deng, Dawn Hershman, Matthew Mumber, Jane Perlmutter, Dugald Seely, Ananda Sen, Suzanna M. Zick, Debu Tripathy) of the document personally. They made the following collective statement about their conflicts of interest: “There are no financial conflicts of interest to disclose. We note that some authors have conducted/authored some of the studies included in the review.” I am a little puzzled to hear that they have no financial conflicts of interest (do not most of them earn their living by practising integrative medicine? Yes they do! The article informs us that: “A multidisciplinary panel of experts in oncology and integrative medicine was assembled to prepare these clinical practice guidelines. Panel members have expertise in medical oncology, radiation oncology, nursing, psychology, naturopathic medicine, traditional Chinese medicine, acupuncture, epidemiology, biostatistics, and patient advocacy.”). I also suspect they have other, potentially much stronger conflicts of interest. They belong to a group of people who seem to religiously believe in the largely nonsensical concept of integrative medicine

The just-published update has a different statement about conflicts of interest:

DISCLOSURES: Linda E. Carlson reports book royalties from New Harbinger and the American Psychological Association. Misha R. Cohen reports royalties from Health Concerns Inc., outside the submitted work. Matthew Mumber owns stock in I Thrive. All remaining authors report no conflicts of interest.

Is this much better than the previous statement? Was the previous statement therefore false?

I wonder.

What do you think?

I have repeatedly cautioned about the often poor quality of research into alternative medicine. This seems particularly necessary with studies of acupuncture, and especially true for such research carried out in China. I have also frequently noted that certain ‘CAM journals’ are notoriously prone to publishing rubbish. So, what can we expect from a paper that:

  • is on alternative medicine,
  • focusses on acupuncture,
  • is authored by Chinese researchers,
  • was published in the Journal of Alternative and Complementary Medicine (JACM)?

The answer is PROBABLY NOT A LOT!

As if for confirming my prediction, The JACM just published this systematic review. It reports pairwise and network meta-analyses to determine the effectiveness of acupuncture and acupuncture-related techniques for the treatment of psoriasis. A total of 13 RCTs were included. The methodological quality of these studies was ‘not rigorous’ according to the authors – in fact, it was lousy. Acupoint stimulation seemed to be more effective than non-acupoint stimulation. The short-term treatment effect was superior to the long-term effect (as one would expect with placebo). Network meta-analysis suggested that acupressure or acupoint catgut embedding generate superior effects compared to medications. It was noted that acupressure was the most effective treatment of all the acupuncture-like therapies.

The authors concluded that acupuncture-related techniques could be considered as an alternative or adjuvant therapy for psoriasis in short term, especially of acupressure and acupoint catgut embedding. This study recommends further well-designed, methodologically rigorous, and more head-to-head randomized trials to explore the effects of acupuncture-related techniques for treating psoriasis.

And what is wrong with that?

EVERYTHING!

  • The review is of very poor quality.
  • The primary studies are even worse.
  • The English language is defective to the point of being not understandable.
  • The conclusions are misleading.

Correct conclusions should read something like this: Due to the paucity and the poor quality of the clinical trials, this review could not determine whether acupuncture and similar therapies are effective for psoriasis.

And then there is, of course, the question about plausibility. How plausible is the assumption that acupuncture might affect a genetic autoimmune disease like psoriasis. The answer, I think, is that the assumption is highly unlikely.

In the above review, most of the 13 primary RCTs were from China. One of the few studies not conducted in China is this one:

56 patients suffering from long-standing plaque psoriasis were randomized to receive either active treatment (electrostimulation by needles placed intramuscularly, plus ear-acupuncture) or placebo (sham, ‘minimal acupuncture‘) twice weekly for 10 weeks. The severity of the skin lesions was scored (PASI) before, during, and 3 months after therapy. After 10 weeks of treatment the PASI mean value had decreased from 9.6 to 8.3 in the ‘active’ group and from 9.2 to 6.9 in the placebo group (p < 0.05 for both groups). These effects are less than the usual placebo effect of about 30%. There were no statistically significant differences between the outcomes in the two groups during or 3 months after therapy. The patient’s own opinion about the results showed no preference for ‘active’ therapy. It was also clear from the answers that the blinded nature of the study had not been discovered by the patients. In conclusion, classical acupuncture is not superior to sham (placebo) ‘minimal acupuncture‘ in the treatment of psoriasis.

Somehow, I trust these conclusions more than the ones from the review!

And somehow, I get very tired of journal editors failing to do their job of rejecting papers that evidently are embarrassing, unethical rubbish.

Systematic reviews are aimed at summarising and critically evaluating the evidence on a specific research question. They are the highest level of evidence and are more reliable than anything else we have. Therefore, they represent a most useful tool for both clinicians and researchers.

But there are, of course, exceptions. Take, for instance, this recent systematic review by researchers from the

  • Texas Chiropractic College, Pasadena, the Palmer Center for Chiropractic Research, Palmer College of Chiropractic, Davenport,
  • Department of Planning, Policy and Design, University of California, Irvine,
  • VA Puget Sound Health Care System, Tacoma,
  • New York Chiropractic College, Seneca Falls,
  • Logan University College of Chiropractic, Chesterfield,
  • University of Western States, Portland.

Its purpose was to evaluate the effectiveness of conservative non-drug, non-surgical interventions, either alone or in combination, for conditions of the shoulder. The review was conducted from March 2016 to November 2016 in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), and was registered with PROSPERO. Eligibility criteria included randomized controlled trials (RCTs), systematic reviews, or meta-analyses studying adult patients with a shoulder diagnosis. Interventions qualified if they did not involve prescription medication or surgical procedures, although these could be used in the comparison group or groups. At least 2 independent reviewers assessed the quality of each study using the Scottish Intercollegiate Guidelines Network checklists. Shoulder conditions addressed were

  • shoulder impingement syndrome (SIS),
  • rotator cuff-associated disorders (RCs),
  • adhesive capsulitis (AC),
  • nonspecific shoulder pain.

Twenty-five systematic reviews and 44 RCTs met inclusion criteria. Low- to moderate-quality evidence supported the use of manual therapies for all 4 shoulder conditions. Exercise, particularly combined with physical therapy protocols, was beneficial for SIS and AC. For SIS, moderate evidence supported several passive modalities. For RC, physical therapy protocols were found beneficial but not superior to surgery in the long term. Moderate evidence supported extracorporeal shockwave therapy for calcific tendinitis RC. Low-level laser was the only modality for which there was moderate evidence supporting its use for all 4 conditions.

The authors concluded that the findings of this literature review may help inform practitioners who use conservative methods (eg, doctors of chiropractic, physical therapists, and other manual therapists) regarding the levels of evidence for modalities used for common shoulder conditions.

This review has so many defects that it would be boring to list them here.

The PRISMA guidelines  – I happen to be a co-author – state, for instance, that the abstract (the above text is from the abstract) should provide a structured summary including, as applicable: background; objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions and implications of key findings; systematic review registration number. It is obvious that the review authors have omitted several of these items.

And that is just the abstract!  There is much, much more to criticise in this paper.

The most striking deficit, in my view, is the useless conclusion: the one from the abstract (the part of the paper that will be read most widely) could have been written before the review had even been started. It is therefore not based on the data presented. Crucially it does not match the stated aim of this review (“to evaluate the effectiveness of conservative…interventions”).

But why? Why did the authors bother to follow PRISMA? Why did they formulate this bizarre conclusion in their abstract? Why did they do a review in the first place?

I fear, the answers might be embarrassingly simple:

  • They only pretended to follow PRISMA guidelines because that gives their review a veneer of respectability.
  • They formulated the conclusions because otherwise they would have needed to state that the evidence for manual therapy is less than convincing.
  • They conducted the review to promote chiropractic, and when the data were not as they had hoped for, they just back-paddled in an attempt to hide the truth as much as possible.

If this were an isolated case, I would not have bothered to mention it. But sadly, in the realm of chiropractic (and alternative medicine in general) we currently witness a plethora of rubbish reviews (published by rubbish journals). To the naïve observer, they might look rigorous and therefore they will be taken seriously. The end-effect of this pollution of the literature with rubbish is that we get a false-positive impression about the validity of the treatments in question. Consequently, we will see a host of wrong decisions on all levels of healthcare.

The big question is: HOW DO WE PROTECT OURSELVES FROM THIS DANGEROUS TREND?

I only see one solution: completely disregard certain journals that have been identified to regularly publish nonsense. Sadly, the wider medical community is far from having arrived at this point. As far as I can see, the problem has not even been identified yet as a serious issue that needs addressing. For the foreseeable future, we will probably have to live with this type of pollution of our medical literature.

The ‘Daily Mail’ is not a paper famed for its objective reporting. In politics, this can influence elections; in medicine, it can endanger public health.

A recent article is a case in point, I think.

START OF QUOTE

Traditional Chinese medicines could help prevent heart disease and the progression of pre-diabetes, according to research. Some herbal treatments proved as effective in lowering blood pressure as Western drugs and improved heart health by lowering cholesterol, scientists found. Certain alternative medicines could lower blood sugar and insulin levels, too.

Chinese medicines could be used alongside conventional treatments, say researchers from Shandong University Qilu Hospital in China. Or they can be beneficial as an alternative for patients intolerant of Western drugs, they said in their review of medical studies over a ten-year period. Senior review author from the university’s department of traditional Chinese medicine said: ‘The pharmacological effects and the underlying mechanisms of some active ingredients of traditional Chinese medications have been elucidated. Thus, some medications might be used as a complementary and alternative approach for primary and secondary prevention of cardiovascular disease.”

It’s potentially good news for people living with diabetes, which is now a global epidemic and has proved a tricky condition to manage for many people. High blood pressure is very common too, affecting more than one in four adults in the UK,  although many won’t show symptoms and realise it. If untreated, it increases your risk of serious problems including heart disease, the number one killer globally.

The Chinese have used herbs for treating diseases for thousands of years and have become increasingly popular in Europe and North America, mainly as complement to Western medicine. But the researchers also warn that much of the research conducted have limitations and so their long-term effects are not proven.

Key findings  

Herbs for high blood pressure

The blood pressure-lowering effect of herb zhongfujiangya was found to be similar to that of oral anti-hypertension medication benazeprilm, which goes by the brand name Lotensin. Similarly, patients treated for eight weeks with herbal tiankuijiangya had a lower reading than those given a placebo. Herbal Jiangya tablets were found to ‘significantly lower’ systolic blood pressure, that is the amount of pressure in your arteries during contraction of your heart muscle compared to a fake treatment. The herb Jiangyabao also had a significant effect compared to a placebo, but just at night. But overall, compared to the drug Nimodipine, a calcium channel blocker, it worked just as well. Qiqilian capsules also proved more effective compared to a placebo.

Herbs for diabetes

The team report some Chinese medicines medications – such as xiaoke, tangminling, jinlida, and jianyutangkang – have a ‘potent’ effect on lowering blood sugar levels and b-cell function, which controls the release of insulin. Some remedies – such as tangzhiping and tianqi – might prevent the progression of pre-diabetes to diabetes, they note.

Herbs for cholesterol 

The researchers looked at research on dyslipidemia, the term for unbalanced or unhealthy cholesterol levels. They found that jiangzhitongluo, salviamiltiorrhiza and pueraria lobata, and zhibitai capsule all have a ‘potent lipid-lowing effect’.

Herbs for heart disease

Some traditional Chinese medicines such as qiliqiangxin, nuanxin, shencaotongmai, and yangxinkang, might be effective in improving function in patients with chronic heart failure, they wrote.

Limitations with trials

But Western scientists often reject Chinese medicine for specific reasons, warned Dr Zhao’s team. Chinese medicines are frowned upon because they do not go through the same exhaustive approval process as trials conducted domestically, they pointed out. Plus, one treatment can be made of many different ingredients with various chemical compounds, making it hard to pinpoint how their benefits work. ‘One should bear in mind that traditional Chinese medicine medications are usually prescribed as complex formulae, which are often further manipulated by the practitioner on a personalized basis,’ said Dr Zhao.

END OF QUOTE

Apart from the fact that this article is badly written, it is also misleading to the point of being outright dangerous. Regular readers of my blog will be aware that Chinese research is everything but reliable; there are practically no Chinese TCM-trials that report negative results. Furthermore, the safety of Chinese herbal preparations is as good as unknown and they are often contaminated with toxic substances as well as adulterated with synthetic drugs. Most of these preparations are also unavailable outside China. Moreover, Chinese herbal treatments are usually individualised (mixtures are tailor-made for each individual patient), and there is no good evidence that this approach is effective. Crucially, the trial evidence is often of such poor quality that it would be a dangerous mistake to trust these findings.

None of these important caveats, it seems, are important enough to get a mention in the Daily Mail.

Don’t let the truth get in the way of a sensational story!

Let’s just for a moment imagine what would happen if people took the Mail article seriously (is there anyone out there who does take the Mail seriously?). In a best case scenario, they would take Chinese herbs in addition to their prescribed medication. This might case plenty of unwanted side-effects and herb-drug interactions. In addition, people would lose a lot of their hard-earned cash. In a worst case scenario, they would abandon their prescribed medication for dubious Chinese herbal mixtures. This could cause thousands of premature deaths.

With just a little research, I managed to find the original article on which the Mail’s report was based. Here is its abstract:

Traditional Chinese medicine (TCM) has more than 2,000 years of history and has gained widespread clinical applications. However, the explicit role of TCM in preventing and treating cardiovascular disease remains unclear due to a lack of sound scientific evidence. Currently available randomized controlled trials on TCM are flawed, with small sample sizes and diverse outcomes, making it difficult to draw definite conclusions about the actual benefits and harms of TCM. Here, we systematically assessed the efficacy and safety of TCM for cardiovascular disease, as well as the pharmacological effects of active TCM ingredients on the cardiovascular system and potential mechanisms. Results indicate that TCM might be used as a complementary and alternative approach to the primary and secondary prevention of cardiovascular disease. However, further rigorously designed randomized controlled trials are warranted to assess the effect of TCM on long-term hard endpoints in patients with cardiovascular disease.

In my view, the authors of this review are grossly over-optimistic in their conclusions (but nowhere near as bad as the Mail journalist). If the trials are of poor quality, as the review-authors admit, no firm conclusions should be permissible about the usefulness of the therapies in question.

As the Mail article is obviously based on a press release (several other papers worldwide reported about the review as well), it seems interesting to note what the editor of the Journal of the American College of Cardiology (the journal that published the review) recently had to say about the responsibility of journalists and researchers:

START OF QUOTE

…I would like to suggest that journalists and researchers must share equally in shouldering the burden of responsibility to improve appropriate communication about basic and clinical research.

First, there is an obligation on the part of the researchers not to inflate the importance of their findings. This has been widely recognized as damaging, especially if bias is introduced in the paper…

Second, researchers should take some responsibility for the creation of the press release about their research, which is written by the media or press relations department at their hospital or society. Press releases are often how members of the media get introduced to a particular study, and these releases can often introduce errors or exaggerations. In fact, British researchers evaluated 462 press releases on biomedical and health-related science issued by 20 leading U.K. universities in 2011, alongside their associated peer-reviewed research papers and the news stories that followed (n = 668). They found that 40% of the press releases contained exaggerated advice, 33% contained exaggerated causal claims, and 36% contained exaggerated inference to humans from animal research. When press releases contained such exaggeration, 58%, 81%, and 86% of news stories, respectively, contained further exaggeration, compared with rates of 17%, 18%, and 10% in the news when the press releases were not exaggerated. Researchers should not be excused from being part of the press release process, as the author(s) should at least review the release before it gets disseminated to the media. I would even encourage researchers to engage in the process at the writing stage and to not allow their hospital’s or society’s public relations department to extrapolate their study’s results. Ultimately, the authors and the journals in which the studies are published will be held accountable for the information that trickles into the headlines, not the public relations departments, so we must make sure that the information is accurate and representative of the study’s actual findings.

END OF QUOTE

Sound advice indeed.

Now we only need to ALL follow it!!!

This study tested chondroitin sulfate 800 mg/day (CS) pharmaceutical-grade in the management of symptomatic knee osteoarthritis. It was designed as a prospective, randomised, 6-month, 3-arm, double-blind, double-dummy, placebo and celecoxib (200 mg/day)-controlled trial.  The primary endpoints were changes in pain on a Visual Analogue Scale (VAS) and in the Lequesne Index (LI). Minimal-Clinically Important Improvement (MCII), Patient-Acceptable Symptoms State (PASS) were used as secondary endpoints.

A total of 604 patients, diagnosed according to American College of Rheumalogy (ACR) criteria, were recruited in five European countries and followed for 182 days. CS and celecoxib showed a greater significant reduction in pain and LI than placebo. In the intention-to-treat (ITT) population, pain reduction in VAS at day 182 in the CS group (−42.6 mm) and in celecoxib group (−39.5 mm) was significantly greater than the placebo group (−33.3 mm) (p=0.001 for CS and p=0.009 for celecoxib). No difference observed between CS and celecoxib. Similar trend for the LI, as reduction in this metric in the CS group (−4.7) and celecoxib group (−4.6) was significantly greater than the placebo group (−3.7) (p=0.023 for CS and p=0.015 for celecoxib). Again, no difference was observed between CS and celecoxib. Both secondary endpoints (MCII and PASS) at day 182 improved significantly in the CS and celecoxib groups. All treatments demonstrated excellent safety profiles.
The authors concluded that a 800 mg/day pharmaceutical-grade CS is superior to placebo and similar to celecoxib in reducing pain and improving function over 6 months in symptomatic knee osteoarthritis (OA) patients. This formulation of CS should be considered a first-line treatment in the medical management of knee OA.

In my view, this is a good study with clear and useful results: CS seems to be efficacious and safe. Another recent study confirmed the superiority of CS over celecoxib at reducing cartilage volume loss in knee OA patients.

The current Cochrane review does not yet account for the new data; it concluded cautiously positive: A review of randomized trials of mostly low quality reveals that chondroitin (alone or in combination with glucosamine) was better than placebo in improving pain in participants with osteoarthritis in short-term studies. The benefit was small to moderate with an 8 point greater improvement in pain (range 0 to 100) and a 2 point greater improvement in Lequesne’s index (range 0 to 24), both seeming clinically meaningful. These differences persisted in some sensitivity analyses and not others. Chondroitin had a lower risk of serious adverse events compared with control. More high-quality studies are needed to explore the role of chondroitin in the treatment of osteoarthritis. The combination of some efficacy and low risk associated with chondroitin may explain its popularity among patients as an over-the-counter supplement.

The call for more high quality trials was justified but has now been answered. In my view, CS can be considered an evidence-based option in the management of OA.

How often have I pointed out that most studies of chiropractic (and other alternative therapies) are overtly unethical because they fail to report adverse events? And if you think this is merely my opinion, you are mistaken. This new analysis by a team of chiropractors aimed to describe the extent of adverse events reporting in published RCTs of Spinal Manipulative Therapy (SMT), and to determine whether the quality of reporting has improved since publication of the 2010 Consolidated Standards Of Reporting Trials (CONSORT) statement.

The Physiotherapy Evidence Database and the Cochrane Central Register of Controlled Trials were searched for RCTs involving SMT. Domains of interest included classifications of adverse events, completeness of adverse events reporting, nomenclature used to describe the events, methodological quality of the study, and details of the publishing journal. Data were analysed using descriptive statistics. Frequencies and proportions of trials reporting on each of the specified domains above were calculated. Differences in proportions between pre- and post-CONSORT trials were calculated with 95% confidence intervals using standard methods, and statistical comparisons were analysed using tests for equality of proportions with continuity correction.

Of 7,398 records identified in the electronic searches, 368 articles were eligible for inclusion in this review. Adverse events were reported in 140 (38.0%) articles. There was a significant increase in the reporting of adverse events post-CONSORT (p=.001). There were two major adverse events reported (0.3%). Only 22 articles (15.7%) reported on adverse events in the abstract. There were no differences in reporting of adverse events post-CONSORT for any of the chosen parameters.

The authors concluded that although there has been an increase in reporting adverse events since the introduction of the 2010 CONSORT guidelines, the current level should be seen as inadequate and unacceptable. We recommend that authors adhere to the CONSORT statement when reporting adverse events associated with RCTs that involve SMT.

We conducted a very similar analysis back in 2012. Specifically, we evaluated all 60 RCTs of chiropractic SMT published between 2000 and 2011 and found that 29 of them did not mention adverse effects at all. Sixteen RCTs reported that no adverse effects had occurred (which I find hard to believe since reliable data show that about 50% of patients experience adverse effects after consulting a chiropractor). Complete information on incidence, severity, duration, frequency and method of reporting of adverse effects was included in only one RCT. Conflicts of interests were not mentioned by the majority of authors. Our conclusion was that adverse effects are poorly reported in recent RCTs of chiropractic manipulations.

The new paper suggests that the situation has improved a little, yet it is still wholly unacceptable. To conduct a clinical trial and fail to mention adverse effects is not, as the authors of the new article suggest, against current guidelines; it is a clear and flagrant violation of medical ethics. I blame the authors of such papers, the reviewers and the journal editors for behaving dishonourably and urge them to get their act together.

The effects of such non-reporting are obvious: anyone looking at the evidence (for instance via systematic reviews) will get a false-positive impression of the safety of SMT. Consequently, chiropractors are able to claim that very few adverse effects have been reported in the literature, therefore our hallmark therapy SMT is demonstrably safe. Those who claim otherwise are quite simply alarmist.

A recent post discussed a ‘STATE OF THE ART REVIEW’ from the BMJ. When I wrote it, I did not know that there was more to come. It seems that the BMJ is planning an entire series on the state of the art of BS! The new paper certainly looks like it:

Headaches, including primary headaches such as migraine and tension-type headache, are a common clinical problem. Complementary and integrative medicine (CIM), formerly known as complementary and alternative medicine (CAM), uses evidence informed modalities to assist in the health and healing of patients. CIM commonly includes the use of nutrition, movement practices, manual therapy, traditional Chinese medicine, and mind-body strategies. This review summarizes the literature on the use of CIM for primary headache and is based on five meta-analyses, seven systematic reviews, and 34 randomized controlled trials (RCTs). The overall quality of the evidence for CIM in headache management is generally low and occasionally moderate. Available evidence suggests that traditional Chinese medicine including acupuncture, massage, yoga, biofeedback, and meditation have a positive effect on migraine and tension headaches. Spinal manipulation, chiropractic care, some supplements and botanicals, diet alteration, and hydrotherapy may also be beneficial in migraine headache. CIM has not been studied or it is not effective for cluster headache. Further research is needed to determine the most effective role for CIM in patients with headache.

My BS-detector struggled with the following statements:

  • integrative medicine (CIM), formerly known as complementary and alternative medicine (CAM) – the fact that CIM is a nonsensical new term has been already mentioned in the previous post;
  • evidence informed modalities – another new term! evidence-BASED would be too much? because it would require using standards that do not apply to CIM? double standards promoted by the BMJ, what next?
  • CIM commonly includes the use of nutrition – yes, so does any healthcare or indeed life!
  • the overall quality of the evidence for CIM in headache management is generally low and occasionally moderate – in this case, no conclusions should be drawn from it (see below);
  • evidence suggests that traditional Chinese medicine including acupuncture, massage, yoga, biofeedback, and meditation have a positive effect on migraine and tension headaches – no, it doesn’t (see above)!
  • further research is needed to determine the most effective role for CIM in patients with headache – this sentence does not even make the slightest sense to me; have the reviewers of this article been asleep?

And this is just the abstract!

The full text provides enough BS to fertilise many acres of farmland!

Moreover, the article is badly researched, cherry-picked, poorly constructed, devoid of critical input, and poorly written. Is there anything good about it? You tell me – I did not find much!

My BS-detector finally broke when we came to the conclusions:

The use of CIM therapies has the potential to empower patients and help them take an active role in their care. Many CIM modalities, including mind-body therapies, are both self selected and self administered after an education period. This, coupled with patients’ increased desire to incorporate integrative medicine, should prompt healthcare providers to consider and discuss its inclusion in the overall management strategy. Low to moderate quality evidence exists for the effectiveness of some CIM therapies in the management of primary headache. The evidence for and use of CIM is continuously changing so healthcare professionals should direct their patients to reliable and updated resources, such as NCCIH.

WHAT IS HAPPENING TO THE BMJ?

IT USED TO BE A GOOD JOURNAL!

The website of BMJ Clinical Evidence seems to be popular with fans of alternative medicine (FAMs). That sounds like good news: it’s an excellent source, and one can learn a lot about EBM when studying it. But there is a problem: FAMs don’t seem to really study it (alternatively they do not have the power of comprehension to understand the data); they merely pounce on this figure and cite it endlessly:

They interpret it to mean that only 11% of what conventional clinicians do is based on sound evidence. This is water on their mills, because now they feel able to claim:

THE MAJORITY OF WHAT CONVENTIONAL CLINICIANS DO IS NOT EVIDENCE-BASED. SO, WHY DO SO-CALLED RATIONAL THINKERS EXPECT ALTERNATIVE THERAPIES TO BE EVIDENCE-BASED? IF WE NEEDED PROOF THAT THEY ARE HYPOCRITES, HERE IT IS!!!

The question is: are these FAMs correct?

The answer is: no!

They are merely using a logical fallacy (tu quoque); what is worse, they use it based on misunderstanding the actual data summarised in the above figure.

Let’s look at this in a little more detail.

The first thing we need to understand the methodologies used by ‘Clinical Evidence’ and what the different categories in the graph mean. Here is the explanation:


So, arguably the top three categories amounting to 42% signify some evidential support (if we decided to be more rigorous and merely included the two top categories, we would still arrive at 35%). This is not great, but we must remember two things here:

  • EBM is fairly new;
  • lots of people are working hard to improve the evidence base of medicine so that, in future, these figures will be better (by contrast, in alternative medicine, no similar progress is noticeable).

The second thing that strikes me is that, in alternative medicine, these figures would surely be much, much worse. I am not aware of reliable estimates, but I guess that the percentages might be one dimension smaller.

The third thing to mention is that the figures do not cover the entire spectrum of treatments available today but are based on ~ 3000 selected therapies. It is unclear how they were chosen, presumably the choice is pragmatic and based on the information available. If an up-to date systematic review has been published and provided the necessary information, the therapy was included. This means that the figures include not just mainstream but also plenty of alternative treatments (to the best of my knowledge ‘Clinical Evidence’ makes no distinction between the two). It is thus nonsensical to claim that the data highlight the weakness of the evidence in conventional medicine. It is even possible that the figures would be better, if alternative treatments had been excluded (I estimate that around 2 000 systematic reviews of alternative therapies have been published [I am the author of ~400 of them!]).

The fourth and possibly the most important thing to mention is that the percentage figures in the graph are certainly NOT a reflection of what percentage of treatments used in routine care are based on good evidence. In conventional practice, clinicians would, of course, select where possible those treatments with the best evidence base, while leaving the less well documented ones aside. In other words, they will use the ones in the two top categories much more frequently than those from the other categories.

At this stage, I hear some FAMs say: how does he know that?

Because several studies have been published that investigated this issue in some detail. They have monitored what percentage of interventions used by conventional clinicians in their daily practice are based on good evidence. In 2004, I reviewed these studies; here is the crucial passage from my paper:

“The most conclusive answer comes from a UK survey by Gill et al who retrospectively reviewed 122 consecutive general practice consultations. They found that 81% of the prescribed treatments were based on evidence and 30% were based on randomised controlled trials (RCTs). A similar study conducted in a UK university hospital outpatient department of general medicine arrived at comparable figures; 82% of the interventions were based on evidence, 53% on RCTs. Other relevant data originate from abroad. In Sweden, 84% of internal medicine interventions were based on evidence and 50% on RCTs. In Spain these percentages were 55 and 38%, respectively. Imrie and Ramey pooled a total of 15 studies across all medical disciplines, and found that, on average, 76% of medical treatments are supported by some form of compelling evidence — the lowest was that mentioned above (55%),6 and the highest (97%) was achieved in anaesthesia in Britain. Collectively these data suggest that, in terms of evidence-base, general practice is much better than its reputation.”

My conclusions from all this:

FAMs should study the BMJ Clinical Evidence more thoroughly. If they did, they might comprehend that the claims they tend to make about the data shown there are, in fact, bogus. In addition, they might even learn a thing or two about EBM which might eventually improve the quality of the debate.

1 2 3 17
Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted.


Click here for a comprehensive list of recent comments.

Categories