MD, PhD, FMedSci, FRSB, FRCP, FRCPEd.

bias

One theory as to how acupuncture works is that it increases endorphin levels in the brain. These ‘feel-good’ chemicals could theoretically be helpful for weaning alcohol-dependent people off alcohol. So, for once, we might have a (semi-) plausible mechanism as to how acupuncture could be clinically effective. But a ‘beautiful hypothesis’ does not necessarily mean acupuncture works for alcohol dependence. To answer this question, we need clinical trials or systematic reviews of clinical trials.

A new systematic review assessed the effects and safety of acupuncture for alcohol withdrawal syndrome (AWS). All RCTs of drug plus acupuncture or acupuncture alone for the treatment of AWS were included. Eleven RCTs with a total of 875 participants were included. In the acute phase, two trials reported no difference between drug plus acupuncture and drug plus sham acupuncture in the reduction of craving for alcohol; however, two positive trials reported that drug plus acupuncture was superior to drug alone in the alleviation of psychological symptoms. In the protracted phase, one trial reported acupuncture was superior to sham acupuncture in reducing the craving for alcohol, one trial reported no difference between acupuncture and drug (disulfiram), and one trial reported acupuncture was superior to sham acupuncture for the alleviation of psychological symptoms. Adverse effects were tolerable and not severe.

The authors concluded that there was no significant difference between acupuncture (plus drug) and sham acupuncture (plus drug) with respect to the primary outcome measure of craving for alcohol among participants with AWS, and no difference in completion rates (pooled results). There was limited evidence from individual trials that acupuncture may reduce alcohol craving in the protracted phase and help alleviate psychological symptoms; however, given concerns about the quantity and quality of included studies, further large-scale and well-conducted RCTs are needed.

There is little to add here. Perhaps just two short points:

1. The quality of the trials was poor; only one study of the 11 trials was of acceptable rigor. Here is its abstract:

We report clinical data on the efficacy of acupuncture for alcohol dependence. 503 patients whose primary substance of abuse was alcohol participated in this randomized, single blind, placebo controlled trial. Patients were assigned to either specific acupuncture, nonspecific acupuncture, symptom based acupuncture or convention treatment alone. Alcohol use was assessed, along with depression, anxiety, functional status, and preference for therapy. This article will focus on results pertaining to alcohol use. Significant improvement was shown on nearly all measures. There were few differences associated with treatment assignment and there were no treatment differences on alcohol use measures, although 49% of subjects reported acupuncture reduced their desire for alcohol. The placebo and preference for treatment measures did not materially effect the results. Generally, acupuncture was not found to make a significant contribution over and above that achieved by conventional treatment alone in reduction of alcohol use.

To me, this does not sound all that encouraging.

2.  Of the 11 RCTs, 8 failed to report on adverse effects of acupuncture. In my book, this means these trials were in violation with basic research ethics.

My conclusion of all this: another ugly fact kills a beautiful hypothesis.

The authors of this paper wanted to establish and compare the effectiveness of Healing Touch (HT) and Oncology Massage (OM) therapies on cancer patients’ pain. They conducted pre-test/post-test, observational, retrospective study. A total of 572 outpatient oncology were recruited and asked to report pain before and after receiving a single session of either HT or OM from a certified practitioner.

Both HT and OM significantly reduced pain. Unadjusted rates of clinically significant pain improvement (defined as ≥2-point reduction in pain score) were 0.68 HT and 0.71 OM. Adjusted for pre-therapy pain, OM was associated with increased odds of pain improvement. For patients with severe pre-therapy pain, OM was not more effective in yielding clinically significant pain reduction when adjusting for pre-therapy pain score.

The authors concluded that both HT and OM provided immediate pain relief. Future research should explore the duration of pain relief, patient attitudes about HT compared with OM, and how this may differ among patients with varied pretherapy pain levels.

This paper made me laugh out loud; no, not because of the ‘certified’ practitioners (in the UK, we use this term to indicate that someone is not quite sane), but because of the admission that the authors aimed at establishing the effectiveness of their therapies. Most researchers of alternative medicine have exactly this motivation, but few make the mistake to write it into the abstract of their papers. Little do they know that this admission discloses a fatal amount of bias. Science is supposed to test hypotheses, and researchers who aim at establishing the effectiveness of their pet-therapy oust themselves as pseudo-researchers.

It comes therefore as no surprise that the study turns out to be a pseudo-study. As there was no adequate control group, these outcomes cannot be attributed to the interventions administered. The results could therefore be due to:

  • the time that has passed;
  • regression to the mean;
  • the attention provided by the therapists;
  • the expectation of the patient;
  • social desirability;
  • all of the above.

It follows that – just as with the study discussed in the previous post – the conclusion is wholly misleading. In fact, the data are consistent with the hypothesis that HT and OM both aggravated the pain (the results might have been better without HT and OM). The devils advocate concludes that both HT and OM provided an immediate increase in pain.

This study was aimed at evaluating group-level and individual-level change in health-related quality of life among persons with chronic low back pain or neck pain receiving chiropractic care in the United States.

A 3-month longitudinal study was conducted of 2,024 patients with chronic low back pain or neck pain receiving care from 125 chiropractic clinics at 6 locations throughout the US. Ninety-one percent of the sample completed the baseline and 3-month follow-up survey (n = 1,835). Average age was 49, 74% females, and most of the sample had a college degree, were non-Hispanic White, worked full-time, and had an annual income of $60,000 or more. Group-level and individual-level changes on the Patient-Reported Outcomes Measurement Information System (PROMIS) v2.0 profile measure were evaluated: 6 multi-item scales (physical functioning, pain, fatigue, sleep disturbance, social health, emotional distress) and physical and mental health summary scores.

Within group t-tests indicated significant group-level change for all scores except for emotional distress, and these changes represented small improvements in health. From 13% (physical functioning) to 30% (PROMIS-29 Mental Health Summary Score) got better from baseline to 3 months later.

The authors concluded that chiropractic care was associated with significant group-level improvement in health-related quality of life over time, especially in pain. But only a minority of the individuals in the sample got significantly better (“responders”). This study suggests some benefits of chiropractic on functioning and well-being of patients with low back pain or neck pain.

These conclusions are worded carefully to avoid any statement of cause and effect. But I nevertheless feel that the authors strongly imply that chiropractic caused the observed outcomes. This is perhaps most obvious when they state that this study suggests some benefits of chiropractic on functioning and well-being of patients with low back pain or neck pain.

To me, it is obvious that this is wrong. The data are just as consistent with the opposite conclusion. There was no control group. It is therefore conceivable that the patients would have improved more and/or faster, if they had never consulted a chiropractor. The devil’s advocate therefore concludes this: the results of this study suggest that chiropractic has significant detrimental effects on functioning and well-being of patients with low back pain or neck pain.

Try to prove me wrong!

PS

I am concerned that a leading journal (Spine) publishes such rubbish.

The ‘Schwaebische Tageblatt’ is not on my regular reading list. But this article of yesterday (16/10/2018) did catch my attention. For those who read German, I will copy it below, and for those who don’t I will provide a brief summary and comment thereafter:

Die grün-schwarze Landesregierung lässt 2019 den ersten Lehrstuhl für Naturheilkunde und Integrative Medizin in Baden-Württemberg einrichten. Lehrstuhl für Naturheilkunde und Integrative Medizin

Ihren Schwerpunkt soll die Professur im Bereich Onkologie haben. Strömungen wie Homöopathie oder Anthroposophie sollen nicht gelehrt, aber innerhalb der Lehre beleuchtet werden, sagte Ingo Autenrieth, Dekan der Medizinischen Fakultät in Tübingen am Dienstag der Deutschen Presse-Agentur. «Ideologien und alles, was nichts mit Wissenschaft zu tun hat, sortieren wir aus.»

Die Professur soll sich demnach mit Themen wie Ernährung, Probiotika und Akupunktur beschäftigten. Geplant ist laut Wissenschaftsministerium, die Lehre in Tübingen anzusiedeln; die Erforschung der komplementären Therapien soll vorwiegend am Centrum für Tumorerkrankungen des Robert-Bosch-Krankenhauses in Stuttgart stattfinden. Die Robert-Bosch-Stiftung finanziert die Professur in den ersten fünf Jahren mit insgesamt 1,84 Millionen Euro, danach soll das Land die Mittel dafür bereitstellen.

«Naturheilkunde und komplementäre Behandlungsmethoden werden von vielen Menschen ganz selbstverständlich genutzt, beispielsweise zur Ergänzung konventioneller Therapieangebote», begründete Wissenschaftsministerin Theresia Bauer (Grüne) das Engagement. Sogenannte sanfte oder natürliche Methoden könnten schwere Krankheiten wie etwa Krebs alleine nicht heilen, heißt es in einer Mitteilung des Ministeriums. Wissenschaftliche Ergebnisse zeigten aber, dass sie häufig zu Therapieerfolgen beitragen könnten, da sie den Patienten helfen, schulmedizinische Therapien gut zu überstehen – etwa die schweren Nebenwirkungen von Chemotherapien mindern.

Im Gegensatz zur Schulmedizin gebe es bisher aber kaum kontrollierte klinische Studien zur Wirksamkeit solcher Therapien, ergänzte Ingo Autenrieth. Ihre Erforschung am neuen Lehrstuhl solle Patienten Sicherheit bringen und ermöglichen, dass die gesetzlichen Krankenkassen die Kosten dafür übernehmen.

Hersteller alternativer Arzneimittel loben den Schritt der Politik. «Baden-Württemberg nimmt damit eine Vorreiterrolle in Deutschland und in Europa ein», heißt es beim Unternehmen Wala Heilmittel GmbH in Bad Boll. Die Landesregierung trage mit der Entscheidung dem Wunsch vieler Patienten und Ärzte nach umfassenden Behandlungskonzepten Rechnung.

Auch hoffen die Unternehmen, dass Licht in die oft kritische Debatte um Homöopathie gebracht wird. «Wir sehen mit Erstaunen und Befremden, dass eine bewährte Therapierichtung wie die Homöopathie, die Teil der Vielfalt des therapeutischen Angebots in Deutschland ist, diskreditiert werden soll», sagte ein Sprecher des Herstellers Weleda AG mit Sitz in Schwäbisch Gmünd der Deutschen Presse-Agentur. Deshalb begrüße man den Lehrstuhl: «Es ist gut, dass Forschung und Lehre ausgebaut werden, da eine Mehrheit der Bevölkerung Komplementärmedizin wünscht und nachfragt. Es braucht Ärzte, die in diesen Bereichen auch universitär ausgebildet werden.»

Laut Koalitionsvertrag will Baden-Württemberg künftig eine Vorreiterrolle in der Erforschung der Komplementärmedizin einnehmen. Bisher gab es im Südwesten mit dem Akademischen Zentrum für Komplementäre und Integrative Medizin (AZKIM) zwar einen Verbund der Unikliniken Tübingen, Freiburg, Ulm und Heidelberg, aber keinen eigenen Lehrstuhl. Bundesweit existieren nach Angaben der Hufelandgesellschaft, dem Dachverband der Ärztegesellschaften für Naturheilkunde und Komplementärmedizin, Lehrstühle für Naturheilkunde noch an den Universitäten Duisburg-Essen, Rostock und Witten/Herdecke sowie drei Stiftungsprofessuren an der Berliner Charité.

END OF QUOTE

And here is my English summary:

The black/green government of Baden-Wuerttemberg has decided to create a ‘chair of naturopathy and integrated medicine’ at the university of Tuebingen in 2019. The chair will focus in the area of oncology. Treatments such as homeopathy and anthroposophical medicine will not be taught but merely mentioned in lectures. Ideologies and everything that is not science will be omitted.

The chair will thus deal with nutrition, acupuncture and probiotics. The teaching activities will be in the medical faculty at Tuebingen, while the research will be located at the Robert-Bosch Hospital in Stuttgart. The funds for the first 5 years – 1.84 million Euro – will come from the Robert-Bosch Foundation; thereafter they will be provided by the government of the county.

So-called gentle or natural therapies cannot cure serious diseases on their own, but as adjuvant treatments they can be helful, for instance, in alleviating the adverse effects of chemotherapy. There are only few studies on this, and the new chair will increase patient safety and facilitate the reimbursement of these treatments by health insurances.

Local anthroposophy manufacturers like Wala welcomed the move stating it would be in accordance with the wishes of many patients and doctors. They also hope that the move will bring light in the current critical debate about homeopathy. A spokesperson of Weleda added that they ‘note with surprise that time-tested therapies like homeopathy are being discredited. Therefore, it is laudable that research and education in this realm will be extended. The majority of the public want complementary medicine and need doctors who are also university-trained.’

Baden-Wurttemberg aims for a leading role in researching complementary medicine. Thus far, chairs of complementary medicine existed only at the universities of Duisburg-Essen, Rostock und Witten/Herdecke as well as three professorships at the Charité in Berlin.

END OF MY SUMMARY

As I have occupied a chair of complementary medicine for 19 years, I am tempted to add a few points here.

  • In principle, a new chair can be a good thing.
  • The name of the chair is odd, to say the least.
  • As the dean of the Tuebingen medical school pointed out, it has to be based on science. But how do they define science?
  • Where exactly does the sponsor, the Robert-Bosch Stiftung, stand on alternative medicine. Do they have a track-record of being impractical and scientific?
  • In order to prevent this becoming a unrealistic prospect, it is essential that the new chair needs to fall into the hands of a scientist with a proven track record of critical thinking.
  • Rigorous scientist with a proven track record of critical thinking are very rare in the realm of alternative medicine.
  • The ridiculous comments by Wala and Weleda, both local firms with considerable local influence, sound ominous and let me suspect that proponents of alternative medicine aim to exert their influence on the new chair.
  • The above-voiced notion that the new chair is to facilitate the reimbursement of alternative treatments by the health insurances seems even more ominous. Proper research has to be objective and could, depending on its findings, have the opposite effect. To direct it in this way seems to determine its results before the research has started.
  • I miss a firm commitment to medical ethics, to the principles of EBM, and to protecting the independence of the new chair.

Thus, I do harbour significant anxieties about this new chair. It is in danger of becoming a chair of promoting pseudoscience. I hope the dean of the Tuebingen medical school might read these lines.

I herewith offer him all the help I can muster in keeping pseudoscience out of this initiative, in defining the remit of the chair and, crucially, in finding the right individual for doing the job.

Often referred to as “Psychological acupressure”, the emotional freedom technique (EFT) works by releasing blockages within the energy system which are the source of emotional intensity and discomfort. These blockages in our energy system, in addition to challenging us emotionally, often lead to limiting beliefs and behaviours and an inability to live life harmoniously. Resulting symptoms are either emotional and/ or physical and include lack of confidence and self esteem, feeling stuck anxious or depressed, or the emergence of compulsive and addictive behaviours. It is also now finally widely accepted that emotional disharmony is a key factor in physical symptoms and dis-ease and for this reason these techniques are being extensively used on physical issues, including chronic illness with often astounding results. As such these techniques are being accepted more and more in medical and psychiatric circles as well as in the range of psychotherapies and healing disciplines.

An EFT treatment involves the use of fingertips rather than needles to tap on the end points of energy meridians that are situated just beneath the surface of the skin. The treatment is non-invasive and works on the ethos of making change as simple and as pain free as possible.

EFT is a common sense approach that draws its power from Eastern discoveries that have been around for over 5,000 years. In fact Albert Einstein also told us back in the 1920’s that everything (including our bodies) is composed of energy. These ideas have been largely ignored by Western Healing Practices and as they are unveiled in our current times, human process is reopening itself to the forgotten truth that everything is Energy and the potential that this offers us.

END OF QUOTE

If you ask me, this sounds as though EFT combines pseudo-psychological with acupuncture-BS.

But I may be wrong.

What does the evidence tell us?

A systematic review included 14 RCTs of EFT with a total of 658 patients.  The pre-post effect size for the EFT treatment group was 1.23 (95% confidence interval, 0.82-1.64; p < 0.001), whereas the effect size for combined controls was 0.41 (95% confidence interval, 0.17-0.67; p = 0.001). Emotional freedom technique treatment demonstrated a significant decrease in anxiety scores, even when accounting for the effect size of control treatment. However, there were too few data available comparing EFT to standard-of-care treatments such as cognitive behavioural therapy, and further research is needed to establish the relative efficacy of EFT to established protocols.  Meta-analyses indicate large effect sizes for posttraumatic stress disorder, depression, and anxiety; however, treatment effects may be due to components EFT shares with other therapies.

Another, more recent analysis reviewed whether EFTs acupressure component was an active ingredient. Six studies of adults with diagnosed or self-identified psychological or physical symptoms were compared (n = 403), and three (n = 102) were identified. Pretest vs. posttest EFT treatment showed a large effect size, Cohen’s d = 1.28 (95% confidence interval [CI], 0.56 to 2.00) and Hedges’ g = 1.25 (95% CI, 0.54 to 1.96). Acupressure groups demonstrated moderately stronger outcomes than controls, with weighted posttreatment effect sizes of d = -0.47 (95% CI, -0.94 to 0.0) and g = -0.45 (95% CI, -0.91 to 0.0). Meta-analysis indicated that the acupressure component was an active ingredient and outcomes were not due solely to placebo, nonspecific effects of any therapy, or non-acupressure components.

From these and other reviews, one could easily get the impression that my above-mentioned suspicion is erroneous and EFT is an effective therapy. But I still do have my doubts.

Why?

These reviews conveniently forget to mention that the primary studies tend to be of poor or even very poor quality. The most common flaws include tiny sample sizes, wrong statistical approach, lack of blinding, lack of control of placebo and other nonspecific effects. Reviews of such studies thus turn out to be a confirmation of the ‘rubbish in, rubbish out’ principle: any summary of flawed studies are likely to produce a flawed result.

Until I have good quality trials to convince me otherwise, EFT is in my view:

  1. implausible and
  2. not of proven effectiveness for any condition.

Bee venom acupuncture is a form of acupuncture in which bee venom is applied to the tips of acupuncture needles, stingers are extracted from bees, or bees are held with an instrument exposing the stinger, and applied to acupoints on the skin.

Bee venom consisting of multiple anti-inflammatory compounds such as melittin, adolapin, apamin. Other substances such as phospholipase A2 can be anti-inflammatory in low concentrations and pro-inflammatory in others. However, bee venom also contains proinflammatory substances, melittin, mast cell degranulation peptide 401, and histamine.

Bee venom acupuncture has been used to treat a number of conditions such as lumbar disc disease, osteoarthritis of the knee, rheumatoid arthritis, adhesive capsulitis, lateral epicondylitis, peripheral neuropathies, stroke and Parkinson’s Disease. The quality of these studies tends to be so poor that any verdict on the effectiveness of bee venom acupuncture would be premature.

A new clinical trial of bee-venom acupuncture for rheumatoid arthritis (RA) might change this situation. A total of 120 cases of RA patients were randomized into bee-sting acupuncture group (treatment) and western medicine group (control). The patients of the control group were treated by oral administration of Methotrexate (10 mg, once a week) and Celecoxlb (0.2 g, once a day). Those of the treatment group received 5 to 15 bee stings of Ashi-points or acupoints according to different conditions and corporeity, and with the bee-sting retained for about 5 min every time, once every other day. The treatment lasted for 8 weeks. The therapeutic effect was assessed by examining:

  • symptoms and signs of the affected joints as morning stiffness duration,
  • swollen/tender joint counts (indexes),
  • handgrip strength,
  • 15 m-walking time,
  • visual analogue scale (VAS),
  • Disease Activity Score including a 28-joint count (DAS 28),
  • rheumatoid factor (RF),
  • erythrocyte sedimentation rate (ESR),
  • C-reactive protein (CRP),
  • anti-cyclic citrullinated peptide antibody (ACCPA).

For assessing the safety of bee-venom acupuncture, the patients’ responses of fever, enlargement of lymph nodes, regional red and swollen, itching, blood and urine tests for routine were examined.

Findings of DAS 28 responses displayed that of the two 60 cases in the control and bee-venom acupuncture groups, 15 and 18 experienced marked improvement, 33 and 32 were effective, 12 and 10 ineffective, with the effective rates being 80% and 83. 33%, respectively. No significant difference was found between the two groups in the effective rate (P>0.05). After the treatment, both groups have witnessed a marked decrease in the levels of morning stiffness duration, arthralgia index, swollen joint count index, joint tenderness index, 15 m walking time, VAS, RF, ESR, CRP and ACCPA, and an obvious increase of handgrip strength relevant to their own levels of pre-treatment in each group (P<0.05). There were no significant differences between the two groups in the abovementioned indexes (P>0.05). The routine blood test, routine urine test, routine stool test, electrocardiogram result, the function of liver and kidney and other security index were within the normal range, without any significant adverse effects found after bee-stinging treatment.

The authors (from the Department of Acupuncture and Moxibustion, Bao’an Hospital of Traditional Chinese Medicine, Shenzhen, China) concluded that bee-venom acupuncture therapy for RA patients is safe and effective, worthy of popularization and application in clinical practice.

Where to start? There is so much – perhaps I just comment on the conclusion:

  • Safety cannot be assessed on the basis of such a small sample. Bee venom can cause anaphylaxis, and several deaths have been reported in patients who successfully received the therapy prior to the adverse event. Because there is no adverse-effect monitoring system, the incidence of adverse events is unknown. Stating that it is safe, is therefore a big mistake.
  • The trial was a non-superiority study. As such, it needs a much larger sample to be able to make claims about effectiveness.
  • From the above two points, it follows that popularization and application in clinical practice would be a stupid exercise.

So, what is left over from this seemingly rigorous RCT?

NOTHING!

(except perhaps a re-affirmation of my often-voiced fear that we must take TCM-studies from China with more than just one pinch of salt)

Do musculoskeletal conditions contribute to chronic non-musculoskeletal conditions? The authors of a new paper – inspired by chiropractic thinking, it seems – think so. Their meta-analysis was aimed to investigate whether the most common musculoskeletal conditions, namely neck or back pain or osteoarthritis of the knee or hip, contribute to the development of chronic disease.

The authors searched several electronic databases for cohort studies reporting adjusted estimates of the association between baseline neck or back pain or osteoarthritis of the knee or hip and subsequent diagnosis of a chronic disease (cardiovascular disease , cancer, diabetes, chronic respiratory disease or obesity).

There were 13 cohort studies following 3,086,612 people. In the primary meta-analysis of adjusted estimates, osteoarthritis (n= 8 studies) and back pain (n= 2) were the exposures and cardiovascular disease (n=8), cancer (n= 1) and diabetes (n= 1) were the outcomes. Pooled adjusted estimates from these 10 studies showed that people with a musculoskeletal condition have a 17% increase in the rate of developing a chronic disease compared to people without a musculoskeletal condition.

The authors concluded that musculoskeletal conditions may increase the risk of chronic disease. In particular, osteoarthritis appears to increase the risk of developing cardiovascular disease. Prevention and early

treatment of musculoskeletal conditions and targeting associated chronic disease risk factors in people with long

standing musculoskeletal conditions may play a role in preventing other chronic diseases. However, a greater

understanding about why musculoskeletal conditions may increase the risk of chronic disease is needed.

For the most part, this paper reads as if the authors are trying to establish a causal relationship between musculoskeletal problems and systemic diseases at all costs. Even their aim (to investigate whether the most common musculoskeletal conditions, namely neck or back pain or osteoarthritis of the knee or hip, contribute to the development of chronic disease) clearly points in that direction. And certainly, their conclusion that musculoskeletal conditions may increase the risk of chronic disease confirms this suspicion.

In their discussion, they do concede that causality is not proven: While our review question ultimately sought to assess a causal connection between common musculoskeletal conditions and chronic disease, we cannot draw strong conclusions  due  to  poor  adjustment,  the  analysis methods employed by the included studies, and a lack of studies investigating conditions other than OA and cardiovascular disease…We did not find studies that satisfied all of Bradford Hill’s suggested criteria for casual inference (e.g. none estimated dose–response effects) nor did we find studies that used contemporary causal inference methods for observational data (e.g. a structured identification approach for selection of confounding variables or assessment of the effects of unmeasured or residual confounders. As such, we are unable to infer a strong causal connection between musculoskeletal conditions and chronic diseases.

In all honesty, I would see this a little differently: If their review question ultimately sought to assess a causal connection between common musculoskeletal conditions and chronic disease, it was quite simply daft and unscientific. All they could ever hope is to establish associations. Whether these are causal or not is an entirely different issue which is not answerable on the basis of the data they searched for.

An example might make this clearer: people who have yellow stains on their 2nd and 3rd finger often get lung cancer. The yellow fingers are associated with cancer, yet the link is not causal. The association is due to the fact that smoking stains the fingers and causes cancer. What the authors of this new article seem to suggest is that, if we cut off the stained fingers of smokers, we might reduce the cancer risk. This is clearly silly to the extreme.

So, how might the association between musculoskeletal problems and systemic diseases come about? Of course, the authors might be correct and it might be causal. This would delight chiropractors because DD Palmer, their founding father, said that 95% of all diseases are caused by subluxation of the spine, the rest by subluxations of other joints. But there are several other and more likely explanations for this association. For instance,  many people with a systemic disease might have had subclinical problems for years. These problems would prevent them from pursuing a healthy life-style which, in turn, resulted is musculoskeletal problems. If this is so, musculoskeletal conditions would not increase the risk of chronic disease, but chronic diseases would lead to musculoskeletal problems.

Don’t get me wrong, I am not claiming that this reverse causality is the truth; I am simply saying that it is one of several possibilities that need to be considered. The fact that the authors failed to do so, is remarkable and suggests that they were bent on demonstrating what they put in their conclusion. And that, to me, is an unfailing sign of poor science.

Proof of Principle or Concept studies are investigations usually for an early stage of clinical drug development when a compound has shown potential in animal models and early safety testing. This step often links between Phase-I and dose ranging Phase-II studies. These small-scale studies are designed to detect a signal that the drug is active on a patho-physiologically relevant mechanism, as well as preliminary evidence of efficacy in a clinically relevant endpoint.

For therapies that have been in use for many years, proof of concept studies are unusual to say the least. A proof of concept study of osteopathy has never been heard of. This is why I was fascinated by this new paper. The objective of this ‘proof of concept’ study was to evaluate the effect of osteopathic manipulative therapy (OMTh) on chronic symptoms of multiple sclerosis (MS).

Patients (n=22) with MS received 5 forty-minute MS health education sessions (control group) or 5 OMTh sessions (OMTh group). All participants completed a questionnaire that assessed their level of clinical disability, fatigue, depression, anxiety, and quality of life before the first session, one week after the final session, and 6 months after the final session. The Extended Disability Status Scale, a modified Fatigue Impact Scale, the Beck Depression Inventory-II, the Beck Anxiety Inventory, and the 12-item Short Form Health Survey were used to assess clinical disability, fatigue, depression, anxiety, and quality of life, respectively. In the OMTh group, statistically significant improvements in fatigue and depression were found one week after the final session. A non-significant increase in quality of life was also found in the OMTh group one week after the final session.

The authors concluded that the results demonstrate that OMTh should be considered in the treatment of patients with chronic symptoms of MS.

Who said that reading alternative medicine research papers is not funny? I for one laughed heartily when I read this (no need at all to go into the many obvious flaws of the study). Calling a pilot study ‘proof of concept’ is certainly not without hilarity. Drawing definitive conclusions about the effectiveness of OMTh is outright laughable. But issuing a far-reaching recommendation for use of OMTh in MS is just better than the best comedy. This had me in stiches!

I congratulate the Journal of the American Osteopathic Association and the international team of authors for providing us with such fun.

Osteopathy is a form of manual therapy invented by the American Andrew Taylor Still (1828-1917). Today, US osteopaths (doctors of osteopathy or DOs) practise no or little manual therapy; they are fully recognised as medical doctors who can specialise in any medical field after their training which is almost identical with that of MDs. Outside the US, osteopaths practice almost exclusively manual treatments and are considered alternative practitioners. This post deals with the latter category of osteopaths.

Still defined his original osteopathy as a science which consists of such exact, exhaustive, and verifiable knowledge of the structure and function of the human mechanism, anatomical, physiological and psychological, including the chemistry and physics of its known elements, as has made discoverable certain organic laws and remedial resources, within the body itself, by which nature under the scientific treatment peculiar to osteopathic practice, apart from all ordinary methods of extraneous, artificial, or medicinal stimulation, and in harmonious accord with its own mechanical principles, molecular activities, and metabolic processes, may recover from displacements, disorganizations, derangements, and consequent disease, and regained its normal equilibrium of form and function in health and strength.

Based on such vague and largely nonsensical statements, traditional osteopaths feel entitled to offer treatments for most human diseases, conditions and symptoms. The studies they produce to back up their claims tend to be as poor as Still’s original assumptions were fantastic.

Here is an apt example:

The aim of this new study was to study the effect of osteopathic manipulation on pain relief and quality of life improvement in hospitalized oncology geriatric patients.

The researchers conducted a non-randomized controlled clinical trial with 23 cancer patients. They were allocated to two groups: the study group (OMT [osteopathic manipulative therapy] group, N = 12) underwent OMT in addition to physiotherapy (PT), while the control group (PT group, N = 12) underwent only PT. Included were postsurgical cancer patients, male and female, age ⩾65 years, with an oncology prognosis of 6 to 24 months and chronic pain for at least 3 months with an intensity score higher than 3, measured with the Numeric Rating Scale. Exclusion criteria were patients receiving chemotherapy or radiotherapy treatment at the time of the study, with mental disorders (Mini-Mental State Examination [MMSE] = 10-20), with infection, anticoagulation therapy, cardiopulmonary disease, or clinical instability post-surgery. Oncology patients were admitted for rehabilitation after cancer surgery. The main cancers were colorectal cancer, osteosarcoma, spinal metastasis from breast and prostatic cancer, and kidney cancer.

The OMT, based on osteopathic principles of body unit, structure-function relationship, and homeostasis, was designed for each patient on the basis of the results of the osteopathic examination. Diagnosis and treatment were founded on 5 models: biomechanics, neurologic, metabolic, respiratory-circulatory, and behaviour. The OMT protocol was administered by an osteopath with clinical experience of 10 years in one-on-one individual sessions. The techniques used were: dorsal and lumbar soft tissue, rib raising, back and abdominal myofascial release, cervical spine soft tissue, sub-occipital decompression, and sacroiliac myofascial release. Back and abdominal myofascial release techniques are used to improve back movement and internal abdominal pressure. Sub-occipital decompression involves traction at the base of the skull, which is considered to release restrictions around the vagus nerve, theoretically improving nerve function. Sacroiliac myofascial release is used to improve sacroiliac joint movement and to reduce ligament tension. Strain-counter-strain and muscle energy technique are used to diminish the presence of trigger points and their pain intensity. OMT was repeated once every week during 4 weeks for each group, for a total of 4 treatments. Each treatment lasted 45 minutes.

At enrolment (T0), the patients were evaluated for pain intensity and quality of life by an external examiner. All patients were re-evaluated every week (T1, T2, T3, and T4) for pain intensity, and at the end of the study treatment (T4) for quality of life.

The OMT added to physiotherapy produced a significant reduction in pain both at T2 and T4. The difference in quality of life improvements between T0 and T4 was not statistically significant. Pain improved in the PT group at T4. Between-group analysis of pain and quality of life did not show any significant difference between the two treatments.

The authors concluded that our study showed a significant improvement in pain relief and a nonsignificant improvement in quality of life in hospitalized geriatric oncology patients during osteopathic manipulative treatment.

GOOD GRIEF!

Where to begin?

Even if there had been a difference in outcome between the two groups, such a finding would not have shown an effect of OMT per se. More likely, it would have been due to the extra attention and the expectation in the OMT group (or caused by the lack of randomisation). The A+B vs B design used for this study  does not control for non-specific effects. Therefore it is incapable of establishing a causal relationship between the therapy and the outcome.

As it turns out, there were no inter-group differences. How can this be? I have often stated that A+B is always more than B alone. And this is surely true!

So, how can I explain this?

As far as I can see, there are two possibilities:

  1. The study was underpowered, and thus an existing difference was not picked up.
  2. The OMT had a detrimental effect on the outcome measures thus neutralising the positive effects of the extra attention and expectation.

And which possibility does apply in this case?

Nobody can know from these data.

Integrative Cancer Therapies, the journal that published this paper, states that it focuses on a new and growing movement in cancer treatment. The journal emphasizes scientific understanding of alternative and traditional medicine therapies, and the responsible integration of both with conventional health care. Integrative care includes therapeutic interventions in diet, lifestyle, exercise, stress care, and nutritional supplements, as well as experimental vaccines, chrono-chemotherapy, and other advanced treatments. I feel that the editors should rather focus more on the quality of the science they publish.

My conclusion from all this is the one I draw so depressingly often: fatally flawed science is not just useless, it is unethical, gives clinical research a bad name, hinders progress, and can be harmful to patients.

I remember reading this paper entitled ‘Comparison of acupuncture and other drugs for chronic constipation: A network meta-analysis’ when it first came out. I considered discussing it on my blog, but then decided against it for a range of reasons which I shall explain below. The abstract of the original meta-analysis is copied below:

The objective of this study was to compare the efficacy and side effects of acupuncture, sham acupuncture and drugs in the treatment of chronic constipation. Randomized controlled trials (RCTs) assessing the effects of acupuncture and drugs for chronic constipation were comprehensively retrieved from electronic databases (such as PubMed, Cochrane Library, Embase, CNKI, Wanfang Database, VIP Database and CBM) up to December 2017. Additional references were obtained from review articles. With quality evaluations and data extraction, a network meta-analysis (NMA) was performed using a random-effects model under a frequentist framework. A total of 40 studies (n = 11032) were included: 39 were high-quality studies and 1 was a low-quality study. NMA showed that (1) acupuncture improved the symptoms of chronic constipation more effectively than drugs; (2) the ranking of treatments in terms of efficacy in diarrhoea-predominant irritable bowel syndrome was acupuncture, polyethylene glycol, lactulose, linaclotide, lubiprostone, bisacodyl, prucalopride, sham acupuncture, tegaserod, and placebo; (3) the ranking of side effects were as follows: lactulose, lubiprostone, bisacodyl, polyethylene glycol, prucalopride, linaclotide, placebo and tegaserod; and (4) the most commonly used acupuncture point for chronic constipation was ST25. Acupuncture is more effective than drugs in improving chronic constipation and has the least side effects. In the future, large-scale randomized controlled trials are needed to prove this. Sham acupuncture may have curative effects that are greater than the placebo effect. In the future, it is necessary to perform high-quality studies to support this finding. Polyethylene glycol also has acceptable curative effects with fewer side effects than other drugs.

END OF 1st QUOTE

This meta-analysis has now been retracted. Here is what the journal editors have to say about the retraction:

After publication of this article [1], concerns were raised about the scientific validity of the meta-analysis and whether it provided a rigorous and accurate assessment of published clinical studies on the efficacy of acupuncture or drug-based interventions for improving chronic constipation. The PLOS ONE Editors re-assessed the article in collaboration with a member of our Editorial Board and noted several concerns including the following:

  • Acupuncture and related terms are not mentioned in the literature search terms, there are no listed inclusion or exclusion criteria related to acupuncture, and the outcome measures were not clearly defined in terms of reproducible clinical measures.
  • The study included acupuncture and electroacupuncture studies, though this was not clearly discussed or reported in the Title, Methods, or Results.
  • In the “Routine paired meta-analysis” section, both acupuncture and sham acupuncture groups were reported as showing improvement in symptoms compared with placebo. This finding and its implications for the conclusions of the article were not discussed clearly.
  • Several included studies did not meet the reported inclusion criteria requiring that studies use adult participants and assess treatments of >2 weeks in duration.
  • Data extraction errors were identified by comparing the dataset used in the meta-analysis (S1 Table) with details reported in the original research articles. Errors included aspects of the study design such as the experimental groups included in the study, the number of study arms in the trial, number of participants, and treatment duration. There are also several errors in the Reference list.
  • With regard to side effects, 22 out of 40 studies were noted as having reported side effects. It was not made clear whether side effects were assessed as outcome measures for the other 18 studies, i.e. did the authors collect data clarifying that there were no side effects or was this outcome measure not assessed or reported in the original article. Without this clarification the conclusion comparing side effect frequencies is not well supported.
  • The network geometry presented in Fig 5 is not correct and misrepresents some of the study designs, for example showing two-arm studies as three-arm studies.
  • The overall results of the meta-analysis are strongly reliant on the evidence comparing acupuncture versus lactulose treatment. Several of the trials that assessed this comparison were poorly reported, and the meta-analysis dataset pertaining to these trials contained data extraction errors. Furthermore, potential bias in studies assessing lactulose efficacy in acupuncture trials versus lactulose efficacy in other trials was not sufficiently addressed.

While some of the above issues could be addressed with additional clarifications and corrections to the text, the concerns about study inclusion, the accuracy with which the primary studies’ research designs and data were represented in the meta-analysis, and the reporting quality of included studies directly impact the validity and accuracy of the dataset underlying the meta-analysis. As a consequence, we consider that the overall conclusions of the study are not reliable. In light of these issues, the PLOS ONE Editors retract the article. We apologize that these issues were not adequately addressed during pre-publication peer review.

LZ disagreed with the retraction. YM and XD did not respond.

END OF 2nd QUOTE

Let me start by explaining why I initially decided not to discuss this paper on my blog. Already the first sentence of the abstract put me off, and an entire chorus of alarm-bells started ringing once I read further.

  • A meta-analysis is not a ‘study’ in my book, and I am somewhat weary of researchers who employ odd or unprecise language.
  • We all know (and I have discussed it repeatedly) that studies of acupuncture frequently fail to report adverse effects (in doing this, their authors violate research ethics!). So, how can it be a credible aim of a meta-analysis to compare side-effects in the absence of adequate reporting?
  • The methodology of a network meta-analysis is complex and I know not a lot about it.
  • Several things seemed ‘too good to be true’, for instance, the funnel-plot and the overall finding that acupuncture is the best of all therapeutic options.
  • Looking at the references, I quickly confirmed my suspicion that most of the primary studies were in Chinese.

In retrospect, I am glad I did not tackle the task of criticising this paper; I would probably have made not nearly such a good job of it as PLOS ONE eventually did. But it was only after someone raised concerns that the paper was re-reviewed and all the defects outlined above came to light.

While some of my concerns listed above may have been trivial, my last point is the one that troubles me a lot. As it also related to dozens of Cochrane reviews which currently come out of China, it is worth our attention, I think. The problem, as I see it, is as follows:

  • Chinese (acupuncture, TCM and perhaps also other) trials are almost invariably reporting positive findings, as we have discussed ad nauseam on this blog.
  • Data fabrication seems to be rife in China.
  • This means that there is good reason to be suspicious of such trials.
  • Many of the reviews that currently flood the literature are based predominantly on primary studies published in Chinese.
  • Unless one is able to read Chinese, there is no way of evaluating these papers.
  • Therefore reviewers of journal submissions tend to rely on what the Chinese review authors write about the primary studies.
  • As data fabrication seems to be rife in China, this trust might often not be justified.
  • At the same time, Chinese researchers are VERY keen to publish in top Western journals (this is considered a great boost to their career).
  • The consequence of all this is that reviews of this nature might be misleading, even if they are published in top journals.

I have been struggling with this problem for many years and have tried my best to alert people to it. However, it does not seem that my efforts had even the slightest success. The stream of such reviews has only increased and is now a true worry (at least for me). My suspicion – and I stress that it is merely that – is that, if one would rigorously re-evaluate these reviews, their majority would need to be retracted just as the above paper. That would mean that hundreds of papers would disappear because they are misleading, a thought that should give everyone interested in reliable evidence sleepless nights!

So, what can be done?

Personally, I now distrust all of these papers, but I admit, that is not a good, constructive solution. It would be better if Journal editors (including, of course, those at the Cochrane Collaboration) would allocate such submissions to reviewers who:

  • are demonstrably able to conduct a CRITICAL analysis of the paper in question,
  • can read Chinese,
  • have no conflicts of interest.

In the case of an acupuncture review, this would narrow it down to perhaps just a handful of experts worldwide. This probably means that my suggestion is simply not feasible.

But what other choice do we have?

One could oblige the authors of all submissions to include full and authorised English translations of non-English articles. I think this might work, but it is, of course, tedious and expensive. In view of the size of the problem (I estimate that there must be around 1 000 reviews out there to which the problem applies), I do not see a better solution.

(I would truly be thankful, if someone had a better one and would tell us)

Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted but you must tick the box: “Save my name, email, and website in this browser for the next time I comment.”

The most recent comments from all posts can be seen here.

Categories