MD, PhD, FMedSci, FRSB, FRCP, FRCPEd.

study design

1 2 3 16

During the last 30 years, I must have read a few thousand studies of so-called alternative medicine (SCAM). Some made me angry because of their methodological flaws or wrong conclusions. A few impressed me. Many made me giggle. But none has ever caused me to laugh out so long as this one entitled ‘A STUDY ON THE PROPHYLACTIC EFFICACY OF HOMOEOPATHIC PREVENTIVE’.

Here is its abstract:

Homoeopathy has established its supremacy in the control of infectious viral diseases. The widespread acclaim in this regard is now supported by this study. The study was conducted in the Chikungunya fever hit areas of Kerala. The genus epidemicus was selected after detailed analysis of the first cases of Chikungunya. This preventive medicine was widely distributed in the disease prevalent areas. A survey was conducted for the evaluation of prophylactic efficacy. The study showed a very high significant effect of Homeopathic medicine in the prevention of Chikungunya fever.

You are, of course, correct to defend the Indian authors: it is unfair to judge a study purely on its abstract. So, let’s have a look at the rest. After a lengthy introduction, the heart of the full paper discloses the amazing details of the study.

Here I present the unabridged text of the study; the only part I have omitted is the introduction:

Aims and Objectives

1. To assess the efficacy of Homoeopathic medicine in the prevention of Chikungunya.
2. To determine the magnitude of incidence, clinical features, mortality , social & economic impact of the Chikungunya epidemic.

Conclusion

The Homoeopathic preventive medicine distributed for Chikungunya epidemic was highly effective.

THAT’S ALL!

The objective of this analysis was to evaluate the impact of chiropractic utilization upon use of prescription opioids among patients with spinal pain. The researchers employed a retrospective cohort design for analysis of health claims data from three contiguous US states for the years 2012-2017.

They included adults aged 18-84 years enrolled in a health plan and with office visits to a primary care physician or chiropractor for spinal pain. Two cohorts of subjects were thus identified:

  1. patients who received both primary care and chiropractic care,
  2. Patients who received primary care but not chiropractic care.

The total number of subjects was 101,221. Overall, between 1.55 and 2.03 times more nonrecipients of chiropractic care filled an opioid prescription, as compared with recipients.

The authors concluded that patients with spinal pain who saw a chiropractor had half the risk of filling an opioid prescription. Among those who saw a chiropractor within 30 days of diagnosis, the reduction in risk was greater as compared with those with their first visit after the acute phase.

Similar findings have been reported before and we have discussed them on this blog (see here, here and here). As before, one has to ask: WHAT DO THEY ACTUALLY MEAN?

The short answer is NOTHING MUCH! And certainly not what many chiros make of them.

They do not suggest that chiropractic care is a substitute for opioids in the management of spinal pain.

Why?

There are several reasons. Perhaps the most important ones are that such analyses lack any clinical outcome data, and that comparing one mistake (opioid-overuse) whith what might be another (chiropractic care) is a wrong apporoach. Imagine a scenario where half to the patients had received, in addition to their usual care, the services of:

  • a paranormal healer,
  • a crystal therapist,
  • a shaman,
  • or a homeopath.

Nobody would be surprised to see a very similar result, particularly if all of these practitioners were in the habit of discouraging their patients from using conventional drugs. Or imagine a scenario where half of all patients suffering from spinal pain are entered into an environment where they receive no treatment at all. Who would not expect that this regimen does not dramatically reduce the risk of filling an opioid prescription? But would that indicate that zero treatment is a good solution for managing spinal pain?

The thing is this:

  • If you want to reduce opioid use, you need to prescribe less opioids (for instance, by re-educating doctors to do as they have been told in med school and curb over-prescribing).
  • If you discourage patients to use opioids (as many other healthcare professionals would), many will not use opioids.
  • If you want to know whether chiropractic is effective in managing spinal pain, you need to conduct a well-designed clinical trial.

Or, to put it simply:

CORRELATION IS NOT CAUSATION!

 

Dry needling (DN), also known as myofascial trigger point dry needling, is a SCAM similar to acupuncture. It involves the use of solid filiform needles or hollow-core hypodermic needles and is usually employed for treating muscle pain. Instead of sticking them into acupuncture points, like with acupuncture, they are inserted into myofascial trigger points usually identified by palpation. There are some theories how DN might work, but whether it is clinically effective remains unclear.

This single-blind RCT determined, if the addition of upper quarter DN to a rehabilitation protocol is more effective in improving ROM, pain, and functional outcome scores when compared to a rehabilitation protocol alone after shoulder stabilization surgery. Thirty-nine post-operative shoulder patients were randomly allocated into two groups: (1) standard of care rehabilitation (control group) (2) standard of care rehabilitation plus dry needling (experimental group). Patient’s pain, ROM, and functional outcome scores were assessed at baseline (4 weeks post-operative), and at 8 weeks, 12 weeks, and 6 months post-operative.

Of 39 enrolled patients, 20 were allocated to the control group and 19 to the experimental group. At six-month follow up, there was a statistically significant improvement in shoulder flexion ROM in the control group. Aside from this, there were no significant differences in outcomes between the two treatment groups. Both groups showed improvement over time. No adverse events were reported.

The authors concluded that dry needling of the shoulder girdle in addition to standard of care rehabilitation after shoulder stabilization surgery did not significantly improve shoulder ROM, pain, or functional outcome scores when compared with standard of care rehabilitation alone. Both group’s improvement was largely equal over time. The significant difference in flexion at the six-month follow up may be explained by additional time spent receiving passive range of motion (PROM) in the control group. These results provide preliminary evidence that dry needling in a post-surgical population is safe and without significant risk of iatrogenic infection or other adverse events.

How odd!

This trial followed the infamous A+B versus B design. As [A+B] is more than [B] alone, one would have expected that the experimental group has a better outcome than the control group.

But this was not the case!

Why?

Theoretically it can mean one of two things:

  1. DN did not even convey a placebo effect.
  2. DN had a negative effect on the outcome.

Yesterday’s blog disclosed the fact that the German ‘Natur und Medizin’, an organisation of the ‘Carstens Stiftung’, had published slanderous lies about me. Consequently, I published an ‘open letter’ urging them to correct their mistake so that they would spare us the agony and cost of using legal action.

I never doubted for a minute that they would do this (I do not assume they are stupid, just a tiny bit dishonest) – and, as it turned out, I was correct. Here is a reminder of what they had originally published:

… er ist dafür bekannt, dass er kein gutes Haar an komplementären Therapieverfahren lässt. Notfalls greift er auch zu absichtlichen Falschdarstellungen[17], erfindet Daten[18] oder behauptet einfach, klinische Studien, die nicht die Negativ-Ergebnisse erbringen, die er erwartet, seien schlicht und ergreifend Betrug.[19]…

My rough translation:

… he [Edzard Ernst] is known for not finding anything positive in SCAM. If all else fails, he uses deliberate misrepresentation [17], invents data [18], or simply claims that clinical trials which did not generate the negative findings he expected are simply falsifications [19]…

The corrected new text passage is a little longer and now reads as follows (my rough translation):

… he [Edzard Ernst] is known for not finding anything positive in SCAM. Analyses of his publications by independent scientists draw the conclusion that he represents case-reports demonstrably wrongly [17] and that he arbitrarily alters or omits data [18]. He claims occasionally that high-quality studies of SCAM which do not generate the negative findings he expected appeared to be scientifically sound, but are nevertheless not believable [19]…

… er ist dafür bekannt, dass er kein gutes Haar an komplementären Therapieverfahren lässt. Analysen seiner Publikationen durch unabhängige Wissenschaftler gelangen zu der Schlussfolgerung, dass er Fallberichte nachweislich falsch darstelle[17] und Daten willkürlich verändere oder auslasse[18]. Er selbst behauptet mitunter über methodisch hochwertige Studien zur Komplementärmedizin, die nicht die Negativ-Ergebnisse erbringen, die er erwartet, sie sähen zwar nach wissenschaftlichen Maßstäben überzeugend aus, seien aber dennoch ‚unglaubwürdig‘.[19]… 

I would like to take this occasion to sincerely thank the ‘Natur und Medizin’ and the ‘Carstens Stiftung’ for this – much obliged guys, you made my day!

  • They have shown wisdom in not wasting money on expensive lawyers (even though my brother, who is a lawyer, might have enjoyed the windfall).
  • They have shown courage to hide behind papers like the one by Robert Hahn which have been discussed on this blog and elsewhere and found to be deluded.
  • They have shown strength by not meekly apologising to me about their attempt to slander me and my work.
  • They show leadership and innovative spirit by employing Jens Behnke, the author of the above lines, who does not seem to let the truth get in the way of a good story.

Last not least, my personal thanks to dear Jens (after your generosity, I am thinking about dedicating an entire blog post to you; your employer needs to know what a genius they have in you – watch this space) for yet again having demonstrated that the phenomenon known as ERNST’ S LAW is 100% correct.

The website of this organisation is always good for a surprise. A recent announcement relates to a course of Thought Field Therapy (TFT):

As part of our ongoing programme to explore prospects for improved healthcare, the College is pleased to announce a course on TFT – a “Tapping” therapy – independently provided by Janet Thomson MSc.

In healthcare we may find ourselves exhausting the evidence-based options and still looking for ways to help our patients. So when trusted practitioners suggest simple and safe approaches that appear to have benefit we are interested.

TFT is a simple non-invasive, technique that anyone can learn, for themselves or to pass on to their patients, to help cope with negative thoughts and emotions. It was developed by Roger Callahan who discovered that tapping on certain meridian points could help counter negative emotions. Janet trained with Roger and has become an accomplished exponent of the technique.

Janet has contracted her usual two-day course into one: to get the most from this will require access to her Tapping For Life book and there will be pre-course videos demonstrating some of the key techniques.  The second consecutive day is available for advanced TFT training, to help in dealing with difficult cases, as well as how to integrate TFT with other modalities.

How much does it cost (excluding booking fee)?  Day One only – £195; Day Two only – £195 (only available if you have previously completed day one); Both Days – £375.

When is it?  Saturday & Sunday 7th-8th March – 09:30-17:30

What, you don’t know what TFT is? Let me fill you in.

According to Wiki, TFT is a fringe psychological treatment developed by an American psychologist, Roger Callahan.[2] Its proponents say that it can heal a variety of mental and physical ailments through specialized “tapping” with the fingers at meridian points on the upper body and hands. The theory behind TFT is a mixture of concepts “derived from a variety of sources. Foremost among these is the ancient Chinese philosophy of chi, which is thought to be the ‘life force’ that flows throughout the body”. Callahan also bases his theory upon applied kinesiology and physics.[3] There is no scientific evidence that TFT is effective, and the American Psychological Association has stated that it “lacks a scientific basis” and consists of pseudoscience.[2]

Other assessments are even less complimentary: Thought field therapy (TFT) is a New Age psychotherapy dressed up in the garb of traditional Chinese medicine. It was developed in 1981 by Dr. Roger Callahan, a cognitive psychologist. While treating a patient for water phobia:

He asked her to think about water, tap with two fingers on the point that connected with the stomach meridian and much to his surprise, her fear of water completely disappeared.*

Callahan attributes the cure to the tapping, which he thinks unblocked “energy” in her stomach meridian. I don’t know how Callahan got the idea that tapping on a particular point would have anything to do with relieving a phobia, but he claims he has developed taps for just about anything that ails you, including a set of taps that can cure malaria (NPR interview).

TFT allegedly “gives immediate relief for post traumatic stress disorder (PTSD ), addictions, phobias, fears, and anxieties by directly treating the blockage in the energy flow created by a disturbing thought pattern. It virtually eliminates any negative feeling previously associated with a thought.”*

The theory behind TFT is that negative emotions cause energy blockage and if the energy is unblocked then the fears will disappear. Tapping acupressure points is thought to be the means of unblocking the energy. Allegedly, it only takes five to six minutes to elicit a cure. Dr. Callahan claims an 85% success rate. He even does cures over the phone using “Voice Technology” on infants and animals; by analyzing the voice he claims he can determine what points on the body the patient should tap for treatment.

_____________________________________________________________

Yes, TFT seems utterly implausible – but what about the clinical evidence?

There are quite a few positive controlled clinical trials of TFT. They all have one thing in common: they smell fishy to me! I know, that’s not a very scientific judgement. Let me rephrase it: I am not aware of a single trial that proves TFT to have effects beyond placebo (if you know one, please post the link).

And Janet Thomson, MSc (the therapist who runs the course), who is she? Her website is revealing; have a look if you are interested. If not, it might suffice to say that she modestly claims that she is an outstanding Life Coach, Therapist & Trainer.

So, considering that TFT is so very implausible and unproven, why does the ‘College of Medicine and Integrated Healthcare’ promote it in such strong terms?

I have to admit, I do not know the answer – perhaps they want at all costs to become known as the ‘College of Quack Medicine’?

In 2011, the following leading researchers of so-called alternative medicine (SCAM) – no I was not invited – had a meeting in Italy, did a brainstorm and decided what we would need to know about SCAM by 2020 (today, in other words):

They proposed 6 core areas of research that should be investigated to achieve a robust knowledge base and to allow stakeholders to make informed decisions:

  1. Research into the prevalence of SCAM in Europe: Reviews show that we do not know enough about the circumstances in which SCAM is used by Europeans. To enable a common European strategic approach, a clear picture of current use is of the utmost importance.
  2. Research into differences regarding citizens’ attitudes and needs towards SCAM: Citizens are the driver for CAM utilization. Their needs and views on SCAM are a key priority, and their interests must be investigated and addressed in future SCAM research.
  3. Research into safety of SCAM: Safety is a key issue for European citizens. SCAM is considered safe, but reliable data is scarce although urgently needed in order to assess the risk and cost-benefit ratio of SCAM.
  4. Research into the comparative effectiveness of SCAM: Everybody needs to know in what situation SCAM is a reasonable choice. Therefore, we recommend a clear emphasis on concurrent evaluation of the overall effectiveness of SCAM as an additional or alternative treatment strategy in real-world settings.
  5. Research into effects of context and meaning: The impact of effects of context and meaning on the outcome of SCAM treatments must be investigated; it is likely that they are significant.
  6. Research into different models of SCAM health care integration: There are different models of SCAM being integrated into conventional medicine throughout Europe, each with their respective strengths and limitations. These models should be described and concurrently evaluated; innovative models of SCAM provision in health care systems should be one focus for SCAM research.

‘Look, half the work is done! All you need to do is fill in the top part so we can legally say the bottom part.’

The researchers then added:

We also propose a methodological framework for SCAM research. We consider that a framework of mixed methodological approaches is likely to yield the most useful information. In this model, all available research strategies including comparative effectiveness research utilising quantitative and qualitative methods should be considered to enable us to secure the greatest density of knowledge possible. Stakeholders, such as citizens, patients and providers, should be involved in every stage of developing the specific and relevant research questions, study design and the assurance of real-world relevance for the research.

Furthermore, structural and sufficient financial support for research into SCAM is needed to strengthen SCAM research capacity if we wish to understand why it remains so popular within the EU. In order to consider employing SCAM as part of the solution to the health care, health creation and self-care challenges we face by 2020, it is vital to obtain a robust picture of SCAM use and reliable information about its cost, safety and effectiveness in real-world settings. We need to consider the availability, accessibility and affordability of SCAM. We need to engage in research excellence and utilise comparative effectiveness approaches and mixed methods to obtain this data.

Our recommendations are both strategic and methodological. They are presented for the consideration of researchers and funders while being designed to answer the important and implicit questions posed by EU citizens currently using SCAM in apparently increasing numbers. We propose that the EU actively supports an EU-wide strategic approach that facilitates the development of SCAM research. This could be achieved in the first instance through funding a European SCAM coordinating research office dedicated to foster systematic communication between EU governments, public, charitable and industry funders as well as researchers, citizens and other stakeholders. The aim of this office would be to coordinate research strategy developments and research funding opportunities, as well as to document and disseminate international research activities in this field.

With the aim to develop sustainability as second step, a European Centre for SCAM should be established that takes over the monitoring and further development of a coordinated research strategy for SCAM, as well as it should have funds that can be awarded to foster high quality and robust independent research with a focus on citizens health needs and pan-European collaboration.

We wish to establish a solid funding for SCAM research to adequately inform health care and health creation decision-making throughout the EU. This centre would ensure that our vision of a common, strategic and scientifically rigorous approach to SCAM research becomes our legacy and Europe’s reality. We are confident that our recommendations will serve these essential goals for EU citizens.

As I know all of the members of the panel personally, I am not surprised by the content of this document. That does not mean, however, that I do not find it remarkable. In my view, it is remarkable because of the nature of the 6 items that we allegedly need to know by 2020, and because of the fact that, even though none of them seem particularly demanding, today we have clarity or sound information on none of them. I also thought that both the research topics and the research methods were on the woolly side and, to a large degree, avoided what would be standard in conventional medicine. The ‘vision’ of the 13 researchers thus turns out to be the view of 13 partially sighted people on an array of platitudes.

Being just a bit sarcastic, the document could be seen as a plea for letting SCAM researchers:

  • continue to play on their far from level playing field,
  • use their preferred and largely inadequate methodologies,
  • pretend they do cutting edge science,
  • continue to avoid the real issues,
  • enjoy a life free of demanding challenges,
  • have pots of EU money for doing largely useless work.

In a word, I am confident that their recommendations would not have served any essential goals for EU citizens.

Ever since the government in Bavaria has been misguided enough to agree to a research programme testing whether homeopathy has a role in curtailing the over-use of anti-biotics, the subject of homeopathics as a replacement of antibiotics has been revived.

In this paper, homeopaths describe four female cases with recurrent urinary tract infections. The patients were treated successfully with the homeopathic strategy after several conventional approaches revealed no improvement. The follow-up period was a minimum of 3 years and the frequency of episodes with urinary tract infection as well as of antibiotic treatment was documented. Additionally, the patients were asked to assess the treatment outcome retrospectively in a validated questionnaire.

The treatment resulted in a reduction of urinary tract infections and the need for antibiotics from monthly to less than 3 times a year. Three of the four women had no cystitis and related intake of antibiotics for more than 1.5 years. A relapse of symptoms could be treated efficiently with a repetition of the homeopathic remedy. All subjective outcome assessments resulted positive.

The authors concluded that this case series suggests a possible benefit of individualized homeopathic treatment for female patients with recurrent urinary tract infections. Larger observational studies and controlled investigations are warranted. 

Such articles make me quite angry! They have the potential to mislead many patients and, in extreme cases, might even cost lives.

The ‘possible benefit’ of any treatment cannot be demonstrated with such flimsy case series. It has to be shown in properly controlled clinical trials. The findings of case series are confounded by dozens of variables and tell us next to nothing about cause and effect.

Case series make sense when they explore possible new therapeutic avenues. Homeopathy does certainly not fall into this category. The notion that homeopathics might be an alternative to antibiotics has been tested many times before in different settings, in animals, in humans, it vivo and in vitro. This has never generated convincingly positive findings. To re-address it by reporting uncontrolled cases is not just a nonsense; in my view, it is an unethical attempt to mislead us.

About 85% of German children are treated with herbal remedies. Yet, little is known about the effects of such interventions. A new study might tell us more.

This analysis accessed 2063 datasets from the paediatric population in the PhytoVIS data base, screening for information on indication, gender, treatment, co-medication and tolerability. The results suggest that the majority of patients was treated with herbal medicine for the following conditions:

  • common cold,
  • fever,
  • digestive complaints,
  • skin diseases,
  • sleep disturbances
  • anxiety.

The perceived effect of the therapy was rated in 84% of the patients as very good or good without adverse events.

The authors concluded that the results confirm the good clinical effects and safety of herbal medicinal products in this patient population and show that they are widely used in Germany.

If you are a fan of herbal medicine, you will be jubilant. If, on the other hand, you are a critical thinker or a responsible healthcare professional, you might wonder what this database is, why it was set up and how exactly these findings were produced. Here are some details:

The data were collected by means of a retrospective, anonymous, one-off survey consisting of 20 questions on the user’s experience with herbal remedies. The questions included complaints/ disease, information on drug use, concomitant factors/diseases as well as basic patient data. Trained interviewers performed the interviews in pharmacies and doctor’s offices. Data were collected in the Western Part of Germany between April 2014 and December 2016. The only inclusion criterion was the intake of herbal drugs in the last 8 weeks before the individual interview. The primary endpoint was the effect and tolerability of the products according to the user.

And who participated in this survey? If I understand it correctly, the survey is based on a convenience sample of parents using herbal remedies. This means that those parents who had a positive experience tended to volunteer, while those with a negative experience were absent or tended to refuse. (Thus the survey is not far from the scenario I often use where people in a hamburger restaurant are questioned whether they like hamburgers.)

So, there are two very obvious factors other than the effectiveness of herbal remedies determining the results:

  1. selection bias,
  2. lack of objective outcome measure.

This means that conclusions about the clinical effects of herbal remedies in paediatric patients are quite simply not possible on the basis of this survey. So, why do the authors nevertheless draw such conclusions (without a critical discussion of the limitations of their survey)?

Could it have something to do with the sponsor of the research?

The PhytoVIS study was funded by the Kooperation Phytopharmaka GbR Bonn, Germany.

Or could it have something to do with the affiliations of the paper’s authors:

1 Institute of Pharmacy, University of Leipzig, Brüderstr. 34, 04103, Leipzig, Germny. nieberkaren@gmx.de.

2 Kooperation Phytopharmaka GbR, Plittersdorfer Str. 218, 573, Bonn, Germany. nieberkaren@gmx.de.

3 Institute of Medical Statistics and Computational Biology, Faculty of Medicine, University of Cologne, Kerpener Str. 62, 50937, Cologne, Germany.

4 ClinNovis GmbH, Genter Str. 7, 50672, Cologne, Germany.

5 Bayer Consumer Health, Research & Development, Phytomedicines Supply and Development Center, Steigerwald Arzneimittelwerk GmbH, Havelstr. 5, 64295, Darmstadt, Germany.

6 Kooperation Phytopharmaka GbR, Plittersdorfer Str. 218, 53173, Bonn, Germany.

7 Institute of Pharmaceutical Biology, Goethe University Frankfurt, Max-von-Laue-Str. 9, 60438, Frankfurt, Germany.

8 Chair of Naturopathy, University Medicine Rostock, Ernst-Heydemann Str. 6, 18057, Rostock, Germany.

WHAT DO YOU THINK?

I have been sceptical about Craniosacral Therapy (CST) several times (see for instance here, here and here). Now, a new paper might change all this:

The systematic review assessed the evidence of Craniosacral Therapy (CST) for the treatment of chronic pain. Randomized clinical trials (RCTs) assessing the effects of CST in chronic pain patients were eligible. Pain intensity and functional disability were the primary outcomes. Risk of bias was assessed using the Cochrane tool.

Ten RCTs with a total of 681 patients suffering from neck and back pain, migraine, headache, fibromyalgia, epicondylitis, and pelvic girdle pain were included.

Compared to treatment as usual, CST showed greater post intervention effects on:

  • pain intensity (SMD=-0.32, 95%CI=[−0.61,-0.02])
  • disability (SMD=-0.58, 95%CI=[−0.92,-0.24]).

Compared to manual/non-manual sham, CST showed greater post intervention effects on:

  • pain intensity (SMD=-0.63, 95%CI=[−0.90,-0.37])
  • disability (SMD=-0.54, 95%CI=[−0.81,-0.28]) ;

Compared to active manual treatments, CST showed greater post intervention effects on:

  • pain intensity (SMD=-0.53, 95%CI=[−0.89,-0.16])
  • disability (SMD=-0.58, 95%CI=[−0.95,-0.21]) .

At six months, CST showed greater effects on pain intensity (SMD=-0.59, 95%CI=[−0.99,-0.19]) and disability (SMD=-0.53, 95%CI=[−0.87,-0.19]) versus sham. Secondary outcomes were all significantly more improved in CST patients than in other groups, except for six-month mental quality of life versus sham. Sensitivity analyses revealed robust effects of CST against most risk of bias domains. Five of the 10 RCTs reported safety data. No serious adverse events occurred. Minor adverse events were equally distributed between the groups.

The authors concluded that, in patients with chronic pain, this meta-analysis suggests significant and robust effects of CST on pain and function lasting up to six months. More RCTs strictly following CONSORT are needed to further corroborate the effects and safety of CST on chronic pain.

Robust effects! This looks almost convincing, particularly to an uncritical proponent of so-called alternative medicine (SCAM). However, a bit of critical thinking quickly discloses numerous problems, not with this (technically well-made) review, but with the interpretation of its results and the conclusions. Let me mention a few that spring into my mind:

  1. The literature searches were concluded in August 2018; why publish the paper only in 2020? Meanwhile, there might have been further studies which would render the review outdated even on the day it was published. (I know that there are many reasons for such a delay, but a responsible journal editor must insist on an update of the searches before publication.)
  2. Comparisons to ‘treatment as usual’ do not control for the potentially important placebo effects of CST and thus tell us nothing about the effectiveness of CST per se.
  3. The same applies to comparisons to ‘active’ manual treatments and ‘non-manual’ sham (the purpose of a sham is to blind patients; a non-manual sham defies this purpose).
  4. This leaves us with exactly two trials employing a sham that might have been sufficiently credible to be able to fool patients into believing that they were receiving the verum.
  5. One of these trials (ref 44) is far too flimsy to be taken seriously: it was tiny (n=23), did not adequately blind patients, and failed to mention adverse effects (thus violating research ethics [I cannot take such trials seriously]).
  6. The other trial (ref 41) is by the same research group as the review, and the authors award themselves a higher quality score than any other of the primary studies (perhaps even correctly, because the other trials are even worse). Yet, their study has considerable weaknesses which they fail to discuss: it was small (n=54), there was no check to see whether patient-blinding was successful, and – as with all the CST studies – the therapist was, of course, no blind. The latter point is crucial, I think, because patients can easily be influenced by the therapists via verbal or non-verbal communication to report the findings favoured by the therapist. This means that the small effects seen in such studies are likely to be due to this residual bias and thus have nothing to do with the intervention per se.
  7. Despite the fact that the review findings depend critically on their own primary study, the authors of the review declared that they have no conflict of interest.

Considering all this plus the rather important fact that CST completely lacks biological plausibility, I do not think that the conclusions of the review are warranted. I much prefer the ones from my own systematic review of 2012. It included 6 RCTs (all of which were burdened with a high risk of bias) and concluded that the notion that CST is associated with more than non‐specific effects is not based on evidence from rigorous RCTs.

So, why do the review authors first go to the trouble of conducting a technically sound systematic review and meta-analysis and then fail utterly to interpret its findings critically? I might have an answer to this question. Back in 2016, I included the head of this research group, Gustav Dobos, into my ‘hall of fame’ because he is one of the many SCAM researchers who never seem to publish a negative result. This is what I then wrote about him:

Dobos seems to be an ‘all-rounder’ whose research tackles a wide range of alternative treatments. That is perhaps unremarkable – but what I do find remarkable is the impression that, whatever he researches, the results turn out to be pretty positive. This might imply one of two things, in my view:

I let my readers chose which possibility they deem to be more likely.

Yesterday, we discussed a paper concluding (amongst other things) that there are insufficient high‐quality RCTs to judge the efficacy of acupuncture for cancer‐related pain. Today, we are looking at one that overtly contradicts this verdict.

This systematic review (published in JAMA Oncology) evaluated the existing randomized clinical trials (RCTs) for evidence of the association of acupuncture and acupressure with reduction in cancer pain. Randomized clinical trials that compared acupuncture and acupressure with a sham control, analgesic therapy, or usual care for managing cancer pain were included. The primary outcome was pain intensity measured by the Brief Pain Inventory, Numerical Rating Scale, Visual Analog Scale, or Verbal Rating Scale.

A total of 17 RCTs (with 1111 patients) were included, and data from 14 RCTs (with 920 patients) were used in the meta-analysis. Seven sham-controlled RCTs (35%) were notable for their high quality, being judged to have a low risk of bias for all of their domains, and showed that real (compared with sham) acupuncture was associated with reduced pain intensity. A favourable association was also seen when acupuncture and acupressure were combined with analgesic therapy in 6 RCTs for reducing pain intensity and in 2 RCTs for reducing opioid dose. The evidence grade was moderate because of the substantial heterogeneity among studies.

The authors concluded that this systematic review and meta-analysis found that acupuncture and/or acupressure was significantly associated with reduced cancer pain and decreased use of analgesics, although the evidence level was moderate. This finding suggests that more rigorous trials are needed to identify the association of acupuncture and acupressure with specific types of cancer pain and to integrate such evidence into clinical care to reduce opioid use.

So, which of the two conclusions should we trust?

Personally, I find the JAMA paper unimpressive to the point of being suspect. Here are some of my reasons:

  • About half of the primary studies are Chinese; and we have seen repeatedly that they are unreliable and report only positive results.
  • Many of the trials are published in Chinese and can thus not be checked by non-Chinese readers (nor, presumably, by the experts who acted as peer-reviewers for JAMA Oncology).
  • I have my doubts about the rigor of the peer-review of some of the journals that published the primary studies included in the review.
  • One paper included in the review is even a mere doctoral thesis which usually is not peer-reviewed in the usual sense.
  • The authors state that they included only clinical trials that compared acupuncture and acupressure with a sham control, analgesic therapy, or usual care. However, this is evidently not true; many of the studies had the infamous A+B versus B design comparing acupuncture plus a conventional therapy against the conventional therapy. As we have discussed ad nauseam on this blog, such trials cannot produce a negative finding even if ‘A’ is a placebo.
  • Contrary to what the authors claim, the quality of most of the included studies was extremely poor, as far as I can see.
  • One included paper which I cannot access is entitled ‘Clinical observation on 30 cases of moderate and severe cancer pain of bone metastasis treated by auricular acupressure‘. Are the review authors seriously claiming that this is an RCT?

The more I study the details of the JAMA Oncology paper, the more I feel it might be worth a complaint to the editor with a view of initiating a thorough investigation and a possible retraction.

 

1 2 3 16
Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted but you must tick the box: “Save my name, email, and website in this browser for the next time I comment.”

The most recent comments from all posts can be seen here.

Archives
Categories