Can I tempt you to run a little (hopefully instructive) thought-experiment with you? It is quite simple: I will tell you about the design of a clinical trial, and you will tell me what the likely outcome of this study would be.
Are you game?
Here we go:
Imagine we conduct a trial of acupuncture for persistent pain (any type of pain really). We want to find out whether acupuncture is more than a placebo when it comes to pain-control. Of course, we want our trial to look as rigorous as possible. So, we design it as a randomised, sham-controlled, partially-blinded study. To be really ‘cutting edge’, our study will not have two but three parallel groups:
1. Standard needle acupuncture administered according to a protocol recommended by a team of expert acupuncturists.
2. Minimally invasive sham-acupuncture employing shallow needle insertion using short needles at non-acupuncture points. Patients in groups 1 and 2 are blinded, i. e. they are not supposed to know whether they receive the sham or real acupuncture.
3. No treatment at all.
We apply the treatments for a sufficiently long time, say 12 weeks. Before we start, after 6 and 12 weeks, we measure our patients’ pain with a validated method. We use sound statistical methods to compare the outcomes between the three groups.
WHAT DO YOU THINK THE RESULT WOULD BE?
You are not sure?
Well, let me give you some hints:
Group 3 is not going to do very well; not only do they receive no therapy at all, but they are also disappointed to have ended up in this group as they joined the study in the hope to get acupuncture. Therefore, they will (claim to) feel a lot of pain.
Group 2 will be pleased to receive some treatment. However, during the course of the 6 weeks, they will get more and more suspicious. As they were told during the process of obtaining informed consent that the trial entails treating some patients with a sham/placebo, they are bound to ask themselves whether they ended up in this group. They will see the short needles and the shallow needling, and a percentage of patients from this group will doubtlessly suspect that they are getting the sham treatment. The doubters will not show a powerful placebo response. Therefore, the average pain scores in this group will decrease – but only a little.
Group 1 will also be pleased to receive some treatment. As the therapists cannot be blinded, they will do their best to meet the high expectations of their patients. Consequently, they will benefit fully from the placebo effect of the intervention and the pain score of this group will decrease significantly.
So, now we can surely predict the most likely result of this trial without even conducting it. Assuming that acupuncture is a placebo-therapy, as many people do, we now see that group 3 will suffer the most pain. In comparison, groups 1 and 2 will show better outcomes.
Of course, the main question is, how do groups 1 and 2 compare to each other? After all, we designed our sham-controlled trial in order to answer exactly this issue: is acupuncture more than a placebo? As pointed out above, some patients in group 2 would have become suspicious and therefore would not have experienced the full placebo-response. This means that, provided the sample sizes are sufficiently large, there should be a significant difference between these two groups favouring real acupuncture over sham. In other words, our trial will conclude that acupuncture is better than placebo, even if acupuncture is a placebo.
THANK YOU FOR DOING THIS THOUGHT EXPERIMENT WITH ME.
Now I can tell you that it has a very real basis. The leading medical journal, JAMA, just published such a study and, to make matters worse, the trial was even sponsored by one of the most prestigious funding agencies: the NIH.
Here is the abstract:
Musculoskeletal symptoms are the most common adverse effects of aromatase inhibitors and often result in therapy discontinuation. Small studies suggest that acupuncture may decrease aromatase inhibitor-related joint symptoms.
To determine the effect of acupuncture in reducing aromatase inhibitor-related joint pain.
Design, Setting, and Patients:
Randomized clinical trial conducted at 11 academic centers and clinical sites in the United States from March 2012 to February 2017 (final date of follow-up, September 5, 2017). Eligible patients were postmenopausal women with early-stage breast cancer who were taking an aromatase inhibitor and scored at least 3 on the Brief Pain Inventory Worst Pain (BPI-WP) item (score range, 0-10; higher scores indicate greater pain).
Patients were randomized 2:1:1 to the true acupuncture (n = 110), sham acupuncture (n = 59), or waitlist control (n = 57) group. True acupuncture and sham acupuncture protocols consisted of 12 acupuncture sessions over 6 weeks (2 sessions per week), followed by 1 session per week for 6 weeks. The waitlist control group did not receive any intervention. All participants were offered 10 acupuncture sessions to be used between weeks 24 and 52.
Main Outcomes and Measures:
The primary end point was the 6-week BPI-WP score. Mean 6-week BPI-WP scores were compared by study group using linear regression, adjusted for baseline pain and stratification factors (clinically meaningful difference specified as 2 points).
Among 226 randomized patients (mean [SD] age, 60.7 [8.6] years; 88% white; mean [SD] baseline BPI-WP score, 6.6 [1.5]), 206 (91.1%) completed the trial. From baseline to 6 weeks, the mean observed BPI-WP score decreased by 2.05 points (reduced pain) in the true acupuncture group, by 1.07 points in the sham acupuncture group, and by 0.99 points in the waitlist control group. The adjusted difference for true acupuncture vs sham acupuncture was 0.92 points (95% CI, 0.20-1.65; P = .01) and for true acupuncture vs waitlist control was 0.96 points (95% CI, 0.24-1.67; P = .01). Patients in the true acupuncture group experienced more grade 1 bruising compared with patients in the sham acupuncture group (47% vs 25%; P = .01).
Conclusions and Relevance:
Among postmenopausal women with early-stage breast cancer and aromatase inhibitor-related arthralgias, true acupuncture compared with sham acupuncture or with waitlist control resulted in a statistically significant reduction in joint pain at 6 weeks, although the observed improvement was of uncertain clinical importance.
Do you see how easy it is to deceive (almost) everyone with a trial that looks rigorous to (almost) everyone?
My lesson from all this is as follows: whether consciously or unconsciously, SCAM-researchers often build into their trials more or less well-hidden little loopholes that ensure they generate a positive outcome. Thus even a placebo can appear to be effective. They are true masters of producing false-positive findings which later become part of a meta-analysis which is, of course, equally false-positive. It is a great shame, in my view, that even top journals (in the above case JAMA) and prestigious funders (in the above case the NIH) cannot (or want not to?) see behind this type of trickery.
Having yesterday been to a ‘Skeptics in the Pub’ event on MEDITATION in Cambridge (my home town since last year) I had to think about the subject quite a bit. As I have hardly covered this topic on my blog, I am today trying to briefly summarise my view on it.
The first thing that strikes me when looking at the evidence on meditation is that it is highly confusing. There seem to be:
- a lack of clear definitions,
- hundreds of studies, most of which are of poor or even very poor quality,
- lots of people with ’emotional baggage’,
- plenty of strange links to cults and religions,
- dozens of different meditation methods and regimen,
- unbelievable claims by enthusiasts,
- lots of weirdly enthusiastic followers.
What was confirmed yesterday is the fact that, once we look at the reliable medical evidence, we are bound to find that the health claims of various meditation techniques are hugely exaggerated. There is almost no strong evidence to suggest that meditation does affect any condition. The small effects that do emerge from some meta-analyses could easily be due to residual bias and confounding; it is not possible to rigorously control for placebo effects in clinical trials of meditation.
Another thing that came out clearly yesterday is the fact that meditation might not be as risk-free as it is usually presented. Several cases of psychoses after meditation are on record; some of these are both severe and log-lasting. How often do they happen? Nobody knows! Like with most alternative therapies, there is no reporting system in place that could possibly give us anything like a reliable answer.
For me, however, the biggest danger with (certain forms of) meditation is not the risk of psychosis. It is the risk of getting sucked into a cult that then takes over the victim and more or less destroys his or her personality. I have seen this several times, and it is a truly frightening phenomenon.
In our now 10-year-old book THE DESKTOP GUIDE TO COMPLEMENTARY AND ALTERNATIVE MEDICINE, we included a chapter on meditation. It concluded that “meditation appears to be safe for most people and those with sufficient motivation to practise regularly will probably find a relaxing experience. Evidence for effectiveness in any indication is week.” Even today, this is not far off the mark, I think. If I had to re-write it now, I would perhaps mention the potential for harm and also add that, as a therapy, the risk/benefit balance of meditation fails to be convincingly positive.
I highly recommend ‘Skeptics in the Pub’ events to anyone who likes stimulating talks and critical thinking.
I stumbled over an article entitled ‘The myths of homeopathy: Resounding answers‘. I thought it was great fun, so much so, that I copied it below – not just once but twice. The second time I took the liberty of replacing the little porkies told by homeopaths with the truth.
Homeopathic medicines are not placebos! Little “pellets” of sugar cannot have an effect!
Of course, the sugar in homeopathic pellets doesn’t have any effect. This is why we also have tablets and drops that contain homeopathic active substances.
The sugar is simply a medium for these active substances. The important element is what has been added to the sugar – the active ingredient!
As homeopathic remedies have very slow action, they cannot be used to treat acute illnesses!
This is incorrect. You can successfully use homeopathy in acute circumstances such as infections, fevers and colds.
Homeopathy seems to be a kind of magic!
Homeopathy is not magic! Homeopathy is a field of medicine that has the capacity to heal, but if course, it has its limits, just like any other medicine, including conventional medicine.
To give you a clear example – it’s unlikely that homeopathy will replace a surgical intervention.
During homeopathic treatment you have to follow a strict diet!
Well, it’s not such a bad thing…but of course, you need to eat healthily and avoid smoking, drinking alcohol and coffee.
In some cases you can’t eat onion or garlic as they contain sulphur, which is a homeopathic remedy in itself. All of these things have little to do with a strict diet.
Diabetes sufferers can’t use homeopathic remedies!
This is not true. The amount of sugar in the pellets is negligible. These homeopathic pellets could even be taken on a daily basis. The foods we eat contain much more sugar, even those that are especially for diabetics.
MY CORRECTED VERSION
Homeopathic medicines are not placebos! Little “pellets” of sugar cannot have an effect!
Of course, the sugar in homeopathic pellets doesn’t have any effect. And the drops added also contain no active substances.
In other words, there is no active ingredient!
As homeopathic remedies have very slow action, they cannot be used to treat acute illnesses!
This is correct. You cannot successfully use homeopathy in acute circumstances such as infections, fevers and colds. In fact, you cannot use it to cure any condition, chronic or acute.
Homeopathy seems to be a kind of magic!
Homeopathy is not magic! It relies on the placebo and other non-specific effects, and that is no magic.
During homeopathic treatment you have to follow a strict diet!
Hahnemann gave very clear instructions to avoid a whole range of things while taking homeopathic remedies – otherwise, they don’t work, he claimed. This is as wrong as everything else Hahnemann said about homeopathy: these remedies don’t work whatever you do.
Diabetes sufferers can’t use homeopathic remedies!
This is not true. The amount of sugar in the pellets is negligible. These homeopathic pellets could even be taken on a daily basis. The foods we eat contain much more sugar, even those that are especially for diabetics. But that does, of course, not mean that diabetics ought to take homeopathic remedies. There is no reason why they should; these remedies are pure placebos.
Few people reading these lines will be surprised that the ‘resounding answers’ turn out to be resounding lies. And what I above called ‘great fun’, turns out to be a serious deception.
The fascinating thing here is, I think, the way homeopaths try to mislead the public: one seemingly innocent untruth about the ‘active substance’ is used as the basis for an entire house of cards. It tumbles at the slightest attempt to provide the facts. Sadly, many consumers do not know the facts and are therefore prone to fall victim of these resounding lies.
There is perhaps not a law against such lies, but there certainly are moral and ethical principles that must not be violated:
TELLING LIES OF THIS NATURE IS UNETHICAL AND ENDANGERS THE HEALTH OF THOSE WHO DO NOT KNOW THE FACTS.
Why do most alternative practitioners show such dogged determination not to change their view of the efficacy of their therapy, even if good evidence shows that it is a placebo? This is the question that I have been pondering for some time. I have seen many doctors change their mind about this or that treatment in the light of new evidence. In fact, I have not seen one who has not done so at some stage. Yet I have never seen an alternative therapist change his/her mind about his/her alternative therapy. Why is that?
You might say that the answers are obvious:
- because they have heavily invested in their therapy, both emotionally and financially;
- because their therapy has ‘stood the test of time’;
- because they believe what they were taught;
- because they are deluded, not very bright, etc.;
- because they need to earn a living.
All of these reasons may apply. But do they really tell the whole story? While contemplating about this question, I thought of something that had previously not been entirely clear to me: they simply KNOW that the evidence MUST be wrong.
Let me try to explain.
Consider an acupuncturist (I could have chosen almost any other type of alternative practitioner) who has many years of experience. He has grown to be a well-respected expert in the world of acupuncture. He sits on various committees and has advised important institutions. He knows the literature and has treated thousands of patients.
This experience has taught him one thing for sure: his patients do benefit from his treatment. He has seen it happening too many times; it cannot be a coincidence. Acupuncture works, no question about it.
And this is also what the studies tell him. Even the most sceptical scientist cannot deny the fact that patients do get better after acupuncture. So, what is the problem?
The problem is that sceptics say that this is due to a placebo effect, and many studies seem to confirm this to be true. Yet, our acupuncturist completely dismisses the placebo explanation.
- Because he has heavily invested in their therapy? Perhaps.
- Because acupuncture has ‘stood the test of time’? Perhaps.
- Because he believes what he has been taught? Perhaps.
- Because he is deluded, not very bright, etc.? Perhaps.
- Because he needs to earn a living? Perhaps.
But there is something else.
He has only ever treated his patients with acupuncture. He has therefore no experience of real medicine, or other therapeutic options. He has no perspective. Therefore, he does not know that patients often get better, even if they receive an ineffective treatment, even if they receive no treatment, and even if they receive a harmful treatment. Every improvement he notes in his patients, he relates to his acupuncture. Our acupuncturist never had the opportunity to learn to doubt cause and effect in his clinical routine. He never had to question the benefits of acupuncture. He never had to select from a pool of therapies the optimal one, because he only ever used acupuncture.
It is this lack of experience that never led him to think critically about acupuncture. He is in a similar situation as physicians were 200 years ago; they only (mainly) had blood-letting, and because some patients improved with it, they had no reason to doubt it. He only ever saw his successes (not that all his patients improved, but those who did not, did not return). He simply KNOWS that acupuncture works, because his own, very limited experience never forced him to consider anything else. And because he KNOWS, the evidence that does not agree with his knowledge MUST be wrong.
I am of course exaggerating and simplifying in order to make a point. And please don’t get me wrong.
I am not saying that doctors cannot be stubborn. And I am not saying that all alternative practitioners have such limited experience and are unable to change their mind in the light of new evidence. However, I am trying to say that many alternative practitioners have a limited perspective and therefore find it impossible to be critical about their own practice.
If I am right, there would be an easy (and entirely alternative) cure to remedy this situation. We should sent our acupuncturist to a homeopath (or any other alternative practitioner whose practice he assumes to be entirely bogus) and ask him to watch what kind of therapeutic success the homeopath is generating. The acupuncturist would soon see that it is very similar to his own. He would then have the choice to agree that highly diluted homeopathic remedies are effective in curing illness, or that the homeopath relies on the same phenomenon as his own practice: placebo.
Sadly, this is not going to happen, is it?
When NHS England announced several months ago that they plan to stop the reimbursement of homeopathy, UK homeopaths were understandably upset and decided to launch a legal challenge to this decision. Yesterday, the result of this challenge has been published in a NHS press-release:
START OF QUOTE
NHS chief Simon Stevens has today welcomed the High Court’s decisive rejection of a legal challenge by the British Homeopathic Association to overturn plans to no longer routinely fund homeopathy on the NHS.
As part of action to clamp down on waste, over the last two years NHS England has taken action to curb prescriptions for medicines that can be bought over the counter or are of low value.
At the end of last year NHS England published guidance to curb prescriptions for 18 ineffective, unsafe or low clinical priority treatments, such as coproxamol, some dietary supplements, herbal treatments and homeopathy, saving up to £141 million a year. Earlier this year NHS England published a further list of 35 minor, short-term conditions for which over the counter medicines should not routinely be prescribed, saving around a further £100 million a year.
Savings form a key building block of the NHS’s 10 point efficiency plan contained in the Next Steps on the NHS Five Year Forward View, published in March 2017.
NHS chief, Simon Stevens said: “There is no robust evidence to support homeopathy which is at best a placebo and a misuse of scarce NHS funds.
“So we strongly welcome the High Court’s clear cut decision to kick out this costly and spurious legal challenge.”
Guidance on items which should not be routinely prescribed in primary care is available on the NHS England website.
END OF QUOTE
The reaction of UK homeopaths was as swift and deluded as it was predictable. The British Homeopathy Association, for instance, commented thus:
… The charity’s main claims against NHS England were that the consultation misrepresented homeopathy and therefore was unfair; and a report used in the consultation to inform the public was so complicated it would deter rather than encourage people to respond. Although the judge found there were sufficient grounds for a judicial review, after four days of lengthy legal arguments he dismissed the claims. Margaret Wyllie, BHA Chair, said the case highlighted how health bosses were unfairly manipulating the consultation process and making decisions about healthcare services without genuine patient engagement. “That NHS England attracted fewer than 3,000 responses from patients to a national consultation that ran for three months highlights its failure to genuinely engage with the public on important decisions about healthcare provision. Although 18 medicines were under review the only negative statement in NHS England’s press release promoting its public consultation was about homeopathy. The statement was so prejudicial it was widely reported in the media that the decision to deny patients homeopathic medicines had already been taken. How the judge failed to recognise that this was a deliberate attempt by NHS England to unfairly influence the public is astonishing,” said Mrs Wyllie.
In The Telegraph, Wyllie is quoted saying: “It appears NHS England can fail to engage with patients properly on removing services and get away with it. That is not good enough, for it is important to remember that the real losers in this case are the patients who are now being refused a treatment on which they have come to depend.”
One can only marvel at the lack of insight and self-criticism. I should to add that the BHA is a charity; with this court case, they have wasted significant amounts of public money for their own, hopelessly misguided interests. To me, this indicates that they no longer deserve a charitable status.
Personally, I had very little doubt that the court would decide as it did. The evidence was simply overwhelming and indisputable. In the written judgement, the judge stated that “I am satisfied that NHSE was rigorous in the discharge of the duty to have “due regard” to relevant matters, and that it was entitled, on the evidence before it, to conclude that the Guidance would not have an adverse impact on the statutory equality objectives, but rather, as the Analysis found, “would [enable] patients to have access to the most effective medications to achieve the best outcomes”.
In all this, the open questions, in my view, were whether
- Prince Charles, the prime defender of quackery in the UK, would intervene,
- and whether his intervention might change anything.
So, did he try to intervene?
I do not know. It could be that, after all the hoo-ha about his spider memos, he now is a little more cautions about meddling in health politics. It could also be that, as he is getting ready to become king, he wants to keep a low profile about his more bizarre ideas. Lastly, it could be that his opinion does not weigh as heavily as I had feared.
In any case, The High Court ruling is most welcome and unquestionably just, progressive, and long-overdue. I applaud all who have worked hard to bring it about, and am sure that (most of) my readers agree.
Reflexology is an alternative therapy that is subjectively pleasant and objectively popular; it has been the subject on this blog before (see also here and here). Reflexologists assume that certain zones on the sole of our feet correspond to certain organs, and that their manual treatment can influence the function of these organs. Thus reflexology is advocated for all sorts of conditions, including infant colic.
The aim of this new study was to explore the effect of reflexology on infantile colic.
A total of 64 babies with colic were included in this study. Following a paediatrician’s diagnosis, two groups (study and control) were created. Socio-demographic data (including mother’s age, educational status, and smoking habits of parents) and medical history of the baby (including gender, birth weight, mode of delivery, time of the onset breastfeeding after birth, and nutrition style) were collected. The Infant Colic Scale (ICS) was used to estimate the colic severity in the infants. Reflexology was applied to the study group by the researcher and their mother 2 days a week for 3 weeks. The babies in the control group did not receive reflexology. Assessments were performed before and after the intervention in both groups.
The results show that the two groups were similar regarding socio-demographic background and medical history. While there was no difference between the groups in ICS scores before application of reflexology, the mean ICS score of the study group was significantly lower than that of control group at the end of the intervention.
The authors concluded that reflexology application for babies suffering from infantile colic may be a promising method to alleviate colic severity.
The authors seem to attribute the outcome to specific effects of reflexology.
However, they are mistaken!
Because their study does not control for the non-specific effects of the intervention.
Reflexology has not been shown to work for anything (“the best clinical evidence does not demonstrate convincingly reflexology to be an effective treatment for any medical condition“), and there is plenty of evidence to show that holding the baby, massaging it, cuddling it, rocking it or doing just about anything with it will have an effect, e. g.:
I think, in a way, this is rather good news; we do not need to believe in the hocus-pocus of reflexology in order to help our crying infants.
Today, enthusiasts of homeopathy celebrate the start of the HOMEOPATHY AWARENESS WEEK. Let’s join them by re-addressing one of their favourite themes: their personal experience with homeopathy.
Most homeopathy-fans argue that the negative scientific evidence must be wrong because they have had positive experiences. Whenever I give a lecture, for instance, there will be at least one person in the audience who presents such an experience (and I too could contribute a few such stories from my own past). Such ‘case reports’ can, of course, be interesting, illuminating or leading to further research, but they can never be conclusive.
This concept is often profoundly confusing for patients and consumers. They tend to feel that I am doubting their words, but nothing could be further from the truth. Their experience is certainly true – what might be false is their interpretation of it. I think, I better explain this in more detail using a concrete, published example.
After the publication of our 2003 RCT of homeopathic Arnica which showed that two different potencies have effects that do not differ from those of placebo, I received lots of angry responses from people who told me that they had the opposite experience or observed positive outcomes on their pets. In my subsequent publication in the journal ‘Homeopathy‘ entitled ‘The benefits of Arnica: 16 case reports‘, I have tried my best to explain their experiences in the light of our finding that highly diluted homeopathic Arnica is a placebo:
Sixteen case reports of the apparent beneﬁts of Arnica … raise several relevant points. Firstly, topical Arnica preparations are often wrongly equated with homeopathic Arnica, the subject of our trial. The former are herbal preparations (ie not homeopathically diluted), which have undisputed pharmacological activity. Taken orally they would even be toxic. Thus all Arnica for oral administration must be highly diluted and has therefore no pharmacological effects. The case reports show that many lay people seem to be unclear about the difference between herbal and homeopathic Arnica.
Secondly, if animals seem to respond to homeopathic Arnica, as claimed in several of the case reports, this is not necessarily a proof of its effectiveness. Animals are not immune to placebo effects. Think of Pavlov’s experiments and the fact that conditioning is clearly an element in the placebo response.
Thirdly, the natural history of the condition can mimic clinical improvement caused by therapy. Many of the 16 cases summarized can be explained through a placebo response or the natural history of disease or the combination of both phenomena…
Many of the letters I received were outspoken to say the least. The authors stated that they were ‘appalled’, ‘saddened and angry’ by our research. Others implied that I was paid by the pharmaceutical industry to abolish homeopathy in the UK. One person felt that ‘it is highly irresponsible to dismiss a natural healing remedy with no evidence at all’. I believe the case reports … convey an important message about the power of belief, anecdotes, placebos and expectation.
END OF QUOTE
The thing about case reports and personal experiences is quite simply this: they may seem almost overwhelmingly convincing, but they can NEVER serve as a proof that the treatment in question was effective. The reason for this fact could not be more simple. Any therapeutic response is due to a complex combination of factors: placebo effects, natural history of the condition, regression to the mean, etc.
See it this way: you wake up one morning with an enormous hangover. You try to identify the cause of it. Was it the beer you had in the pub? The wine you drank before you went out? Or the whiskey you consumed before you went to bed? Perhaps you think it was the Cognac you enjoyed at a friend’s house? Only one thing is for sure: it was not the glass of shaken water you drank during the night.
The plethora of dodgy meta-analyses in alternative medicine has been the subject of a recent post – so this one is a mere update of a regular lament.
This new meta-analysis was to evaluate evidence for the effectiveness of acupuncture in the treatment of lumbar disc herniation (LDH). (Call me pedantic, but I prefer meta-analyses that evaluate the evidence FOR AND AGAINST a therapy.) Electronic databases were searched to identify RCTs of acupuncture for LDH, and 30 RCTs involving 3503 participants were included; 29 were published in Chinese and one in English, and all trialists were Chinese.
The results showed that acupuncture had a higher total effective rate than lumbar traction, ibuprofen, diclofenac sodium and meloxicam. Acupuncture was also superior to lumbar traction and diclofenac sodium in terms of pain measured with visual analogue scales (VAS). The total effective rate in 5 trials was greater for acupuncture than for mannitol plus dexamethasone and mecobalamin, ibuprofen plus fugui gutong capsule, loxoprofen, mannitol plus dexamethasone and huoxue zhitong decoction, respectively. Two trials showed a superior effect of acupuncture in VAS scores compared with ibuprofen or mannitol plus dexamethasone, respectively.
The authors from the College of Traditional Chinese Medicine, Jinan University, Guangzhou, Guangdong, China, concluded that acupuncture showed a more favourable effect in the treatment of LDH than lumbar traction, ibuprofen, diclofenac sodium, meloxicam, mannitol plus dexamethasone and mecobalamin, fugui gutong capsule plus ibuprofen, mannitol plus dexamethasone, loxoprofen and huoxue zhitong decoction. However, further rigorously designed, large-scale RCTs are needed to confirm these findings.
Why do I call this meta-analysis ‘dodgy’? I have several reasons, 10 to be exact:
- There is no plausible mechanism by which acupuncture might cure LDH.
- The types of acupuncture used in these trials was far from uniform and included manual acupuncture (MA) in 13 studies, electro-acupuncture (EA) in 10 studies, and warm needle acupuncture (WNA) in 7 studies. Arguably, these are different interventions that cannot be lumped together.
- The trials were mostly of very poor quality, as depicted in the table above. For instance, 18 studies failed to mention the methods used for randomisation. I have previously shown that some Chinese studies use the terms ‘randomisation’ and ‘RCT’ even in the absence of a control group.
- None of the trials made any attempt to control for placebo effects.
- None of the trials were conducted against sham acupuncture.
- Only 10 studies 10 trials reported dropouts or withdrawals.
- Only two trials reported adverse reactions.
- None of these shortcomings were critically discussed in the paper.
- Despite their affiliation, the authors state that they have no conflicts of interest.
- All trials were conducted in China, and, on this blog, we have discussed repeatedly that acupuncture trials from China never report negative results.
And why do I find the journal ‘dodgy’?
Because any journal that publishes such a paper is likely to be sub-standard. In the case of ‘Acupuncture in Medicine’, the official journal of the British Medical Acupuncture Society, I see such appalling articles published far too frequently to believe that the present paper is just a regrettable, one-off mistake. What makes this issue particularly embarrassing is, of course, the fact that the journal belongs to the BMJ group.
… but we never really thought that science publishing was about anything other than money, did we?
The pro arguments essentially are the well-rehearsed points acupuncture-fans like to advance:
- Some guidelines do recommend acupuncture.
- Sham acupuncture is not a valid comparator.
- The largest meta-analysis shows a small effect.
- Acupuncture is not implausible.
- It improves quality of life.
Cummings concludes as follows: In summary, the pragmatic view sees acupuncture as a relatively safe and moderately effective intervention for a wide range of common chronic pain conditions. It has a plausible set of neurophysiological mechanisms supported by basic science.12 For those patients who choose it and who respond well, it considerably improves health related quality of life, and it has much lower long term risk for them than non-steroidal anti-inflammatory drugs. It may be especially useful for chronic musculoskeletal pain and osteoarthritis in elderly patients, who are at particularly high risk from adverse drug reactions.
Our arguments are also not new; essentially, we stress that:
- The effects of acupuncture are too small to be clinically relevant.
- They are probably not even caused by acupuncture, but the result of residual bias.
- Pragmatic trials are of little value in defining efficacy.
- Acupuncture is not free of risks.
- Regular acupuncture treatments are expensive.
- There is no generally accepted, plausible mechanism.
We concluded that after decades of research and hundreds of acupuncture pain trials, including thousands of patients, we still have no clear mechanism of action, insufficient evidence for clinically worthwhile benefit, and possible harms. Therefore, doctors should not recommend acupuncture for pain.
Neither Asbjorn nor I have any conflicts of interests to declare.
Dr Cummings, by contrast, states that he is the salaried medical director of the British Medical Acupuncture Society, which is a membership organisation and charity established to stimulate and promote the use and scientific understanding of acupuncture as part of the practice of medicine for the public benefit. He is an associate editor for Acupuncture in Medicine, published by BMJ. He has a modest private income from lecturing outside the UK, royalties from textbooks, and a partnership teaching veterinary surgeons in Western veterinary acupuncture. He has participated in a NICE guideline development group as an expert adviser discussing acupuncture. He has used Western medical acupuncture in clinical practice following a chance observation as a medical officer in the Royal Air Force in 1989.
My question to you is this: WHICH OF THE TWO POSITION IS THE MORE REASONABLE ONE?
Please, do let us know by posting a comment here, or directly at the BMJ article (better), or both (best).
The question whether spinal manipulative therapy (SMT) has any specific therapeutic effects is still open. This fact must irritate ardent chiropractors, and they therefore try everything to dispel our doubts. One way would be to demonstrate a dose-effect relationship between SMT and the clinical outcome. But, for several reasons, this is not an easy task.
This RCT was aimed at identifying the dose-response relationship between visits for SMT and chronic cervicogenic headache (CGH) outcomes; to evaluate the efficacy of SMT by comparison with a light massage control.
The study included 256 adults with chronic CGH. The primary outcome was days with CGH in the prior 4 weeks evaluated at the 12- and 24-week primary endpoints. Secondary outcomes included CGH days at remaining endpoints, pain intensity, disability, perceived improvement, medication use, and patient satisfaction. Participants were randomized to 4 different dose levels of chiropractic SMT: 0, 6, 12, or 18 sessions. They were treated 3 times per week for 6 weeks and received a focused light-massage control at sessions when SMT was not assigned. Linear dose effects and comparisons to the no-manipulation control group were evaluated at 6, 12, 24, 39, and 52 weeks.
A linear dose-response was observed for all follow-ups, a reduction of approximately 1 CGH day/4 weeks per additional 6 SMT visits (p<.05); a maximal effective dose could not be determined. CGH days/4 weeks were reduced from about 16 to 8 for the highest and most effective dose of 18 SMT visits. Mean differences in CGH days/4 weeks between 18 SMT visits and control were -3.3 (p=.004) and -2.9 (p=.017) at the primary endpoints, and similar in magnitude at the remaining endpoints (p<.05). Differences between other SMT doses and control were smaller in magnitude (p > .05). CGH intensity showed no important improvement nor differed by dose. Other secondary outcomes were generally supportive of the primary.
The authors concluded that there was a linear dose-response relationship between SMT visits and days with CGH. For the highest and most effective dose of 18 SMT visits, CGH days were reduced by half, and about 3 more days per month than for the light-massage control.
This trial would make sense, if the effectiveness of SMT for CGH had been a well-documented fact, and if the study had rigorously controlled for placebo-effects.
But guess what?
Neither of these conditions were met.
A recent review concluded that there are few published randomized controlled trials analyzing the effectiveness of spinal manipulation and/or mobilization for TTH, CeH, and M in the last decade. In addition, the methodological quality of these papers is typically low. Clearly, there is a need for high-quality randomized controlled trials assessing the effectiveness of these interventions in these headache disorders. And this is by no means the only article making such statements; similar reviews arrive at similar conclusions. In turn, this means that the effects observed after SMT are not necessarily specific effects due to SMT but could easily be due to placebo or other non-specific effects. In order to avoid confusion, one would need a credible placebo – one that closely mimics SMT – and make sure that patients were ‘blinded’. But ‘light massage’ clearly does not mimic SMT, and patients obviously were aware of which interventions they received.
So, an alternative – and I think at least as plausible – conclusion of the data provided by this new RCT is this:
Chiropractic SMT is associated with a powerful placebo response which, of course, obeys a dose-effect relationship. Thus these findings are in keeping with the notion that SMT is a placebo.
And why would the researchers – who stress that they have no conflicts of interest – mislead us by making this alternative interpretation of their findings not abundantly clear?
I fear, the reason might be simple: they also seem to mislead us about their conflicts of interest: they are mostly chiropractors with a long track record of publishing promotional papers masquerading as research. What, I ask myself, could be a stronger conflict of interest?
(Pity that a high-impact journal like SPINE did not spot these [not so little] flaws)