Yesterday, I wrote about a new acupuncture trial. Amongst other things, I wanted to find out whether the author who had previously insisted I answer his questions about my view on the new NICE guideline would himself answer a few questions when asked politely. To remind you, this is what I wrote:
This new study was designed as a randomized, sham-controlled trial of acupuncture for persistent allergic rhinitis in adults investigated possible modulation of mucosal immune responses. A total of 151 individuals were randomized into real and sham acupuncture groups (who received twice-weekly treatments for 8 weeks) and a no acupuncture group. Various cytokines, neurotrophins, proinflammatory neuropeptides, and immunoglobulins were measured in saliva or plasma from baseline to 4-week follow-up.
Statistically significant reduction in allergen specific IgE for house dust mite was seen only in the real acupuncture group. A mean (SE) statistically significant down-regulation was also seen in pro-inflammatory neuropeptide substance P (SP) 18 to 24 hours after the first treatment. No significant changes were seen in the other neuropeptides, neurotrophins, or cytokines tested. Nasal obstruction, nasal itch, sneezing, runny nose, eye itch, and unrefreshed sleep improved significantly in the real acupuncture group (post-nasal drip and sinus pain did not) and continued to improve up to 4-week follow-up.
The authors concluded that acupuncture modulated mucosal immune response in the upper airway in adults with persistent allergic rhinitis. This modulation appears to be associated with down-regulation of allergen specific IgE for house dust mite, which this study is the first to report. Improvements in nasal itch, eye itch, and sneezing after acupuncture are suggestive of down-regulation of transient receptor potential vanilloid 1.
…Anyway, the trial itself raises a number of questions – unfortunately I have no access to the full paper – which I will post here in the hope that my acupuncture friend, who are clearly impressed by this paper, might provide the answers in the comments section below:
- Which was the primary outcome measure of this trial?
- What was the power of the study, and how was it calculated?
- For which outcome measures was the power calculated?
- How were the subjective endpoints quantified?
- Were validated instruments used for the subjective endpoints?
- What type of sham was used?
- Are the reported results the findings of comparisons between verum and sham, or verum and no acupuncture, or intra-group changes in the verum group?
- Was the success of patient-blinding checked, quantified and successful?
- What other treatments did each group of patients receive?
- Does anyone really think that this trial shows that “acupuncture is a safe, effective and cost-effective treatment for allergic rhinitis”?
In the comments section, the author wrote: “after you have read the full text and answered most of your questions for yourself, it might then be a more appropriate time to engage in any meaningful discussion, if that is in fact your intent”, and I asked him to send me his paper. As he does not seem to have the intention to do so, I will answer the questions myself and encourage everyone to have a close look at the full paper [which I can supply on request].
- The myriad of lab tests were defined as primary outcome measures.
- Two sentences are offered, but they do not allow me to reconstruct how this was done.
- No details are provided.
- Most were quantified with a 3 point scale.
- Mostly not.
- Needle insertion at non-acupoints.
- The results are a mixture of inter- and intra-group differences.
- Patient blinding was checked but no quantitative results of the success of blinding are reported. Crucially, McDonald did all the treatments himself, also the sham treatments.
- Patients were allowed to use conventional treatments and the frequency of this use was reported in patient diaries.
- I don’t think so.
So, here is my interpretation of this study:
- It lacked power for many outcome measures, certainly the clinical ones.
- There were hardly any differences between the real and the sham acupuncture group.
- Most of the relevant results were based on intra-group changes, rather than comparing sham with real acupuncture, a fact, which is obfuscated in the abstract.
- In a controlled trial fluctuations within one group must never be interpreted as caused by the treatment.
- There were dozens of tests for statistical significance, and there seems to be no correction for multiple testing.
- Thus the few significant results that emerged when comparing sham with real acupuncture might easily be false positives.
- Patient-blinding seems questionable.
- McDonald as the only therapist of the study might be suspected to have influenced his patients through verbal and non-verbal communications.
I am sure there are many more flaws, particularly in the stats, and I leave it to others to identify them. The ones I found are, however, already serious enough, in my view, to call for a withdrawal of this paper. Essentially, the authors seem to have presented a study with largely negative findings as a trial with positive results showing that acupuncture is an effective therapy for allergic rhinitis. Subsequently, McDonald went on social media to inflate his findings even more. One might easily ask: is this scientific misconduct?
I would be most interested to hear what you think about it [if you want to see the full article, please send me an email].
“Conflicts of interest should always be disclosed.”
This is what I wrote in the ‘RULES’ of this blog when I first started it almost 4 years ago. Sadly, very few people writing comments observe this rule. Perhaps, I just thought, I did not observe it either? So, here are my conflicts of interest: none.
Not true!!! I hear some people say. But it is!
I have no financial interest in any ‘Big Pharma’ or ‘TINY CAM’, and I get not a penny for writing this blog.
How do I pay for my living? Mind your own business… well, on second thought, even that must not be a secret: I get a small pension and have some savings.
Still not convinced?
Perhaps it’s time to define what ‘conflicts of interests’ are. According to Wikipedia, they can be defined as situations in which a person or organization is involved in multiple interests, financial interest, or otherwise, one of which could possibly corrupt the motivation of the individual or organization.
So, not having financial benefits from my current work does not necessarily mean that I have no conflicts of interest. The above definitions vaguely mentions ‘or otherwise’ – and that could be important. What could this mean in the context of this blog?
Well, I might have very strong beliefs, for instance (for instance, very strong beliefs that acupuncture is by definition nonsense [see below]). We all know that strong beliefs can corrupt motivation (and a lot more). And if I ask myself, do you have strong beliefs?, I have to say: Yes, absolutely!
I believe that:
- good evidence is a prerequisite for progress in healthcare,
- good evidence must be established by rigorous research,
- we should not tolerate double standards in healthcare,
- patients deserve to be treated with the best available treatments,
- making therapeutic claims that are not supported by sound evidence is wrong.
These strong beliefs might make me biased in the eyes of many who comment on this blog. In Particular, we recently had a bunch of acupuncturists who went on the rampage attacking me personally the best they could. However, a rational analysis of my beliefs can hardly produce evidence for bias against anything other than the promotion of unproven therapies to the unsuspecting public.
The above mentioned acupuncturists seem to think that I have always been against acupuncture for the sake of being against acupuncture. However, this is not true. The proof for this statement is very simple: I have published quite a bit of articles that concluded positively – even (WOULD YOU BELIEVE IT?) about acupuncture for back pain! A prominently published meta-analysis of 2005 (with me as senior author) concluded: “Acupuncture effectively relieves chronic low back pain.” (This of course was 11 years ago when the evidence was, in fact, positive; today, this seems to have changed – just like the NICE guidelines [probably not a coincidence!])
Conflicts of interest? No, not on my side, I think.
But what about the ‘other side’?
The unruly horde of acupuncturists (no, this is not an ad hominem attack, it’s a fact) who recently made dozens of ad hominem attacks against me, what about them?
- They earn their money with acupuncture.
- They have invested in acupuncture training often for long periods of time.
- They have invested in practice equipment etc.
- Some of them sell books on acupuncture.
- Others run courses.
- And all of them very clearly and demonstrably have strong beliefs about acupuncture.
I think the latter point constitutes by far the most important conflict of interest in this context.
And this is where the somewhat trivial story has an unexpected twist and gets truly bizarre:
I have just leant that the same group of conflicted acupuncturists are now planning to publicly attack the panel of experts responsible for drafting the NICE guidelines. The reason? They feel that this panel had significant conflicts of interest that led them to come out against acupuncture.
Perhaps I should mention that I was not a member of this group, but I suspect that some of its members might have links to the pharmaceutical industry. It is almost impossible to find top experts in any area of medicine who do not have such links. You either gather experts with potential conflicts of interest, or you get non-experts without them. Would that bias them against acupuncture or any other alternative therapy? I very much doubt it.
What I do not doubt for a minute is that conflicts of interest are of major importance in these discussions. And by that I mean the more than obvious (but nevertheless undeclared) conflicts of interest of the acupuncturists. It seems that those with the strongest conflicts of interest shout the loudest about the non-existent or irrelevant conflict of interest of those who do not happen to share their quasi-religious belief in acupuncture.
On this blog, I have repeatedly tried to explain why integrative (or integrated) medicine is such a deceptive nonsense; see for instance here, here and here. Today, I have reason to make another attempt: The International Congress on Integrative Medicine & Health.
In 2012, I published an analysis of the ‘3rd European Congress of Integrated Medicine’ which had taken place in December 2010 in Berlin (in Europe they call it ‘integrated’ and in the US ‘integrative’ medicine). For this purpose, I simply read all the 222 abstracts and labelled them according to their contents. The results showed that the vast majority were on unproven alternative therapies and none on conventional treatments.
The abstracts from the International Congress on Integrative Medicine & Health (ICIMH, Green Valley Ranch Resort, Las Vegas, Nevada, USA, May 17–20, 2016) which were just published provide me with the opportunity to check whether this situation has changed. There were around 400 abstracts, and I did essentially the same type of analysis (attributing one subject area to each abstract). And what a tedious task this was! I spotted just two articles of interest, and will report about them shortly.
This time I also assessed whether the conclusions of each paper were positive (expressing something favourable about the subject at hand), negative (expressing something negative about the subject at hand) or neither of the two (surveys, for instance, rarely show positive or negative results).
Here are the results: mind-body therapies were the top subject with 49 papers, followed by acupuncture (44), herbal medicine (37), integrative medicine (36), chiropractic and other manual therapies (26), TCM (19), methodological issues (16), animal and other pre-clinical investigations (15) and Tai Chi (5). The rest of the abstracts were on a diverse array of other subjects. There was not a single paper on a conventional therapy and only 4 focussed on risk assessments.
The 36 articles on integrative medicine deserve perhaps a special mention. The majority of these papers were about using alternative therapies as an add-on to conventional care. They focussed on the alternative therapies used and usually concluded that this ‘integration’ was followed by good results. None of these papers discussed integrative medicine and its assumptions critically, and none of these investigations cast any doubt about the assumption that integrative medicine is a positive thing.
I should also mention that my attributions of the subject areas were not always straight forward. I allowed myself only one subject per paper, but there were, of course, many that could be categorised in more than one subject area ( for instance, a paper on an herbal medicine might be in that category, or in TCM or in pre-clinical). So I tried to attribute the subject that seemed to dominate the abstract in question.
My analysis according to the direction of the conclusions was equally revealing: I categorised 260 papers as positive, 5 as negative and 116 as neither of the two. That means for every negative result there were 52 positive ones. I find this most remarkable.
Essentially, my two analyses of conference abstracts published 6 years apart show the same phenomenon: on the ‘scientific level’, integrative medicine is not about the ‘best of both worlds’ (i. e. the best alternative medicine has to offer integrated with the best conventional medicine offers) – the slogan by which advocates of integrative medicine usually try to ‘sell’ their dubious approach to us. It is almost exclusively about alternative therapies which advocates of integrative medicine aim to smuggle into mainstream healthcare. Critical analysis seems to be unwelcome in this area, and – perhaps worse of all – in the last 6 years, there does not seem to have been any improvement.
And that’s just on the ‘scientific level’, as I said. If you wonder what is happening on the ‘practical level’, you will find that, in the realm of integrative medicine, every quackery under the sun is being promoted at often exorbitant prices to the often gullible and always unsuspecting public. If you don’t believe me, search for ‘integrative medicine clinic’ on the Internet; I promise, you will be surprised!
Personally, I am sometimes amused by the sheer idiocy of all this, but more often I am enraged and ask myself:
- Why are we allowing quackery to make such a spectacular come-back?
- Why is hardly anyone voicing strong objections?
- Is it not our ethical duty to do something about it and try to prevent the worse?
It has long been argued that chiropractic spinal manipulations are mere placebo interventions. Yet few controlled trials have assessed the efficacy of spinal manipulative therapy (SMT). No high quality trials have been performed to test the efficacy and effectiveness of Graston Technique® (GT), an instrument-assisted soft tissue therapy.
The objective of this trial was to determine the efficacy of SMT and GT compared to sham therapy for the treatment of non-specific thoracic spine pain.
People with non-specific thoracic pain were randomly allocated to one of three groups: SMT, GT, or a placebo (de-tuned ultrasound). GT is a popular soft-tissue technique in the United States and becoming more popular in other developed countries. GT is an instrument-assisted soft-tissue therapy involving the use of hand-held stainless steel instruments. The promoters of the GT claim that the instruments resonate in the clinician’s hands allowing the clinician to isolate soft-tissue “adhesions and restrictions”, and treat them precisely. Each participant received up to 10 supervised treatment sessions at Murdoch University chiropractic student clinic over a 4 week period.
The two outcome measures were self-administered instruments. Participants were given blank questionnaires in a package by a research assistant following their first treatment. Participants were instructed to complete the instruments at each assessment time point. After completion of the forms the participant posted them back to the Murdoch University Chiropractic Clinic. Research assistants remained blind to the outcome data for the entire study period. The participants and treatment providers were not blinded to the treatment allocation as it was clear that the groups were receiving different treatments. Participants in the placebo group were blinded to their placebo allocation until follow-up was complete at 12 months. Participants were surveyed for the adequacy of the placebo blinding at the end of the study.
Treatment outcomes were measured at baseline, 1 week, and at one, three, six and 12 months. Primary outcome measures included a modified Oswestry Disability Index, and the Visual Analogue Scale (VAS). Treatment effects were estimated with intention to treat analysis and linear mixed models.
One hundred and forty three participants were randomly allocated to the three groups (SMT = 36, GT = 63 and Placebo = 44). Baseline data for the three groups did not show any meaningful differences. Results of the intention to treat analyses revealed no time by group interactions, indicating no statistically significant between-group differences in pain or disability at 1 week, 1 month, 3 months, 6 months, or 12 months. There were significant main effects of time (p < 0.01) indicating improvements in pain and disability from baseline among all participants regardless of intervention. No significant adverse events were reported.
The authors concluded that this study indicates that there is no difference in outcome at any time point for pain or disability when comparing SMT, Graston Technique® or sham therapy for thoracic spine pain, however all groups improved with time. These results constitute the first from a fully powered randomised controlled trial comparing SMT, Graston technique® and a placebo.
Some people claim that there is little wrong with placebo therapy, as long as it helps patients. This is not what I think, but even the proponents of this argument would agree that the placebo used in this way has to be safe. As SMT is by no means free of adverse effects, the argument cannot be applied here.
Other people will argue that this is about SMT and not chiropractic implying that I am conducting a vendetta against the poor chiropractors. I would disagree: we have just learnt that 93% of chiropractors consider SMT as their primary treatment. Yes, osteopaths and physiotherapists also use SMT but certainly not to this extent. Thus this discussion is mostly about chiropractic, and the onus is on chiropractors to demonstrate beyond doubt that SMT does more good than harm.
The true significance of this study is, I think, that the chiropractic profession now must convince us that spinal manipulation has any usefulness at all. They will have to conduct rigorous trials along the lines of this study to test for which condition these interventions generate outcomes that are significantly better than those achievable by sham.
Until such data are available, it would be wise, I think, to consider all therapeutic claims made for chiropractic unproven and bogus.
WATCH THIS SPACE!
Ad hominem attacks, I have previously pointed out, are victories of reason over unreason. And they are used frequently by supporters of alternative medicine!
If you doubt it, see for yourself.
I recently posted a comment on new Nice guidelines. It generated lots of comments, and mostly they were rational discussions of the issues involved. This changed abruptly when, on 16 May, Mel’s comment started a new, concerted wave of discussion at a time when the usual debate had already subsided. In the course of this new and heated debate, I was repeatedly accused of being rude.
As I have stated repeatedly on this blog, I try to keep rudeness out of the comments as much as I can. Therefore, the claim surprised me and today I reviewed the entire comment section selecting all potential ad hominem attacks. Here are the results:
ACTUAL OR POTENTIAL AD HOMINEM ATTACKS AGAINST ME
Peter Deadman on Tuesday 17 May 2016 at 12:55 Edward Ernst, I always thought you were a bully and a fraud. You’re very macho when it comes to slapping down people who may have experiential reasons for supporting acupuncture and other therapies but don’t have the skill to challenge you on the clinical evidence. Now as soon as somebody does, you back off, cry ‘enough’, say you can’t possibly comment till some undetermined future date and generally act like a wuss. I say put up or shut up. I’d prefer the former because it would be good to see you eat crow but I lean towards the latter because of the substantial harm you are causing and the beautiful silence that would ensue if you did indeed go quiet.
tonto on Tuesday 17 May 2016 at 13:19 You appear as weak in your arguments, as some pendulum swinging, new age dowser, who vainly holds sticks to their guns, not because they can back their position up with scientific evidence, but because it is what they “believe”.
Jill Onyett on Tuesday 17 May 2016 at 14:29 …an unfortunate creature too keen on the sound of his own voice.
Tracey Phillips on Tuesday 17 May 2016 at 13:16 …to date you have been fairly opinionated …
Peter Deadman on Tuesday 17 May 2016 at 16:34 I made an ad hominem response because your blog is all about you as a person. You are constantly rude to others and bypass or ignore responses that you don’t like. It’s you who makes it hominem.
Peter Deadman on Tuesday 17 May 2016 at 16:52 You are hyper-emotional, extremely biased, hostile and contemptuous of anyone you think ‘beneath you’. You gloat over people’s real or imagined inconsistencies and generally come across as a nasty piece of work.
Peter Deadman on Tuesday 17 May 2016 at 19:30 How can such a childish provocateur remain in his post. It demeans the University and it’s time they let him go.
Kylee Junghans on Wednesday 18 May 2016 at 08:42 …you, kind Sir, with your rhetoric and tantrums, are exhibiting a prime example of confirmation bias.
Peter Deadman on Wednesday 18 May 2016 at 08:48 [Ernst] professes a scholarly detachment, a commitment to evidence and an open mind, but in fact is deeply biased… He clearly loves his childish provocative stance and is as far from a disinterested observer as it’s possible to be. I wouldn’t waste my time or breath on him if he didn’t have an influence that far exceeds his worth.
Carol Cooke on Wednesday 18 May 2016 at 09:27 I have followed this discussion with interest. Some of the rudest and most discourteous posts I can see are from Mr Ernst himself. But I get that, I imagine you seek to maintain a bold and authoritative tone simply by dismissing others. Being a bit controversial in your discourse has obviously served you well in that you have built a media profile on it.
ACTUAL OR POTENTIAL AD HOMINEM ATTACKS BY MYSELF
Edzard on Wednesday 18 May 2016 at 09:18 “it is also difficult to get a man to read something, when he is foaming from his mouth”.
I know, this is not really ‘ad hominem’ but I could not find anything more dramatic. Surely, some will disagree this me here, and I do invite them to cite my rudeness from this threat, if they spot it. You are more than welcome!
You may think this is a bit trivial, but I disagree. The main reason I did this little exercise is to demonstrate a point which I think is important and carries a relevant lesson for future comments and discussions:
- WHEN I OR ANYONE ELSE DEFENDING RATIONALITY GET AGGRESSED, WE NATURALLY TEND TO RESPOND SLIGHTLY MORE FORCEFULLY.
- SUBSEQUENTLY, THE OTHER SIDE OFTEN REACTS BY ATTACKING US PERSONALLY.
- THIS OFTEN LEADS TO AN ESCALATION OF TONE.
- EVENTUALLY THE OTHER SIDE CLAIMS WITH INDIGNATION THAT WE ARE THE ONES DOING THE PERSONAL ATTACKS.
- IT IS A TACTIC THAT IS EFFECTIVE BUT DISHONEST, IN MY VIEW.
- THE LESSON IS SIMPLE: DO NOT LET YOURSELF GET PROVOKED INTO ISSUING AD HOMINEM ATTACKS, BE POLITE AND PATIENT.
I know this sounds simpler than it is, and I am far from being immune to the problem, but we owe it to reason to give it a try.
Anyone who really wants to get an insight into the ‘homeopathic mind-set’ should read the regular newsletter ‘HOMEOPATHY 4 EVERYONE’. Its current issue is focussed on cardiology. An article on coronary heart disease, a condition that kills about 40% of the population, informs us how homeopaths tackle this killer-disease:
If anything permanent is to be accomplished by treatment, a most careful examination of the individual case must be made. Not the attack alone, but the habits of the patient, his family history and environments must all be studied in every possible light. In the management, each case must be considered separately and the causes that excite an attack sought after. Many of these patients already have recognized the cause in their own case and often it is some irregularity of diet, exercise or mental condition. Many times it is not an easy matter to control the mental state, as the worry and strain of business life presses upon many of these patients, and is responsible for many cases of arterial degeneration that give rise to apoplexy, Bright ‘s disease, aneurysm or angina pectoris. The age and occupation of the patient, and the condition of the vascular system should be taken into consideration.
Following an attack the condition of the heart may require absolute rest, from a day to a week or more; this is especially true if the attacks are precipitated by a slight degree of exercise, which shows that the heart is not able to propel the blood under anything but normal conditions. Under no condition should quick movements and strong emotions be associated. Steady quiet exercise as walking upon level ground is beneficial. If the cardiac weakness is such as to forbid this, massage, or the resistance exercise of the Schott’s method may be tried. This exercise should not follow immediately after a meal.
But this is not all. There are plenty more papers on life-threatening cardiac conditions. Take the article on pericarditis for instance. This is how homeopaths are told how to treat this medical emergency:
Remedies that may be indicated are as follows: If traumatic, Arnica. For the inflammatory outset, Aconite or Vera- trum viride. The anguish of Aconite distinguishes its inflammation from that attending the stupor of Veratrum. For the pain Bryonia or Spigelia. They may be indicated in this order, Bryonia for the first stage and Spigelia for the subsequent myalgia. In these cases there may be met with indications for Belladonna (its flushed face), Arsenicum (dyspnoea on lying down), Digitalis (its weak pulse), Cactus (severe myalgia) or Kali carb (stitching pains). General symptoms may call for Colchicum, Aesculus, Kali iod., Cimicifuga, Kahnia, Squilla
A further article tackles diseases of the blood vessels. The article on thrombosis informs the homeopath that
Thrombosis is a blocking of the local circulation either spontaneously, after injuries or from slow and imperfect circulation forming a clot. In thrombosis the part becomes pale and edematous. The remedies are Aconite for first stage. Hamamelis, Lachesis or Lycopodium may be indicated. If suppuration threatens Sulphur or Hepar. Rest and a supporting diet.
The same article also tells us how to treat aneurysms:
Select the remedy carefully. Lycopodium 12 has cured aneurism of the carotid (Hughes). If the attack is due to a sudden strain or injury, Arnica; if from fear or fright, Aconite; if from syphilis, Mercurius, Kali hydr. or Nitric acid; if from alcoholism, Arsenicum or Nux vomica; if from fatty degeneration, Phosphorus; if from fibrous inflammation and degeneration, Bryonia; if there is great arterial excitement and delirium, Veratrum viride; if circulation sluggish, Digitalis. Secale has cured aneurism. Consult Carbo veg., Spigelia. See Heart Therapeutics.
After reading the entire issue, I was not sure whether this wasn’t a hoax. Are we supposed to laugh or to cry? Personally I did giggle a lot while reading this. But if I imagine for a minute that some homeopaths might take this seriously, I am not far from crying.
While over on my post about the new NICE GUIDELINES on acupuncture for back pain, the acupuncturists’ assassination attempts of my character, competence, integrity and personality are in full swing, I have decided to employ my time more fruitfully and briefly comment on a new piece of acupuncture research.
This new Italian study was to determine the effectiveness of acupuncture for the management of hot flashes in women with breast cancer.
A total of 190 women with breast cancer were randomly assigned to two groups. Random assignment was performed with stratification for hormonal therapy; the allocation ratio was 1:1. Both groups received a booklet with information about climacteric syndrome and its management to be followed for at least 12 weeks. In addition, the acupuncture group received 10 traditional acupuncture treatment sessions involving needling of predefined acupoints.
The primary outcome was hot flash score at the end of treatment (week 12), calculated as the frequency multiplied by the average severity of hot flashes. The secondary outcomes were climacteric symptoms and quality of life, measured by the Greene Climacteric and Menopause Quality of Life scales. Health outcomes were measured for up to 6 months after treatment. Expectation and satisfaction of treatment effect and safety were also evaluated. We used intention-to-treat analyses.
Of the participants, 105 were randomly assigned to enhanced self-care and 85 to acupuncture plus enhanced self-care. Acupuncture plus enhanced self-care was associated with a significantly lower hot flash score than enhanced self-care at the end of treatment (P < .001) and at 3- and 6-month post-treatment follow-up visits (P = .0028 and .001, respectively). Acupuncture was also associated with fewer climacteric symptoms and higher quality of life in the vasomotor, physical, and psychosocial dimensions (P < .05).
The authors concluded that acupuncture in association with enhanced self-care is an effective integrative intervention for managing hot flashes and improving quality of life in women with breast cancer.
This hardly needs a comment, as I have been going on about this study design many times before: the ‘A+B versus B’ design can only produce positive findings. Any such study concluding that ‘acupuncture (or whatever other intervention) is effective’ can therefore not be a legitimate test of a hypothesis and ought to be categorised as pseudo-science. Sadly, this problem seems more the rule than the exception in the realm of acupuncture research. That’s a pity really… because, if there is potential in acupuncture at all, this sort of thing can only distract from it.
I think the JOURNAL OF CLINICAL ONCOLOGY, its editors and reviewers, should be ashamed of having published such misleading rubbish.
Polycystic ovarian syndrome (PCOS) is a common condition characterised by oligo-amenorrhoea, infertility and hirsutism. Conventional treatment of PCOS includes a range of oral pharmacological agents, lifestyle changes and surgical modalities. Some studies have suggested that acupuncture might be helpful but the evidence is often flawed and the results are mixed. What is needed in such a situation is, of course, a systematic review.
The aim of this new Cochrane review was to assess the effectiveness and safety of acupuncture treatment of oligo/anovulatory women with polycystic ovarian syndrome (PCOS). The authors identified relevant studies from databases including the Cochrane Central Register of Controlled Trials (CENTRAL), Ovid MEDLINE, EMBASE, PsycINFO, CNKI and trial registries. The data are current to 19 October 2015.
They included randomised controlled trials (RCTs) that studied the efficacy of acupuncture treatment for oligo/anovulatory women with PCOS. We excluded quasi- or pseudo-RCTs. Primary outcomes were live birth and ovulation (primary outcomes), and secondary outcomes were clinical pregnancy, restoration of menstruation, multiple pregnancy, miscarriage and adverse events. We assessed the quality of the evidence using GRADE methods.
Two review authors independently selected the studies, extracted data and assessed risk of bias. They calculated Mantel-Haenszel odds ratios (ORs) and mean difference (MD) and 95% confidence intervals (CIs).
Five RCTs with 413 women were included. They compared true acupuncture versus sham acupuncture (two RCTs), true acupuncture versus relaxation (one RCT), true acupuncture versus clomiphene (one RCT) and electroacupuncture versus physical exercise (one RCT). Four of the studies were at high risk of bias in at least one domain. No study reported live birth rate. Two studies reported clinical pregnancy and found no evidence of a difference between true acupuncture and sham acupuncture (OR 2.72, 95% CI 0.69 to 10.77, two RCTs, 191 women, very low quality evidence). Three studies reported ovulation. One RCT reported number of women who had three ovulations during three months of treatment but not ovulation rate. One RCT found no evidence of a difference in mean ovulation rate between true and sham acupuncture (MD -0.03, 95% CI -0.14 to 0.08, one RCT, 84 women, very low quality evidence). However, one other RCT reported very low quality evidence to suggest that true acupuncture might be associated with higher ovulation frequency than relaxation (MD 0.35, 95% CI 0.14 to 0.56, one RCT, 28 women). Two studies reported menstrual frequency. One RCT reported true acupuncture reduced days between menstruation more than sham acupuncture (MD 220.35, 95% CI 252.85 to 187.85, 146 women). One RCT reported electroacupuncture increased menstrual frequency more than no intervention (0.37, 95% CI 0.21 to 0.53, 31 women). There was no evidence of a difference between the groups in adverse events. Evidence was very low quality with very wide CIs and very low event rates. Overall evidence was low or very low quality. The main limitations were failure to report important clinical outcomes and very serious imprecision.
The authors concluded that, thus far, only a limited number of RCTs have been reported. At present, there is insufficient evidence to support the use of acupuncture for treatment of ovulation disorders in women with PCOS.
This is, in my view, a rigorous assessment of the evidence leading to a clear conclusion. Foremost, I applaud the authors from the Faculty of Science, University of Technology Sydney for using such clear language. Such clarity seems to be getting a rare event in reviews of alternative medicine. To demonstrated this point, here are the most recent 5 systematic reviews which came up on my screen when I searched today Medline for ‘complementary alternative medicine, systematic review’.
The combination of TGP and LEF in treatment of RA presented the characteristics of notably decreasing the levels of laboratory indexes and higher safety in terms of liver function. However, this conclusion should be further investigated based on a larger sample size.
CHM as an adjunctive therapy is associated with a decreased risk of in-hospital mortality compared with WT in patients with AKI. Further studies with high quality and large sample size are needed to verify our conclusions.
As an important supplementary treatment, TCM may provide benefits in repair of injured spinal cord. With a general consensus that future clinical approaches will be diversified and a combination of multiple strategies, TCM is likely to attract greater attention in SCI treatment.
I think the phenomenon is fairly obvious: authors of such papers are far too often not able or willing to express the bottom line of their work openly. As systematic reviews are supposed to be the ultimate type of evidence, this trend is very worrying, I think. In my view, such conclusions merely display the bias of the authors. If the evidence is not convincingly positive (which it very rarely is), authors have an ethical obligation to clearly say so.
If they don’t do it, journal editors have the duty to correct the error. If neither of these actions happen, funding agencies should make sure that such teams get no further research money until they can demonstrate that they have learnt the lesson.
This may sound a bit drastic but I think such steps would be both necessary and urgent. The problem is now extremely common, and if we do not quickly implement some effective preventative measures, our scientific literature will become contaminated to the point of becoming useless. This surely would be a disaster that affects us all.
There can, of course, be several reasons for the evidence being not positive:
- there can be a paucity of data
- the results might be contradictory
- the trials might be open to bias
- some of the primary data might look suspicious
In all of these cases, the evidence would be not convincingly positive, and it would be wrong and unhelpful not to be frank about it. Beating about the bush, like so many authors nowadays do, is misleading, unhelpful, unethical and borderline fraudulent. Therefore it constitutes a disservice to everyone concerned.
In a previous post, I asked this important question: how can research into alternative medicine ever save a single life?
The answer I suggested was as follows:
Since about 20 years, I am regularly pointing out that the most important research questions in my field relate to the risks of alternative medicine. I have continually published articles about these issues in the medical literature and, more recently, I have also made a conscious effort to step out of the ivory towers of academia and started writing for a much wider lay-audience (hence also this blog). Important landmarks on this journey include:
Alternative medicine is cleverly, heavily and incessantly promoted as being natural and hence harmless. Several of my previous posts and the ensuing discussions on this blog strongly suggest that some chiropractors deny that their neck manipulations can cause a stroke. Similarly, some homeopaths are convinced that they can do no harm; some acupuncturists insist that their needles are entirely safe; some herbalists think that their medicines are risk-free, etc. All of them tend to agree that the risks are non-existent or so small that they are dwarfed by those of conventional medicine, thus ignoring that the potential risks of any treatment must be seen in relation to their proven benefit.
For 20 years, I have tried my best to dispel these dangerous myths and fallacies. In doing so, I had to fight many tough battles (sometimes even with the people who should have protected me, e.g. my peers at Exeter university), and I have the scars to prove it. If, however, I did save just one life by conducting my research into the risks of alternative medicine and by writing about it, the effort was well worth it.
END OF QUOTE FROM MY PREVIOUS POST
Just now, I received an email from someone who clearly and vehemently disagrees with any of the above. As this blog is a forum where all sorts of opinions can and should be voiced, I thought I share this communication with you. Here it is:
Having been out of chiropractic practice for a while, I was thrilled to hear that you have been forced into early retirement on today’s Radio 4 programme. You have caused so many good people anguish and pain and your tunnel-visioned arrogance is staggering and detrimental to humanity. You REALLY think modern science has all the answers? Wow.
The question I ask myself is who is correct, the (ex-)chiropractor or I?
- Have I caused anguish and pain to many?
- Do I suffer from tunnel-vision?
- Am I arrogant?
- Is my work detrimental to humanity?
- Do I believe that modern science has all the answers?
Here is what I think about these specific questions:
- I have probably caused anguish (but no pain, as far as I am aware). This sadly is unavoidable if one seeks the truth in an area as alternative medicine.
- I am not the best person to judge this.
- Possibly; again I cannot judge.
- I truly don’t see this at all.
- No, not for one second.
In case you wonder what programme the author of the above email had been listening to, you can find it here.
Is there a bottom line? I am not sure. Perhaps this: whenever strong believes clash with scientific facts, some people are going to be unhappy. If we want to make progress, this seems to be almost unavoidable; all we can try to do is to minimize the anguish by being humble and by showing human decency.
In 2010, we published an investigation of 200 chiropractor websites and 9 chiropractic associations’ World Wide Web claims in Australia, Canada, New Zealand, the United Kingdom, and the United States. The outcome measure was claims (either direct or indirect) regarding the eight reviewed conditions, made in the context of chiropractic treatment.
We found evidence that 190 (95%) chiropractor websites made unsubstantiated claims regarding at least one of the conditions. When colic and infant colic data were collapsed into one heading, there was evidence that 76 (38%) chiropractor websites made unsubstantiated claims about all the conditions not supported by sound evidence. Fifty-six (28%) websites and 4 of the 9 (44%) associations made claims about lower back pain, whereas 179 (90%) websites and all 9 associations made unsubstantiated claims about headache/migraine. Unsubstantiated claims were made about asthma, ear infection/earache/otitis media, neck pain.
At the time, we concluded that the majority of chiropractors and their associations in the English-speaking world seem to make therapeutic claims that are not supported by sound evidence, whilst only 28% of chiropractor websites promote lower back pain, which is supported by some evidence. We suggest the ubiquity of the unsubstantiated claims constitutes an ethical and public health issue.
Have things changed since?
I fear not! I regularly come across websites of chiropractors where they happily make bogus claims. On this website, for instance, chiropractor Karen Smith claims that muscles in the upper neck affect the ear canals. “We don’t actually treat the ear infection, or the symptoms. What we do is, we assist the body’s natural healing ability,” says Smith. “So if there’s something going on with the joints and the muscles soft tissue, the nerves coming out that supply those muscles, those muscles can’t relax, so then they’re almost tight and in spasm, so that can’t allow the drainage to happen properly.”
When fluid builds up in the ears, it’s a breeding ground for bacteria and infection. Smith says specific, gentle adjustments, can help the body drain those fluids through the nose. “What we do is we get some motion in the upper neck, with my hands, or I might use an instrument as well,” says Smith. “There’s a few other techniques that we can do. We can do some sinus drainage. We can drain some of the fluid in the ear.”
A simple ear pull technique can also help. “So what we do is, we just take the ear of the child and we do a little pull and that can actually drain the fluid as well,” says Smith. Smith says a child’s overall health and immune system impacts how quickly they see results from the treatment. In some cases, relief can be instant. “What we notice right after an adjustment is a lot of times you’ll actually see the fluid drain through the nose,” says Smith… Smith says she also treats adults who have had chronic ear issues as a child or who are experiencing pain in the ear.
When I or others expose such nonsense, the apologists say that these are just a few ‘rotten apples’, and that the chiropractic profession is fast progressing. Yet, I very much doubt this claim. For any fast progression, one would want to see the profession taking decisive and effective action against the ‘rotten apples’. This is clearly not happening, at least not to an extend that would stop such dangerous quackery.
What practical lesson can be learnt from such insights?
The only responsible advice I can think of is this: IF YOU OR YOUR CHILD IS ILL, AVOID CONSULTING A CHIROPRACTOR.