Conventional cough syrups do not have the best of reputations – but the repute of homeopathic cough syrups is certainly not encouraging. So what should one do with such a preparation? Forget about it? No, one conducts a clinical trial, of course! Not just any old trial but one where science, ethics and common sense are absent. Here are the essentials of a truly innovative study that, I think, has all of these remarkable qualities:
The present prospective observational study investigated children affected by wet acute cough caused by non-complicated URTIs, comparing those who received the homeopathic syrup versus those treated with the homeopathic syrup plus antibiotic. The aims were: 1) to assess whether the addition of antibiotics to a symptomatic treatment had a role in reducing the severity and duration of acute cough in a pediatric population, as well as in improving cough resolution; 2) to verify the safety of the two treatments. Eighty-five children were enrolled in an open study: 46 children received homeopathic syrup alone for 10 days and 39 children received homeopathic syrup for 10 days plus oral antibiotic treatment (amoxicillin/clavulanate, clarithromycin, and erythromycin) for 7 days. To assess cough severity we used a subjective verbal category-descriptive (VCD) scale. Cough VCD score was significantly (P < 0.001) reduced in both groups starting from the second day of treatment (−0.52 ± 0.66 in the homeopathic syrup group and −0.56 ± 0.55 in children receiving homeopathic syrup plus oral antibiotic treatment). No significant differences in cough severity or resolution were found between the two groups of children in any of the 28 days of the study. After the first week (day 8) cough was completely resolved in more than one-half of patients in both groups. Two children (4.3 %) reported adverse effects in the group treated with the homeopathic syrup alone, versus 9 children (23.1 %) in the group treated with the homeopathic syrup plus antibiotics (P = 0.020).
Our data confirm that the homeopathic treatment in question has potential benefits for cough in children as well, and highlight the strong safety profile of this treatment. Additional antibiotic prescription was not associated with a greater cough reduction, and presented more adverse events than the homeopathic syrup alone.
Let us be clear about what has happened here. I think, the events can be summarised as follows:
- the researchers come across a homeopathic syrup (anyone who understands respiratory problems and/or therapeutics would be more than a little suspicious of this product, but this team is exceptional),
- they decide to do a trial with it (a decision which would make some ethicists already quite nervous, but the ethics committee is exceptional too),
- the question raises, what should the researchers give to the control group?
- someone has the idea, why not compare our dodgy syrup against something that is equally dodgy, perhaps even a bit unsafe?
- the researchers are impressed and ask: but what precisely could we use?
- let’s take antibiotics; they are often used for acute coughs, but the best evidence fails to show that they are helpful and they have, of course, risks,
- another member of the team adds: let’s use children, they and their mothers are unlikely to understand what we are up to,
- the team is in agreement,
- Boiron, the world’s largest producer of homeopathic products, accepts to finance the study,
- a protocol is written,
- ethics approval is obtained,
- the trial is conducted and even published by a journal with the help of peer-reviewers who are less than critical.
And the results of the trial? Contrary to the authors’ conclusion copied above, they show that two bogus treatments are worse that one.
BOB’S YOUR UNCLE!
EVERYONE SEEMS HAPPY: THE RESEARCHERS CAN ADD AN ARTICLE TO THEIR PUBLICATION LIST, BOIRON HAS MORE ‘EVIDENCE’ IN FAVOUR OF HOMEOPATHY, AND THE ETHICS COMMITTEE SLEEP JUST AS SOUNDLY AS THE PEER-REVIEWERS.
The press officers of journals like to send out press-releases of articles which are deemed to be particularly good and important. Sadly, it is not often that articles on alternative medicine fulfil these criteria. I was therefore excited to receive this press-release which seemed encouraging, to say the least:
Medical evidence supports the potential for acupuncture to be significantly more effective in the treatment of dermatologic conditions such as dermatitis, pruritus, and urticaria than alternative treatment options, “placebo acupuncture,” or no treatment, according to a review of the medical literature published in The Journal of Alternative and Complementary Medicine, a peer-reviewed publication from Mary Ann Liebert, Inc., publishers…
The abstract was equally promising:
Objectives: Acupuncture is a form of Traditional Chinese Medicine that has been used to treat a broad range of medical conditions, including dermatologic disorders. This systematic review aims to synthesize the evidence on the use of acupuncture as a primary treatment modality for dermatologic conditions.
Methods: A systematic search of MEDLINE, EMBASE, and the Cochrane Central Register was performed. Studies were limited to clinical trials, controlled studies, case reports, comparative studies, and systematic reviews published in the English language. Studies involving moxibustion, electroacupuncture, or blood-letting were excluded.
Results: Twenty-four studies met inclusion criteria. Among these, 16 were randomized controlled trials, 6 were prospective observational studies, and 2 were case reports. Acupuncture was used to treat atopic dermatitis, urticaria, pruritus, acne, chloasma, neurodermatitis, dermatitis herpetiformis, hyperhidrosis, human papillomavirus wart, breast inflammation, and facial elasticity. In 17 of 24 studies, acupuncture showed statistically significant improvements in outcome measurements compared with placebo acupuncture, alternative treatment options, and no intervention.
Conclusions: Acupuncture improves outcome measures in the treatment of dermatitis, chloasma, pruritus, urticaria, hyperhidrosis, and facial elasticity. Future studies should ideally be double-blinded and standardize the control intervention.
One has to read the actual full text article to understand that the evidence presented here is dodgy to the extreme. In fact, one has to go into the tedious details of the methods section to find the reasons why: All searches were limited to clinical trials, controlled studies, case reports, comparative studies, and systematic reviews published in the English language.
There are many more weaknesses of this review, but the inclusion of uncontrolled studies and even anecdotes is, in my view, a virtual death sentence to its credibility. It means that no general conclusions about the effectiveness of acupuncture, such as the authors have decided to make, are possible.
Such overt exaggerations are sadly no rarities in the realm of alternative medicine. I think, this begs a number of serious questions:
- Does this cross the line between flawed research and scientific misconduct?
- Why did the reviewers not pick up these flaws?
- Why did the editor pass this article for publication?
- How can the publisher tolerate such dubious behaviour?
- Should this journal (which I have commented on before here and which is one with the highest impact factor of all the alt med journals) be de-listed from Medline?
I don’t think that we will get answers from the people responsible for this disgrace, but I would like to learn my readers’ opinions.
Chiropractors are back pain specialists, they say. They do not pretend to treat non-spinal conditions, they claim.
If such notions were true, why are so many of them still misleading the public? Why do many chiropractors pretend to be primary care physicians who can take care of most illnesses regardless of any connection with the spine? Why do they continue to happily promote bogus treatments? Why do chiropractors, for instance, claim they can treat gastrointestinal diseases?
This recent narrative review of the literature, for example, was aimed at summarising studies describing the management of disorders of the gastrointestinal (GI) tract using ‘chiropractic therapy’ broadly defined here as spinal manipulation therapy, mobilizations, soft tissue therapy, modalities and stretches.
Twenty-one articles were found through searching the published literature to meet the authors’ inclusion criteria. The retrieved articles included case reports to clinical trials to review articles. The majority of articles chronicling patient experiences under chiropractic care reported that they experienced mild to moderate improvements in GI symptoms. No adverse effects were reported.
From this, the authors concluded that chiropractic care can be considered as an adjunctive therapy for patients with various GI conditions providing there are no co-morbidities.
I think, we would need to look for a long time to find an article with conclusions that are more ridiculous, false and unethical than these.
The old adage applies: rubbish in, rubbish out. If we include unreliable reports such as anecdotes, our finding will be unreliable as well. If we do not make this mistake and conduct a proper systematic review, we will arrive at very different conclusions. My own systematic review, for instance, of controlled clinical trials drew the following conclusion: There is no supportive evidence that chiropractic is an effective treatment for gastrointestinal disorders.
That probably says it all. I only want to add a short question: SHOULD THIS LATEST CHIROPRACTIC ATTEMPT TO MISLEAD THE PUBLIC BE CONSIDERED ‘SCIENTIFIC MISCONDUCT’ OR ‘FRAUD’?
The last time I had contact with Dr Fisher was when he fired me from the editorial board of his journal ‘Homeopathy’. He did that by sending me the following letter:
Dear Professor Ernst,
This is to inform you that you have been removed from the Editorial Board of Homeopathy. The reason for this is the statement you published on your blog on Holocaust Memorial Day 2013 in which you smeared homeopathy and other forms of complementary medicine with a ‘guilt by association’ argument, associating them with the Nazis.
I should declare a personal interest….[Fisher goes on to tell a story which is personal and which I therefore omit]… I mention this only because it highlights the absurdity of guilt by association arguments.
Peter Fisher Editor-in-Chief, Homeopathy
I did not expect to have any more dealings with him after this rather unpleasant encounter. But, as it turns out, I recently did have a further encounter.
When the BMJ invited me to write a debate article about the question whether homeopathy should continue to be available on the NHS, I accepted (with some reservations, I hasten to add). At the time, I did not know who would do the ‘other side’ of this debate. It turned out to be Peter Fisher, and our two articles have just been published.
As one would expect from a good journal, the articles were both peer reviewed. One of the peer-reviewers of my piece was most scathing of it essentially claiming that it was entirely worthless. Feeling that this was a bit harsh and very impolite, I was keen to see who this reviewer had been; it was none other than Andrew Vickers. This is remarkable because Vickers had not only published several homeopathic papers with Fisher, but also had been in the employment of the ‘Royal London Homeopathic Hospital’ under Fisher. To the best of my knowledge, his conflicts of interested had not been disclosed. I did point that out to the BMJ, but they seemed to think nothing of it.
Anyway, I was pleased to eventually (the whole procedure took many months) see the articles published, but at the same time somewhat irritated by Fisher’s piece. It contained plenty of misleading information that the peer-reviewers obviously had failed to correct. Here is a small sample from Fishers piece:
… recent overviews have had more favourable conclusions, including a health technology assessment commissioned by the Swiss federal government that concluded that homeopathy is “probably” effective for upper respiratory tract infections and allergies.
Readers interested in the clinical evidence can access the CORE-HOM database of clinical research in homeopathy free of charge (www.carstens-stiftung.de/core-hom). It includes 1117 clinical trials of homeopathy, of which about 300 are randomised controlled trials.
In the podcast that accompanies the articles Fisher insists that, on this database, there are well over 300 RCT, and I had to admit that this was new to me. Keen to learn more, I registered with the database and had a look. What I found startled me. True, the database does claim that almost 500 RCTs are available, but just a very superficial scrutiny of these studies reveals that
- some are not truly randomised,
- some are not even clinical trials,
- the list includes dual publications, re-analyses of already published studies as well as aborted trials,
- many have never been peer-reviewed,
- many are not double-blind,
- many are not placebo controlled,
- the majority are of poor methodological quality.
As to the other thing mentioned in the above excerpt from Fisher’s article, the famous ‘health technology assessment commissioned by the Swiss federal government’, I can refer my readers to a blog post by J W Nienhuys which probably says it all, if not, there is plenty more criticism of this report available on the Internet.
My conclusion from all this?
THE QUEEN’S HOMEOPATH USES ARGUMENTS THAT SEEM JUST AS BOGUS AS HOMEOPATHY ITSELF.
I thought I had seen everything that is lamentable about homeopathy. When I came across this article, I had to change my opinion. It is a more despicable, unethical and dangerous promotion of falsehoods than I could have imagined.
Strong words? Read for yourself:
There are treatments that can heal vaccine damage, but few physicians in the conventional medical care system know about them, since vaccine injuries are usually denied as the cause of any illness. Some parents with autistic children report that homeopathy has completely reversed their children’s autism and healed other serious health conditions caused by vaccines. This article explains how homeopathic remedies can bring about healing for many types of vaccine injuries.
Homeopathy is not the only treatment that has helped children and adults recover from vaccine damage, but it is the one that is the focus of this article. I will describe how homeopathy can bring about a true cure for the harm that vaccines have caused to children and adults…
It is a tragedy when a normal young child suddenly starts losing the ability to speak sentences or even to speak words after receiving vaccines. The ability to have positive social interactions with other children or adults can disappear in a matter of days after vaccines have been given to children. Intellectual development can be lost and even successful potty training skills can disappear. The ability to sit quietly, listen to a story being read, and the ability to learn can suddenly be replaced with hand flapping, body spinning, head banging, food allergies, asthma, agitation, hyperactivity, learning disabilities, chronic colds and fevers, constant stomach pain, constipation, and a general failure to grow and thrive. There are also serious consequences for adults who use vaccines. Formerly productive adults can lose their independence and become paralyzed, infertile, chronically ill, and even die, because of vaccine damage. It happens every day, yet few people make the connection between their illnesses and vaccine use…
By the time parents fully awaken to the harm that has occurred to their children, many have already resigned themselves to a lifetime of caretaking their disabled children. Some parents will even receive counsel from their physicians to give up their children to the care of the state, because they have no treatments to offer and can offer no hope of recovery. Some physicians will try to convince parents that this is a genetic problem that might be cured someday, but not in the near future. The conventional medical care system leaves parents feeling like helpless victims without any good options. The truth is there are good options for restoring health after vaccine damage, and homeopathy is one of them!…
Homeopathy does not wage war on disease and seek to destroy the symptoms of disease through brute force. It does not bring substances into the body as is done with allopathic drugs, for the purpose of doing hand to hand combat against disease. Instead, homeopathy and its remedies are intended to gently stimulate and strengthen the body so that it can overcome illness through its own vital force and strength. Homeopathic remedies restore the natural ability of the body to defend itself against illness and to heal itself. When this happens, a person is truly cured of what ails him…
Allopathic drugs and treatments do not have a positive effect upon the vital force in the body. They do not improve the strength of a person, and they do not provide for physical, emotional, or mental renewal. Rather, they just suppress symptoms, and add side effects…
You may also wish to ask for a referral from your chiropractor, osteopath, or acupuncturist. Such practitioners are often aware of good homeopaths in the area. Sometimes the person who is responsible for managing supplements and remedies sold at health food stores will be aware of experienced homeopaths as well…
I know, apologists will claim that such extreme idiocy is always the work of a few ‘rotten apples’, even most homeopaths would object to such dangerous and amoral lunacy. But the fact is, they don’t! If you disagree, please show me the protests from homeopaths or other alternative practitioners.
When Wakefield was shown to be a fraud endangering public health with his bogus claims about vaccine damage, there were protests in abundance, and he was ousted by the medical and scientific communities. Where are the protests by the alternative medicine fraternity against this article and the many, many others like it?
NOBODY SHOULD BE ALLOWED TO ENDANGER PUBLIC HEALTH IN THIS WAY.
In case you wonder who wrote the above article, it is John P. Thomas. He is a health writer for Health Impact News. He holds a B.A. in Psychology from the University of Michigan, and a Master of Science in Public Health (M.S.P.H.) from the School of Public Health, Department of Health Administration, at the University of North Carolina at Chapel Hill. John specializes in environmental health, but writes on a variety of issues.
Prince Charles’ meddling beyond his constitutional role is yet again in the press today. I was sent the 2nd batch of ‘spider memos’ yesterday, studied them and commented as follows:
The letters demonstrate yet again that Prince Charles relentlessly meddles in UK health politics and thus disrespects his constitutional role. His arguments in favour of CAM, and in particular homeopathy, show a devastating lack of knowledge and understanding; they are ill-informed, invalid and embarrassingly naïve – but at the same time they are remarkably persistent. Charles tries to give the impression that he is motivated by passion and compassion, but in health care such drivers need solid evidence and expertise. Charles has neither which is not just regrettable, it is arrogant on his part and potentially harmful for public health.
If you get the feeling that I have little patience with Charles’ meddling, you may be right. I have little doubt that it was his interfering that led to the closure of my research unit at Exeter. In my book, A SCIENTIST IN WONDERLAND, I provide the full details of what happened. Here I would just like to reproduce the incredible ‘spider memo’ in question.
It was not actually authored by Charles but by Sir Michael Peat, his 1st private secretary. On 22 September 2005, Peat wrote on Clarence House note-paper to the Vice Chancellor of Exeter University wrongly accusing me of a serious breach of confidentiality:
I am writing both as The Prince of Wales’ Private Secretary and as Acting Chairman of His Royal Highness’ Foundation for Integrated Health.
There has been a breach of confidence by Professor Edzard Ernst in respect of a draft report on the efficacy of certain complementary therapies sent to him by Mr. Christopher Smallwood. The report was commissioned by The Prince of Wales.
Mr. Smallwood sent Professor Ernst an early and, at that stage, incomplete draft of the report for comment. The accompanying e-mail requested and stressed the need for confidentiality. Professor Ernst implicitly agreed to comment on the report on this basis but then, as you probably saw, gave his views about the report to the national press. I attach a copy of a letter from the Editor of the Lancet published by The Times which summarises the issues well. I also attach a copy of the e-mail sent to Professor Ernst by Mr. Smallwood.
I apologise for troubling you, but I felt that you should have this matter drawn to your attention.
What followed was embarrassing and shameful: my uni started a 13 month investigation, eventually I was pronounced innocent but my unit was closed down. Unbelievably, Clarence house denied that Charles even knew about this amazing attempt to meddle in academic affairs. As I say, you need to read my memoir to understand this story fully.
The founder of Johrei Healing (JH), Mokichi Okada, believed that “all human beings have toxins in their physical bodies. Some are inherited, others are acquired by ingesting medicines, food additives, unnatural food, unclean air, most drugs, etc. all of these contain chemicals which cannot be used by the body and are treated as poisons…….. Illness is no more than the body’s way of purifying itself to regain health…… The more we resist illness by taking suppressive medications, the harder and more built up the toxins become…… If we do not allow the toxins to be eliminated from the body, we will suffer more, and have more difficult purification…..on the other hand, if we allow illness to take its course by letting the toxins become naturally eliminated from our bodies, we will be healthier.”
Johrei healers channel light or energy or warmth etc. into the patient’s or recipient’s body in order to stimulate well-being and healing. Sounds wacky? Yes!
Still, at one stage my team conducted research into all sorts of wacky healing practices (detailed reasons and study designs can be found in my recent book ‘A SCIENTIST IN WONDERLAND‘). Despite the wackiness, we even conducted a study of JH. Dr Michael Dixon, who was closely collaborating with us at the time, had persuaded me that it would be reasonable to do such a study. He brought some Japanese JH-gurus to my department to discuss the possibility, and (to my utter amazement) they were happy to pay £ 70 000 into the university’s research accounts for a small pilot study. I made sure that all the necessary ethical safe-guards were in place, and eventually we all agreed to design and conduct a study. Here is the abstract of the paper published once the results were available and written up.
“Johrei is a form of spiritual healing comprising “energy channelling” and light massage given either by a trained healer or, after some basic training, by anyone. This pilot trial aimed to identify any potential benefits of family-based Johrei practice in childhood eczema and for general health and to establish the feasibility of a subsequent randomised controlled trial. Volunteer families of 3-5 individuals, including at least one child with eczema were recruited to an uncontrolled pilot trial lasting 12 months. Parents were trained in Johrei healing and then practised at home with their family. Participants kept diaries and provided questionnaire data at baseline, 3,6 and 12 months. Eczema symptoms were scored at the same intervals. Scepticism about Johrei is presently an obstacle to recruitment and retention of a representative sample in a clinical trial, and to its potential use in general practice. The frequency and quality of practise at home by families may be insufficient to bring about the putative health benefits. Initial improvements in eczema symptoms and diary recorded illness, could not be separated from seasonal factors and other potential confounders. There were no improvements on other outcomes measuring general health and psychological wellbeing of family members.”
Our findings were hugely disappointing for the JH-gurus, of course, but we did insist on our right to publish them. Dr Dixon was not involved in the day to day running of our trial, nor in evaluating its results, nor in writing up the paper. He nevertheless showed a keen interest in the matter, kept in contact with the Japanese sponsors, and arranged regular meetings to discuss our progress. It was at one of those gatherings when he mentioned that he was about to fly to Japan to give a progress report to the JH organisation that had financed the study. My team felt this was odd (not least because, at this point, the study was far from finished) and we were slightly irritated by this interference.
When Dixon had returned from Japan, we asked him how the meeting had been. He said the JH sponsors had received him extremely well and had appreciated his presentation of our preliminary findings. As an ‘aside’, he mentioned something quite extraordinary: he, his wife and his three kids had all flown business class paid for by the sponsors of our trial. This, we all felt, was an overt abuse of potential research funds, unethical and totally out of line with academic behaviour. Recently, I found this fascinating clip on youtube, and I wonder whether it was filmed when Dr Dixon visited Japan on that occasion. One does get the impression that the Johrei organisation is not short of money.
A few months later, I duly reported this story to my dean, Prof Tooke, who was about to get involved with Dr Dixon in connection with a postgraduate course on integrated medicine for our medical school (more about this episode here or in my book). He agreed with me that such a thing was a most regrettable violation of academic and ethical standards. To my great surprise, he then asked me not to tell anybody about it. Today I feel very little loyalty to either of these two people and have therefore decided to publish my account – which, by the way, is fully documented as I have kept all relevant records and a detailed diary (in case anyone should feel like speaking to libel lawyers).
Homeopathy has many critics who claim that there is no good evidence for this type of therapy. Homeopaths invariably find this most unfair and point to a plethora of studies that show an effect. They are, of course, correct! There are plenty of trials that suggest that homeopathic remedies do work. The question, however, is HOW RELIABLE ARE THESE STUDIES?
Here is a brand new one which might stand for dozens of others.
In this study, homeopaths treated 50 multimorbid patients with homeopathic remedies identifies by a method called ‘polarity analysis’ (PA) and prospectively followed them over one year (PA enables homeopaths to calculate a relative healing probability, based on Boenninghausen’s grading of polar symptoms).
The 43 patients (86%) who completed the observation period experienced an average improvement of 91% in their initial symptoms. Six patients dropped out, and one did not achieve an improvement of 80%, and was therefore also counted as a treatment failure. The cost of homeopathic treatment was 41% of projected equivalent conventional treatment.
Good news then for enthusiasts of homeopathy? 91% improvement!
Yet, I am afraid that critics might not be bowled over. They might smell a whiff of selection bias, lament the lack of a control group or regret the absence of objective outcome measures. But I was prepared to go as far as stating that such results might be quite interesting… until I read the authors’ conclusions that is:
Polarity Analysis is an effective method for treating multimorbidity. The multitude of symptoms does not prevent the method from achieving good results. Homeopathy may be capable of taking over a considerable proportion of the treatment of multimorbid patients, at lower costs than conventional medicine.
Virtually nothing in these conclusions is based on the data provided. They are pure extrapolation and wild assumptions. Two questions seem to emerge from this:
- How on earth can we take this and so many other articles on homeopathy seriously?
- When does this sort of article cross the line between wishful thinking and scientific misconduct?
There must be well over 10 000 clinical trials of acupuncture; Medline lists ~5 000, and many more are hidden in the non-Medline listed literature. That should be good news! Sadly, it isn’t.
It should mean that we now have a pretty good idea for what conditions acupuncture is effective and for which illnesses it does not work. But we don’t! Sceptics say it works for nothing, while acupuncturists claim it is a panacea. The main reason for this continued controversy is that the quality of the vast majority of these 10 000 studies is not just poor, it is lousy.
“Where is the evidence for this outraging statement???” – I hear the acupuncture-enthusiasts shout. Well, how about my own experience as editor-in-chief of FACT? No? Far too anecdotal?
How about looking at Cochrane reviews then; they are considered to be the most independent and reliable evidence in existence? There are many such reviews (most, if not all [co-]authored by acupuncturists) and they all agree that the scientific rigor of the primary studies is fairly awful. Here are the crucial bits of just the last three; feel free to look for more:
Or how about providing an example? Good idea! Here is a new trial which could stand for numerous others:
This study was performed to compare the efficacy of acupuncture versus corticosteroid injection for the treatment of Quervain’s tendosynovitis (no, you do not need to look up what condition this is for understanding this post). Thirty patients were treated in two groups. The acupuncture group received 5 acupuncture sessions of 30 minutes duration. The injection group received one methylprednisolone acetate injection in the first dorsal compartment of the wrist. The degree of disability and pain was evaluated by using the Quick Disabilities of the Arm, Shoulder, and Hand (Q-DASH) scale and the Visual Analogue Scale (VAS) at baseline and at 2 weeks and 6 weeks after the start of treatment. The baseline means of the Q-DASH and the VAS scores were 62.8 and 6.9, respectively. At the last follow-up, the mean Q-DASH scores were 9.8 versus 6.2 in the acupuncture and injection groups, respectively, and the mean VAS scores were 2 versus 1.2. Thus there were short-term improvements of pain and function in both groups.
The authors drew the following conclusions: Although the success rate was somewhat higher with corticosteroid injection, acupuncture can be considered as an alternative option for treatment of De Quervain’s tenosynovitis.
The flaws of this study are exemplary and numerous:
- This should have been a study that compares two treatments – the technical term is ‘equivalence trial – and such studies need to be much larger to produce a meaningful result. Small sample sizes in equivalent trials will always make the two treatments look similarly effective, even if one is a pure placebo.
- There is no gold standard treatment for this condition. This means that a comparative trial makes no sense at all. In such a situation, one ought to conduct a placebo-controlled trial.
- There was no blinding of patients; therefore their expectation might have distorted the results.
- The acupuncture group received more treatments than the injection group; therefore the additional attention might have distorted the findings.
- Even if the results were entirely correct, one cannot conclude from them that acupuncture was effective; the notion that it was similarly ineffective as the injections is just as warranted.
These are just some of the most fatal flaws of this study. The sad thing is that similar criticisms can be made for most of the 10 000 trials of acupuncture. But the point here is not to nit-pick nor to quack-bust. My point is a different and more serious one: fatally flawed research is not just a ‘poor show’, it is unethical because it is a waste of scarce resources and, even more importantly, an abuse of patients for meaningless pseudo-science. All it does is it misleads the public into believing that acupuncture might be good for this or that condition and consequently make wrong therapeutic decisions.
In acupuncture (and indeed in most alternative medicine) research, the problem is so extremely wide-spread that it is high time to do something about it. Journal editors, peer-reviewers, ethics committees, universities, funding agencies and all others concerned with such research have to work together so that such flagrant abuse is stopped once and for all.
When someone has completed a scientific project, it is customary to publish it [‘unpublished science is no science’, someone once told me many years ago]. To do so, he needs to write it up and submit it to a scientific journal. The editor of this journal will then submit it to a process called ‘peer review’.
What does ‘peer review’ entail? Well, it means that 2-3 experts are asked to critically assess the paper in question, make suggestions as to how it can be improved and submit a recommendation as to whether or not the article deserves to be published.
Peer review has many pitfalls but, so far, nobody has come up with a solution that is convincingly better. Many scientists are under pressure to publish [‘publish or perish’], and therefore some people resort to cheating. A most spectacular case of fraudulent peer review has been reported recently in this press release:
London, UK (08 July 2014) – SAGE announces the retraction of 60 articles implicated in a peer review and citation ring at the Journal of Vibration and Control (JVC). The full extent of the peer review ring has been uncovered following a 14 month SAGE-led investigation, and centres on the strongly suspected misconduct of Peter Chen, formerly of National Pingtung University of Education, Taiwan (NPUE) and possibly other authors at this institution.
In 2013 the then Editor-in-Chief of JVC, Professor Ali H. Nayfeh,and SAGE became aware of a potential peer review ring involving assumed and fabricated identities used to manipulate the online submission system SAGE Track powered by ScholarOne Manuscripts™. Immediate action was taken to prevent JVC from being exploited further, and a complex investigation throughout 2013 and 2014 was undertaken with the full cooperation of Professor Nayfeh and subsequently NPUE.
In total 60 articles have been retracted from JVC after evidence led to at least one author or reviewer being implicated in the peer review ring. Now that the investigation is complete, and the authors have been notified of the findings, we are in a position to make this statement.
While investigating the JVC papers submitted and reviewed by Peter Chen, it was discovered that the author had created various aliases on SAGE Track, providing different email addresses to set up more than one account. Consequently, SAGE scrutinised further the co-authors of and reviewers selected for Peter Chen’s papers, these names appeared to form part of a peer review ring. The investigation also revealed that on at least one occasion, the author Peter Chen reviewed his own paper under one of the aliases he had created.
Unbelievable? Perhaps, but sadly it is true; some scientists seem to be criminally ingenious when it comes to getting their dodgy articles into peer-reviewed journals.
And what does this have to do with ALTERNATIVE MEDICINE, you may well ask. The Journal of Vibration and Control is not even medical and certainly would never consider publishing articles on alternative medicine. Such papers go to one of the many [I estimate more that 1000] journals that cover either alternative medicine in general or any of the modalities that fall under this wide umbrella. Most of these journals, of course, pride themselves with being peer-reviewed – and, at least nominally, that is correct.
I have been on the editorial board of most of the more important journals in alternative medicine, and I cannot help thinking that their peer review process is not all that dissimilar from the fraudulent scheme set up by Peter Chen and disclosed above. What happens in alternative medicine is roughly as follows:
- a researcher submits a paper for publication,
- the editor sends it out for peer review,
- the peer reviewers are either those suggested by the original author or members of the editorial board of the journal,
- in either case, the reviewers are more than likely to be uncritical and recommend publication,
- in the end, peer review turns out to be a farcical window dressing exercise with no consequence,
- thus even very poor research and pseudo-research are being published abundantly.
The editorial boards of journals of alternative medicine tend to be devoid of experts who are critical about the subject at hand. If you think that I am exaggerating, have a look at the editorial board members of ‘HOMEOPATHY’ (or any other journal of alternative medicine) and tell me who might qualify as a critic of homeopathy. When the editor, Peter Fisher, recently fired me from his board because he felt I had tarnished the image of homeopathy, this panel lost the only person who understood the subject matter and, at the same time, was critical about it (the fact that the website still lists me as an editorial board member is merely a reflection of how slow things are in the world of homeopathy: Fisher fired me more than a year ago).
The point I am trying to make is simple: peer review is never a perfect method but when it is set up to be deliberately uncritical, it cannot possibly fulfil its function to prevent the publication of dodgy research. In this case, the quality of the science will be inadequate and generate false-positive messages that mislead the public.