The recent meta-analysis by Mathie et al for non-individualised homeopathy (recently discussed here) identified just 3 RCTs that were rated as ‘reliable evidence’. But just how rigorous are these ‘best’ studies? Let’s find out!
THE FIRST STUDY
The objective of the first trial was “to evaluate the efficacy of the non-hormonal treatment BRN-01 in reducing hot flashes in menopausal women.” Its design was that of a multicentre (35 centres in France), randomized, double-blind, placebo-controlled. One hundred and eight menopausal women, ≥50 years of age, were enrolled in the study. The eligibility criteria included menopause for <24 months and ≥5 hot flashes per day with a significant negative effect on the women’s professional and/or personal life. Treatment was either BRN-01 tablets, a registered homeopathic medicine [not registered in the UK] containing Actaea racemosa (4 centesimal dilutions [4CH]), Arnica montana (4CH), Glonoinum (4CH), Lachesis mutus (5CH), and Sanguinaria canadensis (4CH), or placebo tablets, prepared by Laboratoires Boiron according to European Pharmacopoeia standards [available OTC in France]. Oral treatment (2 to 4 tablets per day) was started on day 3 after study enrolment and was continued for 12 weeks. The main outcome measure was the hot flash score (HFS) compared before, during, and after treatment. Secondary outcome criteria were the quality of life (QoL) [measured using the Hot Flash Related Daily Interference Scale (HFRDIS)], severity of symptoms (measured using the Menopause Rating Scale), evolution of the mean dosage, and compliance. All adverse events (AEs) were recorded. One hundred and one women were included in the final analysis (intent-to-treat population: BRN-01, n = 50; placebo, n = 51). The global HFS over the 12 weeks, assessed as the area under the curve (AUC) adjusted for baseline values, was significantly lower in the BRN-01 group than in the placebo group (mean ± SD 88.2 ± 6.5 versus 107.2 ± 6.4; p = 0.0411). BRN-01 was well tolerated; the frequency of AEs was similar in the two treatment groups, and no serious AEs were attributable to BRN-01. The authors concluded that BRN-01 seemed to have a significant effect on the HFS, compared with placebo. According to the results of this clinical trial, BRN-01 may be considered a new therapeutic option with a safe profile for hot flashes in menopausal women who do not want or are not able to take hormone replacement therapy or other recognized treatments for this indication.
Laboratoires Boiron provided BRN-01, its matching placebo, and financial support for the study. Randomization and allocation were carried out centrally by Laboratoires Boiron. I would argue that the treatment time in this study was way too short for generating a therapeutic response. The evolution of the HFS in the two groups was assessed by analysis of the area under the curve (AUC) of the mean scores recorded weekly from each patient in each group over the duration of the study, including those at enrollment (before any treatment). I wonder whether this method was chosen only when the researchers noted that the HFS at the pre-defined time points did not yield a significant result or whether it was pre-determined (elsewhere in the methods section we are told that “The primary evaluation criterion was the effect of BRN-01 on the HFS, compared with placebo. The HFS was defined as the product of the daily frequency and intensity of all hot flashes experienced by the patient, graded by the women from 1 to 4 (1 = mild; 2 = moderate; 3 = strong; 4 = very strong). These data were recorded by the women on a self-administered questionnaire, assisted by a telephone call from a clinical research associate. Data were collected (i) during the first 2 days after enrolment and before any medication had been taken; (ii) then every Tuesday and Wednesday of each week until the 11th week of treatment, inclusive; and (iii) finally, every day of the 12th week of treatment.”). Two of the authors of this paper are employees of Boiron.
THE SECOND STUDY
The second trial was aimed at finding out “whether a well-known and frequently prescribed homeopathic preparation could mitigate post-operative pain.” It was a randomized, double-blind, placebo-controlled trial to evaluate the efficacy of the homeopathic preparation Traumeel S® in minimizing post-operative pain and analgesic consumption following surgical correction of hallux valgus. Eighty consecutive patients were randomized to receive either Traumeel tablets or an indistinguishable placebo, and took primary and rescue oral analgesics as needed. Maximum numerical pain scores at rest and consumption of oral analgesics were recorded on day of surgery and for 13 days following surgery. Traumeel was not found superior to placebo in minimizing pain or analgesic consumption over the 14 days of the trial, however a transient reduction in the daily maximum post-operative pain score favoring the Traumeel arm was observed on the day of surgery, a finding supported by a treatment-time interaction test (p = 0.04). The authors concluded that Traumeel was not superior to placebo in minimizing pain or analgesic consumption over the 14 days of the trial. A transient reduction in the daily maximum post-operative pain score on the day of surgery is of questionable clinical importance.
Traumeel is a mixture of 6 ingredients, 4 of which are in the D2 potency. Thus it neither is administered as a homeopathic remedy (no ‘like cures like’) nor is it highly diluted. In fact, it is not homeopathy at all but belongs to a weird offspring of homeopathy called ‘homotoxicology’ [this is an explanation from my book: Homotoxicology is a method inspired by homeopathy which was developed by Hans Heinrich Reckeweg (1905 – 1985). He believed that all or most illness is caused by an overload of toxins in the body. The toxins originate, according to Reckeweg, both from the environment and from the malfunction of physiological processes within the body. His treatment consists mainly in applying homeopathic remedies which usually consist of combinations of single remedies, because health cannot be achieved without ridding the body of toxins. The largest manufacturer and promoter of remedies used in homotoxicology is the German firm Heel.] The HEEL Company (Baden-Baden, Germany) provided funding for the performance and monitoring of this project, supplied the study medication and placebo, and prepared the randomization list. The positive outcome mentioned in the authors’ conclusion refers to a secondary endpoint. I would argue that the authors should not have noted it there and should have made it clear that the trial generated a negative result.
THE THIRD STUDY
Finally, the third of the 3 ‘rigorous’ studies “evaluated the effectiveness of the homeopathic preparation Plumbum Metallicum (PM) in reducing the blood lead levels of workers exposed to this metal.” The Brazilian researchers recruited 131 workers to this RCT who took PM in the CH15 potency or placebo for 35 days (10 drops twice daily). Thereafter, the percentage of workers whose lead level had fallen by at least 25% did not differ between the groups, both on intention to treat and per protocol analyses. The authors concluded that PM “had no effect in this study in terms of reducing serum lead in workers exposed to lead.”
This study lacks a power calculation, and arguably the period might have been too short to show an effect. The trial was published in the journal HOMEOPATHY which, some might argue, has not the most rigorous of peer-review procedures.
The third study seems the most rigorous by far, in my view. The other two trials are seriously under-whelming in several respects, primarily because we cannot be sure how much influence the commercial interests of the sponsor had on their findings. I am sure others will spot weaknesses in all three trials that I failed to see.
Mathie et al partly disagree with my assessment when they write in their paper: “We report separately our model validity assessments of these trials, evaluating consequently their overall quality based on a GRADE-like principle of ‘downgrading’ : two trials [23, 25] rated here as reliable evidence were downgraded to ‘low quality’ overall due to the inadequacy of their model validity; the remaining trial with reliable evidence  was judged to have adequate model validity. The latter study  thus comprises the sole RCT that can be designated ‘high quality’ overall by our approach, a stark finding that reveals further important aspects of the preponderantly low quality of the current body of evidence in non-individualised homeopathy.”
What Mathie et al seem to forget entirely is that none of the 3 RCTs is a trial of homeopathy as defined by treatment according to the ‘like cures like’ principle. The authors of the second study acknowledge this fact by stating: “Homeopathic purists may find fault in the administration of a standardized combination homeopathic formula to all patients, based upon clinical diagnosis – as opposed to the individualized manner dictated by standard homeopathic practice.”
So, which ever way we look upon this evidence, we cannot possibly deny that the evidence for non-individualised homeopathy is rubbish.
‘What Doctors Don’t Tell You’ (WDDTY) have been shown to be strangely economical with the truth many times before (for instance here, here and here). Now they have published an article entitled ‘Ombudsman investigates ‘flawed’ homeopathic study that claimed it doesn’t work’ It attacks in no uncertain terms the ‘NHMRC Statement on Homeopathy and NHMRC Information Paper – Evidence on the effectiveness of homeopathy for treating health conditions’ which I believe to be a sound evaluation of homeopathy and therefore have mentioned repeatedly on this blog. Here is what WDDTY stated:
START OF QUOTE
A major and influential review of homeopathy concluded that the controversial therapy doesn’t work—but it was so riddled with error and bad science that it’s sparked an official ombudsman investigation.
The world’s media announced that homeopathy was a scam after the Australia government’s National Health and Medical Research Council (NHMRC) published its findings in 2015 that “there are no health conditions for which there is reliable evidence that homeopathy is effective.”
But now the Commonwealth Ombudsman is investigating the review’s procedures after receiving reports of inaccuracies, mishandling of evidence and conflicts of interest.
The review has been triggered by the Australian Homeopathic Association (AHA), supported by the Homeopathic Research Institute (HRI), which began questioning the review’s processes after several solid studies that demonstrated homeopathy’s benefits had been overlooked.
The NHMRC review team set arbitrary parameters that only studies that involved more than 150 people—and which met standards that even drug trials rarely achieve—would be considered. Those requirements reduced the number of qualifying studies to just five—from an initial pool of more than 1,800 trials—and none of these showed that homeopathy was effective.
One of the NHMRC’s own reviewers produced a mysterious first report that has never been published, and hasn’t been released despite Freedom of Information requests.
And the AHA has discovered that Prof Peter Brooks, chair of the NHMRC committee that carried out the homeopathy review, never declared that he was a member of the anti-homeopathy lobby group, Friends of Science in Medicine.
There are solid studies that demonstrate homeopathy is effective against childhood diarrhea, sinusitis and hay fever—but they all involve fewer than 150 people, said HRI chief executive Rachel Roberts. “The public has a right to know that there are high quality studies showing homeopathy works for some medical conditions—information that was lost only due to NHMRC’s mishandling of the evidence.”
The homeopaths aren’t alone in challenging the NHMRC review: Australia’s independent Cochrane Centre said its conclusions are not an accurate reflection of the evidence, and a second expert also said he felt “uncertain of the definitive nature of the report’s conclusions.”
END OF QUOTE
As it happens, I am in contact with the lead author of this report, Paul Glasziou, not least because he very kindly wrote the foreword for my book HOMEOPATHY, THE UNDILUTED FACTS. So, we corresponded and discussed the latest WDDTY diatribe. Thus I am now in a position to put a few things straight (I hope Paul does not mind).
ISSUE 1. – The NHMRC review team set arbitrary parameters that only studies involving more than 150 people—and which met standards that even drug trials rarely achieve—would be considered.
The truth is that report focused on systematic reviews of trials, not individual trials. The 57 included systematic reviews found 176 individual trials which covered 61 conditions: an average of about 3 trials per condition. But some conditions only had 1 trial, and one small trial would, of course, not be considered a reasonable basis for reliable conclusions. GRADE – the international standard for assessing evidence – downgrades reviews for “imprecision” – the GRADE Handbook suggests “whenever there are sample sizes that are less than 400, review authors and guideline developers should certainly consider rating down for imprecision.” Hence the criterion of 150 which the Australians decided to use is considerably more lenient than the current GRADE guideline.
ISSUE 2 – Those requirements reduced the number of qualifying studies to just five—from an initial pool of more than 1,800 trials—and none of these showed that homeopathy was effective.
This is simply not correct. The report found 57 systematic reviews that contained 176 individual trials, not 5. These 176 trials, which covered 61 conditions, formed the body of evidence for the NHMRC report’s conclusions.
ISSUE 3 – There are solid studies that demonstrate homeopathy is effective against childhood diarrhoea, sinusitis and hay fever—but they all involve fewer than 150 people, said HRI chief executive Rachel Roberts.
The NHMRC report focused on systematic reviews that covered all trials for individual conditions. Given the conventional p-value of 0.05, one would expect 1 in 20 single trials to be “false positives”. So with 176 trials, we expect about 9 “false positive” trials. But using systematic reviews that combine all trials for individual conditions, reduces this risk of false positives. Most national evidence review bodies require more than 1 trial, e.g, the FDA requires 2 positive trials, whereas many others require a systematic review which has at least 2 trials. Replication of findings is obviously a cornerstone of science.
ISSUE 4 The homeopaths aren’t alone in challenging the NHMRC review: Australia’s independent Cochrane Centre said its conclusions are not an accurate reflection of the evidence, and a second expert also said he felt “uncertain of the definitive nature of the report’s conclusions.”
The truth is that the Cochrane Centre, which provided an independent check during the processes of the NHMRC review, concluded that “Overall, the conclusions arising from the review appear justified based on the evidence presented.”
I REST MY CASE.
Tui Na is a massage technique that is based on the Taoist principles of TCM. It involves a range of manipulations usually performed by an operator’s finger, hand, elbow, knee, or foot applied to muscle or soft tissue at specific parts of the body. According to one website of TCM-proponents “Tui Na makes use of various hand techniques in combination with acupuncture and other manipulation techniques. To enhance the healing process, the practitioner may recommend the use of Chinese herbs. Many of the techniques used in this massage resemble that of a western massage like gliding, kneading, vibration, tapping, friction, pulling, rolling, pressing and shaking. In Tui Na massage, the muscles and tendons are massaged with the help of hands, and an acupressure technique is applied to directly affect the flow of Qi at different acupressure points of the body, thus facilitating the healing process. It removes the blockages and keeps the energy moving through the meridians as well as the muscles. A typical session of Tui Na massage may vary from thirty minutes to an hour. The session timings may vary depending on the patient’s needs and condition. The best part of the therapy is that it relaxes as well as energizes the person. The main benefit of Tui Na massage is that it focuses on the specific problem, whether it is an acute or a chronic pain associated with the joints, muscles or a skeletal system. This technique is very beneficial in reducing the pain of neck, shoulders, hips, back, arms, highs, legs and ankle disorders. It is a very effective therapy for arthritis, pain, sciatica and muscle spasms. Other benefits of this massage therapy include alleviation of the stress related disorders like insomnia, constipation, headaches and other disorders related to digestive, respiratory and reproductive systems. The greatest advantage of Tui Na is that it focuses on maintaining overall balance with both physical and mental health. Any one who wants to avoid the side effects of drugs or a chemical based treatment can adopt this effective massage technique to alleviate their pain. Tui Na massage therapy is now becoming a more common therapy method due to its focus on specific problems rather than providing a general treatment.”
This clearly begs the question IS IT EFFECTIVE?
This systematic review assessed the evidence of Tui Na for cervical radiculopathy. Seven databases were searched. Randomised controlled trials (RCTs) incorporating Tui Na alone or Tui Na combined with conventional treatment were included. Five studies involving 448 patients were found. The pooled analysis from the 3 trials indicated that Tui Na alone showed a significant lowering immediate effects on pain score with moderate heterogeneity compared to cervical traction. The meta-analysis from 2 trials revealed significant immediate effects of Tui Na plus cervical traction in improving pain score with no heterogeneity compared to cervical traction alone. None of the RCTs mentioned adverse effects. There was very low quality or low quality evidence to support the results.
The authors concluded that “Tui Na alone or Tui Na plus cervical traction may be helpful to cervical radiculopathy patients, but supportive evidence seems generally weak. Future clinical studies with low risk of bias and adequate follow-up design are recommended.”
In my view, this is a misleading conclusion. A correct one would have been: THE CURRENT EVIDENCE IS INSUFFICIENT TO DRAW ANY CONCLUSIONS ABOUT THE EFFECTIVENESS OF TUI NA.
Here are some of the most obvious reasons:
- there are far too few studies for a firm conclusion,
- the included RCTs lack scientific rigour,
- all trials originate from China where reliability seems to be a serious problem,
- traction is not a useful therapy for radiculopathy,
- the primary studies violate research ethics by not reporting adverse effects.
Personally, I am getting very tired of conclusions stating ‘…XY MAY BE EFFECTIVE/HELPFUL/USEFUL/WORTH A TRY…’ It is obvious that the therapy in question MAY be effective, otherwise one would surely not conduct a systematic review. If a review fails to produce good evidence, it is the authors’ ethical, moral and scientific obligation to state this clearly. If they don’t, they simply misuse science for promotion and mislead the public. Strictly speaking, this amounts to scientific misconduct.
Yes, homeopaths are incredibly fond of the notion that homeopathy has been proven to work in numerous population studies of outbreaks of infectious diseases. The argument is bound to come up in any discussion with a ‘well-informed’ homeopathy fan. Therefore, it might be worth addressing it once and for all.
This website offers a fairly good summary of what homeopaths consider to be convincing evidence. It also provides links to the original articles which is valuable for all who want to study them in full detail. I will therefore present the crucial passage here unchanged.
START OF QUOTE
By the end of year 2014, there have been 19 papers published on Epidemiological studies on 7 epidemic diseases (scarlet fever, typhus fever, Cholera, Dengue, meningococcal, influenza and Leptospirosis) in 11 peer-reviewed (beyond year 1893) journals in evidence of Homeopathy including 2 Randomised Controlled Trials.
1. Samuel Hahnemann, “The Cure and prevention of scarlet fever”, Zeitschrift für Praktischen Medizin (Journal of Practical Medicine), 1801, Republished in Lesser Writings. B.Jain Publishing, New Delhi
Preventive use of homeopathy was first applied in 1799 during an epidemic of scarlet fever in Königslütter, Germany, when Dr. Hahnemann prescribed a single dose of Belladonna, as the remedy of the genus epidemicus to susceptible children in the town with more than 95% success rate. In this paper, he also specified how the Belladonna has to be potentised to 1/24,000,000 dilution. His recommended dose of Belladonna was 0.0416 nanograms to be repeated every 72 hrs. This is the first recorded nano dose of medicine used in treatment of any disease . It was another 125 years before Gladys Henry and George Frederick developed a vaccine for scarlet fever in 1924.
2. Samuel Hahnemann, “Scarlet fever and Purpura miliaris, two different diseases”, Zeitschrift für Praktischen Medizin, vol. 24, part. 1, 1806
3. Samuel Hahnemann, “Observations on scarlet fever”, Allgemeine Reichanzeiger (General Reich Gazette), No. 160, Germany, 1808
4. Samuel Hahnemann, “Reply to a question about the prophylactic for scarlet fever”, Zeitschrift für Praktischen Medizin, vol. 27, part. 4, p. 152-156, 1808
5. Samuel Hahnemann, “Treatment of typhus & fever at present prevailing”, Allgemeine Reichanzeiger, No. 6, Jan. 1814.
6. Hufeland, Prophylactic powers of Belladonna against Scarlet Fever , The Lancet, 1829
The proper use of belladonna has, in most cases, prevented infection. Numerous observations have shown that, by the general use of belladonna, epidemics of scarlet fever have actually been arrested. In those few instances where the use of belladonna was insufficient to prevent infection, the disease has been invariably slight. The Prussian (German Empire) Government ordered the use of the prophylactic during all scarlet fever epidemics
7. Samuel Hahnemann, “Cure and prevention of Asiatic cholera”, Archiv für die homöopathische Heilkunst (Archives for the Homoeopathic Healing Art), Vol. 11, part 1, 1831.
Cuprum 30c once every week as preventive medicine
8. Samuel Hahnemann, “On the contagiousness of cholera”. British Homoeopathic Journal, Vol. 7, 1849
9. Samuel Hahnemann, “Appeal to Thinking Philanthropists Respecting the Mode of Propagation of the Asiatic Cholera”, 20 pages, 1831. Republished in British Homoeopathic Journal, Oct 1849.
He said, “On board ships – in those confined spaces, filled with mouldy watery vapours, the cholera-miasm finds a favourable element for its multiplication, and grows into an enormously increased brood of those excessively minute, invisible, living creatures, so inimical to human life, of which the contagious matter of the cholera most probably consists millions of those miasmatic animated beings, which, at first developed on the broad marshy banks or the tepid Ganges– on board these ships, I say, this concentrated aggravated miasm kills several of the crew …” .
It was another 59 years (1890) before Koch saw these organisms, and later on orthodox medicine gave them the name ‘germs’
10. Charles Woodhull Eaton, The Facts about Variolinum, Transactions of the American Institute of Homoeopathy, 1907
2806 patients were treated prophylactically with Variolinum 30 (a nosode) for prevention of smallpox in Iowa. Of the 547 patients definitely exposed, only 14 developed the disease. Efficacy rate of 97.5%
11. Taylor Smith A, Poliomyelitis and prophylaxis British Homoeopathic Journal, 1950
In 1950 during an epidemic of poliomyelitis, Dr Taylor Smith of Johannesburg, South Africa protected 82 people with homoeopathic Lathyrus sativus. Of the 82 so immunised, 12 came into direct contact with disease. None were infected.
12. Oscillococcinum 200c in the treatment of influenza during epidemic in France from 1984-1987, British Journal of Clinical Pharmacology (1989)
A DBRPCT, Oscillococcinum 200c taken twice daily for 5 days significantly increased the rate of cure within two days (n=487, 237 treated and 241 on placebo), absence of symptoms at 48 hours, relative risk estimate significantly favour homeopathy (p=0.048), no pain and no fever (p=0.048), recovery rate (headache, stiffness, articular pain, shivering reduction) at 48 hours better in homeopathy group (p=0.032)
13. Bernard Leary, Cholera 1854 Update, British Homoeopathic Journal, 1994
Sir William Wilde, the well-known allopathic doctor of Dublin, which in his work entitled “Austria and its Institutions”, wrote: “Upon comparing the report of the treatment of Cholera in the Homeopathic hospital testified to by two allopathic medical inspectors appointed by Government with that of the treatment of the same disease in the other hospitals of Vienna during the same period the epidemic of 1836, it appeared that while two-thirds of the cases treated by Dr. Fleischmann the physician of the Homeopathic hospital, recovered, two-thirds of those treated by the ordinary methods in the other hospitals died.”
14. Meningococcinum – its protective effect against meningococcal disease, Homeopathy Links, 2001 (2001)
A total of 65,826 people between the ages of 0–20 were immunised homeopathically to protect against meningococcal disease while 23,532 were not. Over a year period, 4 out of 65,826 protected homeopathically developed meningococcal infection. 20 out of 23,532 not protected developed meningococcal infection. Based on the infection rate in the unprotected group, 58 cases of infection could have been expected in the homeopathically protected group. Instead, there were only four cases of meningococcal infection. Statistical analysis showed that homeopathic immunisation offered 95% protection in the first six months and 91% protection over the year against meningococcal disease. 
15. Contribution of homeopathy to the control of an outbreak of dengue epidemic in Macaé, Rio de Janeiro, Brazil in 2007-8 , International Journal of High Dilution Research, 2008
In a campaign ‘Homeopathy campaign against dengue’ by Brazilian Govt, “156,000 doses of homeopathic remedy were freely distributed in April and May 2007 to asymptomatic patients and 129 doses to symptomatic patients treated in outpatient clinics, according to the notion of genus epidemicus . The remedy used was a homeopathic complex against dengue containing Phosphorus 30c, Crotalus horridus 30c and Eupatorium perfoliatum 30c. The incidence of the disease in the first three months of 2008 fell 93% by comparison to the corresponding period in 2007, whereas in the rest of the State of Rio de Janeiro there was an increase of 128%.”
16. Marino R. Eupatorium perfoliatum 30c for the Dengue Epidemics in Brazil in 2007. International Journal of High Dilution Research, 2008
In May 2001, prophylactic use of Eupatorium perfoliatum 30c single dose was given during a dengue outbreak to 40% of residents in the most highly affected neighbourhood which resulted in significant decrease in dengue incidence by 81.5% (p<0.0001) when compared with those neighbourhoods that did not receive homeopathic prophylaxis.
17. Bracho et. al. Application of 200C potency of bacteria for Leptospirosis epidemic control in Cuba 2007-8 (2010)
Conducted by the Finlay Institute, a vaccines producer in Cuba gave 2.308562 million (70% of the target population above the age of 1 year) people in Cuba given two doses (1 dose=5 drops) of 200C potency of a nosode prepared from Leptospirosis bacteria, each (7-9 days apart), for protection against Leptospirosis (fever+jaundice+ inflammation in kidney+enlargement of spleen) with 84% decrease in disease incidence and only 10 reported cases. Dramatic decrease in morbidity within two weeks and zero morbidity of hospitalised patients, non-treated (8.8 millions) area saw an increase in number of cases from 309 cases in 2007 to 376 in 2008 representing a 21% increase. The cost of homeopathic immunization =1/15th of conventional vaccine.
18. Effect of individualized homoeopathic treatment in influenza like illness, Indian Journal of Research in Homeopathy (2013)
A multicenter, single blind, randomized, placebo controlled study to evaluate the effect of homoeopathic medicines in the treatment of Influenza like illness and to compare the efficacy of LM (50 millisimal) potency vis-à-vis centesimal (C) potency. In LM group (n=152), C group (n=147) or placebo (n=148) group. The study revealed the significant effect of individualized homoeopathic treatment in the patients suffering from ILI with no marked difference between LM and Centesimal groups. The medicines which were commonly prescribed were: Arsenic album, Bryonia alba, Rhus tox., Belladonna, Nux vomica, Sepia, Phosphorus, Gelsemium, Sulphur, Natrum mur. and Aconitum napellus. 
19. Reevaluation of the Effectiveness of Homoeoprophylaxis Against Leptospirosis in Cuba in 2007-8, Journal of Evidence-based Complementary & Alternative Medicine (2014)
The results support the previous conclusions that homoeoprophylaxis can be used to effectively immunize people against targeted infectious diseases such as leptospirosis.
 Iman Navab, Lives saved by Homeopathy in Epidemics and Pandemics, https://drnancymalik.wordpress.com/2013/01/23/epidemics-and-pandemics/
 Reshu Agarwal, Natural History of Disease and Homeopathy at different levels of Intervention, http://www.homeorizon.com/homeopathic-articles/homeopathic-philosophy/disease-history
 Homoeopathy- Science of Gentle Healing, Deptt. of AYUSH, Ministry of Health & Family Welfare, Govt, of India, 2013, http://www.ccrhindia.org/Dossier/content/page22.html
 Conversation with David Little, http://hpathy.com/homeopathy-papers/conversations-with-david-little/
 Nancy Malik, Principles of Homeopathy Explained, 2015, https://drnancymalik.wordpress.com/article/homeopathy-explained/
 Nancy Malik, Recent Advances in Nanoparticle Research in Homeopathy, Homeopathy 4 Everyone, Vol.12, Issue 6, 18 June 2015, http://hpathy.com/scientific-research/recent-advances-in-nanoparticle-research-in-homeopathy/
 Samuel Hahnemann, “Appeal to Thinking Philanthropists Respecting the Mode of Propagation of the Asiatic Cholera”, 20 pages, 1831, Translated by R E Dudgeon, M.D. in The Lesser Writings of Samuel Hahnemann, 1851, B Jain Publishers, reproduced edition, 2002, p. 758
 Fran Sheffield, Homeoprophylaxis: Human Records, Studies and Trials, 2014, http://homeopathyplus.com/Homeoprophylaxis-Human-Records-Studies-Trials.pdf
 Homoeopathy in Flu-like Illness- Factsheet, Central Council for Research in Homoeopathy, Deptt. of AYUSH, Ministry of Health & Family Welfare, Govt, of India, 2015, http://ccrhindia.org/pdf/swineflu.pdf
END OF QUOTE
Whenever I read articles of this nature, I get a little embarrassed. It seems obvious to me that the authors of such reviews have done some ‘research’ and believe strongly in the correctness in what they write. It embarrasses me to see how such people, full of good will, can be so naïve, ignorant and wrong. They clearly fail to understand several crucial issues. To me. this seems like someone such as me lecturing others about car mechanics, quantum physics or kite flying. I have no idea about these subjects, and therefore it would be idiotic to lecture others about them. But homeopaths tend to be different! And this is when my embarrassment quickly turns into anger: articles like the above spread nonsense and misguide people about important issues. THEY ARE DANGEROUS! There is little room for embarrassment and plenty of room for criticism. So, let’s criticise the notions advanced above.
In my recent book, I briefly touched upon epidemics in relation to homeopathy:
Epidemics are outbreaks of disease occurring at the same time in one geographical area and affecting large number of people. In homeopathy, epidemics are important because, in its early days, they seemed to provide evidence for the notion that homeopathy is effective. The results of homeopathic treatment seemed often better than those obtained by conventional means. Today we know that this was not necessarily due to the effects of homeopathy per se, but might have been a false impression caused by bias and confounding.
This tells us the main reason why the much-treasured epidemiological evidence of homeopaths is far from compelling. The review above does not mention these caveats at all. But it is lousy also for a whole host of other reasons, for instance:
- The text contains several errors (which I find too petty to correct here).
- The list of studies is the result of cherry-picking the evidence.
- It confuses what epidemiological studies are; RCTs are certainly not epidemiological studies, for instance.
- It also omits some of the most important epidemiological studies suggesting homeopathy works.
- It cites texts that are clearly not epidemiological studies.
- Several studies are on prevention of illness rather than on treatment.
- Some studies do not even employ homeopathy at all.
In the typical epidemiological case/control study, one large group of patients [A] is retrospectively compared to another group [B]. By large, I mean with a sample size of thousands of patients. In our case, group A has been treated homeopathically, while group B received the treatments available at the time. It is true that several of such reports seemed to suggest that homeopathy works. But this does by no means prove anything; the result might have been due to a range of circumstances, for instance:
- group A might have been less ill than group B,
- group A might have been richer and therefore better nourished,
- group A might have benefitted from better hygiene in the homeopathic hospital,
- group A might have received better care, e. g. hydration,
- group B might have received treatments that made the situation not better but worse.
Because these are RETROSPECTIVE studies, there is no way to account for these and many other factors that might have influenced the outcome. This means that epidemiological studies of this nature can generate interesting results which, in turn, need testing in properly controlled studies where these confounding factors are adequately controlled for. Without such tests, they are next to worthless for recommendations regarding clinical practice.
As it happens, the above author also included two RCT in the review (these are NOT epidemiological studies, as I already mentioned). Let’s have a quick look at them.
The first RCT is flawed for a range of reasons and has been criticised many times before. Even its authors state that “the result cannot be explained given our present state of knowledge, but it calls for further rigorously designed clinical studies.” More importantly, the current Cochrane review of Oscillococcinum, the remedy used in this study, concluded: “There is insufficient good evidence to enable robust conclusions to be made about Oscillococcinum® in the prevention or treatment of influenza and influenza-like illness.”
The second RCT is equally flawed; for instance, its results could be due to the concomitant use of paracetamol, and it seems as though the study was not double blind. The findings of this RCT have so far not been confirmed by an independent replication.
What puzzles me most with these regularly voiced notions about the ‘epidemiological evidence’ for homeopathy is not the deplorable ineptitude of those who promote them, but it is this: do homeopaths really believe that conventional medics and scientists would ignore such evidence, if it were sound or even just encouraging? This assumes that all healthcare professionals (except homeopaths) are corrupt and cynical enough not to follow up leads with the potential to change medicine for ever. It assumes that we would supress knowledge that could save the lives of millions for the sole reason that we are against homeopathy or bribed by ‘BIG PHARMA’.
Surely, this shows more clearly than anything else how deluded homeopaths really are!!!
Acupuncture is often recommended as a treatment for shoulder pain, but its effectiveness is far from proven. A new study has just been published; but does it change this uncertainty?
A total of 227 patients with subacromial pain syndrome were recruited to this RCT. The patients were allocated to three groups who received either A) group exercise, B) group exercise plus acupuncture or C) group exercise plus electro-acupuncture. The primary outcome measure was the Oxford Shoulder Score. Follow-up was post treatment, and at 6 and 12 months. Data were analysed on intention-to-treat principles with imputation of missing values.
Treatment groups were similar at baseline. All treatment groups demonstrated improvements over time. Between-group estimates were, however, small and non-significant.
The authors concluded that neither acupuncture nor electro-acupuncture were found to be more beneficial than exercise alone in the treatment of subacromial pain syndrome.
Well, that was to be expected!… I hear the rationalists amongst us exclaim.
Actually, I am not so sure.
One could easily have expected that the acupuncture groups (B and C) show a significant advantage over group A.
Because acupuncture is a ‘theatrical placebo’, a ritual that impresses patients and thus impacts on results, particularly on subjective outcomes like pain. If the results had shown a benefit for acupuncture + exercise (groups B and C) versus exercise alone (group A), what would we have made of it? Acupuncture fans would surely have claimed that it is evidence confirming acupuncture’s effectiveness. Sceptics, on the other hand, would have rightly insisted that it demonstrates nothing of the sort – it merely confirms that placebo effects can affect clinical outcomes such as pain.
As it turned out, however, this trial results happened to indicate that these placebo-effects can be so small that they fail to reach the level of statistical significance.
I think there is one noteworthy message here: RCTs with such a design (no adequate control for placebo effects) can easily generate false-positive results (in this case, this did not happen, but it was nevertheless a possible outcome). Such studies are popular but utterly useless: they don’t advance our knowledge one single iota. If that is so, we should not waste our resources on them because, in the final analysis, this is not ethical. In other words, we must stop funding research that has little or no chance of advancing our knowledge.
If you want to scientifically investigate this question, it might be a good idea NOT to start with the following sentence: “Auricular acupuncture (AA) is effective in the treatment of preoperative anxiety”. Yet, this is exactly what the authors did in their recent publication.
The aim of this new study was to investigate whether AA can reduce exam anxiety as compared to placebo and no intervention. Forty-four medical students were randomized to receive AA, placebo, or no intervention in a crossover manner. Subsequently they completed three comparable oral anatomy exams with an interval of one month between the exams/interventions.
A licensed acupuncturist with more than five years of experience with this technique applied AA at the acupuncture points MA-IC1 (Lung), MA-TF1 (ear Shenmen), MA-SC (Kidney), MA-AT1 (Subcortex) and MA-TG (Adrenal gland) bilaterally. Indwelling fixed ‘New Pyonex’ needles embedded in a skin-coloured adhesive tape were used for AA. The participants were instructed by the acupuncturist to stimulate the auricular needles for 3–5 minutes, if they felt anxious. For the placebo procedure, ‘New Pyonex’ placebo needles were attached to five sites on the helix of the auricle bilaterally. ‘New Pyonex’ placebo needles have the same appearance as AA needles but consist of self-adhesive tape only. In order to avoid potential physiologic effects of acupressure, the participants were not instructed to stimulate the attached ‘New Pyonex’ placebo needles. AA and placebo needles were left in situ until the next day and were removed out of sight of the participants after the exam by the investigator, who was not involved in acupuncture procedure
Levels of anxiety were measured using a visual analogue scale before and after each intervention as well as before each exam. Additional measures included the State-Trait-Anxiety Inventory, duration of sleep at night, blood pressure, heart rate and the extent of participant blinding.
All included participants finished the study. Anxiety levels were reduced after AA and placebo intervention compared to baseline and the no intervention condition (p < 0.003). Moreover, AA was also better at reducing anxiety than placebo in the evening before the exam (p = 0.018). Participants were able to distinguish between AA and placebo intervention.
The authors concluded that both auricular acupuncture and placebo procedure were shown to be effective in reducing levels of exam anxiety in medical students. The superiority of verum AA over placebo AA and no intervention is considered to be due to stimulation of cranial nerves, but may have been increased in effect by insufficient participant blinding.
Here are just three of the major concerns I have about this study:
- The trial design seems odd: a crossover study can only work well, if there is a stable baseline. This may not be the case with three consecutive exams; the anxiety experienced by students is bound to get less as time goes by. I think anyone who has passed a series of exams will confirm that there is a large degree of habituation.
- It seems inadequate to employ just one acupuncturist; it means that the trial might end up testing not acupuncture per se but the skills of the therapist.
- The placebo used for this study cannot possibly have fooled anyone into believing that it was real AA; volunteers were not even instructed to ‘stimulate’ the placebo devices. The difference to the ‘real thing’ must have been very clear to all involved. This means that the control for placebo-effects was woefully incomplete. In turn, this means that the observed outcomes are most likely due to residual bias.
In view of these concerns, allow me to re-phrase the authors’ conclusions:
THE RESULTS OF THIS POORLY-DESIGNED STUDY ARE DIFFICULT TO INTERPRET. MOST LIKELY THEY SHOW THAT ACUPUNCTURE IS NOT EFFECTIVE BUT MERELY WORKS THROUGH A PLACEBO-RESPONSE.
Perhaps I have a weak spot for fish oil; more likely, however, I just like positive news – and, in alternative medicine, there is not much of it. That’s why I have written about the potential benefits of fish-oil again and again and again and again.
Reduced intake of fish oil, i.e. n−3 long-chain polyunsaturated fatty acids (LCPUFAs), may be a contributing factor to the increasing prevalence of asthma and other wheezing disorders. Yet the evidence is neither clear nor strong. This study was aimed at shedding more light on the issue; specifically, it tested the effect of supplementation with n−3 LCPUFAs in pregnant women on the risk of persistent wheeze and asthma in their offspring.
The investigators randomly assigned 736 pregnant women at 24 weeks of gestation to receive 2.4 g of n−3 LCPUFA (fish oil) or placebo (olive oil) per day. Their children were followed prospectively with extensive clinical phenotyping. Neither the investigators nor the participants were aware of group assignments during follow-up for the first 3 years of the children’s lives, after which there was a 2-year follow-up period during which only the investigators were unaware of group assignments. The primary end point was persistent wheeze or asthma, and the secondary end points included lower respiratory tract infections, asthma exacerbations, eczema, and allergic sensitization.
A total of 695 children were included in the trial, and 95.5% completed the 3-year, double-blind follow-up period. The risk of persistent wheeze or asthma in the treatment group was 16.9%, versus 23.7% in the control group (hazard ratio, 0.69; 95% confidence interval [CI], 0.49 to 0.97; P=0.035), corresponding to a relative reduction of 30.7%. Prespecified subgroup analyses suggested that the effect was strongest in the children of women whose blood levels of eicosapentaenoic acid and docosahexaenoic acid were in the lowest third of the trial population at randomization: 17.5% versus 34.1% (hazard ratio, 0.46; 95% CI, 0.25 to 0.83; P=0.011). Analyses of secondary end points showed that supplementation with n−3 LCPUFA was associated with a reduced risk of infections of the lower respiratory tract (31.7% vs. 39.1%; hazard ratio, 0.75; 95% CI, 0.58 to 0.98; P=0.033), but there were no statistically significant associations between supplementation and asthma exacerbations, eczema, or allergic sensitization.
The authors concluded that supplementation with n−3 LCPUFA in the third trimester of pregnancy reduced the absolute risk of persistent wheeze or asthma and infections of the lower respiratory tract in offspring by approximately 7 percentage points, or one third.
The authors must be congratulated. This trial is stunning in many ways: it was carefully designed and executed; its results are clear and important; its write-up is excellent. The research was supported by private and public research funds, all of which are listed at www.copsac.com. The Lundbeck Foundation, the Danish Ministry of Health, the Danish Council for Strategic Research, the Danish Council for Independent Research, and the Capital Region Research Foundation provided core support.
It is debatable whether the intake of fish oil falls under the umbrella of alternative medicine. In a way, it reminds me of the famous saying: what do we call alternative medicine that works? We call it medicine. It also holds an important reminder for all who make claims about the benefit of alternative therapies: extraordinary claims require extraordinary evidence.
Can these findings be translated into practical advice to consumers? The NEJM discussed this question in an accompanying article in which the case of a fictional pregnant woman (Ms. Franklin) was considered. Here is what they concluded: …there is benefit and little risk associated with n−3 LCPUFA supplementation. Even though we do not know Ms. Franklin’s EPA and DHA levels, there is likely to be a benefit for her child, at little risk, cost, or inconvenience. She should start taking n−3 LCPUFA supplements.
Despite my soft spot for fish oil, I might add that, while we give advice of this nature, we nevertheless need to insist on independent replications to have certainty.
The common cold is one of the indications for which homeopathy is deemed to be effective… by homeopaths that is! Non-homeopaths are understandably critical about this claim, not least because there is no good evidence for it. But, hold on, there is a new study which might change all this.
This study was recently published in COMPLEMENTARY THERAPIES IN MEDICINE which is supposed to be one of the better journals in this area. According to its authors, it was conducted “to determine if a homeopathic syrup was effective in treating cold symptoms in preschool children.” Children diagnosed with an upper respiratory tract infection were randomized to receive a commercial homeopathic cold syrup containing allium cepa 6X, hepar sulf calc 12X, natrum muriaticum 6X, phosphorous 12X, pulsatilla 6X, sulphur 12X, and hydrastasis 6X or placebo. Parents administered the study medication as needed for 3 days. The primary outcome was change in symptoms one hour after each dose. Parents also assessed the severity of each of the symptoms of runny nose, cough, congestion and sneezing at baseline and twice daily for 3 days, using a 4-point rating scale. A composite cold score was calculated by combining the values for each of the four symptoms. Among 261 eligible participants, data on 957 doses of study medication in 154 children were analyzed. There was no significant difference in improvement one hour after the dose for any symptom between the two groups. Analysis of twice daily data on the severity of cold symptoms compared to baseline values found that improvements in sneezing, cough and the composite cold score were significantly greater at both the first and second assessments among those receiving the cold syrup compared to placebo recipients.
The authors concluded that the homeopathic syrup appeared to be effective in reducing the severity of cold symptoms in the first day after beginning treatment.
Where to start? There are so many problems with this study that I find it difficult to chose the most crucial ones:
- The study had a clearly defined primary endpoint; it was not affected by the homeopathic treatment which doubtlessly makes the study a negative trial. The only correct conclusion therefore is that THE HOMEOPATHIC SYRUP FAILED TO AFFECT THE PRIMARY OUTCOME MEASURE OF THIS STUDY. THEREFORE THE TRIAL DID NOT PRODUCE ANY EVIDENCE TO ASSUME THAT THE EXPERIMENTAL TREATMENT WAS EFFICACIOUS.
- I don’t think that many of the primary or secondary outcome measures are validated or reliable.
- All the positive results reported in the abstract and the article relate to secondary endpoints which are purely explanatory by nature. They should, in my view, not be mentioned in the conclusions at all.
- The fact that some results turned out to be positive can be explained by the fact that the investigators ran dozens of tests for statistical significance which means that, by simple chance, some will turn out to produce a positive result.
- A further explanation for the seemingly positive results might be the fact disclosed in the text of the article that the children in the homeopathy group received more conventional drugs than those in the placebo group.
- Whatever the reason for these positive results, they certainly had nothing to do with the homeopathic syrup.
- The study was funded by the company producing the syrup and for which one of the authors was employed as a consultant. This might be an explanation for the abominably poor science. In other words, this paper is not an exercise in testing a hypothesis but one in marketing.
While I might forgive the company for trying to maximise their sales figures, I do find it harder to forgive the authors, reviewers and editors for publishing such overtly false conclusions. In my view, they are all guilty of scientific misconduct.
Meniscus-injuries are common and there is no consensus as to how best treat them. Physiotherapists tend to advocate exercise, while surgeons tend to advise surgery.
Of course, exercise is not a typical alternative therapy but, as many alternative practitioners might disagree with this statement because they regularly recommend it to their patients, it makes sense to cover it on this blog. So, is exercise better than surgery for meniscus-problems?
The aim of this recent Norwegian study aimed to shed some light on this question. Specifically wanted to determine whether exercise therapy is superior to arthroscopic partial meniscectomy for knee function in patients with degenerative meniscal tears.
A total of 140 adults with degenerative medial meniscal tear verified by magnetic resonance imaging were randomised to either receiving 12 week supervised exercise therapy alone, or arthroscopic partial meniscectomy alone. Intention to treat analysis of between group difference in change in knee injury and osteoarthritis outcome score (KOOS4), defined a priori as the mean score for four of five KOOS subscale scores (pain, other symptoms, function in sport and recreation, and knee related quality of life) from baseline to two-year follow-up and change in thigh muscle strength from baseline to three months.
The results showed no clinically relevant difference between the two groups in change in KOOS4 at two years (0.9 points, 95% confidence interval −4.3 to 6.1; P=0.72). At three months, muscle strength had improved in the exercise group (P≤0.004). No serious adverse events occurred in either group during the two-year follow-up. 19% of the participants allocated to exercise therapy crossed over to surgery during the two-year follow-up, with no additional benefit.
The authors concluded that the observed difference in treatment effect was minute after two years of follow-up, and the trial’s inferential uncertainty was sufficiently small to exclude clinically relevant differences. Exercise therapy showed positive effects over surgery in improving thigh muscle strength, at least in the short-term. Our results should encourage clinicians and middle-aged patients with degenerative meniscal tear and no definitive radiographic evidence of osteoarthritis to consider supervised exercise therapy as a treatment option.
As I stated above, I mention this trial because exercise might be considered by some as an alternative therapy. The main reason for including it is, however, that it is in many ways an exemplary good study from which researchers in alternative medicine could learn.
Like so many alternative therapies, exercise is a treatment for which placebo-controlled studies are difficult, if not impossible. But that does not mean that rigorous tests of its value are impossible. The present study shows the way how it can be done.
Meaningful clinical research is no rocket science; it merely needs well-trained scientists who are willing to test the (rather than promote) their hypotheses. Sadly such individuals are as rare as gold dust in the realm of alternative medicine.
On 25 and 26 May of this year I wrote two posts about an acupuncture trial that, in my view, was dodgy. To refresh your memory, here is the relevant part of the 2nd post:
This new study was designed as a randomized, sham-controlled trial of acupuncture for persistent allergic rhinitis in adults investigated possible modulation of mucosal immune responses. A total of 151 individuals were randomized into real and sham acupuncture groups (who received twice-weekly treatments for 8 weeks) and a no acupuncture group. Various cytokines, neurotrophins, proinflammatory neuropeptides, and immunoglobulins were measured in saliva or plasma from baseline to 4-week follow-up.
Statistically significant reduction in allergen specific IgE for house dust mite was seen only in the real acupuncture group. A mean (SE) statistically significant down-regulation was also seen in pro-inflammatory neuropeptide substance P (SP) 18 to 24 hours after the first treatment. No significant changes were seen in the other neuropeptides, neurotrophins, or cytokines tested. Nasal obstruction, nasal itch, sneezing, runny nose, eye itch, and unrefreshed sleep improved significantly in the real acupuncture group (post-nasal drip and sinus pain did not) and continued to improve up to 4-week follow-up.
The authors concluded that acupuncture modulated mucosal immune response in the upper airway in adults with persistent allergic rhinitis. This modulation appears to be associated with down-regulation of allergen specific IgE for house dust mite, which this study is the first to report. Improvements in nasal itch, eye itch, and sneezing after acupuncture are suggestive of down-regulation of transient receptor potential vanilloid 1.
…the trial itself raises a number of questions:
- Which was the primary outcome measure of this trial?
- What was the power of the study, and how was it calculated?
- For which outcome measures was the power calculated?
- How were the subjective endpoints quantified?
- Were validated instruments used for the subjective endpoints?
- What type of sham was used?
- Are the reported results the findings of comparisons between verum and sham, or verum and no acupuncture, or intra-group changes in the verum group?
- What other treatments did each group of patients receive?
- Does anyone really think that this trial shows that “acupuncture is a safe, effective and cost-effective treatment for allergic rhinitis”?
In the comments section, the author wrote: “after you have read the full text and answered most of your questions for yourself, it might then be a more appropriate time to engage in any meaningful discussion, if that is in fact your intent”, and I asked him to send me his paper. As he does not seem to have the intention to do so, I will answer the questions myself and encourage everyone to have a close look at the full paper [which I can supply on request].
- The myriad of lab tests were defined as primary outcome measures.
- Two sentences are offered, but they do not allow me to reconstruct how this was done.
- No details are provided.
- Most were quantified with a 3 point scale.
- Mostly not.
- Needle insertion at non-acupoints.
- The results are a mixture of inter- and intra-group differences.
- Patients were allowed to use conventional treatments and the frequency of this use was reported in patient diaries.
- I don’t think so.
So, here is my interpretation of this study:
- It lacked power for many outcome measures, certainly the clinical ones.
- There were hardly any differences between the real and the sham acupuncture group.
- Most of the relevant results were based on intra-group changes, rather than comparing sham with real acupuncture, a fact, which is obfuscated in the abstract.
- In a controlled trial fluctuations within one group must never be interpreted as caused by the treatment.
- There were dozens of tests for statistical significance, and there seems to be no correction for multiple testing.
- Thus the few significant results that emerged when comparing sham with real acupuncture might easily be false positives.
- Patient-blinding seems questionable.
- McDonald as the only therapist of the study might be suspected to have influenced his patients through verbal and non-verbal communications.
I am sure there are many more flaws, particularly in the stats, and I leave it to others to identify them. The ones I found are, however, already serious enough, in my view, to call for a withdrawal of this paper. Essentially, the authors seem to have presented a study with largely negative findings as a trial with positive results showing that acupuncture is an effective therapy for allergic rhinitis. Subsequently, McDonald went on social media to inflate his findings even more. One might easily ask: is this scientific misconduct or just poor science?
END OF QUOTE
This and the previous post created lots of discussion and comments. However, the question whether the study in question amounted to scientific misconduct was never satisfactorily resolved. Therefore, I decided to write to the editor of ‘Ann Allergy Asthma Immunol‘ where the trial had been published. He answered by saying I would need to file an official complaint for him to address the issue. On 13 June, I therefore sent him the following email:
Thank you for your letter of 3/6/2016 suggesting I make a formal complaint about the paper entitled ‘EFFECT OF ACUPUNCTURE ON HOUSE DUST MITE…’ [ Ann Allergy Asthma Immunol 2016] by McDonald et al. I herewith wish to file such a complaint.
The article in question reports an RCT of acupuncture for persistent allergic rhinitis. It followed a parallel group design with 3 groups receiving the following interventions:
3. No treatment
There was a plethora of outcome measures and time points on which they were measured. A broad range of parameters was defined as primary endpoints.
The conclusion reached by the authors essentially was that acupuncture affected several outcome measures in a positive sense, thus supporting the notion that acupuncture is efficacious [“Symptoms and quality of life improved significantly and were still continuing to improve 4 weeks after treatment ceased.”] This conclusion, however, is misleading and needs correcting.
The main reasons for this are as follows:
· Despite the fact that the authors did many dozens of statistical tests for significance, they did not correct for this multiplicity of tests. Consequently, some or most of the significant results are likely to be false positive.
· Many of the positive results of this paper were not obtained by comparing one group to another but by doing before/after comparisons within one group. This approach defies the principle of a controlled clinical trial. For doing intra-group comparisons, we obviously do not need any control group at all. The findings from intra-group comparisons are prominently reported in the paper, for instance in the abstract, giving the impression that they originate from inter-group comparisons. One has to read the paper very carefully to find that, when inter-group comparisons were conducted, their results did NOT confirm the findings from the reported intra-group comparisons. As this is the case for most of the symptomatic endpoints, the impression given is seriously misleading and needs urgent correction.
On the whole, the article is a masterpiece of obfuscation and misrepresentation of the actual data. I urge you to consider the harm than can be done by such a misleading publication. In my view, the best way to address this problem is to withdraw the article.
I look forward to your decision.
END OF QUOTE
I had to send several reminders but my most recent one prompted the following response dated 7/11/ 2016:
- Our editorial team recognizes that this is not the best clinical trial we have published in the Annals of Allergy, Asthma and Immunology. However, neither is is the worst. As in most published research studies, there are always things that could have been done better to make it a stronger paper. Never-the-less, the criticism falls fall short of any sort of remedy that would include withdrawal of the manuscript.
- Regarding your accusation that the multiple positive endpoint resulted in the authors making specific therapeutic claims, our assessment is that no specific therapeutic claim was made but rather the authors maintained that the data support the value of acupuncture in improving symptoms and quality of life in patients with AR. We do not believe there was overreach in those statements.
- The authors’ stated intent was to show immune changes associated with clinical markers of improvement in the active acupuncture group compared to controls. The authors maintain (and our editors agree) that their data assessments were primarily based upon three statistical tests not “dozens” (as stated in your original letter of complaint). The power analysis and sample size calculations were presented to us and deemed adequate , making the probability of a type I error quite low.
- The authors acknowledge in their paper that there could be limitations to their data interpretation based upon potential disparities between intra- and intergroup comparisons. The editors felt their transparency was adequately disclosed.