MD, PhD, FMedSci, FSB, FRCP, FRCPEd

scientific misconduct

1 2 3 7

I just came across a new article entitled ” Vaccinated children four times more likely to suffer from ADHD, autism“. It was published in WDDTY, my favourite source of misleading information. Here it is:

Vaccinated children are nearly four times more likely to suffer from learning disabilities, ADHD and autism, a major new study has discovered—and they are six times more likely to suffer from one of these neuro-developmental problems if they were also born prematurely.

The vaccinated child is also more likely to suffer from otitis media, the ear infection, and nearly six times more likely to contract pneumonia.

But the standard childhood vaccines do at least do their job: the vaccinated child is nearly eight times less likely than the unvaccinated to develop chicken pox, and also less likely to suffer from whooping cough (pertussis).

Researchers from Jackson State University are some of the first to look at the long-term effects of vaccination. They monitored the health of 666 children for six years from the time they were six—when the full vaccination programme had been completed—until they were 12. All the children were being home-schooled because it was one of the few communities where researchers could find enough unvaccinated children for comparison; 261 of the children hadn’t been vaccinated and 208 hadn’t had all their vaccinations, while 197 had received the full 48-dose course.

The vaccinated were more likely to suffer from allergic rhinitis, such as hay fever, eczema and atopic dermatitis, learning disability, ADHD (attention-deficit, hyperactive disorder), and autism. The risk was lower among the children who had been partially vaccinated.

Vaccinated children were also more likely to have taken medication, such as an antibiotic, or treatment for allergies or for a fever, than the unvaccinated.

END OF QUOTE

I looked up the original study to check and found several surprises.

The first surprise was that the study was called a ‘pilot’ by its authors, even in the title of the paper: “Pilot comparative study on the health of vaccinated and unvaccinated 6- to 12-year-old U.S. children.”

The second surprise was that even the authors admit to important limitations of their research:

We did not set out to test a specific hypothesis about the association between vaccination and health. The aim of the study was to determine whether the health outcomes of vaccinated children differed from those of unvaccinated homeschool children, given that vaccines have nonspecific effects on morbidity and mortality in addition to protecting against targeted pathogens [11]. Comparisons were based on mothers’ reports of pregnancy-related factors, birth histories, vaccinations, physician-diagnosed illnesses, medications, and the use of health services. We tested the null hypothesis of no difference in outcomes using chi-square tests, and then used Odds Ratios and 96% Confidence Intervals to determine the strength and significance of the association…

What credence can be given to the findings? This study was not intended to be based on a representative sample of homeschool children but on a convenience sample of sufficient size to test for significant differences in outcomes. Homeschoolers were targeted for the study because their vaccination completion rates are lower than those of children in the general population. In this respect our pilot survey was successful, since data were available on 261 unvaccinated children…

Mothers’ reports could not be validated by clinical records because the survey was designed to be anonymous. However, self-reports about significant events provide a valid proxy for official records when medical records and administrative data are unavailable [70]. Had mothers been asked to provide copies of their children’s medical records it would no longer have been an anonymous study and would have resulted in few completed questionnaires. We were advised by homeschool leaders that recruitment efforts would have been unsuccessful had we insisted on obtaining the children’s medical records as a requirement for participating in the study.

A further potential limitation is under-ascertainment of disease in unvaccinated children. Could the unvaccinated have artificially reduced rates of illness because they are seen less often by physicians and would therefore have been less likely to be diagnosed with a disease? The vaccinated were indeed more likely to have seen a doctor for a routine checkup in the past 12 months (57.5% vs. 37.1%, p < 0.001; OR 2.3, 95% CI: 1.7, 3.1). Such visits usually involve vaccinations, which nonvaccinating families would be expected to refuse. However, fewer visits to physicians would not necessarily mean that unvaccinated children are less likely to be seen by a physician if their condition warranted it. In fact, since unvaccinated children were more likely to be diagnosed with chickenpox and whooping cough, which would have involved a visit to the pediatrician, differences in health outcomes are unlikely to be due to under-ascertainment.

The third surprise was that the authors were not at all as certain as WDDTY in their conclusions: “the study findings should be interpreted with caution. First, additional research is needed to replicate the findings in studies with larger samples and stronger research designs. Second, subject to replication, potentially detrimental factors associated with the vaccination schedule should be identified and addressed and underlying mechanisms better understood. Such studies are essential in order to optimize the impact of vaccination of children’s health.”

The fourth surprise was to find the sponsors of this research:

Generation Rescue is, according to Wikipedia, a nonprofit organization that advocates the incorrect view that autism and related disorders are primarily caused by environmental factors, particularly vaccines. These claims are biologically implausible and are disproven by scientific evidence. The organization was established in 2005 by Lisa and J.B. Handley. They have gained attention through use of a media campaign, including full page ads in the New York Times and USA Today. Today, Generation Rescue is known as a platform for Jenny McCarthy‘s autism and anti-vaccine advocacy.

The Children’s Medical Safety Research Institute (CMSRI) was, according to Vaxopedia, created by and is funded by the Dwoskin Family Foundation. It provides grants to folks who will do research on “vaccine induced brain and immune dysfunction” and on what they believe are other “gaps in our knowledge about vaccines and vaccine safety”, including:

While they claim that they are not an anti-vaccine organization, it should be noted that  Claire Dwoskin once said that “Vaccines are a holocaust of poison on our children’s brains and immune systems.”

Did I say SURPRISE?

I take it back!

When it comes to WDDTY, nothing does surprise me.

To a significant extend, this blog has always exposed untruths in the realm of alternative medicine – not just one or two, but hundreds. Obviously, some of them are more clear-cut than others. If, for instance, someone claims that acupuncture has been proven to be effective for a given condition, this many seem like a lie or untruth to you, like a misinterpretation of the evidence to someone else, or like the truth to a third person.

But there are some statements which are demonstrably false. These are often the most irritating lies, frequently forwarded by people who should know better and who nevertheless insist on not being truthful. Below I have listed a few, randomly-chosen examples upon which I have previously commented. For clarity, I have copied the quotes in question, linked them to my original posts, named the authors in brackets, and added a brief comment by myself in bold print.

I was at Exeter when Ernst took over what was already a successful Chair in CAM. (anonymous reviewer of my book at Amazon)

Anyone can check this fairly easily, for instance, in my memoir ‘A SCIENTIST IN WONDERLAND’, there was no pre-existing chair at Exeter.

Ernst’s leak of the Smallwood report (also front page lead in The Times, August 2005), (Dr Peter Fisher, homeopath of the Queen)

This was painfully investigated during a 13 (!) months inquiry which found that I did not leak this report. Again you find the full details in my memoir.

…homeopathic care is recommended for people who have been exposed (or think they have been exposed) to toxic substances… (Dana Ullman US homeopath)

As far as this statement implies that homeopathy is effective for treating intoxications, this is not only a lie but a very dangerous nonsense.

Homeopathy has a long history of being used successfully in veterinary practice for both domestic and farm animals. (UK Faculty of Homeopathy)

If this is to suggest that homeopathy is of proven effectiveness in treating diseases of animals, this is a lie.

Homeoprophylaxis, the homeopathic vaccine alternative, prevents disease through nosodes. (Lisa is the mastermind behind All Natural Ideas)

Homeoprophylaxis has never been proven to prevent any disease; this lie could kill millions.

There are essentially two categories of critics. The first category consists of individuals who are totally ignorant of homeopathy and just repeating propaganda they’ve been exposed to. The second category is people who know that homeopathy works, but have a vested financial interest in destroying it. (Alan Schmukler, US homeopathy)

This lie is quite funny in its transparent defamation of the truth, I think.

Homeopathy works like a vaccine. (Dr Batra, Indian homeopath)

Homeopathy does not even remotely work like a vaccine; in fact, it works like a placebo, if at all.

…UK invests 0% of its research budget on CAM… (Dr Michael Dixon, GP and advisor to Prince Charles)

There has always been a sizable budget for CAM-research in the UK.

Even cancer viruses have, on record, been put into vaccinations. There is no actual vaccine for cancer. The only reason to put cancer viruses in the mix is to create more cases of cancer. In this day and age, one of the most dangerous things you can do for your health is to get vaccinated… (US homeopath)

In this short quote, there are more lies than I care to comment on. The paranoia of the anti-vaccination brigade is astounding and endangers many lives.

A lie is a statement used intentionally for the purpose of deception. In alternative medicine, we encounter so many lies that one would need to continually publish volume after volume to expose just the most harmful untruths. The danger of these lies is that some people might believe them. This could seriously harm their health. Another danger is that we might get used to them, trivialise them, or – like Trump and co – start thinking of them as ‘alternative facts’.

I will continue to do my best to prevent any of this from happening.

 

I have repeatedly cautioned about the often poor quality of research into alternative medicine. This seems particularly necessary with studies of acupuncture, and especially true for such research carried out in China. I have also frequently noted that certain ‘CAM journals’ are notoriously prone to publishing rubbish. So, what can we expect from a paper that:

  • is on alternative medicine,
  • focusses on acupuncture,
  • is authored by Chinese researchers,
  • was published in the Journal of Alternative and Complementary Medicine (JACM)?

The answer is PROBABLY NOT A LOT!

As if for confirming my prediction, The JACM just published this systematic review. It reports pairwise and network meta-analyses to determine the effectiveness of acupuncture and acupuncture-related techniques for the treatment of psoriasis. A total of 13 RCTs were included. The methodological quality of these studies was ‘not rigorous’ according to the authors – in fact, it was lousy. Acupoint stimulation seemed to be more effective than non-acupoint stimulation. The short-term treatment effect was superior to the long-term effect (as one would expect with placebo). Network meta-analysis suggested that acupressure or acupoint catgut embedding generate superior effects compared to medications. It was noted that acupressure was the most effective treatment of all the acupuncture-like therapies.

The authors concluded that acupuncture-related techniques could be considered as an alternative or adjuvant therapy for psoriasis in short term, especially of acupressure and acupoint catgut embedding. This study recommends further well-designed, methodologically rigorous, and more head-to-head randomized trials to explore the effects of acupuncture-related techniques for treating psoriasis.

And what is wrong with that?

EVERYTHING!

  • The review is of very poor quality.
  • The primary studies are even worse.
  • The English language is defective to the point of being not understandable.
  • The conclusions are misleading.

Correct conclusions should read something like this: Due to the paucity and the poor quality of the clinical trials, this review could not determine whether acupuncture and similar therapies are effective for psoriasis.

And then there is, of course, the question about plausibility. How plausible is the assumption that acupuncture might affect a genetic autoimmune disease like psoriasis. The answer, I think, is that the assumption is highly unlikely.

In the above review, most of the 13 primary RCTs were from China. One of the few studies not conducted in China is this one:

56 patients suffering from long-standing plaque psoriasis were randomized to receive either active treatment (electrostimulation by needles placed intramuscularly, plus ear-acupuncture) or placebo (sham, ‘minimal acupuncture‘) twice weekly for 10 weeks. The severity of the skin lesions was scored (PASI) before, during, and 3 months after therapy. After 10 weeks of treatment the PASI mean value had decreased from 9.6 to 8.3 in the ‘active’ group and from 9.2 to 6.9 in the placebo group (p < 0.05 for both groups). These effects are less than the usual placebo effect of about 30%. There were no statistically significant differences between the outcomes in the two groups during or 3 months after therapy. The patient’s own opinion about the results showed no preference for ‘active’ therapy. It was also clear from the answers that the blinded nature of the study had not been discovered by the patients. In conclusion, classical acupuncture is not superior to sham (placebo) ‘minimal acupuncture‘ in the treatment of psoriasis.

Somehow, I trust these conclusions more than the ones from the review!

And somehow, I get very tired of journal editors failing to do their job of rejecting papers that evidently are embarrassing, unethical rubbish.

Systematic reviews are aimed at summarising and critically evaluating the evidence on a specific research question. They are the highest level of evidence and are more reliable than anything else we have. Therefore, they represent a most useful tool for both clinicians and researchers.

But there are, of course, exceptions. Take, for instance, this recent systematic review by researchers from the

  • Texas Chiropractic College, Pasadena, the Palmer Center for Chiropractic Research, Palmer College of Chiropractic, Davenport,
  • Department of Planning, Policy and Design, University of California, Irvine,
  • VA Puget Sound Health Care System, Tacoma,
  • New York Chiropractic College, Seneca Falls,
  • Logan University College of Chiropractic, Chesterfield,
  • University of Western States, Portland.

Its purpose was to evaluate the effectiveness of conservative non-drug, non-surgical interventions, either alone or in combination, for conditions of the shoulder. The review was conducted from March 2016 to November 2016 in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), and was registered with PROSPERO. Eligibility criteria included randomized controlled trials (RCTs), systematic reviews, or meta-analyses studying adult patients with a shoulder diagnosis. Interventions qualified if they did not involve prescription medication or surgical procedures, although these could be used in the comparison group or groups. At least 2 independent reviewers assessed the quality of each study using the Scottish Intercollegiate Guidelines Network checklists. Shoulder conditions addressed were

  • shoulder impingement syndrome (SIS),
  • rotator cuff-associated disorders (RCs),
  • adhesive capsulitis (AC),
  • nonspecific shoulder pain.

Twenty-five systematic reviews and 44 RCTs met inclusion criteria. Low- to moderate-quality evidence supported the use of manual therapies for all 4 shoulder conditions. Exercise, particularly combined with physical therapy protocols, was beneficial for SIS and AC. For SIS, moderate evidence supported several passive modalities. For RC, physical therapy protocols were found beneficial but not superior to surgery in the long term. Moderate evidence supported extracorporeal shockwave therapy for calcific tendinitis RC. Low-level laser was the only modality for which there was moderate evidence supporting its use for all 4 conditions.

The authors concluded that the findings of this literature review may help inform practitioners who use conservative methods (eg, doctors of chiropractic, physical therapists, and other manual therapists) regarding the levels of evidence for modalities used for common shoulder conditions.

This review has so many defects that it would be boring to list them here.

The PRISMA guidelines  – I happen to be a co-author – state, for instance, that the abstract (the above text is from the abstract) should provide a structured summary including, as applicable: background; objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions and implications of key findings; systematic review registration number. It is obvious that the review authors have omitted several of these items.

And that is just the abstract!  There is much, much more to criticise in this paper.

The most striking deficit, in my view, is the useless conclusion: the one from the abstract (the part of the paper that will be read most widely) could have been written before the review had even been started. It is therefore not based on the data presented. Crucially it does not match the stated aim of this review (“to evaluate the effectiveness of conservative…interventions”).

But why? Why did the authors bother to follow PRISMA? Why did they formulate this bizarre conclusion in their abstract? Why did they do a review in the first place?

I fear, the answers might be embarrassingly simple:

  • They only pretended to follow PRISMA guidelines because that gives their review a veneer of respectability.
  • They formulated the conclusions because otherwise they would have needed to state that the evidence for manual therapy is less than convincing.
  • They conducted the review to promote chiropractic, and when the data were not as they had hoped for, they just back-paddled in an attempt to hide the truth as much as possible.

If this were an isolated case, I would not have bothered to mention it. But sadly, in the realm of chiropractic (and alternative medicine in general) we currently witness a plethora of rubbish reviews (published by rubbish journals). To the naïve observer, they might look rigorous and therefore they will be taken seriously. The end-effect of this pollution of the literature with rubbish is that we get a false-positive impression about the validity of the treatments in question. Consequently, we will see a host of wrong decisions on all levels of healthcare.

The big question is: HOW DO WE PROTECT OURSELVES FROM THIS DANGEROUS TREND?

I only see one solution: completely disregard certain journals that have been identified to regularly publish nonsense. Sadly, the wider medical community is far from having arrived at this point. As far as I can see, the problem has not even been identified yet as a serious issue that needs addressing. For the foreseeable future, we will probably have to live with this type of pollution of our medical literature.

How often have I pointed out that most studies of chiropractic (and other alternative therapies) are overtly unethical because they fail to report adverse events? And if you think this is merely my opinion, you are mistaken. This new analysis by a team of chiropractors aimed to describe the extent of adverse events reporting in published RCTs of Spinal Manipulative Therapy (SMT), and to determine whether the quality of reporting has improved since publication of the 2010 Consolidated Standards Of Reporting Trials (CONSORT) statement.

The Physiotherapy Evidence Database and the Cochrane Central Register of Controlled Trials were searched for RCTs involving SMT. Domains of interest included classifications of adverse events, completeness of adverse events reporting, nomenclature used to describe the events, methodological quality of the study, and details of the publishing journal. Data were analysed using descriptive statistics. Frequencies and proportions of trials reporting on each of the specified domains above were calculated. Differences in proportions between pre- and post-CONSORT trials were calculated with 95% confidence intervals using standard methods, and statistical comparisons were analysed using tests for equality of proportions with continuity correction.

Of 7,398 records identified in the electronic searches, 368 articles were eligible for inclusion in this review. Adverse events were reported in 140 (38.0%) articles. There was a significant increase in the reporting of adverse events post-CONSORT (p=.001). There were two major adverse events reported (0.3%). Only 22 articles (15.7%) reported on adverse events in the abstract. There were no differences in reporting of adverse events post-CONSORT for any of the chosen parameters.

The authors concluded that although there has been an increase in reporting adverse events since the introduction of the 2010 CONSORT guidelines, the current level should be seen as inadequate and unacceptable. We recommend that authors adhere to the CONSORT statement when reporting adverse events associated with RCTs that involve SMT.

We conducted a very similar analysis back in 2012. Specifically, we evaluated all 60 RCTs of chiropractic SMT published between 2000 and 2011 and found that 29 of them did not mention adverse effects at all. Sixteen RCTs reported that no adverse effects had occurred (which I find hard to believe since reliable data show that about 50% of patients experience adverse effects after consulting a chiropractor). Complete information on incidence, severity, duration, frequency and method of reporting of adverse effects was included in only one RCT. Conflicts of interests were not mentioned by the majority of authors. Our conclusion was that adverse effects are poorly reported in recent RCTs of chiropractic manipulations.

The new paper suggests that the situation has improved a little, yet it is still wholly unacceptable. To conduct a clinical trial and fail to mention adverse effects is not, as the authors of the new article suggest, against current guidelines; it is a clear and flagrant violation of medical ethics. I blame the authors of such papers, the reviewers and the journal editors for behaving dishonourably and urge them to get their act together.

The effects of such non-reporting are obvious: anyone looking at the evidence (for instance via systematic reviews) will get a false-positive impression of the safety of SMT. Consequently, chiropractors are able to claim that very few adverse effects have been reported in the literature, therefore our hallmark therapy SMT is demonstrably safe. Those who claim otherwise are quite simply alarmist.

A recent post discussed a ‘STATE OF THE ART REVIEW’ from the BMJ. When I wrote it, I did not know that there was more to come. It seems that the BMJ is planning an entire series on the state of the art of BS! The new paper certainly looks like it:

Headaches, including primary headaches such as migraine and tension-type headache, are a common clinical problem. Complementary and integrative medicine (CIM), formerly known as complementary and alternative medicine (CAM), uses evidence informed modalities to assist in the health and healing of patients. CIM commonly includes the use of nutrition, movement practices, manual therapy, traditional Chinese medicine, and mind-body strategies. This review summarizes the literature on the use of CIM for primary headache and is based on five meta-analyses, seven systematic reviews, and 34 randomized controlled trials (RCTs). The overall quality of the evidence for CIM in headache management is generally low and occasionally moderate. Available evidence suggests that traditional Chinese medicine including acupuncture, massage, yoga, biofeedback, and meditation have a positive effect on migraine and tension headaches. Spinal manipulation, chiropractic care, some supplements and botanicals, diet alteration, and hydrotherapy may also be beneficial in migraine headache. CIM has not been studied or it is not effective for cluster headache. Further research is needed to determine the most effective role for CIM in patients with headache.

My BS-detector struggled with the following statements:

  • integrative medicine (CIM), formerly known as complementary and alternative medicine (CAM) – the fact that CIM is a nonsensical new term has been already mentioned in the previous post;
  • evidence informed modalities – another new term! evidence-BASED would be too much? because it would require using standards that do not apply to CIM? double standards promoted by the BMJ, what next?
  • CIM commonly includes the use of nutrition – yes, so does any healthcare or indeed life!
  • the overall quality of the evidence for CIM in headache management is generally low and occasionally moderate – in this case, no conclusions should be drawn from it (see below);
  • evidence suggests that traditional Chinese medicine including acupuncture, massage, yoga, biofeedback, and meditation have a positive effect on migraine and tension headaches – no, it doesn’t (see above)!
  • further research is needed to determine the most effective role for CIM in patients with headache – this sentence does not even make the slightest sense to me; have the reviewers of this article been asleep?

And this is just the abstract!

The full text provides enough BS to fertilise many acres of farmland!

Moreover, the article is badly researched, cherry-picked, poorly constructed, devoid of critical input, and poorly written. Is there anything good about it? You tell me – I did not find much!

My BS-detector finally broke when we came to the conclusions:

The use of CIM therapies has the potential to empower patients and help them take an active role in their care. Many CIM modalities, including mind-body therapies, are both self selected and self administered after an education period. This, coupled with patients’ increased desire to incorporate integrative medicine, should prompt healthcare providers to consider and discuss its inclusion in the overall management strategy. Low to moderate quality evidence exists for the effectiveness of some CIM therapies in the management of primary headache. The evidence for and use of CIM is continuously changing so healthcare professionals should direct their patients to reliable and updated resources, such as NCCIH.

WHAT IS HAPPENING TO THE BMJ?

IT USED TO BE A GOOD JOURNAL!

Shiatsu is one of those alternative therapies where there is almost no research. Therefore, every new study is of interest, and I was delighted to find this new trial.

Italian researchers tested the efficacy and safety of combining shiatsu and amitriptyline to treat refractory primary headaches in a single-blind, randomized, pilot study. Subjects with a diagnosis of primary headache and who experienced lack of response to ≥2 different prophylactic drugs were randomized in a 1:1:1 ratio to receive one of the following treatments:

  1. shiatsu plus amitriptyline,
  2. shiatsu alone,
  3. amitriptyline alone

The treatment period lasted 3 months and the primary endpoint was the proportion of patients experiencing ≥50%-reduction in headache days. Secondary endpoints were days with headache per month, visual analogue scale, and number of pain killers taken per month.

After randomization, 37 subjects were allocated to shiatsu plus amitriptyline (n = 11), shiatsu alone (n = 13), and amitriptyline alone (n = 13). Randomization ensured well-balanced demographic and clinical characteristics at baseline.

The results show that all the three groups improved in terms of headache frequency, visual analogue scale score, and number of pain killers and there was no between-group difference in the primary endpoint. Shiatsu (alone or in combination) was superior to amitriptyline in reducing the number of pain killers taken per month. Seven (19%) subjects reported adverse events, all attributable to amitriptyline, while no side effects were related with shiatsu treatment.

The authors concluded that shiatsu is a safe and potentially useful alternative approach for refractory headache. However, there is no evidence of an additive or synergistic effect of combining shiatsu and amitriptyline. These findings are only preliminary and should be interpreted cautiously due to the small sample size of the population included in our study.

Yes, I would advocate great caution indeed!

The results could easily be said to demonstrate that shiatsu is NOT effective. There is NO difference between the groups when looking at the primary endpoint. This plus the lack of a placebo-group renders the findings uninterpretable:

  • If we take the comparison 2 versus 3, this might indicate efficacy of shiatsu.
  • If we take the comparison 1 versus 3, it would indicate the opposite.
  • If we finally take the comparison 1 versus 2, it would suggest that the drug was ineffective.

So, we can take our pick!

Moreover, I do object to the authors’ conclusion that “shiatsu is a safe”. For such a statement, we would need sample sizes that are about two dimensions greater that those of this study.

So, what might be an acceptable conclusion from this trial? I see only one that is in accordance with the design and the results of this study:

 

POORLY DESIGNED RESEARCH CANNOT LEAD TO ANY CONCLUSIONS ABOUT THERAPEUTIC EFFICACY OR SAFETY. IT IS A WASTE OF RESOURCES AND A VIOLATION OF RESEARCH ETHICAL.

The aim of this pragmatic study was “to investigate the effectiveness of acupuncture in addition to routine care in patients with allergic asthma compared to treatment with routine care alone.”

Patients with allergic asthma were included in a controlled trial and randomized to receive up to 15 acupuncture sessions over 3 months plus routine care, or to a control group receiving routine care alone. Patients who did not consent to randomization received acupuncture treatment for the first 3 months and were followed as a cohort. All trial patients were allowed to receive routine care in addition to study treatment. The primary endpoint was the asthma quality of life questionnaire (AQLQ, range: 1–7) at 3 months. Secondary endpoints included general health related to quality of life (Short-Form-36, SF-36, range 0–100). Outcome parameters were assessed at baseline and at 3 and 6 months.

A total of 1,445 patients were randomized and included in the analysis (184 patients randomized to acupuncture plus routine care and 173 to routine care alone, and 1,088 in the nonrandomized acupuncture plus routine care group). In the randomized part, acupuncture was associated with an improvement in the AQLQ score compared to the control group (difference acupuncture vs. control group 0.7 [95% confidence interval (CI) 0.5–1.0]) as well as in the physical component scale and the mental component scale of the SF-36 (physical: 2.5 [1.0–4.0]; mental 4.0 [2.1–6.0]) after 3 months. Treatment success was maintained throughout 6 months. Patients not consenting to randomization showed similar improvements as the randomized acupuncture group.

The authors concluded that in patients with allergic asthma, additional acupuncture treatment to routine care was associated with increased disease-specific and health-related quality of life compared to treatment with routine care alone.

We have been over this so many times (see for instance here, here and here) that I am almost a little embarrassed to explain it again: it is fairly easy to design an RCT such that it can only produce a positive result. The currently most popular way to achieve this aim in alternative medicine research is to do a ‘A+B versus B’ study, where A = the experimental treatment, and B = routine care. As A always amounts to more than nothing – in the above trial acupuncture would have placebo effects and the extra attention would also amount to something – A+B must always be more than B alone. The easiest way of thinking of this is to imagine that A and B are both finite amounts of money; everyone can understand that A+B must always be more than B!

Why then do acupuncture researchers not get the point? Are they that stupid? I happen to know some of the authors of the above paper personally, and I can assure you, they are not stupid!

So, why?

I am afraid there is only one reason I can think of: they know perfectly well that such an RCT can only produce a positive finding, and precisely that is their reason for conducting such a study. In other words, they are not using science to test a hypothesis, they deliberately abuse it to promote their pet therapy or hypothesis.

As I stated above, it is fairly easy to design an RCT such that it can only produce a positive result. Yet, it is arguably also unethical, perhaps even fraudulent, to do this. In my view, such RCTs amount to pseudoscience and scientific misconduct.

‘What Doctors Don’t Tell You’ (WDDTY) have been shown to be strangely economical with the truth many times before (for instance here, here and here). Now they have published an article entitled ‘Ombudsman investigates ‘flawed’ homeopathic study that claimed it doesn’t work’ It attacks in no uncertain terms the ‘NHMRC Statement on Homeopathy and NHMRC Information Paper – Evidence on the effectiveness of homeopathy for treating health conditions’ which I believe to be a sound evaluation of homeopathy and therefore have mentioned repeatedly on this blog. Here is what WDDTY stated:

START OF QUOTE

A major and influential review of homeopathy concluded that the controversial therapy doesn’t work—but it was so riddled with error and bad science that it’s sparked an official ombudsman investigation.

The world’s media announced that homeopathy was a scam after the Australia government’s National Health and Medical Research Council (NHMRC) published its findings in 2015 that “there are no health conditions for which there is reliable evidence that homeopathy is effective.”

But now the Commonwealth Ombudsman is investigating the review’s procedures after receiving reports of inaccuracies, mishandling of evidence and conflicts of interest.

The review has been triggered by the Australian Homeopathic Association (AHA), supported by the Homeopathic Research Institute (HRI), which began questioning the review’s processes after several solid studies that demonstrated homeopathy’s benefits had been overlooked.

The NHMRC review team set arbitrary parameters that only studies that involved more than 150 people—and which met standards that even drug trials rarely achieve—would be considered. Those requirements reduced the number of qualifying studies to just five—from an initial pool of more than 1,800 trials—and none of these showed that homeopathy was effective.

One of the NHMRC’s own reviewers produced a mysterious first report that has never been published, and hasn’t been released despite Freedom of Information requests.

And the AHA has discovered that Prof Peter Brooks, chair of the NHMRC committee that carried out the homeopathy review, never declared that he was a member of the anti-homeopathy lobby group, Friends of Science in Medicine.

There are solid studies that demonstrate homeopathy is effective against childhood diarrhea, sinusitis and hay fever—but they all involve fewer than 150 people, said HRI chief executive Rachel Roberts. “The public has a right to know that there are high quality studies showing homeopathy works for some medical conditions—information that was lost only due to NHMRC’s mishandling of the evidence.”

The homeopaths aren’t alone in challenging the NHMRC review: Australia’s independent Cochrane Centre said its conclusions are not an accurate reflection of the evidence, and a second expert also said he felt “uncertain of the definitive nature of the report’s conclusions.”

END OF QUOTE

As it happens, I am in contact with the lead author of this report, Paul Glasziou, not least because he very kindly wrote the foreword for my book HOMEOPATHY, THE UNDILUTED FACTS. So, we corresponded and discussed the latest WDDTY diatribe. Thus I am now in a position to put a few things straight (I hope Paul does not mind).

ISSUE 1. – The NHMRC review team set arbitrary parameters that only studies involving more than 150 people—and which met standards that even drug trials rarely achieve—would be considered.

The truth is that report focused on systematic reviews of trials, not individual trials. The 57 included systematic reviews found 176 individual trials which covered 61 conditions: an average of about 3 trials per condition. But some conditions only had 1 trial, and one small trial would, of course, not be considered a reasonable basis for reliable conclusions. GRADE – the international standard for assessing evidence – downgrades reviews for “imprecision” – the GRADE Handbook suggests “whenever there are sample sizes that are less than 400, review authors and guideline developers should certainly consider rating down for imprecision.” Hence the criterion of 150 which the Australians decided to use is considerably more lenient than the current GRADE guideline.

ISSUE 2 – Those requirements reduced the number of qualifying studies to just five—from an initial pool of more than 1,800 trials—and none of these showed that homeopathy was effective.

This is simply not correct. The report found 57 systematic reviews that contained 176 individual trials, not 5. These 176 trials, which covered 61 conditions, formed the body of evidence for the NHMRC report’s conclusions.

ISSUE 3 – There are solid studies that demonstrate homeopathy is effective against childhood diarrhoea, sinusitis and hay fever—but they all involve fewer than 150 people, said HRI chief executive Rachel Roberts.

The NHMRC report focused on systematic reviews that covered all trials for individual conditions. Given the conventional p-value of 0.05, one would expect 1 in 20 single trials to be “false positives”. So with 176 trials, we expect about 9 “false positive” trials. But using systematic reviews that combine all trials for individual conditions, reduces this risk of false positives. Most national evidence review bodies require more than 1 trial, e.g, the FDA requires 2 positive trials, whereas many others require a systematic review which has at least 2 trials. Replication of findings is obviously a cornerstone of science.

ISSUE 4 The homeopaths aren’t alone in challenging the NHMRC review: Australia’s independent Cochrane Centre said its conclusions are not an accurate reflection of the evidence, and a second expert also said he felt “uncertain of the definitive nature of the report’s conclusions.”

The truth is that the Cochrane Centre, which provided an independent check during the processes of the NHMRC review, concluded that “Overall, the conclusions arising from the review appear justified based on the evidence presented.”

I REST MY CASE.

The title of the press-release was impressive: ‘Columbia and Harvard Researchers Find Yoga and Controlled Breathing Reduce Depressive Symptoms’. It certainly awoke my interest and I looked up the original article. Sadly, it also awoke the interest of many journalists, and the study was reported widely – and, as we shall see, mostly wrongly.

According to its authors, the aims of this study were “to assess the effects of an intervention of Iyengar yoga and coherent breathing at five breaths per minute on depressive symptoms and to determine optimal intervention yoga dosing for future studies in individuals with major depressive disorder (MDD)”.

Thirty two subjects were randomized to either the high-dose group (HDG) or low-dose group (LDG) for a 12-week intervention of three or two intervention classes per week, respectively. Eligible subjects were 18–64 years old with MDD, had baseline Beck Depression Inventory-II (BDI-II) scores ≥14, and were either on no antidepressant medications or on a stable dose of antidepressants for ≥3 months. The intervention included 90-min classes plus homework. Outcome measures were BDI-II scores and intervention compliance.

Fifteen HDG and 15 LDG subjects completed the intervention. BDI-II scores at screening and compliance did not differ between groups. BDI-II scores declined significantly from screening (24.6 ± 1.7) to week 12 (6.0 ± 3.8) for the HDG (–18.6 ± 6.6; p < 0.001), and from screening (27.7 ± 2.1) to week 12 (10.1 ± 7.9) in the LDG. There were no significant differences between groups, based on response (i.e., >50% decrease in BDI-II scores; p = 0.65) for the HDG (13/15 subjects) and LDG (11/15 subjects) or remission (i.e., number of subjects with BDI-II scores <14; p = 1.00) for the HDG (14/15 subjects) and LDG (13/15 subjects) after the 12-week intervention, although a greater number of subjects in the HDG had 12-week BDI-II scores ≤10 (p = 0.04).

The authors concluded that this dosing study provides evidence that participation in an intervention composed of Iyengar yoga and coherent breathing is associated with a significant reduction in depressive symptoms for individuals with MDD, both on and off antidepressant medications. The HDG and LDG showed no significant differences in compliance or in rates of response or remission. Although the HDG had significantly more subjects with BDI-II scores ≤10 at week 12, twice weekly classes (plus home practice) may rates of response or remission. Although the HDG, thrice weekly classes (plus home practice) had significantly more subjects with BDI-II scores ≤10 at week 12, the LDG, twice weekly classes (plus home practice) may constitute a less burdensome but still effective way to gain the mood benefits from the intervention. This study supports the use of an Iyengar yoga and coherent breathing intervention as a treatment to alleviate depressive symptoms in MDD.

The authors also warn that this study must be interpreted with caution and point out several limitations:

  • the small sample size,
  • the lack of an active non-yoga control (both groups received Iyengar yoga plus coherent breathing),
  • the supportive group environment and multiple subject interactions with research staff each week could have contributed to the reduction in depressive symptoms,
  • the results cannot be generalized to MDD with more acute suicidality or more severe symptoms.

In the press-release, we are told that “The practical findings for this integrative health intervention are that it worked for participants who were both on and off antidepressant medications, and for those time-pressed, the two times per week dose also performed well,” says The Journal of Alternative and Complementary Medicine Editor-in-Chief John Weeks

At the end of the paper, we learn that the authors, Dr. Brown and Dr. Gerbarg, teach and have published Breath∼Body∼Mind©, a technique that uses coherent breathing. Dr. Streeter is certified to teach Breath∼Body∼Mind©. No competing financial interests exist for the remaining authors.

Taking all of these issues into account, my take on this study is different and a little more critical:

  • The observed effects might have nothing at all to do with the specific intervention tested.
  • The trial was poorly designed.
  • The aims of the study are not within reach of its methodology.
  • The trial lacked a proper control group.
  • It was published in a journal that has no credibility.
  • The limitations outlined by the authors are merely the tip of an entire iceberg of fatal flaws.
  • The press-release is irresponsibly exaggerated.
  • The authors have little incentive to truly test their therapy and seem to use research as a means of promoting their business.
1 2 3 7

Gravityscan Badge

Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted.


Click here for a comprehensive list of recent comments.

Categories