The question whether infant colic can be effectively treated with manipulative therapies might seem rather trivial – after all, this is a benign condition which the infant quickly grows out of. However, the issue becomes a little more tricky, if we consider that it was one of the 6 paediatric illnesses which were at the centre of the famous libel case of the BCA against my friend and co-author Simon Singh. At the time, Simon had claimed that there was ‘not a jot of evidence’ for claiming that chiropractic was an effective treatment of infant colic, and my systematic review of the evidence strongly supported his statement. The BCA eventually lost their libel case and with it the reputation of chiropractic. Now a new article on this intriguing topic has become available; do we have to reverse our judgements?
The aim of this new systematic review was to evaluate the efficacy or effectiveness of manipulative therapies for infantile colic. Six RCTs of chiropractic, osteopathy or cranial osteopathy alone or in conjunction with other interventions were included with a total of 325 infants. Of the 6 included studies, 5 were “suggestive of a beneficial effect” and one found no evidence of benefit. Combining all the RCTs suggested that manipulative therapies had a significant effect. The average crying time was reduced by an average of 72 minutes per day. This effect was sustained for studies with a low risk of selection bias and attrition bias. When analysing only those studies with a low risk of performance bias (i.e. parental blinding) the improvement in daily crying hours was no longer statistically significant.
The quality of the studies was variable. There was a generally low risk of selection bias but a high risk of performance bias. Only one of the studies recorded adverse events and none were encountered.
From these data, the authors drew the following conclusion: Parents of infants receiving manipulative therapies reported fewer hours crying per day than parents whose infants did not and this difference was statistically significant. Most studies had a high risk of performance bias due to the fact that the assessors (parents) were not blind to who had received the intervention. When combining only those trials with a low risk of such performance bias the results did not reach statistical significance.
Does that mean that chiropractic does work for infant colic? No, it does not!
The first thing to point out is that the new systematic review included not just RCTs of chiropractic but also osteopathy and cranio-sacral therapy.
The second important issue is that the effects disappear, once performance bias is being accounted for which clearly shows that the result is false positive.
The third relevant fact is that the majority of the RCTs were of poor quality. The methodologically best studies were negative.
And the fourth thing to note is that only one study mentioned adverse effects, which means that the other 5 trials were in breach of one of rather elementary research ethics.
What makes all of this even more fascinating is the fact that the senior author of the new publication, George Lewith, is the very expert who advised the BCA in their libel case against Simon Singh. He seems so fond of his work that he even decided to re-publish it using even more misleading language than before. It is, of course, far from me to suggest that his review was an attempt to white-wash the issue of chiropractic ‘bogus’ claims. However, based on the available evidence, I would have formulated conclusions which are more than just a little different from his; something like this perhaps:
The current best evidence suggests that the small effects that emerge when we pool the data from mostly unreliable studies are due to bias and therefore not real. This systematic review therefore fails to show that manipulative therapies are effective. It furthermore points to a serious breach of research ethics by the majority of researchers in this field.
Many reader of this blog will remember the libel case of the British Chiropractic Association (BCA) against Simon Singh. Simon had disclosed in a Guardian comment that the BCA was happily promoting bogus chiropractic treatments for 6 paediatric conditions, including infant colic. The BCA not only lost the case but the affair almost destroyed this strange organisation and resulted in an enormous reputational damage of chiropractors worldwide. In an article entitled AFTER THE STORM, the then-president of the BCA later described the defeat in his own words: “in 2009, events in the UK took a turn which was to consume the British Chiropractic Association (BCA) for two years and force the wider profession to confront key issues that for decades had kept it distanced from its medical counterparts and attracting ridicule from its critics…the BCA began one of the darkest periods in its history; one that was ultimately to cost it financially, reputationally and politically…The GCC itself was in an unprecedented situation. Faced with a 1500% rise in complaints, Investigating Committees were assembled to determine whether there was a case to answer…The events of the past two years have exposed a blind adherence to outdated principles amongst a small but significant minority of the profession. Mindful of the adage that it’s the squeaky wheel that gets the grease, the vocalism of this group has ensured that chiropractic is characterised by its critics as unscientific, unsafe and slightly wacky. Claims that the vertebral subluxation complex is the cause of illness and disease have persisted despite the three UK educational establishments advising the GCC that no evidence of acceptable quality exists to support such claims.”
Only a few years AFTER THE STORM, this story seems to have changed beyond recognition. Harald Walach, who is known to readers of this blog because I reported that he was elected ‘pseudo-scientist of the year’ in 2012, recently published a comment on the proceedings of the European Congress of Integrated Medicine where we find the following intriguing version of the libel case:
Mein Freund und Kollege George Lewith aus Southampton hatte einen Hauptvortrag über seine Überblicksarbeit über chiropraktische Interventionen für kleinkindliche Koliken vorgelegt. Sie ist ausgelöst worden durch die Behauptung, die Singh und Ernst vor einigen Jahren erhoben hatten, dass Chiropraktik gefährlich ist, dass es keine Daten dafür gäbe, dass sie wirksam sei und dass sie gefährliche Nebenwirkungen habe, speziell wenn sie bei Kindern angewendet würde. Die Chiropraktiker hatten den Wissenschaftsjournalisten Singh damals wegen Verleumdung verklagt und recht erhalten. George Lewith hatte dem Gericht die Expertise geliefert und nun seine Analyse auf Kinder ausgedehnt.
Kurz gefasst: Die Intervention wirkt sogar ziemlich stark, etwa eine Standardabweichung war der Effekt groß. Die Kinder schreien kürzer und weniger. Und die Durchforstung der Literatur nach gefährlichen Nebenwirkungen hatte keinen, wortwörtlich: nicht einen, Fall zu Tage gefördert, der von Nebenwirkungen, geschweige denn gefährlichen, berichtet hätte. Die Aufregung war seinerzeit dadurch entstanden, dass eine unqualifizierte Person einer zart gebauten Frau über den Rücken gelaufen ist und ihr dabei das Genick gebrochen hat. Die Presse hatte das ganze dann zu „tödlicher Nebenwirkung chiropraktischer Intervention“ aufgebauscht.
Oh, I almost forgot, you don’t read German? Here is my translation of this revealing text:
“My friend and colleague Geoorge Lewith from Southampton gave a keynote lecture on his review of chiropractic interventions for infant colic. This was prompted by the claim, made by Singh and Ernst a few years ago, that chiropractic was dangerous, that no data existed showing its effectiveness, and that it had dangerous side-effects, particularly for children. The chiropractors had sued the science journalist Singh for libel and won the case. George Lewith had provided the expert report for the court and has now extended his analysis on children.
To put it briefly: the intervention is even very effective; the effect-size is about one standard deviation. The children cry less long and more rarely. And the search of the literature for dangerous side-effects resulted in no – literally: not one – case of side-effects, not to mention dangerous ones. The fuzz had started back then because an unqualified person had walked over the back of a thin woman and had thus broken her neck. The press had subsequently hyped the whole thing to a “deadly side-effect of a chiropractic intervention”. (I am sorry for the clumsy language but the original is even worse.)
Now, isn’t that remarkable? Not only has the truth about the libel case been turned upside down, but also the evidence on chiropractic as a treatment for infant colic seems mysteriously improved; other reviews which might just be a bit more independent and objective come to the following conclusions:
The literature concerning this topic is surprisingly scarce, of poor quality and lack of convincing conclusions. With the present day data on this topic, it is impossible to say whether this kind of treatment has a significant effect.
And what should we make of all this? I don’t know about you, but I conclude that, for some apologists of alternative medicine, the truth is a rather flexible commodity.
A recent post of mine seems to have stimulated a lively discussion about the question IS THERE ANY GOOD EVIDENCE AT ALL FOR OSTEOPATHIC TREATMENTS? By and large, osteopaths commented that they are well aware that their signature interventions for their most frequently treated condition (back pain) lack evidential support and that more research is needed. At the same time, many osteopaths seemed to see little wrong in making unsubstantiated therapeutic claims. I thought this was remarkable and feel encouraged to write another post about a similar topic.
Most osteopaths treat children for a wide range of conditions and claim that their interventions are helpful. They believe that children are prone to structural problems which can be corrected by their interventions. Here is an example from just one of the numerous promotional websites on this topic:
STRUCTURAL PROBLEMS, such as those affecting the proper mobility and function of the body’s framework, can lead to a range of problems. These may include:
- Postural – such as scoliosis
- Respiratory – such as asthma
- Manifestations of brain injury – such as cerebral palsy and spasticity
- Developmental – with delayed physical or intellectual progress, perhaps triggering learning behaviour difficulties
- Infections – such as ear and throat infections or urinary disturbances, which may be recurrent.
OSTEOPATHY can assist in the prevention of health problems, helping children to make a smooth transition into normal, healthy adult life.
As children cannot give informed consent, this is even more tricky than treating adults with therapies of questionable value. It is therefore important, I think, to ask whether osteopathic treatments of children is based on evidence or just on wishful thinking or the need to maximise income. As it happens, my team just published an article about these issues in one of the highest-ranking paediatrics journal.
The objective of our systematic review was to critically evaluate the effectiveness of osteopathic manipulative treatment (OMT) as a treatment of paediatric conditions. Eleven databases were searched from their respective inceptions to November 2012. Only randomized clinical trials (RCTs) were included, if they tested OMT against any type of control intervention in paediatric patients. The quality of all included RCTs was assessed using the Cochrane criteria.
Seventeen trials met our inclusion criteria. Only 5 RCTs were of high methodological quality. Of those, 1 favoured OMT, whereas 4 revealed no effect compared with various control interventions. Replications by independent researchers were available for two conditions only, and both failed to confirm the findings of the previous studies. Seven RCTs suggested that OMT leads to a significantly greater reduction in the symptoms of asthma, congenital nasolacrimal duct obstruction, daily weight gain and length of hospital stay, dysfunctional voiding, infantile colic, otitis media, or postural asymmetry compared with various control interventions. Seven RCTs indicated that OMT had no effect on the symptoms of asthma, cerebral palsy, idiopathic scoliosis, obstructive apnoea, otitis media, or temporo-mandibular disorders compared with various control interventions. Three RCTs did not report between-group comparisons. The majority of the included RCTs did not report the incidence rates of adverse-effects.
Our conclusion is likely to again dissatisfy many osteopaths: The evidence of the effectiveness of OMT for paediatric conditions remains unproven due to the paucity and low methodological quality of the primary studies.
So, what does this tell us? I am sure osteopaths will disagree, but I think it shows that for no paediatric condition do we have sufficient evidence to show that OMT is effective. The existing RCTs are mostly of low quality. There is a lack of independent replication of the few studies that suggested a positive outcome. And to make matters even worse, osteopaths seem to be violating the most basic rule of medical research by not reporting adverse-effects in their clinical trials.
I rest my case – at least for the moment.
The question whether spinal manipulation is an effective treatment for infant colic has attracted much attention in recent years. The main reason for this is, of course, that a few years ago Simon Singh had disclosed in a comment that the British Chiropractic Association (BCA) was promoting chiropractic treatment for this and several other childhood condition on their website. Simon famously wrote “they (the BCA) happily promote bogus treatments” and was subsequently sued for libel by the BCA. Eventually, the BCA lost the libel action as well as lots of money, and the entire chiropractic profession ended up with enough egg on their faces to cook omelets for all their patients.
At the time, the BCA had taken advice from several medical and legal experts; one of their medical advisers, I was told, was Prof George Lewith. Intriguingly, he and several others have just published a Cochrane review of manipulative therapies for infant colic. Here are the unabbreviated conclusions from their article:
“The studies included in this meta-analysis were generally small and methodologically prone to bias, which makes it impossible to arrive at a definitive conclusion about the effectiveness of manipulative therapies for infantile colic. The majority of the included trials appeared to indicate that the parents of infants receiving manipulative therapies reported fewer hours crying per day than parents whose infants did not, based on contemporaneous crying diaries, and this difference was statistically significant. The trials also indicate that a greater proportion of those parents reported improvements that were clinically significant. However, most studies had a high risk of performance bias due to the fact that the assessors (parents) were not blind to who had received the intervention. When combining only those trials with a low risk of such performance bias, the results did not reach statistical significance. Further research is required where those assessing the treatment outcomes do not know whether or not the infant has received a manipulative therapy. There are inadequate data to reach any definitive conclusions about the safety of these interventions”
Cochrane reviews also carry a “plain language” summary which might be easier to understand for lay people. And here are the conclusions from this section of the review:
The studies involved too few participants and were of insufficient quality to draw confident conclusions about the usefulness and safety of manipulative therapies. Although five of the six trials suggested crying is reduced by treatment with manipulative therapies, there was no evidence of manipulative therapies improving infant colic when we only included studies where the parents did not know if their child had received the treatment or not. No adverse effects were found, but they were only evaluated in one of the six studies.
If we read it carefully, this article seems to confirm that there is no reliable evidence to suggest that manipulative therapies are effective for infant colic. In the analyses, the positive effect disappears, if the parents are properly blinded; thus it is due to expectation or placebo. The studies that seem to show a positive effect are false positive, and spinal manipulation is, in fact, not effective.
The analyses disclose another intriguing aspect: most trials failed to mention adverse effects. This confirms the findings of our own investigation and amounts to a remarkable breach of publication ethics (nobody seems to be astonished by this fact; is it normal that chiropractic researchers ignore generally accepted rules of ethics?). It also reflects badly on the ability of the investigators of the primary studies to be objective. They seem to aim at demonstrating only the positive effects of their intervention; science is, however, not about confirming the researchers’ prejudices, it is about testing hypotheses.
The most remarkable thing about the new Cochrane review is, I think, the in-congruence of the actual results and the authors’ conclusion. To a critical observer, the former are clearly negative but the latter sound almost positive. I think this begs the question about the possibility of reviewer bias.
We have recently discussed on this blog whether reviews by one single author are necessarily biased. The new Cochrane review has 6 authors, and it seems to me that its conclusions are considerably more biased than my single-author review of chiropractic spinal manipulation for infant colic; in 2009, I concluded simply that “the claim [of effectiveness] is not based on convincing data from rigorous clinical trials”.
Which of the two conclusions describe the facts more helpfully and more accurately?
I think, I rest my case.