The question whether infant colic can be effectively treated with manipulative therapies might seem rather trivial – after all, this is a benign condition which the infant quickly grows out of. However, the issue becomes a little more tricky, if we consider that it was one of the 6 paediatric illnesses which were at the centre of the famous libel case of the BCA against my friend and co-author Simon Singh. At the time, Simon had claimed that there was ‘not a jot of evidence’ for claiming that chiropractic was an effective treatment of infant colic, and my systematic review of the evidence strongly supported his statement. The BCA eventually lost their libel case and with it the reputation of chiropractic. Now a new article on this intriguing topic has become available; do we have to reverse our judgements?
The aim of this new systematic review was to evaluate the efficacy or effectiveness of manipulative therapies for infantile colic. Six RCTs of chiropractic, osteopathy or cranial osteopathy alone or in conjunction with other interventions were included with a total of 325 infants. Of the 6 included studies, 5 were “suggestive of a beneficial effect” and one found no evidence of benefit. Combining all the RCTs suggested that manipulative therapies had a significant effect. The average crying time was reduced by an average of 72 minutes per day. This effect was sustained for studies with a low risk of selection bias and attrition bias. When analysing only those studies with a low risk of performance bias (i.e. parental blinding) the improvement in daily crying hours was no longer statistically significant.
The quality of the studies was variable. There was a generally low risk of selection bias but a high risk of performance bias. Only one of the studies recorded adverse events and none were encountered.
From these data, the authors drew the following conclusion: Parents of infants receiving manipulative therapies reported fewer hours crying per day than parents whose infants did not and this difference was statistically significant. Most studies had a high risk of performance bias due to the fact that the assessors (parents) were not blind to who had received the intervention. When combining only those trials with a low risk of such performance bias the results did not reach statistical significance.
Does that mean that chiropractic does work for infant colic? No, it does not!
The first thing to point out is that the new systematic review included not just RCTs of chiropractic but also osteopathy and cranio-sacral therapy.
The second important issue is that the effects disappear, once performance bias is being accounted for which clearly shows that the result is false positive.
The third relevant fact is that the majority of the RCTs were of poor quality. The methodologically best studies were negative.
And the fourth thing to note is that only one study mentioned adverse effects, which means that the other 5 trials were in breach of one of rather elementary research ethics.
What makes all of this even more fascinating is the fact that the senior author of the new publication, George Lewith, is the very expert who advised the BCA in their libel case against Simon Singh. He seems so fond of his work that he even decided to re-publish it using even more misleading language than before. It is, of course, far from me to suggest that his review was an attempt to white-wash the issue of chiropractic ‘bogus’ claims. However, based on the available evidence, I would have formulated conclusions which are more than just a little different from his; something like this perhaps:
The current best evidence suggests that the small effects that emerge when we pool the data from mostly unreliable studies are due to bias and therefore not real. This systematic review therefore fails to show that manipulative therapies are effective. It furthermore points to a serious breach of research ethics by the majority of researchers in this field.
The dismal state of chiropractic research is no secret. But is anything being done about it? One important step would be to come up with a research strategy to fill the many embarrassing gaps in our knowledge about the validity of the concepts underlying chiropractic.
A brand-new article might be a step in the right direction. The aim of this survey was to identify chiropractors’ priorities for future research in order to best channel the available resources and facilitate advancement of the profession. The researchers recruited 60 academic and clinician chiropractors who had attended any of the annual European Chiropractors’ Union/European Academy of Chiropractic Researchers’ Day meetings since 2008. A Delphi process was used to identify a list of potential research priorities. Initially, 70 research priorities were identified, and 19 of them reached consensus as priorities for future research. The following three items were thought to be most important:
- cost-effectiveness/economic evaluations,
- identification of subgroups likely to respond to treatment,
- initiation and promotion of collaborative research activities.
The authors state that this is the first formal and systematic attempt to develop a research agenda for the chiropractic profession in Europe. Future discussion and study is necessary to determine whether the themes identified in this survey should be broadly implemented.
Am I the only one who finds these findings extraordinary?
The chiropractic profession only recently lost the libel case against Simon Singh who had disclosed that chiropractors HAPPILY PROMOTE BOGUS TREATMENTS. One would have thought that this debacle might prompt the need for rigorous research testing the many unsubstantiated claims chiropractors still make. Alas, the collective chiropractic wisdom does not consider such research as a priority!
Similarly, I would have hoped that chiropractors perceive an urgency to investigate the safety of their treatments. Serious complications after spinal manipulation are well documented, and I would have thought that any responsible health care profession would consider it essential to generate reliable evidence on the incidence of such events.
The fact that these two areas are not considered to be priorities is revealing. In my view, it suggests that chiropractic is still very far from becoming a mature and responsible profession. It seems that chiropractors have not learned the most important lessons from recent events; on the contrary, they continue to bury their heads in the sand and carry on seeing research as a tool for marketing.
Systematic reviews are widely considered to be the most reliable type of evidence for judging the effectiveness of therapeutic interventions. Such reviews should be focused on a well-defined research question and identify, critically appraise and synthesize the totality of the high quality research evidence relevant to that question. Often it is possible to pool the data from individual studies and thus create a new numerical result of the existing evidence; in this case, we speak of a meta-analysis, a sub-category of systematic reviews.
One strength of systematic review is that they minimise selection and random biases by considering at the totality of the evidence of a pre-defined nature and quality. A crucial precondition, however, is that the quality of the primary studies is critically assessed. If this is done well, the researchers will usually be able to determine how robust any given result is, and whether high quality trials generate similar findings as those of lower quality. If there is a discrepancy between findings from rigorous and flimsy studies, it is obviously advisable to trust the former and discard the latter.
And this is where systematic reviews of alternative treatments can run into difficulties. For any given research question in this area we usually have a paucity of primary studies. Equally important is the fact that many of the available trials tend to be of low quality. Consequently, there often is a lack of high quality studies, and this makes it all the more important to include a robust critical evaluation of the primary data. Not doing so would render the overall result of the review less than reliable – in fact, such a paper would not qualify as a systematic review at all; it would be a pseudo-systematic review, i.e. a review which pretends to be systematic but, in fact, is not. Such papers are a menace in that they can seriously mislead us, particularly if we are not familiar with the essential requirements for a reliable review.
This is precisely where some promoters of bogus treatments seem to see their opportunity of making their unproven therapy look as though it was evidence-based. Pseudo-systematic reviews can be manipulated to yield a desired outcome. In my last post, I have shown that this can be done by including treatments which are effective so that an ineffective therapy appears effective (“chiropractic is so much more than just spinal manipulation”). An even simpler method is to exclude some of the studies that contradict one’s belief from the review. Obviously, the review would then not comprise the totality of the available evidence. But, unless the reader bothers to do a considerable amount of research, he/she would be highly unlikely to notice. All one needs to do is to smuggle the paper past the peer-review process – hardly a difficult task, given the plethora of alternative medicine journals that bend over backwards to publish any rubbish as long as it promotes alternative medicine.
Alternatively (or in addition) one can save oneself a lot of work and omit the process of critically evaluating the primary studies. This method is increasingly popular in alternative medicine. It is a fool-proof method of generating a false-positive overall result. As poor quality trials have a tendency to deliver false-positive results, it is obvious that a predominance of flimsy studies must create a false-positive result.
A particularly notorious example of a pseudo-systematic review that used this as well as most of the other tricks for misleading the reader is the famous ‘systematic’ review by Bronfort et al. It was commissioned by the UK GENERAL CHIROPRACTIC COUNCIL after the chiropractic profession got into trouble and was keen to defend those bogus treatments disclosed by Simon Singh. Bronfort and his colleagues thus swiftly published (of course, in a chiro-journal) an all-encompassing review attempting to show that, at least for some conditions, chiropractic was effective. Its lengthy conclusions seemed encouraging: Spinal manipulation/mobilization is effective in adults for: acute, subacute, and chronic low back pain; migraine and cervicogenic headache; cervicogenic dizziness; manipulation/mobilization is effective for several extremity joint conditions; and thoracic manipulation/mobilization is effective for acute/subacute neck pain. The evidence is inconclusive for cervical manipulation/mobilization alone for neck pain of any duration, and for manipulation/mobilization for mid back pain, sciatica, tension-type headache, coccydynia, temporomandibular joint disorders, fibromyalgia, premenstrual syndrome, and pneumonia in older adults. Spinal manipulation is not effective for asthma and dysmenorrhea when compared to sham manipulation, or for Stage 1 hypertension when added to an antihypertensive diet. In children, the evidence is inconclusive regarding the effectiveness for otitis media and enuresis, and it is not effective for infantile colic and asthma when compared to sham manipulation. Massage is effective in adults for chronic low back pain and chronic neck pain. The evidence is inconclusive for knee osteoarthritis, fibromyalgia, myofascial pain syndrome, migraine headache, and premenstrual syndrome. In children, the evidence is inconclusive for asthma and infantile colic.
Chiropractors across the world cite this paper as evidence that chiropractic has at least some evidence base. What they omit to tell us (perhaps because they do not appreciate it themselves) is the fact that Bronfort et al
- failed to formulate a focussed research question,
- invented his own categories of inconclusive findings,
- included all sorts of studies which had nothing to do with chiropractic,
- and did not to make an assessment of the quality of the included primary studies they included in their review.
If, for a certain condition, three trials were included, for instance, two of which were positive but of poor quality and one was negative but of good quality, the authors would conclude that, overall, there is sound evidence.
Bronfort himself is, of course, more than likely to know all that (he has learnt his trade with an excellent Dutch research team and published several high quality reviews) - but his readers mostly don’t. And for chiropractors, this ‘systematic’ review is now considered to be the most reliable evidence in their field.
Many reader of this blog will remember the libel case of the British Chiropractic Association (BCA) against Simon Singh. Simon had disclosed in a Guardian comment that the BCA was happily promoting bogus chiropractic treatments for 6 paediatric conditions, including infant colic. The BCA not only lost the case but the affair almost destroyed this strange organisation and resulted in an enormous reputational damage of chiropractors worldwide. In an article entitled AFTER THE STORM, the then-president of the BCA later described the defeat in his own words: “in 2009, events in the UK took a turn which was to consume the British Chiropractic Association (BCA) for two years and force the wider profession to confront key issues that for decades had kept it distanced from its medical counterparts and attracting ridicule from its critics…the BCA began one of the darkest periods in its history; one that was ultimately to cost it financially, reputationally and politically…The GCC itself was in an unprecedented situation. Faced with a 1500% rise in complaints, Investigating Committees were assembled to determine whether there was a case to answer…The events of the past two years have exposed a blind adherence to outdated principles amongst a small but significant minority of the profession. Mindful of the adage that it’s the squeaky wheel that gets the grease, the vocalism of this group has ensured that chiropractic is characterised by its critics as unscientific, unsafe and slightly wacky. Claims that the vertebral subluxation complex is the cause of illness and disease have persisted despite the three UK educational establishments advising the GCC that no evidence of acceptable quality exists to support such claims.”
Only a few years AFTER THE STORM, this story seems to have changed beyond recognition. Harald Walach, who is known to readers of this blog because I reported that he was elected ‘pseudo-scientist of the year’ in 2012, recently published a comment on the proceedings of the European Congress of Integrated Medicine where we find the following intriguing version of the libel case:
Mein Freund und Kollege George Lewith aus Southampton hatte einen Hauptvortrag über seine Überblicksarbeit über chiropraktische Interventionen für kleinkindliche Koliken vorgelegt. Sie ist ausgelöst worden durch die Behauptung, die Singh und Ernst vor einigen Jahren erhoben hatten, dass Chiropraktik gefährlich ist, dass es keine Daten dafür gäbe, dass sie wirksam sei und dass sie gefährliche Nebenwirkungen habe, speziell wenn sie bei Kindern angewendet würde. Die Chiropraktiker hatten den Wissenschaftsjournalisten Singh damals wegen Verleumdung verklagt und recht erhalten. George Lewith hatte dem Gericht die Expertise geliefert und nun seine Analyse auf Kinder ausgedehnt.
Kurz gefasst: Die Intervention wirkt sogar ziemlich stark, etwa eine Standardabweichung war der Effekt groß. Die Kinder schreien kürzer und weniger. Und die Durchforstung der Literatur nach gefährlichen Nebenwirkungen hatte keinen, wortwörtlich: nicht einen, Fall zu Tage gefördert, der von Nebenwirkungen, geschweige denn gefährlichen, berichtet hätte. Die Aufregung war seinerzeit dadurch entstanden, dass eine unqualifizierte Person einer zart gebauten Frau über den Rücken gelaufen ist und ihr dabei das Genick gebrochen hat. Die Presse hatte das ganze dann zu „tödlicher Nebenwirkung chiropraktischer Intervention“ aufgebauscht.
Oh, I almost forgot, you don’t read German? Here is my translation of this revealing text:
“My friend and colleague Geoorge Lewith from Southampton gave a keynote lecture on his review of chiropractic interventions for infant colic. This was prompted by the claim, made by Singh and Ernst a few years ago, that chiropractic was dangerous, that no data existed showing its effectiveness, and that it had dangerous side-effects, particularly for children. The chiropractors had sued the science journalist Singh for libel and won the case. George Lewith had provided the expert report for the court and has now extended his analysis on children.
To put it briefly: the intervention is even very effective; the effect-size is about one standard deviation. The children cry less long and more rarely. And the search of the literature for dangerous side-effects resulted in no – literally: not one – case of side-effects, not to mention dangerous ones. The fuzz had started back then because an unqualified person had walked over the back of a thin woman and had thus broken her neck. The press had subsequently hyped the whole thing to a “deadly side-effect of a chiropractic intervention”. (I am sorry for the clumsy language but the original is even worse.)
Now, isn’t that remarkable? Not only has the truth about the libel case been turned upside down, but also the evidence on chiropractic as a treatment for infant colic seems mysteriously improved; other reviews which might just be a bit more independent and objective come to the following conclusions:
The literature concerning this topic is surprisingly scarce, of poor quality and lack of convincing conclusions. With the present day data on this topic, it is impossible to say whether this kind of treatment has a significant effect.
And what should we make of all this? I don’t know about you, but I conclude that, for some apologists of alternative medicine, the truth is a rather flexible commodity.
Five years ago to the day, Simon Singh and I published an article in The Daily Mail to promote our book TRICK OR TREATMENT… which was then about to be launched. We recently learnt that our short article prompted a “confidential” message by the BRITISH CHIROPRACTIC ASSOCIATION to all its members. “Confidential” needs to be put in inverted commas because it is readily available on the Internet. I find it fascinating and of sufficient public interest to reproduce it here in full. I have not altered a thing in the following text, except putting it in italics and putting the section where the BCA quote our text in bold for clarity.
CONFIDENTIAL FOR BCA MEMBERS ONLY
Information for BCA members regarding an article in the Daily Mail – April 8th 2008
A double page spread appeared in the edition of the Daily Mail April 8th 2008 on page 46 and 47 and titled ‘Alternative Medicine The Verdict’.
The article was written by Simon Singh and Edzard Ernst and is a publicity prelude to a book they have written called ‘Trick or Treatment? Alternative Medicine on Trial’, which will be published later this month.
The article covers Alexander Technique, Aromatherapy, Flower Remedy, Chiropractic, Hypnotherapy, Magnet Therapy and Osteopathy.
The coverage of Chiropractic follows a familiar pattern for E Ernst. The treatment is oversimplified in explanation, with a heavy emphasis on words like thrust, strong and aggressive. There is tacit acknowledgement that chiropractic works for back pain, but then there is a long section about caution regarding neck manipulation. The article concludes by advising people not to have their neck manipulated and not to allow children to be treated.
WHAT IS IT? Chiropractors use spinal manipulation to realign the spine to restore mobility. Initial examination often includes X-ray images or MRI scans.
Spinal manipulation can be a fairly aggressive technique, which pushes the spinal joint slightly beyond what it is ordinarily capable of achieving, using a technique called high-velocity, low-amplitude thrust – exerting a relatively strong force in order to move the joint at speed, but the extent of the motion needs to be limited to prevent damage to the joint and its surrounding structures.
Although spinal manipulation is often associated with a cracking sound, this is not a result of the bones crunching or a sign that bones are being put back; the noise is caused by the release and popping of gas bubbles, generated when the fluid in the joint space is put under severe stress.
Some chiropractors claim to treat everything from digestive disorders to ear infections, others will treat only back problems.
DOES IT WORK? There is no evidence to suggest that spinal manipulation is effective for anything but back pain and even then conventional approaches (such as regular exercise and ibuprofen) are just as likely to be effective and are cheaper.
Neck manipulation has been linked to neurological complications such as strokes – in 1998, a 20-year-old Canadian woman died after neck manipulation caused a blood clot which led to stroke. We would strongly recommend physiotherapy exercises and osteopathy ahead of chiropractic therapy because they are at least effective and much safer.
If you do decide to visit a chiropractor despite our concerns and warnings, we very strongly recommend you confirm your chiropractor won’t manipulate your neck. The dangers of chiropractic therapy to children are particularly worrying because a chiropractor would be manipulating an immature spine.
Daily Mail 2008 April 8th.
As we are aware that patients or potential patients of our members will be confronted with questions regarding this article, we have put together some comment and Q&As to assist you.
• Please consider this information as strictly confidential and for your use only.
• Only use this if a patient asks about these specific issues; there is nothing to be gained from releasing any information not asked for.
• Do not duplicate these patient notes and hand out direct to the patient or the media; these are designed for you to use when in direct conversation with a patient.
The BCA will be very carefully considering any questions or approaches we may receive from the press and will respond to them using specially briefed spokespeople. We would strongly advise our members not to speak directly to the press on any of the issues raised as a result of this coverage.
Please note that In the event of you receiving queries from the media, please refer these direct to BCA (0118 950 5950 – Anne Barlow or Sue Wakefield) or Publicasity (0207 632 2400 – Julie Doyle or Sara Bailey).
The following points should assist you in answering questions that patients may ask with regard to the safety and effectiveness of chiropractic care. Potential questions are detailed along with the desired ‘BCA response’:
o “The Daily Mail article seems to suggest chiropractic treatment is not that effective”
Nothing could be further from the truth. The authors have had to concede that chiropractic treatment works for back pain as there is overwhelming evidence to support this. The authors also contest that pain killers and exercises can do the job just as well. What they fail to mention is that research has shown that this might be the case for some patients, but the amount of time it may take to recover is a lot longer and the chance of re-occurrence of the problem is higher. This means that chiropractic treatment works, gets results more quickly and helps prevent re-occurrence of the problem. Chiropractic is the third largest healthcare profession in the world and in the UK is recognised and regulated by the UK Government.
o “The treatment is described as aggressive, can you explain?”
It is important to say that the authors of the article clearly have no direct experience of chiropractic treatment, nor have they bothered to properly research the training and techniques. Chiropractic treatment can take many forms, depending on the nature of the problem, the particular patient’s age and medical history and other factors. The training chiropractors receive is overseen by the government appointed regulator and the content of training is absolutely designed to ensure that an individual chiropractor understands exactly which treatment types are required in each individual patient scenario. Gentle technique, massage and exercise are just some of the techniques available in the chiropractor’s ‘toolkit’. It is a gross generalisation and a demonstration of lack of knowledge of chiropractic to characterise it the way it appeared in the article.
o “The article talked about ‘claims’ of success with other problems”
There is a large and undeniable body of evidence regarding the effectiveness of chiropractic treatment for musculoskeletal problems such as back pain. There is also growing evidence that chiropractic treatment can help many patients with other problems; persistent headaches for example. There is also anecdotal evidence and positive patient experience to show that other kinds of problems have been helped by chiropractic treatment. For many of these kinds of problems, the formal research is just beginning and a chiropractor would never propose their treatment as a substitute for other, ongoing treatments.
o “Am I at risk of having a stroke if I have a chiropractic treatment?”
What is important to understand is that any association between neck manipulation and stroke is extremely rare. Chiropractic is a very safe form of treatment.
Another important point to understand is that the treatments employed by chiropractors are statistically safer than many other conservative treatment options (such as ibuprofen and other pain killers with side effects such as gastric bleeding) for mechanical low back or neck pain conditions.
A research study in the UK, published just last year studied the neck manipulations received by nearly 20,000 chiropractic patients. NO SERIOUS ADVERSE SIDE EFFECTS WERE IDENTIFIED AT ALL. In another piece of research, published in February this year, stroke was found to be a very rare event and the risk associated with a visit to a chiropractor appeared to be no different from the risk of a stroke following a visit to a GP.
Other recent research shows that such an association with stroke may occur once in every 5.85 million adjustments.
To put this in context, a ‘significant risk’ for any therapeutic intervention (such as pain medication) is defined as 1 in 10,000.
Additional info: Stroke is a natural occurring phenomenon, and evidence dictates that a number of key risk factors increase the likelihood of an individual suffering a stroke. Smoking, high blood pressure, high cholesterol and family medical histories can all contribute; rarely does a stroke occur in isolation from these factors. Also, stroke symptoms can be similar to that of upper neck pains, stiffness or headaches, conditions for which patients may seek chiropractic treatment. BCA chiropractors are trained to recognise and diagnose these symptoms and advise appropriate mainstream medical care.
o “Can you tell if I am at risk from stroke?”
As a BCA chiropractor I am trained to identify risk factors and would not proceed with treatment if there was any doubt as to the patient’s suitability. Potential risks may come to light during the taking of a case history, which may include: smoking, high cholesterol, contraceptive pill, Blood clotting problems/blood thinning meds, heart problems, trauma to the head etc and on physical examination e.g. high blood pressure, severe osteoarthritis of the neck, history of rheumatoid arthritis
o “Do you ever tell patients if they are at risk?”
Yes, I would always discuss risks with patients and treatment will not proceed without informed consent.
o “Is it safe for my child to be treated by a chiropractor”
It is a shame that the article so generalises the treatment provided by a chiropractor, that it makes such outrageous claims. My training in anatomy, physiology and diagnosis means that I absolutely understand the demands and needs of spines from the newborn baby to the very elderly patient. The techniques and treatments I might use on a 25 year old are not the same as those I would employ on a 5 year old. I see a lot of children as patients at this clinic and am able to offer help with a variety of problems with the back, joints and muscles. I examine every patient very thoroughly, understand their medical history and discuss my findings with them and their parents before undertaking any treatment.
- Chiropractic is a mature profession and numerous studies clearly demonstrate that chiropractic treatment, including manipulative and spinal adjustment, is both safe and effective.
- Thousands of patients are treated by me and my fellow chiropractors every day in the UK. Chiropractic is a healthcare profession that is growing purely because our patients see the results and GPs refer patients to us because they know we get results!
This article is to promote a book and a controversial one at that. Certainly, in the case of the comments about chiropractic, there is much evidence and research that has formed part of guidelines developed by the Royal Society of General Practitioners, NICE and other NHS/Government agencies, has been conveniently ignored. The statements about chiropractic treatment and technique demonstrate that there has clearly been no research into the actual education that chiropractors in the UK receive – in my case a four year full-time degree course that meets stringent educational standards set down by the government appointed regulator.
Shortly after the article in The Daily Mail, our book was published and turned out to be much appreciated by critical thinkers across the globe — not, however, by chiropractors.
At the time, I did, of course, not know about the above “strictly confidential” message to BCA members, yet I strongly suspected that chiropractors would do everything in their power to dispute our central argument, namely that most of the therapeutic claims by chiropractors were not supported by sufficient evidence. I also knew that our evidence for it was rock solid; after all, I had researched the evidence for or against chiropractic in full depth and minute detail and published dozens of articles on the subject in the medical literature.
When, one and a half weeks after our piece in the Mail, Simon published his now famous Guardian comment stating that the BCA “happily promote bogus treatments”, he was sued for libel by the BCA. I think the above “strictly confidential” message already reveals the BCA’s determination and their conviction to be on firm ground. As it turned out, they were wrong. Not only did they lose their libel suit, but they also dragged chiropractic into a deep crisis.
The “strictly confidential” message is intriguing in several more ways - I will leave it to my readers to pick out some of the many gems hidden in this text. Personally, I find the most remarkable aspect that the BCA seems to attempt to silence its own members regarding the controversy about the value of their treatments. Instead they proscribe answers (should I say doctrines?) of highly debatable accuracy for them, almost as though chiropractors were unable to speak for themselves. To me, this smells of cult-like behaviour, and is by no means indicative of a mature profession – despite their affirmations to the contrary.
The question whether spinal manipulation is an effective treatment for infant colic has attracted much attention in recent years. The main reason for this is, of course, that a few years ago Simon Singh had disclosed in a comment that the British Chiropractic Association (BCA) was promoting chiropractic treatment for this and several other childhood condition on their website. Simon famously wrote “they (the BCA) happily promote bogus treatments” and was subsequently sued for libel by the BCA. Eventually, the BCA lost the libel action as well as lots of money, and the entire chiropractic profession ended up with enough egg on their faces to cook omelets for all their patients.
At the time, the BCA had taken advice from several medical and legal experts; one of their medical advisers, I was told, was Prof George Lewith. Intriguingly, he and several others have just published a Cochrane review of manipulative therapies for infant colic. Here are the unabbreviated conclusions from their article:
“The studies included in this meta-analysis were generally small and methodologically prone to bias, which makes it impossible to arrive at a definitive conclusion about the effectiveness of manipulative therapies for infantile colic. The majority of the included trials appeared to indicate that the parents of infants receiving manipulative therapies reported fewer hours crying per day than parents whose infants did not, based on contemporaneous crying diaries, and this difference was statistically significant. The trials also indicate that a greater proportion of those parents reported improvements that were clinically significant. However, most studies had a high risk of performance bias due to the fact that the assessors (parents) were not blind to who had received the intervention. When combining only those trials with a low risk of such performance bias, the results did not reach statistical significance. Further research is required where those assessing the treatment outcomes do not know whether or not the infant has received a manipulative therapy. There are inadequate data to reach any definitive conclusions about the safety of these interventions”
Cochrane reviews also carry a “plain language” summary which might be easier to understand for lay people. And here are the conclusions from this section of the review:
The studies involved too few participants and were of insufficient quality to draw confident conclusions about the usefulness and safety of manipulative therapies. Although five of the six trials suggested crying is reduced by treatment with manipulative therapies, there was no evidence of manipulative therapies improving infant colic when we only included studies where the parents did not know if their child had received the treatment or not. No adverse effects were found, but they were only evaluated in one of the six studies.
If we read it carefully, this article seems to confirm that there is no reliable evidence to suggest that manipulative therapies are effective for infant colic. In the analyses, the positive effect disappears, if the parents are properly blinded; thus it is due to expectation or placebo. The studies that seem to show a positive effect are false positive, and spinal manipulation is, in fact, not effective.
The analyses disclose another intriguing aspect: most trials failed to mention adverse effects. This confirms the findings of our own investigation and amounts to a remarkable breach of publication ethics (nobody seems to be astonished by this fact; is it normal that chiropractic researchers ignore generally accepted rules of ethics?). It also reflects badly on the ability of the investigators of the primary studies to be objective. They seem to aim at demonstrating only the positive effects of their intervention; science is, however, not about confirming the researchers’ prejudices, it is about testing hypotheses.
The most remarkable thing about the new Cochrane review is, I think, the in-congruence of the actual results and the authors’ conclusion. To a critical observer, the former are clearly negative but the latter sound almost positive. I think this begs the question about the possibility of reviewer bias.
We have recently discussed on this blog whether reviews by one single author are necessarily biased. The new Cochrane review has 6 authors, and it seems to me that its conclusions are considerably more biased than my single-author review of chiropractic spinal manipulation for infant colic; in 2009, I concluded simply that “the claim [of effectiveness] is not based on convincing data from rigorous clinical trials”.
Which of the two conclusions describe the facts more helpfully and more accurately?
I think, I rest my case.