This new RCT was embargoed until today; so, I had to wait until I was able to publish my comments. Here are the essentials of the study:
The Swedish investigators compared the effect of two types of acupuncture versus no acupuncture in infants with colic in public child health centres (CHCs). The study was designed as a multicentre, randomised controlled, single-blind, three-armed trial (ACU-COL) comparing two styles of acupuncture with no acupuncture, as an adjunct to standard care. Among 426 infants whose parents sought help for colic and registered their child’s fussing/crying in a diary, 157 fulfilled the criteria for colic and 147 started the intervention.
Parallel to usual care, study participants visited the study CHC twice a week for 2 weeks. Thus, all infants received usual care plus 4 extra visits to a CHC, during which parents met a nurse for 20–30 min and were able to discuss their infant’s symptoms. Together these were considered to represent gold standard care. The nurse listened, and gave evidence-based advice and calming reassurance. Breastfeeding mothers were encouraged to continue breastfeeding. At each visit, the study nurse carried the infant to a separate treatment room where they were left alone with the acupuncturist for 5 min.
The acupuncturist treated the baby according to group allocation and recorded the treatment procedures and any adverse events. Disposable stainless steel 0.20×13 mm Vinco needles (Helio, Jiangsu Province, China) were used. Infants allocated to group A received standardised MA at LI4. One needle was inserted to a depth of approximately 3 mm unilaterally for 2–5 s and then withdrawn without stimulation. Infants allocated to group B received semi-standardised individualised acupuncture, mimicking clinical TCM practice. Following a manual, the acupuncturists were able to choose one point, or any combination of Sifeng, LI4 and ST36, depending on the infant’s symptoms, as reported in the diary. A maximum of five insertions were allowed per treatment. Needling at Sifeng consisted of 4 insertions, each to a depth of approximately 1 mm for 1 s. At LI4 and ST36, needles were inserted to a depth of approximately 3 mm, uni- or bilaterally. Needles could be retained for 30 seconds. De qi was not sought, therefore stimulation was similarly minimal in groups A and B. Infants in group C spent 5 min alone with the acupuncturist without receiving acupuncture.
The effect of the two types of acupuncture was similar and both were superior to gold standard care alone. Relative to baseline, there was a greater relative reduction in time spent crying and colicky crying by the second intervention week (p=0.050) and follow-up period (p=0.031), respectively, in infants receiving either type of acupuncture. More infants receiving acupuncture cried <3 hours/day, and thereby no longer fulfilled criteria for colic, in the first (p=0.040) and second (p=0.006) intervention weeks. No serious adverse events were reported.
The authors concluded that acupuncture appears to reduce crying in infants with colic safely.
Notice that the investigators are cautious and state in the abstract that “acupuncture appears to reduce crying…” Their conclusions from the actual article are, however, quite different; here they state the following:
Among those initially experiencing excessive infant crying, the majority of parents reported normal values once the infant’s crying had been evaluated in a diary and a diet free of cow’s milk had been introduced. Therefore, objective measurement of crying and exclusion of cow’s milk protein are recommended as first steps, to avoid unnecessary treatment. For those infants that continue to cry >3 hours/day, acupuncture may be an effective treatment option. The two styles of MA tested in ACU-COL had similar effects; both reduced crying in infants with colic and had no serious side effects. However, there is a need for further research to find the optimal needling locations, stimulation and treatment intervals.
Such phraseology is much more assertive and seems to assume acupuncture caused specific therapeutic effects. Yet, I think, this assumption is not warranted.
In fact, I believe, the study shows almost the opposite of what the authors conclude. Both minimal and TCM acupuncture seemed to reduce the symptoms of colic compared to no acupuncture at all. I think, this confirms previous research showing that acupuncture is a ‘theatrical placebo’. The study was designed without an adequate placebo group. It would have been easy to use some form of sham acupuncture in the control group. Why did the authors not do that? Heaven knows, but one might speculate that they were aiming for a positive result – and what better way to ensure it than with a ‘no treatment’ control group?
There are, of course, numerous other flaws. For instance, Prof David Colquhoun FRS, Professor of Pharmacology at University College London, criticised the study because of its lousy statistics:
START OF QUOTE
“It is truly astonishing that, in the 21st century, the BMJ still publishes a journal devoted to a form of pre-scientific medicine which after more than 3000 trials has still not been able to produce convincing evidence of efficacy1. Like most forms of alternative medicine, acupuncture has been advocated for a vast range of problems, and there is little evidence that it works for any of them. Colic has not been prominent in these claims. What parent would think that sticking needles into their baby would stop it crying? The idea sounds bizarre. It is. This paper certainly doesn’t show that it works.
“The statistical analysis in the paper is incompetent. This should have been detected by the referees, but wasn’t. For a start, the opening statement, ‘A two-sided P value ≤0.05 was considered statistically significant’ is simply unacceptable in the light of all recent work about reproducibility. Still worse, Table 1 uses the description ‘statistical tendency towards significance (p=0.051–0.1)’.
“Worst of all, Table 1 reports 24 different P values, of which three are (just) below 0.05. Yet no correction has been used for multiple comparisons. This is very bad practice. It’s highly unlikely that, if the proper correction had been done, any of the results would have given a type 1 error rate below 5%.
“Even were it not for this, most of the ‘significant’ P values are marginal (only slightly less than 0.05). It is now well known that the type 1 error rate gives an optimistic view. What matters is the false positive rate – the chance that a ‘significant’ result is a false positive. A p-value close to 0.05 implies that there is at least a 30% chance that they are false positives. If one thought, a priori, that the chance of colic being cured by sticking needles into a baby was less than 50%, the false positive rate could easily be greater than 80%2. It is now recognised that this misinterpretation of p-values is a major contributor to the crisis of reproducibility.
“Other problems concern the power calculation. A priori calculations of power are well-known to be overoptimistic, because small trials usually overestimate the effect size. In this case the initial estimated sample size was not attained, and a rather mysterious recalculation of power was used.
“Another small problem: the discussion points out that ‘the majority of infants in this cohort did not have colic’.
“The nature of the control group is not very clear. An appropriate control might have been to cuddle the baby – this was used in a study in which another implausible treatment, chiropractic, was shown not to work. This appears not to have been done.
“Lastly, p-values are reported in the text without mention of effect sizes. This is contrary to all statistical advice.
“In conclusion, the design of the trial is reasonable (apart from the control group) but the statistical analysis is appalling. It’s very likely that there aren’t any real effects of acupuncture at all. This paper serves more to muddy the waters than to add useful information. It’s a model for the sort of mistakes that have led to the crisis in reproducibility. The BMJ should not be publishing this sort of stuff, and the referees seem to have no understanding of statistics.”
END OF QUOTE
Despite these rather obvious – some would say fatal – flaws, the editor of ACUPUNCTURE IN MEDICINE (AIM) thought this trial to be so impressively rigorous that he issued a press-release about it. This, I think, is particularly telling, perhaps even humorous: it shows what kind of a journal AIM is, and also provides an insight into the state of acupuncture research in general.
The long and short of it is that conclusions about specific therapeutic effects of acupuncture are not permissible. We know that colicky babies respond even to minimal attention, and this trial confirms that even a little additional TLC in the form of acupuncture will generate an effect. The observed outcome is most likely unrelated to acupuncture.
45 Responses to Acupuncture study on colicky kids shows the opposite of what the authors conclude
Leave a Reply to Alan Henness Cancel reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
more here:
https://www.buzzfeed.com/tomchivers/acupuncture-baby?utm_term=.rqAg4YwzM#.hne9rMD3K
This paper was covered in the Today programme on Radio 4 this morning – listen here from about 2 hours 44 minutes in.
It involved Mike Cummings, an associate editor of Acupuncture in Medicine and medical director of the British Medical Acupuncture Society, and Prof David Colquhoun.
I’ll leave you to listen to the arguments in favour of this study (such as they were), but there was one very odd thing from Cummings. He said there was ‘very limited funding for these type of studies’, yet went on to say that, although this study ‘in technical terms was negative’ we should wait but went on to say ‘there’s about four trials done in this area and we’re waiting for them…all the results to be added up in a review to see’.
If there have now been five trials conducted just on colic, funding doesn’t seem to be all that limited, does it?
i heard the interview; it was odd. i would have questioned why this study is being ‘sold’ to the public as a positive trial, while it was, in fact, negative? why make a press-release on such a scam? Cummings advocated doing a meta-analysis across all available studies. i predict that, if done properly, it would not show efficacy of acupuncture.
Can we also predict whether the results will be announced with the same media fanfare?
what might be predictable is this: in case the results are positive (only possible in my view, if the review is biased) there will be a major media fanfare; if not, there will be nothing of the sort. it seems to follow: poor research, positive result, media fanfare; rigorous research, negative result, no media attention.
Acupuncture trials don’t appear to be in short supply. Short of convincing evidence but – short of money? Doesn’t compute.
Indeed.
May I respectfully suggest that Professor Ernst’s above blog piece is re-written, including (possibly co-authored by) DC’s piece, in a form suitable for publition in AIM.
If they decline to publish (in the interests of balance), let me know and I will raise the issue at next June’s BMA Annual Conference.
The BMJ (and AIM) have editorial freedom from the BMA, but BMA members will not want the BMJ Publishing Group brought into disrepute.
A trial of compromised power, found a barely significant difference between acupuncture and standard care. No placebo control.
My feeling is that the parents were mad to hand over their babies to be taken away and punctured or…
Who knows what exactly went on in the room where babies did not receive acupuncture?
want to see some truly bad reporting of this study?
http://www.telegraph.co.uk/science/2017/01/16/acupuncture-helps-young-babies-stop-crying-new-research/?WT.mc_id=tmg_share_tw
I’m sure Cummings will be only too eager to contact them to correct them.
and some not quite as bad but still very uncritical reporting here:
http://www.nhs.uk/news/2017/01January/Pages/Can-colic-really-be-cured-by-acupuncture.aspx
@Edzard – please could you point out the evidence that proves ‘colicky babies respond even to minimal attention’? I’m not saying it doesn’t exist, I’m just interested. Thanks.
can’t you do your own Medline search? that’s where I saw it some time ago
alright, I think it was this study but am not entirely sure
http://adc.bmj.com/content/84/2/138
The conclusion of that study was ‘Improvement occurred in both the treatment and control groups. This may reflect an effect of general counselling and support from the professional team or a natural spontaneous improvement as a result of increasing age.’ So they don’t put improvements down to attention towards the baby. Perhaps you meant a different study? It seems important, as you’re claiming it is ‘known’ that colicky babies respond to intention, and you’re using this ‘fact’ as a stick to beat the acupuncture trial with.
@Tom
From the ‘methods’ section of the paper: “The infants who did not get spinal manipulation were just held by the nurse for 10 minutes (the approximate time of treatment) after being partially undressed in a similar way as [chiropractically] treated infants.” You, yourself, quoted the conclusion that improvement occurred in both the treatment and the control groups. Therefore, a nurse holding a baby for 10 minutes was associated with improvement. I’d say that supports Edzard’s contention that “colicky babies respond even to minimal attention”, wouldn’t you?
@Frank Not really, no, and certainly not to the extent that ‘know’ it to be true. As the conclusion states, there are other very plausible explanations for the improvements seen in the control group, such as the natural course of the condition. It seems to me to be an example of the double standards so obviously evident in arguments over the scientific basis for CAM, on both sides of the fence. Edzard and others here have decided that acupuncture is nonsense, and therefore make statements like this as if they are facts that confirm their argument. And then whenever a similar tactic is used by someone with a different opinion, they are lambasted for misrepresenting the facts to suit their agenda.
” Edzard and others here have decided that acupuncture is nonsense, and therefore make statements like this as if they are facts that confirm their argument.”
has it occurred to you that the study we are discussing here was negative? what do you want “Edzard and others” to do? claim that “ACUPUNCTURE IS NOT NONSENSE”?
do you see how absurd your argument is?
As far as I can see this study was funded by two parts. One is the ‘Ekhagastiftelsen’-fund, founded by a wealthy Swedish building entrepreneur who was very interested in alternative medicine. It is dedicated to support mostly alternative ideas An interesting conference on ‘Integrative Health Care’ took place under their auspices in 2010, and the present subject was presented there. Apparently there was a pilot study preceding this RCT.
The other supporter of this study seems to be the Uddenäs family one of the wealthiest in Lund, the home town of the Swedish main author.
It does not seem too difficult to find funds for studies of the alternative in this part of the world. Here’s a side story to illustrate this.
Recently three Scandinavian insurance companies jointly funded a large scale study of functional medicine. If I understand correctly from speed-reading the media reports, they want to see if individualised life-style interventions helps for hypothyroidism. Here’s a news item from a branch magazine about insurance. For those who do not read Swedish I suggest pasting this URL:
http://www.nft.nu/sv/kan-funktionell-medicin-revolutionera-framtidens-sjukvard
…into Google’s translator (translate.google.com) The resulting English translation seems quite adequate for general comprehension. The article starts on an hilariously naive quote from Mark Hyman: “There are no diseases, only symptoms”
The study is severely criticised here by a well known Swedish medical blogger Mats Reimer MD at the weekly Dagens Medicin (“Medicine of the day”) a weekly branch-magazine for health care professionals. One of the idiosyncracies he discusses, is the stress relieving technique called EFT (Emotional Freedom Technique) intended to be part of the studied lifestyle interventions. It apparently involves banging your fingers on the head. His critique was countered by Prof. Kerstin Brismar MD who heads the study. She is professor of internal medicine and endocrinology at the ‘Karolinska Institute’ (one of the most prestigious medical institutions of the world, the one that doles out the Nobel prize in medicine) Dr. Brismar declares in her rather indignant response that everything should be studied, also the alternative and points out e.g. that studies have shown EFT to have similar effects as yoga and meditation.
The debate continues here with Dr. Reimer’s swathing critique e.g. of apparent commercial involvement in the study by companies invloved in alternative goods and services e.g. hair mineral analysis and herbal remedies and supplements. People with backgrounds in diverse doltishness such as applied kinesiology, frequency healing and bioresonance seem to figure in running the businesses involved 😀
(For those interested in reading a translation of these references, please see the Google-translate tip above)
I jotted down this information to illustrate the fact that there is quite some money and high-level support to be found for research into the incredible in Scandinavia.
To see where billions are spent inn (re-)search of the alternative in the US, go to https://nccih.nih.gov/
Landgren and Hallström cite three previous studies of acupuncture for colic in infants.
Here’s what I found.
Ref 8. : Reinthal et al. Prospective quasi-randomised Controlled Trial. 40 infants divided into two groups; one received bilateral light needling at L14; the other group received the same time and attention by an acupuncturist but no acupuncture. No placebo group. Significant difference (p=0.0160). How or whether parents were assessed for effective blinding is not discussed; the authors simply stated that no information was given to parents as to which group their child was allocated.
Self-evident limitations of this study are poor randomisation, low power and possible parental bias if blinding was compromised in any way.
In the study under discussion here (Landgren and Hallström 2016) acupuncture groups of over twice the size barely achieved statistical significance – except when they added what had been intended to be separately compared treatments together, to outnumber the non-acupuncture control by two to one.
Ref 11. : Landgren, Kvorning, Hallström, 2010. 90 infants randomised to light needling at L14 or to same time and attention from acupuncturist but no acupuncture. No placebo group. Significant difference (p=0.034).
In this trial one and the same person was responsible for randomisation; for administering acupuncture; and was aware of allocation. This same person had sole access to records during the study, was in contact with the person who transferred infants back and forth from acupuncturist to their parents.
This study was described as double-blinded. Parents were told that usually there was no bleeding or visible marks. By no means can blinding be considered to have been safe in this trial.
Landgren and Hallström have, essentially, replicated the results of their earlier trial by round about means.
Amongst the three previous trials for acupuncture for colic in infants, they describe as an “outlier”, a trial by Skjeie et al. 2013, which found no statistical difference for acupuncture compared with no treatment. 90 infants were randomised to acupuncture or similar treatment without acupuncture. They chose to needle point ST36.
The blinding assessment made by Skjeie et al. suggests that blinding was likely more effective than in the trials of Landgren and Hallström. Randomisation was clearly superior and much greater detail is given about procedure.
I don’t see Skjeie et al. as an outlier, necessarily.
I see Skjeie et al. as the best quality study. Let’s allow their study to cancel the weakest trial of Reinthal et al.
The duplicated trials of Landgren et al. barely signify.
We are left with their highly questionable ad hoc addition of two studies employing different numbers and placements of acupuncture needles showing statistical significance. They make no mention of the clear heterogeneity of these two studies either qualitatively or quantitatively.
I conclude that nobody can reasonably claim that needles should be inserted into babies suffering from colic on the basis of the available evidence.
Link to Skjeie et al. (Ref 10)
http://www.tandfonline.com/doi/full/10.3109/02813432.2013.862915
The glaring drawback of all the trials of acupuncture for colic is the absence of a placebo control.
I wonder if it bothers the authors whether acupuncture is anything more than a placebo? It would seem not.
i suspect they know – that’s why they avoid placebo controls. Cummings said on the TODAY PROGRAMME that it is difficult to design studies large enough for picking up differences between needling and needling. he meant placebo acupuncture using non-acupuncture points. he conveniently forgot that one can also use placebo devices that employ non-penetrating needles. these are effectively non-needling devices.
The paper states:
Why then does the CONSORT diagram say that in one of the acupuncture arms, one infant dropped out, hospitalised due to fever? Why did the authors not consider this to be an AE? No explanation is given.
Because the adverse event was unlikely in the extreme to be connected with the trial, presumably?
Wow.
@Alan Why ‘wow’? I may be wrong but in any trial if someone pulls out through illness I assume that wouldn’t count as an adverse event unless it was due to the intervention? And considering 1) babies quite frequently pick up infections that result in fevers, and 2) when administered by properly trained practitioners acupuncture doesn’t, as far as I’m aware, cause fevers, it seems reasonable that this event was reported but not classed as adverse.
yes, you ARE wrong.
in any trial, one has to seriously consider that a reported side-effect WAS due to the treatment. you cannot assume it was not caused by it but have to consider it possible unless proven to be unrelated.
and acupuncture is an invasive procedure which cannot be ruled out as the cause of a fever.
What Edzard said. Did the the editors or peer reviewers not pick it up?
Well, problem with saying usual evidenced-care is not as good as a placebo,,,or?
@Edzard
‘has it occurred to you that the study we are discussing here was negative? what do you want “Edzard and others” to do? claim that “ACUPUNCTURE IS NOT NONSENSE”?
do you see how absurd your argument is?’
This is a bizarre response. No, I’m not asking you to claim anything. I’m pointing out double-standards in the way you present information.
you lost me.
me too.
Let me put it differently then. If someone wrote ‘we know [insert CAM therapy here] to be useful for [insert condition here]’, I suspect you’d ask for the evidence to support the claim. You state ‘we know colicky babies respond even to minimal attention’ as if this is a fact, and you use this ‘fact’ to counter the claims made by the above trial. I asked you to support your claim, and what you produced was anything but convincing. This, to me, is a clear case of double standards used to suit your agenda.
….and I did show you the evidence… and the therapy was ineffective in the trial discussed in this post. so?
I’m not defending the collick trial, I’m saying the the evidence you provided to support your dismantling of the trial was extremely flimsy. Which is the sort of thing you complain about all the time, and weakens your argument.
so the evidence for TLC is flimsy!
GET OVER IT!!!
btw, this is what ‘NHS Choices’ (http://www.nhs.uk/Conditions/Colic/Pages/Treatment.aspx) offer as the 2 first therapeutic options for a colicky baby:
“The following suggestions may help:
Holding your baby during a crying episode can sometimes help, as can wrapping them snugly in a blanket or baby sling.
Hold your baby in different positions – such as on your shoulder, cradled in your arms, or lying with their tummy faced down along your forearm…”
A strongly aggravating factor in baby colick is stressed and tired parents. Almost anything helps if it reassures and calms the parents.The element of a vicious cycle can be addressed by simply calming and reassuring the parents, which is exactly what happens when people who are susceptible to believing in acupuncture are invited to participate in a trial aimed at confirming[sic] its effects. Only those ready to believe in the effects of the intervention will attend.
Here’s how participants were selected:
(emphasis is mine)
A recipe for selection bias, if I ever saw one.
Ernst et al, I would like to comment: The majority of parents in the Control (C) group either guessed incorrectly (did think their child received acupuncture) or were unsure whether their baby received treatment, across the treatment period of 2 weeks. Hence, the majority of mothers in the control group was blind, thinking that their infant received treatment or were not sure. This scenario was the same in the acupuncture groups, here the parents may not have been blind as again the majority believed that treatment was given or was unsure. So, we have an equal level of bias from group allocation belief (not necessarily blinding) in all three groups, hence the differences of effects between control and treatment groups most likely cannot be explained by this. You claim that the differences between the control and treatment groups come from a placebo effect which is due to lack of attention towards the infants in the control group. By looking at it from a practical point of view, we are talking irritable and excessively crying infants here. It is highly unlikely that the acupuncturist in the room did not interact with the child, unless it was sleeping (which happens sometimes but not too often), so my guess is, that the acupuncturist was talking to the child, may be using toys or even touch the child. It certainly is not in favour of the authors that they have not reported on this as indeed it is a crucial aspect on judging whether the effects seen may be caused by acupuncture itself or not. Besides, all infants were exposed to the nurse on all four visits, regardless of group allocation. So they all got the attention of the nurse. Do you think that 4×5 mins with the acupuncturist in two weeks with respect to attention would make such a difference to the infants in the intervention groups, even if the babys in the control group did not interact with the acupuncturist at all? I seriously doubt that, I would think that by now a finding that infants get better the more they are exposed to different people who pay attention to them would have been described in the literature, or the word would have spread quickly amongst affected moms. But there is none of that.
So, there may be indeed a chance that the effects come from the needles, but again, as pointed out before, other measures should have been taken into account such as using sham needles and/or using different ‘non-therapeutic’ insertion points.
What worries me though is the fact that the infants were left alone in the room with the acupuncturist. This is ethically questionable. First we are talking child safety here, second who is ensuring professional conduct and adherence to protocol?
as you rightly say: you are guessing.
My first thought was whether or not video recordings were made of the acupuncturist’s dealings with the infants.
No mention is made of anyone else being present. Without some sort of check on what was happening in the acupuncture room, the trial is hopelessly compromised, in terms both of ethics and reliability of results.
By Prof David Colquhoun: If your baby is crying. what do you do? Stick pins in it