Would it not be nice to have a world where everything is positive? No negative findings ever! A dream! No, it’s not a dream; it is reality, albeit a reality that exists mostly in the narrow realm of alternative medicine research. Quite a while ago, we have demonstrated that journals of alternative medicine never publish negative results. Meanwhile, my colleagues investigating acupuncture, homeopathy, chiropractic etc. seem to have perfected their strategy of avoiding the embarrassment of a negative finding.
Since several years, researchers in this field have adopted a study-design which is virtually sure to generate nothing but positive results. It is being employed widely by enthusiasts of placebo-therapies, and it is easy to understand why: it allows them to conduct seemingly rigorous trials which can impress decision-makers and invariably suggests even the most useless treatment to work wonders.
One of the latest examples of this type of approach is a trial where acupuncture was tested as a treatment of cancer-related fatigue. Most cancer patients suffer from this symptom which can seriously reduce their quality of life. Unfortunately there is little conventional oncologists can do about it, and therefore alternative practitioners have a field-day claiming that their interventions are effective. It goes without saying that desperate cancer victims fall for this.
In this new study, cancer patients who were suffering from fatigue were randomised to receive usual care or usual care plus regular acupuncture. The researchers then monitored the patients’ experience of fatigue and found that the acupuncture group did better than the control group. The effect was statistically significant, and an editorial in the journal where it was published called this evidence “compelling”.
Due to a cleverly over-stated press-release, news spread fast, and the study was celebrated worldwide as a major breakthrough in cancer-care. Finally, most commentators felt, research has identified an effective therapy for this debilitating symptom which affects so many of the most desperate patients. Few people seemed to realise that this trial tells us next to nothing about what effects acupuncture really has on cancer-related fatigue.
In order to understand my concern, we need to look at the trial-design a little closer. Imagine you have an amount of money A and your friend owns the same sum plus another amount B. Who has more money? Simple, it is, of course your friend: A+B will always be more than A [unless B is a negative amount]. For the same reason, such “pragmatic” trials will always generate positive results [unless the treatment in question does actual harm]. Treatment as usual plus acupuncture is more than treatment as usual, and the former is therefore moer than likely to produce a better result. This will be true, even if acupuncture is no more than a placebo – after all, a placebo is more than nothing, and the placebo effect will impact on the outcome, particularly if we are dealing with a highly subjective symptom such as fatigue.
I can be fairly confident that this is more than a theory because, some time ago, we analysed all acupuncture studies with such an “A+B versus B” design. Our hypothesis was that none of these trials would generate a negative result. I probably do not need to tell you that our hypothesis was confirmed by the findings of our analysis. Theory and fact are in perfect harmony.
You might say that the above-mentioned acupuncture trial does still provide important information. Its authors certainly think so and firmly conclude that “acupuncture is an effective intervention for managing the symptom of cancer-related fatigue and improving patients’ quality of life”. Authors of similarly designed trials will most likely arrive at similar conclusions. But, if they are true, they must be important!
Are they true? Such studies appear to be rigorous – e.g. they are randomised – and thus can fool a lot of people, but they do not allow conclusions about cause and effect; in other words, they fail to show that the therapy in question has led to the observed result.
Acupuncture might be utterly ineffective as a treatment of cancer-related fatigue, and the observed outcome might be due to the extra care, to a placebo-response or to other non-specific effects. And this is much more than a theoretical concern: rolling out acupuncture across all oncology centres at high cost to us all might be entirely the wrong solution. Providing good care and warm sympathy could be much more effective as well as less expensive. Adopting acupuncture on a grand scale would also stop us looking for a treatment that is truly effective beyond a placebo – and that surely would not be in the best interest of the patient.
I have seen far too many of those bogus studies to have much patience left. They do not represent an honest test of anything, simply because we know their result even before the trial has started. They are not science but thinly disguised promotion. They are not just a waste of money, they are dangerous – because they produce misleading results – and they are thus also unethical.
21 Responses to No negatives please, we are alternative!
Leave a Reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
The rates of negative to positive trials in conventional medicine mirror the rates of negative and positive trials in Homeopathy- see Faculty of Homeoapthy research pages: – Oh I forgot- my dear old friend, Edzard, studies these on a regular basis and can provide the link…………
From the conventional medical literature an interesting and informative article was sent to me today. It seems I have a 41% chance of getting flu despite having been vaccinated, but for the over 65s there is simply no scientific data, ergo no evidence based advice, purely extrapolation and unsupported opinion. Might as well take some homeopathic oscillococinum 30C bd as well?!
http://www.telegraph.co.uk/health/flu/9692193/Flu-vaccine-effectiveness-exaggerated-scientists-claim.html
As regards ‘No negatives’ didn’t you not mean to write: No negative trials published by the big medical researchers has led to calls for publication of all trials- including the negative ones to allow Dr’s to correctly advise the patients consulting them. Psychology is in the spotlight in this Guardian article, although other research is alluded to.
http://www.guardian.co.uk/science/2012/sep/13/scientific-research-fraud-bad-practice
Or perhaps this BMJ article is relevant: http://www.lymeneteurope.org/forum/viewtopic.php?f=5&t=3595
Well done for a sensationalist and unresearched article.
While your criticism of the A+B vs A design is completely valid you make it sound like this nudging of designs towards producing positive results was a particular trait of people promoting alternative medicine, but it’s clearly not. Exactly the same can be found in research on classical western medicine/drugs testing, and it’s likely to be a side-effect of the publication bias in current academic research in general. In a way the publication of trials of which we can guess the results before they’ve even started are actually less harmful than results which only get published when one of many trials happens to exhibit significant results. Pharmaceutical companies ‘have a field-day claiming that their interventions are effective’ all the time (see e.g. here) and I find criticism there much more important as established companies can exert more influence on policy makers and thus help maintain their position and exploit the status quo.
Kevin: you are probably correct; dirty tricks are being played everywhere. But specifically the A+B verus B design seems particularly prevalent in alternative medicine. I do not know many pharma studies where a drug is tested alongside standard care against standard care alone in an RCT. But I might be wrong, as my field is alt med and not pharma.
Just an observation: Ernst Quoteming to make a beginning. The Nature study cited is from 1997!
Today CAM journals if they publish negative studies.
By 2007, the situation regarding CAM-journals publishing only or mostly positive results had improved only marginally http://www.ncbi.nlm.nih.gov/pubmed/17658121. More recently, we have not investigated this intriguing phenomenon, but my impression is that there is little reason for optimism.
Ok, thanks!.
Wouldn’t a fairer test be A compared to A+B and A+C, C being, say, the distribution of jelly beans, with similar ceremonies, sympathy etc. I don’t know obviously, but I am willing to advocate for jelly bean therapists….
Why not A+B versus A+C where A is the standard care that has to be given for ethical reasons, B is the experimental treatment and C is a placebo or the nearest to a placebo. The more arms you have in a study, the more hypotheses you are testing simultaneously; and that can lead to confusion and uninterpretable findings.
And, the “depression” doctors can do little about”? Of course they can.
The depression is through the same path as the KRAS oncogenesis through the activation of it.
Turning it off, where I see it is overrun, is the same door that closes fibromyalgia and a host of disease.
I’ve compiled the info.
Please. Somebody tell me where and with whom to share it.