Can one design a clinical study in such a way that it looks highly scientific but, at the same time, has zero chances of generating a finding that the investigators do not want? In other words, can one create false positive findings at will and get away with it? I think it is possible; what is more, I believe that, in alternative medicine, this sort of thing happens all the time. Let me show you how it is done; four main points usually suffice:
- The first rule is that it ought to be an RCT, if not, critics will say the result was due to selection bias. Only RCTs have the reputation of being ‘top notch’.
- Once we are clear about this design feature, we need to define the patient population. Here the trick is to select individuals with an illness that cannot be quantified objectively. Depression, stress, fatigue…the choice is vast. The aim must be to employ an outcome measure that is well-accepted, validated etc. but which nevertheless is entirely subjective.
- Now we need to consider the treatment to be “tested” in our study. Obviously we take the one we are fond of and want to “prove”. It helps tremendously, if this intervention has an exotic name and involves some exotic activity; this raises our patients’ expectations which will affect the result. And it is important that the treatment is a pleasant experience; patients must like it. Finally it should involve not just one but several sessions in which the patient can be persuaded that our treatment is the best thing since sliced bread – even if, in fact, it is entirely bogus.
- We also need to make sure that, for our particular therapy, no universally accepted placebo exists which would allow patient-blinding. That would be fairly disastrous. And we certainly do not want to be innovative and create such a placebo either; we just pretend that controlling for placebo-effects is impossible or undesirable. By far the best solution would be to give the control group no treatment at all. Like this, they are bound to be disappointed for missing out a pleasant experience which, in turn, will contribute to unfavourable outcomes in the control group. This little trick will, of course, make the results in the experimental group look even better.
That’s about it! No matter how ineffective our treatment is, there is no conceivable way our study can generate a negative result; we are in the pink!
Now we only need to run the trial and publish the positive results. It might be advisable to recruit several co-authors for the publication – that looks more serious and is not too difficult: people are only too keen to prolong their publication-list. And we might want to publish our study in one of the many CAM-journals that are not too critical, as long as the result is positive.
Once our article is in print, we can legitimately claim that our bogus treatment is evidence-based. With a bit of luck, other research groups will proceed in the same way and soon we will have not just one but several positive studies. If not, we need to do two or three more trials along the same lines. The aim is to eventually do a meta-analysis that yields a convincingly positive verdict on our phony intervention.
You might think that I am exaggerating beyond measure. Perhaps a bit, I admit, but I am not all that far from the truth, believe me. You want proof? What about this one?
Researchers from the Charite in Berlin just published an RCT to investigate the effectiveness of a mindful walking program in patients with high levels of perceived psychological distress.
To prevent allegations of exaggeration, selective reporting, spin etc. I take the liberty of reproducing the abstract of this study unaltered:
Participants aged between 18 and 65 years with moderate to high levels of perceived psychological distress were randomized to 8 sessions of mindful walking in 4 weeks (each 40 minutes walking, 10 minutes mindful walking, 10 minutes discussion) or to no study intervention (waiting group). Primary outcome parameter was the difference to baseline on Cohen’s Perceived Stress Scale (CPSS) after 4 weeks between intervention and control.
Seventy-four participants were randomized in the study; 36 (32 female, 52.3 ± 8.6 years) were allocated to the intervention and 38 (35 female, 49.5 ± 8.8 years) to the control group. Adjusted CPSS differences after 4 weeks were -8.8 [95% CI: -10.8; -6.8] (mean 24.2 [22.2; 26.2]) in the intervention group and -1.0 [-2.9; 0.9] (mean 32.0 [30.1; 33.9]) in the control group, resulting in a highly significant group difference (P < 0.001).
Conclusion. Patients participating in a mindful walking program showed reduced psychological stress symptoms and improved quality of life compared to no study intervention. Further studies should include an active treatment group and a long-term follow-up
This whole thing could just be a bit of innocent fun, but I am afraid it is neither innocent nor fun, it is, in fact, quite serious. If we accept manipulated trials as evidence, we do a disservice to science, medicine and, most importantly, to patients. If the result of a trial is knowable before the study has even started, it is unethical to run the study. If the trial is not a true test but a simple promotional exercise, research degenerates into a farcical pseudo-science. If we abuse our patients’ willingness to participate in research, we jeopardise more serious investigations for the benefit of us all. If we misuse the scarce funds available for research, we will not have the money to conduct much needed investigations. If we tarnish the reputation of clinical research, we hinder progress.
Research is essential for progress, and research in alternative medicine is important for advancing alternative medicine, one would assume. But why then do I often feel that research in this area hinders progress? One of the reasons is, in my view, the continuous drip, drip, drip of misleading conclusions usually drawn from weak studies. I could provide thousands of examples; here is one recently published article chosen at random which seems as good as any other to make the point.
Researchers from the Department of Internal and Integrative Medicine, Faculty of Medicine, University of Duisburg-Essen, Germany set out to investigate associations of regular yoga practice with quality of life and mental health in patients with chronic diseases. Using a case-control study design, 186 patients with chronic diseases who had elected to regularly practice yoga were selected and compared to controls who had chosen to not regularly practice yoga. Patients were matched individually on gender, main diagnosis, education, and age. Patients’ quality of life, mental health, life satisfaction, and health satisfaction were also assessed. The analyses show that patients who regularly practiced yoga had a significantly better general health status, a higher physical functioning, and physical component score on the SF-36 than those who did not.
The authors concluded that practicing yoga under naturalistic conditions seems to be associated with increased physical health but not mental health in chronically diseased patients.
Why do I find these conclusions misleading?
In alternative medicine, we have an irritating abundance of such correlative research. By definition, it does not allow us to make inferences about causation. Most (but by no means all) authors are therefore laudably careful when choosing their terminology. Certainly, the present article does not claim that regular yoga practice has caused increased physical health; it rightly speaks of “associations“. And surely, there is nothing wrong with that – or is there?
Perhaps, I will be accused of nit-picking, but I think the results are presented in a slightly misleading way, and the conclusions are not much better.
Why do the authors claim that patients who regularly practiced yoga had a significantly better general health status, a higher physical functioning, and physical component score on the SF-36 than those who did not than those who did not? I know that the statement is strictly speaking correct, but why do they not write that “patients who had a significantly better general health status, a higher physical functioning, and physical component score on the SF-36 were more likely to practice yoga regularly”? After all, this too is correct! And why does the conclusion not state that better physical health seems to be associated with a greater likelihood of practicing yoga?
The possibility that the association is the other way round deserves serious consideration, in my view. Is it not logical to assume that, if someone is relatively fit and healthy, he/she is more likely to take up yoga (or table-tennis, sky-diving, pole dancing, etc.)?
It’s perhaps not a hugely important point, so I will not dwell on it – but, as the alternative medicine literature is full with such subtly misleading statements, I don’t find it entirely irrelevant either.
Yesterday, I received a letter from the editor-in-chief of the journal ‘Homeopathy‘ informing me that I have been struck off the editorial board of his publication. As the letter is not marked confidential, I feel that I can reproduce parts of it here:
Dear Professor Ernst,
This is to inform you that you have been removed from the Editorial Board of Homeopathy. The reason for this is the statement you published on your blog on Holocaust Memorial Day 2013 in which you smeared homeopathy and other forms of complementary medicine with a ‘guilt by association’ argument, associating them with the Nazis.
I should declare a personal interest….[Fisher goes on to tell a story which is personal and which I therefore omit]… I mention this only because it highlights the absurdity of guilt by association arguments.
Peter Fisher Editor-in-Chief, Homeopathy
I do agree with Dr Fisher that guilt by association is absurd. However, I disagree with the notion that I used this fallacy in my post the full text of which be found here. After re-reading it several times, I still do not see that it employs a ‘guilt by association argument’. It merely recounts historical facts which are not well-known and therefore worth mentioning. Importantly, the post consits in essence of quotes from my previous publications on the subject. My motives for writing it could not have been clearer and are emphacised in the last paragraph:
So, why bring all of this up today? Is it not time that we let grass grow over these most disturbing events? I think not! For many years, I actively researched this area (you can find many of my articles on Medline) because I am convinced that the unprecedented horrors of Nazi medicine need to be told and re-told – not just on HOLOCAUST MEMORIAL DAY, but continually. This, I hope, will minimize the risk of such incredible abuses ever happening again.
Perhaps a comparison might make it a little clearer why, in my opinion, Fisher’s is so utterly bizarre. Imagine an eminent researcher in the area of psychiatry who has been on the editorial board of a journal in his area for many years and contributed numerous articles to this journal. He then decides to research and subsequently write about the infamous Nazi past of German psychiatry. As a result, he is fired from his editorial board position because the editor feels that he has smeared the reputation of psychiatry.
I think most observers might find this odd and unjustified. Such a thing would not happen, I think, in a field with a mature research-culture. That it did happen in homeopathy might be interpreted as a reflection of the fact that homeopathy lacks such a culture.
So how precisely can we explain my dismissal? My article and my motives for writing it could have been thoroughly misunderstood – in my view, this is unlikely because I explained my motives in some detail both in the article and in the comments that follow the article. Here is my last of several posts clarifying my motives:
i am sorry that some have misunderstood the message of this blog and the reason why i wrote it.
i did certainly not want to engage in the GUILTY BY ASSOCIATION fallacy.
here is the truth:
i have had a research interest in nazi medicine and published about it.
in the course of these activities, i discovered that, contrary to what most people seem to assumue, alt med was involved as well. so i published this too many years ago.
this blog was simply and purely aimed at re-telling this story because it deserves to be re-told, in my view.
i regret that some people have read things into it which i did not intend.
Another explanation could be that Dr Fisher, who also is the Queen’s homeopath, lacks sufficient skills of critical thinking to understand the article and its purpose. Alternatively, he has been waiting for an occasion to fire me ever since I became more openly critical of homeopathy about five years ago.
Whatever the explanation, I think it is regrettable that the journal ‘Homeopathy’ has now lost the only editorial board member who had the ability to openly and repeatedly display a critical attitude about homeopathy – remember: without a critical attitude progress is unlikely!
There probably is no area in health care that produces more surveys than alternative medicine. I estimate that about 500 surveys are published every year; this amounts to about two every working day which is substantially more than the number of clinical trials in this field.
I have long been critical of this ‘survey-mania’. The reason is simple: most of these articles are of such poor quality that they tell us nothing of value.
The vast majority of these surveys attempts to evaluate the prevalence of use of alternative medicine, and it is this type of investigation that I intend to discuss here.
For a typical prevalence survey, a team of enthusiastic researchers might put together a few questions and design a questionnaire to find out what percentage of a group of individuals have tried alternative medicine in the past. Subsequently, the investigators might get one or two hundred responses. They then calculate simple descriptive statistics and demonstrate that xy% (let’s assume it is 45%) use alternative medicine. This finding eventually gets published in one of the many alternative medicine journals, and everyone is happy – well, almost everybody.
How can I be such a spoil-sport and claim that this result tells us nothing of value? At the very minimum, some might argue, it shows that enthusiasts of alternative medicine are interested in and capable of conducting research. I beg to differ: this is not research, it is pseudo-research which ignores most of the principles of survey-design.
The typical alternative medicine prevalence survey has none of the features that would render it a scientific investigation:
1) It lacks an accepted definition of what is being surveyed. There is no generally accepted definition of alternative medicine, and even if the researchers address specific therapies, they run into huge problems. Take prayer, for instance – some see this as alternative medicine, while others would, of course, argue that it is a religious pursuit. Or take herbal medicine – many consumers confuse it with homeopathy, some might think that drinking tea is herbal medicine, while others would probably disagree.
2) The questionnaires used for such surveys are almost never validated. Essentially, this means that we cannot be sure they evaluate what we think they evaluate. We all know that the way we formulate a question can determine the answer. There are many potential sources of bias here, and they are rarely taken into consideration.
3) Enthusiastic researchers of alternative medicine usually use a small convenience sample of participants for their surveys. This means they ask a few people who happen to be around to fill their questionnaire. As a consequence, there is no way the survey is representative of the population in question.
4) The typical survey has a low response rate; sometimes the response rate is not even provided or remains unknown even to the investigators. This means we do not know how the majority of patients/consumers who received but did not fill the questionnaire would have answered. Often there is good reason to suspect that those who have a certain attitude did respond, while those with a different opinion did not. This self-selection process is likely to produce misleading findings.
And why I am so sure about all of theses limitations? To my embarrassment, I know about them not least because I have made most these mistakes myself at some time in my career. You might also ask why this is important: what’s the harm in publishing a few flimsy surveys?
In my view, these investigations are regrettably counter-productive because:
they tend to grossly over-estimate the popularity of alternative medicine,
they distract money, manpower and attention from the truly important research questions in this field,
they give a false impression of a buoyant research activity,
and their results are constantly misused.
The last point is probably the most important one. The argument that is all too often spun around such survey data goes roughly as follows: a large percentage of the population uses alternative medicine; people pay out of their own pocket for these treatments; they are satisfied with them (if not, they would not pay for them). BUT THIS IS GROSSLY UNFAIR! Why should only those individuals who are rich enough to afford alternative medicine benefit from it? ALTERNATIVE MEDICINE SHOULD BE MADE AVAILABLE FOR ALL.
I rest my case.
Still in the spirit of ACUPUNCTURE AWARENESS WEEK, I have another critical look at a recent paper. If you trust some of the conclusions of this new article, you might think that acupuncture is an evidence-based treatment for coronary heart disease. I think this would be a recipe for disaster.
This condition affects millions and eventually kills a frighteningly large percentage of the population. Essentially, it is caused by the fact that, as we get older, the blood vessels supplying the heart also change, become narrower and get partially or even totally blocked. This causes lack of oxygen in the heart which causes pain known as angina pectoris. Angina is a most important warning sign indicating that a full blown heart attack might be not far.
The treatment of coronary heart disease consists in trying to let more blood flow through the narrowed coronaries, either by drugs or by surgery. At the same time, one attempts to reduce the oxygen demand of the heart, if possible. Normalisation of risk factors like hypertension and hypercholesterolaemia are key preventative strategies. It is not immediate clear to me how acupuncture might help in all this – but I have been wrong before!
The new meta-analysis included 16 individual randomised clinical trials. All had a high or moderate risk of bias. Acupuncture combined with conventional drugs (AC+CD) turned out to be superior to conventional drugs alone in reducing the incidence of acute myocardial infarction (AMI). AC+CD was superior to conventional drugs in reducing angina symptoms as well as in improving electrocardiography (ECG). Acupuncture by itself was also superior to conventional drugs for angina symptoms and ECG improvement. AC+CD was superior to conventional drugs in shortening the time to onset of angina relief. However, the time to onset was significantly longer for acupuncture treatment than for conventional treatment alone.
From these results, the authors [who are from the Chengdu University of Traditional Chinese Medicine in Sichuan, China] conclude that “AC+CD reduced the occurrence of AMI, and both acupuncture and AC+CD relieved angina symptoms and improved ECG. However, compared with conventional treatment, acupuncture showed a longer delay before its onset of action. This indicates that acupuncture is not suitable for emergency treatment of heart attack. Owing to the poor quality of the current evidence, the findings of this systematic review need to be verified by more RCTs to enhance statistical power.”
As in the meta-analysis discussed in my previous post, the studies are mostly Chinese, flawed, and not obtainable for an independent assessment. As in the previous article, I fail to see a plausible mechanism by which acupuncture might bring about the effects. This is not just a trivial or coincidental observation – I could cite dozens of systematic reviews for which the same criticism applies.
What is different, however, from the last post on gout is simple and important: if you treat gout with a therapy that is ineffective, you have more pain and eventually might opt for an effective one. If you treat coronary heart disease with a therapy that does not work, you might not have time to change, you might be dead.
Therefore I strongly disagree with the authors of this meta-analysis; “the findings of this systematic review need NOT to be verified by more RCTs to enhance statistical power” — foremost, I think, the findings need to be interpreted with much more caution and re-written. In fact, the findings show quite clearly that there is no good evidence to use acupuncture for coronary heart disease. To pretend otherwise is, in my view, not responsible.
There might be an important lesson here: A SEEMINGLY SLIGHT CORRECTION OF CONCLUSIONS OF SUCH SYSTEMATIC REVIEWS MIGHT SAVE LIVES.
There are at least two dramatically different kinds of herbal medicine, and the proper distinction of the two is crucially important. The first type is supported by some reasonably sound evidence and essentially uses well-tested herbal remedies against specific conditions; this approach has been called by some experts RATIONAL PHYTOTHERAPY. An example is the use of St John’s Wort for depression.
The second type of herbal medicine. It entails consulting a herbal practitioner who takes a history, makes a diagnosis (usually according to obsolete concepts) and prescribes a mixture of several herbal remedies tailor-made to the characteristics of his patient. Thus 10 patients with the identical diagnosis (say depression) might receive 10 different mixtures of herbs. This is true for individualized herbalism of all traditions, e.g. Chinese, Indian or European, and virtually every herbalist you might consult will employ this individualized, traditional approach.
Many consumers know that, in principle, there is some reasonably good evidence for herbal medicine. They fail to appreciate, however, that this does only apply to (sections of) rational phytotherapy. So, they consult herbal practitioners in the belief that they are about to receive an evidence-based therapy. Nothing could be further from the truth! The individualised approach is not evidence-based; even if the individual extracts employed were all supported by sound data (which they frequently are not) the mixutres applied are clearly not.
And this is where the danger of traditional herbalism lies; over the years, herbalists have fooled us all with this fundamental misunderstanding. In the UK, they might even achieve statutory regulation on the back of this self-serving misconception. When this happens, we would have a situation where a completely unproven practice has obtained the same status as doctors, nurses and physiotherapists. If this is not grossly misleading for the consumer, I do not know what is!!!
Some claim that individualized herbalism cannot be tested in clinical trials. This notion can very easily be shown to be wrong: several such studies testing individualized herbalism have been published. To the dismay of traditional herbalists, their results fail to confirm that such treatments are effective for any condition.
Now a further trial has become available that importantly contributes to this knowledge-base. Its authors (all enthusiasts of individualized herbalism) randomized 102 patients suffering from hip or knee-osteoarthritis into two groups. The experimental group received tailor-made mixtures of 7 to 10 Chinese herbs which were traditionally assumed to be helpful. The control group took a mixture of plants known to be ineffective but tasting similar. After 20 weeks of treatment, there were no differences between the groups in any of the outcome measures: pain, stiffness and function. These results thus confirm that this approach is not effective. Not only that, it also carries more risks.
As individualized herbalism employs a multitude of ingredients, the dangers of adverse-effects and herb-drug interactions, contamination, adulteration etc. are bigger that those with the use of single herbal extracts. It seems to follow therefore that the risks of individualized herbalism do not outweigh its benefit.
My recommendations are thus fairly straight forward: if we consider herbal medicine, it is vital to differentiate between the two types. Rational phytotherapy might be fine – of course, depending on the remedy and the condition we are aiming to treat. Individualised or traditional herbalism is not fine; it is not demonstrably effective and has considerable risks. This means consulting a herbalist is not a reasonable approach to treating any human ailment. It also means that regulating herbalists (as we are about to do in the UK) is a seriously bad idea: the regulation of non-sense will result in non-sense!
Rigorous studies of homeopathy are a bit like gold dust; they are so rare that we see perhaps only one or two per year. It is therefore good news that very recently one such trial has been published.
This randomized, placebo-controlled study tested the efficacy of a complex homeopathic medicine, Cocculine, for chemotherapy-induced nausea and vomiting (CINV) in non-metastatic breast cancer patients treated by standard chemotherapy regimens.
Chemotherapy-naive patients with non-metastatic breast cancer scheduled to receive 6 cycles of chemotherapy were randomized to receive standard anti-emetic treatment plus either the complex homeopathic remedy or the matching placebo. The primary endpoint was nausea score measured after the 1st chemotherapy course.
In total, 431 patients were randomized: 214 to Cocculine (C) and 217 to placebo (P). Patient characteristics were well-balanced between the 2 arms. Overall, compliance to study treatments was excellent and similar between the 2 arms. A total of 205 patients (50.9%; 103 patients in the placebo and 102 in the homeopathy arms) had nausea scores > 6 indicative of no impact of nausea on quality of life during the 1st chemotherapy course. There was no difference between the 2 arms when primary endpoint analysis was performed by chemotherapy stratum; or in the subgroup of patients with susceptibility to nausea and vomiting before inclusion. In addition, nausea, vomiting and global emesis scores were not statistically different at any time between the two study arms.
The authors’ conclusions could not be clearer: “This double-blinded, placebo-controlled, randomised Phase III study showed that adding a complex homeopathic medicine (Cocculine) to standard anti-emetic prophylaxis does not improve the control of CINV in early breast cancer patients.”
COCCULINE is manufactured by Boiron and contains Cocculus indicus 4CH, Strychnos nux vomica 4CH, Nicotiana tabacum 4CH, Petroleum rectificatum 4CH aa 0,375 mg. Boiron informs us that “this homeopathic preparation is indicated in sickness during travelling (kinetosis). Preventive dosage is 2 tablets 3 times a day one day before departure and on the day of journey. Treatment dosage is 2 tablets every hour. The interval is prolonged in dependence on improvement. Dosage in children is the same as in adults. The tablets are left to dissolve in mouth or in a small amount of water.”
Homeopaths might argue that this trail did not follow the rules of classical homeopathy where treatments need to be individualised. This may be true but, in this case, they should campaign for all OTC homeopathy to be banned. As they do not do that, I suggest they live with yet another rigorous clinical trial demonstrating that homeopathic remedies are pure placebos.
If I had a pint of beer for every time I have been accused of bias against chiropractic, I would rarely be sober. The thing is that I do like to report about decent research in this field and I am almost every day looking out for new articles which might be worth writing about – but they are like gold dust!
“Huuuuuuuuh, that just shows how very biased he is” I hear the chiro community shout. Well let’s put my hypothesis to the test. Here is a complete list of recent (2013)Medline-listed articles on chiropractic; no omission, no bias, just facts (for clarity, the Pubmed-link is listed first, then the title in bold followed by a short comment in italics):
Towards establishing an occupational threshold for cumulative shear force in the vertebral joint – An in vitro evaluation of a risk factor for spondylolytic fractures using porcine specimens.
This is an interesting study of the shear forces observed in porcine vertebral specimen during maneuvers which might resemble spinal manipulation in humans. The authors conclude that “Our investigation suggested that pars interarticularis damage may begin non-linearly accumulating with shear forces between 20% and 40% of failure tolerance (approximately 430 to 860N”
Development of an equation for calculating vertebral shear failure tolerance without destructive mechanical testing using iterative linear regression.
This is a mathematical modelling of the forces that might act on the spine during manipulation. The authors draw no conclusions.
Collaborative Care for Older Adults with low back pain by family medicine physicians and doctors of chiropractic (COCOA): study protocol for a randomized controlled trial.
This is merely the publication of a trial that is about to commence.
Military Report More Complementary and Alternative Medicine Use than Civilians.
This is a survey which suggests that ~45% of all military personnel use some form of alternative medicine.
Complementary and Alternative Medicine Use by Pediatric Specialty Outpatients
This is another survey; it concludes that ” that CAM use is high among pediatric specialty clinic outpatients”
Extending ICPC-2 PLUS terminology to develop a classification system specific for the study of chiropractic encounters
This is an article on chiropractic terminology which concludes that “existing ICPC-2 PLUS terminology could not fully represent chiropractic practice, adding terms specific to chiropractic enabled coding of a large number of chiropractic encounters at the desired level. Further, the new system attempted to record the diversity among chiropractic encounters while enabling generalisation for reporting where required. COAST is ongoing, and as such, any further encounters received from chiropractors will enable addition and refinement of ICPC-2 PLUS (Chiro)”.
US Spending On Complementary And Alternative Medicine During 2002-08 Plateaued, Suggesting Role In Reformed Health System
This is a study of the money spent on alternative medicine concluding as follows “Should some forms of complementary and alternative medicine-for example, chiropractic care for back pain-be proven more efficient than allopathic and specialty medicine, the inclusion of complementary and alternative medicine providers in new delivery systems such as accountable care organizations could help slow growth in national health care spending”
A Royal Chartered College joins Chiropractic & Manual Therapies.
This is a short comment on the fact that a chiro institution received a Royal Charter.
Exposure-adjusted incidence rates and severity of competition injuries in Australian amateur taekwondo athletes: a 2-year prospective study.
This is a study by chiros to determine the frequency of injuries in taekwondo athletes.
The first thing that strikes me is the paucity of articles; ok, we are talking of just january 2013 but by comparison most medical fields like neurology, rheumatology have produced hundreds of articles during this period and even the field of acupuncture research has generated about three times more.
The second and much more important point is that I fail to see much chiropractic research that is truly meaningful or tells us anything about what I consider the most urgent questions in this area, e.g. do chiropractic interventions work? are they safe?
My last point is equally critical. After reading the 9 papers, I have to honestly say that none of them impressed me in terms of its scientific rigor.
So, what does this tiny investigation suggest? Not a lot, I have to admit, but I think it supports the hypothesis that research into chiropractic is not very active, nor high quality, nor does it address the most urgent questions.
In my very first post on this blog, I proudly pronounced that this would not become one of those places where quack-busters have field-day. However, I am aware that, so far, I have not posted many complimentary things about alternative medicine. My ‘excuse’ might be that there are virtually millions of sites where this area is uncritically promoted and very few where an insider dares to express a critical view. In the interest of balance, I thus focus of critical assessments.
Yet I intend, of course, report positive news when I think it is relevant and sound. So, today I shall discuss a new trial which is impressively sound and generates some positive results:
French rheumatologists conducted a prospective, randomised, double blind, parallel group, placebo controlled trial of avocado-soybean-unsaponifiables (ASU). This dietary supplement has complex pharmacological activities and has been used since years for osteoarthritis (OA) and other conditions. The clinical evidence has, so far, been encouraging, albeit not entirely convincing. My own review arrived at the conclusion that “the majority of rigorous trial data available to date suggest that ASU is effective for the symptomatic treatment of OA and more research seems warranted. However, the only real long-term trial yielded a largely negative result”.
For the new trial, patients with symptomatic hip OA and a minimum joint space width (JSW) of the target hip between 1 and 4 mm were randomly assigned to three years of 300 mg/day ASU-E or placebo. The primary outcome was JSW change at year 3, measured radiographically at the narrowest point.
A total of 399 patients were randomised. Their mean baseline JSW was 2.8 mm. There was no significant difference on mean JSW loss, but there was 20% less progressors in the ASU than in the placebo group (40% vs 50%, respectively). No difference was observed in terms of clinical outcomes. Safety was excellent.
The authors concluded that 3 year treatment with ASU reduces the speed of JSW narrowing, indicating a potential structure modifying effect in hip OA. They cautioned that their results require independent confirmation and that the clinical relevance of their findings require further assessment.
I like this study, and here are just a few reasons why:
It reports a massive research effort; I think anyone who has ever attempted a 3-year RCT might agree with this view.
It is rigorous; all the major sources of bias are excluded as far as humanly possible.
It is well-reported; all the essential details are there and anyone who has the skills and funds would be able to attempt an independent replication.
The authors are cautious in their interpretation of the results.
The trial tackles an important clinical problem; OA is common and any treatment that helps without causing significant harm would be more than welcome.
It yielded findings which are positive or at least promising; contrary to what some people seem to believe, I do like good news as much as anyone else.
I WISH THERE WERE MORE ALT MED STUDIES/RESEARCHERS OF THIS CALIBER!
The ‘Samueli Institute’ might be known to many readers of this blog; it is a wealthy institution that is almost entirely dedicated to promoting the more implausible fringe of alternative medicine. The official aim is “to create a flourishing society through the scientific exploration of wellness and whole-person healing“. Much of its activity seems to be focused on military medical research. Its co-workers include Harald Walach who recently was awarded a rare distinction for his relentless efforts in introducing esoteric pseudo-science into academia.
Now researchers from the Californian branch of the Samueli Institute have published an articles whic, in my view, is another landmark in nonsense.
Jain and colleagues conducted a randomized controlled trial to determine whether Healing Touch with Guided Imagery [HT+GI] reduced post-traumatic stress disorder (PTSD) compared to treatment as usual (TAU) in “returning combat-exposed active duty military with significant PTSD symptoms“. HT is a popular form of para-normal healing where the therapist channels “energy” into the patient’s body; GI is a self-hypnotic from of relaxation-therapy. While the latter approach might be seen as plausible and, at least to some degree, evidence-based, the former cannot.
123 soldiers were randomized to 6 sessions of HT+GI, while the control group had no such therapies. All patients also received standard conventional therapies, and the treatment period was three weeks. The results showed significant reductions in PTSD symptoms as well as depression for HT+GI compared to controls. HT+GI also showed significant improvements in mental quality of life and cynicism.
The authors concluded that HT+GI resulted in a clinically significant reduction in PTSD and related symptoms, and that further investigations of biofield therapies for mitigating PTSD in military populations are warranted.
The Samueli Institute claims to “support science grounded in observation, investigation, and analysis, and [to have] the courage to ask challenging questions within a framework of systematic, high-quality, research methods and the peer-review process“. I do not think that the above-named paper lives up to these standards.
As discussed in some detail in a previous post, this type of study-design is next to useless for determining whether any intervention does any good at all: A+B is always more than B alone! Moreover, if we test HT+GI as a package, how can we conclude about the effectiveness of one of the two interventions? Thus this trial tells us next to nothing about the effectiveness of HT, nor about the effectiveness of HT+GI.
Previously, I have argued that conducting a trial for which the result is already clear before the first patient has been recruited, is not ethical. Samueli Institute, however, claims that it “acts with the highest respect for the public it serves by ensuring transparency, responsible management and ethical practices from discovery to policy and application“. Am I the only one who senses a contradiction here?
Perhaps other research in this area might be more informative? Even the most superficial Medline-search brings to light a flurry of articles on HT and other biofield therapies that are relevant.
Several trials have indeed produces promissing evidence suggesting positive effects of such treatments on anxiety and other symptoms. But the data are far from uniform, and most investigations are wide open to bias. The more rigorous studies seem to suggest that these interventions are not effective beyond placebo. Our review demonstrated that “the evidence is insufficient” to suggest that reiki, another biofield therapy, is an effective treatment for any condition.
Another study showed that tactile touch led to significantly lower levels of anxiety. Conventional massage may even be better than HT, according to some trials. The conclusion from this body of evidence is, I think, fairly obvious: touch can be helpful (most clinicians knew that anyway) but this has nothing to do with energy, biofields, healing energy or any of the other implausible assumptions these treatments are based on.
I therefore disagree with the authors’ conclusion that “further investigation into biofield therapies… is warranted“. If we really want to help patients, let’s find out more about the benefits of touch and let’s not mislead the public about some mystical energies and implausible quackery. And if we truly want to improve heath care, as the Samueli Institute claims, let’s use our limited resources for research which meaningfully contributes to our knowledge.