Can one design a clinical study in such a way that it looks highly scientific but, at the same time, has zero chances of generating a finding that the investigators do not want? In other words, can one create false positive findings at will and get away with it? I think it is possible; what is more, I believe that, in alternative medicine, this sort of thing happens all the time. Let me show you how it is done; four main points usually suffice:
- The first rule is that it ought to be an RCT, if not, critics will say the result was due to selection bias. Only RCTs have the reputation of being ‘top notch’.
- Once we are clear about this design feature, we need to define the patient population. Here the trick is to select individuals with an illness that cannot be quantified objectively. Depression, stress, fatigue…the choice is vast. The aim must be to employ an outcome measure that is well-accepted, validated etc. but which nevertheless is entirely subjective.
- Now we need to consider the treatment to be “tested” in our study. Obviously we take the one we are fond of and want to “prove”. It helps tremendously, if this intervention has an exotic name and involves some exotic activity; this raises our patients’ expectations which will affect the result. And it is important that the treatment is a pleasant experience; patients must like it. Finally it should involve not just one but several sessions in which the patient can be persuaded that our treatment is the best thing since sliced bread – even if, in fact, it is entirely bogus.
- We also need to make sure that, for our particular therapy, no universally accepted placebo exists which would allow patient-blinding. That would be fairly disastrous. And we certainly do not want to be innovative and create such a placebo either; we just pretend that controlling for placebo-effects is impossible or undesirable. By far the best solution would be to give the control group no treatment at all. Like this, they are bound to be disappointed for missing out a pleasant experience which, in turn, will contribute to unfavourable outcomes in the control group. This little trick will, of course, make the results in the experimental group look even better.
That’s about it! No matter how ineffective our treatment is, there is no conceivable way our study can generate a negative result; we are in the pink!
Now we only need to run the trial and publish the positive results. It might be advisable to recruit several co-authors for the publication – that looks more serious and is not too difficult: people are only too keen to prolong their publication-list. And we might want to publish our study in one of the many CAM-journals that are not too critical, as long as the result is positive.
Once our article is in print, we can legitimately claim that our bogus treatment is evidence-based. With a bit of luck, other research groups will proceed in the same way and soon we will have not just one but several positive studies. If not, we need to do two or three more trials along the same lines. The aim is to eventually do a meta-analysis that yields a convincingly positive verdict on our phony intervention.
You might think that I am exaggerating beyond measure. Perhaps a bit, I admit, but I am not all that far from the truth, believe me. You want proof? What about this one?
Researchers from the Charite in Berlin just published an RCT to investigate the effectiveness of a mindful walking program in patients with high levels of perceived psychological distress.
To prevent allegations of exaggeration, selective reporting, spin etc. I take the liberty of reproducing the abstract of this study unaltered:
Participants aged between 18 and 65 years with moderate to high levels of perceived psychological distress were randomized to 8 sessions of mindful walking in 4 weeks (each 40 minutes walking, 10 minutes mindful walking, 10 minutes discussion) or to no study intervention (waiting group). Primary outcome parameter was the difference to baseline on Cohen’s Perceived Stress Scale (CPSS) after 4 weeks between intervention and control.
Seventy-four participants were randomized in the study; 36 (32 female, 52.3 ± 8.6 years) were allocated to the intervention and 38 (35 female, 49.5 ± 8.8 years) to the control group. Adjusted CPSS differences after 4 weeks were -8.8 [95% CI: -10.8; -6.8] (mean 24.2 [22.2; 26.2]) in the intervention group and -1.0 [-2.9; 0.9] (mean 32.0 [30.1; 33.9]) in the control group, resulting in a highly significant group difference (P < 0.001).
Conclusion. Patients participating in a mindful walking program showed reduced psychological stress symptoms and improved quality of life compared to no study intervention. Further studies should include an active treatment group and a long-term follow-up
This whole thing could just be a bit of innocent fun, but I am afraid it is neither innocent nor fun, it is, in fact, quite serious. If we accept manipulated trials as evidence, we do a disservice to science, medicine and, most importantly, to patients. If the result of a trial is knowable before the study has even started, it is unethical to run the study. If the trial is not a true test but a simple promotional exercise, research degenerates into a farcical pseudo-science. If we abuse our patients’ willingness to participate in research, we jeopardise more serious investigations for the benefit of us all. If we misuse the scarce funds available for research, we will not have the money to conduct much needed investigations. If we tarnish the reputation of clinical research, we hinder progress.
A lengthy article posted by THE HOMEOPATHIC COLLEGE recently advocated treating cancer with homeopathy. Since I doubt that many readers access this publication, I take the liberty of reproducing here their (also fairly lengthy) CONCLUSIONS in full:
Laboratory studies in vitro and in vivo show that homeopathic drugs, in addition to having the capacity to reduce the size of tumors and to induce apoptosis, can induce protective and restorative effects. Additionally homeopathic treatment has shown effects when used as a complementary therapy for the effects of conventional cancer treatment. This confirms observations from our own clinical experience as well as that of others that when suitable remedies are selected according to individual indications as well as according to pathology and to cell-line indications and administered in the appropriate doses according to the standard principles of homeopathic posology, homeopathic treatment of cancer can be a highly effective therapy for all kinds of cancers and leukemia as well as for the harmful side effects of conventional treatment. More research is needed to corroborate these clinical observations.
Homeopathy over almost two decades of its existence has developed more than four hundred remedies for cancer treatment. Only a small fraction have been subjected to scientific study so far. More homeopathic remedies need to be studied to establish if they have any significant action in cancer. Undoubtedly the next big step in homeopathic cancer research must be multiple comprehensive double-blinded, placebo-controlled, randomized clinical trials. To assess the effect of homeopathic treatment in clinical settings, volunteer adult patients who prefer to try homeopathic treatment instead of conventional therapy could be recruited, especially in cases for which no conventional therapy has been shown to be effective.
Many of the researchers conducting studies — cited here but not discussed — on the growing interest in homeopathic cancer treatment have observed that patients are driving the demand for access to homeopathic and other alternative modes of cancer treatment. So long as existing cancer treatment is fraught with danger and low efficacy, it is urgent that the research on and the provision of quality homeopathic cancer treatment be made available for those who wish to try it.
When I report about nonsense like that, I find it hard not to go into a fuming rage. But doing that would not be very constructive – so let me instead highlight (in random order) eight simple techniques that seem to be so common when unsubstantiated claims are being promoted for alternative treatments:
1) cherry pick the data
2) use all sorts of ‘evidence’ regardless how flimsy or irrelevant it might be
3) give yourself the flair of being highly scientific and totally impartial
4) point out how dangerous and ineffective all the conventional treatments are
5) do not shy away from overt lies
6) do not forget to stress that the science is in full agreement with your exhaustive clinical experience
7) stress that patients want what you are offering
8) ignore the biological plausibility of the underlying concepts
Provided we adhere to these simple rules, we can convince the unsuspecting public of just about anything – even of the notion that homeopathy is a cure for cancer!
Swiss chiropractors have just published a clinical trial to investigate outcomes of patients with radiculopathy due to cervical disk herniation (CDH). All patients had neck pain and dermatomal arm pain; sensory, motor, or reflex changes corresponding to the involved nerve root and at least one positive orthopaedic test for cervical radiculopathy were included. CDH was confirmed by magnetic resonance imaging. All patients received regular neck manipulations.
Baseline data included two pain numeric rating scales (NRSs), for neck and arm, and the Neck Disability Index (NDI). At two, four and twelve weeks after the initial consultation, patients were contacted by telephone, and the data for NDI, NRSs, and patient’s global impression of change were collected. High-velocity, low-amplitude thrusts were administered by experienced chiropractors. The proportion of patients reporting to feel “better” or “much better” on the patient’s global impression of change scale was calculated. Pre-treatment and post-treatment NRSs and NDIs were analysed.
Fifty patients were included. At two weeks, 55.3% were “improved,” 68.9% at four and 85.7% at twelve weeks. Statistically significant decreases in neck pain, arm pain, and NDI scores were noted at 1 and 3 months compared with baseline scores. 76.2% of all sub-acute/chronic patients were improved at 3 months.
The authors concluded that most patients in this study, including sub-acute/chronic patients, with symptomatic magnetic resonance imaging-confirmed CDH treated with spinal manipulative therapy, reported significant improvement with no adverse events.
In the presence of disc herniation, chiropractic manipulations have been described to cause serious complications. Some experts therefore believe that CDH is a contra-indication for spinal manipulation. The authors of this study imply, however, that it is not – on the contrary, they think it is an effective intervention for CDH.
One does not need to be a sceptic to notice that the basis for this assumption is less than solid. The study had no control group. This means that the observed effect could have been due to:
a placebo response,
the regression towards the mean,
the natural history of the condition,
or other factors which have nothing to do with the chiropractic intervention per se.
And what about the interesting finding that no adverse-effects were noted? Does that mean that the treatment is safe? Sorry, but it most certainly does not! In order to generate reliable results about possibly rare complications, the study would have needed to include not 50 but well over 50 000 patients.
So what does the study really tell us? I have pondered over this question for some time and arrived at the following answer: NOTHING!
Is that a bit harsh? Well, perhaps yes. And I will revise my verdict slightly: the study does tell us something, after all – chiropractors tend to confuse research with the promotion of very doubtful concepts at the expense of their patients. I think, there is a name for this phenomenon: PSEUDO-SCIENCE.
One thing that unites all (well, almost all, in my experience) proponents of alternative medicine is their intense dislike for BIG PHARMA. Essentially, they see this sector as:
- Driven by profit
- Employing unethical means to maximise profit
- Not caring for the needs of patients
- Attacking alternative medicine for fear of losing profit
And, of course, they claim that alternative medicine, LITTLE ALT MED, is fundamentally different from the cynically capitalist, malign BIG PHARMA.
I have no intention to defend the ways of the pharmaceutical industry – neither on this blog nor anywhere else. This industry is usually responsible to their share-holders and that constellation can lead to excesses which are counter-productive to our needs, to put it mildly. However, what I will question is the notion that LITTLE ALT MED is fundamentally different from BIG PHARMA.
We all have to make a living; to some extend at least we are therefore all driven by our desire to earn money. In alternative medicine, there are certainly not as many mega-enterprises as in the pharmaceutical industry but nobody can deny that many sizable firms exist which make a profit selling alternative remedies of one type or another.
And those parts of alternative medicine which are not into the sale of remedies, you may well ask – think of acupuncture, for instance. Well, those therapists are not exempt either from the need to make a living. Sure, this is on a different scale from BIG PHARMA, but it constitutes still a need for profit. If we multiply the relatively small sums involved by the vast number of therapists, the grand total of LITTLE ALT MED might approach similar orders of magnitude as that of BIG PHARMA.
Ok, but the alternative sector would not employ unethical means for securing or maximising profits! Wrong again, I am afraid.
My 20 years of experience of LITTLE ALT MED have let me witness several incidents which I would not hesitate to call unethical. One of the most recent and least pleasant, from my point of view, was the discovery that several German homeopathic manufacturers had given money to a ‘journalist’ who used these funds to systematically defame me.
What about the charge that BIG PHARMA does not care for the suffering patient? LITTLE ALT MED would never do that!!! Sadly this is a myth too.
Alternative practitioners and their organisations make a plethora of therapeutic claims which are not substantiated. Who would deny that misleading patients into making wrong health care decisions is not the opposite from ‘caring’? What seems even worse, in my view, is the behaviour that might follow the exposure of such behaviour. If someone is courageous enough to disclose the irresponsibility of bogus claims, he might be attacked or even taken to court by those who, in reality should be in the dock or, at least, do their utmost to get their house in order.
Finally, we have the notion of BIG PHARMA trying to suppress LITTLE ALT MED. I call this a myth too because I see absolutely no evidence for this rumour. Even those who circulate it can, when challenged, not produce any.
And, anyway, we all know how many of the big pharmaceutical firms buy into the alternative medicine market as soon as they see a commercially viable opportunity. Does that look like suppression?
So, what is the conclusion? BIG PHARMA can behave badly, no doubt, and when they do, I am as disgusted as the next man. However, LITTLE ALT MED also can behave badly – in fact, wherever there is money to be made, some people will behave badly some of the time.
Perhaps we should not judge an entire sector just by the bad actions of some of its members, but perhaps we should also consider whether or not it has done any good. Who would doubt that BIG PHARMA has helped to save lives – millions of lives?!
And now ask yourself: can we honestly say the same about LITTLE ALT MED?
Two of the top US general medical journals have just published articles which somehow smell of the promotion of quackery. A relatively long comment on alternative medicine, entitled THE FUTURE OF INTEGRATIVE MEDICINE appeared in THE AMERICAN JOURNAL OF MEDICINE and another one entitled PERSPECTIVES ON COMPLEMENTARY AND ALTERNATIVE MEDICINE RESEARCH in THE JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION. As this sort of thing does not happen that often, it is perhaps worth having a closer look at these publications. The JAMA-article has already been analysed skilfully by Orac, so I will not criticise it further. In the following text, the passages which are in italics are direct quotes from the AJM-article, while the interceptions in normal print are my comments on it.
…a field of unconventional medicine has evolved that has been known by a progression of names: holistic medicine, complementary and alternative medicine, and now integrative medicine. These are NOT synonyms, and there are many more names which have been forgotten, e.g. fringe, unorthodox, natural medicine It is hoped that the perspectives offered by integrative medicine will eventually transform mainstream medicine by improving patient outcomes, reducing costs, improving safety, and increasing patient satisfaction. Am I the only one to feel this sentence is a platitude?
Integrative medicine has been defined as “the practice of medicine that reaffirms the importance of the relationship between practitioner and patient, focuses on the whole person, is informed by evidence, and makes use of all appropriate therapeutic approaches, healthcare professionals and disciplines to achieve optimal health and healing.” There is, in fact, no accepted definition; the most remarkable bit in this one is perhaps the term “informed by evidence” which, as we will see shortly, is by no means the same as “evidence-based”, the accepted term and principle in medicine.
The most obvious differences between integrative and conventional medicine are its practitioners, who offer longer consultations and emphasize minimally invasive therapies, such as mind-body approaches, nutrition, prevention, and lifestyle changes, and focus on healing and wellness. Come again! Is that supposed to mean that conventional doctors do not employ “minimally invasive therapies or prevention or nutrition etc.”? In addition to conventional therapies, they may recommend alternatives, such as acupuncture, dietary supplements, and botanicals. BINGO! The difference between integrative and conventional physicians is quite simply that the former put an emphasis on unproven treatments; evidence my foot! This is just quackery by a different name. The doctor-patient relationship emphasizes joint decision-making by the patient and the physician. Yes, that may be true, but it does so in any type of good health care. To imply that the doctor-patient relationship and joint decision-making is an invention of integrative medicine is utter nonsense.
More and more patients seek integrative medicine practitioners. By 2007, approximately 40% of adult Americans and 12% of children were using some form of alternative therapies compared with 33% in 1991.
The number of US hospitals offering integrative therapies, such as acupuncture, massage therapy, therapeutic touch, and guided imagery, has increased from 8% in 1998 to 42% in 2010.Many academic cancer centers offer these integrative practices as part of a full spectrum of care. Other hospitals offer programs in integrative women’s health, cardiology, and pain management. But why? I think the authors forgot to mention that the main reason here is to make money.
Despite the increasing number of patients seeking alternative therapies, until recently, many of these skills were not routinely offered in medical schools or graduate medical education. Yet they are critical competencies and essential to stemming the tide of chronic diseases threatening to overwhelm both our health care and our financial systems. Essential? Really? Most alternative therapies are, in fact, unproven or disproven! Further, conventional medical journals rarely contained articles about alternative therapies until 1998 when the Journal of the American Medical Association and its affiliated journals published more than 60 articles on the theme of complementary and alternative medicine.
The National Institutes of Health established an office in 1994 and a National Center for Complementary and Alternative Medicine in 1998. Because many alternative therapies date back thousands of years, their efficacy has not been tested in randomized clinical trials. The reasons for the lack of research may be complex but they have very little to do with the long history of the modalities in question. The National Center for Complementary and Alternative Medicine provides the funds to conduct appropriate trials of these therapies. The NCCAM- funded studies have been criticised over and over again and most scientists find them not at all “appropriate”. They also have funded education research and programs in both conventional medical nursing schools and complementary and alternative medicine professional schools. Outcomes of these studies are being published in the conventional medical literature. Not exactly true! Much of it is published in journals of alternative medicine. Also, the authors forgot to mention that none of the studies of NCCAM have ever convincingly shown an alternative treatment to work.
Integrative medicine began to have an impact on medical education when 8 medical school deans met in 1999 to discuss complementary and alternative medicine. This meeting led to the establishment of the Consortium of Academic Health Centers for Integrative Medicine, composed initially of 11 academic centers. By 2012, this group had grown to 54 medical and health profession schools in the United States, Canada, and Mexico that have established integrative medicine programs. The consortium’s first international research conference on integrative medicine was held in 2006, with subsequent research conferences being held in 2009 and 2012. Three conferences? Big deal! I have hosted 14 research conferences in Exeter in as many years. I think, the authors are here blowing up a mouse to look like an elephant.
Multiple academic integrative medicine programs across the country have been supported by National Institutes of Health funding and private contributions, including the Bravewell Collaborative that was founded in 2002 by a group of philanthropists. The goal of the Bravewell Collaborative is “to transform the culture of healthcare by advancing the adoption of Integrative Medicine.” It foremost was an organisation of apologists of alternative medicine and quackery. A high water mark also occurred in 2009 when the Institute of Medicine held a Summit on Integrative Medicine led by Dr Ralph Snyderman.
There is clear evidence that integrative medicine is becoming part of current mainstream medicine. Really? I would like to see it. Increasing numbers of fellowships in integrative medicine are being offered in our academic health centers. In 2013, there are fellowships in integrative medicine in 13 medical schools. In 2000, the University of Arizona established a 1000-hour online fellowship that has been completed by more than 1000 physicians, nurse practitioners, and physician assistants. This online fellowship makes it possible for fellows to continue their clinical practice during their fellowship. I see, this is supposed to be the evidence?
A 200-hour curriculum for Integrative Medicine in Residency has been developed and is now in place in 30 family practice and 2 internal medicine residencies. The curriculum includes many of the topics that are not covered in the medical school curriculum, such as nutrition, mind–body therapies, nutritional and botanical supplements, alternative therapies (eg, acupuncture, massage, and chiropractic), and lifestyle medicine. It is not true that conventional medical schools do not teach about nutrition, psychology etc. Not all might, however, teach overt quackery. A similar curriculum for pediatric residencies is being developed. The eventual goal is to include integrative medicine skills and competencies in all residency programs.
Integrative medicine now has a broad presence in medical education, having evolved because of public demand, student and resident interest, increased research, institutional support, and novel educational programs. Now on the horizon is a more pluralistic, pragmatic approach to medicine that is patient-centered, that offers the broadest range of potential therapies, and that advocates not only the holistic treatment of disease but also prevention, health, and wellness.
Is it not an insult to conventional medicine to imply it is not pluralistic, pragmatic, patient-centred, that it does not offer a broad range of therapies, holism and prevention? This article displays much of what is wrong with the mind-set of the apologists of alternative medicine. The more I think about it, the more I feel that it is a bonanza of fallacies, follies and attempts to white-wash quackery. But I would be interested in how my readers see it.
In a recent comment, US chiropractors stated that there is a growing recognition within the profession that the practicing chiropractor must be able to do the following: formulate a searchable clinical question, rapidly access the best evidence available, assess the quality of that evidence, determine if it is applicable to a particular patient or group of patients, and decide if and how to incorporate the evidence into the care being offered. In a word, they believe, that evidence-based chiropractic is possible, perhaps even (almost) a reality. For evidence-based practice to penetrate and transform a profession, the penetration must occur at two levels, they explain. One level is the degree to which individual practitioners possess the willingness and basic skills to search and assess the literature.
The second level, the authors explain, relates to whether the therapeutic interventions commonly employed by a particular health care discipline are supported by clinical research. The authors believe that a growing body of randomized controlled trials provides evidence of the effectiveness and safety of manual therapies. Is this really true, I wonder.
In support of these fairly bold statements, they cite a paper by Bronfort et al which, in their view, is currently the most comprehensive review of the evidence for the efficacy of manual therapies. According to these authors, the ‘Bronfort-report’ stated that evidence is inconclusive for pneumonia, stage 1 hypertension, pre-menstrual syndrome, nocturnal enuresis, and otitis media. The authors also believe that it is unlikely manipulation of the neck is causally related to stroke.
When I read this article, I could not stop myself from giggling. It seems to me that it provides pretty good evidence for the fact that the chiropractic profession is nowhere near reaching the stage where anyone could reasonably claim that chiropractors practice evidence-based medicine – not even the authors themselves seem to abide by the rules of evidence-based medicine! If they had truly been able to access the best evidence available and assess the quality of that evidence surely they would not have (mis-) quoted the ‘Bronfort-report’.
Bronfort’s overview was commissioned by the General Chiropractic Council, it was hastily compiled by ardent believers of chiropractic, published in a journal that non-chiropractors would not touch with a barge pole, and crucially it lacks some of the most important qualities of an unbiased systematic review. In my view, it is nothing short of a white-wash and not worth the paper it was printed on. Conclusions, such as the evidence regarding pneumonia, bed-wetting and otitis is inconclusive are just embarrassing; the correct conclusion is that the evidence fails to be positive for these and most other indications.
Similarly, if the authors had really studied and quoted the best evidence, how on earth could they have stated that manipulation of the neck cannot cause a stroke? The evidence for that is fairly overwhelming, and the only open question here is, how often do such complications occur? And even the biased ‘Bronfort-report’ states: Adverse events associated with manual treatment can be classified into two categories: 1) benign, minor or non-serious and 2) serious. Generally those that are benign are transient, mild to moderate in intensity, have little effect on activities, and are short lasting. Most commonly, these involve pain or discomfort to the musculoskeletal system. Less commonly, nausea, dizziness or tiredness are reported. Serious adverse events are disabling, require hospitalization and may be life-threatening. The most documented and discussed serious adverse event associated with spinal manipulation (specifically to the cervical spine) is vertebrobasilar artery (VBA) stroke. Less commonly reported are serious adverse events associated with lumbar spine manipulation, including lumbar disc herniation and cauda equina syndrome.
Evidence-based practice? Who are these chiropractors kidding? This article very neatly reflects the exact opposite. It ignores hundreds of peer-reviewed papers which are critical of chiropractic. The best one can do with this paper, I think, is to use it as a hilarious bit of involuntary humour or as a classic example of cherry-picking.
Come to think of it, chiropractic and evidence-based practice are contradictions in terms. Either a therapist claims to adjust mystical subluxations, in which case he/she does not practice evidence-based medicine. Or he/she practices evidence-based medicine, in which case adjusting mystical subluxations cannot be part of their therapeutic repertoire.
Towards the end of the article, we learn further fascinating things: the authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article – oh, really?!?! Furthermore, we are told that this ‘research’ was funded by the ‘National Center of Complementary and Alternative Medicine’ (NCCAM) of the National Institutes of Health.
Can it be true? Does the otherwise most respectable NIH really give its name for such overt nonsense? Yes, it is true, and it is by no means the first time. In fact, our analysis shows that, when it comes to chiropractic, this organisation has sponsored almost nothing but utter rubbish, and our conclusion was blunt: the criticism repeatedly aimed at NCCAM seems justified, as far as their RCTs of chiropractic is concerned. It seems questionable whether such research is worthwhile.
Research is essential for progress, and research in alternative medicine is important for advancing alternative medicine, one would assume. But why then do I often feel that research in this area hinders progress? One of the reasons is, in my view, the continuous drip, drip, drip of misleading conclusions usually drawn from weak studies. I could provide thousands of examples; here is one recently published article chosen at random which seems as good as any other to make the point.
Researchers from the Department of Internal and Integrative Medicine, Faculty of Medicine, University of Duisburg-Essen, Germany set out to investigate associations of regular yoga practice with quality of life and mental health in patients with chronic diseases. Using a case-control study design, 186 patients with chronic diseases who had elected to regularly practice yoga were selected and compared to controls who had chosen to not regularly practice yoga. Patients were matched individually on gender, main diagnosis, education, and age. Patients’ quality of life, mental health, life satisfaction, and health satisfaction were also assessed. The analyses show that patients who regularly practiced yoga had a significantly better general health status, a higher physical functioning, and physical component score on the SF-36 than those who did not.
The authors concluded that practicing yoga under naturalistic conditions seems to be associated with increased physical health but not mental health in chronically diseased patients.
Why do I find these conclusions misleading?
In alternative medicine, we have an irritating abundance of such correlative research. By definition, it does not allow us to make inferences about causation. Most (but by no means all) authors are therefore laudably careful when choosing their terminology. Certainly, the present article does not claim that regular yoga practice has caused increased physical health; it rightly speaks of “associations“. And surely, there is nothing wrong with that – or is there?
Perhaps, I will be accused of nit-picking, but I think the results are presented in a slightly misleading way, and the conclusions are not much better.
Why do the authors claim that patients who regularly practiced yoga had a significantly better general health status, a higher physical functioning, and physical component score on the SF-36 than those who did not than those who did not? I know that the statement is strictly speaking correct, but why do they not write that “patients who had a significantly better general health status, a higher physical functioning, and physical component score on the SF-36 were more likely to practice yoga regularly”? After all, this too is correct! And why does the conclusion not state that better physical health seems to be associated with a greater likelihood of practicing yoga?
The possibility that the association is the other way round deserves serious consideration, in my view. Is it not logical to assume that, if someone is relatively fit and healthy, he/she is more likely to take up yoga (or table-tennis, sky-diving, pole dancing, etc.)?
It’s perhaps not a hugely important point, so I will not dwell on it – but, as the alternative medicine literature is full with such subtly misleading statements, I don’t find it entirely irrelevant either.
A recently published study by Danish researchers aimed at comparing the effectiveness of a patient education (PEP) programme with or without the added effect of chiropractic manual therapy (MT) to a minimal control intervention (MCI). Its results seem to indicate that chiropractic MT is effective. Is this the result chiropractors have been waiting for?
To answer this question, we need to look at the trial and its methodology in more detail.
A total of 118 patients with clinical and radiographic unilateral hip osteoarthritis (OA) were randomized into one of three groups: PEP, PEP+ MT or MCI. The PEP was taught by a physiotherapist in 5 sessions. The MT was delivered by a chiropractor in 12 sessions, and the MCI included a home stretching programme. The primary outcome measure was the self-reported pain severity on an 11-box numeric rating scale immediately following the 6-week intervention period. Patients were subsequently followed for one year.
The primary analyses included 111 patients. In the PEP+MT group, a statistically and clinically significant reduction in pain severity of 1.9 points was noted compared to the MCI of 1.90. The number needed to treat for PEP+MT was 3. No difference was found between the PEP and the MCI groups. At 12 months, the difference favouring PEP+MT was maintained.
The authors conclude that for primary care patients with osteoarthritis of the hip, a combined intervention of manual therapy and patient education was more effective than a minimal control intervention. Patient education alone was not superior to the minimal control intervention.
This is an interesting, pragmatic trial with a result suggesting that chiropractic MT in combination with PEP is effective in reducing the pain of hip OA. One could easily argue about the small sample size, the need for independent replication etc. However, my main concern is the fact that the findings can be interpreted in not just one but in at least two very different ways.
The obvious explanation would be that chiropractic MT is effective. I am sure that chiropractors would be delighted with this conclusion. But how sure can we be that it would reflect the truth?
I think an alternative explanation is just as (possibly more) plausible: the added time, attention and encouragement provided by the chiropractor (who must have been aware what was at stake and hence highly motivated) was the effective element in the MT-intervention, while the MT per se made little or no difference. The PEP+MT group had no less than 12 sessions with the chiropractor. We can assume that this additional care, compassion, empathy, time, encouragement etc. was a crucial factor in making these patients feel better and in convincing them to adhere more closely to the instructions of the PEP. I speculate that these factors were more important than the actual MT itself in determining the outcome.
In my view, such critical considerations regarding the trial methodology are much more than an exercise in splitting hair. They are important in at least two ways.
Firstly, they remind us that clinical trials, whenever possible, should be designed such that they allow only one interpretation of their results. This can sometimes be a problem with pragmatic trials of this nature. It would be wise, I think, to conduct pragmatic trials only of interventions which have previously been proven to work. To the best of my knowledge, chiropractic MT as a treatment for hip OA does not belong to this category.
Secondly, it seems crucial to be aware of such methodological issues and to consider them carefully before research findings are translated into clinical practice. If not, we might end up with therapeutic decisions (or guidelines) which are quite simply not warranted.
I would not be in the least surprised, if chiropractic interest groups were to use the current findings for promoting chiropractic in hip-OA. But what, if the MT per se was ineffective, while the additional care, compassion and encouragement was? In this case, we would not need to recruit (and pay for) chiropractors and put up with the considerable risks chiropractic treatments can entail; we would merely need to modify the PE programme such that patients are better motivated to adhere to it.
As it stands, the new study does not tell us much that is of any practical use. In my view, it is a pragmatic trial which cannot readily be translated into evidence-based practice. It might get interpreted as good news for chiropractic but, in fact, it is not.
Sorry, but I am fighting a spell of depression today.
Why? I came across this website which lists the 10 top blogs on alternative medicine. To be precise, here is what they say about their hit-list: this list includes the top 10 alternative medicine bloggers on Twitter, ranked by Klout score. Using Cision’s media database, we compiled the list based on Cision’s proprietary research, with results limited to bloggers who dedicate significant coverage to alternative medicine and therapies…
And here are the glorious top ten:
All of these sites are promotional and lack even the slightest hint of critical evaluation. All of them sell or advertise products and are thus out to make money. All of them are full of quackery, in my view. Some of the most popular bloggers are world-famous quacks!
What about impartial information for the public? What about critical review of the evidence? What about a degree of balance? What about guiding consumers to make responsible, evidence-based decisions? What about preventing harm? What about using scarce resources wisely?
I don’t see any of this on any of the sites.
You see, now I have depressed you too!
Quick, buy some herbal, natural, holistic and integrative anti-depressant! As it happens, I have some for sale….
Even after all these years of full-time research into alternative medicine and uncounted exchanges with enthusiasts involved in this sector, I find the logic that is often applied in this field bewildering and the unproductiveness of the dialogue disturbing.
To explain what I mean, it be might best to publish a (fictitious, perhaps slightly exaggerated) debate between a critical thinker or scientist (S) and an uncritical proponent (P) of one particular form of alternative medicine.
P: Did you see this interesting study demonstrating that treatment X is now widely accepted, even by highly critical GPs at the cutting edge of health care?
S: This was a survey, not a ‘study’, and I never found the average GP “highly critical”. Surveys of this nature are fairly useless and they “demonstrate” nothing of real value.
P: Whatever, but it showed that GPs accept treatment X. This can only mean that they realise how safe and effective it is.
S: Not necessarily, GPs might just give in to consumer demand, or the sample was cleverly selected, or the question was asked in a leading manner, etc.
P: Hardly, because there is plenty of good evidence for treatment X.
S: Really? Show me.
P: There is this study here which proves that treatment X works and is risk-free.
S: The study was far too small to demonstrate safety, and it is wide open to multiple sources of bias. Therefore it does not conclusively show efficacy either.
P: You just say this because you don’t like its result! You have a closed mind!
In any case, it was merely an example! There are plenty more positive studies; do your research properly before you talk such nonsense.
S: I did do some research and I found a recent, high quality systematic review that arrived at a negative conclusion about the value of treatment X.
P: That review was done by sceptics who clearly have an axe to grind. It is based on studies which do not account for the intrinsic subtleties of treatment X. Therefore they are unfair tests of treatment X. These trials don’t really count at all. Every insider knows that! The fact that you cite it merely confirms that you do not understand what you are talking about.
S: It seems to me, that you like scientific evidence only when it confirms your belief. This, I am afraid, is what quacks tend to do!
P: I strongly object to being insulted in this way.
S: I did not insult you, I merely made a statement of fact.
P: If you like facts, you have to see that one needs to have sufficient expertise in treatment X in order to apply it properly and effectively. This important fact is neglected in all of those trials that report negative results; and that’s why they are negative. Simple! I really don’t understand why you are too stupid to understand this. Such studies do not show that treatment X is ineffective, but they demonstrate that the investigators were incompetent or hired with the remit to discredit treatment X.
S: I would have thought they are negative because they minimised bias and the danger of generating a false positive result.
P: No, by minimising bias, as you put it, these trials eliminated the factors that are important elements of treatment X.
S: Such as the placebo-effect?
P: That’s what you call it because you irrationally believe in reductionist science.
S: Science requires no belief, I think you are the believer here.
P: The fact is that scientists of your ilk negate all factors related to human interactions. Patients are no machines, you know, they need compassion; we clinicians know that because we work at the coal face of health care. Scientists in their ivory towers have no idea about patient care and just want science for science sake. This is not how you help patients. Show some compassion man!
S: I do know about the importance of compassion and care, but here we are discussing an entirely different topic, namely tests the efficacy or effectiveness of treatments, not patient-care. Let’s focus on one issue at a time.
P: You cannot separate things in this way. We have to take a holistic view. Patients are whole individuals, and you cannot do them justice by running artificial experiments. Every patient is different; clinical trials fail to account for this fact and are therefore fairly irrelevant to us and to our patients. Real life is very different from your imagined little experiments, you know.
S: These are platitudes that are nonsensical in this context and do not contribute anything meaningful to the present discussion. You do not seem to understand the methodology or purpose of a clinical trial.
P: That is typical! Whenever you run out of arguments, you try to change the subject or throw a few insults at me.
S: Not at all, I thought we were talking about clinical trials evaluating the effectiveness of treatment X.
P: That’s right; and they do show that it is effective, provided you consider those which are truly well-done by experts who know about treatment X and believe in it.
S: Not true. Only if you cherry-pick the data will you be able to produce an overall positive result for treatment X.
P: In any case, the real world results of clinical practice show very clearly that it works. It would not have survived for so long, if it didn’t. Nobody can deny that, and nobody should claim that silly little trials done in artificial circumstances are more meaningful than a wealth of experience.
S: Experience has little to do with reliable evidence.
P: To deny the value of experience is just stupid and clearly puts you in the wrong. I have shown you plenty of reliable evidence but you just ignore everything I say that does not go along with your narrow-minded notions about science; science is not the only way of knowing or comprehending things! Stop being obsessed with science.
S: No, you show me rubbish data and have little understanding of science, I am afraid.
P: Here we go again! I have had about enough of that and your blinkered arguments. We are going in circles because you are ignorant and arrogant. I have tried my best to show you the light, but your mind is closed. I offer true insight and you pay me back with insults. You and your cronies are in the pocket of BIG PHARMA. You are cynical, heartless and not interested in the wellbeing of patients. Next you will tell me to vaccinate my kids!
S: I think this is a waste of time.
P: Precisely! Everyone who has followed this debate will see very clearly that you are obsessed with reductionist science and incapable of considering the suffering of whole individuals. You want to deny patients a treatment that really helps them simply because you do not understand how treatment X works. Shame on you!!!