research methodology
In the UK, we have about 150000 practitioners of Spiritual Healing (SH). They treat all sorts of conditions claiming to channel ‘healing energy’ into the patient’s body which enables him/her to heal itself. The plausibility of SH is very close to zero and, despite numerous trials, its clinical effectiveness remains unproven. A new and, in my view, remarkable study of SH was aimed at investigating whether “SH could support patients with breast cancer”.
Spiritual Healing was provided by 4 healers registered with the National Federation of Spiritual Healers. Twelve patients with breast cancer undergoing long-term hormone treatment and experiencing its adverse-effects as onerous, self-referred themselves and were given ten weekly sessions of approximately 40 minutes each. Data collected included participant’s daily records, direct observations noted by the healers, the researcher’s field diary and a one-to-one semi-structured interview.
The alleged positive effects of SH included alleviation of the physical adverse-effects of their treatment, increased energy levels, enhanced well-being, emotional relaxation, and re-engagement with pre-cancer activities. Although one participant admitted considering a drug holiday prior to joining the study, none of the participants felt tempted to stop their hormonal treatments while receiving SH. The authors concluded that “these qualitative findings indicate that SH has the potential to support patients with breast cancer in the maintenance of their long-term orthodox treatments. Further research is needed to test SH as a cost-effective complementary therapy, for those undergoing long-term cancer treatments.”
As I already mentioned, I think this study is remarkable. Having done quite a bit of research into SH myself, I know how bizarre this intervention truly is. A typical treatment session might be with the patient lying on a couch in a relaxing atmosphere, often accompanied by soothing background music; the healer would talk gently but very little to enable the patient to be comfortable and relax; the SH itself might be performed by the healer moving his/her hands at a distance over the body of the patient; the healer would explain that this may cause the sensation of warmth as the ‘healing energy’ enters the body. Altogether, the experience is exotic to say the least.
It is therefore not surprising that SH generates a host of non-specific effects, including the most enormous placebo-response I have ever witnessed in any clinical trial which I have been involved in. I am mentioning this, of course, to point out that the above-noted effects are entirely compatible with those of placebo. As the study has no control group, there is no way of knowing what the effects of SH per se might have been. The fact that patients self-referred themselves to SH would only amplify this placebo-response. In the discussion of the paper, we find a further interesting pointer regarding patients’ prior experience with conventional health care professionals: “participants felt they were left to cope alone as their side-effects were trivialized.” This seems to suggest that the group of patients were indeed highly selected and all had suffered badly from previous experiences of poorly administered heath care. Thus their expectations of SH were probably high which, in turn, would exaggerate the placebo-response even further.
All of these phenomena might well be fascinating and could provide ample material for relevant research. They deserve to be analysed carefully and discussed openly and critically. Unfortunately none of this happened in the present study. The authors do not even consider the possibility that the observed effects could be related to anything else than their SH. Their stated aim to investigate whether SH supports cancer patients is not even approached; the authors simply assume a cause-effect relationship without demonstrating one. I find this is more than just a missed opportunity; in my view, it is pseudo-science. And this is the reason why I find this study remarkable.
This post has an odd title and addresses an odd subject. I am sure some people reading it will ask themselves “has he finally gone potty; is he a bit xenophobic, chauvinistic, or what?” I can assure you none of the above is the case.
Since many years, I have been asked to peer-review Chinese systematic reviews and meta-analyses of TCM-trials submitted to various journals and to the Cochrane Collaboration for publication, and I estimate that around 300 such articles are available today. Initially, I thought they were a valuable contribution to our knowledge, particularly for the many of us who cannot read Chinese languages. I hoped they might provide reliable information about this huge and potentially important section of the TCM-evidence. After doing this type of work for some time, I became more and more frustrated; now I have decided not to accept this task any longer – not because it is too much trouble, but because I have come to the conclusion that these articles are far less helpful than I had once assumed; in fact, I now fear that they are counter-productive.
In order to better understand what I mean, it might be best to use an example; this recent systematic review seems as good for that purpose as any.
Its Chinese authors “hypothesized that the eligible trials would provide evidence of the effect of Chinese herbs on bone mineral density (BMD) and the therapeutic benefits of Chinese medicine treatment in patients with bone loss“. Randomized controlled trials (RCTs) were thus retrieved for a systematic review from Medline and 8 Chinese databases. The authors identified 12 RCTs involving a total of 1816 patients. The studies compared Chinese herbs with placebo or standard anti-osteoporotic therapy. The pooled data from these RCTs showed that the change of BMD in the spine was more pronounced with Chinese herbs compared to the effects noted with placebo. Also, in the femoral neck, Chinese herbs generated significantly higher increments of BMD compared to placebo. Compared to conventional anti-osteoporotic drugs, Chinese herbs generated greater BMD changes.
In their abstract, the part on the paper that most readers access, the authors reached the following conclusions: “Our results demonstrated that Chinese herb significantly increased lumbar spine BMD as compared to the placebo or other standard anti-osteoporotic drugs.” In the article itself, we find this more detailed conclusion: “We conclude that Chinese herbs substantially increased BMD of the lumbar spine compared to placebo or anti-osteoporotic drugs as indicated in the current clinical reports on osteoporosis treatment. Long term of Chinese herbs over 12 months of treatment duration may increase BMD in the hip more effectively. However, further studies are needed to corroborate the positive effect of increasing the duration of Chinese herbs on outcome as the results in this analysis are based on indirect comparisons. To date there are no studies available that compare Chinese herbs, Chinese herbs plus anti-osteoporotic drugs, and anti-osteoporotic drug versus placebo in a factorial design. Consequently, we are unable to draw any conclusions on the possible superiority of Chinese herbs plus anti-osteoporotic drug versus anti-osteoporotic drug or Chinese herb alone in the context of BMD.“
Most readers will feel that this evidence is quite impressive and amazingly solid; they might therefore advocate routinely using Chinese herbs for the common and difficult to treat problem of osteoporosis. The integration of TCM might avoid lots of human suffering, prolong the life of many elderly patients, and save us all a lot of money. Why then am I not at all convinced?
The first thing to notice is the fact that we do not really know which of the ~7000 different Chinese herbs should be used. The article tells us surprisingly little about this crucial point. And even, if we manage to study this question in more depth, we are bound to get thoroughly confused; there are simply too many herbal mixtures and patent medicines to easily identify the most promising candidates.
The second and more important hurdle to making sense of these data is the fact that most of the primary studies originate from inaccessible Chinese journals and were published in Chinese languages which, of course, few people in the West can understand. This is entirely our fault, some might argue, but it does mean that we have to believe the authors, take their words at face value, and cannot check the original data. You may think this is fine, after all, the paper has gone through a rigorous peer-review process where it has been thoroughly checked by several top experts in the field. This, however, is a fallacy; like you and me, the peer-reviewers might not read Chinese either! (I don’t, and I reviewed quite a few of these papers; in some instances, I even asked for translations of the originals to do the job properly but this request was understandably turned down) In all likelihood, the above paper and most similar articles have not been properly peer-reviewed at all.
The third and perhaps most crucial point can only be fully appreciated, if we were able to access and understand the primary studies; it relates to the quality of the original RCTs summarised in such systematic reviews. The abstract of the present paper tells us nothing at all about this issue. In the paper, however, we do find a formal assessment of the studies’ risk of bias which shows that the quality of the included RCTs was poor to very poor. We also find a short but revealing sentence: “The reports of all trials mentioned randomization, but only seven described the method of randomization.” This remark is much more significant than it may seem: we have shown that such studies use such terminology in a rather adventurous way; reviewing about 2000 of these allegedly randomised trials, we found that many Chinese authors call a trial “randomised” even in the absence of a control group (one cannot randomise patients and have no control group)! They seem to like the term because it is fashionable and makes publication of their work easier. We thus have good reason to fear that some/many/most of the studies were not RCTs at all.
The fourth issue that needs mentioning is the fact that very close to 100% of all Chinese TCM-trials report positive findings. This means that either TCM is effective for every indication it is tested for (most unlikely, not least because there are many negative non-Chinese trials of TCM), or there is something very fundamentally wrong with Chinese research into TCM. Over the years, I have had several Chinese co-workers in my team and was invariably impressed by their ability to work hard and efficiently; we often discussed the possible reasons for the extraordinary phenomenon of 0% negative Chinese trials. The most plausible answer they offered was this: it would be most impolite for a Chinese researcher to produce findings which contradict the opinion of his/her peers.
In view of these concerns, can we trust the conclusions of such systematic reviews? I don’t think so – and this is why I have problems with research of this nature. If there are good reasons to doubt their conclusions, these reviews might misinform us systematically, they might not further but hinder progress, and they might send us up the garden path. This could well be in the commercial interest of the Chinese multi-billion dollar TCM-industry, but it would certainly not be in the interest of patients and good health care.
My aim with this blog is to eventually cover most of the 400 or so different alternative therapies and diagnostic techniques. So far, I have focused on some of the most popular modalities; and this means, I have neglected many others. Today, it is time, I think, to discuss aromatherapy, after all, it is one of the most popular forms of alternative medicine in the UK.
Aromatherapists use essential oils, and this is where the confusion starts. They are called “essential” not because humans cannot do without them, like essential nutrients, for instance; they are called “essential” because they are made of flower ESSENCES. The man who ‘discovered’ aromatherapy was a chemist who accidentally had burnt his hand and put some lavender essence on the burn. It healed very quickly, and he thus concluded that essential oils can be useful therapeutics.
Today’s aromatherapists would rarely use the pure essential oil; they dilute it in an inert carrier oil and usually apply it via a very gentle massage to the skin. They believe that specific oils have specific effects for specific conditions. As these oils contain pharmacologically active ingredients, some of these assumptions might even be correct. The question, however, is one of concentration. Do these ingredients reach the target organ in sufficient quantities? Are they absorbed through the skin at all? Does smelling them have a sufficiently large effect to produce the claimed benefit?
The ‘acid test’ for any therapeutic claim is, as always, the clinical trial. As it happens a new paper has just become available. The aim of this randomised study was to determine the effects of inhalation aromatherapy on pregnant women. Essential oils with high linalool and linalyl acetate content were selected and among these the one preferred by the participant was used. Thirteen pregnant women in week 28 of a single pregnancy were randomly assigned into an aromatherapy and a control group. The main outcome measures were several validated scores to assess mood and the heart-rate variability. The results showed significant differences in the Tension-Anxiety score and the Anger-Hostility score after aromatherapy. Heart rate variability changes indicated that the parasympathetic nerve activity increased significantly in the verum group. The authors concluded that aromatherapy inhalation was effective and suggest that more research is warranted.
I have several reasons for mentioning this study here.
1st research into aromatherapy is rare and therefore any new trial of this popular treatment might be important.
2nd aromatherapy is mostly (but not in this study) used in conjunction with a gentle, soothing massage; any outcome of such an intervention is difficult to interpret: we cannot then know whether it was the massage or the oil that produced the observed effect. The present trial is different and might allow conclusions specifically about the effects of the essential oils.
3rd the study displays several classic methodological mistakes which are common in trials of alternative medicine. By exposing them, I hope that they might become less frequent in future.
The most obvious flaw is its tiny sample size. What is an adequate size, people often ask. This question is unfortunately unanswerable. To determine the adequate sample size, it is best to conduct a pilot study or use published data to calculate the required number of patients needed for the specific trial you are planning. Any statistician will be able to help you with this.
The second equally obvious flaw relates to the fact that the results and the conclusions of this study were based on comparing the outcome measures before with those after the interventions within one intervention group. The main reason for taking the trouble of running a control group in a clinical trial is that the findings from the experimental group are compared to those of the control group. Only such inter-group comparisons can tell us whether the results were actually caused by the intervention and not by other factors such as the passage of time, a placebo-effect etc.
In the present study, the authors seem to be aware of their mistake and mention that there were no significant differences in outcomes when the two groups were compared. Yet they fail to draw the right conclusion from this fact. It means that their study demonstrated that aromatherapy inhalation had no effect on the outcomes studied.
So, what does the reliable trial evidence on aromatherapy tell us?
A clinical trial in which I was involved failed to show that it improves the mood or quality of life of cancer patients. But one swallow does not make a summer; what do systematic reviews of all available trials indicate?
The first systematic review was probably the one we published in 2000. We then located 12 randomised clinical trials: six of them had no independent replication; six related to the relaxing effects of aromatherapy combined with massage. These 6 studies collectively suggested that aromatherapy massage has a mild but short-lasting anxiolytic effect. These effects of aromatherapy are probably not strong enough for it to be considered for the treatment of anxiety. We concluded that the hypothesis that it is effective for any other indication is not supported by the findings of rigorous clinical trials.
Since then several other systematic reviews have emerged. We therefore decided to summarise their findings in an overview of all available reviews. We searched 12 electronic databases and our departmental files without restrictions of time or language. The methodological quality of all systematic reviews was evaluated independently by two authors. Of 201 potentially relevant publications, 10 met our inclusion criteria. Most of the systematic reviews were of poor methodological quality. The clinical subject areas were hypertension, depression, anxiety, pain relief, and dementia. For none of the conditions was the evidence convincing. Our conclusions therefore had to be cautious: due to a number of caveats, the evidence is not sufficiently convincing that aromatherapy is an effective therapy for any condition.
Finally, we also investigated the safety of aromatherapy by assessing all published data regarding adverse effects. Forty two primary reports met our inclusion criteria. In total, 71 patients had experienced adverse effects after aromatherapy which ranged from mild to severe and included one fatality. The most common adverse effect was dermatitis. Lavender, peppermint, tea tree oil and ylang-ylang were the most common essential oils responsible for adverse effects. We concluded that aromatherapy has the potential to cause adverse effects some of which are serious. Their frequency remains unknown.
And what is the conclusion of all this? To me, it seems fairly straight forward: Aromatherapy is not demonstrably effective for any condition. It also is not entirely free of risks. Its risk/benefit profile is thus not positive which can only mean that it is not a useful or recommendable treatment for anybody who is ill.
There probably is no area in health care that produces more surveys than alternative medicine. I estimate that about 500 surveys are published every year; this amounts to about two every working day which is substantially more than the number of clinical trials in this field.
I have long been critical of this ‘survey-mania’. The reason is simple: most of these articles are of such poor quality that they tell us nothing of value.
The vast majority of these surveys attempts to evaluate the prevalence of use of alternative medicine, and it is this type of investigation that I intend to discuss here.
For a typical prevalence survey, a team of enthusiastic researchers might put together a few questions and design a questionnaire to find out what percentage of a group of individuals have tried alternative medicine in the past. Subsequently, the investigators might get one or two hundred responses. They then calculate simple descriptive statistics and demonstrate that xy% (let’s assume it is 45%) use alternative medicine. This finding eventually gets published in one of the many alternative medicine journals, and everyone is happy – well, almost everybody.
How can I be such a spoil-sport and claim that this result tells us nothing of value? At the very minimum, some might argue, it shows that enthusiasts of alternative medicine are interested in and capable of conducting research. I beg to differ: this is not research, it is pseudo-research which ignores most of the principles of survey-design.
The typical alternative medicine prevalence survey has none of the features that would render it a scientific investigation:
1) It lacks an accepted definition of what is being surveyed. There is no generally accepted definition of alternative medicine, and even if the researchers address specific therapies, they run into huge problems. Take prayer, for instance – some see this as alternative medicine, while others would, of course, argue that it is a religious pursuit. Or take herbal medicine – many consumers confuse it with homeopathy, some might think that drinking tea is herbal medicine, while others would probably disagree.
2) The questionnaires used for such surveys are almost never validated. Essentially, this means that we cannot be sure they evaluate what we think they evaluate. We all know that the way we formulate a question can determine the answer. There are many potential sources of bias here, and they are rarely taken into consideration.
3) Enthusiastic researchers of alternative medicine usually use a small convenience sample of participants for their surveys. This means they ask a few people who happen to be around to fill their questionnaire. As a consequence, there is no way the survey is representative of the population in question.
4) The typical survey has a low response rate; sometimes the response rate is not even provided or remains unknown even to the investigators. This means we do not know how the majority of patients/consumers who received but did not fill the questionnaire would have answered. Often there is good reason to suspect that those who have a certain attitude did respond, while those with a different opinion did not. This self-selection process is likely to produce misleading findings.
And why I am so sure about all of theses limitations? To my embarrassment, I know about them not least because I have made most these mistakes myself at some time in my career. You might also ask why this is important: what’s the harm in publishing a few flimsy surveys?
In my view, these investigations are regrettably counter-productive because:
they tend to grossly over-estimate the popularity of alternative medicine,
they distract money, manpower and attention from the truly important research questions in this field,
they give a false impression of a buoyant research activity,
and their results are constantly misused.
The last point is probably the most important one. The argument that is all too often spun around such survey data goes roughly as follows: a large percentage of the population uses alternative medicine; people pay out of their own pocket for these treatments; they are satisfied with them (if not, they would not pay for them). BUT THIS IS GROSSLY UNFAIR! Why should only those individuals who are rich enough to afford alternative medicine benefit from it? ALTERNATIVE MEDICINE SHOULD BE MADE AVAILABLE FOR ALL.
I rest my case.
Clinical trials of acupuncture can be quite challenging. In particular, it is often difficult to make sure that any observed outcome is truly due to the treatment and not caused by some other factor(s). How tricky this can be, shows a recently published study.
A new RCT has all (well, almost all) the features of a rigorous study. It tested the effects of acupuncture in patients suffering from hay fever. The German investigators recruited 46 specialized physicians in 6 hospital clinics and 32 private outpatient clinics. In total, 422 patients with IgE sensitization to birch and grass pollen were randomized into three groups: 1) acupuncture plus rescue medication (RM) (n= 212), 2) sham acupuncture plus RM (n= 102), or 3) RM alone (n= 108). Twelve acupuncture sessions were provided in groups 1 and 2 over 8 weeks. The outcome measures included changes in the Rhinitis Quality of Life Questionnaire (RQLQ) overall score and the RM score (RMs) from baseline to weeks 7, 8 and 16 in the first year as well as week 8 in the second year after randomization.
Compared with sham acupuncture and with RM, acupuncture was associated with improvement in RQLQ score and RMS. There were no differences after 16 weeks in the first year. After the 8-week follow-up phase in the second year, small improvements favoring real acupuncture over sham were noted.
Based on these results, the authors concluded that “acupuncture led to statistically significant improvements in disease-specific quality of life and antihistamine use measures after 8 weeks of treatment compared with sham acupuncture and with RM alone, but the improvements may not be clinically significant.”
The popular media were full of claims that this study proves the efficacy of acupuncture. However, I am not at all convinced that this conclusion is not hopelessly over-optimistic.
It might not have been the acupuncture itself that led to the observed improvements; they could well have been caused by several factors unrelated to the treatment itself. To understand my concern, we need to look closer at the actual interventions employed by the investigators.
The real acupuncture was done on acupuncture points thought to be indicated for hay fever. The needling was performed as one would normally do it, and the acupuncturists were asked to treat the patients in group 1 in such a way that they were likely to experience the famous ‘de-qi’ feeling.
The sham acupuncture, by contrast, was performed on non-acupuncture points; acupuncturists were asked to use shallow needling only and they were instructed to try not to produce ‘de-qi’.
This means that the following factors in combination or alone could have caused [and in my view probably did cause] the observed differences in outcomes between the acupuncture and the sham group:
1) verbal or non-verbal communication between the acupuncturists and the patient [previous trials have shown this factor to be of crucial importance]
2) the visibly less deep needling in the sham-group
3) the lack of ‘de-qi’ experience in the sham-group.
Sham-treatments in clinical trials serve the purpose of a placebo. They are thus meant to be indistinguishable from the verum. If that is not the case [as in the present study], the trial cannot be accepted as being patient-blind. If a trial is not patient-blind, the expectations of patients will most certainly influence the results.
Therefore I believe that the marginal differences noted in this study were not due to the effects of acupuncture per se, but were an artifact caused through de-blinding of the patients. De facto, neither the patients nor the acupuncturists were blinded in this study.
If that is true, the effects were not just not clinically relevant, as noted by the authors, they also had nothing to do with acupuncture. In other words, acupuncture is not of proven efficacy for this condition – a verdict which is also supported by our systematic review of the subject which concluded that “the evidence for the effectiveness of acupuncture for the symptomatic treatment or prevention of allergic rhinitis is mixed. The results for seasonal allergic rhinitis failed to show specific effects of acupuncture…”
Once again, we have before us a study which looks impressive at first glance. At closer scrutiny, we find, however, that it had important design flaws which led to false positive results and conclusions. In my view, it would have been the responsibility of the authors to discuss these limitations in full detail and to draw conclusions that take them into account. Moreover, it would have been the duty of the peer-reviewers and journal editors to pick up on these points. Instead the editors even commissioned an accompanying editorial which displays an exemplary lack of critical thinking.
Having failed to do any of this, they are in my opinion all guilty of misleading the world media who reported extensively and often uncritically on this new study thus misleading us all. Sadly, the losers in this bonanza of incompetence are the many hay fever sufferers who will now be trying (and paying for) useless treatments.
The UK General Chiropractic Council has commissioned a survey of chiropractic patients’ views of chiropractic. Initially, 600 chiropractors were approached to recruit patients, but only 47 volunteered to participate. Eventually, 70 chiropractors consented and recruited a total of 544 patients who completed the questionnaire in 2012. The final report of this exercise has just become available.
I have to admit, I found it intensely boring. This is mainly because the questions asked avoided contentious issues. One has to dig deep to find nuggets of interest. Here are some of the findings that I thought were perhaps mildly intriguing:
15% of all patients did not receive information about possible adverse effects (AEs) of their treatment.
20% received no explanations why investigations such as X-rays were necessary and what risks they carried.
17% were not told how much their treatment would cost during the initial consultation.
38% were not informed about complaint procedures.
9% were not told about further treatment options for their condition.
18% said they were not referred to another health care professional when the condition failed to improve.
20% noted that the chiropractor did not liaise with the patient’s GP.
I think, one has to take such surveys with more than just a pinch of salt. At best, they give a vague impression of what patients believe. At worst, they are not worth the paper they are printed on.
Perhaps the most remarkable finding from the report is the unwillingness of chiropractors to co-operate with the GCC which, after all, is their regulating body. To recruit only ~10% of all UK chiropractors is more than disappointing. This low response rate will inevitably impact on the validity of the results and the conclusions.
It can be assumed that those practitioners who did volunteer are a self-selected sample and thus not representative of the UK chiropractic profession; they might be especially good, correct or obedient. This, in turn, also applies to the sample of patients recruited for this research. If that is so, the picture that emerged from the survey is likely to be be far too positive.
In any case, with a response rate of only ~10%, any survey is next to useless. I would therefore put it in the category of ‘not worth the paper it is printed on’.
If I had a pint of beer for every time I have been accused of bias against chiropractic, I would rarely be sober. The thing is that I do like to report about decent research in this field and I am almost every day looking out for new articles which might be worth writing about – but they are like gold dust!
“Huuuuuuuuh, that just shows how very biased he is” I hear the chiro community shout. Well let’s put my hypothesis to the test. Here is a complete list of recent (2013)Medline-listed articles on chiropractic; no omission, no bias, just facts (for clarity, the Pubmed-link is listed first, then the title in bold followed by a short comment in italics):
http://www.ncbi.nlm.nih.gov/pubmed/23360894
Towards establishing an occupational threshold for cumulative shear force in the vertebral joint – An in vitro evaluation of a risk factor for spondylolytic fractures using porcine specimens.
This is an interesting study of the shear forces observed in porcine vertebral specimen during maneuvers which might resemble spinal manipulation in humans. The authors conclude that “Our investigation suggested that pars interarticularis damage may begin non-linearly accumulating with shear forces between 20% and 40% of failure tolerance (approximately 430 to 860N”
http://www.ncbi.nlm.nih.gov/pubmed/23337706
Development of an equation for calculating vertebral shear failure tolerance without destructive mechanical testing using iterative linear regression.
This is a mathematical modelling of the forces that might act on the spine during manipulation. The authors draw no conclusions.
http://www.ncbi.nlm.nih.gov/pubmed/23324133
Collaborative Care for Older Adults with low back pain by family medicine physicians and doctors of chiropractic (COCOA): study protocol for a randomized controlled trial.
This is merely the publication of a trial that is about to commence.
http://www.ncbi.nlm.nih.gov/pubmed/23323682
Military Report More Complementary and Alternative Medicine Use than Civilians.
This is a survey which suggests that ~45% of all military personnel use some form of alternative medicine.
http://www.ncbi.nlm.nih.gov/pubmed/23319526
Complementary and Alternative Medicine Use by Pediatric Specialty Outpatients
This is another survey; it concludes that ” that CAM use is high among pediatric specialty clinic outpatients”
http://www.ncbi.nlm.nih.gov/pubmed/23311664
Extending ICPC-2 PLUS terminology to develop a classification system specific for the study of chiropractic encounters
This is an article on chiropractic terminology which concludes that “existing ICPC-2 PLUS terminology could not fully represent chiropractic practice, adding terms specific to chiropractic enabled coding of a large number of chiropractic encounters at the desired level. Further, the new system attempted to record the diversity among chiropractic encounters while enabling generalisation for reporting where required. COAST is ongoing, and as such, any further encounters received from chiropractors will enable addition and refinement of ICPC-2 PLUS (Chiro)”.
http://www.ncbi.nlm.nih.gov/pubmed/23297270
US Spending On Complementary And Alternative Medicine During 2002-08 Plateaued, Suggesting Role In Reformed Health System
This is a study of the money spent on alternative medicine concluding as follows ”Should some forms of complementary and alternative medicine-for example, chiropractic care for back pain-be proven more efficient than allopathic and specialty medicine, the inclusion of complementary and alternative medicine providers in new delivery systems such as accountable care organizations could help slow growth in national health care spending”
http://www.ncbi.nlm.nih.gov/pubmed/23289610
A Royal Chartered College joins Chiropractic & Manual Therapies.
This is a short comment on the fact that a chiro institution received a Royal Charter.
http://www.ncbi.nlm.nih.gov/pubmed/23242960
Exposure-adjusted incidence rates and severity of competition injuries in Australian amateur taekwondo athletes: a 2-year prospective study.
This is a study by chiros to determine the frequency of injuries in taekwondo athletes.
The first thing that strikes me is the paucity of articles; ok, we are talking of just january 2013 but by comparison most medical fields like neurology, rheumatology have produced hundreds of articles during this period and even the field of acupuncture research has generated about three times more.
The second and much more important point is that I fail to see much chiropractic research that is truly meaningful or tells us anything about what I consider the most urgent questions in this area, e.g. do chiropractic interventions work? are they safe?
My last point is equally critical. After reading the 9 papers, I have to honestly say that none of them impressed me in terms of its scientific rigor.
So, what does this tiny investigation suggest? Not a lot, I have to admit, but I think it supports the hypothesis that research into chiropractic is not very active, nor high quality, nor does it address the most urgent questions.
On January 27, 1945, the concentration camp in Auschwitz was liberated. By May of the same year, around 20 similar camps had been discovered. What they revealed is so shocking that it is difficult to put it in words.
Today, on ‘HOCOCAUST MEMORIAL DAY’, I quote (shortened and slightly modified) from articles I published many years ago (references can be found in the originals) to remind us of the unspeakable atrocities that occurred during the Nazi period and of the crucial role the German medical profession played in them.
The Nazi’s euthanasia programme, also known as ”Action T4″, started in specialized medicinal departments in 1939. Initially, it was aimed at children suffering from “idiocy, Down’s syndrome, hydrocephalus and other abnormalities”. By the end of 1939, the programme was extended to adults “unworthy of living.” We estimate that, when it was stopped, more than 70,000 patients had been killed.
Action T4 (named after its address: Tiergarten Strasse 4) was the Berlin headquarters of the euthanasia programme. It was run by approximately 50 physicians who, amongst other activities, sent questionnaires to (mostly psychiatric) hospitals urging them to return lists of patients for euthanasia. The victims were transported to specialized centers where they were gassed or poisoned. Action T4 was thus responsible for medically supervised, large-scale murder. Its true significance, however, lies elsewhere. Action T4 turned out to be nothing less than a “pilot project” for the extinction of millions of prisoners of the concentration camps.
The T4 units had developed the technology for killing on an industrial scale. It was only with this know-how that the total extinction of all Jews of the Reich could be planned. This truly monstrous task required medical expertise.
Almost without exception, those physicians who had worked for T4 went on to take charge of what the Nazis called the ‘Final Solution’. While action T4 had killed thousands, its offspring would murder millions under the trained instructions of Nazi doctors.
The medical profession’s role in these crimes was critical and essential. German physicians had been involved at all levels and stages. They had created and embraced the pseudo-science of race hygiene. They were instrumental in developing it further into applied racism. They had generated the know-how of mass extinction. Finally, they also performed outrageously cruel and criminal experiments under the guise of scientific inquiry [see below]. German doctors had thus betrayed all the ideals medicine had previously stood for, and had become involved in criminal activities unprecedented in the history of medicine (full details and references on all of this are provided in my article, see link above).
Alternative medicine
It is well-documented that alternative medicine was strongly supported by the Nazis. The general belief is that this had nothing to do with the sickening atrocities of this period. I believe that this assumption is not entirely correct. In 2001, I published an article which reviews the this subject; I take the liberty of borrowing from it here.
Based on a general movement in favour of all things natural, a powerful trend towards natural ways of healing had developed in the 19(th)century. By 1930, this had led to a situation in Germany where roughly as many lay-practitioners of alternative medicine as conventional doctors were in practice.This had led to considerable tensions between the two camps. To re-unify German medicine under the banner of ‘Neue Deutsche Heilkunde’ (New German Medicine), Nazi officials eventually decided to create the profession of the ’Heilpraktiker‘ (healing practitioner). Heilpraktiker were not allowed to train students and their profession was thus meant to become extinct within one generation; Goebbels spoke of having created the cradle and the grave of the Heilpraktiker. However, after 1945, this decision was challenged in the courts and eventually over-turned – and this is why Heilpraktiker are still thriving today.
The ‘flag ship’ of the ‘Neue Deutsche Heilkunde’ was the ‘Rudolf Hess Krankenhaus‘ in Dresden (which was re-named into Gerhard Wagner Krankenhaus after Hess’ flight to the UK). It represented a full integration of alternative and orthodox medicine.
‘Research’
An example of systematic research into alternative medicine is the Nazi government’s project to validate homoeopathy. The data of this massive research programme are now lost (some speculate that homeopaths made them disappear) but, according to an eye-witness report, its results were entirely negative (full details and references on alt med in 3rd Reich are in the article cited above).
There is,of course, plenty of literature on the subject of Nazi ‘research’ (actually, it was pseudo-research) and the unspeakable crimes it entailed. By contrast, there is almost no published evidence that these activities included in any way alternative medicine, and the general opinion seems to be that there are no connections whatsoever. I fear that this notion might be erroneous.
As far as I can make out, no systematic study of the subject has so far been published, but I found several hints and indications that the criminal experiments of Nazi doctors also involved alternative medicine (the sources are provided in my articles cited above or in the links provided below). Here are but a few leads:
Dr Wagner, the chief medical officer of the Nazis was a dedicated and most active proponent of alternative medicine.
Doctors in the alternative “Rudolf Hess Krankenhaus” [see above] experimented on speeding up the recovery of wounded soldiers, on curing syphilis with fasting, and on various other projects to help the war effort.
The Dachau concentration camp housed the largest plantation of medicinal herbs in Germany.
Dr Madaus (founder of the still existing company for natural medicines by the same name) experimented on the sterilisation of humans with herbal and homeopathic remedies, a project that was deemed of great importance for controlling the predicted population growth in the East of the expanding Reich.
Dr Grawitz infected Dachau prisoners with various pathogens to test the effectiveness of homeopathic remedies.
Schuessler salts were also tested on concentration camp inmates.
So, why bring all of this up today? Is it not time that we let grass grow over these most disturbing events? I think not! For many years, I actively researched this area (you can find many of my articles on Medline) because I am convinced that the unprecedented horrors of Nazi medicine need to be told and re-told – not just on HOLOCAUST MEMORIAL DAY, but continually. This, I hope, will minimize the risk of such incredible abuses ever happening again.
As I am drafting this post, I am in a plane flying back from Finland. The in-flight meal reminded me of the fact that no food is so delicious that it cannot be spoilt by the addition of too many capers. In turn, this made me think about the paper I happened to be reading at the time, and I arrived at the following theory: no trial design is so rigorous that it cannot to be turned into something utterly nonsensical by the addition of a few amateur researchers.
The paper I was reading when this idea occurred to me was a randomised, triple-blind, placebo-controlled cross-over trial of homeopathy. Sounds rigorous and top quality? Yes, but wait!
Essentially, the authors recruited 86 volunteers who all claimed to be suffering from “mental fatigue” and treated them with Kali-Phos 6X or placebo for one week (X-potencies signify dilution steps of 1: 10, and 6X therefore means that the salt had been diluted 1: 1000000 ). Subsequently, the volunteers were crossed-over to receive the other treatment for one week.
The results failed to show that the homeopathic medication had any effect (not even homeopaths can be surprised about this!). The authors concluded that Kali-Phos was not effective but cautioned that, because of the possibility of a type-2-error, they might have missed an effect which, in truth, does exist.
In my view, this article provides an almost classic example of how time, money and other resources can be wasted in a pretence of conducting reasonable research. As we all know, clinical trials usually are for testing hypotheses. But what is the hypothesis tested here?
According to the authors, the aim was to “assess the effectiveness of Kali-Phos 6X for attention problems associated with mental fatigue”. In other words, their hyposesis was that this remedy is effective for treating the symptom of mental fatigue. This notion, I would claim, is not a scientific hypothesis, it is a foolish conjecture!
Arguably any hypothesis about the effectiveness of a highly diluted homeopathic remedy is mere wishful thinking. But, if there were at least some promissing data, some might conclude that a trial was justified. By way of justification for the RCT in question, the authors inform us that one previous trial had suggested an effect; however, this study did not employ just Kali-Phos but a combined homeopathic preparation which contained Kalium-Phos as one of several components. Thus the authors’ “hypothesis” does not even amount to a hunch, not even to a slight incling! To me, it is less than a shot in the dark fired by blind optimists - nobody should be surprised that the bullet failed to hit anything.
It could even be that the investigators themselves dimly realised that something is amiss with the basis of their study; this might be the reason why they called it an “exploratory trial”. But an exploratory study is one whithout a hypothesis, and the trial in question does have a hyposis of sorts – only that it is rubbish. And what exactly did the authos meant to explore anyway?
That self-reported mental fatigue in healthy volunteers is a condition that can be mediatised such that it merits treatment?
That the test they used for quantifying its severity is adequate?
That a homeopathic remedy with virtually no active ingredient generates outcomes which are different from placebo?
That Hahnemann’s teaching of homeopathy was nonsense and can thus be discarded (he would have sharply condemned the approach of treating all volunteers with the same remedy, as it contradicts many of his concepts)?
That funding bodies can be fooled to pay for even the most ridiculous trial?
That ethics-committees might pass applications which are pure nonsense and which are thus unethical?
A scientific hypothesis should be more than a vague hunch; at its simplest, it aims to explain an observation or phenomenon, and it ought to have certain features which many alt med researchers seem to have never heard of. If they test nonsense, the result can only be nonsense.
The issue of conducting research that does not make much sense is far from trivial, particularly as so much (I would say most) of alt med research is of such or even worst calibre (if you do not believe me, please go on Medline and see for yourself how many of the recent articles in the category “complementary alternative medicine” truly contribute to knowledge worth knowing). It would be easy therefore to cite more hypothesis-free trials of homeopathy.
One recent example from Germany will have to suffice: in this trial, the only justification for conducting a full-blown RCT was that the manufacturer of the remedy allegedly knew of a few unpublished case-reports which suggested the treatment to work – and, of course, the results of the RCT eventually showed that it didn’t. Anyone with a background in science might have predicied that outcome – which is why such trials are so deplorably wastefull.
Research-funds are increasingly scarce, and they must not be spent on nonsensical projects! The money and time should be invested more fruitfully elsewhere. Participants of clinical trials give their cooperation willingly; but if they learn that their efforts have been wasted unnecessarily, they might think twice next time they are asked. Thus nonsensical research may have knock-on effects with far-reaching consequences.
Being a researcher is at least as serious a profession as most other occupations; perhaps we should stop allowing total amateurs wasting money while playing at being professioal. If someone driving a car does something seriously wrong, we take away his licence; why is there not a similar mechanism for inadequate researchers, funders, ethics-committees which prevents them doing further damage?
At the very minimum, we should critically evaluate the hypothesis that the applicants for research-funds propose to test. Had someone done this properly in relatiom to the two above-named studies, we would have saved about £150,000 per trial (my estimate). But as it stands, the authors will probably claim that they have produced fascinating findings which urgently need further investigation – and we (normally you and I) will have to spend three times the above-named amount (again, my estimate) to finance a “definitive” trial. Nonsense, I am afraid, tends to beget more nonsense.
In my last post, we discussed the “A+B versus B” trial design as a tool to produce false positive results. This method is currently very popular in alternative medicine, yet it is by no means the only approach that can mislead us. Today, let’s look at other popular options with a view of protecting us against trialists who naively or willfully might fool us.
The crucial flaw of the “A+B versus B” design is that it fails to account for non-specific effects. If the patients in the experimental group experience better outcomes than the control group, this difference could well be due to effects that are unrelated to the experimental treatment. There are, of course, several further ways to ignore non-specific effects in clinical research. The simplest option is to include no control group at all. Homeopaths, for instance, are very proud of studies which show that ~70% of their patients experience benefit after taking their remedies. This type of result tends to impress journalists, politicians and other people who fail to realise that such a result might be due to a host of factors, e.g. the placebo-effect, the natural history of the disease, regression towards the mean or treatments which patients self-administered while taking the homeopathic remedies. It is therefore misleading to make causal inferences from such data.
Another easy method to generate false positive results is to omit blinding. The purpose of blinding the patient, the therapist and the evaluator of the outcomes in clinical trials is to make sure that expectation is not the cause of or contributor to the outcome. They say that expectation can move mountains; this might be an exaggeration, but it can certainly influence the result of a clinical trial. Patients who hope for a cure regularly do get better even if the therapy they receive is useless, and therapists as well as evaluators of the outcomes tend to view the results through rose-tinted spectacles, if they have preconceived ideas about the experimental treatment. Similarly, the parents of a child or the owners of an animal can transfer their expectations, and this is one of several reasons why it is incorrect to claim that children and animals are immune to placebo-effects.
Failure to randomise is another source of bias which can make an ineffective therapy look like an effective one when tested in a clinical trial. If we allow patients or trialists to select or choose which patients receive the experimental and which get the control-treatment, it is likely that the two groups differ in a number of variables. Some of these variables might, in turn, impact on the outcome. If, for instance, doctors allocate their patients to the experimental and control groups, they might select those who will respond to the former and those who don’t to the latter. This may not happen with malicious intent but through intuition or instinct: responsible health care professionals want those patients who, in their experience, have the best chances to benefit from a given treatment to receive that treatment. Only randomisation can, when done properly, make sure we are comparing comparable groups of patients, and non-randomisation is likely to produce misleading findings.
While these options for producing false positives are all too obvious, the next possibility is slightly more intriguing. It refers to studies which do not test whether an experimental treatment is superior to another one (often called superiority trials), but to investigations attempting to assess whether it is equivalent to a therapy that is generally accepted to be effective. The idea is that, if both treatments produce the same or similarly positive results, both must be effective. For instance, such a study might compare the effects of acupuncture to a common pain-killer. Such trials are aptly called non-superiority or equivalence trials, and they offer a wide range of possibilities for misleading us. If, for example, such a trial has not enough patients, it might show no difference where, in fact, there is one. Let’s consider a deliberately silly example: someone comes up with the idea to compare antibiotics to acupuncture as treatments of bacterial pneumonia in elderly patients. The researchers recruit 10 patients for each group, and the results reveal that, in one group, 2 patients died, while, in the other, the number was 3. The statistical tests show that the difference of just one patient is not statistically significant, and the authors therefore conclude that acupuncture is just as good for bacterial infections as antibiotics.
Even trickier is the option to under-dose the treatment given to the control group in an equivalence trial. In our hypothetical example, the investigators might subsequently recruit hundreds of patients in an attempt to overcome the criticism of their first study; they then decide to administer a sub-therapeutic dose of the antibiotic in the control group. The results would then apparently confirm the researchers’ initial finding, namely that acupuncture is as good as the antibiotic for pneumonia. Acupuncturists might then claim that their treatment has been proven in a very large randomised clinical trial to be effective for treating this condition, and people who do not happen to know the correct dose of the antibiotic could easily be fooled into believing them.
Obviously, the results would be more impressive, if the control group in an equivalence trial received a therapy which is not just ineffective but actually harmful. In such a scenario, the most useless or even slightly detrimental treatment would appear to be effective simply because it is equivalent to or less harmful than the comparator.
A variation of this theme is the plethora of controlled clinical trials which compare one unproven therapy to another unproven treatment. Perdicatbly, the results indicate that there is no difference in the clinical outcome experienced by the patients in the two groups. Enthusiastic researchers then tend to conclude that this proves both treatments to be equally effective.
Another option for creating misleadingly positive findings is to cherry-pick the results. Most trails have many outcome measures; for instance, a study of acupuncture for pain-control might quantify pain in half a dozen different ways, it might also measure the length of the treatment until pain has subsided, the amount of medication the patients took in addition to receiving acupuncture, the days off work because of pain, the partner’s impression of the patient’s health status, the quality of life of the patient, the frequency of sleep being disrupted by pain etc. If the researchers then evaluate all the results, they are likely to find that one or two of them have changed in the direction they wanted. This can well be a chance finding: with the typical statistical tests, one in 20 outcome measures would produce a significant result purely by chance. In order to mislead us, the researchers only need to “forget” about all the negative results and focus their publication on the ones which by chance have come out as they had hoped.
One fail-proof method for misleading the public is to draw conclusions which are not supported by the data. Imagine you have generated squarely negative data with a trial of homeopathy. As an enthusiast of homeopathy, you are far from happy with your own findings; in addition you might have a sponsor who puts pressure on you. What can you do? The solution is simple: you only need to highlight at least one positive message in the published article. In the case of homeopathy, you could, for instance, make a major issue about the fact that the treatment was remarkably safe and cheap: not a single patient died, most were very pleased with the treatment which was not even very expensive.
And finally, there is always the possibility of overt cheating. Researchers are only human and are thus not immune to temptation. They may have conflicts of interest or may know that positive results are much easier to publish than negative ones. Certainly they want to publish their work – “publish or perish”! So, faced with disappointing results of a study, they might decide to prettify them or even invent new ones which are more pleasing to them, their peers, or their sponsors.
Am I claiming that this sort of thing only happens in alternative medicine? No! Obviously, the way to minimise the risk of such misconduct is to train researchers properly and make sure they are able to think critically. Am I suggesting that investigators of alternative medicine are often not well-trained and almost always uncritical? Yes.