As reported, the Bavarian government has set aside almost half a million Euros for research to determine whether the over-use of antibiotics can be reduced by replacing them with homeopathic remedies. Homeopaths in and beyond Germany were delighted, of course, but many experts were bewildered (see also this or this, if you read German).
While the Bavarians are entering the planning stage of this research, I want to elaborate on the question what methodology might be employed for this task. As far as I can see, there are, depending on the precise research questions, various options.
IN VITRO TESTS OF HOMEOPATHICS
The most straight forward way to find out whether homeopathics are an alternative to antibiotics would be to screen them for antibiotic activity. For this, we would take all homeopathic remedies in several potencies that are commonly used, for instance D12 and C30, and add them to bacterial cultures. To cover even part of the range of homeopathic remedies, several thousand such tests would be required. The remedies that show activity in vitro would then be candidates for further clinical tests.
I doubt that this will generate meaningful findings. As homeopaths would probably point out quickly, they never claimed that their remedies have any antibiotic effects. Homeopathics work not via pharmacological mechanisms (there is none), they stimulate the vital force, the immune system, or whatever mystical force you fancy. Faced with the inevitably negative results of in vitro tests, homeopaths would merely shrug their shoulders and say: ‘we told you so’.
Thus it might be more constructive to go directly into animal models. Such tests could take several shapes and forms. For instance, scientists could infect animals with a bacterium and subsequently treat one group with a high potency homeopathic remedy and the control group with a placebo. If the homeopathic animals survive, while the controls die, the homeopathic treatment was effective.
Such concepts would run into problems on at least two levels. Firstly, any ethics committee worth its name would refuse to pass such a protocol and argue that it is not ethical to infect and then treat animals with two different types of placebo. Secondly, the homeopathic fraternity would explain that homeopathy must be individualised which cannot be done properly in animals. Faced with the inevitably negative results of such animal studies, homeopaths would merely shrug their shoulders and say: ‘we told you so’.
Homeopathy may, according to some homeopaths, defy in vitro and animal tests, but it is most certainly amenable to being tested in clinical trials. The simplest version of a clinical study would entail randomising a group of patients with bacterial infections – say pneumonia – into receiving either individualised homeopathy or placebo. Possibly, one could add a third group of patients being treated with appropriate antibiotics.
The problem here would again be the ethics; no proper ethic committee would pass such a concept (see above). Another problem might be that even the homeopathic fraternity would oppose such a study. Why? Because all but the most deluded homeopaths know only too well that the result of such a trial would be devastatingly negative for homeopathy.
Therefore, homeopaths are likely to go for a different study design, for instance, one where patients suspected to have a bacterial infection are randomised to two groups of GPs. One group of ‘normal’ GPs would proceed as usual, while the other group are also trained in homeopathy and would be free to give whatever they feel is right for each individual patient. With a bit of luck, the ‘normal’ GPs would over-prescribe antibiotics (because that’s what they are apparently doing routinely), while the homeopathic GPs would often use homeopathics instead.
Such a study would indeed generate a result alleging that the use of homeopathy reduces the use of antibiotics. Of course, to be truly ‘positive’ it would need to exclude any clinical outcome such as time to recovery, because that might not be in favour of homeopathy.
The problem might again be the ethics committee. Assuming they are scientifically switched on, they will see through the futility of a trial designed to produce the desired result. They might also argue that science is not for testing one faulty approach (over-prescribing) against another (homeopathy) and insist that science is about finding the best treatment (which is neither of the two).
There are, of course, many other study designs that could be considered. Generally, they fall into two different categories: if they are rigorous tests of a hypothesis, they are sure to produce a result unfavourable to homeopathy. Such studies will therefore be opposed to by the powerful homeopathic fraternity. If, however, studies are flimsily designed to generate a positive finding, they might be liked by homeopaths, yet rejected by scientists and ethicists.
A much easier solution to the question ‘does the use of homeopathy reduce the use of antibiotics’ might be to not do a trial at all, but to run a simple survey. For instance, one could retrospectively assess how many antibiotics 100 homeopathic GPs have prescribed during the last year and compare this to the figure of 100 over-prescribing, ‘normal’ GPs. This type of ‘research’ is a sure winner for the homeopaths. Therefore, I predict that they will advocate this or a similarly flawed concept.
Most politicians are scientifically illiterate to such a degree that they might actually agree to finance such a survey and then confuse correlation with causation by triumphantly stating that the use of homeopathy reduces over-prescribing of antibiotics. Few, I fear, will realise that there is only one method for reducing the over-prescribing of antibiotics: remind doctors what they all learnt in medical school, namely to prescribe antibiotics only in cases where they are indicated. And for that we evidently need no homeopathy or other SCAM.
An intercessory prayer (IP) is an intervention characterized by one or more individuals praying for the well-being or a positive outcome of another person. There have been several trials of IP, but the evidence is far from clear-cut. Perhaps this new study will bring clarity?
The goal of this double-blind RCT was to assess the effects of intercessory prayer on psychological, spiritual and biological scores of breast 31 cancer patients who were undergoing radiotherapy (RT). The experimental group was prayed for, while the controla group received no such treatment. The intercessory prayer was performed by a group of six Christians, who prayed daily during 1 h while participant where under RT. The prayers asked for calm, peace, harmony and recovery of health and spiritual well-being of all participants. Data collection was performed in three time points (T0, T1 and T2).
Significant changes were noted in the intra-group analysis, concerning the decrease in spiritual distress score; negative religious/spiritual coping prevailed, while the total religious/spiritual coping increased between the posttest T2 to T0.
The authors concluded that begging a higher being for health recovery is a common practice among people, regardless of their spirituality and religiosity. In this study, this practice was performed through intercessory prayer, which promoted positive health effects, since spiritual distress and negative spiritual coping have reduced. Also, spiritual coping has increased, which means that participants facing difficult situations developed strategies to better cope and solve the problems. Given the results related to the use of intercession prayer, as a complementary therapeutic intervention, holistic nursing care should integrate this intervention, which is included in the Nursing Interventions Classification. Additionally, further evidence and research is needed about the effect of this nursing spiritual intervention in other cultures, in different clinical settings and with larger samples.
The write-up of this study is very poor and most confusing – so much so that I find it hard to make sense of the data provided. If I understand it correctly, the positive findings relate to changes within the experimental group. As RCTs are about compating one group to another, these changes are irrelevant. Therefore (and for several other methodological flaws as well), the conclusion that IP generates positive effects is not warranted by these new findings.
Like all other forms of paranormal healing, IP is implausible and lacks support of clinical effectiveness.
I have recently gone to the trouble of evaluating 150 different modalities from the realm of so-called alternative medicine (SCAM) in a book. This is what it tells you about Reiki:
Reiki is a form of paranormal or energy healing popularised by Japanese Mikao Usui (1865-1926). Rei means universal spirit (sometimes thought of as a supreme being) and ki is the assumed universal life energy.
- Reiki is based on the assumptions of Traditional Chinese Medicine and the existence of ‘chi’, the life-force that determines our health.
- Reiki practitioners believe that, with their hands-on healing method, they can transfer ‘healing energy’ to a patient which, in turn, stimulates the self-healing properties of the body. They assume that the therapeutic effects of this technique are obtained from a ‘universal life energy’ that provides strength, harmony, and balance to the body and mind.
- There is no scientific basis for such notions, and reiki is therefore not plausible.
- Reiki is used for a number of conditions, including the relief of stress, tension and pain.
- There have been several clinical trials testing the effectiveness of reiki. Unfortunately, their methodological quality is usually poor.
- A systematic review summarising this evidence concluded that the evidence is insufﬁcient to suggest that reiki is an effective treatment for any condition. Therefore, the value of reiki remains unproven. And a Cochrane review found that there is insufficient evidence to say whether or not Reiki is useful for people over 16 years of age with anxiety or depression or both.
- Reiki appears to be generally safe, and serious adverse effects have not been reported. Some practitioners advise caution about using reiki in people with psychiatric illnesses because of the risk of bringing out underlying psychopathology.
So, Reiki is both implausible and unproven. Now a new, large trial has emerged that might change this verdict. The main purpose of this study (published in JCAM) was to measure the effect of a single session of Reiki on physical and psychological health in a large nonclinical sample.
The study design was a single arm effectiveness trial with measures at pre-and postintervention. The study took place at private Reiki practices across the United States. Reiki practitioners were recruited from an online mailing list to participate in the study with their Reiki clients. A total of 99 Reiki practitioners met the inclusion criteria and participated in the study. Reiki practitioners were instructed to give a flyer to each of their Reiki clients that contained information about the study and invited the client to complete a survey before and after their Reiki session.
Trained and certified Reiki Masters conducted the Reiki sessions in person, with each session lasting between 45 and 90 min. The 20-item Positive and Negative Affect Schedule was used to assess affect, and brief, single-item self-report measures were used to assess a wide range of physical and psychological variables immediately before (pre) and after (post) the Reiki session.
A total of N = 1411 Reiki sessions were conducted and included in the analysis. Statistically significant improvements were observed for all outcome measures, including positive affect, negative affect, pain, drowsiness, tiredness, nausea, appetite, shortness of breath, anxiety, depression, and overall well-being (all p-values <0.001).
The authors concluded that the results from this large-scale multisite effectiveness trial suggest that a single session of Reiki improves multiple variables related to physical and psychological health.
This ‘large scale’ effectiveness trial’ could make you laugh and cry at the same time.
- Laugh, because it is almost comically daft.
- Cry, because the conclusion is bound to mislead a lot of gullible people.
Without a control group, the study cannot even attempt to determine anything like the effectiveness of Reiki. What the results truly show is that consumers who consult (and pay) a Reiki master expect to have a positive effect. The expectation translates into a sizable placebo response. The investigators seem to be clueless scientists, or they wilfully mislead the public (the senior author is from the ‘The Center for Reiki Research‘ which, according to its mission statement, is dedicated to gaining acceptance for the practice of Reiki by the medical community).
The only conclusion that can honestly be drawn from the data is that consumers who pay for a serivce often like this service (otherwise they would not use it!). It’s a bit like the thing with the hamburger joint that I often cite: if you ask people eating in a McDonalds whether they enjoy hamburgers, most will answer in the affirmative.
But there might be a valuable lesson in this paper after all: never trust the JACM further than you can throw it.
The medical literature is currently swamped with reviews of acupuncture (and other forms of TCM) trials originating from China. Here is the latest example (but, trust me, there are hundreds more of the same ilk).
The aim of this review was to evaluate the effectiveness of scalp, tongue, and Jin’s 3-needle acupuncture for the improvement of post-apoplectic aphasia. PubMed, Cochrane, Embase databases were searched using index words to identify qualifying randomized controlled trials (RCTs). Meta-analyses of odds ratios (OR) or standardized mean differences (SMD) were performed to evaluate the outcomes between investigational (scalp / tongue / Jin’s 3-needle acupuncture) and control (traditional acupuncture; TA and/or rehabilitation training; RT) groups.
Thirty-two RCTs (1310 participants in investigational group and 1270 in control group) were included. Compared to TA, (OR 3.05 [95% CI: 1.77, 5.28]; p<0.00001), tongue acupuncture (OR 3.49 [1.99, 6.11]; p<0.00001), and Jin’s 3-needle therapy (OR 2.47 [1.10, 5.53]; p = 0.03) had significantly better total effective rate. Compared to RT, scalp acupuncture (OR 4.24 [95% CI: 1.68, 10.74]; p = 0.002) and scalp acupuncture with tongue acupuncture (OR 7.36 [3.33, 16.23]; p<0.00001) had significantly better total effective rate. In comparison with TA/RT, scalp acupuncture, tongue acupuncture, scalp acupuncture with tongue acupuncture, and Jin’s three-needling significantly improved ABC, oral expression, comprehension, writing and reading scores.
The authors concluded that compared to traditional acupuncture and/or rehabilitation training, scalp acupuncture, tongue acupuncture, and Jin’ 3-needle acupuncture can better improve post-apoplectic aphasia as depicted by the total effective rate, the ABC score, and comprehension, oral expression, repetition, denomination, reading and writing scores. However, quality of the included studies was inadequate and therefore further high-quality studies with lager samples and longer follow-up times and with patient outcomes are necessary to verify the results presented herein. In future studies, researchers should also explore the efficacy and differences between scalp acupuncture, tongue acupuncture and Jin’s 3-needling in the treatment of post-apoplectic aphasia.
I’ll be frank: I find it hard to believe that sticking needles in a patient’s tongue restores her ability to speak. What is more, I do not believe a word of this review and its conclusion. And now I better explain why.
- All the primary studies originate from China, and we have often discussed how untrustworthy such studies are.
- All the primary studies were published in Chinese and cannot therefore be checked by most readers of the review.
- The review authors fail to provide the detail about a formal assessment of the rigour of the included studies; they merely state that their methodological quality was low.
- Only 6 of the 32 studies can be retrieved at all via the links provided in the articles.
- As far as I can find out, some studies do not even exist at all.
- Many of the studies compare acupuncture to unproven therapies such as bloodletting.
- Many do not control for placebo effects.
- Not one of the 32 studies reports findings that are remotely convincing.
I conclude that such reviews are little more than pseudo-scientific propaganda. They seem aim at promoting acupuncture in the West and thus serve the interest of the People’s Republic of China. They pollute our medical literature and undermine the trust in science.
I seriously ask myself, are the editors and reviewers all fast asleep?
The journal ‘BMC Complement Altern Med‘ has, in its 18 years of existence, published almost 4 000 Medline-listed papers. They currently charge £1690 for handling one paper. This would amount to about £6.5 million! But BMC are not alone; as I have pointed out repeatedly, EBCAM is arguably even worse.
And this is, in my view, the real scandal. We are being led up the garden path by people who make a very tidy profit doing so. BMC (and EBCAM) must put an end to this nonsense. Alternatively, PubMed should de-list these publications.
This has been going on for far too long; urgent action is required!
Hirudotherapy, also known as leech therapy, has been used to treat a wide range of disorders for thousands of years. It is also mentioned as a minimal invasive technique called Jalaukavacharana in the Sushruta Samhita, an ancient Sanskrit text of Ayurvedic medicine.
But a long history is a fallacious argument (appeal to tradition) when used to imply efficacy. So, does this treatment work?
A review located a total of 834 articles were found of which 89.8% were original articles. USA was the leading country with 280 publications, followed by UK, Germany and France (128, 101 and 41 items, respectively). The most productive countries regarding hirudotherapy were the UK (1.93), Slovenia (1.44), and Israel (1.32). The peak publication year for hirudotherapy literature was 2011 with 41 papers.
What does that tell us about the efficacy of leech therapy?
The authors of another review concluded that reached the following conclusion: ” Given the low number of reported adverse events, leech therapy may be a useful approach in treating this condition. Further high-quality RCTs are required for the conclusive judgment of its effectiveness and safety.”
The few clinical trials that were reviewed are mostly by one research group – and yes, you guessed it: it was also this group who published the review.
And anyway: why do they conclude that there is a low number of adverse events? Firstly, there is no reporting system for such events; so, a low number is next to meaningless. Secondly, there are several reports of adverse events. Here are three recent cases:
A 59-year-old woman was admitted to the emergency department with complaints of redness and swelling in both eyes and face. She had a long history of headache, therefore applied leech treatment occasionally. Swelling began on the face after the treatment of leech therapy. Vital signs were as follows; fever: 36.5°C, BP: 126/81 mmHg, heart rate: 84/min and sO2: 98%; respiratory rate: 12/min. In physical examination, GCS was 15, conscious, oriented cooperative. There was no lymphadenopathy in the palpation of the head and neck examination. Oropharynx was in natural appearance and no uvula edema. Facial palpation revealed redness, pain and heat rise. Other systemic findings were normal. Laboratory tests showed leukocytes: 11,000/mm3 (4,000-10,000/mm3), haemoglobin: 12.8 g/dL (12,00-14,00 g/dL) platelet: 271,000 (100,000-400,000/mm3) CRP: 3.45 mg/L (0-0.5mg/L). Other parameters were within normal limits. Computed tomography (CT) showed bilateral periorbital, frontal subcutaneous soft tissue oedema and lymphatic dilatations. She was hospitalized with the diagnosis of orbital cellulite due to leech therapy.
Anorectal sepsis usually presents with anal abscesses, which may evolve to become anorectal fistulas. Most of these cases are either of cryptoglandular origin, or they develop secondary to inflammatory bowel diseases. A 32-year-old male patient applied to our Proctology Unit with severe anal pain and swelling. Three days before admission, leeches were applied to the hemorrhoidal swellings in a medical center. The abscess was drained with appropriate unroofing and search for any compartments. The patient recovered rapidly. The abscess culture and microscopy revealed mix flora with predominant Escherichia coli. After 6 months, he has been symptom-free with perfect healing of the surgical site. We need to check up on possible handicaps in our modern patient care policies that divert people to such methods. Nevertheless, such alternative methods should be regarded as nonscientific and out of context unless their efficacy and safety are documented.
Pseudolymphoma, also known as Jessner’s lymphocytic infiltration, is a benign but usually chronic, T-cell infiltrating disease with erythematous papules and plaques usually seen on the skin of the face, neck, and back. The use of leech therapy also known as hirudotherapy has increased in recent years. Here, we report a 52-year-old male patient who had undergone hirudotherapy in his neck and developed infiltrating plaques after four months. A skin biopsy confirmed the diagnosis of Jessner’s lymphocytic infiltration. In parallel with the increasing use of hirudotherapy in recent years, the side-effect reports will likely to increase. Indications and contraindications of hirudotherapy, which is being used officially in hospitals, should be taken into consideration.
So, what do we make of this evidence?
I don’t know about you, but I am not likely to try or recommend leech therapy in a hurry.
Spinal manipulation is a treatment employed by several professions, including physiotherapists and osteopaths; for chiropractors, it is the hallmark therapy.
- They use it for (almost) every patient.
- They use it for (almost) every condition.
- They have developed most of the techniques.
- Spinal manipulation is the focus of their education and training.
- All textbooks of chiropractic focus on spinal manipulation.
- Chiropractors are responsible for most of the research on spinal manipulation.
- Chiropractors are responsible for most of the adverse effects of spinal manipulation.
Spinal manipulation has traditionally involved an element of targeting the technique to a level of the spine where the proposed movement dysfunction is sited. This study evaluated the effects of a targeted manipulative thrust versus a thrust applied generally to the lumbar region.
Sixty patients with low back pain were randomly allocated to two groups: one group received a targeted manipulative thrust (n=29) and the other a general manipulation thrust (GT) (n=31) to the lumbar spine. Thrust was either localised to a clinician-defined symptomatic spinal level or an equal force was applied through the whole lumbosacral region. The investigators measured pressure-pain thresholds (PPTs) using algometry and muscle activity (magnitude of stretch reflex) via surface electromyography. Numerical ratings of pain and Oswestry Disability Index scores were collected.
Repeated measures of analysis of covariance revealed no between-group differences in self-reported pain or PPT for any of the muscles studied. The authors concluded that a GT procedure—applied without any specific targeting—was as effective in reducing participants’ pain scores as targeted approaches.
The authors point out that their data are similar to findings from a study undertaken with a younger, military sample, showing no significant difference in pain response to a general versus specific rotation, manipulation technique. They furthermore discuss that, if ‘targeted’ manipulation proves to be no better than ‘general’ manipulation (when there has been further research, more studies), it would challenge the need for some current training courses that involve comprehensive manual skill training and teaching of specific techniques. If simple SM interventions could be delivered with less training, than the targeted approach currently requires, it would mean a greater proportion of the population who have back pain could access those general manipulation techniques.
Assuming that the GT used in this trial was equivalent to a placebo control, another interpretation of these results is that the effects of spinal manipulation are largely or even entirely due to a placebo response. If this were confirmed in further studies, it would be yet one more point to argue that spinal manipulation is not a treatment of choice for back pain or any other condition.
A systematic review of the evidence for effectiveness and harms of specific spinal manipulation therapy (SMT) techniques for infants, children and adolescents has been published by Dutch researchers. I find it important to stress from the outset that the authors are not affiliated with chiropractic institutions and thus free from such conflicts of interest.
They searched electronic databases up to December 2017. Controlled studies, describing primary SMT treatment in infants (<1 year) and children/adolescents (1–18 years), were included to determine effectiveness. Controlled and observational studies and case reports were included to examine harms. One author screened titles and abstracts and two authors independently screened the full text of potentially eligible studies for inclusion. Two authors assessed risk of bias of included studies and quality of the body of evidence using the GRADE methodology. Data were described according to PRISMA guidelines and CONSORT and TIDieR checklists. If appropriate, random-effects meta-analysis was performed.
Of the 1,236 identified studies, 26 studies were eligible. In all but 3 studies, the therapists were chiropractors. Infants and children/adolescents were treated for various (non-)musculoskeletal indications, hypothesized to be related to spinal joint dysfunction. Studies examining the same population, indication and treatment comparison were scarce. Due to very low quality evidence, it is uncertain whether gentle, low-velocity mobilizations reduce complaints in infants with colic or torticollis, and whether high-velocity, low-amplitude manipulations reduce complaints in children/adolescents with autism, asthma, nocturnal enuresis, headache or idiopathic scoliosis. Five case reports described severe harms after HVLA manipulations in 4 infants and one child. Mild, transient harms were reported after gentle spinal mobilizations in infants and children, and could be interpreted as side effect of treatment.
The authors concluded that, based on GRADE methodology, we found the evidence was of very low quality; this prevented us from drawing conclusions about the effectiveness of specific SMT techniques in infants, children and adolescents. Outcomes in the included studies were mostly parent or patient-reported; studies did not report on intermediate outcomes to assess the effectiveness of SMT techniques in relation to the hypothesized spinal dysfunction. Severe harms were relatively scarce, poorly described and likely to be associated with underlying missed pathology. Gentle, low-velocity spinal mobilizations seem to be a safe treatment technique in infants, children and adolescents. We encourage future research to describe effectiveness and safety of specific SMT techniques instead of SMT as a general treatment approach.
We have often noted that, in chiropractic trials, harms are often not mentioned (a fact that constitutes a violation of research ethics). This was again confirmed in the present review; only 4 of the controlled clinical trials reported such information. This means harms cannot be evaluated by reviewing such studies. One important strength of this review is that the authors realised this problem and thus included other research papers for assessing the risks of SMT. Consequently, they found considerable potential for harm and stress that under-reporting remains a serious issue.
Another problem with SMT papers is their often very poor methodological quality. The authors of the new review make this point very clearly and call for more rigorous research. On this blog, I have repeatedly shown that research by chiropractors resembles more a promotional exercise than science. If this field wants to ever go anywhere, if needs to adopt rigorous science and forget about its determination to advance the business of chiropractors.
I feel it is important to point out that all of this has been known for at least one decade (even though it has never been documented so scholarly as in this new review). In fact, when in 2008, my friend and co-author Simon Singh, published that chiropractors ‘happily promote bogus treatments’ for children, he was sued for libel. Since then, I have been legally challenged twice by chiropractors for my continued critical stance on chiropractic. So, essentially nothing has changed; I certainly do not see the will of leading chiropractic bodies to bring their house in order.
May I therefore once again suggest that chiropractors (and other spinal manipulators) across the world, instead of aggressing their critics, finally get their act together. Until we have conclusive data showing that SMT does more good than harm to kids, the right thing to do is this: BEHAVE LIKE ETHICAL HEALTHCARE PROFESSIONALS: BE HONEST ABOUT THE EVIDENCE, STOP MISLEADING PARENTS AND STOP TREATING THEIR CHILDREN!
Controlled clinical trials are methods for testing whether a treatment works better than whatever the control group is treated with (placebo, a standard therapy, or nothing at all). In order to minimise bias, they ought to be randomised. This means that the allocation of patients to the experimental and the control group must not be by choice but by chance. In the simplest case, a coin might be thrown – heads would signal one, tails the other group.
In so-called alternative medicine (SCAM) where preferences and expectations tend to be powerful, randomisation is particularly important. Without randomisation, the preference of patients for one or the other group would have considerable influence on the result. An ineffective therapy might thus appear to be effective in a biased study. The randomised clinical trial (RCT) is therefore seen as a ‘gold standard’ test of effectiveness, and most researchers of SCAM have realised that they ought to produce such evidence, if they want to be taken seriously.
But, knowingly or not, they often fool the system. There are many ways to conduct RCTs that are only seemingly rigorous but, in fact, are mere tricks to make an ineffective SCAM look effective. On this blog, I have often mentioned the A+B versus B study design which can achieve exactly that. Today, I want to discuss another way in which SCAM researchers can fool us (and even themselves) with seemingly rigorous studies: the de-randomised clinical trial (dRCT).
The trick is to use random allocation to the two study groups as described above; this means the researcher can proudly and honestly present his study as an RCT with all the kudos these three letters seem to afford. And subsequent to this randomisation process, the SCAM researcher simply de-randomises the two groups.
To understand how this is done, we need first to be clear about the purpose of randomisation. If done well, it generates two groups of patients that are similar in all factors that might impact on the results of the study. Perhaps the most obvious factor is disease severity; one could easily use other methods to make sure that both groups of an RCT are equally severely ill. But there are many other factors which we cannot always quantify or even know about. By using randomisation, we make sure that there is an similar distribution of ALL of them in the two study groups, even those factors we are not even aware of.
De-randomisation is thus a process whereby the two previously similar groups are made to differ in terms of any factor that impacts on the results of the trial. In SCAM, this is often surprisingly simple.
Let’s use a concrete example. For our study of spiritual healing, the 5 healers had opted during the planning period of the study to treat both the experimental group and the control group. In the experimental group, they wanted to use their full healing power, while in the control group they would not employ it (switch it off, so to speak). It was clear to me that this was likely to lead to de-randomisation: the healers would have (inadvertently or deliberately) behaved differently towards the two groups of patients. Before and during the therapy, they would have raised the expectation of the verum group (via verbal and non-verbal communication), while sending out the opposite signals to the control group. Thus the two previously equal groups would have become unequal in terms of their expectation. And who can deny that expectation is a major determinant of the outcome? Or who can deny that experienced clinicians can manipulate their patients’ expectation?
For our healing study, we therefore chose a different design and did all we could to keep the two groups comparable. Its findings thus turned out to show that healing is not more effective than placebo (It was concluded that a specific effect of face-to-face or distant healing on chronic pain could not be demonstrated over eight treatment sessions in these patients.). Had we not taken these precautions, I am sure the results would have been very different.
In RCTs of some SCAMs, this de-randomisation is difficult to avoid. Think of acupuncture, for instance. Even when using sham needles that do not penetrate the skin, the therapist is aware of the group allocation. Hoping to prove that his beloved acupuncture can be proven to work, acupuncturists will almost automatically de-randomise their patients before and during the therapy in the way described above. This is, I think, the main reason why some of the acupuncture RCTs using non-penetrating sham devices or similar sham-acupuncture methods suggest that acupuncture is more than a placebo therapy. Similar arguments also apply to many other SCAMs, including for instance chiropractic.
There are several ways of minimising this de-randomisation phenomenon. But the only sure way to avoid this de-randomisation is to blind not just the patient but also the therapists (and to check whether both remained blind throughout the study). And that is often not possible or exceedingly difficult in trials of SCAM. Therefore, I suggest we should always keep de-randomisation in mind. Whenever we are confronted with an RCT that suggest a result that is less than plausible, de-randomisation might be a possible explanation.
Autologous whole blood (AWB) therapy is a treatment where a patients blood is first drawn from a vein and then (unmodified or treated in various bizarre ways) reinjected intra-muscularly. This sounds barmy, not least because there is no remotely plausible mode of action. Nonetheless, the therapy is popular in some countries (like Germany, where it is practised by many doctors and Heilpraktikers) and recommended for all sorts of illnesses, particularly for strengthening the immune system and fend off infections.
I have personally used it quite a bit and even conducted the first but very small double-blind, placebo-controlled RCT of AWB therapy which showed promising results. Now two systematic reviews of AWB therapy have become available almost simultaneously.
The first systematic review included our plus 7 more clinical studies. The authors included all prospective controlled trials concerning intra-muscular AWB therapy with the exception of trials using oxygenated, UV radiated or heated blood. Information was extracted on the indication, design, additions to AWB and outcome. Full texts were screened for information about the effector mechanisms.
Eight trials met their inclusion criteria. In three controlled trials with patients suffering from atopic dermatitis and urticaria, AWB therapy showed beneficial effects. In five randomized controlled trials (RCTs), two of which concerned respiratory tract infections, two urticaria and one ankylosing spondylitis, no efficacy could be found. A quantitative assessment was not possible due to the heterogeneity of the included studies. The authors found only 4 controlled trials with sample sizes bigger than 37 individuals per group. Only one study investigated the effector mechanisms of AWB.
The German authors concluded that there is some evidence for efficacy of AWB therapy in urticaria patients and patients with atopic eczema. Firm conclusions can, however, not be drawn. We see a great need for further RCTs with adequate sample sizes and for investigation of the effector mechanisms of AWB therapy.
The second systematic review had a slightly different focus in that it assessed AWB therapy as well as autologous serum therapy (AST) for patients suffering from chronic spontaneous urticaria (CSU). Its authors managed to include 8 clinical trials. AST was not more effective than the placebo treatment in alleviating CSU symptoms at the end of treatment (p = .161), and AWB injection was also not more effective in response rates than the placebo at the end of follow-up (p = .099). Furthermore, the efficacy of AST or AWB injection for CSU and the ASST status were not significantly related. No remarkable adverse events were recorded during therapy.
The Taiwanese authors concluded that their meta-analysis suggested that AWB therapy and AST are not significantly more effective in alleviating CSU symptoms than the placebo treatment.
These somewhat contradictory conclusions will confuse most readers. Personally, I think that caution is well-justified. The trials are mostly flawed, and even our positive study (which received the highest possible quality marks by the authors of the first review) can in no way be definitive, because it was far too small for allowing firm conclusions.
Yet, despite all this, I do think that AWB therapy merits further study.
“There is a ton of chiropractor journals. If you want evidence then read some.”
This was the comment by a defender of chiropractic to a recent post of mine. And it’s true, of course: there are quite a few chiro journals, but are they a reliable source of information?
One way of quantifying the reliability of medical journals is to calculate what percentage of its published articles arrive at negative conclusion. In the extreme instance of a journal publishing nothing but positive results, we cannot assume that it is a credible publication. In this case, it would be not a scientific journal at all, but it would be akin to a promotional rag.
Back in 1997, we published our first analysis of journals of so-called alternative medicine (SCAM). It showed that just 1% of the papers published in SCAM journals reported findings that were not positive. In the years that followed, we confirmed this deplorable state of affairs repeatedly, and on this blog I have shown that the relatively new EBCAM journal is similarly dubious.
But these were not journals focussing specifically on chiropractic. Therefore, the question whether chiro journals are any different from the rest of SCAM is as yet unanswered. Enough reason for me to bite the bullet and test this hypothesis. I thus went on Medline and assessed all the articles published in 2018 in two of the leading chiro journals.
- JOURNAL OF CHIROPRACTIC MEDICINE (JCM)
- CHIROPRACTIC AND MANUAL THERAPY (CMT)
I evaluated them according to
- TYPE OF ARTICLE
- DIRECTION OF CONCLUSION
The results of my analysis are as follows:
- The JCM published 39 Medline-listed papers in 2018.
- The CMT published 50 such papers in 2018.
- Together, the 2 journals published:
- 18 surveys,
- 17 case reports,
- 10 reviews,
- 8 diagnostic papers,
- 7 pilot studies,
- 4 protocols,
- 2 RCTs,
- 2 non-randomised trials,
- 2 case-series,
- the rest are miscellaneous types of articles.
4. None of these papers arrived at a conclusion that is negative or contrary to chiropractors’ current belief in chiropractic care. The percentage of publishing negative findings is thus exactly 0%, a figure that is almost identical to the 1% we found for SCAM journals in 1997.
I conclude: these results suggest that the hypothesis of chiro journals publishing reliable information is not based on sound evidence.