For some researchers, the question whether homeopathy works beyond a placebo effect is not as relevant as the question whether it works as well as an established treatment. To answer it, they must conduct RCTs comparing homeopathy with a therapy that has been shown beyond reasonable doubt to be effective, i.e better than placebo. Such a drug is, for instance, Ibuprofen.
The purpose of this study was to compare the efficacy of Ibuprofen and homeopathic Belladonna for orthodontic pain. 51 females and 21 males, were included in this study. Cases with non-extraction treatment plan having proper contacts’ mesial and distal to permanent first molar and currently not taking any analgesics or antibiotics were included in the study. They were randomly divided into two groups; one group was assigned to ibuprofen 400 mg and second group took Belladonna 6C (that’s a dilution of 1: 1000000000000). Patients were given two doses of medication of their respective remedies one hour before placement of elastomeric separators (Ormco Separators, Ormco Corporation, CA, USA) and one dose 6 h after the placement. Pain scores were recorded on a visual analogue scale (VAS) 2 h after placement, 6 h after placement, bedtime, day 1 morning, day 2 morning, day 3 morning and day 5 morning.
The comparisons showed that there were no differences between the two groups at any time point.
(Mean visual analogue scale pain score at different time intervals after separator placement in Ibuprofen and Belladonna group)
The authors concluded that Ibuprofen and Belladonna 6C are effective and provide adequate analgesia with no statistically significant difference. Lack of adverse effects with Belladonna 6C makes it an effective and viable alternative.
FINALLY, THE PROOF HOMEOPATHS HAVE BEEN WAITING FOR: HOMEOPATHY DOES WORK AFTER ALL!
Not so fast – before we draw any conclusions, let’s have a closer look at this study. Here are a few of its limitations (apart from the fact that it was published in a journal that does not exactly belong to the ‘crème de la crème’ of medical publications):
- Patients obviously knew which group they were assigned to; thus their expectations would have influenced the outcome.
- The same applies to the researchers (the study could have been ‘blind’ using a ‘double dummy’ method, but the researchers did not use it).
- The study was an equivalence trial (it did not test whether homeopathy is superior to placebo, but whether its effects are equivalent to Ibuprofen); such studies need sample sizes that are about one dimension larger than was the case here.
Therefore, all this trial does demonstrate that the sample was too small for an existing group difference in favour of Ibuprofen to show.
So sorry, my homeopathic friends!
The American Dance Therapy Association defines Dance Movement Therapy as the psychotherapeutic use of movement to promote emotional, social, cognitive, and physical integration of the participant.
Dance/movement therapy is:
- Focused on movement behavior as it emerges in the therapeutic relationship. Expressive, communicative, and adaptive behaviors are all considered for group and individual treatment. Body movement, as the core component of dance, simultaneously provides the means of assessment and the mode of intervention for dance/movement therapy.
- Practiced in mental health, rehabilitation, medical, educational and forensic settings, and in nursing homes, day care centers, disease prevention, health promotion programs and in private practice.
- Effective for individuals with developmental, medical, social, physical and psychological impairments.
- Used with people of all ages, races and ethnic backgrounds in individual, couples, family and group therapy formats.
This sounds interesting, but does dance therapy work?
The aim of this paper was to perform a systematic review on the effectiveness of dance-based programs in patients with fibromyalgia, as well as calculate the overall effect size of the improvements, through a meta-analysis.
A total of 7 RCTs fulfilled all inclusion criteria. Their methodological quality was low. Duration of dance programs ranged from 12 to 24 weeks. Sessions lasted between 60 and 120 minutes and were performed 1-2 times per week. The overall effect size for pain was -1.64 with a 95% CI from -2.69 to -0.59 which can be interpreted as large. In addition, significant improvements were observed in quality of life, depression, impact of the disease, anxiety, and physical function.
The authors concluded that dance-based intervention programs can be an effective intervention for people suffering from fibromyalgia, leading to a significant reduction of the level of pain with an effect size that can be considered as large. However, findings and conclusions from this meta-analysis must be taken with caution due to the small number of articles and the large heterogeneity.
I don’t doubt that physical activity can ease pain, particularly, if combined with the often positive social interactions of dance. What is unclear to me is whether dance therapy generates results that are better than other forms of physical activity.
And then again, is that question really all that important? Perhaps the best advice to patients is to engage in the type of physical exercise the like best. At the very least, this would minimise the often poor compliance with such programs and might thus maximise their potential benefits.
The public is often impressed by scenes shown on TV where surgeons in China operate patients apparently with no other anaesthesia than acupuncture. Such films have undoubtedly contributed significantly to the common belief that acupuncture cannot possibly be a placebo (every single time I give a public talk about acupuncture, the issue comes up, and someone asks me: how can you doubt the efficacy of acupuncture when, in China, they use it for major operations?).
Some years ago, I have myself been involved is such a BBC broadcast and had to learn the hard way that such scenes are more than just a bit misleading.
Unfortunately, the experts rarely object to any of this. They seem to have become used to the false claims and overt propaganda that is rife in the promotion of acupuncture, and have resigned to the might of poor journalism.
The laudable exception is a team of French authors of a recent and excellent paper.
This unusual article analysed a clip from the program “Acupuncture, osteopathy, hypnosis: do complementary medicines have superpowers?” about acupuncture as an anaesthetic for surgical procedures in China. Their aim was to propose a rational explanation for the phenomena observed and to describe the processes leading a public service broadcasting channel to offer this type of content at prime time and the potential consequences in terms of public health. For this purpose, they used critical thinking attitudes and skills, along with a bibliographical search of Medline, Google Scholar and Cochrane Library databases.
Their results reveal that the information delivered in the television clip is ambiguous. It did not allow the viewer to form an informed opinion on the relevance of acupuncture as an anaesthetic for surgical procedures. It is reasonable to assume that the clip shows surgery performed with undisclosed epidural anaesthesia coupled with mild intravenous anaesthesia, sometimes performed in other countries.
What needs to be highlighted, the authors of this critique state, is the overestimation of acupuncture added to the protocol. The media tend to exaggerate the risks and expected effects of the treatments they report on, which can lead patients to turn to unproven therapies.
The authors concluded that broadcasting such a clip at prime time underlines the urgent need for the public and all health professionals to be trained in sorting and critically analysing health information.
In my view, broadcasting such misleading films also underlines the urgent need for journalists to be conscious of their responsibility not to mislead the public and do more rigorous research before reporting on matters of health.
Acupuncture is a branch of alternative medicine where pseudo-science abounds. Here is yet another example of this deplorable phenomenon.
This study was conducted to evaluate the efficacy of acupuncture in the management of primary dysmenorrhea.
Sixty females aged 17-23 years were randomly assigned to either a study group or a control group.
- The study group received acupuncture for the duration of 20 minutes/day, for 15 days/month, for the period of 90 days.
- The control group did not receive acupuncture for the same period.
Both groups were assessed on day 1; day 30 and day 60; and day 90. The results showed a significant reduction in all the variables such as the visual analogue scale score for pain, menstrual cramps, headache, dizziness, diarrhoea, faint, mood changes, tiredness, nausea, and vomiting in the study group compared with those in the control group.
The authors concluded that acupuncture could be considered as an effective treatment modality for the management of primary dysmenorrhea.
These findings contradict those of a recent Cochrane review (authored by known acupuncture-proponents) which included 42 RCTs and concluded that there is insufficient evidence to demonstrate whether or not acupuncture or acupressure are effective in treating primary dysmenorrhoea, and for most comparisons no data were available on adverse events. The quality of the evidence was low or very low for all comparisons. The main limitations were risk of bias, poor reporting, inconsistency and risk of publication bias.
The question that I ask myself is this: why do researchers bother to conduct studies that contribute NOTHING to our knowledge and progress? The new study had a no-treatment control group which means it cannot control for the effects of placebo, the extra attention, social desirability etc. In view of the fact that already 42 poor quality trials exist, it is not just useless to add a 43rd but, in my view, it is scandalous! A 43rd useless trial:
- tells us nothing of value;
- misleads the public;
- pollutes the medical literature;
- is a waste of resources;
- undermines the trust in clinical research;
- is deeply unethical.
It is high time to stop such redundant, foolish, wasteful and unethical pseudo-science.
Mini-scalpel acupuncture or acupotomy is a relatively new type of non-invasive acupuncture/ micro surgery using a small needle-scalpel invented by Professor Zhu Hanzhang around 30 years ago in China. It is a slightly thicker and more blunt instrument that gets under the skin and is able to break apart adhesions and muscle knots more effectively than a regular acupuncture needle would.
Never mind, the question is does it work!
A systematic review showed that almost all studies reported an effect of acupotomy on joint pain compared to a variety of controls. On reflection, this is hardly surprising:
- all the trials were from China;
- all had major methodological flaws.
This means that we need better studies to decide the efficacy question.
This new study investigated the efficacy and safety of mini-scalpel acupuncture (MA) for knee osteoarthritis (KOA) in an assessor-blinded randomized controlled pilot trial; this would provide information for a large-scale randomized controlled trial.
Participants (n = 24) were recruited and randomly allocated to the MA group (experimental) or acupuncture group (control). The MA group received treatment once a week for 3 weeks (total of 3 treatments), while the acupuncture group received treatment two times per week for 3 weeks (total of 6 treatments). The primary outcome was pain as assessed by a visual analogue scale (VAS). The secondary outcomes (intensity of current pain, stiffness, and physical function) were assessed using the short-form McGill Pain Questionnaire (SF-MPQ) and Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC). Assessments were performed at baseline, 1, 2, and 3 during treatment and at week 5 (2 weeks after the end of treatment).
Of the 24 participants, 23 completed the study. Both groups showed significant improvements in VAS, SF-MPQ, and WOMAC. There were no significant differences between the MA and acupuncture groups. No serious adverse event occurred and blood test results were within normal limits.
The authors concluded that although both MA and acupuncture provide similar effects with regard to pain control in patients with KOA, MA may be more effective in providing pain relief because the same relief was obtained with fewer treatments. A large-scale clinical study is warranted to further clarify these findings.
I can recommend this article to anyone who wants a quick introduction into the critical analysis of clinical trials. It is a veritable treasure trove of mistakes, flaws, errors, fallacies etc. Here are just a few:
- The authors aim of investigating the safety of MA is unobtainable. It would require not 24 but probably 24 000 patients.
- The authors aim of investigating the efficacy of MA is equally unobtainable. It would require a much larger sample than 24, a sham control arm, identical treatment schedules, patient-blinding, etc.
- Calling the trial a ‘pilot’ is endearing but, except for the title and the insufficient sample size, this study has none of the characteristics of a pilot study.
- In their ‘introduction’, the authors state that miniscalpel acupuncture (MA) is a new subtype of acupuncture that is effective in treating chronic soft tissue injuries such as adhesions and contractures. This is clearly wrong but discloses their bias very plainly.
- The authors statement that both MA and acupuncture provide similar effects with regard to pain control in patients with KOA is misleading. It implies that both interventions had specific effects. Without a sham control arm, this is pure speculation.
- Similarly their assumption that MA may be more effective in providing pain relief because the same relief was obtained with fewer treatments, is pure fantasy.
- In fact, as MA requires injections of local anaesthetics, any outcome is heavily confounded by this addition.
- In the discussion section, the authors state that because MA is invasive and provides a strong stimulus, some participants complained of stiff and dull pain for few days after treatment. Yet, when reporting adverse effects in the results section, this was not mentioned.
- The way this study was designed, it should have been clear from the start that it would not produce any meaningful findings. Seen from this perspective, running the trial could even be seen as a breach of research ethics.
- According to the aims of a pilot study and the authors hope that their study would provide information for a large-scale randomized controlled trial, all reporting of outcomes is misplaced and should be replaced by information as to how a definitive trial should be conducted.
The following footnote is worth mentioning: This study is supported by a grant from the Ministry of Health & Welfare, Korea. It suggests to me that this ministry should urgently re-think its funding strategy and recruit some reviewers who are capable of critical analysis.
In my view, this is a lousy study which the authors decides to call ‘a pilot’ in order to get it published in a lousy journal.
Bee venom acupuncture is a form of acupuncture in which bee venom is applied to the tips of acupuncture needles, stingers are extracted from bees, or bees are held with an instrument exposing the stinger, and applied to acupoints on the skin.
Bee venom consisting of multiple anti-inflammatory compounds such as melittin, adolapin, apamin. Other substances such as phospholipase A2 can be anti-inflammatory in low concentrations and pro-inflammatory in others. However, bee venom also contains proinflammatory substances, melittin, mast cell degranulation peptide 401, and histamine.
Bee venom acupuncture has been used to treat a number of conditions such as lumbar disc disease, osteoarthritis of the knee, rheumatoid arthritis, adhesive capsulitis, lateral epicondylitis, peripheral neuropathies, stroke and Parkinson’s Disease. The quality of these studies tends to be so poor that any verdict on the effectiveness of bee venom acupuncture would be premature.
A new clinical trial of bee-venom acupuncture for rheumatoid arthritis (RA) might change this situation. A total of 120 cases of RA patients were randomized into bee-sting acupuncture group (treatment) and western medicine group (control). The patients of the control group were treated by oral administration of Methotrexate (10 mg, once a week) and Celecoxlb (0.2 g, once a day). Those of the treatment group received 5 to 15 bee stings of Ashi-points or acupoints according to different conditions and corporeity, and with the bee-sting retained for about 5 min every time, once every other day. The treatment lasted for 8 weeks. The therapeutic effect was assessed by examining:
- symptoms and signs of the affected joints as morning stiffness duration,
- swollen/tender joint counts (indexes),
- handgrip strength,
- 15 m-walking time,
- visual analogue scale (VAS),
- Disease Activity Score including a 28-joint count (DAS 28),
- rheumatoid factor (RF),
- erythrocyte sedimentation rate (ESR),
- C-reactive protein (CRP),
- anti-cyclic citrullinated peptide antibody (ACCPA).
For assessing the safety of bee-venom acupuncture, the patients’ responses of fever, enlargement of lymph nodes, regional red and swollen, itching, blood and urine tests for routine were examined.
Findings of DAS 28 responses displayed that of the two 60 cases in the control and bee-venom acupuncture groups, 15 and 18 experienced marked improvement, 33 and 32 were effective, 12 and 10 ineffective, with the effective rates being 80% and 83. 33%, respectively. No significant difference was found between the two groups in the effective rate (P>0.05). After the treatment, both groups have witnessed a marked decrease in the levels of morning stiffness duration, arthralgia index, swollen joint count index, joint tenderness index, 15 m walking time, VAS, RF, ESR, CRP and ACCPA, and an obvious increase of handgrip strength relevant to their own levels of pre-treatment in each group (P<0.05). There were no significant differences between the two groups in the abovementioned indexes (P>0.05). The routine blood test, routine urine test, routine stool test, electrocardiogram result, the function of liver and kidney and other security index were within the normal range, without any significant adverse effects found after bee-stinging treatment.
The authors (from the Department of Acupuncture and Moxibustion, Bao’an Hospital of Traditional Chinese Medicine, Shenzhen, China) concluded that bee-venom acupuncture therapy for RA patients is safe and effective, worthy of popularization and application in clinical practice.
Where to start? There is so much – perhaps I just comment on the conclusion:
- Safety cannot be assessed on the basis of such a small sample. Bee venom can cause anaphylaxis, and several deaths have been reported in patients who successfully received the therapy prior to the adverse event. Because there is no adverse-effect monitoring system, the incidence of adverse events is unknown. Stating that it is safe, is therefore a big mistake.
- The trial was a non-superiority study. As such, it needs a much larger sample to be able to make claims about effectiveness.
- From the above two points, it follows that popularization and application in clinical practice would be a stupid exercise.
So, what is left over from this seemingly rigorous RCT?
(except perhaps a re-affirmation of my often-voiced fear that we must take TCM-studies from China with more than just one pinch of salt)
The aim of this RCT was to investigate the effects of an osteopathic manipulative treatment (OMT) which includes a diaphragm intervention compared to the same OMT with a sham diaphragm intervention in chronic non-specific low back pain (NS-CLBP).
Participants (N=66) with a diagnosis of NS-CLBP lasting at least 3 months were randomized to receive either an OMT protocol including specific diaphragm techniques (n=33) or the same OMT protocol with a sham diaphragm intervention (n=33), conducted in 5 sessions provided during 4 weeks.
The primary outcomes were pain (evaluated with the Short-Form McGill Pain Questionnaire [SF-MPQ] and the visual analog scale [VAS]) and disability (assessed with the Roland-Morris Questionnaire [RMQ] and the Oswestry Disability Index [ODI]). Secondary outcomes were fear-avoidance beliefs, level of anxiety and depression, and pain catastrophization. All outcome measures were evaluated at baseline, at week 4, and at week 12.
A statistically significant reduction was observed in the experimental group compared to the sham group in all variables assessed at week 4 and at week 12. Moreover, improvements in pain and disability were clinically relevant.
The authors concluded that an OMT protocol that includes diaphragm techniques produces significant and clinically relevant improvements in pain and disability in patients with NS-CLBP compared to the same OMT protocol using sham diaphragm techniques.
This seems to be a rigorous study. The authors describe in detail their well-standardised interventions in the full text of their paper. This, of course, will be essential, if someone wants to repeat the trial.
I have but a few points to add:
- What I fail to understand is this: why the authors call the interventions osteopathic? The therapist was a physiotherapist and the techniques employed are, if I am not mistaken, as much physiotherapeutic as osteopathic.
- The findings of this trial are encouraging but almost seem a little too good to be true. They need, of course, to be independently replicated in a larger study.
- If that is done, I would suggest to check whether the blinding of the patient was successful. If not, there is a suspicion that the diaphragm technique works partly or mostly via a placebo effect.
- I would also try to make sure that the therapist cannot influence the results in any way, for instance, by verbal or non-verbal suggestions.
- Finally, I suggest to employ more than one therapist to increase generalisability.
Once all these hurdles are taken, we might indeed have made some significant progress in the manual therapy of NS-CLBP.
Osteopathy is a form of manual therapy invented by the American Andrew Taylor Still (1828-1917). Today, US osteopaths (doctors of osteopathy or DOs) practise no or little manual therapy; they are fully recognised as medical doctors who can specialise in any medical field after their training which is almost identical with that of MDs. Outside the US, osteopaths practice almost exclusively manual treatments and are considered alternative practitioners. This post deals with the latter category of osteopaths.
Still defined his original osteopathy as a science which consists of such exact, exhaustive, and verifiable knowledge of the structure and function of the human mechanism, anatomical, physiological and psychological, including the chemistry and physics of its known elements, as has made discoverable certain organic laws and remedial resources, within the body itself, by which nature under the scientific treatment peculiar to osteopathic practice, apart from all ordinary methods of extraneous, artificial, or medicinal stimulation, and in harmonious accord with its own mechanical principles, molecular activities, and metabolic processes, may recover from displacements, disorganizations, derangements, and consequent disease, and regained its normal equilibrium of form and function in health and strength.
Based on such vague and largely nonsensical statements, traditional osteopaths feel entitled to offer treatments for most human diseases, conditions and symptoms. The studies they produce to back up their claims tend to be as poor as Still’s original assumptions were fantastic.
Here is an apt example:
The aim of this new study was to study the effect of osteopathic manipulation on pain relief and quality of life improvement in hospitalized oncology geriatric patients.
The researchers conducted a non-randomized controlled clinical trial with 23 cancer patients. They were allocated to two groups: the study group (OMT [osteopathic manipulative therapy] group, N = 12) underwent OMT in addition to physiotherapy (PT), while the control group (PT group, N = 12) underwent only PT. Included were postsurgical cancer patients, male and female, age ⩾65 years, with an oncology prognosis of 6 to 24 months and chronic pain for at least 3 months with an intensity score higher than 3, measured with the Numeric Rating Scale. Exclusion criteria were patients receiving chemotherapy or radiotherapy treatment at the time of the study, with mental disorders (Mini-Mental State Examination [MMSE] = 10-20), with infection, anticoagulation therapy, cardiopulmonary disease, or clinical instability post-surgery. Oncology patients were admitted for rehabilitation after cancer surgery. The main cancers were colorectal cancer, osteosarcoma, spinal metastasis from breast and prostatic cancer, and kidney cancer.
The OMT, based on osteopathic principles of body unit, structure-function relationship, and homeostasis, was designed for each patient on the basis of the results of the osteopathic examination. Diagnosis and treatment were founded on 5 models: biomechanics, neurologic, metabolic, respiratory-circulatory, and behaviour. The OMT protocol was administered by an osteopath with clinical experience of 10 years in one-on-one individual sessions. The techniques used were: dorsal and lumbar soft tissue, rib raising, back and abdominal myofascial release, cervical spine soft tissue, sub-occipital decompression, and sacroiliac myofascial release. Back and abdominal myofascial release techniques are used to improve back movement and internal abdominal pressure. Sub-occipital decompression involves traction at the base of the skull, which is considered to release restrictions around the vagus nerve, theoretically improving nerve function. Sacroiliac myofascial release is used to improve sacroiliac joint movement and to reduce ligament tension. Strain-counter-strain and muscle energy technique are used to diminish the presence of trigger points and their pain intensity. OMT was repeated once every week during 4 weeks for each group, for a total of 4 treatments. Each treatment lasted 45 minutes.
At enrolment (T0), the patients were evaluated for pain intensity and quality of life by an external examiner. All patients were re-evaluated every week (T1, T2, T3, and T4) for pain intensity, and at the end of the study treatment (T4) for quality of life.
The OMT added to physiotherapy produced a significant reduction in pain both at T2 and T4. The difference in quality of life improvements between T0 and T4 was not statistically significant. Pain improved in the PT group at T4. Between-group analysis of pain and quality of life did not show any significant difference between the two treatments.
The authors concluded that our study showed a significant improvement in pain relief and a nonsignificant improvement in quality of life in hospitalized geriatric oncology patients during osteopathic manipulative treatment.
Where to begin?
Even if there had been a difference in outcome between the two groups, such a finding would not have shown an effect of OMT per se. More likely, it would have been due to the extra attention and the expectation in the OMT group (or caused by the lack of randomisation). The A+B vs B design used for this study does not control for non-specific effects. Therefore it is incapable of establishing a causal relationship between the therapy and the outcome.
As it turns out, there were no inter-group differences. How can this be? I have often stated that A+B is always more than B alone. And this is surely true!
So, how can I explain this?
As far as I can see, there are two possibilities:
- The study was underpowered, and thus an existing difference was not picked up.
- The OMT had a detrimental effect on the outcome measures thus neutralising the positive effects of the extra attention and expectation.
And which possibility does apply in this case?
Nobody can know from these data.
Integrative Cancer Therapies, the journal that published this paper, states that it focuses on a new and growing movement in cancer treatment. The journal emphasizes scientific understanding of alternative and traditional medicine therapies, and the responsible integration of both with conventional health care. Integrative care includes therapeutic interventions in diet, lifestyle, exercise, stress care, and nutritional supplements, as well as experimental vaccines, chrono-chemotherapy, and other advanced treatments. I feel that the editors should rather focus more on the quality of the science they publish.
My conclusion from all this is the one I draw so depressingly often: fatally flawed science is not just useless, it is unethical, gives clinical research a bad name, hinders progress, and can be harmful to patients.
Kinesiology tape KT is fashionable, it seems. Gullible consumers proudly wear it as decorative ornaments to attract attention and show how very cool they are.
Am I too cynical?
But does KT really do anything more?
A new trial might tell us.
The aim of this study was to investigate whether adding kinesiology tape (KT) to spinal manipulation (SM) can provide any extra effect in athletes with chronic non-specific low back pain (CNLBP).
Forty-two athletes (21males, 21females) with CNLBP were randomized into two groups of SM (n = 21) and SM plus KT (n = 21). Pain intensity, functional disability level and trunk flexor-extensor muscles endurance were assessed by Numerical Rating Scale (NRS), Oswestry pain and disability index (ODI), McQuade test, and unsupported trunk holding test, respectively. The tests were done before and immediately, one day, one week, and one month after the interventions and compared between the two groups.
After treatments, pain intensity and disability level decreased and endurance of trunk flexor-extensor muscles increased significantly in both groups. Repeated measures analysis, however, showed that there was no significant difference between the groups in any of the evaluations.
The authors, physiotherapists from Iran, concluded that the findings of the present study showed that adding KT to SM does not appear to have a significant extra effect on pain, disability and muscle endurance in athletes with CNLBP. However, more studies are needed to examine the therapeutic effects of KT in treating these patients.
Regular readers of my blog will be able to predict what I have to say about this study design: A+B versus B is not a meaningful test of anything. I used to claim that it cannot possibly produce a negative result – and yet, here it seems to have done exactly that!
The way I see it, there are two possibilities to explain this:
- the KT has a mildly negative effect on CNLBP; thus the expected positive placebo-effect was neutralised to result in a null-effect overall;
- the study was under-powered such that the true inter-group difference could not manifest itself.
I think the second possibility is more likely, but it does really not matter at all. Because the only lesson we can learn from this trial is this: inadequate study designs will hardly ever generate anything worthwhile.
And this is, I think, a lesson that would be valuable for many researchers.
Comparing spinal manipulation with and without Kinesio Taping® in the treatment of chronic low back pain.
Can I tempt you to run a little (hopefully instructive) thought-experiment with you? It is quite simple: I will tell you about the design of a clinical trial, and you will tell me what the likely outcome of this study would be.
Are you game?
Here we go:
Imagine we conduct a trial of acupuncture for persistent pain (any type of pain really). We want to find out whether acupuncture is more than a placebo when it comes to pain-control. Of course, we want our trial to look as rigorous as possible. So, we design it as a randomised, sham-controlled, partially-blinded study. To be really ‘cutting edge’, our study will not have two but three parallel groups:
1. Standard needle acupuncture administered according to a protocol recommended by a team of expert acupuncturists.
2. Minimally invasive sham-acupuncture employing shallow needle insertion using short needles at non-acupuncture points. Patients in groups 1 and 2 are blinded, i. e. they are not supposed to know whether they receive the sham or real acupuncture.
3. No treatment at all.
We apply the treatments for a sufficiently long time, say 12 weeks. Before we start, after 6 and 12 weeks, we measure our patients’ pain with a validated method. We use sound statistical methods to compare the outcomes between the three groups.
WHAT DO YOU THINK THE RESULT WOULD BE?
You are not sure?
Well, let me give you some hints:
Group 3 is not going to do very well; not only do they receive no therapy at all, but they are also disappointed to have ended up in this group as they joined the study in the hope to get acupuncture. Therefore, they will (claim to) feel a lot of pain.
Group 2 will be pleased to receive some treatment. However, during the course of the 6 weeks, they will get more and more suspicious. As they were told during the process of obtaining informed consent that the trial entails treating some patients with a sham/placebo, they are bound to ask themselves whether they ended up in this group. They will see the short needles and the shallow needling, and a percentage of patients from this group will doubtlessly suspect that they are getting the sham treatment. The doubters will not show a powerful placebo response. Therefore, the average pain scores in this group will decrease – but only a little.
Group 1 will also be pleased to receive some treatment. As the therapists cannot be blinded, they will do their best to meet the high expectations of their patients. Consequently, they will benefit fully from the placebo effect of the intervention and the pain score of this group will decrease significantly.
So, now we can surely predict the most likely result of this trial without even conducting it. Assuming that acupuncture is a placebo-therapy, as many people do, we now see that group 3 will suffer the most pain. In comparison, groups 1 and 2 will show better outcomes.
Of course, the main question is, how do groups 1 and 2 compare to each other? After all, we designed our sham-controlled trial in order to answer exactly this issue: is acupuncture more than a placebo? As pointed out above, some patients in group 2 would have become suspicious and therefore would not have experienced the full placebo-response. This means that, provided the sample sizes are sufficiently large, there should be a significant difference between these two groups favouring real acupuncture over sham. In other words, our trial will conclude that acupuncture is better than placebo, even if acupuncture is a placebo.
THANK YOU FOR DOING THIS THOUGHT EXPERIMENT WITH ME.
Now I can tell you that it has a very real basis. The leading medical journal, JAMA, just published such a study and, to make matters worse, the trial was even sponsored by one of the most prestigious funding agencies: the NIH.
Here is the abstract:
Musculoskeletal symptoms are the most common adverse effects of aromatase inhibitors and often result in therapy discontinuation. Small studies suggest that acupuncture may decrease aromatase inhibitor-related joint symptoms.
To determine the effect of acupuncture in reducing aromatase inhibitor-related joint pain.
Design, Setting, and Patients:
Randomized clinical trial conducted at 11 academic centers and clinical sites in the United States from March 2012 to February 2017 (final date of follow-up, September 5, 2017). Eligible patients were postmenopausal women with early-stage breast cancer who were taking an aromatase inhibitor and scored at least 3 on the Brief Pain Inventory Worst Pain (BPI-WP) item (score range, 0-10; higher scores indicate greater pain).
Patients were randomized 2:1:1 to the true acupuncture (n = 110), sham acupuncture (n = 59), or waitlist control (n = 57) group. True acupuncture and sham acupuncture protocols consisted of 12 acupuncture sessions over 6 weeks (2 sessions per week), followed by 1 session per week for 6 weeks. The waitlist control group did not receive any intervention. All participants were offered 10 acupuncture sessions to be used between weeks 24 and 52.
Main Outcomes and Measures:
The primary end point was the 6-week BPI-WP score. Mean 6-week BPI-WP scores were compared by study group using linear regression, adjusted for baseline pain and stratification factors (clinically meaningful difference specified as 2 points).
Among 226 randomized patients (mean [SD] age, 60.7 [8.6] years; 88% white; mean [SD] baseline BPI-WP score, 6.6 [1.5]), 206 (91.1%) completed the trial. From baseline to 6 weeks, the mean observed BPI-WP score decreased by 2.05 points (reduced pain) in the true acupuncture group, by 1.07 points in the sham acupuncture group, and by 0.99 points in the waitlist control group. The adjusted difference for true acupuncture vs sham acupuncture was 0.92 points (95% CI, 0.20-1.65; P = .01) and for true acupuncture vs waitlist control was 0.96 points (95% CI, 0.24-1.67; P = .01). Patients in the true acupuncture group experienced more grade 1 bruising compared with patients in the sham acupuncture group (47% vs 25%; P = .01).
Conclusions and Relevance:
Among postmenopausal women with early-stage breast cancer and aromatase inhibitor-related arthralgias, true acupuncture compared with sham acupuncture or with waitlist control resulted in a statistically significant reduction in joint pain at 6 weeks, although the observed improvement was of uncertain clinical importance.
Do you see how easy it is to deceive (almost) everyone with a trial that looks rigorous to (almost) everyone?
My lesson from all this is as follows: whether consciously or unconsciously, SCAM-researchers often build into their trials more or less well-hidden little loopholes that ensure they generate a positive outcome. Thus even a placebo can appear to be effective. They are true masters of producing false-positive findings which later become part of a meta-analysis which is, of course, equally false-positive. It is a great shame, in my view, that even top journals (in the above case JAMA) and prestigious funders (in the above case the NIH) cannot (or want not to?) see behind this type of trickery.