Clinical trials are a most useful tool, but they can easily be abused. It is not difficult to misuse them in such a way that even the most useless treatment appears to be effective. Sadly, this sort of thing happens all too often in the realm of alternative medicine. Take for instance this recently published trial of homeopathy.
The objective of this study was to investigate the usefulness of classical homeopathy for the prevention of recurrent urinary tract infections (UTI) in patients with spinal cord injury (SCI). Patients were admitted to this trial, if they had chronic SCI and had previously suffered from at least three UTI/year. They were treated either with a standardized prophylaxis alone, or with a standardized prophylaxis in combination with homeopathy. The number of UTIs, general and specific quality of life (QoL), and satisfaction with homeopathic treatment were assessed prospectively over the period of one year. Ten patients were in the control group and 25 patients received adjunctive homeopathic treatment. The median number of self-reported UTI in the homeopathy group decreased significantly, whereas it remained unchanged in the control group. The domain incontinence impact of the KHQ improved significantly, whereas the general QoL did not change. The satisfaction with homeopathic care was high.
The authors concluded that adjunctive homeopathic treatment lead to a significant decrease of UTI in SCI patients. Therefore, classical homeopathy could be considered in SCI patients with recurrent UTI.
Where to begin?
Here are just some of the most obvious flaws of and concerns with this study:
- There is no plausible rationale to even plan such a study.
- The sample size was far too small for allowing generalizable conclusions.
- There was no adequate randomisation and patients were able to chose the homeopathy option.
- The study seems to lack objective outcome measures.
- The study design did not allow to control for non-specific effects; therefore, it seems likely that the observed outcomes are unrelated to the homeopathic treatments but are caused by placebo and other non-specific effects.
- Even if the study had been rigorous, we would need independent replications before we draw such definitive conclusions.
- Two of the authors are homeopaths, and it is in their clinics that the study took place.
- Some of the authors have previously published a very similar paper – except that this ‘case series’ included no control group at all.
- The latter paper seems to have been published more than once.
- Of this paper, one of the authors claimed that ” the usefulness of classical homeopathy as an adjunctive measure for UTI prophylaxis in patients with NLUTD due to SCI has been demonstrated in a case series”. He seems to be unaware of the fact that a case series cannot possible lend itself to demonstrate this.
- I do wonder: did they just add a control group to their case series thus pretending it became a controlled clinical trial?
What strikes me most with such pseudo-research is its abundance and the naivety – or should I call it ignorance? – of the enthusiasts who conduct it. Most of them, I am fairly sure do not mean to do harm; but by Jove they do!
Do chiropractors even know the difference between promotion and research?
Probably a rhetorical question.
Personally, I have seen them doing so much pseudo-research that I doubt they recognise the real thing, even if they fell over it.
Here is a recent example that stands for many, many more such ‘research’ projects (some of which have been discussed on this blog).
But first a few sentences on the background of this new ‘study’.
The UD chiropractic profession is currently on the ‘opioid over-use bandwagon’ hoping that this move might promote their trade. Most chiropractors have always been against using (any type of) pharmaceutical treatment and advise their patients accordingly. D D Palmer, the founder of chiropractic, was adamant that drugs are to be avoided; he stated for instance that Drugs are delusive; they do not adjust anything. And “as the Founder intended, chiropractic has existed as a drug-free healthcare profession for better than 120 years.” To this day, chiropractors are educated and trained to argue against non-drug treatments and regularly claim that chiropractic is a drug-free alternative to traditional medicine.
Considering this background, this new piece of (pseudo) research is baffling, in my view.
The objective of this investigation was to evaluate the association between utilization of chiropractic services and the use of prescription opioid medications. The authors used a retrospective cohort design to analyse health insurance claims data. The data source was the all payer claims database administered by the State of New Hampshire. The authors chose New Hampshire because health claims data were readily available for research, and in 2015, New Hampshire had the second-highest age-adjusted rate of drug overdose deaths in the United States.
The study population comprised New Hampshire residents aged 18-99 years, enrolled in a health plan, and with at least two clinical office visits within 90 days for a primary diagnosis of low-back pain. The authors excluded subjects with a diagnosis of cancer. They measured likelihood of opioid prescription fill among recipients of services delivered by chiropractors compared with a control group of patients not consulting a chiropractor. They also compared the cohorts with regard to rates of prescription fills for opioids and associated charges.
The adjusted likelihood of filling a prescription for an opioid analgesic was 55% lower among chiropractic compared to non-chiropractic patients. Average charges per person for opioid prescriptions were also significantly lower among the former group.
The authors concluded that among New Hampshire adults with office visits for noncancer low-back pain, the likelihood of filling a prescription for an opioid analgesic was significantly lower for recipients of services delivered by doctors of chiropractic compared with nonrecipients. The underlying cause of this correlation remains unknown, indicating the need for further investigation.
The underlying cause remains unknown???
Let me speculate, or even better, let me extrapolate by drawing an analogy:
Employees by a large Hamburger chain set out to study the association between utilization of Hamburger restaurant services and vegetarianism. The authors used a retrospective cohort design. The study population comprised New Hampshire residents aged 18-99 years, who had entered the premises of a Hamburger restaurant within 90 days for a primary purpose of eating. The authors excluded subjects with a diagnosis of cancer. They measured the likelihood of vegetarianism among recipients of services delivered by Hamburger restaurants compared with a control group of individuals not using meat-dispensing facilities. They also compared the cohorts with regard to the money spent in Hamburger restaurants.
The adjusted likelihood of being a vegetarian was 55% lower among the experimental group compared to controls. The average money spent per person in Hamburger restaurants were also significantly lower among the Hamburger group.
The authors concluded that among New Hampshire adults visiting Hamburger restaurants, the likelihood of vegetarianism was significantly lower for consumers frequenting Hamburger restaurants compared with those who failed to frequent such places. The underlying cause of this correlation remains unknown, indicating the need for further investigation.
The question whether spinal manipulative therapy (SMT) has any specific therapeutic effects is still open. This fact must irritate ardent chiropractors, and they therefore try everything to dispel our doubts. One way would be to demonstrate a dose-effect relationship between SMT and the clinical outcome. But, for several reasons, this is not an easy task.
This RCT was aimed at identifying the dose-response relationship between visits for SMT and chronic cervicogenic headache (CGH) outcomes; to evaluate the efficacy of SMT by comparison with a light massage control.
The study included 256 adults with chronic CGH. The primary outcome was days with CGH in the prior 4 weeks evaluated at the 12- and 24-week primary endpoints. Secondary outcomes included CGH days at remaining endpoints, pain intensity, disability, perceived improvement, medication use, and patient satisfaction. Participants were randomized to 4 different dose levels of chiropractic SMT: 0, 6, 12, or 18 sessions. They were treated 3 times per week for 6 weeks and received a focused light-massage control at sessions when SMT was not assigned. Linear dose effects and comparisons to the no-manipulation control group were evaluated at 6, 12, 24, 39, and 52 weeks.
A linear dose-response was observed for all follow-ups, a reduction of approximately 1 CGH day/4 weeks per additional 6 SMT visits (p<.05); a maximal effective dose could not be determined. CGH days/4 weeks were reduced from about 16 to 8 for the highest and most effective dose of 18 SMT visits. Mean differences in CGH days/4 weeks between 18 SMT visits and control were -3.3 (p=.004) and -2.9 (p=.017) at the primary endpoints, and similar in magnitude at the remaining endpoints (p<.05). Differences between other SMT doses and control were smaller in magnitude (p > .05). CGH intensity showed no important improvement nor differed by dose. Other secondary outcomes were generally supportive of the primary.
The authors concluded that there was a linear dose-response relationship between SMT visits and days with CGH. For the highest and most effective dose of 18 SMT visits, CGH days were reduced by half, and about 3 more days per month than for the light-massage control.
This trial would make sense, if the effectiveness of SMT for CGH had been a well-documented fact, and if the study had rigorously controlled for placebo-effects.
But guess what?
Neither of these conditions were met.
A recent review concluded that there are few published randomized controlled trials analyzing the effectiveness of spinal manipulation and/or mobilization for TTH, CeH, and M in the last decade. In addition, the methodological quality of these papers is typically low. Clearly, there is a need for high-quality randomized controlled trials assessing the effectiveness of these interventions in these headache disorders. And this is by no means the only article making such statements; similar reviews arrive at similar conclusions. In turn, this means that the effects observed after SMT are not necessarily specific effects due to SMT but could easily be due to placebo or other non-specific effects. In order to avoid confusion, one would need a credible placebo – one that closely mimics SMT – and make sure that patients were ‘blinded’. But ‘light massage’ clearly does not mimic SMT, and patients obviously were aware of which interventions they received.
So, an alternative – and I think at least as plausible – conclusion of the data provided by this new RCT is this:
Chiropractic SMT is associated with a powerful placebo response which, of course, obeys a dose-effect relationship. Thus these findings are in keeping with the notion that SMT is a placebo.
And why would the researchers – who stress that they have no conflicts of interest – mislead us by making this alternative interpretation of their findings not abundantly clear?
I fear, the reason might be simple: they also seem to mislead us about their conflicts of interest: they are mostly chiropractors with a long track record of publishing promotional papers masquerading as research. What, I ask myself, could be a stronger conflict of interest?
(Pity that a high-impact journal like SPINE did not spot these [not so little] flaws)
The chiropractor Oakley Smith had graduated under D D Palmer in 1899. Smith was a former Iowa medical student who also had investigated Andrew Still’s osteopathy in Kirksville, before going to Palmer in Davenport. Eventually, Smith came to reject the Palmer concept of vertebral subluxation and developed his own concept of “the connective tissue doctrine” or naprapathy. Today, naprapathy is a popular form of manual therapy, particularly in Scandinavia and the US.
But what exactly is naprapathy? This website explains it quite well: Naprapathy is defined as a system of specific examination, diagnostics, manual treatment and rehabilitation of pain and dysfunction in the neuromusculoskeletal system. The therapy is aimed at restoring function through treatment of the connective tissue, muscle- and neural tissues within or surrounding the spine and other joints. Naprapathic treatment consists of combinations of manual techniques for instance spinal manipulation and mobilization, neural mobilization and Naprapathic soft tissue techniques, in additional to the manual techniques Naprapaths uses different types of electrotherapy, such as ultrasound, radial shockwave therapy and TENS. The manual techniques are often combined with advice regarding physical activity and ergonomics as well as medical rehabilitation training in order to decrease pain and disability and increase work ability and quality of life. A Dr. of Naprapathy is specialized in the diagnosis of structural and functional neuromusculoskeletal disorders, treatment and rehabilitation of patients with problems of such origin as well as to differentiate pain of other origin.
DOCTOR OF NAPRAPATHY? I hear you shout.
Yes, in the US, the title exists: The National College of Naprapathic Medicine is chartered by the State of Illinois and recognized by the State Board of Higher Education to grant the degree, Doctor of Naprapathy (D.N.). Graduates of the College are eligible to take the Naprapathic Medicine examination for licensure in the State of Illinois. The D.N. Degree requires:
- 66 hours – Basic Sciences
- 64 hours – Naprapathic Sciences
- 60 hours – Clinical Internship
Things become even stranger when we ask, what does the evidence show?
I found all of three clinical trials on Medline.
A 2016 clinical trial was designed to compare the treatment effect on pain intensity, pain related disability and perceived recovery from a) naprapathic manual therapy (spinal manipulation, spinal mobilization, stretching and massage) to b) naprapathic manual therapy without spinal manipulation and to c) naprapathic manual therapy without stretching for male and female patients seeking care for back and/or neck pain.
Participants were recruited among patients, ages 18-65, seeking care at the educational clinic of Naprapathögskolan – the Scandinavian College of Naprapathic Manual Medicine in Stockholm. The patients (n = 1057) were randomized to one of three treatment arms a) manual therapy (i.e. spinal manipulation, spinal mobilization, stretching and massage), b) manual therapy excluding spinal manipulation and c) manual therapy excluding stretching. The primary outcomes were minimal clinically important improvement in pain intensity and pain related disability. Treatments were provided by naprapath students in the seventh semester of eight total semesters. Generalized estimating equations and logistic regression were used to examine the association between the treatments and the outcomes.
At 12 weeks follow-up, 64% had a minimal clinically important improvement in pain intensity and 42% in pain related disability. The corresponding chances to be improved at the 52 weeks follow-up were 58% and 40% respectively. No systematic differences in effect when excluding spinal manipulation and stretching respectively from the treatment were found over 1 year follow-up, concerning minimal clinically important improvement in pain intensity (p = 0.41) and pain related disability (p = 0.85) and perceived recovery (p = 0.98). Neither were there disparities in effect when male and female patients were analyzed separately.
The authors concluded that the effect of manual therapy for male and female patients seeking care for neck and/or back pain at an educational clinic is similar regardless if spinal manipulation or if stretching is excluded from the treatment option.
Even though this study is touted as showing that naprapathy works by advocates, in all honesty, it tells us as good as nothing about the effect of naprapathy. The data are completely consistent with the interpretation that all of the outcomes were to the natural history of the conditions, regression towards the mean, placebo, etc. and entirely unrelated to any specific effects of naprapathy.
A 2010 study by the same group was to compare the long-term effects (up to one year) of naprapathic manual therapy and evidence-based advice on staying active regarding non-specific back and/or neck pain.
Subjects with non-specific pain/disability in the back and/or neck lasting for at least two weeks (n = 409), recruited at public companies in Sweden, were included in this pragmatic randomized controlled trial. The two interventions compared were naprapathic manual therapy such as spinal manipulation/mobilization, massage and stretching, (Index Group), and advice to stay active and on how to cope with pain, provided by a physician (Control Group). Pain intensity, disability and health status were measured by questionnaires.
89% completed the 26-week follow-up and 85% the 52-week follow-up. A higher proportion in the Index Group had a clinically important decrease in pain (risk difference (RD) = 21%, 95% CI: 10-30) and disability (RD = 11%, 95% CI: 4-22) at 26-week, as well as at 52-week follow-ups (pain: RD = 17%, 95% CI: 7-27 and disability: RD = 17%, 95% CI: 5-28). The differences between the groups in pain and disability considered over one year were statistically significant favoring naprapathy (p < or = 0.005). There were also significant differences in improvement in bodily pain and social function (subscales of SF-36 health status) favoring the Index Group.
The authors concluded that combined manual therapy, like naprapathy, is effective in the short and in the long term, and might be considered for patients with non-specific back and/or neck pain.
This study is hardly impressive either. The results are consistent with the interpretation that the extra attention and care given to the index group was the cause of the observed outcomes, unrelated to ant specific effects of naprapathy.
The last study was published in 2017 again by the same group. It was designed to compare naprapathic manual therapy with evidence-based care for back or neck pain regarding pain, disability, and perceived recovery.
Four hundred and nine patients with pain and disability in the back or neck lasting for at least 2 weeks, recruited at 2 large public companies in Sweden in 2005, were included in this randomized controlled trial. The 2 interventions were naprapathy, including spinal manipulation/mobilization, massage, and stretching (Index Group) and support and advice to stay active and how to cope with pain, according to the best scientific evidence available, provided by a physician (Control Group). Pain, disability, and perceived recovery were measured by questionnaires at baseline and after 3, 7, and 12 weeks.
At 7-week and 12-week follow-ups, statistically significant differences between the groups were found in all outcomes favoring the Index Group. At 12-week follow-up, a higher proportion in the naprapathy group had improved regarding pain [risk difference (RD)=27%, 95% confidence interval (CI): 17-37], disability (RD=18%, 95% CI: 7-28), and perceived recovery (RD=44%, 95% CI: 35-53). Separate analysis of neck pain and back pain patients showed similar results.
The authors thought that this trial suggests that combined manual therapy, like naprapathy, might be an alternative to consider for back and neck pain patients.
As the study suffers from the same limitations as the one above (in fact, it might be a different analysis of the same trial), they might be mistaken. I see no good reason to assume that any of the three studies provide good evidence for the effectiveness of naprapathy.
So, what should we conclude from all this?
If you ask me, naprapathy is something between chiropractic (without some of the woo) and physiotherapy (without its expertise). There is no good evidence that it works. Crucially, there is no evidence that it is superior to other therapeutic options.
I was going to finish on a positive note stating that ‘at least the ‘naprapathologists’ (I refuse to even consider the title of ‘doctor of naprapathy’) do not claim to treat conditions other than musculoskeletal problems’. But then I found this advertisement of a ‘naprapathologist’ on Twitter:
And now, I am going to finish by stating that A LOT OF NAPRAPATHY LOOKS VERY MUCH LIKE QUACKERY TO ME.
Sipjeondaebo-tang is an East Asian herbal supplement containing Angelica root (Angelicae Gigantis Radix), the rhizome of Cnidium officinale Makino (Cnidii Rhizoma), Radix Paeoniae, Rehmannia glutinosa root (Rehmanniae Radix Preparata), Ginseng root (Ginseng Radix Alba), Atractylodes lancea root (Atractylodis Rhizoma Alba), the dried sclerotia of Poria cocos (Poria cocos Sclerotium), Licorice root (Glycyrrhizae Radix), Astragalus root (Astragali Radix), and the dried bark of Cinnamomum verum (Cinnamomi Cortex).
But does this herbal mixture actually work? Korean researchers wanted to find out.
The purpose of their study was to examine the feasibility of Sipjeondaebo-tang (Juzen-taiho-to, Shi-Quan-Da-Bu-Tang) for cancer-related anorexia. A total of 32 participants with cancer anorexia were randomized to either Sipjeondaebo-tang group or placebo group. Participants were given 3 g of Sipjeondaebo-tang or placebo 3 times a day for 4 weeks. The primary outcome was a change in the Anorexia/Cachexia Subscale of Functional Assessment of Anorexia/Cachexia Therapy (FAACT). The secondary outcomes included Visual Analogue Scale (VAS) of anorexia, FAACT scale, and laboratory tests.
The results showed that anorexia and quality of life measured by FAACT and VAS were improved after 4 weeks of Sipjeondaebo-tang treatment. However, there was no significant difference between changes of Sipjeondaebo-tang group and placebo group.
From this, the authors of the study concluded that sipjeondaebo-tang appears to have potential benefit for anorexia management in patients with cancer. Further large-scale studies are needed to ensure the efficacy.
Well, isn’t this just great? Faced with a squarely negative result, one simply ignores it and draws a positive conclusion!
As we all know – and as trialists certainly must know – controlled trials are designed to compare the outcomes of two groups. Changes within one of the groups can be caused by several factors unrelated to the therapy and are therefore largely irrelevant. This means that “no significant difference between changes of Sipjeondaebo-tang group and placebo group” indicates that the herbal mixture had no effect. In turn this means that a conclusion stating that “sipjeondaebo-tang appears to have potential benefit for anorexia” is just fraudulent.
This level of scientific misconduct is remarkable, even for the notoriously poor Evid Based Complement Alternat Med.
I strongly suggest that:
- The journal is de-listed from Medline because similarly misleading nonsense has been coming out of this rag for some time.
- The paper is withdrawn because it can only mislead vulnerable patients.
Cranio-sacral therapy is firstly implausible, and secondly it lacks evidence of effectiveness (see for instance here, here, here and here). Yet, some researchers are nevertheless not deterred to test it in clinical trials. While this fact alone might be seen as embarrassing, the study below is a particular and personal embarrassment to me, in fact, I am shocked by it and write these lines with considerable regret.
Why? Bear with me, I will explain later.
The purpose of this trial was to evaluate the effectiveness of osteopathic manipulative treatment and osteopathy in the cranial field in temporomandibular disorders. Forty female subjects with temporomandibular disorders lasting at least three months were included. At enrollment, subjects were randomly assigned into two groups: (1) osteopathic manipulative treatment group (n=20) and (2) osteopathy in the cranial field [craniosacral therapy for you and me] group (n=20). Examinations were performed at baseline (E0) and at the end of the last treatment (E1), and consisted of subjective pain intensity with the Visual Analog Scale, Helkimo Index and SF-36 Health Survey. Subjects had five treatments, once a week. 36 subjects completed the study.
Patients in both groups showed significant reduction in Visual Analog Scale score (osteopathic manipulative treatment group: p = 0.001; osteopathy in the cranial field group: p< 0.001), Helkimo Index (osteopathic manipulative treatment group: p = 0.02; osteopathy in the cranial field group: p = 0.003) and a significant improvement in the SF-36 Health Survey – subscale “Bodily Pain” (osteopathic manipulative treatment group: p = 0.04; osteopathy in the cranial field group: p = 0.007) after five treatments (E1). All subjects (n = 36) also showed significant improvements in the above named parameters after five treatments (E1): Visual Analog Scale score (p< 0.001), Helkimo Index (p< 0.001), SF-36 Health Survey – subscale “Bodily Pain” (p = 0.001). The differences between the two groups were not statistically significant for any of the three endpoints.
The authors concluded that both therapeutic modalities had similar clinical results. The findings of this pilot trial support the use of osteopathic manipulative treatment and osteopathy in the cranial field as an effective treatment modality in patients with temporomandibular disorders. The positive results in both treatment groups should encourage further research on osteopathic manipulative treatment and osteopathy in the cranial field and support the importance of an interdisciplinary collaboration in patients with temporomandibular disorders. Implications for rehabilitation Temporomandibular disorders are the second most prevalent musculoskeletal condition with a negative impact on physical and psychological factors. There are a variety of options to treat temporomandibular disorders. This pilot study demonstrates the reduction of pain, the improvement of temporomandibular joint dysfunction and the positive impact on quality of life after osteopathic manipulative treatment and osteopathy in the cranial field. Our findings support the use of osteopathic manipulative treatment and osteopathy in the cranial field and should encourage further research on osteopathic manipulative treatment and osteopathy in the cranial field in patients with temporomandibular disorders. Rehabilitation experts should consider osteopathic manipulative treatment and osteopathy in the cranial field as a beneficial treatment option for temporomandibular disorders.
This study has so many flaws that I don’t know where to begin. Here are some of the more obvious ones:
- There is, as already mentioned, no rationale for this study. I can see no reason why craniosacral therapy should work for the condition. Without such a rationale, the study should never even have been conceived.
- Technically, this RCTs an equivalence study comparing one therapy against another. As such it needs to be much larger to generate a meaningful result and it also would require a different statistical approach.
- The authors mislabelled their trial a ‘pilot study’. However, a pilot study “is a preliminary small-scale study that researchers conduct in order to help them decide how best to conduct a large-scale research project. Using a pilot study, a researcher can identify or refine a research question, figure out what methods are best for pursuing it, and estimate how much time and resources will be necessary to complete the larger version, among other things.” It is not normally a study suited for evaluating the effectiveness of a therapy.
- Any trial that compares one therapy of unknown effectiveness to another of unknown effectiveness is a complete and utter nonsense. Equivalent studies can only ever make sense, if one of the two treatments is of proven effectiveness – think of it as a mathematical equation: one equation with two unknowns is unsolvable.
- Controlled studies such as RCTs are for comparing the outcomes of two or more groups, and only between-group differences are meaningful results of such trials.
- The ‘positive results’ which the authors mention in their conclusions are meaningless because they are based on such within-group changes and nobody can know what caused them: the natural history of the condition, regression towards the mean, placebo-effects, or other non-specific effects – take your pick.
- The conclusions are a bonanza of nonsensical platitudes and misleading claims which do not follow from the data.
As regular readers of this blog will doubtlessly have noticed, I have seen plenty of similarly flawed pseudo-research before – so, why does this paper upset me so much? The reason is personal, I am afraid: even though I do not know any of the authors in person, I know their institution more than well. The study comes from the Department of Physical Medicine and Rehabilitation, Medical University of Vienna, Austria. I was head of this department before I left in 1993 to take up the Exeter post. And I had hoped that, even after 25 years, a bit of the spirit, attitude, knowhow, critical thinking and scientific rigor – all of which I tried so hard to implant in my Viennese department at the time – would have survived.
Perhaps I was wrong.
Some say that Chinese herbal medicine offers a solution.
This Chinese multi-centre RCT included 588 mothers considering breastfeeding. The intervention group received the Chinese herbal mixture Zengru Gao, while the control group received no therapy. The primary outcomes were the percentages of fully and partially breastfeeding mothers, and a secondary outcome was baby’s daily formula intake.
At day 3 and 7 after delivery, significant differences were found in favour of Zengru Gao group on the percentage of full/ partial breastfeeding. At day 7, the percentage of full/ partial breastfeeding of the active group increased to 71.48%/20.70% versus 58.67%/30.26% in the control group, the differences remained significant. No statistically significant differences were detected on primary measures at day. While intake of formula differed between groups at day 1 and 3, this difference did not achieve statistical significance, but this difference was apparent by day 7.
The authors concluded that the Chinese Herbal medicine Zengru Gao enhanced breastfeeding success during one week postpartum. The approach is acceptable to participants and merits further evaluation.
To the naïve observer, this study might look rigorous, but it is a seriously flawed RCT. Here are just some of its most obvious limitations:
- All we get in the methods section is this explanation: Participants were randomly allocated to the blank control group or the intervention group: Zengru Gao, orally, 30 g a time and 3 times a day. This seems to indicate that the control group got no treatment at all which means there was no blinding nor placebo control. The authors even comment on this point in the discussion section of their paper stating that because we included new mothers who received no treatment as a control group, we were able to prove that the improvement in breastfeeding was not due to the placebo effect. However, this is a totally nonsensical argument.
- The experimental treatment is not reproducible. The authors state: Zengru Gao, a Chinese herbal formula, which is composed of 8 herbs: Semen Vaccariae, Medulla Tetrapanacis, Radix Rehmanniae Praeparata, Radix Angelicae Sinensis, Radix Paeoniae Alba,Rhizoma Chuanxiong, Herba Leonuri, Radix Trichosanthis. This is not enough information to replicate the study outside China where the mixture is not commercially available.
- The primary outcome was the percentage of fully, and partially breastfeeding mothers. Breastfeeding was defined as mother’s milk given by direct breast feeding. Full breastfeeding meant that no other types of milk or solids were given. Partially breastfeeding meant that sustained latch with deep rhythmic sucking through the length of the feed, with some pause, on either/ or both breasts. We are not being told how the endpoint was quantified. Presumably women kept diaries. We cannot guess how accurate this process was.
- As far as I can see, there was no correction for multiple testing for statistical significance. This means that some or all of the significant results might be false-positive.
- There is insufficient data to show that the herbal mixture is safe for the mothers and the babies. At the very minimum, the researchers should have measured essential safety parameters. This omission is a gross violation of research ethics.
- Towards the end of the paper, we find the following statement: The authors would like to thank the Research and Development Department of Zhangzhou Pien Tze Huang Pharmaceutical co., Ltd. … The authors declare that they have no competing interests. And the 1st and 3rd authors are “affiliated with” Guangzhou Hipower Pharmaceutical Technology Co., Ltd, Guangzhou, China, i. e. work for the manufacturer of the mixture. This does clearly not make any sense whatsoever.
I have seen too many flawed studies of alternative medicine to be shocked or even surprised by this level of incompetence and nonsense. Yet, I still find it lamentable. But, in my view, the worst is that supposedly peer-reviewed journals such as ‘BMC Complement Altern Med’ publish such overt rubbish.
It would be easy to shrug one’s shoulder and bin the paper. But the effect of such fatally flawed research is too serious for that. In our recent book MORE HARM THAN GOOD? THE MORAL MAZE OF COMPLEMENTARY AND ALTERNATIVE MEDICINE, we discuss that such flawed science amounts to a violation of medical ethics: CAM journals allocate peer review tasks to a narrow range of CAM enthusiasts who often have been chosen by the authors of the article in question. The raison d’être of CAM journals and CAM researchers is inextricably tied to a belief in CAM, resulting in a self-referential situation which is permissive to the acceptance of weak or ﬂawed reports of clinical effectiveness… Defective research—whether at the design, execution, analysis, or reporting stage—corrupts the repository of reliable medical knowledge. Ultimately, this leads to suboptimal and erroneous treatment decisions…
The authors of this systematic review aimed to summarize the evidence of clinical trials on cupping for athletes. Randomized controlled trials on cupping therapy with no restriction regarding the technique, or co-interventions, were included, if they measured the effects of cupping compared with any other intervention on health and performance outcomes in professionals, semi-professionals, and leisure athletes. Data extraction and risk of bias assessment using the Cochrane Risk of Bias Tool were conducted independently by two pairs of reviewers.
Eleven trials with n = 498 participants from China, the United States, Greece, Iran, and the United Arab Emirates were included, reporting effects on different populations, including soccer, football, and handball players, swimmers, gymnasts, and track and field athletes of both amateur and professional nature. Cupping was applied between 1 and 20 times, in daily or weekly intervals, alone or in combination with, for example, acupuncture. Outcomes varied greatly from symptom intensity, recovery measures, functional measures, serum markers, and experimental outcomes. Cupping was reported as beneficial for perceptions of pain and disability, increased range of motion, and reductions in creatine kinase when compared to mostly untreated control groups. The majority of trials had an unclear or high risk of bias. None of the studies reported safety.
The authors concluded that no explicit recommendation for or against the use of cupping for athletes can be made. More studies are necessary for conclusive judgment on the efficacy and safety of cupping in athletes.
Considering the authors’ stated aim, this conclusion seems odd. Surely, they should have concluded that THERE IS NO CONVINCING EVIDENCE FOR THE USE OF CUPPING IN ATHLETES. But this sounds rather negative, and the JCAM does not seem to tolerate negative conclusions, as discussed repeatedly on this blog.
The discussion section of this paper is bar of any noticeable critical input (for those who don’t know: the aim of any systematic review must be to CRITICALLY EVALUATE THE PRIMARY DATA). The authors even go as far as stating that the trials reported in this systematic review found beneficial effects of cupping in athletes when compared to no intervention. I find this surprising and bordering on scientific misconduct. The RCTs were mostly not on cupping but on cupping in combination with some other treatments. More importantly, they were of such deplorable quality that they allow no conclusions about effectiveness. Lastly, they mostly failed to report on adverse effects which, as I have often stated, is a violation of research ethics.
In essence, all this paper proves is that, if you have rubbish trials, you can produce a rubbish review and publish it in a rubbish journal.
If you ask me, the field of alternative medicine is plagued with surveys; too many are published and most are complete, meaningless rubbish which serve merely the purpose of being misinterpreted as a means of popularising bogus treatments. Yet, every now and then, a decent and informative article appears – like this survey from Canada.
It yields a number of fascinating findings:
- More than three-quarters of Canadians (79%) had used at least one from of CAM sometime in their lives in 2016 (74% in 2006 and 73% in 1997). British Columbians were most likely to have used an alternative therapy during their lifetime (89%), followed by Albertans (84%) and Ontarians (81%).
- More than half (56%) of Canadians had used at least one CAM therapy in the year prior to the 2016 survey, compared to 54% in 2006 and 50% in 1997.
- In 2016, massage was the most common type of therapy that Canadians used over their lifetime with 44 percent having tried it, followed by chiropractic care (42%), yoga (27%), relaxation techniques (25%), and acupuncture (22%).
- The most rapidly expanding therapies over the past two decades were massage, yoga, acupuncture, chiropractic care, osteopathy, and naturopathy.
- High dose/mega vitamins, herbal therapies, and folk remedies were in declining use over that same time period.
- The most likely users of CAM over the past 12 months in 2016 were from the 35- to 44-year-old age group (61%). The use of CAM diminished with age, and generally rose with both income and education. These trends are similar to those observed in 2006 and 1997.
- The majority of people choosing to use CAM in the 12 months preceding the 2016 survey did so for “wellness”.
- Canadians spent an estimated $8.8 billion on CAM in the last 12 months ($8.0 billion in 2005/06 and $6.3 billion in 1996/97.
- Of the $8.8 billion spent in 2016, more than $6.5 billion was spent on providers of CAM, while another $2.3 billion was spent on herbs, vitamins, special diet programs, books, classes, and equipment.
- The majority of Canadians believe that CAM should be paid for privately and not by provincial health.
The strengths of this survey are that it is methodologically rigorous, and that it provides longitudinal data (this is in sharp contrast to the plethora of CAM surveys published recently). Many of its findings confirm what has already been known. Yet some results are new and noteworthy.
To many readers of this blog, the high CAM-usage will be disturbing. However, I am mildly encouraged by the results of this survey.
- Firstly, the choice of CAM by Canadians seems rather more reasonable than that by other nations. Canadians seem to avoid the more ridiculous types of CAM, such as homeopathy or para-normal healing.
- Secondly, many Canadians seem to view CAM not as medicine, but as a sort of luxurious pampering that they use to relax and feel well. Consequently, most are not pushing to get it reimbursed which I find more sensible than consumers’ attitudes in many other countries.
As I have stated repeatedly, I am constantly on the look-out for positive news about alternative medicine. Usually, I find plenty – but when I scrutinise it, it tends to crumble in the type of misleading report that I often write about on this blog. Truly good research in alternative medicine is hard to find, and results that are based on rigorous science and show a positive finding are a bit like gold-dust.
But hold on, today I have something!
This systematic review was aimed at determining whether physical exercise is effective in improving cognitive function in the over 50s. The authors evaluated all randomised controlled trials of physical exercise interventions in community-dwelling adults older than 50 years with an outcome measure of cognitive function.
39 studies were included in the systematic review. Analysis of 333 dependent effect sizes from 36 studies showed that physical exercise improved cognitive function. Interventions of aerobic exercise, resistance training, multicomponent training and tai chi, all had significant point estimates. When exercise prescription was examined, a duration of 45–60 min per session and at least moderate intensity, were associated with benefits to cognition. The results of the meta-analysis were consistent and independent of the cognitive domain tested or the cognitive status of the participants.
The authors concluded that physical exercise improved cognitive function in the over 50s, regardless of the cognitive status of participants. To improve cognitive function, this meta-analysis provides clinicians with evidence to recommend that patients obtain both aerobic and resistance exercise of at least moderate intensity on as many days of the week as feasible, in line with current exercise guidelines.
But this is not alternative medicine, I hear you say.
You are right, mostly, it isn’t. There were a few RCTs of tai chi and yoga, but the majority was of conventional exercise. Moreover, most of these ‘alternative’ RCTs were less convincing than the conventional RCTs; here is one of the former category:
Community-dwelling older adults (N = 118; mean age = 62.0) were randomized to one of two groups: a Hatha yoga intervention or a stretching-strengthening control. Both groups participated in hour-long exercise classes 3×/week over the 8-week study period. All participants completed established tests of executive function including the task switching paradigm, n-back and running memory span at baseline and follow-up. Analysis of covariances showed significantly shorter reaction times on the mixed and repeat task switching trials (partial η(2) = .04, p < .05) for the Hatha yoga group. Higher accuracy was recorded on the single trials (partial η(2) = .05, p < .05), the 2-back condition of the n-back (partial η(2) = .08, p < .001), and partial recall scores (partial η(2) = .06, p < .01) of running span task.
I just wanted to be generous and felt the need to report a positive result. I guess, this just shows how devoid of rigorous research generating a positive finding alternative medicine really is.
Of course, there are many readers of this blog who are convinced that their pet therapy is supported by excellent evidence. For them, I have this challenge: if you think you have good evidence for an alternative therapy, show it to me (send it to me via the ‘contact’ option of this blog or post the link as a comment below). Please note that any evidence I would consider analysing in some detail (writing a full blog post about it) would need to be recent, peer-reviewed and rigorous.