This paper notes that, according to the World Naturopathic Federation (WNF), the naturopathic profession is based on two fundamental philosophies of medicine (vitalism and holism) and seven principles of practice (healing power of nature; treat the whole person; treat the cause; first, do no harm; doctor as teacher; health promotion and disease prevention; and wellness). The philosophy, theory, and principles are translated to clinical practice through a range of therapeutic modalities. The WNF has identified seven core modalities: (1) clinical nutrition and diet modification/counselling; (2) applied nutrition (use of dietary supplements, traditional medicines, and natural health care products); (3) herbal medicine; (4) lifestyle counselling; (5) hydrotherapy; (6) homeopathy, including complex homeopathy; and (7) physical modalities (based on the treatment modalities taught and allowed in each jurisdiction, including yoga, naturopathic manipulation, and muscle release techniques).
The ‘scoping’ review was to summarize the current state of the research evidence for whole-system, multi-modality naturopathic medicine. Studies were included, if they met the following criteria:
- Controlled clinical trials, longitudinal cohort studies, observational trials, or case series involving five or more cases presented in any language
- Human studies
- Multi-modality treatment administered by a naturopath (naturopathic clinician, naturopathic physician) as an intervention
- Non-English language studies in which an English title and abstract provided sufficient information to determine effectiveness
- Case series in which five or more individual cases were pooled and authors provided a summative discussion of the cases in the context of naturopathic medicine
- All human research evaluating the effectiveness of naturopathic medicine, where two or more naturopathic modalities are delivered by naturopathic clinicians, were included in the review.
- Case studies of five or more cases were included.
Thirty-three published studies with a total of 9859 patients met inclusion criteria (11 US; 4 Canadian; 6 German; 7 Indian; 3 Australian; 1 UK; and 1 Japanese) across a range of mainly chronic clinical conditions. A majority of the included studies were observational cohort studies (12 prospective and 8 retrospective), with 11 clinical trials and 2 case series. The studies predominantly showed evidence for the efficacy of naturopathic medicine for the conditions and settings in which they were based. Overall, these studies show naturopathic treatment results in a clinically significant benefit for treatment of hypertension, reduction in metabolic syndrome parameters, and improved cardiac outcomes post-surgery.
The authors concluded that to date, research in whole-system, multi-modality naturopathic medicine shows that it is effective for treating cardiovascular disease, musculoskeletal pain, type 2 diabetes, polycystic ovary syndrome, depression, anxiety, and a range of complex chronic conditions. Overall, these studies show naturopathic treatment results in a clinically significant benefit for treatment of hypertension, reduction in metabolic syndrome parameters, and improved cardiac outcomes post-surgery.
Where to start?
There are many issues here to choose from:
- The definition of naturopathy used in this review may be the one of the WHF, but it has little resemblance to the one used elsewhere. German naturopathic doctors, for instance, would not consider homeopathy to be a naturopathic treatment. They would also not, like the WNF does, subscribe to the long-obsolete humoral theory of disease. The only German professional organisation that is a member of the WNF is thus not one of naturopathic doctors but one of Heilpraktiker (the notorious German lay-practitioner created by the Nazis during the Third Reich).
- A review that includes observational studies and even case series, while drawing far-reaching conclusions on therapeutic effectiveness is, in my view, little more than embarrassing pseudo-science. Such studies are unable to differentiate between specific and non-specific therapeutic effects and therefore can tell us nothing about the effectiveness of a treatment.
- A review on a subject such as naturopathy (an approach which, after all, originated in Europe) that excludes studies not published in English (and without an English abstract providing sufficient information to determine effectiveness) is likely to be incomplete.
- The authors call their review a ‘scoping review’; they nevertheless draw conclusions not about the scope but the effectiveness of naturopathy.
- Many of the studies included in this review do, in fact, not comply with the inclusion criteria set by the review-authors.
- The review does not assess or even comment on the risks of naturopathic treatments.
- Several of the included studies are not investigations of naturopathy but of approaches that squarely fall under the umbrella of integrative or alternative medicine.
- Of the 33 studies included, only 5 were RCTs, and none of these was free of major limitations.
- None of the RCTs have been independently replicated.
- There is a remarkable absence of negative trials suggestion a strong influence of bias.
- The review lacks any trace of critical thinking.
- The authors are affiliated to institutions of naturopathy but declare no conflicts of interest.
- No funding source was named but it seems that it was supported by the WNF; their primary goal is to promote and advance the naturopathic profession.
- The review appeared in the notorious Journal of Complementary and Alternative Medicine.
Prof Dwyer, the founding president of the Australian ‘Friends of Science in Medicine’, said the study damaged Southern Cross University’s reputation. “At the heart of this is the credibility of Southern Cross University,” he said. “There’s been a stand-off between SCU and the rest of the scientific community in Australia for a number of years and there have been challenges to whether they are really upholding the highest standards of evidence-based medicine.” Professor Dwyer also raised questions about the university’s credibility late last year when it accepted a $10 million donation from vitamin company Blackmore’s to establish a National Centre for Naturopathic Medicine.
My conclusion of naturopathy, as defined by the WNF, is that it is an obsolete form of quackery steeped in concepts of vitalism that should be abandoned sooner rather than later. And my conclusion about the new review agrees with Prof Dwyer’s judgement: it is an embarrassment to all concerned.
A new update of the current Cochrane review assessed the benefits and harms of spinal manipulative therapy (SMT) for the treatment of chronic low back pain. The authors included all randomised controlled trials (RCTs) examining the effect of spinal manipulation or mobilisation in adults (≥18 years) with chronic low back pain with or without referred pain. Studies that exclusively examined sciatica were excluded.
The effect of SMT was compared with recommended therapies, non-recommended therapies, sham (placebo) SMT, and SMT as an adjuvant therapy. Main outcomes were pain and back specific functional status, examined as mean differences and standardised mean differences (SMD), respectively. Outcomes were examined at 1, 6, and 12 months.
Forty-seven RCTs including a total of 9211 participants were identified. Most trials compared SMT with recommended therapies. In 16 RCTs, the therapists were chiropractors, in 14 they were physiotherapists, and in 5 they were osteopaths. They used high velocity manipulations in 18 RCTs, low velocity manipulations in 12 studies and a combination of the two in 20 trials.
Moderate quality evidence suggested that SMT has similar effects to other recommended therapies for short term pain relief and a small, clinically better improvement in function. High quality evidence suggested that, compared with non-recommended therapies, SMT results in small, not clinically better effects for short term pain relief and small to moderate clinically better improvement in function.
In general, these results were similar for the intermediate and long term outcomes as were the effects of SMT as an adjuvant therapy.
Low quality evidence suggested that SMT does not result in a statistically better effect than sham SMT at one month. Additionally, very low quality evidence suggested that SMT does not result in a statistically better effect than sham SMT at six and 12 months. Low quality evidence suggested that SMT results in a moderate to strong statistically significant and clinically better effect than sham SMT at one month. Additionally, very low quality evidence suggested that SMT does not result in a statistically significant better effect than sham SMT at six and 12 months.
(Mean difference in reduction of pain at 1, 3, 6, and 12 months (0-100; 0=no pain, 100 maximum pain) for spinal manipulative therapy (SMT) versus recommended therapies in review of the effects of SMT for chronic low back pain. Pooled mean differences calculated by DerSimonian-Laird random effects model.)
About half of the studies examined adverse and serious adverse events, but in most of these it was unclear how and whether these events were registered systematically. Most of the observed adverse events were musculoskeletal related, transient in nature, and of mild to moderate severity. One study with a low risk of selection bias and powered to examine risk (n=183) found no increased risk of an adverse event or duration of the event compared with sham SMT. In one study, the Data Safety Monitoring Board judged one serious adverse event to be possibly related to SMT.
The authors concluded that SMT produces similar effects to recommended therapies for chronic low back pain, whereas SMT seems to be better than non-recommended interventions for improvement in function in the short term. Clinicians should inform their patients of the potential risks of adverse events associated with SMT.
This paper is currently being celebrated (mostly) by chiropractors who think that it vindicates their treatments as being both effective and safe. However, I am not sure that this is entirely true. Here are a few reasons for my scepticism:
- SMT is as good as other recommended treatments for back problems – this may be so but, as no good treatment for back pain has yet been found, this really means is that SMT is as BAD as other recommended therapies.
- If we have a handful of equally good/bad treatments, it stand to reason that we must use other criteria to identify the one that is best suited – criteria like safety and cost. If we do that, it becomes very clear that SMT cannot be named as the treatment of choice.
- Less than half the RCTs reported adverse effects. This means that these studies were violating ethical standards of publication. I do not see how we can trust such deeply flawed trials.
- Any adverse effects of SMT were minor, restricted to the short term and mainly centred on musculoskeletal effects such as soreness and stiffness – this is how some naïve chiro-promoters already comment on the findings of this review. In view of the fact that more than half the studies ‘forgot’ to report adverse events and that two serious adverse events did occur, this is a misleading and potentially dangerous statement and a good example how, in the world of chiropractic, research is often mistaken for marketing.
- Less than half of the studies (45% (n=21/47)) used both an adequate sequence generation and an adequate allocation procedure.
- Only 5 studies (10% (n=5/47)) attempted to blind patients to the assigned intervention by providing a sham treatment, while in one study it was unclear.
- Only about half of the studies (57% (n=27/47)) provided an adequate overview of withdrawals or drop-outs and kept these to a minimum.
- Crucially, this review produced no good evidence to show that SMT has effects beyond placebo. This means the modest effects emerging from some trials can be explained by being due to placebo.
- The lead author of this review (SMR), a chiropractor, does not seem to be free of important conflicts of interest: SMR received personal grants from the European Chiropractors’ Union (ECU), the European Centre for Chiropractic Research Excellence (ECCRE), the Belgian Chiropractic Association (BVC) and the Netherlands Chiropractic Association (NCA) for his position at the Vrije Universiteit Amsterdam. He also received funding for a research project on chiropractic care for the elderly from the European Centre for Chiropractic Research and Excellence (ECCRE).
- The second author (AdeZ) who also is a chiropractor received a grant from the European Chiropractors’ Union (ECU), for an independent study on the effects of SMT.
After carefully considering the new review, my conclusion is the same as stated often before: SMT is not supported by convincing evidence for back (or other) problems and does not qualify as the treatment of choice.
“Most of the supplement market is bogus,” Paul Clayton*, a nutritional scientist, told the Observer. “It’s not a good model when you have businesses selling products they don’t understand and cannot be proven to be effective in clinical trials. It has encouraged the development of a lot of products that have no other value than placebo – not to knock placebo, but I want more than hype and hope.” So, Dr Clayton took a job advising Lyma, a product which is currently being promoted as “the world’s first super supplement” at £199 for a one-month’s supply.
Lyma is a dietary supplement that contains a multitude of ingredients all of which are well known and available in many other supplements costing only a fraction of Lyma. The ingredients include:
- vitamin D3.
Apparently, these ingredients are manufactured in special (and patented) ways to optimise their bioavailabity. According to the website, the ingredients of LYMA have all been clinically trialled with proven efficacy at levels provided within the LYMA supplement… Unless the ingredient has been clinically trialled, and peer reviewed there may be limited (if any) benefit to the body. LYMA’s revolutionary formulation is the most advanced and proven super supplement in the world, bringing together eight outstanding ingredients – seven of which are patented – to support health, wellbeing and beauty. Each ingredient has been selected for its efficacy, purity, quality, bioavailability, stability and ultimately, on the results of clinical studies.
The therapeutic claims made for the product are numerous:
- it will improve your hair, skin and nails (80% improvement in skin smoothness, 30% increase in skin moisture, 17% increase in skin elasticity, 12% reduction in wrinkle depth, 47% increase in hair strength & 35% decrease in hair loss)
- it will support energy levels in both the body and the brain (increase in brain membrane turnover by 26% and increase brain energy by 14%),
- it will improve cognitive function,
- it will enhance endurance (cardiorespiratory endurance increased by 13% compared to a placebo),
- it will improve quality of life,
- it will improve sleep (reducing insomnia by 70%),
- it will improve immunity,
- it will reduce inflammation,
- it will improve your memory,
- it will improve osteoporosis (reduce risk of osteoporosis by 37%).
These claims are backed up by 197 clinical trials, we are being told.
If true, this would be truly sensational – but is it true?
I asked the Lyma firm for the 197 original studies, and they very kindly sent me dozens papers which all referred to the single ingredients listed above. I emailed again and asked whether there are any studies of Lyma with all its ingredients in one supplement. Then I was told that they are ‘looking into a trial on the final Lyma formula‘.
I take this to mean that not a single trial of Lyma has been conducted. In this case, how do we be sure the mixture works? How can we know that the 197 studies have not been cherry-picked? How can we be sure that there are no interactions between the active constituents?
The response from Lyma quoted the above-mentioned Dr Paul Clayton stating this: “In regard to LYMA, clinical trials at this stage are not necessary. The whole point of LYMA is that each ingredient has already been extensively trialled, and validated. They have selected the best of the best ingredients, and amalgamated them; to enable consumers to take them all in a convenient format. You can quite easily go out and purchase all the ingredients separately. They aren’t easy to find, and it would mean swallowing up to 12 tablets and capsules a day; but the choice is always yours.”
It’s kind, to leave the choice to us, rather than forcing us to spend £199 each month on the world’s first super-supplement. Very kind indeed!
Having the choice, I might think again.
I might even assemble the world’s maximally evidence-based, extra super-supplement myself, one that is supported by many more than 197 peer-reviewed papers. To not directly compete with Lyma, I could use entirely different ingredients. Perhaps I should take the following five:
- Vitamin C (it has over 61 000 Medline listed articles to its name),
- Vitanin E (it has over 42 000 Medline listed articles to its name),
- Collagen (it has over 210 000 Medline listed articles to its name),
- Coffee (it has over 14 000 Medline listed articles to its name),
- Aloe vera (it has over 3 000 Medline listed articles to its name).
I could then claim that my extra super-supplement is supported by some 300 000 scientific articles plus 1 000 clinical studies (I am confident I could cherry-pick 1 000 positive trials from the 300 000 papers). Consequently, I would not just charge £199 but £999 for a month’s supply.
But this would be wrong, misleading, even bogus!!!, I hear you object.
On the one hand, I agree.
On the other hand, as Paul Clayton rightly pointed out: Most of the supplement market is bogus.
*If my memory serves me right, I met Paul many years ago when he was a consultant for Boots (if my memory fails me, I might need to order some Lyma).
A recent blog-post pointed out that the usefulness of yoga in primary care is doubtful. Now we have new data to shed some light on this issue.
The new paper reports a ‘prospective, longitudinal, quasi-experimental study‘. Yoga group (n= 49) underwent 24-weeks program of one-hour yoga sessions. The control group had no yoga.
Participation was voluntary and the enrolment strategy was based on invitations by health professionals and advertising in the community (e.g., local newspaper, health unit website and posters). Users willing to participate were invited to complete a registration form to verify eligibility criteria.
The endpoints of the study were:
- quality of life,
- psychological distress,
- satisfaction level,
- adherence rate.
The yoga routine consisted of breathing exercises, progressive articular and myofascial warming-up, followed by surya namascar (sun salutation sequence; adapted to the physical condition of each participant), alignment exercises, and postural awareness. Practice also included soft twists of the spine, reversed and balance postures, as well as concentration exercises. During the sessions, the instructor discussed some ethical guidelines of yoga, as for example, non-violence (ahimsa) and truthfulness (satya), to allow the participant to have a safer and integrated practice. In addition, the participants were encouraged to develop their awareness of the present moment and their body sensations, through a continuous process of self-consciousness, keeping a distance between body sensations and the emotional experience. The instructor emphasized the connection between breathing and movement. Each session ended with a guided deep relaxation (yoga nidra; 5–10 min), followed by a meditation practice (5–10 min).
The results of the study showed that the patients in the yoga group experienced a significant improvement in all domains of quality of life and a reduction of psychological distress. Linear regression analysis showed that yoga significantly improved psychological quality of life.
The authors concluded that yoga in primary care is feasible, safe and has a satisfactory adherence, as well as a positive effect on psychological quality of life of participants.
Are the authors’ conclusions correct?
I think not!
Here are some reasons for my judgement:
- The study was far to small to justify far-reaching conclusions about the safety and effectiveness of yoga.
- There were relatively high numbers of drop-outs, as seen in the graph above. Despite this fact, no intention to treat analysis was used.
- There was no randomisation, and therefore the two groups were probably not comparable.
- Participants of the experimental group chose to have yoga; their expectations thus influenced the outcomes.
- There was no attempt to control for placebo effects.
- The conclusion that yoga is safe would require a sample size that is several dimensions larger than 49.
In conclusion, this study fails to show that yoga has any value in primary care.
Oh, I almost forgot: and yoga is also satanic, of course (just like reading Harry Potter!).
Crohn’s disease (CD) is an inflammatory bowel disease characterized by recurring flares altered by periods of inactive disease and remission, affecting physical and psychological aspects and quality of life (QoL). The aim of this study was to determine the therapeutic benefits of soft non-manipulative osteopathic techniques in patients with CD.
A randomized controlled trial was performed. It included 30 individuals with CD who were divided into 2 groups: 16 in the experimental group (EG) and 14 in the control group (CG). The EG was treated with the 6 manual techniques depicted below. All patients were advised to continue their prescribed medications and diets. The intervention period lasted 30 days (1 session every 10 days). Pain, global quality of life (GQoL) and QoL specific for CD (QoLCD) were assessed before and after the intervention. Anxiety and depression levels were measured at the beginning of the study.
A significant effect was observed of the treatment in both the physical and task subscales of the GQoL and also in the QoLCD but not in pain score. When the intensity of pain was taken into consideration in the analysis of the EG, there was a significantly greater increment in the QoLCD after treatment in people without pain than in those with pain. The improvements in GQoL were independent from the disease status.
The authors concluded that soft, non-manipulative osteopathic treatment is effective in improving overall and physical-related QoL in CD patients, regardless of the phase of the disease. Pain is an important factor that inversely correlates with the improvements in QoL.
Where to begin?
Here are some of the most obvious flaws of this study:
- It was far too small for drawing any far-reaching conclusions.
- Because the sample size was so small, randomisation failed to create two comparable groups.
- Sub-group analyses are based on even smaller samples and thus even less meaningful.
- The authors call their trial a ‘single-blind’ study but, in fact, neither the patients nor the therapists (physiotherapists) were blind.
- The researchers were physiotherapists, their treatments were mostly physiotherapy. It is therefore puzzling why they repeatedly call them ‘osteopathic’.
- It also seems unclear why these and not some other soft tissue techniques were employed.
- The CG did not receive additional treatment at all; no attempt was thus made to control for placebo effects.
- The stated aim to determine the therapeutic benefits… seems to be a clue that this study was never aimed at rigorously testing the effectiveness of the treatments.
My conclusion therefore is (yet again) that poor science has the potential to mislead and thus harm us all.
Researchers tend to report only studies that are positive, while leaving negative trials unpublished. This publication bias can mislead us when looking at the totality of the published data. One solution to this problem is the p-curve. A significant p-value indicates that obtaining the result within the null distribution is improbable. The p-curve is the distribution of statistically significant p-values for a set of studies (ps < .05). Because only true effects are expected to generate right-skewed p-curves – containing more low (.01s) than high (.04s) significant p-values – only right-skewed p-curves are diagnostic of evidential value. By telling us whether we can rule out selective reporting as the sole explanation for a set of findings, p-curve offers a solution to the age-old inferential problems caused by file-drawers of failed studies and analyses.
The authors of this article tested the distributions of sets of statistically significant p-values from placebo-controlled studies of homeopathic ultramolecular dilutions. Such dilute mixtures are unlikely to contain a single molecule of an active substance. The researchers tested whether p-curve accurately rejects the evidential value of significant results obtained in placebo-controlled clinical trials of homeopathic ultramolecular dilutions.
Their inclusion criteria were as follows:
- Study is accessible to the authors.
- Study is a clinical trial comparing ultramolecular dilutions to placebo.
- Study is randomized, with randomization method specified.
- Study is double-blinded.
- Study design and methodology result in interpretable findings (e.g., an appropriate statistical test is used).
- Study reports a test statistic for the hypothesis of interest.
- Study reports a discrete p-value or a test statistic from which a p-value can be derived.
- Study reports a p-value independent of other p-values in p-curve.
The first 20 studies, in the order of search output, that met the inclusion criteria were used for analysis.
The researchers found that p-curve analysis accurately rejects the evidential value of statistically significant results from placebo-controlled, homeopathic ultramolecular dilution trials (1st graph below). This result indicates that replications of the trials are not expected to replicate a statistically significant result. A subsequent p-curve analysis was performed using the second significant p-value listed in the studies, if a second p-value was reported, to examine the robustness of initial results. P-curve rejects evidential value with greater statistical significance (2nd graph below). In essence, this seems to indicate that those studies of highly diluted homeopathics that reported positive findings, i. e. homeopathy is better than placebo, are false-positive results due to error, bias or fraud.
True effects with significant non-central distributions would have a greater density of low p-values than high p-values resulting in a right-skewed p-curve (like the dotted green lines in the above graphs). The fact that such a shape is not observed for studies of homeopathy confirms the many analyses previously demonstrating that ULTRAMOLECULAR HOMEOPATHIC REMEDIES ARE PLACEBOS.
Osteopathic visceral manipulation (OVM) have been our subject several times before. The method has been developed by the French Osteopath and Physical Therapist Jean-Pierre Barral. According to uncounted Internet-sites, books and other promotional literature, OVM is a miracle cure for just about every disease imaginable. Most of us hearing such claims hear alarm bells ringing – rightly so, I think. The evidence for OVM is thin, to put it mildly. But now, there is a new study to consider.
Brazilian researchers designed a placebo-controlled study using placebo visceral manipulation as the control to evaluate the effect of OVM of the stomach and liver on pain, cervical mobility, and electromyographic activity of the upper trapezius (UT) muscle in individuals with nonspecific neck pain (NS-NP) and functional dyspepsia. Twenty-eight NS-NP patients were randomly assigned into two groups: treated with OVM (OVMG; n = 14) and treated with placebo visceral manipulation (PVMG; n = 14). The effects were evaluated immediately and 7 days after treatment through pain, cervical range, and electromyographic activity of the UT muscle.
Significant effects were confirmed immediately after treatment (OVMG and PVMG) for numeric rating scale scores and pain area. Significant increases in EMG amplitude were identified immediately and 7 days after treatment for the OVMG. No differences were identified between the OVMG and the PVMG for cervical range of motion.
The authors concluded that the results of this pilot study indicate that a single session of osteopathic visceral manipulation for the stomach and liver reduces cervical pain and increases the amplitude of the upper trapezius muscle EMG signal immediately and 7 days after treatment in patients with nonspecific neck pain and functional dyspepsia. Patients treated with placebo visceral mobilisation reported a significant decrease in pain immediately after treatment. The effect of this intervention on the cervical range of motion was inconclusive. The results of this study suggest that further investigation is necessary.
There are numerous problems with this study:
- The authors call it a pilot study. Such a trial is for exploring the feasibility of a proper study. With the introduction of a placebo-OVM, this would make sense. The relevant question would then be: is the placebo valid and indistinguishable from the real thing? Sadly, this issue is not even addressed in the trial.
- A pilot study certainly is not for evaluating the effectiveness of an intervention. Sadly, this is precisely what the authors used it for. The label ‘pilot’, it seems, was merely given to excuse the many methodological flaws of their trial.
- For an evaluation of treatment effects, the study was far too small. This means the reported results can be discarded as meaningless.
- If we nevertheless took them seriously, we would want to explain how the findings were generated. The authors believe that they were caused by OVM. I find this most unlikely.
- The more plausible explanation would be that patient-blinding was unsuccessful. In other words, the placebo is not indistinguishable from the real OVM. Looking at the pictures above, one can easily see that the patients were able to tell to which group they had been allocated.
- The failure to blind patients (and, of course, the therapists), in turn, would mean that the verum group were better motivated to out-perform the placebo group in the outcome measures.
- Finally, I disagree with the authors’ view that the results of this study suggest that further investigation is necessary. On the contrary, I think that any further investment into OVM is ill-advised.
My conclusion: OVM is an implausible, non-evidence-based SCAM, and dodgy science is not going to make it look any more convincing.
Acupuncture is all over the news today. The reason is a study just out in BMJ-Open.
The aim of this new RCT was to investigate the efficacy of a standardised brief acupuncture approach for women with moderate-tosevere menopausal symptoms. Nine Danish primary care practices recruited 70 women with moderate-to-severe menopausal symptoms. Nine general practitioners with accredited education in acupuncture administered the treatments.
The acupuncture style was western medical with a standardised approach in the pre-defined acupuncture points CV-3, CV-4, LR-8, SP-6 and SP-9. The intervention group received one treatment for five consecutive weeks. The control group received no acupuncture but was offered treatment after 6 weeks. Outcomes were the differences between the two groups in changes to mean scores using the scales in the MenoScores Questionnaire, measured from baseline to week 6. The primary outcome was the hot flushes scale; the secondary outcomes were the other scales in the questionnaire. All analyses were based on intention-to-treat analysis.
Thirty-six patients received the intervention, and 34 were in the control group. Four participants dropped out before week 6. The acupuncture intervention significantly decreased hot flushes, day-and-night sweats, general sweating, menopausal-specific sleeping problems, emotional symptoms, physical symptoms and skin and hair symptoms compared with the control group at the 6-week follow-up. The pattern of decrease in hot flushes, emotional symptoms, skin and hair symptoms was already apparent three weeks into the study. Mild potential adverse effects were reported by four participants, but no severe adverse effects were reported.
The authors concluded that the standardised and brief acupuncture treatment produced a fast and clinically relevant reduction in moderate-to-severe menopausal symptoms during the six-week intervention.
The only thing that I find amazing here is the fact the a reputable journal published such a flawed trial arriving at such misleading conclusions.
- The authors call it a ‘pragmatic’ trial. Yet it excluded far too many patients to realistically qualify for this characterisation.
- The trial had no adequate control group, i.e. one that can account for placebo effects. Thus the observed outcomes are entirely in keeping with the powerful placebo effect that acupuncture undeniably has.
- The authors nevertheless conclude that ‘acupuncture treatment produced a fast and clinically relevant reduction’ of symptoms.
- They also state that they used this design because no validated sham acupuncture method exists. This is demonstrably wrong.
- In my view, such misleading statements might even amount to scientific misconduct.
So, what would be the result of a trial that is rigorous and does adequately control for placebo-effects? Luckily, we do not need to rely on speculation here; we have a study to demonstrate the result:
Background: Hot flashes (HFs) affect up to 75% of menopausal women and pose a considerable health and financial burden. Evidence of acupuncture efficacy as an HF treatment is conflicting.
Objective: To assess the efficacy of Chinese medicine acupuncture against sham acupuncture for menopausal HFs.
Design: Stratified, blind (participants, outcome assessors, and investigators, but not treating acupuncturists), parallel, randomized, sham-controlled trial with equal allocation. (Australia New Zealand Clinical Trials Registry: ACTRN12611000393954)
Setting: Community in Australia.
Participants: Women older than 40 years in the late menopausal transition or postmenopause with at least 7 moderate HFs daily, meeting criteria for Chinese medicine diagnosis of kidney yin deficiency.
Interventions:10 treatments over 8 weeks of either standardized Chinese medicine needle acupuncture designed to treat kidney yin deficiency or noninsertive sham acupuncture.
Measurements: The primary outcome was HF score at the end of treatment. Secondary outcomes included quality of life, anxiety, depression, and adverse events. Participants were assessed at 4 weeks, the end of treatment, and then 3 and 6 months after the end of treatment. Intention-to-treat analysis was conducted with linear mixed-effects models.
Results: 327 women were randomly assigned to acupuncture (n = 163) or sham acupuncture (n = 164). At the end of treatment, 16% of participants in the acupuncture group and 13% in the sham group were lost to follow-up. Mean HF scores at the end of treatment were 15.36 in the acupuncture group and 15.04 in the sham group (mean difference, 0.33 [95% CI, −1.87 to 2.52]; P = 0.77). No serious adverse events were reported.
Limitation: Participants were predominantly Caucasian and did not have breast cancer or surgical menopause.
Conclusion: Chinese medicine acupuncture was not superior to noninsertive sham acupuncture for women with moderately severe menopausal HFs.
My conclusion from all this is simple: acupuncture trials generate positive findings, provided the researchers fail to test it rigorously.
Highly diluted homeopathic remedies are pure placebos! This is what the best evidence clearly shows. Ergo they cannot be shown in a rigorous study to have effects that differ from placebo. But now there is a study that seems to contradict this widely accepted conclusion.
Can someone please help me to understand what is going on?
In this double-blind, placebo-controlled RCT, 60 patients suffering from insomnia were treated either individualised homeopathy (IH) or placebo for 3 months. Patient-administered sleep diary and Insomnia Severity Index (ISI) were used the primary and secondary outcomes respectively, measured at baseline, and after 3 months.
Five patients dropped out (verum:2,control:3).Intention to treat sample (n=60) was analysed. Trial arms were comparable at baseline. In the verum group, except sleep diary item 3 (P= 0.371), rest of the outcomes improved significantly (all P < 0.01). In the control group, there were significant improvements in diary item 6 and ISI score (P < 0.01) and just significant improvement in item 5 (P= 0.018). Group differences were significant for items 4, 5 and 6(P < 0.01) and just significant (P= 0.014) for ISI score with moderate to large effect sizes; but non-significant (P > 0.01) for rest of the outcomes.
The authors concluded that in this double-blind, randomized, prospective, placebo-controlled, two parallel arms clinical trial conducted on 60 patients suffering from insomnia, there was statistically significant difference measured in sleep efficiency, total sleep time, time in bed, and ISI score in favour of homeopathy over placebo with moderate to large effect sizes. Group differences were non-significant for rest of the outcomes(i.e. latency to fall asleep, minutes awake in middle of night and minutes awake too early). Individualized homeopathy seemed to produce significantly better effect than placebo. Independent replications and adequately powered trials with enhanced methodological rigor are warranted.
I have studied this article in some detail; its methodology is nicely and fully described in the original paper. To my amazement, I cannot find a flaw that is worth mentioning. Sure, the sample was small, the treatment time short, the outcome measure subjective, the paper comes from a dubious journal, the authors have a clear conflict of interest, even though they deny it – but none of these limitations has the potential to conclusively explain the positive result.
In view of what I stated above and considering what the clinical evidence so far tells us, this is most puzzling.
A 2010 systematic review authored by proponents of homeopathy included 4 RCTs comparing homeopathic medicines to placebo. All involved small patient numbers and were of low methodological quality. None demonstrated a statistically significant difference in outcomes between groups.
My own 2011 not Medline-listed review (Focus on Alternative and Complementary Therapies Volume 16(3) September 2011 195–199) included several additional studies. Here is its abstract:
The aim of this review was the critical evaluation of evidence for the effectiveness of homeopathy for insomnia and sleep-related disorders. A search of MEDLINE, AMED, CINAHL, EMBASE and Cochrane Central Register was conducted to find RCTs using any form of homeopathy for the treatment of insomnia or sleep-related disorders. Data were extracted according to pre-defined criteria; risk of bias was assessed using Cochrane criteria. Six randomised, placebo-controlled trials met the inclusion criteria. Two studies used individualised homeopathy, and four used standardised homeopathic treatment. All studies had significant flaws; small sample size was the most prevalent limitation. The results of one study suggested that homeopathic remedies were superior to placebo; however, five trials found no significant differences between homeopathy and placebo for any of the main outcomes. Evidence from RCTs does not show homeopathy to be an effective treatment for insomnia and sleep-related disorders.
It follows that the new trial contradicts previously published evidence. In addition, it clearly lacks plausibility, as the remedies used were highly diluted and therefore should be pure placebos. So, what could be the explanation of the new, positive result?
As far as I can see, there are the following possibilities:
- some undetected/undisclosed bias,
- homeopathy works after all.
I would be most grateful, if someone could help solving this puzzle for me (if needed, I can send you the full text of the new article for assessment).
The objective of this ‘real world’ study was to evaluate the effectiveness of integrative medicine (IM) on patients with coronary artery disease (CAD) and investigate the prognostic factors of CAD in a real-world setting.
A total of 1,087 hospitalized patients with CAD from 4 hospitals in Beijing, China were consecutively selected between August 2011 and February 2012. The patients were assigned to two groups:
- Chinese medicine (CM) plus conventional treatment, i.e., IM therapy (IM group). IM therapy meant that the patients accepted the conventional treatment of Western medicine and the treatment of Chinese herbal medicine including herbal-based injection and Chinese patent medicine as well as decoction for at least 7 days in the hospital or 3 months out of the hospital.
- Conventional treatment alone (CT group).
The endpoint was a major cardiac event [MCE; including cardiac death, myocardial infarction (MI), and the need for revascularization].
A total of 1,040 patients finished the 2-year follow-up. Of them, 49.4% received IM therapy. During the 2-year follow-up, the total incidence of MCE was 11.3%. Most of the events involved revascularization (9.3%). Cardiac death/MI occurred in 3.0% of cases. For revascularization, logistic stepwise regression analysis revealed that age ⩾ 65 years [odds ratio (OR), 2.224], MI (OR, 2.561), diabetes mellitus (OR, 1.650), multi-vessel lesions (OR, 2.554), baseline high sensitivity C-reactive protein level ⩾ 3 mg/L (OR, 1.678), and moderate or severe anxiety/depression (OR, 1.849) were negative predictors (P<0.05); while anti-platelet agents (OR, 0.422), β-blockers (OR, 0.626), statins (OR, 0.318), and IM therapy (OR, 0.583) were protective predictors (P<0.05). For cardiac death/MI, age ⩾ 65 years (OR, 6.389) and heart failure (OR, 7.969) were negative predictors (P<0.05), while statin use (OR, 0.323) was a protective predictor (P<0.05) and IM therapy showed a beneficial tendency (OR, 0.587), although the difference was not statistically significant (P=0.218).
The authors concluded that in a real-world setting, for patients with CAD, IM therapy was associated with a decreased incidence of revascularization and showed a potential benefit in reducing the incidence of cardiac death or MI.
What the authors call ‘real world setting’ seems to be a synonym of ‘lousy science’, I fear. I am not aware of good evidence to show that herbal injections and concoctions are effective treatments for CAD, and this study can unfortunately not change this. In the methods section of the paper, we read that the treatment decisions were made by the responsible physicians without restriction. That means the two groups were far from comparable. In their discussion section, the authors state; we found that IM therapy was efficacious in clinical practice. I think that this statement is incorrect. All they have shown is that two groups of patients with similar diagnoses can differ in numerous ways, including clinical outcomes.
The lessons here are simple:
- In clinical trials, lack of randomisation (the only method to create reliably comparable groups) often leads to false results.
- Flawed research is currently being used by many proponents of SCAM (so-called alternative medicine) to mislead us about the value of SCAM.
- The integration of dubious treatments into routine care does not lead to better outcomes.
- Integrative medicine, as currently advocated by SCAM-proponents, is a nonsense.