MD, PhD, FMedSci, FRSB, FRCP, FRCPEd

methodology

1 2 3 5

This randomized controlled trial was aimed to investigate the effect of aromatherapy massage on anxiety, depression, and physiologic parameters in older patients with acute coronary syndrome. It was conducted on 90 older women with acute coronary syndrome. The participants were randomly assigned into the intervention and control groups. The intervention group received reflexology with lavender essential oil plus routine care and the control group only received routine care. Physiologic parameters, the levels of anxiety and depression in the hospital were evaluated using a checklist and the Hospital’s Anxiety and Depression Scale, respectively, before and immediately after the intervention.

Significant differences in the levels of anxiety and depression were reported between the groups after the intervention. The analysis of physiological parameters revealed a statistically significant reduction in systolic blood pressure, diastolic blood pressure, mean arterial pressure, and heart rate. However, no significant difference was observed in the respiratory rate.

The authors concluded that aromatherapy massage can be considered by clinical nurses an efficient therapy for alleviating psychological and physiological responses among older women suffering from acute coronary syndrome.

WRONG!

This trial does not show remotely what the authors think. It demonstrates that A+B is always more than B. We have discussed this phenomenon so often that I hesitate to mention it again. Any study with the ‘A+B versus B’ design can only produce a positive result. The danger that this result is false-positive is so high that it is best to forget about such investigations altogether.

Ethics committees should not accept such protocols.

Researchers should stop running such studies.

Reviewers should not pass them for publication.

Editors should not publish such trials.

THEY MISLEAD ALL OF US AND GIVE CLINICAL RESEARCH A BAD NAME.

George Vithoulkas * (GV) is one of today’s most influential lay-homeopaths, a real ‘super guru’. He has many bizarre ideas; one of the most peculiar one was recently outlined in his article entitled ‘An innovative proposal for scientific alternative medical journals’. Here are a few excerpts from it:

…the only evidence that homeopathy can present to the scientific world at this moment are these thousands of cured cases. It is a waste of time, money, and energy to attempt to demonstrate the effectiveness of homeopathy through double blind trials.

… the international “scientific” community, which has neither direct perception nor personal experience of the beneficial effects of homeopathy, is forced to repeat the same old mantra: “Where is the evidence? Show us the evidence!” … the successes of homeopathy have remained hidden in the offices of hardworking homeopaths – and thus go largely ignored by the world’s medical authorities, governments, and the whole international scientific community…

… simple questions that are usually asked by the “gnorant”, for example, “Can homeopathy cure cancer, multiple sclerosis, ulcerative colitis, etc.?” are invalid and cannot elicit a direct answer because the reality is that many such cases can be ameliorated significantly, and a number can be cured…

A journal could invite a selected number of good prescribers from all over the world as a start to this project and let them contribute to their honest experience and results, as well as their failures. The possibilities and limitations would soon be revealed…

I admit that an argument against accepting cases is that it is possible that false or unreliable information could be provided. This risk could be minimized by preselecting a well-known group of good prescribers, who could be asked to submit their cases, at least in the first phase of such a radical change in the policy of the journals…

This way, instead of rejecting important homeopathic case studies, in the name of a dry intellectualism and conservatism, homeopathy journals (including alternative and complementary journals) could become lively and interesting: initiating debates and discussions on real issues of therapeutics in medicine…

Our own “Evidence Based Medicine” lies in the multitude of chronic cases treated with homeopathy that we can present to the world and on the better quality of life that such cures offer.

END OF QUOTES

So, GV wants homeopathy to thrive by means of publishing lots of case reports of patients who benefitted from homeopathy. And he believes that this suggestion is ‘innovative’? It is not! Case reports were all the rage 150 years ago before medicine started to become a little more scientific. And today, there are several journals specialising in the publication of case-reports, hundreds of journals that like accepting them, as well as dozens of websites that do little else but publishing case reports of homeopathy.

But case reports essentially are anecdotes. Medicine finally managed to progress from its dark ages when we realised how unreliable case reports truly are. To state it yet again (especially for GV who seems to be a bit slow on the uptake): THE PLURAL OF ANECDOTE IS ANECDOTES, NOT EVIDENCE!

In the above article, GV claims that ‘it is a waste of time, money, and energy to attempt to demonstrate the effectiveness of homeopathy through double blind trials.’ That is most puzzling because, only a few years ago, he did publish this:

Alternative therapies in general, and homeopathy in particular, lack clear scientific evaluation of efficacy. Controlled clinical trials are urgently needed, especially for conditions that are not helped by conventional methods. The objective of this work was to assess the efficacy of homeopathic treatment in relieving symptoms associated with premenstrual syndrome (PMS). It was a randomised controlled double-blind clinical trial. Two months baseline assessment with post-intervention follow-up for 3 months was conducted at Hadassah Hospital outpatient gynaecology clinic in Jerusalem in Israel 1992-1994. The subjects were 20 women, aged 20-48, suffering from PMS. Homeopathic intervention was chosen individually for each patient, according to a model of symptom clusters. Recruited volunteers with PMS were treated randomly with one oral dose of a homeopathic medication or placebo. The main outcome measure was scores of a daily menstrual distress questionnaire (MDQ) before and after treatment. Psychological tests for suggestibility were used to examine the possible effects of suggestion. Mean MDQ scores fell from 0.44 to 0.13 (P<0.05) with active treatment, and from 0.38 to 0.34 with placebo (NS). (Between group P=0.057). Improvement >30% was observed in 90% of patients receiving active treatment and 37.5% receiving placebo (P=0.048). Homeopathic treatment was found to be effective in alleviating the symptoms of PMS in comparison to placebo. The use of symptom clusters in this trial may offer a novel approach that will facilitate clinical trials in homeopathy. Further research is in progress.

I find this intriguing, particularly because the ‘further research’ mentioned prominently in the conclusions never did surface! Perhaps its results turned out to be unfavourable to homeopathy? Perhaps this is why GV dislikes RCTs these days? Perhaps this is why he prefers case reports such as this one which he recently published:

START OF QUOTE

An 81-year-old female patient was admitted in July 2015 to the Cardiovascular Surgery Department of a hospital in Bucharest for an aortic valve replacement surgery.

The patient had a history of mild hypertension, insulin-dependent type 2 diabetes, coronary artery disease, congestive heart failure NYHA 2, severe aortic stenosis, moderate mitral regurgitation, mild pulmonary hypertension, bilateral carotid atheromatosis with a 50% stenosis of the left internal carotid artery, complete right mastectomy for breast cancer (at that moment in remission).

After a preoperative evaluation and preparation, the surgery was completed with the replacement of the aortic valve with a bioprosthesis (Medtronic Hancock II Ultra no. 23) and myocardial revascularization by using a double aortic-coronary bypass.

The post-operatory evolution was a good one in terms of the heart disease. However, the patient did not regain consciousness after the anaesthesia, maintaining a deep comatose state (GCS 7 points – E1V2M4).

A brain CT was performed the third day postoperatively, showing no recent ischemic or haemorrhagic cerebral lesions, moderate diffuse cerebral atrophy and carotid atheromatosis.

After the surgery, the patient was admitted to the Intensive Care Unit and was treated by using a multidisciplinary approach. The patient was treated with inotropic, antiarrhythmic, and diuretic drugs, insulin and antidiabetic drugs were used in order to keep the blood sugar levels under control. The patient was kept hydrated and the electrolytes balanced by using an i.v. line, prophylaxis for deep vein thrombosis, and pulmonary thromboembolism was performed by using low molecular weight heparin. Prophylaxis for bedsores was also performed by using a pressure relieve air mattress.

The patient went into acute respiratory distress, needing mechanical ventilation in order to maintain oxygenation.

Despite these complex and correctly performed therapeutic efforts, the patient did not regain consciousness and was still in a deep coma in the fourteenth day post-operatory (GCS 7 points – E1V2M4), without having a confirmed medical explanation.

At that point, the patient’s family requested a consult from a homeopathic specialist.

The homeopathic examination, which was performed in the fourteenth day postoperatively, revealed the following: old, comatose, tranquil patient, with pale and cold skin, with the need to uncover herself (the few movements that she made with her hands were to remove her blanket and clothes, as if she wanted more air – “thirst for air”), abdominal distension, and bloating.

The thorough evaluation of the patient and the analysis of her symptoms led us to the remedy most appropriate for this critical situation – Carbo Vegetabilis.

Homeopathic treatment was initiated the same day, by using Carbo Vegetabilis 200CH 7 granules twice a day, administered diluted in 20ml of water by using a nasogastric tube.

The patient’s evolution was spectacular. The next day after the initiation of the treatment (fifteenth day postoperatively) the patient was in a superficial coma (GCS 11 points – E2V4M5), and the following day she regained consciousness. Carbo Vegetabilis was administered in the same dose for a total of five days (including the nineteenth day postoperatively).

After these five days, the case was reassessed from a homeopathically point of view and the second evaluation revealed the following: severely dyspnoeic patient (even talking caused exhaustion) with pale skin, severe fatigue aggravated by the slightest movements, a weakness sensation located in the chest area, extreme lack of energy, the wish “to be left alone”.

Considering the state of general exhaustion the patient was in at that moment and her lack of energy, the homeopathic treatment was changed to a new remedy: Stanum metallicum 30CH 7 granules administered sublingually twice a day for a week.

After the administration of the second remedy, the patient’s general condition improved dramatically: she started eating, she was able to get up in a sitting position with only little help, her fatigue diminished significantly.

The patient was then transferred to a recovery clinic in Cluj-Napoca in order to continue the cardiovascular recovery treatment. During her three-week admission in the clinic, she followed an individualized cardiovascular recovery program, which led to her ability to walk short distances with minimal support and has was released from the hospital in September 2015.

The following weeks after release, the patient recovered almost entirely, both physically and mentally. She was able to retake her place in her family and in society in general.

END OF QUOTE

One has to be a homeopath (one who is ignorant of the ‘post hoc propter hoc fallacy’) to believe in a causal link between the intake of the homeopathic remedy and the recovery of this patient. Thankfully, comatose patients do re-gain consciousness all the time! Even without homeopathy! But GV seems to not know that. In the discussion of this paper, he even states this: “ even after a well-conducted therapy, this condition leads to the death of the patient.” Is it ethical to publish such falsehoods, I wonder?

As far as the case report goes, the homeopathic remedy might even have delayed the process – perhaps the patient would have re-gained consciousness quicker and more completely without it! My hypothesis (homeopathy cased harm) is exactly as strong and silly as the one (homeopathy cased benefit) of GV. Anecdotes will never be able to answer the question as to who is correct.

One has to be a homeopath (and a daft one at that) to believe that this sort of evidence will lead to the acceptance of homeopathy by the scientific community. No journal will take GV seriously. No editor can be that stupid!

Oooops! Hold on, I might be wrong here.

Dr Peter Fisher, editor of the journal ‘Homeopathy’ just published an editorial ( Fisher P, Homeopathy and intellectual honesty, Homeopathy (2017), see also my previous post) stating that, in future, ‘we will increase publication of well-documented case-reports’.

Did I just claim that no editor can be that stupid?

 

 

 

  • I should declare a conflict of interest: when he got his ‘Right Livelihood Award’, GV sent me (and other prominent homeopathy-researchers) some of the prize money (I think it was around £ 1000) to support my research in homeopathy. I used it for exactly that purpose.

 

Gastro-oesophageal reflux disease (GORD) is a common, benign condition. It can be treated by changing eating habits or drugs. Many alternative therapies are also on offer, for instance, acupuncture. But does it work? Let’s find out.

The objective of this meta-analysis was to explore the effectiveness of acupuncture for the treatment of gastro-oesophageal reflux disease (GORD). Four English and four Chinese databases were searched through June 2016. Randomised controlled trials investigating the effectiveness of manual acupuncture or electroacupuncture (MA/EA) for GORD versus or as an adjunct to Western medicine (WM) were selected.

A total of 12 trials involving 1235 patients were included. The results demonstrated that patients receiving MA/EA combined with WM had a superior global symptom improvement compared with those receiving WM alone  with no significant heterogeneity. Recurrence rates of those receiving MA/EA alone were lower than those receiving WM  with low heterogeneity, while global symptom improvement (six studies) and symptom scores (three studies) were similar. Descriptive analyses suggested that acupuncture also improves quality of life in patients with GORD.

The authors concluded that this meta-analysis suggests that acupuncture is an effective and safe treatment for GORD. However, due to the small sample size and poor methodological quality of the included trials, further studies are required to validate our conclusions.

I am glad the authors used the verb ‘suggest’ in their conclusions. In fact, even this cautious terminology is too strong, in my view. Here are 9 reasons why:

  1. The hypothesis that acupuncture is effective for GORD lacks plausibility.
  2. All the studies were of poor or very poor methodological quality.
  3. All but one were from China, and we know that all acupuncture trials from this country are positive, thus casting serious doubt on their validity.
  4. Six trials had the infamous ‘A+B versus B’ design which never generates a negative result.
  5. There was evidence of publication bias, i. e. negative trials had disappeared and were thus not included in the meta-analysis.
  6. None of the trials made an attempt to control for placebo effects by using a sham-control procedure.
  7. None used patient-blinding.
  8. The safety of a therapy cannot be assessed on the basis of 12 trials
  9. Seven studies failed to report adverse effects, thus violating research ethics.

Considering these facts, I think that a different conclusion would have been more appropriate:  this meta-analysis provides no good evidence for the assumption that acupuncture is an effective and safe treatment for GORD.

The purpose of the study was to compare utilization of conventional psychotropic drugs among patients seeking care for anxiety and depression disorders (ADDs) from general practitioners (GPs) who

  • strictly prescribe conventional medicines (GP-CM),
  • regularly prescribe homeopathy in a mixed practice (GP-Mx),
  • or are certified homeopathic GPs (GP-Ho).

The investigation was an epidemiological cohort study of general practice in France, which included GPs and their patients consulting for ADDs (scoring 9 or more in the Hospital Anxiety and Depression Scale, HADS). Information on all medication utilization was obtained by a standardised telephone interview at inclusion, 1, 3 and 12 months.

Of 1562 eligible patients consulting for ADDs, 710 (45.5 %) agreed to participate. Adjusted multivariate analyses showed that GP-Ho and GP-Mx patients were less likely to use psychotropic drugs over 12 months, compared to GP-CM patients. The rate of clinical improvement (HADS <9) was marginally superior for the GP-Ho group as compared to the GP-CM group, but not for the GP-Mx group.

The authors concluded that patients with ADD, who chose to consult GPs prescribing homeopathy reported less use of psychotropic drugs, and were marginally more likely to experience clinical improvement, than patients managed with conventional care. Results may reflect differences in physicians’ management and patients’ preferences as well as statistical regression to the mean.

Aren’t we glad they added the last sentence to their conclusion!!!

Without it, one might have thought that the observed differences were due to the homeopathic remedies. In fact, the finding amounts to a self-fulfilling prophecy: Homeopaths tend to be against prescribing conventional drugs. This means that patients consulting homeopaths are bound to use less drugs than patients who consult conventional doctors. In that sense, the study was like monitoring whether consumers who go to the butchers buy more meat than those shopping in a shop for vegetarians.

The only result that requires a more serious consideration is that homeopathically treated patients experienced more clinical improvement than those treated conventionally. But even this difference is not hard to explain: firstly, the difference was merely marginal; secondly, patients with ADD are bound to respond particularly well to the empathetic and long therapeutic encounter most homeopaths offer. In other words, the difference had nothing to do with the alleged effectiveness of the homeopathic remedies.

It has been announced that Susan and Henry Samueli have given US$ 200 million to medical research at the University of California, Irvine (UCI). Surely this is a generous and most laudable gift! How could anyone doubt it?

As with any gift, one ought to ask what precisely it is for. If someone made a donation to research aimed at showing that climate change is a hoax, that white supremacy is justified, or that Brexit is going to give Brits their country back, I doubt that it would be a commendable thing. My point is that research must always be aimed at finding the truth and discovering facts. Research that is guided by creed, belief or misinformation is bound to be counter-productive, and a donation to such activities is likely to be detrimental.

Back to the Samuelis! The story goes that Susan once had a cold, took a homeopathic remedy, and subsequently the cold went away. Ever since, the two Samuelis have been supporters not just of homeopathy but all sorts of other alternative therapies. I have previously called this strikingly common phenomenon an ‘epiphany‘. And the Samuelis’ latest gift is clearly aimed at promoting alternative medicine in the US. We only need to look at what their other major donation in this area has achieved, and we can guess what is now going to happen at UCI. David Gorski has eloquently written about the UCI donation, and I will therefore not repeat the whole, sad story.

Instead I want to briefly comment on what, in my view, should happen, if a wealthy benefactor donates a large sum of money to medical research. How can one maximise the effects of such a donation? Which areas of research should one consider? I think the concept of prior probability can be put to good use in such a situation. If I were the donor, I would convene a panel of recognised experts and let them advise me where there are the greatest chances of generating important breakthroughs. If one followed this path, alternative medicine would not appear anywhere near the top preferences, I dare to predict.

But often, like in the case of the Samuelis, the donors have concrete ideas about the area of research they want to invest in. So, what could be done with a large sum in the field of alternative medicine? I believe that plenty of good could come it. All one needs to do is to make absolutely sure that a few safeguards are in place:

  • believers in alternative medicine must be kept out of any decisions processes;
  • people with a solid background in science and a track-record in critical thinking must be put in charge;
  • the influence of the donor on the direction of the research must be minimised as much as possible;
  • a research agenda must be defined that is meaningful and productive (this could include research into the risks of alternative therapies, the ethical standards in alternative medicine, the fallacious thinking of promoters of alternative medicine, the educational deficits of alternative practitioners, the wide-spread misinformation of the public about alternative medicine, etc., etc.)

Under all circumstances, one needs to avoid that the many pseudo-scientists who populate the field of alternative or integrative medicine get appointed. This, I fear, will not be an easy task. They will say that one needs experts who know all about the subtleties of acupuncture, homeopathy, energy-healing etc. But such notions are merely smoke-screens aimed at getting the believers into key positions. My advice is to vet all candidates using my concept of the ‘trustworthiness index’.

How can I be so sure? Because I have been there, and I have seen it all. I have researched this area for 25 years and published more about it than any of the untrustworthy believers. During this time I trained about 90 co-workers, and I have witnessed one thing over and over again: someone who starts out as a believer, will hardly ever become a decent scientist and therefore never produce any worthwhile research; but a good scientist will always be able to acquire the necessary knowledge in this or that alternative therapy to conduct rigorous and meaningful research.

So, how should the UCI spend the $ 200 million? Apparently the bulk of the money will be to appoint 15 faculty chairs across medicine, nursing, pharmacy and population health disciplines. They envisage that these posts will go to people with expertise in integrative medicine. This sounds extremely ominous to me. If this project is to be successful, these posts should go to scientists who are sceptical about alternative medicine and their main remit should be to rigorously test hypotheses. Remember: testing a hypothesis means trying everything to show that it is wrong. Only when all attempts to do so have failed can one assume that perhaps the hypothesis was correct.

My experience tells me that experts in integrative medicine are quite simply intellectually and emotionally incapable of making serious attempts showing that their beliefs are wrong. If the UCI does, in fact, appoint people with expertise in integrative medicine, it is, I fear, unavoidable that we will see:

  • research that fails to address relevant questions;
  • research that is of low quality;
  • promotion masquerading as research;
  • more and more misleading findings of the type we regularly discuss on this blog;
  • a further boost of the fallacious concept of integrative medicine;
  • a watering down of evidence-based medicine;
  • irreversible damage to the reputation of the UCI.

In a nutshell, instead of making progress, we will take decisive steps back towards the dark ages.

The goal of this study was to assess clinical outcomes observed among adult patients who received acupuncture treatments at a United States Air Force medical center.

This retrospective chart review was performed at the Nellis Family Medicine Residency in the Mike O’Callaghan Military Medical Center at Nellis Air Force Base in Las Vegas, NV. The charts were from 172 consecutive patients who had at least 4 acupuncture treatments within 1 year. These patients were suffering from a wide range of symptoms, including pain, anxiety and sleep problems. The main outcome measures were prescriptions for opioid medications, muscle relaxants, benzodiazepines, and nonsteroidal anti-inflammatory drugs (NSAIDS) in the 60 days prior to the first acupuncture session and in the corresponding 60 days 1 year later; and Measure Yourself Medical Outcome Profile (MYMOP2) values for symptoms, ability to perform activities, and quality of life.

The most common 10 acupuncture treatments in descending order were: (1) the Auricular Trauma Protocol; (2) Battlefield Auricular Acupuncture; (3) Chinese scalp acupuncture, using the upper one-fifth of the sensory area and the Foot Motor Sensory Area; (4) the Koffman Cocktail; (5) lumbar percutaneous electrical nerve stimulation (PENS); (6) various auricular functional points; (7) Chinese scalp acupuncture, using the frontal triangle pattern; (8) cervical PENS; (9) the Great American Malady treatment; and (10) tendinomuscular meridian treatment with surface release.

The results show that opioid prescriptions decreased by 45%, muscle relaxants by 34%, NSAIDs by 42%, and benzodiazepines by 14%. MYMOP2 values decreased 3.50–3.11 (P < 0.002) for question 1, 4.18–3.46 (P < 0.00001) for question 3, and 2.73–2.43 (P < 0.006) for question 4.

The authors concluded that in this military patient population, the number of opioid prescriptions decreased and patients reported improved symptom control, ability to function, and sense of well-being after receiving courses of acupuncture by their primary care physicians.

The phraseology used by the authors is intriguing; they imply that the clinical outcomes were the result of the acupuncture treatment without actually stating it. This is perhaps most obvious in the title of the paper: Reduction in Pain Medication Prescriptions and Self-Reported Outcomes Associated with Acupuncture in a Military Patient Population. Association is not causation! But the implication of a cause effect relationship is clearly there. Once we realise who is behind this research we understand why: This study was funded by the ACUS Foundation as part of a Cooperative Research and Development Agreement with the 99th Medical Group, at Nellis Air Force Base. 

The mission of Acus Foundation is to educate military physicians in the science and art of medical acupuncture, and to facilitate its integration into conventional military care… we are the most experienced team of physician teachers and practitioners of acupuncture in the United States. If they are so experienced, they surely also know that there are many explanations for the observed outcomes which are totally unrelated to acupuncture, e. g.:

  • the natural history of the conditions that were being treated;
  • the conventional therapies the soldiers received;
  • the regression to the mean;
  • social desirability;
  • placebo effects.

In fact the results could even indicate that acupuncture caused a delay of clinical improvement; without a control group, we cannot know either way. All we can safely assume from this study is that it is yet another example of promotion masquerading as research.

Shinrin-yoku means “taking in the forest atmosphere” or “forest bathing.” It was developed in Japan during the 1980s and has, according to its proponents, become a cornerstone of preventive health care and healing in Japanese medicine. Researchers primarily in Japan and South Korea have established a robust body of scientific literature on the health benefits of spending time under the canopy of a living forest… there have been many scientific studies that are demonstrating the mechanisms behind the healing effects of simply being in wild and natural areas. (some of this research is available here). For example, many trees give off organic compounds that support our “NK” (natural killer) cells that are part of our immune system’s way of fighting cancer.

The claimed benefits of Shinrin-yoku are remarkable:

  • Boosted immune system functioning, with an increase in the count of the body’s Natural Killer (NK) cells.
  • Reduced blood pressure
  • Reduced stress
  • Improved mood
  • Increased ability to focus, even in children with ADHD
  • Accelerated recovery from surgery or illness
  • Increased energy level
  • Improved sleep
  • Deeper and clearer intuition
  • Increased flow of energy
  • Increased capacity to communicate with the land and its species
  • Increased flow of eros/life force
  • Deepening of friendships
  • Overall increase in sense of happiness

But is any of this really true?

The aim of this state-of-the-art review was to summarise empirical research conducted on the physiological and psychological effects of Shinrin-Yoku. Research published from 2007 to 2017 was considered. A total of 64 studies met the inclusion criteria. According to the authors, they show that health benefits associated with the immersion in nature continue to be currently researched. Longitudinal research, conducted worldwide, is needed to produce new evidence of the relationships associated with Shinrin-Yoku and clinical therapeutic effects. Nature therapy as a health-promotion method and potential universal health model is implicated for the reduction of reported modern-day “stress-state” and “technostress.”

Odd?

Yes!

A look at the primary studies reveals that they are usually small and of poor quality.

Perhaps a brand new  review aimed more specifically at evaluating preventive or therapeutic effects of Shinrin-Yoku on blood pressure can tell us more. The authors considered all published, randomized, controlled trials, cohort studies, and comparative studies that evaluated the effects of the forest environment on changes in systolic blood pressure. Twenty trials involving 732 participants were reviewed. Systolic and diastolic blood pressure of patients submitted to the forest environment was significantly lower than that of controls. The authors concluded that this systematic review shows a significant effect of Shinrin-yoku on reduction of blood pressure.

I find this paper odd as well:

  • it lacks important methodological detail;
  • the authors included not just controlled clinical trials but all sorts of ‘studies’;
  • there is no assessment of the methodological rigor of the primary trials (from what I could see, they were mostly too poor to draw any conclusions from them).

What does all of this mean?

I have no problems in assuming that relaxation in a forest is beneficial in many ways and a nice experience.

But why call this a therapy?

It is relaxation!

Why make so many unsubstantiated claims?

And why study it in such obviously flawed ways?

All this does, I fear, is giving science a bad name.

This new RCT by researchers from the National Institute of Complementary Medicine in Sydney, Australia was aimed at ‘examining the effect of changing treatment timing and the use of manual, electro acupuncture on the symptoms of primary dysmenorrhea’. It had four arms:

  1. low frequency manual acupuncture (LF-MA),
  2. high frequency manual acupuncture (HF-MA),
  3. low frequency electro acupuncture (LF-EA)
  4. and high frequency electro acupuncture (HF-EA).

A total of 74 women were given 12 treatments over three menstrual cycles, either once per week (LF groups) or three times in the week prior to menses (HF groups). All groups received a treatment in the first 48 hours of menses. The primary outcome was the reduction in peak menstrual pain at 12 months from trial entry.

During the treatment period and 9 month follow-up all groups showed statistically significant reductions in peak and average menstrual pain compared to baseline. However, there were no differences between groups. Health related quality of life increased significantly in 6 domains in groups having high frequency of treatment compared to two domains in low frequency groups. Manual acupuncture groups required less analgesic medication than electro-acupuncture groups. HF-MA was most effective in reducing secondary menstrual symptoms compared to both–EA groups.

The authors concluded that acupuncture treatment reduced menstrual pain intensity and duration after three months of treatment and this was sustained for up to one year after trial entry. The effect of changing mode of stimulation or frequency of treatment on menstrual pain was not significant. This may be due to a lack of power. The role of acupuncture stimulation on menstrual pain needs to be investigated in appropriately powered randomised controlled trials.

If I were not used to reading rubbish research of alternative medicine in general and acupuncture in particular, this RCT would amaze me – not so much because of its design, execution, or write-up, but primarily because of its conclusion (why, oh why, I ask myself, did PLOS ONE publish this paper?). They are, I think, utterly barmy.

Let me explain:

  • acupuncture treatment reduced menstrual pain intensity” – oh no, it didn’t; at least this is not what the study proves; the fact that pain was perceived as less could be due to a host of factors, for instance regression towards the mean, or social desirability; as there was no proper control group, nobody can tell;
  • the lack of difference between treatments “may be due to a lack of power”. Yes, but more likely it is due to the fact that all versions of a placebo therapy generate similar outcomes.
  • acupuncture stimulation on menstrual pain needs to be investigated in appropriately powered randomised controlled trials”. Why? Because the authors have a quasi-religious belief in acupuncture? And if they have, why did they not design their study ‘appropriately’?

The best conclusion I can suggest for this daft trial is this: IN THIS STUDY, THE PRIMARY ENDPOINT SHOWED NO DIFFERENCE BETWEEN THE 4 TREATMENT GROUPS. THE RESULTS ARE THEREFORE FULLY COMPATIBLE WITH THE NOTION THAT ACUPUNCTURE IS A PLACEBO THERAPY.

Something along these lines would, in my view, have been honest and scientific. Sadly, in acupuncture research, we very rarely get such honest science and the ‘National Institute of Complementary Medicine in Sydney, Australia’ has no track record of being the laudable exception to this rule.

I have repeatedly cautioned about the often poor quality of research into alternative medicine. This seems particularly necessary with studies of acupuncture, and especially true for such research carried out in China. I have also frequently noted that certain ‘CAM journals’ are notoriously prone to publishing rubbish. So, what can we expect from a paper that:

  • is on alternative medicine,
  • focusses on acupuncture,
  • is authored by Chinese researchers,
  • was published in the Journal of Alternative and Complementary Medicine (JACM)?

The answer is PROBABLY NOT A LOT!

As if for confirming my prediction, The JACM just published this systematic review. It reports pairwise and network meta-analyses to determine the effectiveness of acupuncture and acupuncture-related techniques for the treatment of psoriasis. A total of 13 RCTs were included. The methodological quality of these studies was ‘not rigorous’ according to the authors – in fact, it was lousy. Acupoint stimulation seemed to be more effective than non-acupoint stimulation. The short-term treatment effect was superior to the long-term effect (as one would expect with placebo). Network meta-analysis suggested that acupressure or acupoint catgut embedding generate superior effects compared to medications. It was noted that acupressure was the most effective treatment of all the acupuncture-like therapies.

The authors concluded that acupuncture-related techniques could be considered as an alternative or adjuvant therapy for psoriasis in short term, especially of acupressure and acupoint catgut embedding. This study recommends further well-designed, methodologically rigorous, and more head-to-head randomized trials to explore the effects of acupuncture-related techniques for treating psoriasis.

And what is wrong with that?

EVERYTHING!

  • The review is of very poor quality.
  • The primary studies are even worse.
  • The English language is defective to the point of being not understandable.
  • The conclusions are misleading.

Correct conclusions should read something like this: Due to the paucity and the poor quality of the clinical trials, this review could not determine whether acupuncture and similar therapies are effective for psoriasis.

And then there is, of course, the question about plausibility. How plausible is the assumption that acupuncture might affect a genetic autoimmune disease like psoriasis. The answer, I think, is that the assumption is highly unlikely.

In the above review, most of the 13 primary RCTs were from China. One of the few studies not conducted in China is this one:

56 patients suffering from long-standing plaque psoriasis were randomized to receive either active treatment (electrostimulation by needles placed intramuscularly, plus ear-acupuncture) or placebo (sham, ‘minimal acupuncture‘) twice weekly for 10 weeks. The severity of the skin lesions was scored (PASI) before, during, and 3 months after therapy. After 10 weeks of treatment the PASI mean value had decreased from 9.6 to 8.3 in the ‘active’ group and from 9.2 to 6.9 in the placebo group (p < 0.05 for both groups). These effects are less than the usual placebo effect of about 30%. There were no statistically significant differences between the outcomes in the two groups during or 3 months after therapy. The patient’s own opinion about the results showed no preference for ‘active’ therapy. It was also clear from the answers that the blinded nature of the study had not been discovered by the patients. In conclusion, classical acupuncture is not superior to sham (placebo) ‘minimal acupuncture‘ in the treatment of psoriasis.

Somehow, I trust these conclusions more than the ones from the review!

And somehow, I get very tired of journal editors failing to do their job of rejecting papers that evidently are embarrassing, unethical rubbish.

Systematic reviews are aimed at summarising and critically evaluating the evidence on a specific research question. They are the highest level of evidence and are more reliable than anything else we have. Therefore, they represent a most useful tool for both clinicians and researchers.

But there are, of course, exceptions. Take, for instance, this recent systematic review by researchers from the

  • Texas Chiropractic College, Pasadena, the Palmer Center for Chiropractic Research, Palmer College of Chiropractic, Davenport,
  • Department of Planning, Policy and Design, University of California, Irvine,
  • VA Puget Sound Health Care System, Tacoma,
  • New York Chiropractic College, Seneca Falls,
  • Logan University College of Chiropractic, Chesterfield,
  • University of Western States, Portland.

Its purpose was to evaluate the effectiveness of conservative non-drug, non-surgical interventions, either alone or in combination, for conditions of the shoulder. The review was conducted from March 2016 to November 2016 in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), and was registered with PROSPERO. Eligibility criteria included randomized controlled trials (RCTs), systematic reviews, or meta-analyses studying adult patients with a shoulder diagnosis. Interventions qualified if they did not involve prescription medication or surgical procedures, although these could be used in the comparison group or groups. At least 2 independent reviewers assessed the quality of each study using the Scottish Intercollegiate Guidelines Network checklists. Shoulder conditions addressed were

  • shoulder impingement syndrome (SIS),
  • rotator cuff-associated disorders (RCs),
  • adhesive capsulitis (AC),
  • nonspecific shoulder pain.

Twenty-five systematic reviews and 44 RCTs met inclusion criteria. Low- to moderate-quality evidence supported the use of manual therapies for all 4 shoulder conditions. Exercise, particularly combined with physical therapy protocols, was beneficial for SIS and AC. For SIS, moderate evidence supported several passive modalities. For RC, physical therapy protocols were found beneficial but not superior to surgery in the long term. Moderate evidence supported extracorporeal shockwave therapy for calcific tendinitis RC. Low-level laser was the only modality for which there was moderate evidence supporting its use for all 4 conditions.

The authors concluded that the findings of this literature review may help inform practitioners who use conservative methods (eg, doctors of chiropractic, physical therapists, and other manual therapists) regarding the levels of evidence for modalities used for common shoulder conditions.

This review has so many defects that it would be boring to list them here.

The PRISMA guidelines  – I happen to be a co-author – state, for instance, that the abstract (the above text is from the abstract) should provide a structured summary including, as applicable: background; objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions and implications of key findings; systematic review registration number. It is obvious that the review authors have omitted several of these items.

And that is just the abstract!  There is much, much more to criticise in this paper.

The most striking deficit, in my view, is the useless conclusion: the one from the abstract (the part of the paper that will be read most widely) could have been written before the review had even been started. It is therefore not based on the data presented. Crucially it does not match the stated aim of this review (“to evaluate the effectiveness of conservative…interventions”).

But why? Why did the authors bother to follow PRISMA? Why did they formulate this bizarre conclusion in their abstract? Why did they do a review in the first place?

I fear, the answers might be embarrassingly simple:

  • They only pretended to follow PRISMA guidelines because that gives their review a veneer of respectability.
  • They formulated the conclusions because otherwise they would have needed to state that the evidence for manual therapy is less than convincing.
  • They conducted the review to promote chiropractic, and when the data were not as they had hoped for, they just back-paddled in an attempt to hide the truth as much as possible.

If this were an isolated case, I would not have bothered to mention it. But sadly, in the realm of chiropractic (and alternative medicine in general) we currently witness a plethora of rubbish reviews (published by rubbish journals). To the naïve observer, they might look rigorous and therefore they will be taken seriously. The end-effect of this pollution of the literature with rubbish is that we get a false-positive impression about the validity of the treatments in question. Consequently, we will see a host of wrong decisions on all levels of healthcare.

The big question is: HOW DO WE PROTECT OURSELVES FROM THIS DANGEROUS TREND?

I only see one solution: completely disregard certain journals that have been identified to regularly publish nonsense. Sadly, the wider medical community is far from having arrived at this point. As far as I can see, the problem has not even been identified yet as a serious issue that needs addressing. For the foreseeable future, we will probably have to live with this type of pollution of our medical literature.

1 2 3 5

Gravityscan Badge

Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted.


Click here for a comprehensive list of recent comments.

Categories