reviewer bias
Homeopaths tend to voice a standard set of arguments when confronted with irrefutable evidence against homeopathy. In the discussion sections of this blog, we heard them all:
- “The negative trials are flawed designed.” They claim these trials were done by ungifted therapists or used the wrong remedies, wrong potencies, wrong dosing schedules, etc. Therefore, they do not reflect true homeopathic practice.
- “Homeopathy is individualised, RCTs can’t capture it.” They argue that randomised clinical trials are inherently unsuitable because homeopathic treatment must be tailored to each patient, rendering RCTs “unfair” or even “unscientific.”
- “Only a fraction of the evidence has been considered.” They assert that critics cherry-pick negative evidence and ignore positive small trials, case series, or observational data that they regard as equally valid.
- “There is much positive evidence.” They point to older or methodologically weak positive studies and claim these outweigh or at least balance the otherwise negative body of evidence.
- “Meta-analyses and systematic reviews are biased and/or politically motivated.” They allege that negative evaluations are driven by ideological hostility to homeopathy, Big Pharma influence, or institutional bias.
- “Statistical significance is not the same as clinical reality.” They argue that statistics miss “real-world” benefits observed in practice and that evidence-based medicine is too narrow.
- “Evidence-based medicine overvalues RCTs and undervalues experience.” They insist that long clinical experience, case reports, patient testimonies, etc. should count as strong evidence and that their accumulated practice is itself proof of efficacy.
- “Patient demand and satisfaction are evidence.” They use high patient satisfaction, repeat consultations, and word-of-mouth popularity as a proxy for effectiveness.
- “Millions use it worldwide.” They argue that longstanding, global usage implies that it must work; otherwise it would have disappeared.
- “Conventional medicine is not perfect either.” They respond to criticism by highlighting harms, errors, and historical reversals in conventional medicine, implying that science-based critics lack moral authority.
- “If it were only placebo, it wouldn’t work on XY.” They claim efficacy in infants, animals, or unconscious patients as evidence that placebo cannot fully explain the effects.
- “Mechanisms aren’t fully known, but that doesn’t matter.” They liken homeopathy to earlier medical advances whose mechanisms were unknown at the time (e.g. aspirin), arguing that lack of a plausible mechanism is not a valid reason to reject positive clinical observations.
- “Physics and chemistry are incomplete; future science will explain it.” They invoke concepts like quantum physics, nanostructures, or complex systems to argue that current science is still too limited to explain homeopathy.
- “Regulatory / institutional conspiracies.” They suggest that powerful pharmaceutical or medical lobbies seek to suppress homeopathy to protect their financial interests.
- “Homeopathy is cheap and safe; risk–benefit favours it.” They argue that even if evidence is thin, the very low risk and low cost justify its use.
- “The therapeutic encounter itself is part of the effect.” They turn criticisms about placebo and context effects into a strength: the long consultation, empathy, and attention are claimed to be legitimate and central components of homeopathy.
- “Freedom of choice / patient autonomy.” They shift from scientific to ethical/political ground, insisting that patients should be free to choose homeopathy regardless of scientific consensus.
- “Skeptics misunderstand what homeopathy really is.” They claim that people conflate homeopathy with herbalism, confuse potencies, or misunderstand Hahnemann’s principles, so their critiques do not address true homeopathy.
- “Critics don’t see the individual ‘miracle’ cases.” They counter population-level data with vivid anecdotes of dramatic improvements which they regard as decisive.
- “Negative evidence is ‘absence of evidence’, not ‘evidence of absence’.” They argue that failed trials or negative reviews merely show that efficacy hasn’t been proved yet, not that homeopathy does not work.
- “Science evolves; today’s ‘overwhelming evidence’ may be overturned.” They claim that scientific consensus has been wrong before and that homeopathy will eventually be vindicated when paradigms shift.
In discussions with homeopaths, these points are repeated endlessly. One could easily get the impression of a broken record. All of the above arguments have in common that – even as some of then contain a kernel of truth – they are erroneous. In theory it could be easy to point this out to the stereotypical homeopathy promoter; in practice, however, it often is impossible, since the broken record continues turning senselessly.
Science is supposed to be self-correcting. Papers get reviewed, checked, challenged – and only then published. That’s the idea, anyway. But the practice often looks different. Between 2022 and 2024, that system didn’t just creak a little, it fell over rather dramatically.
At the centre of the mess was a Hindawi journal with the reassuringly serious name Evidence-Based Complementary and Alternative Medicine (eCAM – we have discussed it repeatedly on this blog, e.g. here and here). What followed was one of the largest clean-ups in modern publishing: hundreds of papers retracted from eCAM, and more than about 10,000 from the publisher’s full catalogue.
The reason?
Investigations had uncovered widespread manipulation of the publication process.
A big part of the story involves “paper mills.” These are businesses that will, for a fee, produce a scientific paper with your name on it. No inconvenient need for actual experiments or real data or the nuicance of writing. Using a mix of recycled material, fabricated results, and increasingly AI-generated texts, these outfits can churn out papers that look perfectly respectable, at least until someone reads them closely. Think of it as fast food for academic careers: quick, convenient, and not especially good for long-term health.
Another key ingredient was the journal’s abundant use of “special issues.” These are themed collections of papers run by guest editors, usually experts invited to oversee a niche topic. In theory, this can be a great idea. In practice, it turned out to be a bit like handing over the keys to your flat and hoping nobody throws a party.
Investigations found that, in some cases, the system had been even more seriously gamed. Fake or compromised editors, reviewer suggestions pointing to non-existent experts, and tightly coordinated “peer review rings” meant papers could sail through the process with both ease and speed.
The whole thing started to fall apart when research integrity teams and independent sleuths noticed odd and concerning patterns: identical images appearing in different papers, statistical results that didn’t add up, and peer reviews that seemed to happen at impossible speed. At that point, Wiley, which by then had acquired Hindawi, stepped in and took a much closer look. The result: mass retractions, suspended special issues, and a lot of nearly identical retraction notices politely stating that the “peer review process had been compromised.”
The fallout hit so-called alternative medicine (SCAM) hard. As we all know, SCAM is an area that already faces plenty of skepticism; discovering that a sizable chunk of its literature had effectively skipped quality control didn’t help. More broadly, the episode exposed a structural problem: when publishing lots of papers becomes the goal – for journals and researchers alike – quality tends to quietly take a back seat.
After all this, the journal eCAM more or less ground to a halt. It stopped accepting new submissions, and its future as an active journal remains uncertain. Its archive is still online but now dotted with retraction notices that read like a greatest hits album of publishing failures. Meanwhile, the publisher has tightened the processes,verifying reviewer identities more carefully, restricting special issues, and deploying tools to catch suspicious patterns earlier.
If there’s a silver lining, it’s that the system eventually did what it’s supposed to do: spot the problem and correct it. But the episode is a reminder that science doesn’t run on trust alone; it runs on verification. And when that verification slips, things can go wrong at scale, and with surprising speed. Or put another way: peer review works … as long as the “peers” actually exist and know their business.
Non-specific low back pain is a common condition with substantial socioeconomic implications. Pulsed electromagnetic field (PEMF) therapy is said to generate benefits in pain reduction and improvement of physical function in patients with pain-associated disorders like osteoarthritis. However, studies had heterogeneous settings. The aim of this systematic review was to assess the effects of PEMF on pain and function on patients with non-specific low back pain.
A systematic literature search of randomized controlled trials in PubMed, MEDLINE, EMBASE, Cochrane Library, and PEDro was performed (from inception until 15/5/2023). Outcome measures assessed pain and function.
Nine randomized controlled trials with 420 participants (n = 420) were included. The studies compared PEMF vs. placebo-PEMF, PEMF and conventional physical therapy vs. conventional physical therapy alone, PEMF and conventional physical therapy vs. placebo-PEMF and conventional physical therapy, PEMF vs. high-intensity laser therapy (HILT) vs. conventional physical therapy, and osteopathic manipulative treatment (OMT) and PEMF vs. PEMF alone vs. placebo-PEMF vs. OMT alone. Five of the nine included studies showed statistically significant pain reduction and improvement in physical function in comparison to their control groups (p < 0.05). There was substantial heterogeneity among the groups of the study, with a wide range of duration (10-30 min), treatments per week (2-7/week), applied frequencies (3-50 Hz), and intensities (2mT-150mT). No serious adverse event had been reported in any study. The included studies showed solid methodological quality, with an overall score of 7.2 points according to the PEDro scale.
The authors concluded that the results of the present systematic review suggest that the use of PEMF for patients with non-specific low back pain is beneficial in terms of pain reduction and enhancement of physical function, particularly if used as an addition to conventional physical therapy modalities. It has also been shown that PEMF is a safe therapy for the treatment of non-specific low back pain. Further high-quality studies with larger sample sizes and standardized protocols are necessary. The studies should also focus on determining the optimal parameters of frequency and intensity to advance PEMF application for all pain disorders.
PEMF therapy is a non-invasive treatment widely used by physiotherapist and some practitioners of so-called alternative medicine (SCAM). It applies low-frequency electromagnetic waves to the body. PEMF therapy acts like a wireless charger for your body, sending pulses of energy that penetrate deep into your tissues to jumpstart cellular recovery. Because your body contains conductive fluids (like blood) and charged ions (like calcium and potassium), the PEMF device is said to induce very small electrical currents within your tissues. In turn, the induced current is claimed to stimulate the mitochondria. This, according to proponents, boosts the production of ATP. The pulses are also said to influence the movement of ions across cell membranes, particularly calcium which triggers a cascade of beneficial biochemical reactions, such as the release of nitric oxide, which dilates blood vessels to improve circulation. Lastly, PEMF are claimed to help “calm” the chemical signals that cause chronic inflammation, switching the cellular environment from a state of stress to a state of healing.
While these assumptions look like wild extrapolations, the more crucial question is whether PEMF is clinically effective. The present review seems to tell us that it works for back pain. Yet, I am not convinced.
Most of the RCTs are not of good quality. Many did not blind patients or therapists. Only 2 made a direct comparison against a placebo, and those did not blind patients or check the success of blinding them. As the authors noted, there is plenty of heterogeneity in the included RCTs. All this adds up to a high level of uncertainty about the effectiveness of PEMF.
In summary, I fear that the authors of this review are too optimistic and lack critical thinking in evaluating the existing evidence. My conclusion therefore differs from theirs:
Many RCTs suggest that PEMF is effective for non-specific low back pain. Yet, due to serious limitations of the primary studies, no firm conclusion can be drawn.
Yet another study by Harald Walach et al on the placebo effect has been retracted.
The objective of the paper was to determine what contributes to the size of the placebo effect in clinical drug trials by meta-regressions of randomized placebo-controlled clinical trials. The authors identified 5 diseases where data on the rates of NT were available to search for a sample of n=150 (5×30) RCTs. They extracted various study descriptors and performed meta-regressions to predict improvement in treatment and placebo groups. The investigators sampled 30 trials each from the following diagnoses:
- osteoarthritis of the knee,
- irritable bowel syndrome,
- depression,
- sleep disorders,
- migraine,
and extracted relevant information. They estimated the effects due to RTM and NT and analyzed the improvement in the placebo and treatment groups by fitting two regression models. Both models were highly significant, explaining 72% of the variance. Improvement in the placebo group can be significantly predicted by improvement in the treatment group (beta= .84), whether a study was analyzed according to intention to treat (beta= -.10) or was a multicenter study (beta= .12). Improvement in the treatment group can be explained by the improvement in the placebo group (beta= .83), whether a study was a multi-center trial (beta = -.16), and by RTM (beta= -.18). The treatment effect is smaller in sleep studies (beta= -.17).
The authors concluded that the high correlation of r= .73 between placebo improvement and treatment improvement rates is genuine and not explainable by study or disease characteristics. We conclude from our data that the placebo-effect is the major driver of treatment effects in clinical trials that alone explains 69% of the variance. This leaves only limited space for effects due to pharmacological substances. Context effects are more important than pharmacological ones in the conditions studied by us.
Walach (senior author of the paper) offered the following explanation for the findings:
“The design of a clinical study, with blinding and randomisation, fulfils the formal criteria necessary to establish a generalised entanglement correlation [6-8]. This would mean: part of the therapeutic effect of a pharmacological substance is also found in the control group; but only because an entanglement correlation was generated by the blinding and randomisation. To put it another way: it is not really possible to draw any conclusions about the true effects of verum interventions based on such studies. Rather, one would have to use very different study types and extract the effect by combining the different data, as we once proposed. If one were to take these effects really seriously, the methodological canon would crumble quite quickly. But they are also the reason why, in interventions where the specific effects may be very small, it is not possible to separate ‘real’ from ‘fake’ effects by means of blinded, placebo-controlled studies.”
In my citicism of the study of Februay 2025, I suggested an alternative explanation,
SLOPPY SCIENCE:
- The authors lumped together trials of various drugs as though they are a homogeneous entity in terms of effectiveness beyond placebo (which, of course, they are not).
- The placebo response is the measured improvement of a patient in a clinical trial after receiving a sham treatment. Yet, the authors claim “the placebo response in clinical trials has four components: regression to the mean (RTM), measurement artefacts, natural tendency (NT) of the disease, and the genuine placebo effect”. This is nonsense, and I even fear that the authors know it.
- The paper’s ‘highlight’ claims that contextual factors like expectations and doctor-patient interactions drive healing. Yet, these phenomena are seperate from the placebo-effect and were not the subject of this investigation.
- Correlation is not causation.
In May 2025, it became known that the paper was under investigarion. Now, ‘Retraction Watch’ reported that the study by Harald Walach, who may be familiar to readers, has been retracted. Thus a record number of Walach’s papers has been withdrawn.
In one now-retracted paper, Walach and his coauthors claimed the COVID-19 vaccines killed two people for every three deaths they prevented. In a different, also retracted paper, Walach and colleagues claimed children’s masks trap carbon dioxide. (They later republished the article in a different journal.)
Walach lost two papers and a university post in 2021 and now holds affiliations with the Change Health Science Institute in Basel, Switzerland, and the Next Society Institute in Vilnius, Lithuania. This retraction brings his total to four.
As ‘Retraction Watch’ wrote in May, Gideon Meyerowitz-Katz, a research fellow at the University of Wollongong in Australia known for sleuthing, told us the results of the placebo effect paper seemed to indicate “simply being in a clinical trial is the main component of healing.”
According to the undated retraction notice, the journal “did not find any evidence of deliberate attempts to mislead or any scientific misconduct on the part of the authors.” The “authors agreed that the title is misleading and the manuscript should have been more nuanced or measured,” the notice continues, indicating the researchers misrepresented a formula and included a retracted study in their analysis. Nevertheless, the authors “judge that none of these mistakes change the overall findings and conclusions.”
Stephen Rhodes, a researcher at University Hospitals Cleveland Medical Center, had criticized the study in a letter to the editor in February, citing a “number of errors that lead to some sweeping conclusions.” In the letter, Rhodes wrote those leaps “reflect a misunderstanding of what a ‘treatment effect’ is.” In a placebo-controlled trial, Rhodes observed, the measure can’t be “due to placebo.”
The retraction notice also states the retraction “has raised important questions in respect of the editorial and peer review process for this manuscript, for which the Journal bears responsibility.” David Tovey, co-editor-in-chief of the JCE, told us “as a journal editor, I have to reflect on whether this situation could have been avoided.”
In a recent post, I suggested that, for the reasons explained in previous posts, Walach is the most illustrious member of our ALTERNATIVE MEDICINE HALL OF FAME:
- Prof Walach: what lies ahead for research into SCAM?
- Walach’s new meta-analysis of homeopathy revisited
- A new review of the ‘efficacy’ of homeopathic Arnica by Harald Walach & Co
- Prof Harald Walach and the intriguing ‘Vaccination Control Group’ project
- Prof Harald Walach is really unlucky
- Is Prof Harald Walach incompetent or dishonest?
- Prof Harald Walach’s new ground breaking study of praying the Rosary
- COVID-19 vaccinations: Prof Walach wants to “dampen the enthusiasm by sober facts”
I think my suggestion might have been correct!
Retractions occur when a paper is found to be flawed due to errors, plagiarism, data fabrication, peer-review fraud or other misconduct. On this blog, we have seen our fair share of such things. Occasionally, we even managed to get a fraudulent paper retracted. A paper in THE TIMES OF INDIA sheds more light on the retraction of scientific articles.
According to the Retraction Watch Database, 40,822 research articles were retracted globally by 2023. China led with 21,999 retractions in 2023, followed by the US with 3,731 and India with 2,737. In India, academic promotions and funding often hinge on publication counts, tempting researchers to cut corners. Paper mills, which produce fraudulent studies for a fee, have exploited this pressure, flooding journals with sham research.
The consequences of this type of scientific fraud are profound, e.g.:
- In 1998, Andrew Wakefield’s study in The Lancet falsely linked the MMR vaccine to autism. Though it was retracted in 2010 after evidence of data manipulation, vaccination rates dropped, leading to measles outbreaks. Public health efforts were diverted for years to counter this misinformation.
- A 2010 Science paper claiming arsenic-based life in a California lake suggested alternative biochemistry for extra-terrestrial life, was debunked in 2012, redirecting astrobiology research and prompting journals to tighten scrutiny.
The slow pace of retraction often worsens the problem. Dubious research lingers, cited and built upon long after its flaws are known. This is alarming given the scale of the crisis. In Feb, a Nature analysis found that retraction volumes at certain institutions in India had risen over the past half decade. Yet, ranking agencies like QS, Times Higher Education, and NIRF rarely adjust for retractions, prioritising publication volume over integrity. This allows a university with a high retraction rate to climb the ranks, sending a perverse message that productivity trumps quality.
India stands at a critical juncture in its journey toward becoming a knowledge superpower. With initiatives such as the National Research Foundation prioritising research-driven innovation, the country has set ambitious goals. However, this vision is under threat as the academic evaluation system increasingly rewards superficial metrics rather than genuine quality. The urgency of this issue cannot be overstated. Addressing the retraction crisis will require a steadfast commitment to academic integrity and decisive action to safeguard the nation’s intellectual aspirations.
Ranking frameworks must incorporate retraction penalties. QS, NIRF, and Times Higher Education should adjust scores to reflect research integrity, using metrics like the ratio of retracted papers to total publications. Institutions with high retraction rates should face scrutiny and attract penalties rather than accolades. Journals must strengthen peer review. The rise of paper mills demand automated tools to detect plagiarism and image manipulation, alongside stricter reviewer vetting. Publishers like Elsevier, Wiley and Springer should lead this effort.
Institutions, on their part, must prioritise quality over quantity. The ‘publish or perish’ culture should give way to one that rewards rigorous and reproducible research. In India, UGC could revise promotion criteria and penalise dubious research publications by faculty. Also, mentoring programs for early-career researchers could help instil ethical practices. Transparency is key. Retraction notices must be prompt, detailed, and widely disseminated. A national database of retracted papers, accessible to all, could serve both as a deterrent and a resource. Individuals with a track record of dubious publications must be permanently barred from holding administrative positions.
By penalising retractions, strengthening oversight mechanisms, and fostering a culture of quality, we can restore trust. Only then will India and the world realise the true potential of science that breeds innovation.
__________________
Allow me to add a few points that are more specific to so-called alternative medicine (SCAM):
- In SCAM, we encounter many researchers who manage to go through their entire career without ever publishing a negative result for their pet therapy. Regular readers will know that I have established the ALTERNATIVE MEDICINE HALL OF FAME for these individuals.
- I have often reported that research originating in China is particularly suspect.
- An intriguing case is the one of Daniel Wirth, not least because of the reluctance of some SCAM journals to retract fraudulent research.
- As we see on an almost daily basis, much if not most of SCAM research is profoundly unreliable due to overt bias, poor methodology, misleading conclusions, etc.
Of course, such problems are not confined to SCAM, they exist in all fields. What is rather unique to the realm of SCAM is, however, this: far too many SCAM researchers seem to be ignorant of the fact that research is about FALSIFYING hypotheses. Instead they abuse science by designing studies for CONFIRMING their pet hypothesis. This, however, is not science and often it borders on fraud.
The fallout of such scientific misconduct can only be harmful to us all. I therefore plea (yet again) to journal editors, reviewers and everyone else involved to be much more vigilant and stop misleading and fraudulent papers from getting published.
In 1993, I was appointed Professor of Complementary Medicine at the University of Exeter. After settling in, I started to wonder what a suitable initial research project would be. Someone alerted me to some funds made available by the University for small projects – if I remember corretly, the amount was less that £30 000. Apparently, the grant was to be unbeaurocratic and, as I was newly appointed and everyone was wondering what I might get up to, I decided to go for it.
I sat down and recycled an idea that I had long wanted to test by never quite has the time to realize: a simple clinical trial where patients suffering from a common, benign condition [I think I chose insomnia] would be randomized into two groups. Both groups would be treated with a mild sleeping aid [I think I chose valarian]. The only difference between the two groups was that
- the experimental group would get an empathetic consultation, while
- the control group would get a normal one.
Thus the aim of the trial was to see whether empathy would impact on the outcome. My plan was that the prescribing clinician would be trained such that he/she could behave either normally or empathetically at will.
After I had submitted the proposal it was ‘peer reviewed’ by University staff. As it happened, my reviewer was John Tooke – he later became professor and medical director of the medical school at Exeter and later again moved to London and became the ‘Head of the School of Life & Medical Sciences’ at UCL but, at the time, he was just a doctor in the local hospital (if you are interested, there is more about Tooke in my memoir). Tooke’s assessment of my proposal was, to put it mildly, not positive and, of course, I never did get the funding. His critique explained that in the English language we call it ‘sympathy’ and not ’empathy’.
So, the study outlined above never got done, and every time I hear the word ’empathy’, I am reminded of this little anecdote. Like, for instance, a few days ago when I read that Elon Musk in his infinite wisdom had pronounced that: “The fundamental weakness of Western civilization is empathy”
I have to admit: this thought scares me!
The question I ask myself is this: to what extent does a neglect of emapthy contribute to the professional success of some people?
In any case, I am jolly glad that in the English language does have a word ’empathy’. I am even more profoundly glad that many people I know do have empathy. And I agree with Hannah Arendt who once wrote that “The death of human emapthy is one of the earliest and most telling signs of a culture about to fall into barbarism”
PS
sympathy = (an expression of) understanding and care for someone else’s suffering
This systematic review and meta-analysis investigated the effectiveness and safety of manual therapy (MT) interventions compared to oral or topical pain medications in the management of neck pain.
The investigators searched from inception to March 2023, in Cochrane Central Register of Controller Trials (CENTRAL), MEDLINE, EMBASE, Allied and Complementary Medicine (AMED) and Cumulative Index to Nursing and Allied Health Literature (CINAHL; EBSCO) for randomized controlled trials that examined the effect of manual therapy interventions for neck pain when compared to oral or topical medication in adults with self-reported neck pain, irrespective of radicular findings, specific cause, and associated cervicogenic headaches. Trials with usual care arms were also included if they prescribed medication as part of the usual care and they did not include a manual therapy component. The authors used the Cochrane Risk of Bias 2 tool to assess the potential risk of bias in the included studies, and the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) approach to grade the quality of the evidence.
Nine trials with a total of 779 participants were included in the meta-analysis.
- low certainty of evidence was found that MT interventions may be more effective than oral pain medication in pain reduction in the short-term (Standardized Mean Difference: -0.39; 95% CI -0.66 to -0.11; 8 trials, 676 participants),
- moderate certainty of evidence was found that MT interventions may be more effective than oral pain medication in pain reduction in the long-term (Standardized Mean Difference: −0.36; 95% CI −0.55 to −0.17; 6 trials, 567 participants),
- low certainty evidence that the risk of adverse events may be lower for patients who received MT compared to the ones that received oral pain medication (Risk Ratio: 0.59; 95% CI 0.43 to 0.79; 5 trials, 426 participants).
The authors conluded that MT may be more effective for people with neck pain in both short and long-term with a better safety profile regarding adverse events when compared to patients receiving oral pain medications. However, we advise caution when interpreting our safety results due to the different level of reporting strategies in place for MT and medication-induced adverse events. Future MT trials should create and adhere to strict reporting strategies with regards to adverse events to help gain a better understanding on the nature of potential MT-induced adverse events and to ensure patient safety.
Let’s have a look at the primary studies. Here they are with their conclusions (and, where appropriate, my comments in capital letters):
- For participants with acute and subacute neck pain, spinal manipulative therapy (SMT) was more effective than medication in both the short and long term. However, a few instructional sessions of home exercise with (HEA) resulted in similar outcomes at most time points. EXERCISE WAS AS EFFECTIVE AS SMT
- Oral ibuprofen (OI) pharmacologic treatment may reduce pain intensity and disability with respect to neural mobilization (MNNM and CLG) in patients with CP during six weeks. Nevertheless, the non-existence of between-groups ROM differences and possible OI adverse effects should be considered. MEDICATION WAS BETTER THAN MT
- It appears that both treatment strategies (usual care + MT vs usual care) can have equivalent positive influences on headache complaints. Additional studies with larger study populations are needed to draw firm conclusions. Recommendations to increase patient inflow in primary care trials, such as the use of an extended network of participating physicians and of clinical alert software applications, are discussed. MT DOES NOT IMPROVE OUTCOMES
- The consistency of the results provides, in spite of several discussed shortcomings of this pilot study, evidence that in patients with chronic spinal pain syndromes spinal manipulation, if not contraindicated, results in greater improvement than acupuncture and medicine. THIS IS A PILOT STUDY, A TRIAL TESTING FEASIBILITY, NOT EFFECTIVENESS
- The consistency of the results provides, despite some discussed shortcomings of this study, evidence that in patients with chronic spinal pain, manipulation, if not contraindicated, results in greater short-term improvement than acupuncture or medication. However, the data do not strongly support the use of only manipulation, only acupuncture, or only nonsteroidal antiinflammatory drugs for the treatment of chronic spinal pain. The results from this exploratory study need confirmation from future larger studies.
- In daily practice, manual therapy is a favorable treatment option for patients with neck pain compared with physical therapy or continued care by a general practitioner.
- Short-term results (at 7 weeks) have shown that MT speeded recovery compared with GP care and, to a lesser extent, also compared with PT. In the long-term, GP treatment and PT caught up with MT, and differences between the three treatment groups decreased and lost statistical significance at the 13-week and 52-week follow-up. MT IS NOT SUPERIOR [SAME TRIAL AS No 6]
- In this randomized clinical trial, for patients with chronic neck pain, Chuna manual therapy was more effective than usual care in terms of pain and functional recovery at 5 weeks and 1 year after randomization. These results support the need to consider recommending manual therapies as primary care treatments for chronic neck pain.
- In patients with chronic spinal pain syndromes, spinal manipulation, if not contraindicated, may be the only treatment modality of the assessed regimens that provides broad and significant long-term benefit. SAME TRIAL AS No 5
- An impairment-based manual physical therapy and exercise (MTE) program resulted in clinically and statistically significant short- and long-term improvements in pain, disability, and patient-perceived recovery in patients with mechanical neck pain when compared to a program comprising advice, a mobility exercise, and subtherapeutic ultrasound. THIS STUDY DID NOT TEST MT ALONE AND SHOULD NOT HAVE BEEN INCLUDED
I cannot bring myself to characterising this as an overall positive result for MT; anyone who can is guilty of wishful thinking, in my view. The small differences in favor of MT that (some of) the trials report have little to do with the effectiveness of MT itself. They are almost certainly due to the fact that none of these studies were placebo-controlled and double blind (even though this would clearly be possible). In contrast to popping a pill, MT involves extra attention, physical touch, empathy, etc. These factors easily suffice to bring about the small differences that some studies report.
It follows that the main conclusion of the authors of the review should be modified:
There is no compelling evidence to show that MT is more effective for people with neck pain in both short and long-term when compared to patients receiving oral pain medications.
The WHO has just released guidelines for non-surgical management of chronic primary low back pain (CPLBP). The guideline considers 37 types of interventions across five intervention classes. With the guidelines, WHO recommends non-surgical interventions to help people experiencing CPLBP. These interventions include:
- education programs that support knowledge and self-care strategies;
- exercise programs;
- some physical therapies, such as spinal manipulative therapy (SMT) and massage;
- psychological therapies, such as cognitive behavioural therapy; and
- medicines, such as non-steroidal anti-inflammatory medicines.
The guidelines also outline 14 interventions that are not recommended for most people in most contexts. These interventions should not be routinely offered, as WHO evaluation of the available evidence indicate that potential harms likely outweigh the benefits. WHO advises against interventions such as:
- lumbar braces, belts and/or supports;
- some physical therapies, such as traction;
- and some medicines, such as opioid pain killers, which can be associated with overdose and dependence.
As you probably guessed, I am particularly intrigued by the WHO’s positive recommendation for SMT. Here is what the guideline tells us about this specific topic:
Considering all adults, the guideline development group (GDG) judged overall net benefits [of spinal manipulation] across outcomes to range from trivial to moderate while, for older people the benefit was judged to be largely uncertain given the few trials and uncertainty of evidence in this group. Overall, harms were judged to be trivial to small for all adults and uncertain for older people due to lack of evidence.
The GDG commented that while rare, serious adverse events might occur with SMT, particularly in older people (e.g. fragility fracture in people with bone loss), and highlighted that appropriate training and clinical vigilance concerning potential harms are important. The GDG also acknowledged that rare serious adverse events were unlikely to be detected in trials. Some GDG members considered that the balance of benefits to harms favoured SMT due to small to moderate benefits while others felt the balance did not favour SMT, mainly due to the very low certainty evidence for some of the observed benefits.
The GDG judged the overall certainty of evidence to be very low for all adults, and very low for older people, consistent with the systematic review team’s assessment. The GDG judged that there was likely to be important uncertainty or variability among people with CPLBP with respect to their values and preferences, with GDG members noting that some people might prefer manual
therapies such as SMT, due to its “hands-on” nature, while others might not prefer such an approach.
Based on their experience and the evidence presented from the included trials which offered an average of eight treatment sessions, the GDG judged that SMT was likely to be associated with moderate costs, while acknowledging that such costs and the equity impacts from out-of-pocket costs would vary by setting.
The GDG noted that the cost-effectiveness of SMT might not be favourable when patients do not experience symptom improvements early in the treatment course. The GDG judged that in most settings, delivery of SMT would be feasible, although its acceptability was likely to vary across
health workers and people with CPLBP.
The GDG reached a consensus conditional recommendation in favour of SMT on the basis of small to moderate benefits for critical outcomes, predominantly pain and function, and the likelihood of rare adverse events.
The GDG concluded by consensus that the likely short-term benefits outweighed potential harms, and that delivery was feasible in most settings. The conditional nature of the recommendation was informed by variability in acceptability, possible moderate costs, and concerns that equity might be negatively impacted in a user-pays model of financing.
___________________________
This clearly is not a glowing endorsement or recommendation of SMT. Yet, in my view, it is still too positive. In particular, the assessment of harm is woefully deficient. Looking into the finer details, we find how the GDG assessed harms:
WHO commissioned quantitative systematic evidence syntheses of randomized controlled
trials (RCTs) to evaluate the benefits and harms (as reported in included trials) of each of the
prioritized interventions compared with no care (including trials where the effect of an
intervention could be isolated), placebo or usual care for each of the critical outcomes (refer to Table 2 for the PICO criteria for selecting evidence). Research designs other than RCTs
were not considered.
That explains a lot!
It is not possible to establish the harms of SMT (or any other therapy) on the basis of just a few RCTs, particularly because the RCTs in question often fail to report adverse events. I can be sure of this phenomenon because we investigated it via a systematic review:
Objective: To systematically review the reporting of adverse effects in clinical trials of chiropractic manipulation.
Data sources: Six databases were searched from 2000 to July 2011. Randomised clinical trials (RCTs) were considered, if they tested chiropractic manipulations against any control intervention in human patients suffering from any type of clinical condition. The selection of studies, data extraction, and validation were performed independently by two reviewers.
Results: Sixty RCTs had been published. Twenty-nine RCTs did not mention adverse effects at all. Sixteen RCTs reported that no adverse effects had occurred. Complete information on incidence, severity, duration, frequency and method of reporting of adverse effects was included in only one RCT. Conflicts of interests were not mentioned by the majority of authors.
Conclusions: Adverse effects are poorly reported in recent RCTs of chiropractic manipulations.
The GDG did not cite our review (or any other of our articles on the subject) but, as it was published in a very well-known journal, they must have been aware of it. I am afraid that this wilfull ignorance caused the WHO guideline to underestimate the level of harm of SMT. As there is no post-marketing surveillance system for SMT, a realistic assessment of the harm is far from easy and needs to include a carefully weighted summary of all the published reports (such as this one).
The GDG seems to have been aware of (some of) these problems, yet they ignored them and simply assumed (based on wishful thinking?) that the harms were small or trivial.
Why?
Even the most cursory look at the composition of the GDG, begs the question: could it be that the GDG was highjacked by chiropractors and other experts biased towards SMT?
The more I think of it, the more I feel that this might actually be the case. One committee even listed an expert, Scott Haldeman, as a ‘neurologist’ without disclosing that he foremost is a chiropractor who, for most of his professional life, has promoted SMT in one form or another.
Altogether, the WHO guideline is, in my view, a shameful example of pro-chiropractic bias and an unethical disservice to evidence-based medicine.
When I was still at Exeter, I used to do an average of about 4 peer reviews per week of articles that had been submitted to all sorts of journals for publication. Now I reject most of these invitations and do perhaps just one per month.
Why?
Conducting a peer review is by no means an easy task. You have to realize that the authors have usually put a lot of hard work into their paper and a lot may depend on it in terms of their future. They thus have the right to receive a fair and responsible review. To do the job properly, it took me (even with plenty of experience in reading scientific papers) between 1 and 3 hours per article. Crucially, low-quality articles typically submitted to low-quality journals are more work than papers that adhere to a certain standard.
I do not think that the journal editors who send the submissions out for review appreciate how much work they ask from the reviewers. They normally pay nothing (even if they charge exorbitant handling fees from the authors) and offer you no benefit at all. In addition, many have systems that are more than tedious asking you to register, create a pin number, etc., etc. Then you have to follow certain rules and formats that differ from journal to journal. In a word, they add an administrative burden to the task of reading, understanding, checking a paper, and composing your judgment on it.
All this can be cumbersome but it’s not the reason why I do less and less peer reviews. The true reason is that research papers on so-called alternative medicine (SCAM) are now mostly published in one of the many 3rd class SCAM journals that have recently sprung up. There are so many of them that they, of course, struggle to get enough articles to fill their pages. In turn, this means that they are far too keen to publish anything regardless of its quality or validity. As a consequence, the quality of these articles and their authors are often dismal.
Here is an example of a (rather shocking but not unusual) email I received only today; it might show you what I mean:
Dear Professor!
…
I want to publish some papers in “Areas related to your research field”. Can you help me? I can provide a thank you fee!
For example, I will give you a $2000 thank you fee for helping me write articles. For example, if you add my name to your article, I will give you a $1000 thank you fee. Or I can help you pay for APC.
I know this email is presumptuous, but my friends and I need to publish dozens of papers every year. If you can help me, we can cooperate for a long time. I’m not kidding, I’m very sincere!
If you are offended, please forgive me!
Look forward to your reply!
Warmly Wishes, …
When I do a review for a low-quality SCAM journal and find major defects in an article, my experience has been that the editor then decides to publish it nonetheless. When this happens, I feel frustrated and ask myself: WHY DID THEY ASK FOR MY OPINION IF THEY DO NOT ABIDE BY IT?
Thus I decided that these journals are just as well off without my contributions. So, if you are an editor of a SCAM journal, do me a favor and do not molest me with your invitations to conduct a peer review and
COUNT ME OUT!
Should Acupuncture-Related Therapies be Considered in Prediabetes Control?
No!
If you are pre-diabetic, consult a doctor and follow his/her advice. Do NOT do what acupuncturists or other self-appointed experts tell you. Do NOT become a victim of quackery.
But the authors of a new paper disagree with my view.
So, let’s have a look at the evidence.
Their systematic review was aimed at evaluating the effects and safety of acupuncture-related therapy (AT) interventions on glycemic control for prediabetes. The Chinese researchers searched 14 databases and 5 clinical registry platforms from inception to December 2020. Randomized controlled trials involving AT interventions for managing prediabetes were included.
Of the 855 identified trials, 34 articles were included for qualitative synthesis, 31 of which were included in the final meta-analysis. Compared with usual care, sham intervention, or conventional medicine, AT treatments yielded greater reductions in the primary outcomes, including fasting plasma glucose (FPG) (standard mean difference [SMD] = -0.83; 95% confidence interval [CI], -1.06, -0.61; P < .00001), 2-hour plasma glucose (2hPG) (SMD = -0.88; 95% CI, -1.20, -0.57; P < .00001), and glycated hemoglobin (HbA1c) levels (SMD = -0.91; 95% CI, -1.31, -0.51; P < .00001), as well as a greater decline in the secondary outcome, which is the incidence of prediabetes (RR = 1.43; 95% CI, 1.26, 1.63; P < .00001).
The authors concluded that AT is a potential strategy that can contribute to better glycemic control in the management of prediabetes. Because of the substantial clinical heterogeneity, the effect estimates should be interpreted with caution. More research is required for different ethnic groups and long-term effectiveness.
But this is clearly a positive result!
Why do I not believe it?
There are several reasons:
- There is no conceivable mechanism by which AT prevents diabetes.
- The findings heavily rely on Chinese RCTs which are known to be of poor quality and often even fabricated. To trust such research would be a dangerous mistake.
- Many of the primary studies were designed such that they failed to control for non-specific effects of AT. This means that a causal link between AT and the outcome is doubtful.
- The review was published in a 3rd class journal of no impact. Its peer-review system evidently failed.
So, let’s just forget about this rubbish paper?
If only it were so easy!
Journalists always have a keen interest in exotic treatments that contradict established wisdom. Predictably, they have been reporting about the new review thus confusing or misleading the public. One journalist, for instance, stated:
Acupuncture has been used for thousands of years to treat a variety of illnesses — and now it could also help fight one of the 21st century’s biggest health challenges.
New research from Edith Cowan University has found acupuncture therapy may be a useful tool in avoiding type 2 diabetes.
The team of scientists investigated dozens of studies covering the effects of acupuncture on more than 3600 people with prediabetes. This is a condition marked by higher-than-normal blood glucose levels without being high enough to be diagnosed as diabetes.
According to the findings, acupuncture therapy significantly improved key markers, such as fasting plasma glucose, two-hour plasma glucose, and glycated hemoglobin. Additionally, acupuncture therapy resulted in a greater decline in the incidence of prediabetes.
The review can thus serve as a prime example for demonstrating how irresponsible research has the power to mislead millions. This is why I have often said that poor research is a danger to public health.
And what can be done about this more and more prevalent problem?
The answer is easy: people need to behave more responsibly; this includes:
- trialists,
- review authors,
- editors,
- peer-reviewers,
- journalists.
Yes, the answer is easy in theory – but the practice is far from it!
