reviewer bias
Science is supposed to be self-correcting. Papers get reviewed, checked, challenged – and only then published. That’s the idea, anyway. But the practice often looks different. Between 2022 and 2024, that system didn’t just creak a little, it fell over rather dramatically.
At the centre of the mess was a Hindawi journal with the reassuringly serious name Evidence-Based Complementary and Alternative Medicine (eCAM – we have discussed it repeatedly on this blog, e.g. here and here). What followed was one of the largest clean-ups in modern publishing: hundreds of papers retracted from eCAM, and more than about 10,000 from the publisher’s full catalogue.
The reason?
Investigations had uncovered widespread manipulation of the publication process.
A big part of the story involves “paper mills.” These are businesses that will, for a fee, produce a scientific paper with your name on it. No inconvenient need for actual experiments or real data or the nuicance of writing. Using a mix of recycled material, fabricated results, and increasingly AI-generated texts, these outfits can churn out papers that look perfectly respectable, at least until someone reads them closely. Think of it as fast food for academic careers: quick, convenient, and not especially good for long-term health.
Another key ingredient was the journal’s abundant use of “special issues.” These are themed collections of papers run by guest editors, usually experts invited to oversee a niche topic. In theory, this can be a great idea. In practice, it turned out to be a bit like handing over the keys to your flat and hoping nobody throws a party.
Investigations found that, in some cases, the system had been even more seriously gamed. Fake or compromised editors, reviewer suggestions pointing to non-existent experts, and tightly coordinated “peer review rings” meant papers could sail through the process with both ease and speed.
The whole thing started to fall apart when research integrity teams and independent sleuths noticed odd and concerning patterns: identical images appearing in different papers, statistical results that didn’t add up, and peer reviews that seemed to happen at impossible speed. At that point, Wiley, which by then had acquired Hindawi, stepped in and took a much closer look. The result: mass retractions, suspended special issues, and a lot of nearly identical retraction notices politely stating that the “peer review process had been compromised.”
The fallout hit so-called alternative medicine (SCAM) hard. As we all know, SCAM is an area that already faces plenty of skepticism; discovering that a sizable chunk of its literature had effectively skipped quality control didn’t help. More broadly, the episode exposed a structural problem: when publishing lots of papers becomes the goal – for journals and researchers alike – quality tends to quietly take a back seat.
After all this, the journal eCAM more or less ground to a halt. It stopped accepting new submissions, and its future as an active journal remains uncertain. Its archive is still online but now dotted with retraction notices that read like a greatest hits album of publishing failures. Meanwhile, the publisher has tightened the processes,verifying reviewer identities more carefully, restricting special issues, and deploying tools to catch suspicious patterns earlier.
If there’s a silver lining, it’s that the system eventually did what it’s supposed to do: spot the problem and correct it. But the episode is a reminder that science doesn’t run on trust alone; it runs on verification. And when that verification slips, things can go wrong at scale, and with surprising speed. Or put another way: peer review works … as long as the “peers” actually exist and know their business.
Non-specific low back pain is a common condition with substantial socioeconomic implications. Pulsed electromagnetic field (PEMF) therapy is said to generate benefits in pain reduction and improvement of physical function in patients with pain-associated disorders like osteoarthritis. However, studies had heterogeneous settings. The aim of this systematic review was to assess the effects of PEMF on pain and function on patients with non-specific low back pain.
A systematic literature search of randomized controlled trials in PubMed, MEDLINE, EMBASE, Cochrane Library, and PEDro was performed (from inception until 15/5/2023). Outcome measures assessed pain and function.
Nine randomized controlled trials with 420 participants (n = 420) were included. The studies compared PEMF vs. placebo-PEMF, PEMF and conventional physical therapy vs. conventional physical therapy alone, PEMF and conventional physical therapy vs. placebo-PEMF and conventional physical therapy, PEMF vs. high-intensity laser therapy (HILT) vs. conventional physical therapy, and osteopathic manipulative treatment (OMT) and PEMF vs. PEMF alone vs. placebo-PEMF vs. OMT alone. Five of the nine included studies showed statistically significant pain reduction and improvement in physical function in comparison to their control groups (p < 0.05). There was substantial heterogeneity among the groups of the study, with a wide range of duration (10-30 min), treatments per week (2-7/week), applied frequencies (3-50 Hz), and intensities (2mT-150mT). No serious adverse event had been reported in any study. The included studies showed solid methodological quality, with an overall score of 7.2 points according to the PEDro scale.
The authors concluded that the results of the present systematic review suggest that the use of PEMF for patients with non-specific low back pain is beneficial in terms of pain reduction and enhancement of physical function, particularly if used as an addition to conventional physical therapy modalities. It has also been shown that PEMF is a safe therapy for the treatment of non-specific low back pain. Further high-quality studies with larger sample sizes and standardized protocols are necessary. The studies should also focus on determining the optimal parameters of frequency and intensity to advance PEMF application for all pain disorders.
PEMF therapy is a non-invasive treatment widely used by physiotherapist and some practitioners of so-called alternative medicine (SCAM). It applies low-frequency electromagnetic waves to the body. PEMF therapy acts like a wireless charger for your body, sending pulses of energy that penetrate deep into your tissues to jumpstart cellular recovery. Because your body contains conductive fluids (like blood) and charged ions (like calcium and potassium), the PEMF device is said to induce very small electrical currents within your tissues. In turn, the induced current is claimed to stimulate the mitochondria. This, according to proponents, boosts the production of ATP. The pulses are also said to influence the movement of ions across cell membranes, particularly calcium which triggers a cascade of beneficial biochemical reactions, such as the release of nitric oxide, which dilates blood vessels to improve circulation. Lastly, PEMF are claimed to help “calm” the chemical signals that cause chronic inflammation, switching the cellular environment from a state of stress to a state of healing.
While these assumptions look like wild extrapolations, the more crucial question is whether PEMF is clinically effective. The present review seems to tell us that it works for back pain. Yet, I am not convinced.
Most of the RCTs are not of good quality. Many did not blind patients or therapists. Only 2 made a direct comparison against a placebo, and those did not blind patients or check the success of blinding them. As the authors noted, there is plenty of heterogeneity in the included RCTs. All this adds up to a high level of uncertainty about the effectiveness of PEMF.
In summary, I fear that the authors of this review are too optimistic and lack critical thinking in evaluating the existing evidence. My conclusion therefore differs from theirs:
Many RCTs suggest that PEMF is effective for non-specific low back pain. Yet, due to serious limitations of the primary studies, no firm conclusion can be drawn.
Yet another study by Harald Walach et al on the placebo effect has been retracted.
The objective of the paper was to determine what contributes to the size of the placebo effect in clinical drug trials by meta-regressions of randomized placebo-controlled clinical trials. The authors identified 5 diseases where data on the rates of NT were available to search for a sample of n=150 (5×30) RCTs. They extracted various study descriptors and performed meta-regressions to predict improvement in treatment and placebo groups. The investigators sampled 30 trials each from the following diagnoses:
- osteoarthritis of the knee,
- irritable bowel syndrome,
- depression,
- sleep disorders,
- migraine,
and extracted relevant information. They estimated the effects due to RTM and NT and analyzed the improvement in the placebo and treatment groups by fitting two regression models. Both models were highly significant, explaining 72% of the variance. Improvement in the placebo group can be significantly predicted by improvement in the treatment group (beta= .84), whether a study was analyzed according to intention to treat (beta= -.10) or was a multicenter study (beta= .12). Improvement in the treatment group can be explained by the improvement in the placebo group (beta= .83), whether a study was a multi-center trial (beta = -.16), and by RTM (beta= -.18). The treatment effect is smaller in sleep studies (beta= -.17).
The authors concluded that the high correlation of r= .73 between placebo improvement and treatment improvement rates is genuine and not explainable by study or disease characteristics. We conclude from our data that the placebo-effect is the major driver of treatment effects in clinical trials that alone explains 69% of the variance. This leaves only limited space for effects due to pharmacological substances. Context effects are more important than pharmacological ones in the conditions studied by us.
Walach (senior author of the paper) offered the following explanation for the findings:
“The design of a clinical study, with blinding and randomisation, fulfils the formal criteria necessary to establish a generalised entanglement correlation [6-8]. This would mean: part of the therapeutic effect of a pharmacological substance is also found in the control group; but only because an entanglement correlation was generated by the blinding and randomisation. To put it another way: it is not really possible to draw any conclusions about the true effects of verum interventions based on such studies. Rather, one would have to use very different study types and extract the effect by combining the different data, as we once proposed. If one were to take these effects really seriously, the methodological canon would crumble quite quickly. But they are also the reason why, in interventions where the specific effects may be very small, it is not possible to separate ‘real’ from ‘fake’ effects by means of blinded, placebo-controlled studies.”
In my citicism of the study of Februay 2025, I suggested an alternative explanation,
SLOPPY SCIENCE:
- The authors lumped together trials of various drugs as though they are a homogeneous entity in terms of effectiveness beyond placebo (which, of course, they are not).
- The placebo response is the measured improvement of a patient in a clinical trial after receiving a sham treatment. Yet, the authors claim “the placebo response in clinical trials has four components: regression to the mean (RTM), measurement artefacts, natural tendency (NT) of the disease, and the genuine placebo effect”. This is nonsense, and I even fear that the authors know it.
- The paper’s ‘highlight’ claims that contextual factors like expectations and doctor-patient interactions drive healing. Yet, these phenomena are seperate from the placebo-effect and were not the subject of this investigation.
- Correlation is not causation.
In May 2025, it became known that the paper was under investigarion. Now, ‘Retraction Watch’ reported that the study by Harald Walach, who may be familiar to readers, has been retracted. Thus a record number of Walach’s papers has been withdrawn.
In one now-retracted paper, Walach and his coauthors claimed the COVID-19 vaccines killed two people for every three deaths they prevented. In a different, also retracted paper, Walach and colleagues claimed children’s masks trap carbon dioxide. (They later republished the article in a different journal.)
Walach lost two papers and a university post in 2021 and now holds affiliations with the Change Health Science Institute in Basel, Switzerland, and the Next Society Institute in Vilnius, Lithuania. This retraction brings his total to four.
As ‘Retraction Watch’ wrote in May, Gideon Meyerowitz-Katz, a research fellow at the University of Wollongong in Australia known for sleuthing, told us the results of the placebo effect paper seemed to indicate “simply being in a clinical trial is the main component of healing.”
According to the undated retraction notice, the journal “did not find any evidence of deliberate attempts to mislead or any scientific misconduct on the part of the authors.” The “authors agreed that the title is misleading and the manuscript should have been more nuanced or measured,” the notice continues, indicating the researchers misrepresented a formula and included a retracted study in their analysis. Nevertheless, the authors “judge that none of these mistakes change the overall findings and conclusions.”
Stephen Rhodes, a researcher at University Hospitals Cleveland Medical Center, had criticized the study in a letter to the editor in February, citing a “number of errors that lead to some sweeping conclusions.” In the letter, Rhodes wrote those leaps “reflect a misunderstanding of what a ‘treatment effect’ is.” In a placebo-controlled trial, Rhodes observed, the measure can’t be “due to placebo.”
The retraction notice also states the retraction “has raised important questions in respect of the editorial and peer review process for this manuscript, for which the Journal bears responsibility.” David Tovey, co-editor-in-chief of the JCE, told us “as a journal editor, I have to reflect on whether this situation could have been avoided.”
In a recent post, I suggested that, for the reasons explained in previous posts, Walach is the most illustrious member of our ALTERNATIVE MEDICINE HALL OF FAME:
- Prof Walach: what lies ahead for research into SCAM?
- Walach’s new meta-analysis of homeopathy revisited
- A new review of the ‘efficacy’ of homeopathic Arnica by Harald Walach & Co
- Prof Harald Walach and the intriguing ‘Vaccination Control Group’ project
- Prof Harald Walach is really unlucky
- Is Prof Harald Walach incompetent or dishonest?
- Prof Harald Walach’s new ground breaking study of praying the Rosary
- COVID-19 vaccinations: Prof Walach wants to “dampen the enthusiasm by sober facts”
I think my suggestion might have been correct!
Retractions occur when a paper is found to be flawed due to errors, plagiarism, data fabrication, peer-review fraud or other misconduct. On this blog, we have seen our fair share of such things. Occasionally, we even managed to get a fraudulent paper retracted. A paper in THE TIMES OF INDIA sheds more light on the retraction of scientific articles.
According to the Retraction Watch Database, 40,822 research articles were retracted globally by 2023. China led with 21,999 retractions in 2023, followed by the US with 3,731 and India with 2,737. In India, academic promotions and funding often hinge on publication counts, tempting researchers to cut corners. Paper mills, which produce fraudulent studies for a fee, have exploited this pressure, flooding journals with sham research.
The consequences of this type of scientific fraud are profound, e.g.:
- In 1998, Andrew Wakefield’s study in The Lancet falsely linked the MMR vaccine to autism. Though it was retracted in 2010 after evidence of data manipulation, vaccination rates dropped, leading to measles outbreaks. Public health efforts were diverted for years to counter this misinformation.
- A 2010 Science paper claiming arsenic-based life in a California lake suggested alternative biochemistry for extra-terrestrial life, was debunked in 2012, redirecting astrobiology research and prompting journals to tighten scrutiny.
The slow pace of retraction often worsens the problem. Dubious research lingers, cited and built upon long after its flaws are known. This is alarming given the scale of the crisis. In Feb, a Nature analysis found that retraction volumes at certain institutions in India had risen over the past half decade. Yet, ranking agencies like QS, Times Higher Education, and NIRF rarely adjust for retractions, prioritising publication volume over integrity. This allows a university with a high retraction rate to climb the ranks, sending a perverse message that productivity trumps quality.
India stands at a critical juncture in its journey toward becoming a knowledge superpower. With initiatives such as the National Research Foundation prioritising research-driven innovation, the country has set ambitious goals. However, this vision is under threat as the academic evaluation system increasingly rewards superficial metrics rather than genuine quality. The urgency of this issue cannot be overstated. Addressing the retraction crisis will require a steadfast commitment to academic integrity and decisive action to safeguard the nation’s intellectual aspirations.
Ranking frameworks must incorporate retraction penalties. QS, NIRF, and Times Higher Education should adjust scores to reflect research integrity, using metrics like the ratio of retracted papers to total publications. Institutions with high retraction rates should face scrutiny and attract penalties rather than accolades. Journals must strengthen peer review. The rise of paper mills demand automated tools to detect plagiarism and image manipulation, alongside stricter reviewer vetting. Publishers like Elsevier, Wiley and Springer should lead this effort.
Institutions, on their part, must prioritise quality over quantity. The ‘publish or perish’ culture should give way to one that rewards rigorous and reproducible research. In India, UGC could revise promotion criteria and penalise dubious research publications by faculty. Also, mentoring programs for early-career researchers could help instil ethical practices. Transparency is key. Retraction notices must be prompt, detailed, and widely disseminated. A national database of retracted papers, accessible to all, could serve both as a deterrent and a resource. Individuals with a track record of dubious publications must be permanently barred from holding administrative positions.
By penalising retractions, strengthening oversight mechanisms, and fostering a culture of quality, we can restore trust. Only then will India and the world realise the true potential of science that breeds innovation.
__________________
Allow me to add a few points that are more specific to so-called alternative medicine (SCAM):
- In SCAM, we encounter many researchers who manage to go through their entire career without ever publishing a negative result for their pet therapy. Regular readers will know that I have established the ALTERNATIVE MEDICINE HALL OF FAME for these individuals.
- I have often reported that research originating in China is particularly suspect.
- An intriguing case is the one of Daniel Wirth, not least because of the reluctance of some SCAM journals to retract fraudulent research.
- As we see on an almost daily basis, much if not most of SCAM research is profoundly unreliable due to overt bias, poor methodology, misleading conclusions, etc.
Of course, such problems are not confined to SCAM, they exist in all fields. What is rather unique to the realm of SCAM is, however, this: far too many SCAM researchers seem to be ignorant of the fact that research is about FALSIFYING hypotheses. Instead they abuse science by designing studies for CONFIRMING their pet hypothesis. This, however, is not science and often it borders on fraud.
The fallout of such scientific misconduct can only be harmful to us all. I therefore plea (yet again) to journal editors, reviewers and everyone else involved to be much more vigilant and stop misleading and fraudulent papers from getting published.
In 1993, I was appointed Professor of Complementary Medicine at the University of Exeter. After settling in, I started to wonder what a suitable initial research project would be. Someone alerted me to some funds made available by the University for small projects – if I remember corretly, the amount was less that £30 000. Apparently, the grant was to be unbeaurocratic and, as I was newly appointed and everyone was wondering what I might get up to, I decided to go for it.
I sat down and recycled an idea that I had long wanted to test by never quite has the time to realize: a simple clinical trial where patients suffering from a common, benign condition [I think I chose insomnia] would be randomized into two groups. Both groups would be treated with a mild sleeping aid [I think I chose valarian]. The only difference between the two groups was that
- the experimental group would get an empathetic consultation, while
- the control group would get a normal one.
Thus the aim of the trial was to see whether empathy would impact on the outcome. My plan was that the prescribing clinician would be trained such that he/she could behave either normally or empathetically at will.
After I had submitted the proposal it was ‘peer reviewed’ by University staff. As it happened, my reviewer was John Tooke – he later became professor and medical director of the medical school at Exeter and later again moved to London and became the ‘Head of the School of Life & Medical Sciences’ at UCL but, at the time, he was just a doctor in the local hospital (if you are interested, there is more about Tooke in my memoir). Tooke’s assessment of my proposal was, to put it mildly, not positive and, of course, I never did get the funding. His critique explained that in the English language we call it ‘sympathy’ and not ’empathy’.
So, the study outlined above never got done, and every time I hear the word ’empathy’, I am reminded of this little anecdote. Like, for instance, a few days ago when I read that Elon Musk in his infinite wisdom had pronounced that: “The fundamental weakness of Western civilization is empathy”
I have to admit: this thought scares me!
The question I ask myself is this: to what extent does a neglect of emapthy contribute to the professional success of some people?
In any case, I am jolly glad that in the English language does have a word ’empathy’. I am even more profoundly glad that many people I know do have empathy. And I agree with Hannah Arendt who once wrote that “The death of human emapthy is one of the earliest and most telling signs of a culture about to fall into barbarism”
PS
sympathy = (an expression of) understanding and care for someone else’s suffering
This systematic review and meta-analysis investigated the effectiveness and safety of manual therapy (MT) interventions compared to oral or topical pain medications in the management of neck pain.
The investigators searched from inception to March 2023, in Cochrane Central Register of Controller Trials (CENTRAL), MEDLINE, EMBASE, Allied and Complementary Medicine (AMED) and Cumulative Index to Nursing and Allied Health Literature (CINAHL; EBSCO) for randomized controlled trials that examined the effect of manual therapy interventions for neck pain when compared to oral or topical medication in adults with self-reported neck pain, irrespective of radicular findings, specific cause, and associated cervicogenic headaches. Trials with usual care arms were also included if they prescribed medication as part of the usual care and they did not include a manual therapy component. The authors used the Cochrane Risk of Bias 2 tool to assess the potential risk of bias in the included studies, and the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) approach to grade the quality of the evidence.
Nine trials with a total of 779 participants were included in the meta-analysis.
- low certainty of evidence was found that MT interventions may be more effective than oral pain medication in pain reduction in the short-term (Standardized Mean Difference: -0.39; 95% CI -0.66 to -0.11; 8 trials, 676 participants),
- moderate certainty of evidence was found that MT interventions may be more effective than oral pain medication in pain reduction in the long-term (Standardized Mean Difference: −0.36; 95% CI −0.55 to −0.17; 6 trials, 567 participants),
- low certainty evidence that the risk of adverse events may be lower for patients who received MT compared to the ones that received oral pain medication (Risk Ratio: 0.59; 95% CI 0.43 to 0.79; 5 trials, 426 participants).
The authors conluded that MT may be more effective for people with neck pain in both short and long-term with a better safety profile regarding adverse events when compared to patients receiving oral pain medications. However, we advise caution when interpreting our safety results due to the different level of reporting strategies in place for MT and medication-induced adverse events. Future MT trials should create and adhere to strict reporting strategies with regards to adverse events to help gain a better understanding on the nature of potential MT-induced adverse events and to ensure patient safety.
Let’s have a look at the primary studies. Here they are with their conclusions (and, where appropriate, my comments in capital letters):
- For participants with acute and subacute neck pain, spinal manipulative therapy (SMT) was more effective than medication in both the short and long term. However, a few instructional sessions of home exercise with (HEA) resulted in similar outcomes at most time points. EXERCISE WAS AS EFFECTIVE AS SMT
- Oral ibuprofen (OI) pharmacologic treatment may reduce pain intensity and disability with respect to neural mobilization (MNNM and CLG) in patients with CP during six weeks. Nevertheless, the non-existence of between-groups ROM differences and possible OI adverse effects should be considered. MEDICATION WAS BETTER THAN MT
- It appears that both treatment strategies (usual care + MT vs usual care) can have equivalent positive influences on headache complaints. Additional studies with larger study populations are needed to draw firm conclusions. Recommendations to increase patient inflow in primary care trials, such as the use of an extended network of participating physicians and of clinical alert software applications, are discussed. MT DOES NOT IMPROVE OUTCOMES
- The consistency of the results provides, in spite of several discussed shortcomings of this pilot study, evidence that in patients with chronic spinal pain syndromes spinal manipulation, if not contraindicated, results in greater improvement than acupuncture and medicine. THIS IS A PILOT STUDY, A TRIAL TESTING FEASIBILITY, NOT EFFECTIVENESS
- The consistency of the results provides, despite some discussed shortcomings of this study, evidence that in patients with chronic spinal pain, manipulation, if not contraindicated, results in greater short-term improvement than acupuncture or medication. However, the data do not strongly support the use of only manipulation, only acupuncture, or only nonsteroidal antiinflammatory drugs for the treatment of chronic spinal pain. The results from this exploratory study need confirmation from future larger studies.
- In daily practice, manual therapy is a favorable treatment option for patients with neck pain compared with physical therapy or continued care by a general practitioner.
- Short-term results (at 7 weeks) have shown that MT speeded recovery compared with GP care and, to a lesser extent, also compared with PT. In the long-term, GP treatment and PT caught up with MT, and differences between the three treatment groups decreased and lost statistical significance at the 13-week and 52-week follow-up. MT IS NOT SUPERIOR [SAME TRIAL AS No 6]
- In this randomized clinical trial, for patients with chronic neck pain, Chuna manual therapy was more effective than usual care in terms of pain and functional recovery at 5 weeks and 1 year after randomization. These results support the need to consider recommending manual therapies as primary care treatments for chronic neck pain.
- In patients with chronic spinal pain syndromes, spinal manipulation, if not contraindicated, may be the only treatment modality of the assessed regimens that provides broad and significant long-term benefit. SAME TRIAL AS No 5
- An impairment-based manual physical therapy and exercise (MTE) program resulted in clinically and statistically significant short- and long-term improvements in pain, disability, and patient-perceived recovery in patients with mechanical neck pain when compared to a program comprising advice, a mobility exercise, and subtherapeutic ultrasound. THIS STUDY DID NOT TEST MT ALONE AND SHOULD NOT HAVE BEEN INCLUDED
I cannot bring myself to characterising this as an overall positive result for MT; anyone who can is guilty of wishful thinking, in my view. The small differences in favor of MT that (some of) the trials report have little to do with the effectiveness of MT itself. They are almost certainly due to the fact that none of these studies were placebo-controlled and double blind (even though this would clearly be possible). In contrast to popping a pill, MT involves extra attention, physical touch, empathy, etc. These factors easily suffice to bring about the small differences that some studies report.
It follows that the main conclusion of the authors of the review should be modified:
There is no compelling evidence to show that MT is more effective for people with neck pain in both short and long-term when compared to patients receiving oral pain medications.
The WHO has just released guidelines for non-surgical management of chronic primary low back pain (CPLBP). The guideline considers 37 types of interventions across five intervention classes. With the guidelines, WHO recommends non-surgical interventions to help people experiencing CPLBP. These interventions include:
- education programs that support knowledge and self-care strategies;
- exercise programs;
- some physical therapies, such as spinal manipulative therapy (SMT) and massage;
- psychological therapies, such as cognitive behavioural therapy; and
- medicines, such as non-steroidal anti-inflammatory medicines.
The guidelines also outline 14 interventions that are not recommended for most people in most contexts. These interventions should not be routinely offered, as WHO evaluation of the available evidence indicate that potential harms likely outweigh the benefits. WHO advises against interventions such as:
- lumbar braces, belts and/or supports;
- some physical therapies, such as traction;
- and some medicines, such as opioid pain killers, which can be associated with overdose and dependence.
As you probably guessed, I am particularly intrigued by the WHO’s positive recommendation for SMT. Here is what the guideline tells us about this specific topic:
Considering all adults, the guideline development group (GDG) judged overall net benefits [of spinal manipulation] across outcomes to range from trivial to moderate while, for older people the benefit was judged to be largely uncertain given the few trials and uncertainty of evidence in this group. Overall, harms were judged to be trivial to small for all adults and uncertain for older people due to lack of evidence.
The GDG commented that while rare, serious adverse events might occur with SMT, particularly in older people (e.g. fragility fracture in people with bone loss), and highlighted that appropriate training and clinical vigilance concerning potential harms are important. The GDG also acknowledged that rare serious adverse events were unlikely to be detected in trials. Some GDG members considered that the balance of benefits to harms favoured SMT due to small to moderate benefits while others felt the balance did not favour SMT, mainly due to the very low certainty evidence for some of the observed benefits.
The GDG judged the overall certainty of evidence to be very low for all adults, and very low for older people, consistent with the systematic review team’s assessment. The GDG judged that there was likely to be important uncertainty or variability among people with CPLBP with respect to their values and preferences, with GDG members noting that some people might prefer manual
therapies such as SMT, due to its “hands-on” nature, while others might not prefer such an approach.
Based on their experience and the evidence presented from the included trials which offered an average of eight treatment sessions, the GDG judged that SMT was likely to be associated with moderate costs, while acknowledging that such costs and the equity impacts from out-of-pocket costs would vary by setting.
The GDG noted that the cost-effectiveness of SMT might not be favourable when patients do not experience symptom improvements early in the treatment course. The GDG judged that in most settings, delivery of SMT would be feasible, although its acceptability was likely to vary across
health workers and people with CPLBP.
The GDG reached a consensus conditional recommendation in favour of SMT on the basis of small to moderate benefits for critical outcomes, predominantly pain and function, and the likelihood of rare adverse events.
The GDG concluded by consensus that the likely short-term benefits outweighed potential harms, and that delivery was feasible in most settings. The conditional nature of the recommendation was informed by variability in acceptability, possible moderate costs, and concerns that equity might be negatively impacted in a user-pays model of financing.
___________________________
This clearly is not a glowing endorsement or recommendation of SMT. Yet, in my view, it is still too positive. In particular, the assessment of harm is woefully deficient. Looking into the finer details, we find how the GDG assessed harms:
WHO commissioned quantitative systematic evidence syntheses of randomized controlled
trials (RCTs) to evaluate the benefits and harms (as reported in included trials) of each of the
prioritized interventions compared with no care (including trials where the effect of an
intervention could be isolated), placebo or usual care for each of the critical outcomes (refer to Table 2 for the PICO criteria for selecting evidence). Research designs other than RCTs
were not considered.
That explains a lot!
It is not possible to establish the harms of SMT (or any other therapy) on the basis of just a few RCTs, particularly because the RCTs in question often fail to report adverse events. I can be sure of this phenomenon because we investigated it via a systematic review:
Objective: To systematically review the reporting of adverse effects in clinical trials of chiropractic manipulation.
Data sources: Six databases were searched from 2000 to July 2011. Randomised clinical trials (RCTs) were considered, if they tested chiropractic manipulations against any control intervention in human patients suffering from any type of clinical condition. The selection of studies, data extraction, and validation were performed independently by two reviewers.
Results: Sixty RCTs had been published. Twenty-nine RCTs did not mention adverse effects at all. Sixteen RCTs reported that no adverse effects had occurred. Complete information on incidence, severity, duration, frequency and method of reporting of adverse effects was included in only one RCT. Conflicts of interests were not mentioned by the majority of authors.
Conclusions: Adverse effects are poorly reported in recent RCTs of chiropractic manipulations.
The GDG did not cite our review (or any other of our articles on the subject) but, as it was published in a very well-known journal, they must have been aware of it. I am afraid that this wilfull ignorance caused the WHO guideline to underestimate the level of harm of SMT. As there is no post-marketing surveillance system for SMT, a realistic assessment of the harm is far from easy and needs to include a carefully weighted summary of all the published reports (such as this one).
The GDG seems to have been aware of (some of) these problems, yet they ignored them and simply assumed (based on wishful thinking?) that the harms were small or trivial.
Why?
Even the most cursory look at the composition of the GDG, begs the question: could it be that the GDG was highjacked by chiropractors and other experts biased towards SMT?
The more I think of it, the more I feel that this might actually be the case. One committee even listed an expert, Scott Haldeman, as a ‘neurologist’ without disclosing that he foremost is a chiropractor who, for most of his professional life, has promoted SMT in one form or another.
Altogether, the WHO guideline is, in my view, a shameful example of pro-chiropractic bias and an unethical disservice to evidence-based medicine.
When I was still at Exeter, I used to do an average of about 4 peer reviews per week of articles that had been submitted to all sorts of journals for publication. Now I reject most of these invitations and do perhaps just one per month.
Why?
Conducting a peer review is by no means an easy task. You have to realize that the authors have usually put a lot of hard work into their paper and a lot may depend on it in terms of their future. They thus have the right to receive a fair and responsible review. To do the job properly, it took me (even with plenty of experience in reading scientific papers) between 1 and 3 hours per article. Crucially, low-quality articles typically submitted to low-quality journals are more work than papers that adhere to a certain standard.
I do not think that the journal editors who send the submissions out for review appreciate how much work they ask from the reviewers. They normally pay nothing (even if they charge exorbitant handling fees from the authors) and offer you no benefit at all. In addition, many have systems that are more than tedious asking you to register, create a pin number, etc., etc. Then you have to follow certain rules and formats that differ from journal to journal. In a word, they add an administrative burden to the task of reading, understanding, checking a paper, and composing your judgment on it.
All this can be cumbersome but it’s not the reason why I do less and less peer reviews. The true reason is that research papers on so-called alternative medicine (SCAM) are now mostly published in one of the many 3rd class SCAM journals that have recently sprung up. There are so many of them that they, of course, struggle to get enough articles to fill their pages. In turn, this means that they are far too keen to publish anything regardless of its quality or validity. As a consequence, the quality of these articles and their authors are often dismal.
Here is an example of a (rather shocking but not unusual) email I received only today; it might show you what I mean:
Dear Professor!
…
I want to publish some papers in “Areas related to your research field”. Can you help me? I can provide a thank you fee!
For example, I will give you a $2000 thank you fee for helping me write articles. For example, if you add my name to your article, I will give you a $1000 thank you fee. Or I can help you pay for APC.
I know this email is presumptuous, but my friends and I need to publish dozens of papers every year. If you can help me, we can cooperate for a long time. I’m not kidding, I’m very sincere!
If you are offended, please forgive me!
Look forward to your reply!
Warmly Wishes, …
When I do a review for a low-quality SCAM journal and find major defects in an article, my experience has been that the editor then decides to publish it nonetheless. When this happens, I feel frustrated and ask myself: WHY DID THEY ASK FOR MY OPINION IF THEY DO NOT ABIDE BY IT?
Thus I decided that these journals are just as well off without my contributions. So, if you are an editor of a SCAM journal, do me a favor and do not molest me with your invitations to conduct a peer review and
COUNT ME OUT!
Should Acupuncture-Related Therapies be Considered in Prediabetes Control?
No!
If you are pre-diabetic, consult a doctor and follow his/her advice. Do NOT do what acupuncturists or other self-appointed experts tell you. Do NOT become a victim of quackery.
But the authors of a new paper disagree with my view.
So, let’s have a look at the evidence.
Their systematic review was aimed at evaluating the effects and safety of acupuncture-related therapy (AT) interventions on glycemic control for prediabetes. The Chinese researchers searched 14 databases and 5 clinical registry platforms from inception to December 2020. Randomized controlled trials involving AT interventions for managing prediabetes were included.
Of the 855 identified trials, 34 articles were included for qualitative synthesis, 31 of which were included in the final meta-analysis. Compared with usual care, sham intervention, or conventional medicine, AT treatments yielded greater reductions in the primary outcomes, including fasting plasma glucose (FPG) (standard mean difference [SMD] = -0.83; 95% confidence interval [CI], -1.06, -0.61; P < .00001), 2-hour plasma glucose (2hPG) (SMD = -0.88; 95% CI, -1.20, -0.57; P < .00001), and glycated hemoglobin (HbA1c) levels (SMD = -0.91; 95% CI, -1.31, -0.51; P < .00001), as well as a greater decline in the secondary outcome, which is the incidence of prediabetes (RR = 1.43; 95% CI, 1.26, 1.63; P < .00001).
The authors concluded that AT is a potential strategy that can contribute to better glycemic control in the management of prediabetes. Because of the substantial clinical heterogeneity, the effect estimates should be interpreted with caution. More research is required for different ethnic groups and long-term effectiveness.
But this is clearly a positive result!
Why do I not believe it?
There are several reasons:
- There is no conceivable mechanism by which AT prevents diabetes.
- The findings heavily rely on Chinese RCTs which are known to be of poor quality and often even fabricated. To trust such research would be a dangerous mistake.
- Many of the primary studies were designed such that they failed to control for non-specific effects of AT. This means that a causal link between AT and the outcome is doubtful.
- The review was published in a 3rd class journal of no impact. Its peer-review system evidently failed.
So, let’s just forget about this rubbish paper?
If only it were so easy!
Journalists always have a keen interest in exotic treatments that contradict established wisdom. Predictably, they have been reporting about the new review thus confusing or misleading the public. One journalist, for instance, stated:
Acupuncture has been used for thousands of years to treat a variety of illnesses — and now it could also help fight one of the 21st century’s biggest health challenges.
New research from Edith Cowan University has found acupuncture therapy may be a useful tool in avoiding type 2 diabetes.
The team of scientists investigated dozens of studies covering the effects of acupuncture on more than 3600 people with prediabetes. This is a condition marked by higher-than-normal blood glucose levels without being high enough to be diagnosed as diabetes.
According to the findings, acupuncture therapy significantly improved key markers, such as fasting plasma glucose, two-hour plasma glucose, and glycated hemoglobin. Additionally, acupuncture therapy resulted in a greater decline in the incidence of prediabetes.
The review can thus serve as a prime example for demonstrating how irresponsible research has the power to mislead millions. This is why I have often said that poor research is a danger to public health.
And what can be done about this more and more prevalent problem?
The answer is easy: people need to behave more responsibly; this includes:
- trialists,
- review authors,
- editors,
- peer-reviewers,
- journalists.
Yes, the answer is easy in theory – but the practice is far from it!
The Lancet is a top medical journal, no doubt. But even such journals can make mistakes, even big ones, as the Wakefield story illustrates. But sometimes, the mistakes are seemingly minor and so well hidden that the casual reader is unlikely to find them. Such mistakes can nevertheless be equally pernicious, as they might propagate untruths or misunderstandings that have far-reaching consequences.
A recent Lancet paper might be an example of this phenomenon. It is entitled “Management of common clinical problems experienced by survivors of cancer“, unquestionably an important subject. Its abstract reads as follows:
_______________________
Improvements in early detection and treatment have led to a growing prevalence of survivors of cancer worldwide.
Models of care fail to address adequately the breadth of physical, psychosocial, and supportive care needs of those who survive cancer. In this Series paper, we summarise the evidence around the management of common clinical problems experienced by survivors of adult cancers and how to cover these issues in a consultation. Reviewing the patient’s history of cancer and treatments highlights potential long-term or late effects to consider, and recommended surveillance for recurrence. Physical consequences of specific treatments to identify include cardiac dysfunction, metabolic syndrome, lymphoedema, peripheral neuropathy, and osteoporosis. Immunotherapies can cause specific immune-related effects most commonly in the gastrointestinal tract, endocrine system, skin, and liver. Pain should be screened for and requires assessment of potential causes and non-pharmacological and pharmacological approaches to management. Common psychosocial issues, for which there are effective psychological therapies, include fear of recurrence, fatigue, altered sleep and cognition, and effects on sex and intimacy, finances, and employment. Review of lifestyle factors including smoking, obesity, and alcohol is necessary to reduce the risk of recurrence and second cancers. Exercise can improve quality of life and might improve cancer survival; it can also contribute to the management of fatigue, pain, metabolic syndrome, osteoporosis, and cognitive impairment. Using a supportive care screening tool, such as the Distress Thermometer, can identify specific areas of concern and help prioritise areas to cover in a consultation.
_____________________________
You can see nothing wrong? Me neither! We need to dig deeper into the paper to find what concerns me.
In the actual article, the authors state that “there is good evidence of benefit for … acupuncture …”[1]; the same message was conveyed in one of the tables. In support of these categorical statements, the authors quote the current Cochrane review entitled “Acupuncture for cancer pain in adults”. Its abstract reads as follows:
Background: Forty per cent of individuals with early or intermediate stage cancer and 90% with advanced cancer have moderate to severe pain and up to 70% of patients with cancer pain do not receive adequate pain relief. It has been claimed that acupuncture has a role in management of cancer pain and guidelines exist for treatment of cancer pain with acupuncture. This is an updated version of a Cochrane Review published in Issue 1, 2011, on acupuncture for cancer pain in adults.
Objectives: To evaluate efficacy of acupuncture for relief of cancer-related pain in adults.
Search methods: For this update CENTRAL, MEDLINE, EMBASE, PsycINFO, AMED, and SPORTDiscus were searched up to July 2015 including non-English language papers.
Selection criteria: Randomised controlled trials (RCTs) that evaluated any type of invasive acupuncture for pain directly related to cancer in adults aged 18 years or over.
Data collection and analysis: We planned to pool data to provide an overall measure of effect and to calculate the number needed to treat to benefit, but this was not possible due to heterogeneity. Two review authors (CP, OT) independently extracted data adding it to data extraction sheets. Data sheets were compared and discussed with a third review author (MJ) who acted as arbiter. Data analysis was conducted by CP, OT and MJ.
Main results: We included five RCTs (285 participants). Three studies were included in the original review and two more in the update. The authors of the included studies reported benefits of acupuncture in managing pancreatic cancer pain; no difference between real and sham electroacupuncture for pain associated with ovarian cancer; benefits of acupuncture over conventional medication for late stage unspecified cancer; benefits for auricular (ear) acupuncture over placebo for chronic neuropathic pain related to cancer; and no differences between conventional analgesia and acupuncture within the first 10 days of treatment for stomach carcinoma. All studies had a high risk of bias from inadequate sample size and a low risk of bias associated with random sequence generation. Only three studies had low risk of bias associated with incomplete outcome data, while two studies had low risk of bias associated with allocation concealment and one study had low risk of bias associated with inadequate blinding. The heterogeneity of methodologies, cancer populations and techniques used in the included studies precluded pooling of data and therefore meta-analysis was not carried out. A subgroup analysis on acupuncture for cancer-induced bone pain was not conducted because none of the studies made any reference to bone pain. Studies either reported that there were no adverse events as a result of treatment, or did not report adverse events at all.
Authors’ conclusions: There is insufficient evidence to judge whether acupuncture is effective in treating cancer pain in adults.
This conclusion is undoubtedly in stark contrast to the categorical statement of the Lancet authors: “there is good evidence of benefit for … acupuncture …“
What should be done to prevent people from getting misled in this way?
- The Lancet should correct the error. It might be tempting to do this by simply exchanging the term ‘good’ with ‘some’. However, this would still be misleading, as there is some evidence for almost any type of bogus therapy.
- Authors, reviewers, and editors should do their job properly and check the original sources of their quotes.
PS
In case someone argued that the Cochrane review is just one of many, here is the conclusion of an overview of 15 systematic reviews on the subject: The … findings emphasized that acupuncture and related therapies alone did not have clinically significant effects at cancer-related pain reduction as compared with analgesic administration alone.
