Yet another of Harald Walach’s papers is under investigation

When Harald Walach et al first published it, I did report about their paper entitled “Treatment effects in pharmacological clinical randomized controlled trials are mainly due to placebo”. The objective of their analysis was to determine what contributes to the size of the placebo effect in clinical drug trials by meta-regressions of randomized placebo-controlled clinical trials.

Walach and his co-authors concluded that the high correlation of r= .73 between placebo improvement and treatment improvement rates is genuine and not explainable by study or disease characteristics. We conclude from our data that the placebo-effect is the major driver of treatment effects in clinical trials that alone explains 69% of the variance. This leaves only limited space for effects due to pharmacological substances. Context effects are more important than pharmacological ones in the conditions studied by us.

At the time, I called this SLOPPY SCIENCE and explained:

• The authors lumped together trials of various drugs as though they are a homogeneous entity in terms of effectiveness beyond placebo (which, of course, they are not).
• The placebo response is the measured improvement of a patient in a clinical trial after receiving a sham treatment. Yet, the authors claim “the placebo response in clinical trials has four components: regression to the mean (RTM), measurement artefacts, natural tendency (NT) of the disease, and the genuine placebo effect”. This is nonsense, and I even fear that the authors know it.
• The paper’s ‘highlight’ claims that contextual factors like expectations and doctor-patient interactions drive healing. Yet, these phenomena are seperate from the placebo-effect and were not the subject of this investigation.
• Correlation is not causation.

Now RETRACTION WATCH have published interesting news about Walach’s article:

… Stephen Rhodes, a researcher at University Hospitals Cleveland Medical Center in Ohio, criticized the study in a letter to the editor in February, citing a “number of errors that lead to some sweeping conclusions.” In the letter, Rhodes wrote those leaps “reflect a misunderstanding of what a ‘treatment effect’ is,” noting that in a placebo-controlled trial, the measure can’t be “due to placebo.”

We asked Gideon Meyerowitz-Katz, a sleuth and research fellow at the University of Wollongong in Australia, to take a look at the paper. Meyerowitz-Katz called the work “bizarre” and told us the results seem to indicate “simply being in a clinical trial is the main component of healing.” The studies included in the analysis had “very low average treatment effects,” meaning they didn’t show a huge benefit for the interventions tested. The clinical trials included had another issue: one was retracted in 2018 after journal editors realized  all its participants had been enrolled and randomized on the same day. “I do not think that a meta-analysis which includes work retracted nearly a decade ago can be relied on as evidence,” Meyerowitz-Katz said. When he attempted to replicate the methods, Meyerowitz-Katz wasn’t able to do so. The researchers claimed to have taken one of the formulas from a previous paper; however, the referenced paper used a regression model, and the formula was not listed, he said. Meyerowitz-Katz also pointed out a potential undisclosed conflict of interest. Walach runs the Change Health Science Institute, which promotes “homeopathy and various COVID-19 conspiracy theories,” he said.

Walach, in response to Meyerowitz-Katz’s comments, called the term “conspiracy theory” “an analytically void terminology, because it is dependent on the political mainstream view, which is in turn dependent on political power.” He did not respond to any of Meyerowitz-Katz’s critiques of the paper.

Meyerowitz-Katz touched on many of the same issues Rhodes had raised in his letter. Rhodes questioned if the results really suggest there is “limited space for effects due to pharmacological substances,” quoting from the original paper. He also wrote by weighing clinical trials by study size rather than standard deviation, the researchers are “throwing information away.”

The researchers responded to Rhodes in their own letter to the editor, conceding they should have used “treatment response” rather than “treatment effect” in the title “to avoid confusion.” The authors also argued they had demonstrated “whenever a treatment is very effective, so is the improvement in the placebo group and vice versa.”

Retraction Watch sent questions to the Journal of Clinical Epidemiology, including the above critiques of the paper. Andrea Tricco, the co-editor-in-chief of the publication, told us the journal was investigating the concerns and was “treating this as a matter of highest urgency.” Stefan Schmidt, the corresponding author of the paper, told us his group has been asked to give “a detailed reply within 30 days.”

PS

I think it is only fair to add this note by RETRACTION WATCH:

Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on X or Bluesky, like us on Facebook, follow us on LinkedIn, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].