Guest post by Norbert Aust and Viktor Weisshäupl

Imagine you recently published an excellent and rigorous trial providing solid evidence that a certain therapy is able to help patients suffering from some inevitably fatal condition. You proved that your therapy is able to significantly prolong the patients’ lifetime, much longer than with the current state-of-the-art therapeutic approach. But the patients not only live considerably longer, but they also do so with a much better quality of life (QoL) and subjective well-being. In short: this therapy marks some progress that would otherwise take years or decades of scientific effort.

And then someone comes forward and points out your data apparently were manipulated. Essential parameters of this trial were modified sometime after data collection was completed, with the patients’ outcome and first analyses available. Thus the results were biased in a certain direction and the critics show that the results as published in your study show characteristics that such manipulations would evoke. After all, this holds an implication of scientific misconduct that could, if verified, ruin your academic reputation more or less completely.

What would you do?

Ignore the preposterous concerns because you know your methods and performance were rigorous and solid? After all, anytime some real academic criticism arises you are ready to prove your findings are well-founded results of accepted scientific methods. Or would you publish data or documents that your critics were too ignorant to find or to understand, and thus to stop such rumours once and for all? Maybe you could even clarify some of the issues raised by those critics, maybe add some follow-up information or data to ensure no more misunderstandings occur. Or would you try to find some clues for a libel lawsuit?

Well, we thought some of the above would happen after we contacted the authors of the recent study on adjunct homeopathy in non-small cell lung cancer. On that date, we forwarded our detailed analysis to the lead author and all the co-authors.

Of course, we even considered the possibility, not very likely though, that we would receive some explanation for the numerous exclusion criteria while other serious conditions that coincide with advanced age did not preclude enrollment. Or an updated CONSORT diagram accounting for the patients excluded. Or some explanation just why the numerous amendments to the protocol were necessary but not important enough to mention them in the published paper.

But nothing of this happened as yet (July 2021). Instead on June 14 and 16, 2021, not two weeks after our messages to the authors, the registration data at ClinicalTrials were updated once again and a new version of the protocol was uploaded [3]. And this update looks pretty much like it is meant to cover up and blur the former data that we based our analysis on. Of course, these data and the former version of the protocol are available still – just one layer further down, and you have to scroll to the bottom of the page to find the small link “history of changes”. Maybe not many visitors will do that.

In contrast to the versions before, now the uploaded data are in line with the study as published, namely, they include a full list of the exclusion criteria and the reduced observation time for QoL, which was the primary outcome. Note: throughout the trial until the end of data assessment those parameters were set with pregnancy as the only exclusion criterion and two years follow up time, only to be amended in the protocol uploaded two months after data collection was complete and analysis presumably was well underway.

In addition, there is a new version of the study protocol, this one dated Feb. 6, 2014. Of course, this protocol is fairly new, in spite of the date it carries. Why would the older version allegedly from January 2011 be uploaded to the register in September 2019, if this more actual version already had existed and was available?

Contrary to the prior version all the clues are removed that would indicate that this document was finished at a much later point in time than given in its date: References to some future software versions that were released years after the protocol was allegedly compiled are dropped. And this strange literature reference “25” that corresponds to the reference list in the final study as published but is pointless in the protocol without any reference list, is removed too. And of course, again contrary to the prior version, the exclusion criteria are identical with the final study as is the shortened follow-up time for QoL.

New to the protocol is a section “Bringing in the patient’s voice”, where the authors disclose how they want to “systematically research the ethical, legal, socio-political, and science theoretical dimensions of homeopathy as in the case of lung cancer (non-small-cell lung carcinoma) exemplified” in some “integral social scientific study”, where some “focus groups” of 4 to 10 participants together with their relatives, friends and caregivers included should be used to study “interactions between individuals, collectively shared and uncontested assumptions, and the emergence of collective meaning”.

But from all of this more or less meaningless but very sciency sounding socio-speak, not a single word found its way into the study. Nothing. So it is pointless to try to figure out what the content of this part of the investigation is all about.

Why then was this chapter added? This “integral social scientific study” was to start after the “third or fourth homeopathic treatment” (But why should patients not be included in this “research” from the very first beginning?). Is it perhaps to give some rationale why the follow-up time for QoL was to end after the third homeopathic treatment?

So what we see, when we look up the study at ClinicalTrials now, is a perfectly matching set of data and a protocol that corresponds to the study as published and looks as if it was published at a time where the trial was underway and the patients were still blinded. If you do not look very closely everything now appears to be perfect.

And here we would like to forward some critique to the register: The purpose of the study register is to prevent not only publication bias but misleading manipulation from happening as well. They do quite a good job in preserving former versions of data and documents and keeping them available to the public. Many national study registers do not offer this service. But you must be of a suspicious mind and of some persistence to actively search and find and check the history of modifications. Thus, a cover-up like the one we are witnessing here might well prove successful. We, therefore, propose to improve the presentation of the registration: If vital amendments occurred that may affect the outcomes – such as protocol changes, extensions of exclusion criteria, modifications of follow-up time – this should be indicated upfront in the study’s record instead of some small hint to “history of changes” at the very bottom of the page.

In conclusion, there appears to be no proof that the results of the study were produced using rigorous scientific methods. And the issues we raised in our report to the authors remain unresolved.