scientific misconduct
I recently discussed the incredible paper by Walach et al. To remind you, here is its abstract again:
COVID-19 vaccines have had expedited reviews without sufficient safety data. We wanted to compare risks and benefits.
Method: We calculated the number needed to vaccinate (NNTV) from a large Israeli field study to prevent one death. We accessed the Adverse Drug Reactions (ADR) database of the European Medicines Agency and of the Dutch National Register (lareb.nl) to extract the number of cases reporting severe side effects and the number of cases
with fatal side effects.
Result: The NNTV is between 200–700 to prevent one case of COVID-19 for the mRNA vaccine marketed by Pfizer, while the NNTV to prevent one death is between 9000 and 50,000 (95% confidence interval), with 16,000 as a point estimate. The number of cases experiencing adverse reactions has been reported to be 700 per 100,000 vaccinations. Currently, we see 16 serious side effects per 100,000 vaccinations, and the number of fatal side effects is at 4.11/100,000 vaccinations. For three deaths prevented by vaccination we have to accept two inflicted by vaccination.
Conclusions: This lack of clear benefit should cause governments to rethink their vaccination policy.
In my post, I pointed out that the analysis was deeply flawed and its conclusion ridiculous. Many other observers agreed, and several editorial board members of the journal, Vaccines, that unbelievably had published this junk resigned. Yesterday, the journal reacted by retracting the paper. Here is their statement:
The journal retracts the article, The Safety of COVID-19 Vaccinations—We Should Rethink the Policy.
Serious concerns were brought to the attention of the publisher regarding misinterpretation of data, leading to incorrect and distorted conclusions.
The article was evaluated by the Editor-in-Chief with the support of several Editorial Board Members. They found that the article contained several errors that fundamentally affect the interpretation of the findings.
These include, but are not limited to:
The data from the Lareb report (https://www.lareb.nl/coronameldingen) in The Netherlands were used to calculate the number of severe and fatal side effects per 100,000 vaccinations. Unfortunately, in the manuscript by Harald Walach et al. these data were incorrectly interpreted which led to erroneous conclusions. The data was presented as being causally related to adverse events by the authors. This is inaccurate. In The Netherlands, healthcare professionals and patients are invited to report suspicions of adverse events that may be associated with vaccination. For this type of reporting a causal relation between the event and the vaccine is not needed, therefore a reported event that occurred after vaccination is not necessarily attributable to vaccination. Thus, reporting of a death following vaccination does not imply that this is a vaccine-related event. There are several other inaccuracies in the paper by Harald Walach et al. one of which is that fatal cases were certified by medical specialists. It should be known that even this false claim does not imply causation, which the authors imply. Further, the authors have called the events ‘effects’ and ‘reactions’ when this is not established, and until causality is established they are ‘events’ that may or may not be caused by exposure to a vaccine. It does not matter what statistics one may apply, this is incorrect and misleading.
The authors were asked to respond to the claims, but were not able to do so satisfactorily. The authors were notified of the retraction and did not agree.
Two questions still remain to be answered:
- Were Walach et al just incompetent or did they wilfully try to mislead us?
- How much nonsense is Walach allowed to publish before he is finally stopped?
Prof Harald Walach is well-known to regular readers of this blog (see, for instance, here, here, and here). Those who are aware of his work will know that he is not an expert in infectious diseases, epidemiology, virology, or vaccinations. This did not stop him to publish an analysis that questions the safety and rationale of the current COVID-19 vaccination programs. Here is the abstract:
COVID-19 vaccines have had expedited reviews without sufficient safety data. We wanted to compare risks and benefits.
Method: We calculated the number needed to vaccinate (NNTV) from a large Israeli field study to prevent one death. We accessed the Adverse Drug Reactions (ADR) database of the European Medicines Agency and of the Dutch National Register (lareb.nl) to extract the number of cases reporting severe side effects and the number of cases
with fatal side effects.
Result: The NNTV is between 200–700 to prevent one case of COVID-19 for the mRNA vaccine marketed by Pfizer, while the NNTV to prevent one death is between 9000 and 50,000 (95% confidence interval), with 16,000 as a point estimate. The number of cases experiencing adverse reactions has been reported to be 700 per 100,000 vaccinations. Currently, we see 16 serious side effects per 100,000 vaccinations, and the number of fatal side effects is at 4.11/100,000 vaccinations. For three deaths prevented by vaccination we have to accept two inflicted by vaccination.
Conclusions: This lack of clear benefit should cause governments to rethink their vaccination policy.
I hesitate to comment because some could think that I have a personal grudge, as Walach propagated lies about me. And crucially, like he, I am not a vaccination expert. Yet, I feel I ought to point out that the data that form the basis of Walach’s calculations should not be used in this way for at least two reasons.
- Death after vaccination does not mean that this event was caused by the vaccine. For example, if someone had a fatal accident after vaccination, it would count as a vaccine incident according to Walach’s calculation.
- Vaccine effectiveness cannot be measured by calculating how many people must receive a vaccine to prevent one case of COVID-19 vaccination. Since vaccines have a protective effect on the community, this would be an outright miscalculation. The more people who receive a vaccine, the fewer people need to receive it to prevent a single case. This situation is the exact opposite of what Walach assumes in his paper.
Conclusion: amongst all his previous nonsense, Walach’s new publication stands out, I feel, as the most stupid and the most dangerous. The mistakes seem too obvious to not be deliberate. Let’s hope the journal editor in chief (who failed miserably when publishing this idiocy) has the wisdom to retract it swiftly. One of its editors already tweeted:
I have resigned from the Editorial Board of
following the publication of this article. It is grossly negligent and I can’t believe it passed peer-review. I hope it will be retracted.
And another ed-board member had this to say:
According to one website, electromagnetic fields (EMFs) are “the new smoking“:
For decades, a group of cigarette companies referred to as ‘Big Tobacco’ financed bogus scientific studies claiming smoking was perfectly safe. This tricked doctors, scientists, politicians, and smokers into a false sense of security. There were early warning signs that smoking was dangerous, but it took 50 years for the government to finally take action. Today we’re facing an even bigger health threat… EMFs. Even if many doctors, politicians and Big Wireless still claim that EMFs are perfectly safe, the early warning signs could not be clearer:
- Many leading EMF scientists say EMFs should be classified as a “Class 1” definite carcinogen (just like smoking and asbestos)
- The best functional medicine doctors like Dr. Dietrich Klinghardt, MD, PhD have observed that EMFs are at the very root cause of “Mystery” symptoms including insomnia, fatigue, depression, and digestive issues.
- New technologies like the “5G” (fifth generation) networks are being rolled out at a frantic pace, while exactly ZERO biological studies prove their safety.
- EMF “safety” standards haven’t been updated since 1996, and are based on short-term exposure to ONE device.
The Atox Bio Computer is one of many devices marketed as the solution. It is a little device that supposedly protects any person who is gullible enough to buy it from electrosmog and other EMFs. It was developed by the Russian physicist, Alexander Tarasov. Worn around the neck, the device allegedly acts by “converting negative information into positive”. Alarmingly, the Atox is also promoted as protection against ionizing radiation. Pseudo-scientific explanations are given for the mode of action, in which there is talk of an ominous “energy-information component” of radiation:
“The revolutionary insight of Dr. Tarasov is that any electromagnetic radiation of any origin consists of two components, the physical and the energy-information component. Whereby the energy-information component precedes the physical vibration and primarily affects the human organism or its bioenergetic field.”
Sounds weird? Yes, I agree! But it must be true because it is supported by a real professor from a leading medical school. In 2007, it was reported in a press release that Prof. Dr. Michael FRASS examined the ATOX Bio Computer and found that 90% of people with too low and 100% with too high initial values achieved normalization of their vegetative performance. Altogether, 92.9% of the persons benefited by wearing the ATOX biocomputer.
With regard to the ratio of sympathetic to parasympathetic impulses, 100% of the people with too low and 82.3% with too high initial values normalize their range of the total autonomic power. Overall, 84.2% of the treated individuals showed a positive course under the influence of ATOX biocomputer. According to Frass, this means that people with a high stress factor have a very high probability of returning to normal values of the autonomic system with the help of the ATOX biocomputer.
Considering the fact that such findings, if true, would necessitate to re-write large parts of the textbooks of physics and medicine, it is surprising that Frass does not include them in his CV. Perhaps he is a deeply modest scientist? Or maybe he does not want to spoil his chances for a Nobel?
Prof. Frass has, of course, featured on this blog before. For instance, because his many studies of homeopathy are invariably positive, or because his results have been shown to contain a few (pro-homeopathy) ‘errors’, or (most recently) because he published a trial of homeopathy that claimed lung cancer patients live longer if they are treated with homeopathy. The latter study is now under investigation for fraud.
Had such an investigation been initiated in back 2007 when Frass came out with his ATOX Bio Computer study (which incidentally was never properly published [at least I could not find it on Medline]), we would now not need to worry whether some desperate cancer patients did take Frass’ ‘science’ seriously.
Ever since I published a post about the irresponsible and aggressive advertising campaign of LYMA (“the world’s 1st super-supplement”), I am pursued by them with emails informing me about the wonders of this supplement. Here is one I received recently:
Here at LYMA we are firm believers that optimal productivity depends on good quality sleep and your day is only as good as the previous night.
Suffering from bad sleep is debilitating whether it’s ourselves or we’re watching someone we love suffer, the search for good rest is something we’re all united in.
Energy levels, positive mindset and strong cognitive function all come from sleep, which is why we spent so long formulating the LYMA supplement. Our patented KSM-66® Ashwagandha is the highest-quality, zero toxicity, concentrated Ashwagandha root in the world. The hefty combination of purity and potency make it unrivalled in its ability to reduce inflammation, neutralise anxiety and promote deep, restful sleep, night after night.
Thousands of customers have told us that after years of bad sleep, they’re finally getting the rest they need and feeling transformed as a result. In fact, it’s one of the very first benefits most people notice. We’re happy to hear it.
And the knock-on effects of a good night’s sleep in how we feel, how we perform and our overall health are far reaching. Which is why we are so delighted to welcome Michael Grandner, world-renowned sleep expert and Director of the Behavioural Sleep Medicine Clinic, Arizona to the LYMA team.
Michael is one of the most cited sleep experts in the world and has himself published over 175 articles on issues relating to sleep and health. We plan on tapping into every area of his expertise to understand our own sleep habits and how we can all become the best at rest.
To introduce Michael to the LYMA community we’re hosting a seminar dedicated to understanding sleep on Tuesday 22nd June…
I was tempted to discard all this as rather pathetic advertising hype. But then I had second thoughts. This text does after all make several medical claims, and the question is: ARE THEY SUPPORTED BY EVIDENCE?
It claims that KSM-66® Ashwagandha:
- is the highest-quality, zero toxicity, concentrated Ashwagandha root in the world.
- That the hefty combination of purity and potency makes it unrivalled in its ability to reduce inflammation.
- That the product neutralises anxiety.
- That it promotes deep, restful sleep, night after night.
I ran a few searches to find out whether there is any sound evidence for any of these claims.
- There seem to be several supplements that contain,KSM-66® Ashwagandha’. The impression that LYMA is the only one is thus wrong. Zero toxicity must also be wrong; not even water has zero toxicity. In fact, epigastric pain/discomfort and loose stools were reported as most common (>5%); and giddiness, drowsiness, hallucinogenic, vertigo, nasal congestion (rhinitis), cough, cold, decreased appetite, nausea, constipation, dry mouth, hyperactivity, nocturnal cramps, blurring of vision, hyperacidity, skin rash and weight gain have all been associated with the herbal remedy. Moreover, if it is true that Ashwagandha stimulates the immune system, it might cause problems for people with autoimmune diseases.
- I found no compelling evidence from clinical trials to show that KSM-66® Ashwagandha reduces inflammatory conditions in humans.
- I found a study concluding that Ashwagandha given as an adjunct offered some potential advantages as a safe and effective adjunctive therapy to SSRIs in GAD. Yet, I found no compelling evidence from clinical trials to show that KSM-66® Ashwagandha as a single supplement reduces anxiety in otherwise healthy individuals.
- A 2021 study suggested that Ashwagandha root extract can improve sleep quality and can help in managing insomnia. Yet the authors cautioned that additional clinical trials are required to generalize the outcome.
So, what does that tell us?
It could mean that:
- My searches were not sufficiently thorough and that I have missed compelling evidence. If so, I would appreciate, if the LYMA promoters would show me their evidence so that I can assess it.
- The LYMA people are irresponsible and mislead the public with untenable claims.
I am looking forward to their response.
On 7/10/2020, I discussed a study suggesting that homeopathy improves the quality of life and survival of cancer patients. Now, these data have been carefully scrutinized by a group of members of the „INH“ and „Initiative für Wissenschaftliche Medizin“.
By guest bloggers Norbert Aust and Viktor Weisshäupl
Abstract
The first impression of the results of the study on the adjunctive homeopathic treatment of patients with non-small cell lung cancer (NSCLC) is that of a seemingly rigorous trial with valid results. But a more thorough review yields different insights:
- The methods and definitions were pre-determined in a protocol and seem to have been maintained up to the end. But the date given in the document pointing at some point in time before enrollment began is wrong and misleading: This protocol was first published by uploading it to the register only two months after data assessment was completed with outcomes presumably available.
- The data initially saved to the register are not in agreement with the information given in the published paper: important definitions were subjected to considerable modifications while the study was underway. None of these modifications are mentioned in the paper, neither a rationale nor a comment of their impact on the results was provided.
- Some of the modifications with presumably heavy impact on the results were introduced with the upload of the protocol only, that is two months after data collection was completed. These were (a) a massive extension of the exclusion criteria: the number increased from 1 during initial registration to 20 in the final paper. and (b) an equally massive reduction of the follow-up time for the primary endpoint from two years to 18 weeks.
- The paper discloses no reason why the additional exclusion criteria were introduced. Their selection seems arbitrary without any apparent necessity arising from the trial itself.
- The patients who did not meet the added criteria and were thus excluded are not mentioned in the publication. The CONSORT flow chart does not give information either of their number or of the point in time when they were excluded.
- The survival curves of the placebo and verum groups show some aspects that arise if the inter-group difference was due to the exclusion of unfavorable data.
- It is hard to imagine that, in this trial, the homeopathic preparations had strong effects on the patients’ health, while other rigorous studies or systematic reviews failed to notice such effects.
Altogether, it seems much more plausible to assume that the positive results were achieved by post hoc data manipulation, namely by omitting patients with unfavorable outcomes, than by rigorous and valid science. A retraction of the paper seems the only appropriate measure to avoid misleading the public.
Introduction
Due to its outstanding results, the study about adjunct homeopathic treatment of non-small cell lung cancer patients was met among homeopaths with enthusiasm. However, in this article, we will show that the enthusiasm is unjustified because the results may not be based on a rigorous trial meeting established scientific criteria. Crucial definitions were modified, while the study was underway or even after data collection was completed. It stands to reason that this introduced bias in favor of homeopathy.
For this analysis, we considered the following sources of information :
- the text of the published paper (link)
- the data that were uploaded during registration (link)
- the history of changes of the registered data (link)
- the study protocol included in the registration (link)
As all of this information is readily available on the internet, it is easy to double-check our findings and verify our statements. We also submitted a letter to the editor of the Oncologist, the journal where the paper was published which has not yet been published (status 06-06-2021).
The study
At first glance, the study meets the requirements for reliable evidence.
- There is a study protocol dated January 11, 2011, well before recruiting of participants started. It provides definitions that were used until the end.
- The study was registered at ClinicalTrials.gov before recruiting started.
- The methods of randomization and blinding are suitable to meet the requirements for a low risk of bias rating.
- The presentation of the paper follows the principles set out in the CONSORT-Statement.
- The paper was published in a peer-reviewed journal of some reputation.
The study yielded formidable results in favor of homeopathy: In the group that received the adjunctive homeopathic treatment, the quality of life improved continuously throughout the follow-up time, while the patients in the placebo group deteriorated. In addition, the median survival time was only about two-thirds compared to the patients in the homeopathy group. However, the impression of a valid study does not stand up to closer scrutiny when the history of changes is taken into account.
Changes in study parameters
Between the initial registration and data upload in January 2012 (Link), shortly before recruiting started in February 2012, and the publication in October 2020, multiple changes in essential study parameters occurred:
| Registration January 2012 | Publication October 2020 | |
| Number of participants | 600 | 150 |
| Number of study arms | 2 | 3 |
| Number of exclusion criteria | 1 | 20 |
| Follow-up time for Quality of life | 104 weeks (*) | 18 weeks |
| Number of cancer types | 3 | 1 |
| (*) Derived from “Time Frame: 7 Years” minus the recruitment period of 5 years. | ||
Note the drastic reduction in the follow-up time for quality of life by more than 80 % which was defined as the primary endpoint. Furthermore, note the substantial increase in the number of exclusion criteria. Both issues will be discussed in more detail below.
In contrast to the requirements for a rigorous and valid trial, these modifications are not mentioned in the published paper, and no rationale is given as to why they became necessary. As a consequence, the authors do not discuss the possible impact these modifications may have had on the results.
The study protocol
A study protocol is available in the registration database. It was first uploaded on September 18, 2019, about two months after the end of data collection in July 2019 (Link). The document itself is dated January 11, 2011, which would place it about a year before the study was registered. However, this date is obviously wrong: there are substantial discrepancies between the parameters specified in the protocol and the data provided one year later during initial registration:
| Protocol, allegedly January 2011 | Registration January 2012 | |
| Number of participants | 300 | 600 |
| Number of study arms | 3 | 2 |
| Number of exclusion criteria | 9 | 1 |
| Follow-up time for Quality of life | 18 weeks | 104 weeks (*) |
| Number of cancer types | 1 | 3 |
| (*) Derived from “Time Frame: 7 Years” minus the recruitment period of 5 years. | ||
We see no sensible explanation why the parameters given in the study protocol allegedly compiled in January 2011 are in line with the publication nine years later, but not with the registration only one year after the protocol was compiled. The only sensible conclusion seems to be that this protocol was not completed on the date indicated, but at a much later point in time, maybe just shortly before its upload (September 2019). This impression is corroborated by the information presented in the document that was not available on the date given: On page 10 the software package used in data analysis is referenced as “IBM SPSS statistics 25.0” while, at the beginning of 2011, when the protocol was allegedly compiled, the current version number of this package was 19 only.
A second clue: Also on page 10 there is a reference “(EORTC-QLQ-C30 remaining dimensions; SF-36; subjective well-being)25.” with the number 25 indicating some reference. And some references that is, but not in the protocol – this does not have any references – but in the published paper, where the 25 indicates a paper on the SF-36 questionnaire. So it stands to reason that the number in the protocol originates from some messed up copy and paste procedure from the draft of the paper. Which would indicate that the paper and the protocol were at least partially developed in parallel.
It seems therefore reasonable to assume, that the protocol was finished only shortly before it being uploaded in September 2019, that is two months after data collection was completed.
However, the obviously inaccurate date given in the protocol supports the impression that the study parameters were set a year before the study began and were consistently maintained during the course of the trial, which is not the case, as the above tables show.
Change in exclusion criteria
The initial registration data list pregnancy as the only exclusion criterion. But with the upload of the protocol, which took place two months after data collection was completed, the number of exclusion criteria was increased to nine, only to be enlarged once again in the final publication to the final number of twenty. It is beyond any doubt that at least the final increase of eleven criteria took place after the data collected from the patients were available. But all this is neither disclosed in the final paper nor is there any rationale given for this action.
The patients excluded by the additional criteria never appear anywhere, they are not included in the CONSORT-flow-chart, Fig 3 in the study. It is obvious that some patients were excluded: What was the reason to define such an abundance of criteria, if they were not to be applied? As a consequence, the CONSORT diagram seems to be incomplete which would be in violation of the CONSORT statement.
Thus, an unknown number of patients seems to have been excluded from the study by criteria defined at a time after data collection was completed with outcomes available. After all, eleven of the exclusion criteria were established even after the protocol had been uploaded, at least those were established well after the patients’ results were available.
This raises the question of why these exclusion criteria were introduced. One would assume that an intervention to treat stage III and stage IV lung cancer patients should be effective under the conditions that are usually present in such patients. One would expect that patients somewhat advanced in age, like in this study, usually suffer from some health problems, regardless of their cancer condition. What is the sense of excluding patients with hematological, hepatic, or renal pathology, with coronary heart disease or rheumatism? Homeopathy is claimed to be able to treat comorbidities based on the assessment of symptoms independent of what disease they belong to. And this apparently was the idea at the start of the trial where only pregnancy was specified as an exclusion criterion, while it was understood that elderly patients to be enrolled in the study would suffer from some additional medical problems.
On the other hand, not all health conditions that are associated with advanced age were excluded. Diabetes, hypertension, gastrointestinal diseases, or COPD were no reason to exclude any patient from participation. Only very few of the criteria are somewhat self-explanatory as to why they were defined as exclusion criteria, e.g. if a patient was unwilling to give her informed consent.
Altogether, the assumption seems reasonable that more patients had participated in the trial than accounted for in the publication, and that an unknown number of them were excluded according to criteria that were not present until after data collection was completed. If so, a substantial bias was introduced.
Median survival time
Here, we will focus on the comparison between the homeopathy and placebo groups and leave aside the third group not receiving any additional treatment at all.
If the favorable result in survival really was established by dropping unfavorable data, this might be recognized in some characteristics of the survival curves. Therefore, we modeled this situation starting with two random distributions somewhat tweaked to resemble the typical shape of natural survival functions.
This graph shows the two distributions (n = 80) defined in the range of 0 to 200 as thin lines. Both are very similar to each other with median survival at 27 weeks. If 15 of the 20 patients with the shortest survival are dropped from the thin blue line this would result in the solid blue line (“Hom”). If, on the other hand, 15 out of 20 patients with the longest survival are dropped from the other distribution this would yield the solid red line (“Plac”).
The new functions show some characteristic properties:
- In the red line, median survival drops by 8 weeks to 19 weeks.
- In the blue line median survival rises by 12 Weeks to 39 weeks.
- The difference between the two functions arises from of the first 12 weeks alone. With the blue line, 8 people died, with the red line 23 people died during the first 12 weeks. After week 12 up to week 80, the same number of fatalities occur in both groups (blue: 36, red: 37).
After week 80, the two functions start to converge, which is due to the fact that at some future point all the patients of both groups will be dead. The survival functions that are reported in the study show the same characteristics.
Assuming that homeopathy did not have any effect, both groups should show more or less identical survival functions. In the paper 10.1 months = 303 days is cited from literature as to be expected under conventional care, maybe with some margin to the better because the data that yielded this value of 10.1 months were more than six years old at the time of the trial. The survival functions allegedly found in the trial show:
- Median survival time with the placebo group is reduced by 46 days compared to conventional care alone.
- Median survival time with the homeopathy group is increased by 132 days compared to conventional care alone.
- The advantage of homeopathy arises within the first 9 weeks alone, where only two patients died (out of 51) in the homeopathy group compared to 11 (out of 47) in the placebo group. After this initial phase, the groups developed in parallel: By the end of the two-year follow-up time an additional 26 patients died with homeopathy, and about the same, namely 25 patients died with placebo.
The inevitable convergence of both functions apparently started outside the two-year follow-up. In other words, the survival functions given in the study for placebo and homeopathy treatments show characteristics that match what you would be expected, if two very similar functions were manipulated by dropping unwanted results, i.e. “good” survival data from placebo and “bad” survival data from homeopathy functions. After week 9, the two functions develop parallel to each other, indicating a lack of effect of homeopathy even though the treatment continued until the death of the patient or the end of the study. However, with ongoing effective treatment, the functions should continue to diverge. It seems implausible that homeopathy should be effective on a short time basis only, with a sudden complete loss of effectiveness later on.
Reduction of observation time
Quality of life was defined as the primary endpoint. On initial registration, it was specified that patients should be observed for the entire seven-year duration of the study which, allowing for the recruitment period of five years, results in a follow-up time of two years or 104 weeks for each individual patient. According to the information provided in the study, this was indeed done: “Patients were followed up every nine weeks until death” (or until the end of the study, of course), and questionnaires were completed to determine the quality of life.
The reduction of follow-up time from 104 to 18 weeks was first introduced when the protocol was uploaded. So it is obvious that this substantial reduction occurred after data collection was completed and that data from more than 80 % of the originally defined follow-up were omitted.
Incomplete outcome reporting, especially when a larger scope was defined at the beginning of the study, is considered a source of substantial bias and a major shortcoming in clinical trials: Maybe patients initially experienced an improvement in their quality of life due to whatever effect – but what were the results after this initial phase? Why were they omitted? Perhaps because they got worse than in the placebo group? The long-term development would have been a vital aspect for the evaluation of efficacy – and the study originally was designed to evaluate such long-term effects. Yet, the authors’ conclusions on the quality of life – notably: the primary outcome criterion – are based on less than 20 % of the follow-up in which a positive effect may have occurred due to bias or by chance. To extrapolate from this short time to the total period is not justified and may be misleading. A detailed review of the quality of life results is meaningless: they do not disclose any long-term effects and they are subject to bias caused by the post hoc exclusion of patients anyway.
Study results
The overall evidence on the effectiveness of homeopathy is not encouraging. The quintessence of all systematic reviews that have looked at homeopathy as a whole is that some marginal effect may be found, if all studies are included in the review, regardless of their quality. But this result is questionable due to the generally low quality of the primary studies (Link, in German). However, when quality is taken into account, the systematic reviews do not produce robust evidence for any positive effect beyond placebo. In addition, no review could identify a single condition in which homeopathy is of well-established therapeutic benefit.
This study on NSCLC contradicts the long-established and often-confirmed evidence. During the follow-up time for the patients who actually received the prescribed homeopathic preparations, the quality of life improved steadily in all subscales – even down to the patients’ financial situation – whereas the opposite was observed in the placebo patients. In addition, the mean survival time was about two-thirds longer for the homeopathy patients than for the placebo group.
After 200 years of clinical research into homeopathy, it seems unlikely that such a powerful effect of homeopathy should not have been noticed before. Another scenario seems to be much more plausible:
- The survival times of the placebo group were worse than the data from the literature. This could be due to the fact that patients with relatively good outcomes were excluded by the introduction of additional exclusion criteria.
- The survival times of the homeopathy group were considerably better than expected. This could also be due to the additional exclusion criteria, in that patients with poor outcomes were excluded retrospectively.
- The long time frame where the survival functions run in parallel from week 9 onwards until the end of the two years observation period indicates the lack of effect of the homeopathic treatment. The advantage occurring in the first nine weeks alone seems to be the result of unwanted data being dropped.
- In the case of quality of life (after all, not a “hard” criterion, but based on information from the patients ), the advantage in survival would have initially created a positive effect for the homeopathy group. Then, reporting was discontinued, once the initial positive effects presumably caused by the selective omission of patients had ended.
In conclusion, it seems likely that the substantial modifications of crucial study parameters that occurred after the study had been started and results had become available biased the results in favor of homeopathy. Therefore, this study does not meet strict scientific standards that were established to exclude any confounding factors or biases. If our analysis is correct, the results of this study are invalid, and the authors’ conclusions are not justified. Retraction of this study seems to be appropriate.
Reference
[1] Frass M, Lechleitner P, Gründling C et al. Homeopathic Treatment as an Add-On Therapy May Improve Quality of Life and Prolong Survival in Patients with Non-Small Cell Lung Cancer: A Prospective, Randomized, Placebo-Controlled, Double-Blind, Three-Arm, Multicenter Study. The Oncologist 2020;25:e1930–e1955 https://theoncologist.onlinelibrary.wiley.com/doi/epdf/10.1002/onco.13548I have not often seen a paper reporting a small case series with such an impressively long list of authors from so many different institutions:
- Hospital of Lienz, Lienz, Austria.
- WissHom: Scientific Society for Homeopathy, Koethen, Germany; Umbrella Organization for Medical Holistic Medicine, Vienna, Austria; Vienna International Academy for Holistic Medicine (GAMED), Otto Wagner Hospital Vienna, Austria; Professor Emeritus, Medical University of Vienna, Department of Medicine I, Vienna, Austria. Electronic address: [email protected].
- Resident Specialist in Hygiene, Medical Microbiology and Infectious Diseases, Außervillgraten, Austria.
- St Mary’s University, London, UK.
- Umbrella Organization for Medical Holistic Medicine, Vienna, Austria.
- Shaare Zedek Medical Center, The Center for Integrative Complementary Medicine, Jerusalem, Israel.
- Apotheke Zum Weißen Engel – Homeocur, Retz, Austria.
- Reeshabh Homeo Consultancy, Nagpur, India.
- Umbrella Organization for Medical Holistic Medicine, Vienna, Austria; Vienna International Academy for Holistic Medicine (GAMED), Otto Wagner Hospital Vienna, Austria; Chair of Complementary Medicine, Medical Faculty, Sigmund Freud University Vienna, Austria; KLITM: Karl Landsteiner Institute for Traditional Medicine and Medical Anthropology, Vienna, Austria.
- WissHom: Scientific Society for Homeopathy, Koethen, Germany.
In fact, there are 12 authors reporting about 13 patients! But that might be trivial – so, let’s look at the paper itself. The aim of this study was to describe the effect of adjunctive individualized homeopathic treatment delivered to hospitalized patients with confirmed symptomatic SARS-CoV-2 infection.
Thirteen patients with COVID-19 were admitted. The mean age was 73.4 ± 15.0 (SD) years. The treating homeopathic doctor was instructed by the hospital on March 27, 2020, to adjunctively treat all inpatient COVID-19 patients homeopathically. The high potency homeopathic medicinal products were administered orally. Five globules were administered sublingually where they dissolved, three times a day. In ventilated patients in the ICU, medication was administered as a sip from a water beaker or 1 ml three times a day using a syringe. All ventilated patients exhibited dry cough resulting in respiratory failure. They were given Influenzinum, as were the patients at the general inpatient ward.
Twelve patients (92.3%) were speedily discharged without relevant sequelae after 14.4 ± 8.9 days. A single patient admitted in an advanced stage of septic disease died in the hospital. A time-dependent improvement of relevant clinical symptoms was observed in the 12 surviving patients. Six (46.2%) were critically ill and treated in the intensive care unit (ICU). The mean stay at the ICU of the 5 surviving patients was 18.8 ± 6.8 days. In six patients (46.2%) gastrointestinal disorders accompanied COVID-19.
The authors conclude that adjunctive homeopathic treatment may be helpful to treat patients with confirmed COVID-19 even in high-risk patients especially since there is no conventional treatment of COVID-19 available at present.
In the discussion section of the paper, the authors state this: “Given the extreme variability of pathology and clinical manifestations, a single universal preventive homeopathic medicinal product does not seem feasible. Yet homeopathy may have a relevant role to play precisely because of the number and diversity of its homeopathic medicinal products which can be matched with the diversity of the presentations. Patients with mild forms of disease can use homeopathic medicinal products at home using our simple algorithm. As this Case series suggests, adjunctive homeopathic treatment can play a valuable role in more serious presentations. For future pandemics, homeopathy agencies should be prepared by establishing rapid-response teams and efficacious lines of communication.”
There is nothing in this paper that would lead me to conclude that the homeopathic remedies had a positive effect on the natural history of the disease. All this article actually does do is this: it provides a near-perfect insight into the delusional megalomania of some homeopaths. These people are even more dangerous than I had feared.
Mind-body interventions (MBIs) are one of the top ten so-called alternative medicine (SCAM) approaches utilized in pediatrics, but there is limited knowledge on associated adverse events (AE). The objective of this review was to systematically review AEs reported in association with MBIs in children.
Electronic databases MEDLINE, Embase, CINAHL, CDSR, and CCRCT were searched from inception to August 2018. The authors included primary studies on participants ≤ 21 years of age that used an MBI. Experimental studies were assessed for whether AEs were reported on or not, and all other study designs were included only if they reported an AE.
A total of 441 were included as primary pediatric MBI studies. Of these, 377 (85.5%) did not explicitly report the presence/absence of AEs or a safety assessment. In total, there were 64 included studies: 43 experimental studies reported that no AE occurred, and 21 studies reported AEs. A total of 37 AEs were found, of which the most serious were grade 3. Most of the studies reporting AEs did not report on severity (81.0%) or duration of AEs (52.4%).
The authors concluded that MBIs are popularly used in children; however associated harms are often not reported and lack important information for meaningful assessment.
SCAM is far too often considered to be risk-free. This phenomenon is particularly stark if the SCAM in question does not involve physical or pharmacological treatments. Thus MBIs are seen and often waved through as especially safe. Consequently, many researchers do not even bother to monitor AEs in their clinical trials. This might be understandable, but it is nevertheless a violation of research ethics.
This new review is important in that it highlights these issues. It is high time that we stop giving researchers in SCAM the benefit of the doubt. They may or may not make honest mistakes when not reporting AEs. In any case, it is clear that they are not properly trained and supervised. All too often, we still see clinical trials run by amateurs who have little idea of methodology and even less of ethics. The harm this phenomenon does is difficult to quantify, but I fear it is huge.
Qigong is a branch of Traditional Chinese Medicine using meditation, exercise, deep breathing, and other techniques with a view of strengthening the assumed life force ‘qi’ and thus improving health and prolong life. There are several distinct forms of qigong which can be categorized into two main groups, internal qigong, and external qigong. Internal qigong refers to a physical and mental training method for the cultivation of oneself to achieve optimal health in both mind and body. Internal qigong is not dissimilar to tai chi but it also employs the coordination of different breathing patterns and meditation. External qigong refers to a treatment where qigong practitioners direct their qi-energy to the patient with the intention to clear qi-blockages or balance the flow of qi within that patient. According to Taoist and Buddhist beliefs, qigong allows access to higher realms of awareness. The assumptions of qigong are not scientifically plausible and its clinical effectiveness remains unproven.
The aim of this study was to investigate the effects of internal Qigong for the management of a symptom cluster comprising fatigue, dyspnea, and anxiety in patients with lung cancer.
A total of 156 lung cancer patients participated in this trial, and they were randomized to a Qigong group (6 weeks of intervention) or a waitlist control group receiving usual care. A professional coach with 12 years of experience in teaching Qigong was employed to guide the participants’ training. The training protocol was developed according to the “Qigong Standard” enacted by the Chulalongkorn University, Thailand. The training involved a series of simple, repeated practices including body posture/movement, breathing practice, and meditation performed in synchrony. It mainly consisted of gentle movements designed to bring about a deep state of relaxation and included 7 postures. The symptom cluster was assessed at baseline, at the end of treatment (primary outcome), and at 12 weeks, alongside measures of cough and quality of life (QOL).
The results showed no significant interaction effect between group and time for the symptom cluster, the primary outcome measure of this study, overall and for fatigue and anxiety. However, a significant trend towards improvement was observed on fatigue (P = .004), dyspnea (P = .002), and anxiety (P = .049) in the Qigong group from baseline assessment to the end of intervention at the 6th week (within-group changes). Improvements in dyspnea and in the secondary outcomes of cough, global health status, functional well-being and QOL symptom scales were statistically significant between the 2 groups (P = .001, .014, .021, .001, and .002, respectively).
The authors concluded that Qigong did not alleviate the symptom cluster experience. Nevertheless, this intervention was effective in reducing dyspnea and cough, and improving QOL. More than 6 weeks were needed, however, for detecting the effect of Qigong on improving dyspnea. Furthermore, men benefited more than women. It may not be beneficial to use Qigong to manage the symptom cluster consisting of fatigue, dyspnea, and anxiety, but it may be effective in managing respiratory symptoms (secondary outcomes needing further verification in future research). Future studies targeting symptom clusters should ensure the appropriateness of the combination of symptoms.
I am getting very tired of negative trials getting published as (almost) positive ones. The primary outcome measure of this study did not yield a positive result. The fact that some other endpoints suggested a positive might provide an impetus for further study but does not demonstrate Qigong to be effective. I know the first author of this study is a fan of so-called alternative medicine (SCAM), but this should not stop him from doing proper science.
Indian homeopaths aimed at evaluating the efficacy of individualized homeopathy (IH) for atopic dermatitis (AD). They conducted a double-blind, randomized, placebo-controlled, short-term, preliminary trial in an Indian homeopathy hospital. Patients were randomized to either IH (n = 30) or identical-looking placebo (n = 30) using computerized randomization and allocation. Outcomes were patient-oriented scoring of AD (PO-SCORAD; primary endpoint), Dermatological Life Quality Index (DLQI) score, and AD burden score for adults (ADBSA; secondary endpoints), measured monthly for 3 months. An intention-to-treat sample was analyzed after adjusting baseline differences.
On PO-SCORAD, improvement was higher in IH against placebo, but nonsignificant statistically (pmonth 1 = 0.433, pmonth 2 = 0.442, pmonth 3 = 0.229). Secondary outcomes were also nonsignificant – both DLQI and ADBSA (p > 0.05). Four adverse events (diarrhea, injury, common cold) were recorded.
The authors concluded that there was a small, but nonsignificant direction of effect towards homeopathy, which renders the trial inconclusive. A properly powered robust trial is indicated.
Thee questions:
- Why use statistics only to ignore its results?
- Why discredit research into so-called alternative medicine (SCAM) in this way?
- Who on earth would publish such misleading conclusions?
This article was published in Complementary Medicine Research which claims to be an international peer-reviewed journal that aims to bridge the gap between conventional and complementary/alternative medicine on a sound scientific basis, promoting their mutual integration. It boasts that “experts of both conventional medicine and complementary/alternative medicine cooperate on the journal’s editorial board, ensuring a high standard of scientific quality”. Its editor is Harald Walach who we have met several times before.
I had a look at the long list of members of the editorial board and was unable to see many ‘experts in conventional medicine’. If that is so, the journal’s peer review process is bound to turn into a farcical procedure where any rubbish will pass.
The journal reminds authors that “published research must comply with internationally-accepted standards for research practice and reporting.” I believe that the internationally accepted standards of research reporting include something about not misleading the public by claiming that the absence of an effect is a small effect in favor of homeopathy. By revealing that there was no significant effect, the authors of this study demonstrate that IH was not effective as a treatment of AD. It is in my mind unethical to try to disguise this result by making it look like a small positive effect or claiming the result was inconclusive.
High standard of scientific quality?
No, quite the opposite!
In the world of homeopathy, Prof Michael Frass is a famous man. He is the First Chairman of the Scientific Society for Homeopathy (WissHom), the president of the Umbrella organization of Austrian Doctors for Holistic Medicine, and the Vicepresident of the Doctors Association for Classical Homeopathy. Frass has featured on this blog before, not least because he has published numerous studies of homeopathy, none of which has ever failed to produce a positive result.
This is not just remarkable, in my view, it defies logic and the laws of nature. Even if homeopathy were a supremely effective therapy – a very broad consensus holds that it is not! – one would occasionally expect some negative results. No treatment works under all circumstances
… that is no treatment except homeopathy, according to Frass.
Recently Frass amazed even the world of oncology by publishing a study suggesting that homeopathy can prolong the survival of lung cancer patients. Every oncologist I know was flabbergasted.
Can this be true? This is the question, many people have been asking for some time in relation to Frass’s research.
In my quest to shine more light on it, I was recently alerted to an article by the formidable Austrian investigative journalist, Alwin Schönberger. In 2015, he came across a press release announcing that “HOMEOPATHY HAD BEEN PROVEN TO WORK AFTER ALL” (strikingly similar to one issued in 2018). It came from Austria’s leading manufacturer who was giving an award to an apparently outstanding thesis supervised by Frass. Even today, this piece of research has not been published in the peer-reviewed literature.
Yet, after some difficulties, Schönberger managed to obtain a copy. What he found was surprising, and he thus published his findings in the respected Austrian journal ‘Profil’ (2. Mai 2015 • profil 22).
Frass’s student had been given the task to systematically review all the homeopathy trials published between 2008 and 2012. Contrary to the hype of the press release, the meta-analysis merely suggested a very small effect. When digging deeper, Schönberger found several inconsistencies and mistakes in the analysis. They all were such that they produced a false-positive picture for homeopathy. Upon their correction, homeopathy turned out to be no longer significantly superior to placebo. Frass was then interviewed about it and claimed that the inconsistencies were only ‘errors’ but insisted that homeopathy is not a placebo therapy.
Yes, of course, errors happen in research. But if they all go in one direction and if that direction coincides with the interests of the researchers, we have the right, perhaps even the duty, to be suspicious. The questions that arise from this story are, I think, as follows:
- Have the errors been corrected?
- Are there perhaps other errors in Frass’s research?
- Can we trust anything that Frass says?
- Is it time to consider an official investigation into Frass’s studies of homeopathy?
