Observational studies of homeopathy avoid the cognitive dissonance of homeopaths

When tested rigorously, the evidence for so-called alternatives medicine (SCAM) is usually weak or even negative. This fact has prompted many SCAM enthusiasts to become utterly disenchanted with rigorous tests such as the randomised clinical trial (RCT). They seem to think that, if the RCT fails to generate the findings we want, let’s use different methodologies instead. In other words, they are in favour of observational studies which often yield positive results.

This line of thinking is prevalent in all forms of SCAM, but probably nowhere more so that in the realm of homeopathy. Homeopaths see that rigorous RCTs tend not to confirm their belief and, to avoid cognitive dissonance, they focus on observational studies which are much more likely to confirm their belief.

In this context, it is worth mentioning a recent article where well-known homeopathy enthusiasts have addressed the issue of observational studies. Here is their abstract:

Background: Randomized placebo-controlled trials are considered to be the gold standard in clinical research and have the highest importance in the hierarchical system of evidence-based medicine. However, from the viewpoint of decision makers, due to lower external validity, practical results of efficacy research are often not in line with the huge investments made over decades.

Method: We conducted a narrative review. With a special focus on homeopathy, we give an overview on cohort, comparative cohort, case-control and cross-sectional study designs and explain guidelines and tools that help to improve the quality of observational studies, such as the STROBE Statement, RECORD, GRACE and ENCePP Guide.

Results: Within the conventional medical research field, two types of arguments have been employed in favor of observational studies. First, observational studies allow for a more generalizable and robust estimation of effects in clinical practice, and if cohorts are large enough, there is no over-estimation of effect sizes, as is often feared. We argue that observational research is needed to balance the current over-emphasis on internal validity at the expense of external validity. Thus, observational research can be considered an important research tool to describe “real-world” care settings and can assist with the design and inform the results of randomised controlled trails.

Conclusions: We present recommendations for designing, conducting and reporting observational studies in homeopathy and provide recommendations to complement the STROBE Statement for homeopathic observational studies.

In their paper, the authors state this:

It is important to realize three areas where observational research can be valuable. For one, as already mentioned, it can be valuable as a preparatory type of research for designing good randomized studies. Second, it can be valuable as a stand-alone type of research, where pragmatic or ethical reasons stand against conducting a randomized study. Additionally, it can be valuable as the only adequate method where choices are involved: for instance, in any type of lifestyle research or where patients have very strong preferences, such as in homeopathy and other CAM. This might also lead to a diversification of research efforts and a broader, more realistic, picture of the effects of therapeutic interventions.

My comments to this are as follows:

1. Observational research can be valuable as a preparatory type of research for designing good randomized studies. This purpose is better fulfilled by pilot studies (which are often abused in SCAM).
2. Observational research can be valuable as a stand-alone type of research, where pragmatic or ethical reasons stand against conducting a randomized study. Such situations rarely arise in the realm of SCAM.
3. Observational research can be valuable as the only adequate method where choices are involved: for instance, in any type of lifestyle research or where patients have very strong preferences, such as in homeopathy and other CAM. I fail to see that this is true.
4. Observational research leads to a diversification of research efforts and a broader, more realistic, picture of the effects of therapeutic interventions. The main aim of research into the effectiveness of SCAM should be, in my view, to determine whether the treatment per se works or not. Observational studies are likely to obscure the truth on this issue.

Don’t get me wrong, I am not saying that observational studies are useless; quite to the contrary, they can provide very important information. But what I am trying to express is this:

• We should not allow double standards in medical research. The standards and issues of observational research as they exist in conventional medicine must also apply to SCAM.
• Observational studies cannot easily determine cause and effect between the therapy and the outcome.
• Observational studies cannot be a substitute for RCTs.
• Depending on their exact design, observational studies measure the outcome caused by a whole range of factors, including the therapy per se, the placebo-effect, the natural history of the disease, the regression towards the mean.
• Observational studies are particularly useful in effectiveness research, AFTER the efficacy of a therapy has been established by RCTs.
• If RCT fail to show that a therapy is effective and observational studies seem to indicate that they work, the therapy in question is probably a placebo.
• SCAM-enthusiasts’ preference for observational studies is transparently due to motivated reasoning.