MD, PhD, FMedSci, FRSB, FRCP, FRCPEd.

When tested rigorously, the evidence for so-called alternatives medicine (SCAM) is usually weak or even negative. This fact has prompted many SCAM enthusiasts to become utterly disenchanted with rigorous tests such as the randomised clinical trial (RCT). They seem to think that, if the RCT fails to generate the findings we want, let’s use different methodologies instead. In other words, they are in favour of observational studies which often yield positive results.

This line of thinking is prevalent in all forms of SCAM, but probably nowhere more so that in the realm of homeopathy. Homeopaths see that rigorous RCTs tend not to confirm their belief and, to avoid cognitive dissonance, they focus on observational studies which are much more likely to confirm their belief.

In this context, it is worth mentioning a recent article where well-known homeopathy enthusiasts have addressed the issue of observational studies. Here is their abstract:

Background: Randomized placebo-controlled trials are considered to be the gold standard in clinical research and have the highest importance in the hierarchical system of evidence-based medicine. However, from the viewpoint of decision makers, due to lower external validity, practical results of efficacy research are often not in line with the huge investments made over decades.

Method: We conducted a narrative review. With a special focus on homeopathy, we give an overview on cohort, comparative cohort, case-control and cross-sectional study designs and explain guidelines and tools that help to improve the quality of observational studies, such as the STROBE Statement, RECORD, GRACE and ENCePP Guide.

Results: Within the conventional medical research field, two types of arguments have been employed in favor of observational studies. First, observational studies allow for a more generalizable and robust estimation of effects in clinical practice, and if cohorts are large enough, there is no over-estimation of effect sizes, as is often feared. We argue that observational research is needed to balance the current over-emphasis on internal validity at the expense of external validity. Thus, observational research can be considered an important research tool to describe “real-world” care settings and can assist with the design and inform the results of randomised controlled trails.

Conclusions: We present recommendations for designing, conducting and reporting observational studies in homeopathy and provide recommendations to complement the STROBE Statement for homeopathic observational studies.

In their paper, the authors state this:

It is important to realize three areas where observational research can be valuable. For one, as already mentioned, it can be valuable as a preparatory type of research for designing good randomized studies. Second, it can be valuable as a stand-alone type of research, where pragmatic or ethical reasons stand against conducting a randomized study. Additionally, it can be valuable as the only adequate method where choices are involved: for instance, in any type of lifestyle research or where patients have very strong preferences, such as in homeopathy and other CAM. This might also lead to a diversification of research efforts and a broader, more realistic, picture of the effects of therapeutic interventions.

My comments to this are as follows:

  1. Observational research can be valuable as a preparatory type of research for designing good randomized studies. This purpose is better fulfilled by pilot studies (which are often abused in SCAM).
  2. Observational research can be valuable as a stand-alone type of research, where pragmatic or ethical reasons stand against conducting a randomized study. Such situations rarely arise in the realm of SCAM.
  3. Observational research can be valuable as the only adequate method where choices are involved: for instance, in any type of lifestyle research or where patients have very strong preferences, such as in homeopathy and other CAM. I fail to see that this is true.
  4. Observational research leads to a diversification of research efforts and a broader, more realistic, picture of the effects of therapeutic interventions. The main aim of research into the effectiveness of SCAM should be, in my view, to determine whether the treatment per se works or not. Observational studies are likely to obscure the truth on this issue.

Don’t get me wrong, I am not saying that observational studies are useless; quite to the contrary, they can provide very important information. But what I am trying to express is this:

  • We should not allow double standards in medical research. The standards and issues of observational research as they exist in conventional medicine must also apply to SCAM.
  • Observational studies cannot easily determine cause and effect between the therapy and the outcome.
  • Observational studies cannot be a substitute for RCTs.
  • Depending on their exact design, observational studies measure the outcome caused by a whole range of factors, including the therapy per se, the placebo-effect, the natural history of the disease, the regression towards the mean.
  • Observational studies are particularly useful in effectiveness research, AFTER the efficacy of a therapy has been established by RCTs.
  • If RCT fail to show that a therapy is effective and observational studies seem to indicate that they work, the therapy in question is probably a placebo.
  • SCAM-enthusiasts’ preference for observational studies is transparently due to motivated reasoning.

15 Responses to Observational studies of homeopathy avoid the cognitive dissonance of homeopaths

  • The usual suspects. Elizabeth Thompson is employed as a `consultant homeopathic physician’ by Bristol University Hospitals Foundation NHS Trust. I am trying to find out why. She has only published two papers purporting to be RCTs, but one is only a protocol and the other had negative findings for homeopathy. Hence the present special pleading article.

    The authors have turned the hierarchy of evidence upside down. An RCT is vital if you want to know if there is actually an effect of the treatment. If there is, then do other studies to test generalisability. It’s too easy to skip the first step if it doesn’t confirm what you believe.

    • its scary: http://www.uhbristol.nhs.uk/for-clinicians/consultant-profiles/dr-elizabeth-thompson/
      and that NHS site in turn links to homeopaths who tell us that:
      “At a time when the NHS is at maximum capacity, we recommend contacting one of the many Faculty of Homeopathy members who are conducting virtual and telephone consultations.”
      surely the NHS should not be giving homeopaths the imprimatur of legitimacy, especially at a time like this.

    • There are regulations for the employment in the NHS of a consultant physician.
      Miss Thompson has not been through approved appointments procedures and calling herself a ‘consultant physician’ coould be an attempt to defraud.
      Bristol should be asked what her status is, and correct it if necessary.

  • Background: Randomized placebo-controlled trials are considered to be the gold standard in clinical research and have the highest importance in the hierarchical system of evidence-based medicine. However, from the viewpoint of decision makers, due to lower external validity, practical results of efficacy research are often not in line with the huge investments made over decades.

    I have no idea what this means.

  • The phrase that mostly makes that orotund sentence meaningless, as far as I can see, is “due to lower external validity”.

    When language is woolly, its probably because the thinking and evidence behind the statement is woolly.

    If one were to attempt a translation, which is really an interpretation (and should not be necessary), I THINK the sentence might be saying:

    “Those allocating funds to a treatment may be disappointed that real-world results are not as good as the results of expensive trials led them to hope”.

    IF that is what is meant, the sentence is of course open to argument. Real-world results may be BETTER than hoped-for. In any case, it’s a rotten piece of writing.

    • When language is woolly, its probably because the thinking and evidence behind the statement is woolly

      I couldn’t agree more. I have had too much of NHS managers using this sort of language; they seem to think that once they have mastered the style there is no need for any content. I am not sure which is worse – management-speak gobbledegook or quantum gobbledegook.

      I am also reminded of the story of a vicar who was found to have written in the margin of one of his sermons: “Argument weak. Shout louder.”

  • “Observational studies are particularly useful in effectiveness research, AFTER the efficacy of a therapy has been established by RCTs.”

    So you’re saying that WHO is wrong?

  • Who is wrong about what?

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

If you want to be able to edit your comment for five minutes after you first submit it, you will need to tick the box: “Save my name, email, and website in this browser for the next time I comment.”
Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted but you must tick the box: “Save my name, email, and website in this browser for the next time I comment.”

The most recent comments from all posts can be seen here.

Archives
Categories