There are dozens of observational studies of homeopathy which seem to suggest – at least to homeopaths – that homeopathic treatments generate health benefits. As these investigations lack a control group, their results can be all to easily invalidated by pointing out that factors like ‘regression towards the mean‘ (RTM, a statistical artefact caused by the phenomenon that a variable that is extreme on its first measurement tends to be closer to the average on its second measurement) might be the cause of the observed change. Thus the debate whether such observational data are reliable or not has been raging for decades. Now, German (pro-homeopathy) investigators have published a paper which potentially could resolve this dispute.
With this re-analysis of an observational study, the investigators wanted to evaluate whether the observed changes in previous cohort studies are due to RTM and to estimate RTM adjusted effects. SF-36 quality-of-life (QoL) data from a cohort of 2827 chronically diseased adults treated with homeopathy were reanalysed using a method described in 1991 by Mee and Chua’s. RTM adjusted effects, standardized by the respective standard deviation at baseline, were 0.12 (95% CI: 0.06-0.19, P < 0.001) in the mental and 0.25 (0.22-0.28, P < 0.001) in the physical summary score of the SF-36. Small-to-moderate effects were confirmed for most individual diagnoses in physical, but not in mental component scores. Under the assumption that the true population mean equals the mean of all actually diseased patients, RTM adjusted effects were confirmed for both scores in most diagnoses.
The authors reached the following conclusion: “In our paper we showed that the effects on quality of life observed in patients receiving homeopathic care in a usual care setting are small or moderate at maximum, but cannot be explained by RTM alone. Due to the uncontrolled study design they may, however, completely be due to nonspecific effects. All our analyses made a restrictive and conservative assumption, so the true treatment effects might be larger than shown.”
Of course, the analysis heavily relies on the validity of Mee and Chua’s modified t-test. It requires the true mean in the target population to be known, a requirement that seldom can be fulfilled. The authors therefore took the SF-36 mean summary scores from the 1998 German health survey as proxies. I am not a statistician and therefore unable to tell how reliable this method might be (- if there is someone out there who can give us some guidance here, please post your comment).
In order to make sense of these data, we need to consider that, during the study period, about half of the patients admitted to have had additional visits to non-homeopathic doctors, and 27% also received conventional drugs. In addition, they would have benefitted from:
- the benign history of the conditions they were suffering from,
- a placebo-effect,
- the care and attention they received
- and all sorts of other non-specific effects.
So, considering these factors, what does this interesting re-analysis really tell us? My interpretation is as follows: the type of observational study that homeopaths are so fond of yields false-positive results. If we correct them – as the authors have done here for just one single factor, the RTM – the effect size gets significantly smaller. If we were able to correct them for some of the other factors mentioned above, the effect size would shrink more and more. And if we were able to correct them for all confounders, their results would almost certainly concur with those of rigorously controlled trials which demonstrate that homeopathic remedies are pure placebos.
I am quite sure that this interpretation is unpopular with homeopaths, but I am equally certain that it is correct.
15 Responses to Important new insights into homeopathy
Leave a Reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
I could write a very long and (to most readers) a sleep-inducing technical essay in response to why RTM and other factors weigh so heavily in trials conducted by CAM proponents. Instead, I shall attempt to be brief, succinct, and acidly terse to sCAM (not terse towards Edzard’s excellent post)…
In both very precise and safety-critical areas of engineering we use such things as raised cosine weighting to reduce the effects of start points that quickly regress to the mean and endpoints that digress from the mean for other reasons. The fundamental pillar of this being that the plural of anecdote (even scientifically collected data points) is not data per se.
The collected raw data sets are, of course, subjected to the oft-used stats analysis tools, but these tools frequently produce results best suited to nefarious agendas and least suited to best health and safety outcomes. The painstaking processes of appropriate data set weightings, examining autocorrelations, Fourier transforms, cepstrums, and other mathematical transforms (especially their waterfall depictions), reveal many truths and falsehoods in the collected data sets that will always remain hidden when using any traditional stats based on p and r values. Without such painstaking analysis, mobile phones wouldn’t exists and helicopters would crash so frequently that they would’ve been banned from use ages ago. If sSCAM were to be subjected to this exemplary level of health and safety scrutiny it would be summarily banned and totally discredited.
Naturally, any sCAM practitioner smart enough to begin to understand the power of Fourier transforms and cepstral analysis of patient data would be smart enough to have given up their practice long ago rather than continue to omit negative outcomes in their client data records.
Note to practitioners: The old joke about a boss demanding “Show me the data missing from our database” is indeed an old joke because revealing missing database records is a mundane and trivial task for those of us who’ve spent decades working with both data records and incompetent managers 🙂
In every observational study patients would have benefitted from
the benign history of the conditions they were suffering from,
a placebo-effect,
the care and attention they received
and all sorts of other non-specific effects.
Are the majority of observational studies useless ?
no, not necessarily. it depends on the research question they are addressing. they are pretty useless when it comes to effectiveness/ efficacy questions.
“My interpretation is as follows: the type of observational study that homeopaths are so fond of yields false-positive results. If we correct them – as the authors have done here for just one single factor, the RTM – the effect size gets significantly smaller. If we were able to correct them for some of the other factors mentioned above, the effect size would shrink more and more. And if we were able to correct them for all confounders, their results would almost certainly concur with those of rigorously controlled trials which demonstrate that homeopathic remedies are pure placebos.”
LOL great conclusion Prof. Now that placebo has been proven “Placebo effect is Real, quantifiable and in fact you are doing quite well with an active therapy if you can get a response as good as a placebo response” to quote a Harvard prof what have you to say? Another Prof, has shown with MRI scans of the brain that placebo can affect the brian in the same way as oppiod drugs!!! The placebo response causes “a real neurological and biological effect” according to another prof. It has been shown to increase dopermein in Parkinsons. Prof what have you got to say?? I don’t imagine you will publish these comments but it really does show that your biased blinkered views clouded your judgement, you never even considered how it could work, you just set about trying to convince yourself and other that it COULDN’T work. Nice science.
Interesting programme on placebo last night. It seems placebo can increase performance, lower GDA at altitude, produce dopaminein in Parkinsons and I quote ” it has a real neurological and biological affect” whats it like Prof realising you have been barking up the wrong tree for most of your professional career and that most of the work you have done was actually pointless?
Homeopathy cannot work???? placebo isn’t some inert nothing it produces real measurable changes, so all CAM crap? or all CAM is placebo that now seems to be quite a powerful treatment. Placebo creates a measurable change in the body even when the person knows its a placebo. I know what your argument will be, but ANY treatment, placebo or not that creates a measurable change is just that. A valuable treatment. Good bye Prof Ernst your research and views are now extinct.
HELLO… I’M STILL HERE!
I have no problem with the program – in fact, they picked my brain in doing it. who do you think doubted that placebo is real and effective?
Interesting programme on placebo last night. It seems placebo can increase performance, lower GDA at altitude, produce dopaminein in Parkinsons and I quote ” it has a real neurological and biological affect” whats it like Prof realising you have been barking up the wrong tree for most of your professional career and that most of the work you have done was actually pointless?
Homeopathy cannot work???? placebo isn’t some inert nothing it produces real measurable changes, so all CAM crap? or all CAM is placebo that now seems to be quite a powerful treatment. Placebo creates a measurable change in the body even when the person knows its a placebo. I know what your argument will be, but ANY treatment, placebo or not that creates a measurable change is just that. A valuable treatment.
Horizon explores why placebos work, despite containing no active ingredient, and explains how we could all benefit from their hidden power.
nothing wrong with the program; yet I still prefer to get my knowledge and expertise from reading the original literature rather than TV
Is placebo real and effective? does it produce a measurable response? is homeopathy placebo?
Do you disagree with any of the research projects or outcomes shown on the programme?
no
how about my first question?
Is placebo real and effective? does it produce a measurable response? is homeopathy placebo?
can I ask this question again?
Fedup, it would really help if you
1) kept your comments on one thread (preferrably one where they are on topic) instead of posting the same comments all over the place
2) actually read the answers and made an effort to understand them.
Your question has already been answered by me and a few other commentators on a few other threads, but I don’t mind repeating:
Yes, placebo effects exist, hence the need for placebo controls.
Yes, homeopathy is a placebo therapy.
The big problem is that homeopaths consistently claim that homeopathy is not a placebo therapy, but is more effective than a placebo therapy. Some symptoms respond to placebo therapies (e.g. pain, nausea), and if there’s no underlying illness (or it cannot be treated effectively with medicine), then by all means, go for a placebo therapy – but you have to tell your patient that they’re treated with placebos. A good, effective therapy is one that offers placebo effects on top of specific effects, not instead of, and no, one professor on a TV program saying differently won’t change that.