While looking up an acupuncturist who has recently commented on this blog trying to teach me how to do science and understand research methodology, I was impressed that he, Dr John McDonald, PhD, has just published a clinical trial. Not many acupuncturists do that, you know, and I very much applaud this action, which even seems to have earned him his PhD! McDonald is understandably proud of his achievement – all the more because the study arrived at positive conclusions. This is what he wrote about it:
…So, in a nutshell, acupuncture is a safe, effective and cost-effective treatment for allergic rhinitis which produces lasting changes in the immune system and hence improvements in symptoms and quality of life. Dr John McDonald
Fascinating! I quickly looked up the paper. Here it is:
This new study was designed as a randomized, sham-controlled trial of acupuncture for persistent allergic rhinitis in adults investigated possible modulation of mucosal immune responses. A total of 151 individuals were randomized into real and sham acupuncture groups (who received twice-weekly treatments for 8 weeks) and a no acupuncture group. Various cytokines, neurotrophins, proinflammatory neuropeptides, and immunoglobulins were measured in saliva or plasma from baseline to 4-week follow-up.
Statistically significant reduction in allergen specific IgE for house dust mite was seen only in the real acupuncture group. A mean (SE) statistically significant down-regulation was also seen in pro-inflammatory neuropeptide substance P (SP) 18 to 24 hours after the first treatment. No significant changes were seen in the other neuropeptides, neurotrophins, or cytokines tested. Nasal obstruction, nasal itch, sneezing, runny nose, eye itch, and unrefreshed sleep improved significantly in the real acupuncture group (post-nasal drip and sinus pain did not) and continued to improve up to 4-week follow-up.
The authors concluded that acupuncture modulated mucosal immune response in the upper airway in adults with persistent allergic rhinitis. This modulation appears to be associated with down-regulation of allergen specific IgE for house dust mite, which this study is the first to report. Improvements in nasal itch, eye itch, and sneezing after acupuncture are suggestive of down-regulation of transient receptor potential vanilloid 1.
These conclusions seem to be based on the data of the study. But they are oddly out of line with the above statement made by McDonald about his trial. What could be the reason for this discrepancy? Could it be that he behaves ‘scientifically’ correct when under the watchful eye of numerous co-authors from the School of Medicine, Menzies Health Institute, Griffith University, Queensland, Australia, the School of Medicine, Menzies Health Institute, Griffith University, Queensland, Australia, the National Institute of Complementary Medicine, Western Sydney University, Sydney, Australia, the Health Innovations Research Institute and School of Health Sciences, RMIT University, Melbourne, Victoria, Australia, and the Stanford University, Palo Alto, California? And could it be that he is a little more ‘liberal’ when on his own? A mere speculation, of course, but it would be nice to know.
Anyway, the trial itself raises a number of questions – unfortunately I have no access to the full paper – which I will post here in the hope that my acupuncture friend, who are clearly impressed by this paper, might provide the answers in the comments section below:
- Which was the primary outcome measure of this trial?
- What was the power of the study, and how was it calculated?
- For which outcome measures was the power calculated?
- How were the subjective endpoints quantified?
- Were validated instruments used for the subjective endpoints?
- What type of sham was used?
- Are the reported results the findings of comparisons between verum and sham, or verum and no acupuncture, or intra-group changes in the verum group?
- Was the success of patient-blinding checked, quantified and successful?
- What other treatments did each group of patients receive?
- Does anyone really think that this trial shows that “acupuncture is a safe, effective and cost-effective treatment for allergic rhinitis”?
I have written about ‘EVIDENCE BASED COMPLEMENTARY AND ALTERNATIVE MEDICINE’ (EBCAM), on of the leading alt med journals before (for instance here and here). To my embarrassment, I must admit to having been a member of its founding editorial-board; but I left when things started looking suspicious. In the latter post, I pointed out that:
- The peer-review system of EBCAM is farcical: potential authors who send their submissions to EBCAM are invited to suggest their preferred reviewers who subsequently are almost invariably appointed to do the job. It goes without saying that such a system is prone to all sorts of serious failures; in fact, this is not peer-review at all, in my opinion, it is an unethical sham.
- As a result, most (I estimate around 80%) of the articles that currently get published on alternative medicine are useless rubbish. They tend to be either pre-clinical investigations which never get followed up and are thus meaningless, or surveys of no relevance whatsoever, or pilot studies that never are succeeded by more definitive trials, or non-systematic reviews that are wide open to bias and can only mislead the reader.
Strong words? Yes, ‘useless rubbish’ is not exactly meant as a compliment. Perhaps you want to judge for yourself – here are the last 20 articles published in EBCAM in 2015:
Kathy Lee SM, Yoon KH, Park J, Kim HS, Woo JS, Lee SR, Lee KH, Jang HH, Kim JB, Kim WS, Lee S, Kim W.
Evid Based Complement Alternat Med. 2015;2015:625645. doi: 10.1155/2015/625645. Epub 2016 Jan 11.
Azmi NH, Ismail M, Ismail N, Imam MU, Alitheen NB, Abdullah MA.
Evid Based Complement Alternat Med. 2015;2015:153684. doi: 10.1155/2015/153684. Epub 2015 Dec 22.
Park W, Mollahaliloglu S, Linnik V, Chae H.
Evid Based Complement Alternat Med. 2015;2015:967087. doi: 10.1155/2015/967087. Epub 2015 Dec 30. No abstract available.
Tang H, Fan H, Chen J, Yang M, Yi X, Dai G, Chen J, Tang L, Rong H, Wu J, Liang F.
Evid Based Complement Alternat Med. 2015;2015:861849. doi: 10.1155/2015/861849. Epub 2015 Dec 30. Review.
Li M, Han Z, Bei W, Rong X, Guo J, Hu X.
Evid Based Complement Alternat Med. 2015;2015:643102. doi: 10.1155/2015/643102. Epub 2015 Dec 30.
Silva FC, de Souza JG, Reichert AM, Antonangelo RP, Suzuki R, Itinose AM, Marek CB.
Evid Based Complement Alternat Med. 2015;2015:762373. doi: 10.1155/2015/762373. Epub 2015 Dec 28.
Hou PW, Hsu HC, Lin YW, Tang NY, Cheng CY, Hsieh CL.
Evid Based Complement Alternat Med. 2015;2015:495684. doi: 10.1155/2015/495684. Epub 2015 Dec 28. Review.
Joray MB, Trucco LD, González ML, Napal GN, Palacios SM, Bocco JL, Carpinella MC.
Evid Based Complement Alternat Med. 2015;2015:912484. doi: 10.1155/2015/912484. Epub 2015 Dec 27.
Wei Y, Ma LX, Yin SJ, An J, Wei Q, Yang JX.
Evid Based Complement Alternat Med. 2015;2015:878164. doi: 10.1155/2015/878164. Epub 2015 Dec 27. Review.
Hawk C, Adams J, Hartvigsen J.
Evid Based Complement Alternat Med. 2015;2015:528487. doi: 10.1155/2015/528487. Epub 2015 Dec 24. No abstract available.
Zhao H, Jin B, Zhang X, Cui Y, Sun D, Gao C, Gu Y, Cai B.
Evid Based Complement Alternat Med. 2015;2015:980583. doi: 10.1155/2015/980583. Epub 2015 Dec 21.
Singsai K, Akaravichien T, Kukongviriyapan V, Sattayasai J.
Evid Based Complement Alternat Med. 2015;2015:970354. doi: 10.1155/2015/970354. Epub 2015 Dec 21.
Chung HS, Hwang I, Oh KJ, Lee MN, Park K.
Evid Based Complement Alternat Med. 2015;2015:913158. doi: 10.1155/2015/913158. Epub 2015 Dec 22.
Liu X, Kanthimathi MS, Heese K.
Evid Based Complement Alternat Med. 2015;2015:828159. doi: 10.1155/2015/828159. Epub 2015 Dec 22. No abstract available.
Murakami S, Goto Y, Ito K, Hayasaka S, Kurihara S, Soga T, Tomita M, Fukuda S.
Evid Based Complement Alternat Med. 2015;2015:824395. doi: 10.1155/2015/824395. Epub 2015 Dec 21.
Xuejuan Z, Jietao Z, Di H, Yu Z, Xiaozi G, Yunfa L, Lihua D.
Evid Based Complement Alternat Med. 2015;2015:527219. doi: 10.1155/2015/527219. Epub 2015 Dec 22.
Hesse T, Holmes LG, Kennedy-Overfelt V, Kerr LM, Giles LL.
Evid Based Complement Alternat Med. 2015;2015:508958. doi: 10.1155/2015/508958. Epub 2015 Dec 22.
Mao S, Li C.
Evid Based Complement Alternat Med. 2015;2015:349721. doi: 10.1155/2015/349721. Epub 2015 Dec 22.
Shen CC, Yang YC, Chiao MT, Chan SC, Liu BS.
Evid Based Complement Alternat Med. 2015;2015:278951. doi: 10.1155/2015/278951. Epub 2015 Dec 21.
Mohd Sairazi NS, Sirajudeen KN, Asari MA, Muzaimi M, Mummedy S, Sulaiman SA.
Evid Based Complement Alternat Med. 2015;2015:972623. doi: 10.1155/2015/972623. Epub 2015 Dec 17. Review.
Yes, we discussed this study on a previous blog post. But, as it is ‘ACUPUNCTURE AWARENESS WEEK’ in the UK, and because of another reason (which will become clear in a minute) I decided to revisit the trial.
In case you have forgotten, here is its abstract once again:
Background: Hot flashes (HFs) affect up to 75% of menopausal women and pose a considerable health and financial burden. Evidence of acupuncture efficacy as an HF treatment is conflicting.
Objective: To assess the efficacy of Chinese medicine acupuncture against sham acupuncture for menopausal HFs.
Design: Stratified, blind (participants, outcome assessors, and investigators, but not treating acupuncturists), parallel, randomized, sham-controlled trial with equal allocation. (Australia New Zealand Clinical Trials Registry: ACTRN12611000393954)
Setting: Community in Australia.
Participants: Women older than 40 years in the late menopausal transition or postmenopause with at least 7 moderate HFs daily, meeting criteria for Chinese medicine diagnosis of kidney yin deficiency.
Interventions: 10 treatments over 8 weeks of either standardized Chinese medicine needle acupuncture designed to treat kidney yin deficiency or noninsertive sham acupuncture.
Measurements: The primary outcome was HF score at the end of treatment. Secondary outcomes included quality of life, anxiety, depression, and adverse events. Participants were assessed at 4 weeks, the end of treatment, and then 3 and 6 months after the end of treatment. Intention-to-treat analysis was conducted with linear mixed-effects models.
Results: 327 women were randomly assigned to acupuncture (n = 163) or sham acupuncture (n = 164). At the end of treatment, 16% of participants in the acupuncture group and 13% in the sham group were lost to follow-up. Mean HF scores at the end of treatment were 15.36 in the acupuncture group and 15.04 in the sham group (mean difference, 0.33 [95% CI, −1.87 to 2.52]; P = 0.77). No serious adverse events were reported.
Limitation: Participants were predominantly Caucasian and did not have breast cancer or surgical menopause.
Conclusion: Chinese medicine acupuncture was not superior to noninsertive sham acupuncture for women with moderately severe menopausal HFs.
When I first discussed this trial, I commented that the trial has several strengths: it includes a large sample size and the patients were adequately blinded to eliminate the effects of expectations. It was published in a top journal, and we can therefore assume that it was properly peer-reviewed. Combined with the evidence from our previous systematic review, this indicates that acupuncture has no effect beyond placebo.
The reason for bringing it up again is that a comment about the study has recently appeared, not just any old comment but one from the British Medical Acupuncture Society. It is, in my view, gratifying and interesting. It was published on ‘facebook’ and is therefore in danger of getting forgotten. I hope to preserve it by citing it in full.
Here it is:
A large rigorous trial published in a prestigious general medical journal, and the usual mantra rings out – acupuncture is no better than sham. In this case there was not a fraction of difference from a non-penetrating sham in a two-armed trial with over 300 women. Ok,…so we have known for some time that we really need 400 in each arm to demonstrate the usual difference over sham seen in meta-analysis in pain conditions, but there really was not even a sniff of a difference here. So is that it for acupuncture in hot flushes? Well, we have a 40% symptom reduction in both groups, and a strong conviction from some practitioners that it really seems to work. Is 40% enough for a strong conviction? I have heard some dramatic stories from medical acupuncturist colleagues that really would be hard to dismiss as non-specific effects, and from others I have heard relative ambivalence about the effects in hot flushes.
Personally I always try to consider mechanisms, and I wish researchers in the field would do the same before embarking on their trials. That is not intended as a criticism of this trial, but some consideration of mechanisms might allow us to explain all our data, including the contribution of this trial.
Acupuncture has recognised effects that are local to the needle, in the spinal cord (mainly in the segments stimulated) and in the brain (as well as humoral effects in CSF and blood). The latter are probably the mildest of the three categories, and require the best group of patient responders for them to be observable in clinical practice.
Menopausal hot flushes are explained by the effects of reduced oestrogens on the thermoregulatory centre in the anterior hypothalamus. It is certainly plausible that the neuro-inhibitory effects of endogenous opioids such as beta-endorphin, which we know can be released by acupuncture stimulation in experimental settings, could stablise neurones in the anterior hypothalamus that have become irritable due to a sudden drop in oestrogens.
So are endogenous opioids always released by acupuncture? Well, they and their effects seem to be measurable in experiments that use what I call proper acupuncture. That is, strong stimulation to deep somatic tissue. In the laboratory, and indeed in my clinic, this is only usually achieved in a palatable manner by electroacupuncture to muscle, although repeated manual stimulation every few minutes may have similar effects.
Ee et al used a relatively gentle acupuncture protocol, so they may have only generated measurable effects, based on mechanistic speculation, in the most responsive patients, perhaps less than 10%.
What does all this tell us? Well this trial clearly demonstrates that gentle acupuncture protocols generate effects in women with hot flushes via context rather than penetrating needling. In conditions that rely on central effects, I think we still need to consider stronger stimulation protocols and enriched enrollment in trials, ie preselecting responders before randomisation.
In my original comment I also predicted: “One does not need to be a clairvoyant to predict that acupuncturists will now find what they perceive as a flaw in the new study and claim that its results were false-negative.”
I am so glad Mike Cummings and the BMAS rushed to prove me right.
It’s so nice to know one can rely on someone in these uncertain times!
Germany is, as we all know, the home of homeopathy. Here it has an unbroken popularity, plenty of high level support and embarrassingly little opposition. The argument that homeopathy has repeatedly been shown to merely rely on placebo effects seems to count for nothing in Germany.
Perhaps this is going to change now. On January 30, a group of experts from all walks of life have met in Freiburg to discuss ways of informing the public responsibly and countering the plethora of misinformation that Germans are regularly exposed to on the subject of homeopathy. They founded the ‘Information Network Homeopathy’ and decided on a range of actions.
No doubt, some will ask where does their financial support come from? And no doubt, some will claim that we are on the payroll of ‘Big Pharma’. The truth is that we have no funding; everyone gives his/her own time free of charge and pays for his/her own expenses etc. And why? Because we believe in progress and feel strongly that it is time to improve healthcare by relegating homeopathy to the history books.
One of the first fruits of the network’s endeavours is the ‘Freiburger Erklärung zur Homöopathie’, the ‘Freiburg Declaration on Homeopathy’. I have the permission to reproduce the document here in full (the translation is mine):
HOMEOPATHY IS NEITHER NATUROPATHY NOR MEDICINE
Despite the support of politicians and the silence of those who should know better, homeopathy has remained a method which is in clear opposition to the proven basics of science. The members and supporter of the ‘Information Network Homeopathy’ view homeopathy as a stubbornly surviving belief system, which cannot be accepted as part of naturopathy nor medicine. The information network is an association of physicians, pharmacists, veterinarians, biologists, scientists and other critics of homeopathy who are united in their aim to disclose this fact more openly and make the public more aware of it.
NO SPECIAL STATUS FOR HOMEOPATHY
During the more than 200 years of its existence, homeopathy has not managed to demonstrate its specific effectiveness. Homeopathy only survives because it has been granted special status in the German healthcare system which is, in the opinion of the experts of the network, unjustified. Drugs have to prove their effectiveness according to objective criteria, but homeopathics are exempt from this obligation. We oppose such double standards in medicine.
Homeopathy has also not managed to demonstrate a plausible mode of action. Instead its proponents pretend that there are uncertainties which need to be clarified. We oppose such notions vehemently. Homeopathy is not an unconventional method that requires further scientific study. Its basis consists of long disproven theories such as the ‘law of similars’, ‘vital force’ or ‘potentisation by dilution’.
SELF-DECEPTION OF PATIENT AND THERAPIST
We do not dispute the therapeutic effects of a homeopathic treatment. But they are unrelated to the specific homeopathic remedy. The perceived effectiveness of homeopathics is due to suggestion and auto-suggestion of the patient and the therapist. The mechanisms of such (self-) deceit are multi-fold but well-known and researched. Symptomatic improvements caused by context-effects must not be causally associated with the homeopathic remedy. We assume that many physicians and alternative practitioners using homeopathy are unaware of the existence and multitude of such mechanisms and are acting in good faith. This, however, does not alter the fact that their conclusions are wrong and thus potentially harmful.
MEDICINE AND SCIENCE
We do not claim that the scientific method which we uphold can currently research and explain everything. However, it enables us to explain that homeopathy cannot explain itself. The scientific method shows the best way we have for differentiating effective from ineffective treatments. A popular belief in therapeutic claims nourished by politicians and journalists can never be a guide for medical activities.
AIM OF THIS DECLARATION
Our criticism is not aimed at needy patients or practising homeopathic clinicians; it is aimed at the school of homeopathy and the healthcare institutions which could have long recognised the nonsensical nature of homeopathy, but have chosen not to interfere. We ask the players within our science-based healthcare system to finally reject homeopathy and other pseudoscientific methods and to return to what should be self-evident: scientifically validated, fair and generally reproducible rules promoting top-quality medicine for he benefit of the patient.
Dr.-Ing. Norbert Aust, Initiator Informationsnetzwerk Homöopathie
Dr. med. Natalie Grams, Leiterin Informationsnetzwerk Homöopathie
Amardeo Sarma, GWUP Vorsitzender und Fellow von CSI (Committee for Skeptical Inquiry)
Edzard Ernst, Emeritus Professor, Universität Exeter, UK
Prof. Dr. Rudolf Happle, Verfasser der Marburger Erklärung zur Homöopathie
Prof. Dr. Wolfgang Hell, Vorsitzender des Wissenschaftsrates der GWUP
Prof. Norbert Schmacke, Institut für Public Health und Pflegeforschung, Universität Bremen
Dr. rer. nat. Christian Weymayr, freier Medizinjournalist
The randomized, placebo-controlled, double-blind trial is usually the methodology to test the efficacy of a therapy that carries the least risk of bias. This fact is an obvious annoyance to some alt med enthusiasts, because such trials far too often fail to produce the results they were hoping for.
But there is no need to despair. Here I provide a few simple tips on how to mislead the public with seemingly rigorous trials.
The most brutal method for misleading people is simply to cheat. The Germans have a saying, ‘Papier ist geduldig’ (paper is patient), implying that anyone can put anything on paper. Fortunately we currently have plenty of alt med journals which publish any rubbish anyone might dream up. The process of ‘peer-review’ is one of several mechanisms supposed to minimise the risk of scientific fraud. Yet alt med journals are more clever than that! They tend to have a peer-review that rarely involves independent and critical scientists, more often than not you can even ask that you best friend is invited to do the peer-review, and the alt med journal will follow your wish. Consequently the door is wide open to cheating. Once your fraudulent paper has been published, it is almost impossible to tell that something is fundamentally wrong.
But cheating is not confined to original research. You can also apply the method to other types of research, of course. For instance, the authors of the infamous ‘Swiss report’ on homeopathy generated a false positive picture using published systematic reviews of mine by simply changing their conclusions from negative to positive. Simple!
Obviously, outright cheating is not always as simple as that. Even in alt med, you cannot easily claim to have conducted a clinical trial without a complex infrastructure which invariably involves other people. And they are likely to want to have some control over what is happening. This means that complete fabrication of an entire data set may not always be possible. What might still be feasible, however, is the ‘prettification’ of the results. By just ‘re-adjusting’ a few data points that failed to live up to your expectations, you might be able to turn a negative into a positive trial. Proper governance is aimed at preventing his type of ‘mini-fraud’ but fortunately you work in alt med where such mechanisms are rarely adequately implemented.
Another very handy method is the omission of aspects of your trial which regrettably turned out to be in disagreement with the desired overall result. In most studies, one has a myriad of endpoints. Once the statistics of your trial have been calculated, it is likely that some of them yield the wanted positive results, while others do not. By simply omitting any mention of the embarrassingly negative results, you can easily turn a largely negative study into a seemingly positive one. Normally, researchers have to rely on a pre-specified protocol which defines a primary outcome measure. Thankfully, in the absence of proper governance, it usually is possible to publish a report which obscures such detail and thus mislead the public (I even think there has been an example of such an omission on this very blog).
Yes – lies, dam lies, and statistics! A gifted statistician can easily find ways to ‘torture the data until they confess’. One only has to run statistical test after statistical test, and BINGO one will eventually yield something that can be marketed as the longed-for positive result. Normally, researchers must have a protocol that pre-specifies all the methodologies used in a trial, including the statistical analyses. But, in alt med, we certainly do not want things to function normally, do we?
5 TRIAL DESIGNS THAT CANNOT GENERATE A NEGATIVE RESULT
All the above tricks are a bit fraudulent, of course. Unfortunately, fraud is not well-seen by everyone. Therefore, a more legitimate means of misleading the public would be highly desirable for those aspiring alt med researchers who do not want to tarnish their record to their disadvantage. No worries guys, help is on the way!
The fool-proof trial design is obviously the often-mentioned ‘A+B versus B’ design. In such a study, patients are randomized to receive an alt med treatment (A) together with usual care (B) or usual care (B) alone. This looks rigorous, can be sold as a ‘pragmatic’ trial addressing a real-fife problem, and has the enormous advantage of never failing to produce a positive result: A+B is always more than B alone, even if A is a pure placebo. Such trials are akin to going into a hamburger joint for measuring the calories of a Big Mac without chips and comparing them to the calories of a Big Mac with chips. We know the result before the research has started; in alt med, that’s how it should be!
I have been banging on about the ‘A+B versus B’ design often enough, but recently I came across a new study design used in alt med which is just as elegantly misleading. The trial in question has a promising title: Quality-of-life outcomes in patients with gynecologic cancer referred to integrative oncology treatment during chemotherapy. Here is the unabbreviated abstract:
Integrative oncology incorporates complementary medicine (CM) therapies in patients with cancer. We explored the impact of an integrative oncology therapeutic regimen on quality-of-life (QOL) outcomes in women with gynecological cancer undergoing chemotherapy.
PATIENTS AND METHODS:
A prospective preference study examined patients referred by oncology health care practitioners (HCPs) to an integrative physician (IP) consultation and CM treatments. QOL and chemotherapy-related toxicities were evaluated using the Edmonton Symptom Assessment Scale (ESAS) and Measure Yourself Concerns and Wellbeing (MYCAW) questionnaire, at baseline and at a 6-12-week follow-up assessment. Adherence to the integrative care (AIC) program was defined as ≥4 CM treatments, with ≤30 days between each session.
Of 128 patients referred by their HCP, 102 underwent IP consultation and subsequent CM treatments. The main concerns expressed by patients were fatigue (79.8 %), gastrointestinal symptoms (64.6 %), pain and neuropathy (54.5 %), and emotional distress (45.5 %). Patients in both AIC (n = 68) and non-AIC (n = 28) groups shared similar demographic, treatment, and cancer-related characteristics. ESAS fatigue scores improved by a mean of 1.97 points in the AIC group on a scale of 0-10 and worsened by a mean of 0.27 points in the non-AIC group (p = 0.033). In the AIC group, MYCAW scores improved significantly (p < 0.0001) for each of the leading concerns as well as for well-being, a finding which was not apparent in the non-AIC group.
An IP-guided CM treatment regimen provided to patients with gynecological cancer during chemotherapy may reduce cancer-related fatigue and improve other QOL outcomes.
A ‘prospective preference study’ – this is the design the world of alt med has been yearning for! Its principle is beautiful in its simplicity. One merely administers a treatment or treatment package to a group of patients; inevitably some patients take it, while others don’t. The reasons for not taking it could range from lack of perceived effectiveness to experience of side-effects. But never mind, the fact that some do not want your treatment provides you with two groups of patients: those who comply and those who do not comply. With a bit of skill, you can now make the non-compliers appear like a proper control group. Now you only need to compare the outcomes and BOB IS YOUR UNCLE!
Brilliant! Absolutely brilliant!
I cannot think of a more deceptive trial-design than this one; it will make any treatment look good, even one that is a mere placebo. Alright, it is not randomized, and it does not even have a proper control group. But it sure looks rigorous and meaningful, this ‘prospective preference study’!
If the Flat Earth Society (FES) really exists at all, I must confess I know nothing about it. Here I use the term ‘FES’ merely as an analogy; you might replace FES with SoH or BHA or BAA or BCA or with most of the other acronyms used in my field of inquiry.
What I do know about is alternative medicine, particularly publications in this area, and the authors of such papers. As it happens, the members of my imaginary FES have a lot in common with the authors of articles on alternative medicine. Their publication policy, for instance, is remarkably simple yet astonishingly effective. Its aim is straight forward: mislead the public. As far as I can see, it is being pursued by just two main strategies.
1 SWAMP THE MARKET WITH TRASH
This is a simple and most successful strategy. It consists of publishing an ever-growing mountain of utter nonsense. Anyone who is interested in alternative medicine and conducts a search would thus find tons of articles listed in Medline or other databases. This will instantly generate the impression that Flat Earth research is highly active. Those who can bear the pain might even try to read a few of these papers; they will soon give up in despair. Too many are hardly understandable; they are often badly written, lack essential methodological detail, and invariably arrive at positive conclusions.
The strategy can only work, if there are journals who publish such rubbish. I am glad to say, there is no shortage of them! To attain a veneer of credibility, the journals need to be peer-reviewed, of course. This is no real problem, as long as the peer-reviewers are carefully chosen to be ‘cooperative’. The trick is to make sure to ask the authors submitting articles to name two or three uncritical friends who might, one day, be happy to act as peer-reviewers for their own papers. This works very smoothly indeed: one pseudo-scientist is sure to help another in their desire to publish some pseudo-science in a ‘peer-reviewed’ journal.
To oil the system well, we need money, of course. Again, no problem: most of these journals ask for a hefty publication fee.
The result is as obvious as it is satisfying. The journal earns well, the pseudo-researchers can publish their pseudo-research at will, and the peer-reviewers know precisely where to go for a favour when they need one. Crucially, the first hurdle to misleading the public is taken with bravura.
2. REFUTE ANY EVIDENCE THAT IS UNFAVOURABLE
There are, of course, journals which refuse to play along. Annoyingly, they adhere to such old-fashioned things like standards and ethics; they have a peer-review system that is critical and independent; and they don’t rely on pseudo-scientists for their income. Every now and then, such a journal publishes an article on alternative medicine. It goes without saying that, in all likelihood, such an article is of high quality and therefore would not be in favour of Flat Earth assumptions.
This is a serious threat to the aim of the FES. What can be done?
No panic, the solution is simple!
An article is urgently needed to criticise the paper with the unfavourable evidence – never mind that it is of much better quality than the average paper in the Flat Earth-journals. If one looks hard enough, one can find a flaw in almost every article. And if there is none, the FES can always invent one. And if the proper science journal refuses to publish the pseudo-criticism as a comment, there are always enough pseudo-journals that are only too keen to oblige.
The important thing is to get something that vaguely looks like a rebuttal in print (the public will not realise that it is phony!).
Once this aim is achieved, the world is back in order again. As soon as someone dares to cite the high quality, negative evidence, the FES members can all shout with one voice: BUT THIS PAPER HAS BEEN HEAVILY CRITICISED; IT IS NOT RELIABLE! WHOEVER CITED THE PAPER IS ILL-INFORMED AND THEREFORE NOT CREDIBLE.
3. MISSION ACCOMPLISHED
The overall effect is clear. The public, journalists, politicians etc. get the impression that the earth is indeed flat – or, at the very minimum, they are convinced that there is a real scientific debate about the question.
Homeopathy seems to attract some kind of miracle worker. Elsewhere I have, for instance, reported the curious case of Prof Claudia Witt who published more than anyone on homeopathy in recent years without hardly ever arriving at a negative conclusion. Recently, I came across a researcher with an even better track record: Prof Michael Frass.
Wikipedia describes his achievements as follows: “Michael Frass studied medicine from 1972 to 1978 at the Medical University of Vienna followed by visits abroad at the Pasteur Institute, Paris and at the Porter Memorial Hospital (USA). Since March 2004 he directs the Outpatients Unit of Homeopathy for Malign Diseases at the Department Clinic for Internal of Medicine I at the Medical University of Vienna. Since 2005 Frass also works as a coordinator of the lecture series Homeopathy at the Medical University of Vienna. Beginning with the winter semester 2001/02 he is the coordinator of a lecture series Basics and practise of complementary medical methods at the Medical University of Vienna. From 2002 to 2005 he led the Ludwig Boltzmanm Institute of Homeopathy. Since 2005 Frass is president of the Institute for Homeopathic Research. Actually he works at the Division of Oncology at the Department of Medicine I in Vienna. He is First Chairman of the Scientific Society for Homeopathy (WissHom), founded in 2010, president of the Umbrella organization of Austrian Doctors for Holistic Medicine.”
He directs the WHAT? The Outpatients Unit of Homeopathy for Malign Diseases at the Department Clinic for Internal of Medicine I at the Medical University of Vienna? This is my former medical school, and I had no idea that such a unit even existed – but, of course, I left in 1993 for Exeter (a few months ago, I followed an invitation to give a lecture on homeopathy at the Medical University of Vienna ; sadly neither Prof Frass nor anyone of his team attended).
And what about the Scientific Society for Homeopathy? I am sure that the name of this organisation will make some people wonder. From the society’s website, we learn that “the intention of WissHom is to contribute to the progress of medicine and to the collective good. To this end, WissHom intents to further develop homeopathy both practically and theoretically. It will be WissHom’s task to breathe life into this committed objective.”
Breathing life into homeopathy seems exactly what Prof Frass does. He seems to have found his way to homeopathy relatively late in his career (the 1st Medline-listed article was published only in 2003) but he has nevertheless published many studies on this subject (I use the term ‘study’ here to describe both clinical, pre-clinical and basic research papers); in total, I found 12 such articles on Medline. They cover extremely diverse areas and a wide range of methodologies. Yet they all have one remarkable feature in common: they arrive at positive conclusions.
You find this hard to believe? Join the club!
But it is undeniably true, here are the conclusions (or the bit that comes close to a conclusion) from the Medline-listed abstracts (only the headings in capital letters are mine, and they simply depict the nature of the paper)
Results suggest that the global health status and subjective wellbeing of cancer patients improve significantly when adjunct classical homeopathic treatment is administered in addition to conventional therapy.
Based on the 2 cases, including 1 extreme situation, we suggest that adjunctive homeopathic treatment has a role in the treatment of acute Amanita phalloides-induced toxicity following mushroom poisoning. Additional studies may clarify a more precise dosing regimen, standardization, and better acceptance of homeopathic medicine in the intensive care setting.
Extended survival time in this sample of cancer patients with fatal prognosis but additive homeopathic treatment is interesting. However, findings are based on a small sample, and with only limited data available about patient and treatment characteristics. The relationship between homeopathic treatment and survival time requires prospective investigation in larger samples possibly using matched-pair control analysis or randomized trials.
The symptoms of patients undergoing homeopathic treatment were shown to improve substantially and conventional medication dosage could be substantially reduced. While the real-life effect assessed indicates that there is a potential for enhancing therapeutic measures and reducing healthcare cost, it does not allow to draw conclusions as to the efficacy of homeopathic treatment per se.
The data suggest that both drugs prepared in ethanolic solution are potent inhibitors of H. pylori induced gene expression.
Most of these clinical studies have been deemed to be high quality trials, according to the three most commonly referenced meta-analyses of homeopathic research. Basic in vitro experimental studies also provide evidence that the effects of homeopathy differ from placebo.
This study is based on 25 well documented reports of cases which responded well to treatment with Petroleum.
Animals treated with the standard test solution thyroxine 10(-30) metamorphosed more slowly than the control animals, ie the effect of the homeopathically prepared thyroxine was opposed to the usual physiological effect of molecular thyroxine.
Our report suggests that homeopathy may be applicable even for critically ill patients.
Our data suggest that homeopathic treatment may be a useful additional therapeutic measure with a long-term benefit for severely septic patients admitted to the intensive care unit. A constraint to wider application of this method is the limited number of trained homeopaths.
These data suggest that potentized (diluted and vigorously shaken) potassium dichromate may help to decrease the amount of stringy tracheal secretions in COPD patients.
These animals reacted to the homeopathically prepared thyroxine with a slowing down of metamorphosis, even when they had not been prestimulated with a molecular dose of the hormone. This effect was observed in all 3 laboratories and is consistent with the results of previous studies.
So am I!
How can homeopathy produce nothing but positive results in the hands of this researcher? How can it work in so many entirely different conditions? How is it possible that homeopathic remedies are better than placebo regardless of the methodology used? Why does homeopathy, in the hands of Prof Frass, not even once produce a result that disappoints the aspirations of homeopaths and its advocates? Why are these sensational results almost invariably published in very minor journals? Crucially, why has not one of the findings (as far as I can see) ever been independently reproduced?
I do not know the answers to these questions.
If anyone does, I would like to hear them.
I just came across a website that promised to”cover 5 common misconceptions about alternative medicine that many people have”. As much of this blog is about this very issue, I was fascinated. Here are Dr Cohen’s 5 points in full:
5 Misconceptions about Alternative Medicine Today
1. Alternative Medicine Is Only an Alternative
In fact, many alternative practitioners are also medical doctors, chiropractors, or other trained medical professionals. Others work closely with MDs to coordinate care. Patients should always let all of their health care providers know about treatments that they receive from all the others.
2. Holistic Medicine Isn’t Mainstream
In fact, scientists and doctors do perform studies on all sorts of alternative therapies to determine their effectiveness. These therapies, like acupuncture and an improved diet, pass the test of science and then get integrated into standard medical practices.
3. Natural Doctors Don’t Use Conventional Medicine
No credible natural doctor will ever tell a patient to replace prescribed medication without consulting with his or her original doctor. In many cases, the MD and natural practitioner are the same person. If not, they will coordinate treatment to benefit the health of the patient.
4. Alternative Medicine Doesn’t Work
Actual licensed health providers won’t just suggest natural therapies on a whim. They will consider scientific studies and their own experience to suggest therapies that do work. Countless studies have, for example, confirmed that acupuncture is an effective treatment for many medical conditions. Also, the right dietary changes are known to help improve health and even minimize or cure some diseases. Numerous other alternative therapies have been proven effective using scientific studies.
5. Big Medical Institutions are Against Alternative Medicine
According to a recent survey, about half of big insurers pay for tested alternative therapies like acupuncture. Also, hospitals and doctors do recognize that lifestyle changes, some herbal remedies, and other kinds of alternative medicine may reduce side effects, allow patients to reduce prescription medicine, and even lower medical bills.
This is not to say that every insurer, doctor, or hospital will support a particular treatment. However, patients are beginning to take more control of their health care. If their own providers won’t suggest natural remedies, it might be a good idea to find one who does.
The Best Medicine Combines Conventional and Alternative Medicine
Everyone needs to find the right health care providers to enjoy the safest and most natural care possible. Good natural health providers will have a solid education in their field. Nobody should just abandon their medical treatment to pursue alternative cures. However, seeking alternative therapies may help many people reduce their reliance on harsh medications by following the advice of alternative providers and coordinating their care with all of their health care providers.
END OF COHEN’S TEXT
COMMENT BY MYSELF
Who the Dickens is Dr Cohen and what is his background? I asked myself after reading this. From his website, it seems that he is a chiropractor from North Carolina – not just any old chiro, but one of the best!!! – who also uses several other dubious therapies. He sums up his ‘philosophy’ as follows:
There is an energy or life force that created us (all 70 trillion cells that we are made of) from two cells (sperm and egg cells). This energy or innate intelligence continues to support you throughout life and allows you to grow, develop, heal, and express your every potential. This life force coordinates all cells, tissues, muscles and organs by sending specific, moment by moment communication via the nervous system. If the nervous system is over-stressed or interfered with in any way, then your life force messages will not be properly expressed.
Here he is on the cover of some magazine and here is also his ‘PAIN CLINIC’
Fascinating stuff, I am sure you agree.
As I do not want to risk a libel case, I will abstain from commenting on Dr Cohen and his methods or beliefs. Instead I will try to clear up a few misconceptions that are pertinent to him and the many other practitioners who are promoting pure BS via the Internet.
- Not everyone who uses the title ‘Dr’ is a doctor in the sense of having studied medicine.
- Chiropractors are not ‘trained medical professionals’.
- The concepts of ‘vitalism’, ‘life force’ etc. have been abandoned in real heath care a long time ago, and medicine has improved hugely because of this.
- Hardly any alternative therapy has ‘passed the test of science’.
- Therefore, it is very doubtful whether alternative practitioners actually will ‘consider scientific studies’.
- True, some trials did suggest that acupuncture is an effective treatment for many medical conditions; but their methodological quality is often far too low to draw firm conclusions and many other, often better studies have shown the contrary.
- Numerous other alternative therapies have been proven ineffective using scientific studies.
- Therefore it might be a good idea to find a health care provider who does not offer unproven treatments simply to make a fast buck.
- Seeking alternative therapies may harm many people.
Dear Professor Robinson,
please forgive me for writing to you in a matter that, you might think, is really none of my business. I have been following the news and discussions about the BLACKMORE CHAIR at your university. Having been a professor of complementary medicine at Exeter for ~20 years and having published more papers on this subject than anyone else on the planet, I am naturally interested and would like to express some concerns, if you allow me to.
With my background, I would probably be the last person to argue that a research chair in alternative medicine is not a good and much-needed thing. However, accepting an endowment from a commercially interested source is, as you are well aware, a highly problematic matter.
I am confident that you intend to keep the sponsor at arm’s length and plan to appoint a true scientist to this post who will not engage in the promotional activities which the alternative medicine scene might be expecting. And I am equally sure that the money will be put to good use resulting in good and fully independent science.
But, even if all of this is the case, there are important problems to consider. By accepting Blackmore’s money, you have, perhaps inadvertently, given credit to a commercially driven business empire. As you probably know, Blackmores have a reputation of being ‘a bit on the cavalier side’ when it comes to rules and regulations. This is evidenced, for instance, by the number of complaints that have been upheld against them by the Australian authorities.
For these reasons, the creation of the new chair is not just a step towards generating research, it could (and almost inevitably will) be seen as a boost for quackery. It is foremost this aspect which might endanger the reputation of your university, I am afraid.
My own experience over the last two decades has taught me to be cautious and sceptical regarding the motives of many involved in the multi-billion alternative medicine business. I have recently published my memoir entitled ‘A SCIENTIST IN WONDERLAND. SEARCHING FOR TRUTH AND FINDING TROUBLE’; it might be a helpful read for you and the new professor.
I hope you take my remarks as they were meant: constructive advice from someone who had to learn it all the hard way. If I can be of further assistance, please do not hesitate to ask me.
A recent comment to a post of mine (by a well-known and experienced German alt med researcher) made the following bold statement aimed directly at me and at my apparent lack of understanding research methodology:
C´mon , as researcher you should know the difference between efficacy and effectiveness. This is pharmacological basic knowledge. Specific (efficacy) + nonspecific effects = effectiveness. And, in fact, everything can be effective – because of non-specific or placebo-like effects. That does not mean that efficacy is existent.
The point he wanted to make is that outcome studies – studies without a control group where the researcher simply observe the outcome of a particular treatment in a ‘real life’ situation – suffice to demonstrate the effectiveness of therapeutic interventions. This belief is very wide-spread in alternative medicine and tends to mislead all concerned. It is therefore worth re-visiting this issue here in an attempt to create some clarity.
When a patient’s condition improves after receiving a therapy, it is very tempting to feel that this improvement reflects the effectiveness of the intervention (as the researcher mentioned above obviously does). Tempting but wrong: there are many other factors involved as well, for instance:
- the placebo effect (mainly based on conditioning and expectation),
- the therapeutic relationship with the clinician (empathy, compassion etc.),
- the regression towards the mean (outliers tend to return to the mean value),
- the natural history of the patient’s condition (most conditions get better even without treatment),
- social desirability (patients tend to say they are better to please their friendly clinician),
- concomitant treatments (patients often use treatments other than the prescribed one without telling their clinician).
So, how does this fit into the statement above ‘Specific (efficacy) + nonspecific effects = effectiveness’? Even if this formula were correct, it would not mean that outcome studies of the nature described demonstrate the effectiveness of a therapy. It all depends, of course, on what we call ‘non-specific’ effects. We all agree that placebo-effects belong to this category. Probably, most experts also would include the therapeutic relationship and the regression towards the mean under this umbrella. But the last three points from my list are clearly not non-specific effects of the therapy; they are therapy-independent determinants of the clinical outcome.
The most important factor here is usually the natural history of the disease. Some people find it hard to imagine what this term actually means. Here is a little joke which, I hope, will make its meaning clear and memorable.
CONVERATION BETWEEN TWO HOSPITAL DOCTORS:
Doc A: The patient from room 12 is much better today.
Doc B: Yes, we stared his treatment just in time; a day later and he would have been cured without it!
I am sure that most of my readers now understand (and never forget) that clinical improvement cannot be equated with the effectiveness of the treatment administered (they might thus be immune to the misleading messages they are constantly exposed to). Yet, I am not at all sure that all ‘alternativists’ have got it.