According to its authors, this RCT was aimed at investigating the 1) specific effect of individualized homeopathic Q-potencies compared to placebo and 2) the effect of an extensive homeopathic case taking (case history I) compared to a shorter, rather conventional one (case history II) in the treatment of acute major depression. In particular the second research question is intriguing, I think – so let’s have a closer look at this trial.
The study was designed as a randomized, partially double-blind, placebo-controlled, four-armed, 2×2 factorial trial with a 6-week study duration. A total of 44 patients were randomized (2∶1∶2∶1 randomization: 16 homeopathic Q-potencies/case history I, 7 placebo/case history I, 14 homeopathic Q-potencies/case history II, 7 placebo/case history II). Because of recruitment problems, the study was terminated prior to full recruitment, and was thus underpowered for the pre-planned confirmatory hypothesis testing. Exploratory data analyses showed heterogeneous and inconclusive results with large variance. The mean difference for the Hamilton-D after 6 weeks was 2.0 (95%CI -1.2;5.2) for Q-potencies vs. placebo, and -3.1 (-5.9;-0.2) for case history I vs. case history II. Overall, no consistent or clinically relevant results between homeopathic Q-potencies versus placebo and homeopathic versus conventional case taking were observed. The frequency of adverse events was comparable for all groups.
The conclusions were remarkable: although our results are inconclusive, given that recruitment into this trial was very difficult and we had to terminate early, we cannot recommend undertaking a further trial addressing this question in a similar setting.
Alright, the authors encountered problems in recruiting enough patients and they therefore decided to stop the trial early. This sort of thing happens. Most researchers would then not publish any data at all. This team, however, did publish a report, and the decision to do so might be perfectly fine: other investigators might learn from the problems which led to early termination of the study.
But why do they conclude that the results were INCONCLUSIVE? I think the results were not inconclusive but non-existent; these were no results to report other than those related to the recruitment problems. And even if one insists on presenting outcome data as an exploratory analysis, one cannot honestly say they were INCONCLUSIVE, all one might state in this case is that the results failed to show an effect of the remedy or the consultation. This is far less favourable for homeopathy than stating the results were INCONCLUSIVE.
And why on earth do the authors conclude “we cannot recommend undertaking a further trial addressing this question in a similar setting”? This does not make the slightest sense to me. If the trialists encountered recruitment problems, others might find ways of overcoming them. The research question asking whether the effects of an extensive homeopathic case taking differ from those of a shorter conventional one seems important. If answered accurately, it could disentangle much of the confusion that surrounds clinical trials of homeopathy.
I have repeatedly commented on the odd conclusions drawn by proponents of alternative medicine on the basis of data that did not quite fulfil their expectations, and I often ask myself at what point this ‘prettification’ of the results via false positive conclusions crosses the line to scientific misconduct. My theory is that these conclusions appear odd to those capable of critical analysis because the authors bend over backwards in order to conclude more positively than the data would seem to permit. If we see it this way, such conclusions might even prove useful as a fairly sensitive ‘bullshit-detector’.
We have probably all fallen into the trap of thinking that something which has stood the ‘test of time’, i.e. something that has been used for centuries with apparent success, must be ok. In alternative medicine, this belief is extremely wide-spread, and one could argue that the entire sector is built on it. Influential proponents of ‘traditional’ medicine like Prince Charles do their best to strengthen this assumption. Sadly, however, it is easily disclosed as a classical fallacy: things that have stood the ‘test of time’ might work, of course, but the ‘test of time’ is never a proof of anything.
A recent study brought this message home loud and clear. This trial tested the efficacy of Rhodiola crenulata (R. crenulata), a traditional remedy which has been used widely in the Himalayan areas and in Tibet to prevent acute mountain sickness . As no scientific studies of this traditional treatment existed, the researchers conducted a double-blind, placebo-controlled crossover RCT to test its efficacy in acute mountain sickness prevention.
Healthy adult volunteers were randomized to two treatment sequences, receiving either 800 mg R. crenulata extract or placebo daily for 7 days before ascent and two days during mountaineering. After a three-month wash-out period, they were crossed over to the alternate treatment. On each occasion, the participants ascended rapidly from 250 m to 3421 m. The primary outcome measure was the incidence of acute mountain sickness with headache and at least one of the symptoms of nausea or vomiting, fatigue, dizziness, or difficulty sleeping.
One hundred and two participants completed the trial. No significant differences in the incidence of acute mountain sickness were found between R. crenulata extract and placebo groups. If anything, the incidence of severe acute mountain sickness with Rhodiola extract was slightly higher compared to the one with placebo: 35.3% vs. 29.4%.
R. crenulata extract was not effective in reducing the incidence or severity of acute mountain sickness as compared to placebo.
Similar examples could be found by the dozen. They demonstrate very clearly that the notion of the ‘test of time’ is erroneous: a treatment which has a long history of usage is not necessarily effective (or safe) – not only that, it might be dangerous. The true value of a therapy cannot be judged by experience, to be sure, we need rigorous clinical trials. Acute mountain sickness is a potentially life-threatening condition for which there are reasonably effective treatments. If people relied on the ‘ancient wisdom’ instead of using a therapy that actually works, they might pay for their error with their lives. The sooner alternative medicine proponents realise that, the better.
Acupressure is a treatment-variation of acupuncture; instead of sticking needles into the skin, pressure is applied over ‘acupuncture points’ which is supposed to provide a stimulus similar to needling. Therefore the effects of both treatments should theoretically be similar.
Acupressure could have several advantages over acupuncture:
- it can be used for self-treatment
- it is suitable for people with needle-phobia
- it is painless
- it is not invasive
- it has less risks
- it could be cheaper
But is acupressure really effective? What do the trial data tell us? Our own systematic review concluded that the effectiveness of acupressure is currently not well documented for any condition. But now there is a new study which might change this negative verdict.
The primary objective of this 3-armed RCT was to assess the effectiveness and cost-effectiveness of self-acupressure using wristbands compared with sham acupressure wristbands and standard care alone in the management of chemotherapy-induced nausea. 500 patients from outpatient chemotherapy clinics in three regions in the UK involving 14 different cancer units/centres were randomised to the wristband arm, the sham wristband arm and the standard care only arm. Participants were chemotherapy-naive cancer patients receiving chemotherapy of low, moderate and high emetogenic risk. The experimental group were given acupressure wristbands pressing the P6 point (anterior surface of the forearm). The Rhodes Index for Nausea/Vomiting, the Multinational Association of Supportive Care in Cancer (MASCC) Antiemesis Tool and the Functional Assessment of Cancer Therapy General (FACT-G) served as outcome measures. At baseline, participants completed measures of anxiety/depression, nausea/vomiting expectation and expectations from using the wristbands.
Data were available for 361 participants for the primary outcome. The primary outcome analysis (nausea in cycle 1) revealed no statistically significant differences between the three arms. The median nausea experience in patients using wristbands (both real and sham ones) was somewhat lower than that in the anti-emetics only group (median nausea experience scores for the four cycles: standard care arm 1.43, 1.71, 1.14, 1.14; sham acupressure arm 0.57, 0.71, 0.71, 0.43; acupressure arm 1.00, 0.93, 0.43, 0). Women responded more favourably to the use of sham acupressure wristbands than men (odds ratio 0.35 for men and 2.02 for women in the sham acupressure group; 1.27 for men and 1.17 for women in the acupressure group). No significant differences were detected in relation to vomiting outcomes, anxiety and quality of life. Some transient adverse effects were reported, including tightness in the area of the wristbands, feeling uncomfortable when wearing them and minor swelling in the wristband area (n = 6). There were no statistically significant differences in the costs associated with the use of real acupressure band.
26 subjects took part in qualitative interviews. Participants perceived the wristbands (both real and sham) as effective and helpful in managing their nausea during chemotherapy.
The authors concluded that there were no statistically significant differences between the three arms in terms of nausea, vomiting and quality of life, although apparent resource use was less in both the real acupressure arm and the sham acupressure arm compared with standard care only; therefore; no clear conclusions can be drawn about the use of acupressure wristbands in the management of chemotherapy-related nausea and vomiting. However, the study provided encouraging evidence in relation to an improved nausea experience and some indications of possible cost savings to warrant further consideration of acupressure both in practice and in further clinical trials.
I could argue about several of the methodological details of this study. But I resist the temptation in order to focus on just one single point which I find important and which has implications beyond the realm of acupressure.
Why on earth do the authors conclude that no clear conclusions can be drawn about the use of acupressure wristbands in the management of chemotherapy-related nausea and vomiting? The stated aim of this RCT was to assess the effectiveness and cost-effectiveness of self-acupressure using wristbands compared with sham acupressure wristbands and standard care. The results failed to show significant differences of the primary outcome measures, consequently the conclusion cannot be “unclear”, it has to be that ACUPRESSURE WRIST BANDS ARE NOT MORE EFFECTIVE THAN SHAM ACUPRESSURE WRIST BANDS AS AN ADJUNCT TO ANTI-EMETIC DRUG TREATMENT (or something to that extent).
As long as RCTs of alternative therapies are run by evangelic believers in the respective therapy, we are bound to regularly encounter this lamentable phenomenon of white-washing negative findings with an inadequate conclusion. In my view, this is not research or science, it is pseudo-research or pseudo-science. And it is much more than a nuisance or a trivial matter; it is a waste of research funds, a waste of patients’ good will that has reached a point where people will lose trust in alternative medicine research. Someone should really do a systematic study to identify those research teams that regularly commit such scientific misconduct and ensure that they are cut off public funding and support.
A recent article by a South African homeopath promoted the concept of homeopaths taking over the role of primary care practitioners. His argument essentially was that, in South Africa, homeopaths are well trained and thus adequately equipped to do this job responsibly. Responsibly, really? You find that hard to believe? Here are the essentials of his arguments including all his references in full. I think they are worth reading.
Currently, the Durban University of Technology (DUT) and the University of Johannesburg (UJ) offer degree’s in homoeopathy. This involves a 5-year full-time theoretical and practical training course, followed by a Master’s level research project. After fulfilment of these criteria, a Master’s Degree in Technology (Homoeopathy) is awarded. The course comprises of a strong core of medical subjects, such as the basic sciences of Anatomy, Physiology, Medical Microbiology, Biochemistry and Epidemiology, and the clinical sciences of Pathology and Diagnostics. This is complemented with subjects in Classical, Clinical and Modern Homoeopathy and Homoeopharmaceutics4,5…
By law, any person practicing homoeopathy in South Africa must be registered with the Allied Health Professions Council of South Africa (AHPCSA). This is essential, as the Council ensures both medical and homoeopathic competency of practitioners, and that the activities of registered practitioners are closely monitored by the Professional Board. The purpose of the AHPCSA is to ensure that only those with legitimate qualifications of a high enough standard are registered and allowed to practice in South Africa, thus protecting the public against any fraudulent behaviour and illegal practitioners. Therefore, in order to ensure effective homoeopathic treatment, it is essential that any person wishing to prescribe homoeopathic medicine or practice homoeopathy in South Africa must be registered as a Homoeopathic Practitioner with the Allied Health Professions Council of South Africa. This includes conventional Medical Practitioners (dual registration is allowed for Medical Practitioners with both the Health Professions Council and AHPCSA)6, as homoeopathy requires several years of training in order to apply effectively in clinical practice…
Registration with the Council affords medico-legal rights similar to those of a medical professional, where treatment is limited to the scope of homoeopathic practice. Thus a homoeopath is firstly a trained diagnostician, and with successful registration with the Council, obtains the title Doctor. A homoeopath is trained and legally obliged to conduct a full medical history, a comprehensive clinical examination, and request further medical investigations, such as blood tests and X-rays, in order to fully assess patients. This is coupled with the ability to consult with specialist pathologists and other medical specialists when necessary, and refer a patient to the appropriate practitioner if the condition falls outside the scope of homoeopathic practice. A homoeopath may also legally issue a certificate of dispensation (‘Doctor’s note’) with appropriate evidence and within reason, and is deemed responsible for the diagnosis and treatment of patients under their care6. A homoeopath is not trained or licensed in any form of surgery, specialist diagnostics (e.g. colonoscopy or angiograms), cannot prescribe prescription medication and is not lawfully allowed to conduct intra-venous treatment of any kind. However, a registered homoeopath is licensed to use intra-muscular homoeopathic injectables in the treatment of various local or systemic complaints when necessary.
Conventional (allopathic) medicine generally targets specific biochemical processes with mostly chemically synthesised medication, in an attempt to suppress a symptom. However, in doing so, this usually negatively affects other biochemical reactions which results in an imbalance within the system. Homoeopathy, by contrast, seeks to re-establish a balance within the natural functioning of the body, restore proper function and results in the reduction or cessation of symptoms. Homoeopathy therefore enables the body to self-regulate and self-heal, a process known as homeostasis that is intrinsic to every living organism.
Conventional medical treatment is by no means risk free. Iatrogenic (medically induced) deaths in the United States are estimated at 786 000 per year, deaths which are considered avoidable by medical doctors7,8. These figures put annual iatrogenic death in the American medical system above that of cardiovascular disease and cancer as the leading cause of death in that country9, a fact that is not widely reported! South African figures are not easily available, but it is likely that we have similar rates. Although conventional medications have a vital role, are sometimes necessary and can of-course be life-saving, all too often too many patients are put on chronic medication when there are numerous effective, natural, safe and scientifically substantiated options available….
According to the World Health Organisation (WHO), homeopathy is the second largest system of medicine in the world, and world-wide use continues to grow in developed and developing nations10. Homoeopathy is widely considered to be safe and effective, with both clinical and laboratory research providing evidence for the efficacy of homoeopathy11. As the range of potential conditions that homoeopathy can treat is almost limitless, and that treatment is not associated with adverse reactions, homoeopathy should be considered a first-line therapy for all ages. As homoeopaths in South Africa are considered primary health care practitioners, if a conventional approach is deemed necessary, and further diagnostics are required, your practitioner will not hesitate to refer you to the appropriate health care practitioner. Homeopathy is also used alongside conventional medicine and any other form of therapy, and should be seen as ‘complementary’ medicine and not ‘alternative’ medicine.
Homoeopathy is an approach that is widely considered to be safe, and when utilised correctly, can be effective for a wide range of conditions. As a primary health care practitioner, a homoeopath is able to handle all aspects of general practice and family health care, including diagnostics, case management and referral to other practitioners or medical specialists. A registered homoeopath is legally responsible to ensure the adequate treatment of their patients, and is accountable for all clinical decisions and advice. A registered homoeopath understands the role of conventional medicine, and will refer to the appropriate specialist in cases that fall outside the legal scope of practice.
1. http://homeopathyresource.wordpress.com/what-is-homeopathy (accessed 31 March 2010)
2. Bloch R, Lewis B. Homoeopathy for the home. Cape Town, South Africa: Struik Publishers: 2003
3. http://www.dut.ac.za/site/awdep.asp?depnum=22609 (accessed 1 April 2010)
4. http://dutweb.dut.ac.za/handbooks/HEALTH%20Homoeopathy.pdf (accessed 1 April 2010)
5. http://www.uj.ac.za/EN/Faculties/health/departments/homeopathy/coursesandprogrammes/undergraduate/Pages/default.aspx (accessed 1 April 2010)
6. http://www.ahpcsa.co.za/pb_pbhnp_homoeopathy.htm (accessed 6 April 2010)
7. Starfield, B. Is US Health Really the Best in the World? JAMA 2000; 284(4).
8. Null G, Dean C, et al. Death by Medicine. Nutrition Institute of America 2003. 9. http://www4.dr-rath-foundation.org/features/death_by_medicine.html (accessed 7 April 2010)
10. http://ukiahcommunityblog.wordpress.com/2010/03/04/worldwide-popularity-grows-for-homeopathy-alternative-medicine/#comments (accessed 7 April 2010)
11. http://liga.iwmh.net/dokumente/upload/556c7_SCIEN_FRA_2009_final_approved.pdf (accessed 7 April 2010)
I found this article extremely revealing and scary. It gives us an important glimpse into the way some or perhaps even most homeopaths think. They clearly believe that:
1) Their training is sufficient for them to become competent primary care professionals, i.e. clinicians who are the first port of call for sick people to be diagnosed and treated effectively.
2) Homeopathy is scientifically proven to be efficacious for an ‘almost limitless’ range of conditions. Interestingly, not a single reference is provided to support this claim. Nevertheless, homeopath believe it, and that seems to be enough.
3) Homeopaths seem convinced that they perfectly understand real medicine; yet all they really do is to denounce it as one of the biggest killer of mankind.
4) The fact that homeopaths cannot prescribe real medicine is not seen as a hindrance to their role as primary care practitioner; if anything, homeopaths consider this to be an advantage.
5) Homeopaths view registration with some sort of governing body as the ultimate legitimation of their trade. Once such regulatory measures are in place, the need to support any of their claims with evidence is nil and void.
This article did remind me of the wry statement that ‘HOMEOPATHY IS TO MEDICINE WHAT THE CARPET INDUSTRY IS TO AVIATION’. Homeopaths truly live on a different planet, a planet where belief is everything and responsibility is an alien concept. I certainly hope that they will not take over planet earth in a hurry. If I imagine a world where homeopaths dominate primary care in the way it is suggested in this article, I start having nightmares. It seems to me that people who harbour ideas of this type are not just deluded to the point of madness but they are a danger to public health.
This post will probably work best, if you have read the previous one describing how the parallel universe of acupuncture research insists on going in circles in order to avoid admitting that their treatment might not be as effective as they pretend. The way they achieve this is fairly simple: they conduct trials that are designed in such a way that they cannot possibly produce a negative result.
A brand-new investigation which was recently vociferously touted via press releases etc. as a major advance in proving the effectiveness of acupuncture is an excellent case in point. According to its authors, the aim of this study was to evaluate acupuncture versus usual care and counselling versus usual care for patients who continue to experience depression in primary care. This sounds alright, but wait!
755 patients with depression were randomised to one of three arms to 1)acupuncture, 2)counselling, and 3)usual care alone. The primary outcome was the difference in mean Patient Health Questionnaire (PHQ-9) scores at 3 months with secondary analyses over 12 months follow-up. Analysis was by intention-to-treat. PHQ-9 data were available for 614 patients at 3 months and 572 patients at 12 months. Patients attended a mean of 10 sessions for acupuncture and 9 sessions for counselling. Compared to usual care, there was a statistically significant reduction in mean PHQ-9 depression scores at 3 and 12 months for acupuncture and counselling.
From this, the authors conclude that both interventions were associated with significantly reduced depression at 3 months when compared to usual care alone.
Acupuncture for depression? Really? Our own systematic review with co-authors who are the most ardent apologists of acupuncture I have come across showed that the evidence is inconsistent on whether manual acupuncture is superior to sham… Therefore, I thought it might be a good idea to have a closer look at this new study.
One needs to search this article very closely indeed to find out that the authors did not actually evaluate acupuncture versus usual care and counselling versus usual care at all, and that comparisons were not made between acupuncture, counselling, and usual care (hints like the use of the word “alone” are all we get to guess that the authors’ text is outrageously misleading). Not even the methods section informs us what really happened in this trial. You find this hard to believe? Here is the unabbreviated part of the article that describes the interventions applied:
Patients allocated to the acupuncture and counselling groups were offered up to 12 sessions usually on a weekly basis. Participating acupuncturists were registered with the British Acupuncture Council with at least 3 years post-qualification experience. An acupuncture treatment protocol was developed and subsequently refined in consultation with participating acupuncturists. It allowed for customised treatments within a standardised theory-driven framework. Counselling was provided by members of the British Association for Counselling and Psychotherapy who were accredited or were eligible for accreditation having completed 400 supervised hours post-qualification. A manualised protocol, using a humanistic approach, was based on competences independently developed for Skills for Health. Practitioners recorded in logbooks the number and length of sessions, treatment provided, and adverse events. Further details of the two interventions are presented in Tables S2 and S3. Usual care, both NHS and private, was available according to need and monitored for all patients in all three groups for the purposes of comparison.
It is only in the results tables that we can determine what treatments were actually given; and these were:
1) Acupuncture PLUS usual care (i.e. medication)
2) Counselling PLUS usual care
3) Usual care
Its almost a ‘no-brainer’ that, if you compare A+B to B (or in this three-armed study A+B vs C+B vs B), you find that the former is more than the latter – unless A is a negative, of course. As acupuncture has significant placebo-effects, it never can be a negative, and thus this trial is an entirely foregone conclusion. As, in alternative medicine, one seems to need experimental proof even for ‘no-brainers’, we have some time ago demonstrated that this common sense theory is correct by conducting a systematic review of all acupuncture trials with such a design. We concluded that the ‘A + B versus B’ design is prone to false positive results…What makes this whole thing even worse is the fact that I once presented our review in a lecture where the lead author of the new trial was in the audience; so there can be no excuse of not being aware of the ‘no-brainer’.
Some might argue that this is a pragmatic trial, that it would have been unethical to not give anti-depressants to depressed patients and that therefore it was not possible to design this study differently. However, none of these arguments are convincing, if you analyse them closely (I might leave that to the comment section, if there is interest in such aspects). At the very minimum, the authors should have explained in full detail what interventions were given; and that means disclosing these essentials even in the abstract (and press release) – the part of the publication that is most widely read and quoted.
It is arguably unethical to ask patients’ co-operation, use research funds etc. for a study, the results of which were known even before the first patient had been recruited. And it is surely dishonest to hide the true nature of the design so very sneakily in the final report.
In my view, this trial begs at least 5 questions:
1) How on earth did it pass the peer review process of one of the most highly reputed medical journals?
2) How did the protocol get ethics approval?
3) How did it get funding?
4) Does the scientific community really allow itself to be fooled by such pseudo-research?
5) What do I do to not get depressed by studies of acupuncture for depression?
Has it ever occurred to you that much of the discussion about cause and effect in alternative medicine goes in circles without ever making progress? I have come to the conclusion that it does. Here I try to illustrate this point using the example of acupuncture, more precisely the endless discussion about how to best test acupuncture for efficacy. For those readers who like to misunderstand me I should explain that the sceptics’ view is in capital letters.
At the beginning there was the experience. Unaware of anatomy, physiology, pathology etc., people started sticking needles in other people’s skin, some 2000 years ago, and observed that they experienced relief of all sorts of symptoms.When an American journalist reported about this phenomenon in the 1970s, acupuncture became all the rage in the West. Acupuncture-fans then claimed that a 2000-year history is ample proof that acupuncture does work.
BUT ANECDOTES ARE NOTORIOUSLY UNRELIABLE!
Even the most enthusiastic advocates conceded that this is probably true. So they documented detailed case-series of lots of patients, calculated the average difference between the pre- and post-treatment severity of symptoms, submitted it to statistical tests, and published the notion that the effects of acupuncture are not just anecdotal; in fact, they are statistically significant, they said.
BUT THIS EFFECT COULD BE DUE TO THE NATURAL HISTORY OF THE CONDITION!
“True enough”, grumbled the acupuncture-fans and conducted the very first controlled clinical trials. Essentially they treated one group of patients with acupuncture while another group received conventional treatments as usual. When they analysed the results, they found that the acupuncture group had improved significantly more. “Now do you believe us?”, they asked triumphantly, “acupuncture is clearly effective”.
NO! THIS OUTCOME MIGHT BE DUE TO SELECTION BIAS. SUCH A STUDY-DESIGN CANNOT ESTABLISH CAUSE AND EFFECT.
The acupuncturists felt slightly embarrassed because they had not thought of that. They had allocated their patients to the treatment according to patients’ choice. Thus the expectation of the patients (or the clinician) to get relief from acupuncture might have been the reason for the difference in outcome. So they consulted an expert in trial-design and were advised to allocate not by choice but by chance. In other words, they repeated the previous study but randomised patients to the two groups. Amazingly, their RCT still found a significant difference favouring acupuncture over treatment as usual.
BUT THIS DIFFERENCE COULD BE CAUSED BY A PLACEBO-EFFECT!
Now the acupuncturists were in a bit of a pickle; as far as they could see, there was no good placebo for acupuncture! Eventually some methodologist-chap came up with the idea that, in order to mimic a placebo, they could simply stick needles into non-acupuncture points. When the acupuncturists tried that method, they found that there were improvements in both groups but the difference between real acupuncture and placebo was tiny and usually neither statistically significant nor clinically relevant.
NOW DO YOU CONCEDE THAT ACUPUNCTURE IS NOT AN EFFECTIVE TREATMENT?
Absolutely not! The results merely show that needling non-acupuncture points is not an adequate placebo. Obviously this intervention also sends a powerful signal to the brain which clearly makes it an effective intervention. What do you expect when you compare two effective treatments?
IF YOU REALLY THINK SO, YOU NEED TO PROVE IT AND DESIGN A PLACEBO THAT IS INERT.
At that stage, the acupuncturists came up with a placebo-needle that did not actually penetrate the skin; it worked like a mini stage dagger that telescopes into itself while giving the impression that it penetrated the skin just like the real thing. Surely this was an adequate placebo! The acupuncturists repeated their studies but, to their utter dismay, they found again that both groups improved and the difference in outcome between their new placebo and true acupuncture was minimal.
WE TOLD YOU THAT ACUPUNCTURE WAS NOT EFFECTIVE! DO YOU FINALLY AGREE?
Certainly not, they replied. We have thought long and hard about these intriguing findings and believe that they can be explained just like the last set of results: the non-penetrating needles touch the skin; this touch provides a stimulus powerful enough to have an effect on the brain; the non-penetrating placebo-needles are not inert and therefore the results merely depict a comparison of two effective treatments.
YOU MUST BE JOKING! HOW ARE YOU GOING TO PROVE THAT BIZARRE HYPOTHESIS?
We had many discussions and consensus meeting amongst the most brilliant brains in acupuncture about this issue and have arrived at the conclusion that your obsession with placebo, cause and effect etc. is ridiculous and entirely misplaced. In real life, we don’t use placebos. So, let’s instead address the ‘real life’ question: is acupuncture better than usual treatment? We have conducted pragmatic studies where one group of patients gets treatment as usual and the other group receives acupuncture in addition. These studies show that acupuncture is effective. This is all the evidence we need. Why can you not believe us?
NOW WE HAVE ARRIVED EXACTLY AT THE POINT WHERE WE HAVE BEEN A LONG TIME AGO. SUCH A STUDY-DESIGN CANNOT ESTABLISH CAUSE AND EFFECT. YOU OBVIOUSLY CANNOT DEMONSTRATE THAT ACUPUNCTURE CAUSES CLINICAL IMPROVEMENT. THEREFORE YOU OPT TO PRETEND THAT CAUSE AND EFFECT ARE IRRELEVANT. YOU USE SOME IMITATION OF SCIENCE TO ‘PROVE’ THAT YOUR PRECONCEIVED IDEAS ARE CORRECT. YOU DO NOT SEEM TO BE INTERESTED IN THE TRUTH ABOUT ACUPUNCTURE AT ALL.
Many reader of this blog will remember the libel case of the British Chiropractic Association (BCA) against Simon Singh. Simon had disclosed in a Guardian comment that the BCA was happily promoting bogus chiropractic treatments for 6 paediatric conditions, including infant colic. The BCA not only lost the case but the affair almost destroyed this strange organisation and resulted in an enormous reputational damage of chiropractors worldwide. In an article entitled AFTER THE STORM, the then-president of the BCA later described the defeat in his own words: “in 2009, events in the UK took a turn which was to consume the British Chiropractic Association (BCA) for two years and force the wider profession to confront key issues that for decades had kept it distanced from its medical counterparts and attracting ridicule from its critics…the BCA began one of the darkest periods in its history; one that was ultimately to cost it financially, reputationally and politically…The GCC itself was in an unprecedented situation. Faced with a 1500% rise in complaints, Investigating Committees were assembled to determine whether there was a case to answer…The events of the past two years have exposed a blind adherence to outdated principles amongst a small but significant minority of the profession. Mindful of the adage that it’s the squeaky wheel that gets the grease, the vocalism of this group has ensured that chiropractic is characterised by its critics as unscientific, unsafe and slightly wacky. Claims that the vertebral subluxation complex is the cause of illness and disease have persisted despite the three UK educational establishments advising the GCC that no evidence of acceptable quality exists to support such claims.”
Only a few years AFTER THE STORM, this story seems to have changed beyond recognition. Harald Walach, who is known to readers of this blog because I reported that he was elected ‘pseudo-scientist of the year’ in 2012, recently published a comment on the proceedings of the European Congress of Integrated Medicine where we find the following intriguing version of the libel case:
Mein Freund und Kollege George Lewith aus Southampton hatte einen Hauptvortrag über seine Überblicksarbeit über chiropraktische Interventionen für kleinkindliche Koliken vorgelegt. Sie ist ausgelöst worden durch die Behauptung, die Singh und Ernst vor einigen Jahren erhoben hatten, dass Chiropraktik gefährlich ist, dass es keine Daten dafür gäbe, dass sie wirksam sei und dass sie gefährliche Nebenwirkungen habe, speziell wenn sie bei Kindern angewendet würde. Die Chiropraktiker hatten den Wissenschaftsjournalisten Singh damals wegen Verleumdung verklagt und recht erhalten. George Lewith hatte dem Gericht die Expertise geliefert und nun seine Analyse auf Kinder ausgedehnt.
Kurz gefasst: Die Intervention wirkt sogar ziemlich stark, etwa eine Standardabweichung war der Effekt groß. Die Kinder schreien kürzer und weniger. Und die Durchforstung der Literatur nach gefährlichen Nebenwirkungen hatte keinen, wortwörtlich: nicht einen, Fall zu Tage gefördert, der von Nebenwirkungen, geschweige denn gefährlichen, berichtet hätte. Die Aufregung war seinerzeit dadurch entstanden, dass eine unqualifizierte Person einer zart gebauten Frau über den Rücken gelaufen ist und ihr dabei das Genick gebrochen hat. Die Presse hatte das ganze dann zu „tödlicher Nebenwirkung chiropraktischer Intervention“ aufgebauscht.
Oh, I almost forgot, you don’t read German? Here is my translation of this revealing text:
“My friend and colleague Geoorge Lewith from Southampton gave a keynote lecture on his review of chiropractic interventions for infant colic. This was prompted by the claim, made by Singh and Ernst a few years ago, that chiropractic was dangerous, that no data existed showing its effectiveness, and that it had dangerous side-effects, particularly for children. The chiropractors had sued the science journalist Singh for libel and won the case. George Lewith had provided the expert report for the court and has now extended his analysis on children.
To put it briefly: the intervention is even very effective; the effect-size is about one standard deviation. The children cry less long and more rarely. And the search of the literature for dangerous side-effects resulted in no – literally: not one – case of side-effects, not to mention dangerous ones. The fuzz had started back then because an unqualified person had walked over the back of a thin woman and had thus broken her neck. The press had subsequently hyped the whole thing to a “deadly side-effect of a chiropractic intervention”. (I am sorry for the clumsy language but the original is even worse.)
Now, isn’t that remarkable? Not only has the truth about the libel case been turned upside down, but also the evidence on chiropractic as a treatment for infant colic seems mysteriously improved; other reviews which might just be a bit more independent and objective come to the following conclusions:
The literature concerning this topic is surprisingly scarce, of poor quality and lack of convincing conclusions. With the present day data on this topic, it is impossible to say whether this kind of treatment has a significant effect.
And what should we make of all this? I don’t know about you, but I conclude that, for some apologists of alternative medicine, the truth is a rather flexible commodity.
Nobody really likes criticism, I suppose. Yet everyone with a functional brain agrees that criticism is a precondition to making progress. So most of us do listen to it, introspect and try to learn a lesson.
Not so in alternative medicine! The last post by Preston Long was a summary of constructive criticism of his own profession; it brought that message home to me much clearer than previous discussions on this blog (probably because it did not directly concern me) and, after some reflection, I realised that apologists of alternative medicine have developed five distinct strategies to avoid progress that otherwise might develop from criticism (alright, these strategies do exist in other fields too, but I think that many of the comments on this blog demonstrate that they are particularly evident in alternative medicine).
We could also call this method ‘The Prince of Wales Technique of Avoiding Progress’ because HRH is famous for making statements ‘ex cathedra’ without ever defending them or facing his critics or allowing others to directly challenge him. When he advocated the Gerson diet for cancer, for instance, Prof Baum challenged him in an open letter asking him to use his influence more wisely. Like with all other criticism directed to him, he decided to ignore it. This strategy is a safe bet for stalling progress and it has the added advantage that it does not require anything other than ignorance.
As it requires some basic understanding of the issues at hand, this method is a little more demanding. You need to look closely at the criticism and subsequently shoot holes in it. If you cannot find any, invent some. For instance, you might state that your critic misquoted the evidence. Very few people will bother to read up the original data, and you are likely to get away even with fairly obvious lies. To beef your response up a bit, pretend that there is plenty of good evidence demonstrating exactly the opposite of what your critic has said. If asked to provide actual references or sources for your claims, don’t listen. An extreme example of the bluff-method is to sue your critic for libel – but be careful, this can backfire in a major way!
A very popular method is to claim that the critic is not actually competent to criticise. The discussion of Long’s post demonstrated that technique in a classic fashion. His detractors argued that he was a failed chiropractor who had an axe to grind and thus had no right to criticise chiropractic (“Preston H Long you are a disgrace to the chiropractic profession…take off your chiropractic hat, you dont deserve to wear it. YOU sir are a shame and a folly!!”). Of course, you need to be a bit simple in order to agree with this type of logic, but lots of people seem to be just that!
Even more popular is the blame-game. It involves arguing that, ok not all is rosy on your side of the fence, but the other side is so, so much worse. Before they dare to challenge you, they should look at their own mess; and while it is not sorted, they must simply shut up. For instance, if the criticism is that chiropractors have put hundreds of their patients into wheelchairs with their neck-manipulations, you must point out that doctors with their nasty drugs are much, much worse (“Long discounts the multitudes that chiropractic has… saved from dangerous drugs and surgery. As far as risks of injury from seeing a chiropractor vs. medicine, all one needs to do is compare malpractice insurance rates to see that insurance carriers rate medicine as an exponentially more dangerous undertaking”). Few people will realise that this is a fallacy and that the risks of any therapy must be seen in relation to its potential benefits.
When criticised, you are understandably annoyed; most people will therefore forgive you calling your critic names which are not normally used in polite circles (“who is this idiot, who wouldnt know the first thing about chiropractic”). Ad hominem attacks are the last resort of apologists of alternative medicine which emerges with depressing regularity when they have run out of rational arguments; they are signs of victories of reason over unreason. In the case of those chiropractors who were unable to stomach Long’s critique, the insults were coming thick and fast. The reason for only very few being visible is quite simple: I often delete the worst excesses of such primitive reactions.
As I write these words, I am travelling back from a medical conference. The organisers had invited me to give a lecture which I concluded saying: “anyone in medicine not believing in evidence-based health care is in the wrong business”. This statement was meant to stimulate the discussion and provoke the audience who were perhaps just a little on the side of those who are not all that taken by science.
I may well have been right, because, in the coffee break, several doctors disputed my point; to paraphrase their arguments: “You don’t believe in the value of experience, you think that science is the way to know everything. But you are wrong! Philosophers and other people, who are a lot cleverer than you, tell us that science is not the way to real knowledge; and in some forms of medicine we have a wealth of experience which we cannot ignore. This is at least as important as scientific knowledge. Take TCM, for instance, thousands of years of tradition must mean something; in fact it tells us more than science will ever be able to. Qi-energy, for instance, is a concept based on experience, and science is useless at verifying it.”
I disagreed, of course. But I am afraid that I did not convince my colleagues. The appeal to tradition is amazingly powerful, so much so that even well-seasoned physicians fall for it. Yet it nevertheless is a fallacy, I am sure.
So what does experience tell us, how is it generated and why should it be unreliable?
On the level of the individual, experience emerges when a clinician makes similar observations several times in a row. This is so persuasive that few doctors are immune to the phenomenon. Let’s assume the experience is about acupuncture, more precisely about acupuncture for smoking cessation. The acupuncturist presumably has learnt during his training that his therapy works for that indication via stimulating the flow of Qi, and promptly tries it on several patients. Some of them come back for more and report that they find it easier to give up cigarettes after consulting him. This happens repeatedly, and our clinician forthwith is convinced – in fact, he knows – that acupuncture is effective for smoking cessation.
If we critically analyse this scenario, what does it tell us? It tells us very little of relevance, I am afraid. The scenario is entirely compatible with a whole host of explanations which have nothing to do with the effects of acupuncture per se:
- Those patients who did not manage to stop smoking might not have returned. Only seeing his successes without his failures, the acupuncturist would have got the wrong end of the stick.
- Human memory is selective such that the few patients who did come back and reported failure might easily get forgotten by the clinician. We all remember the good things and forget the disappointments, particularly if we are clinicians.
- The placebo-effect might have played a dirty trick on the experience of our acupuncturist.
- Some patients might have used nicotine patches that helped him to stop smoking without disclosing this fact to the acupuncturist who then, of course, attributed the benefit to his needling.
- The acupuncturist – being a very kind and empathetic clinician – might have involuntarily induced some of his patients to show kindness in return and thus tell porkies about their smoking habits which would have created a false positive impression about the effectiveness of his treatment.
- Being so empathetic, the acupuncturists would have provided lots of encouragement to stop smoking which, in some patients, might have been sufficient to kick the habit.
The long and short of all this is that our acupuncturist gradually got convinced by this interplay of factors that Qi exists and that acupuncture is an ineffective treatment. Hence forth he would bet his last shirt that he is right about this – after all, he has seen it with his own eyes, not just once but many times. And he will doubt anyone who shows him evidence that says otherwise. In fact, he is likely become very sceptical about scientific evidence in general – just like the doctors who talked to me after my lecture.
On a population level, such experience will be prevalent in not just one but most acupuncturists. Our clinician’s experience is certainly not unique; others will have made it too. In fact, as an acupuncturist, it is hard not to make it. Acupuncturists would have told everyone else about it, perhaps reported it on conferences or published it in articles or books. Experience of this nature is passed on from generation to generation, and soon someone will be able to demonstrate that acupuncture has been used ’effectively’ for smoking cessation since decades or centuries. The creation of a myth out of unreliable experience is thus complete.
Am I saying that experience of this nature is always and necessarily wrong or useless? No, I am not. It can be and often is correct. But, at the same time, it is frequently incorrect. It can serve as a valuable indicator but not more. Experience is not a tool for reliably informing us about the effectiveness of medical interventions. Experience based-medicine is an obsolete pseudo-medicine burdened with concepts that are counter-productive to optimal health care.
Philosophers and other people who are much cleverer than I am have been trying for some time to separate good from bad science and evidence from experience. Most recently, two philosophers, MASSIMO PIGLIUCCI and MAARTEN BOUDRY, commented specifically on this problem in relation to TCM. I leave you with some extensive quotes from what they wrote.
… pointing out that some traditional Chinese remedies (like drinking fresh turtle blood to alleviate cold symptoms) may in fact work, and therefore should not be dismissed as pseudoscience… risks confusing the possible effectiveness of folk remedies with the arbitrary theoretical-metaphysical baggage attached to it. There is no question that some folk remedies do work. The active ingredient of aspirin, for example, is derived from willow bark…
… claims about the existence of “Qi” energy, channeled through the human body by way of “meridians,” though, is a different matter. This sounds scientific, because it uses arcane jargon that gives the impression of articulating explanatory principles. But there is no way to test the existence of Qi and associated meridians, or to establish a viable research program based on those concepts, for the simple reason that talk of Qi and meridians only looks substantive, but it isn’t even in the ballpark of an empirically verifiable theory.
…the notion of Qi only mimics scientific notions such as enzyme actions on lipid compounds. This is a standard modus operandi of pseudoscience: it adopts the external trappings of science, but without the substance.
…The notion of Qi, again, is not really a theory in any meaningful sense of the word. It is just an evocative word to label a mysterious force of which we do not know and we are not told how to find out anything at all.
Still, one may reasonably object, what’s the harm in believing in Qi and related notions, if in fact the proposed remedies seem to help? Well, setting aside the obvious objections that the slaughtering of turtles might raise on ethical grounds, there are several issues to consider. To begin with, we can incorporate whatever serendipitous discoveries from folk medicine into modern scientific practice, as in the case of the willow bark turned aspirin. In this sense, there is no such thing as “alternative” medicine, there’s only stuff that works and stuff that doesn’t.
Second, if we are positing Qi and similar concepts, we are attempting to provide explanations for why some things work and others don’t. If these explanations are wrong, or unfounded as in the case of vacuous concepts like Qi, then we ought to correct or abandon them. Most importantly, pseudo-medical treatments often do not work, or are even positively harmful. If you take folk herbal “remedies,” for instance, while your body is fighting a serious infection, you may suffer severe, even fatal, consequences.
…Indulging in a bit of pseudoscience in some instances may be relatively innocuous, but the problem is that doing so lowers your defenses against more dangerous delusions that are based on similar confusions and fallacies. For instance, you may expose yourself and your loved ones to harm because your pseudoscientific proclivities lead you to accept notions that have been scientifically disproved, like the increasingly (and worryingly) popular idea that vaccines cause autism.
Philosophers nowadays recognize that there is no sharp line dividing sense from nonsense, and moreover that doctrines starting out in one camp may over time evolve into the other. For example, alchemy was a (somewhat) legitimate science in the times of Newton and Boyle, but it is now firmly pseudoscientific (movements in the opposite direction, from full-blown pseudoscience to genuine science, are notably rare)….
The borderlines between genuine science and pseudoscience may be fuzzy, but this should be even more of a call for careful distinctions, based on systematic facts and sound reasoning. To try a modicum of turtle blood here and a little aspirin there is not the hallmark of wisdom and even-mindedness. It is a dangerous gateway to superstition and irrationality
It was 20 years ago today that I started my job as ‘Professor of Complementary Medicine’ at the University of Exeter and became a full-time researcher of all matters related to alternative medicine. One issue that was discussed endlessly during these early days was the question whether alternative medicine can be investigated scientifically. There were many vociferous proponents of the view that it was too subtle, too individualised, too special for that and that it defied science in principle. Alternative medicine, they claimed, needed an alternative to science to be validated. I spent my time arguing the opposite, of course, and today there finally seems to be a consensus that alternative medicine can and should be submitted to scientific tests much like any other branch of health care.
Looking back at those debates, I think it is rather obvious why apologists of alternative medicine were so vehement about opposing scientific investigations: they suspected, perhaps even knew, that the results of such research would be mostly negative. Once the anti-scientists saw that they were fighting a lost battle, they changed their tune and adopted science – well sort of: they became pseudo-scientists (‘if you cannot beat them, join them’). Their aim was to prevent disaster, namely the documentation of alternative medicine’s uselessness by scientists. Meanwhile many of these ‘anti-scientists turned pseudo-scientists’ have made rather surprising careers out of their cunning role-change; professorships at respectable universities have mushroomed. Yes, pseudo-scientists have splendid prospects these days in the realm of alternative medicine.
The term ‘pseudo-scientist’ as I understand it describes a person who thinks he/she knows the truth about his/her subject well before he/she has done the actual research. A pseudo-scientist is keen to understand the rules of science in order to corrupt science; he/she aims at using the tools of science not to test his/her assumptions and hypotheses, but to prove that his/her preconceived ideas were correct.
So, how does one become a top pseudo-scientist? During the last 20 years, I have observed some of the careers with interest and think I know how it is done. Here are nine lessons which, if followed rigorously, will lead to success (… oh yes, in case I again have someone thick enough to complain about me misleading my readers: THIS POST IS SLIGHTLY TONGUE IN CHEEK).
- Throw yourself into qualitative research. For instance, focus groups are a safe bet. This type of pseudo-research is not really difficult to do: you assemble about 5 -10 people, let them express their opinions, record them, extract from the diversity of views what you recognise as your own opinion and call it a ‘common theme’, write the whole thing up, and – BINGO! – you have a publication. The beauty of this approach is manifold: 1) you can repeat this exercise ad nauseam until your publication list is of respectable length; there are plenty of alternative medicine journals who will hurry to publish your pseudo-research; 2) you can manipulate your findings at will, for instance, by selecting your sample (if you recruit people outside a health food shop, for instance, and direct your group wisely, you will find everything alternative medicine journals love to print); 3) you will never produce a paper that displeases the likes of Prince Charles (this is more important than you may think: even pseudo-science needs a sponsor [or would that be a pseudo-sponsor?]).
- Conduct surveys. These are very popular and highly respected/publishable projects in alternative medicine – and they are almost as quick and easy as focus groups. Do not get deterred by the fact that thousands of very similar investigations are already available. If, for instance, there already is one describing the alternative medicine usage by leg-amputated police-men in North Devon, and you nevertheless feel the urge of going into this area, you can safely follow your instinct: do a survey of leg-amputated police men in North Devon with a medical history of diabetes. There are no limits, and as long as you conclude that your participants used a lot of alternative medicine, were very satisfied with it, did not experience any adverse effects, thought it was value for money, and would recommend it to their neighbour, you have secured another publication in an alternative medicine journal.
- If, for some reason, this should not appeal to you, how about taking a sociological, anthropological or psychological approach? How about studying, for example, the differences in worldviews, the different belief systems, the different ways of knowing, the different concepts about illness, the different expectations, the unique spiritual dimensions, the amazing views on holism – all in different cultures, settings or countries? Invariably, you will, of course, conclude that one truth is at least as good as the next. This will make you popular with all the post-modernists who use alternative medicine as a playground for getting a few publications out. This approach will allow you to travel extensively and generally have a good time. Your papers might not win you a Nobel prize, but one cannot have everything.
- It could well be that, at one stage, your boss has a serious talk with you demanding that you start doing what (in his narrow mind) constitutes ‘real science’. He might be keen to get some brownie-points at the next RAE and could thus want you to actually test alternative treatments in terms of their safety and efficacy. Do not despair! Even then, there are plenty of possibilities to remain true to your pseudo-scientific principles. By now you are good at running surveys, and you could, for instance, take up your boss’ suggestion of studying the safety of your favourite alternative medicine with a survey of its users. You simply evaluate their experiences and opinions regarding adverse effects. But be careful, you are on somewhat thinner ice here; you don’t want to upset anyone by generating alarming findings. Make sure your sample is small enough for a false negative result, and that all participants are well-pleased with their alternative medicine. This might be merely a question of selecting your patients cleverly. The main thing is that your conclusion is positive. If you want to go the extra pseudo-scientific mile, mention in the discussion of your paper that your participants all felt that conventional drugs were very harmful.
- If your boss insists you tackle the daunting issue of therapeutic efficacy, there is no reason to give up pseudo-science either. You can always find patients who happened to have recovered spectacularly well from a life-threatening disease after receiving your favourite form of alternative medicine. Once you have identified such a person, you write up her experience in much detail and call it a ‘case report’. It requires a little skill to brush over the fact that the patient also had lots of conventional treatments, or that her diagnosis was assumed but never properly verified. As a pseudo-scientist, you will have to learn how to discretely make such irritating details vanish so that, in the final paper, they are no longer recognisable. Once you are familiar with this methodology, you can try to find a couple more such cases and publish them as a ‘best case series’ – I can guarantee that you will be all other pseudo-scientists’ hero!
- Your boss might point out, after you have published half a dozen such articles, that single cases are not really very conclusive. The antidote to this argument is simple: you do a large case series along the same lines. Here you can even show off your excellent statistical skills by calculating the statistical significance of the difference between the severity of the condition before the treatment and the one after it. As long as you show marked improvements, ignore all the many other factors involved in the outcome and conclude that these changes are undeniably the result of the treatment, you will be able to publish your paper without problems.
- As your boss seems to be obsessed with the RAE and all that, he might one day insist you conduct what he narrow-mindedly calls a ‘proper’ study; in other words, you might be forced to bite the bullet and learn how to plan and run an RCT. As your particular alternative therapy is not really effective, this could lead to serious embarrassment in form of a negative result, something that must be avoided at all cost. I therefore recommend you join for a few months a research group that has a proven track record in doing RCTs of utterly useless treatments without ever failing to conclude that it is highly effective. There are several of those units both in the UK and elsewhere, and their expertise is remarkable. They will teach you how to incorporate all the right design features into your study without there being the slightest risk of generating a negative result. A particularly popular solution is to conduct what they call a ‘pragmatic’ trial, I suggest you focus on this splendid innovation that never fails to produce anything but cheerfully positive findings.
- It is hardly possible that this strategy fails – but once every blue moon, all precautions turn out to be in vain, and even the most cunningly designed study of your bogus therapy might deliver a negative result. This is a challenge to any pseudo-scientist, but you can master it, provided you don’t lose your head. In such a rare case I recommend to run as many different statistical tests as you can find; chances are that one of them will nevertheless produce something vaguely positive. If even this method fails (and it hardly ever does), you can always home in on the fact that, in your efficacy study of your bogus treatment, not a single patient died. Who would be able to doubt that this is a positive outcome? Stress it clearly, select it as the main feature of your conclusions, and thus make the more disappointing findings disappear.
- Now that you are a fully-fledged pseudo-scientist who has produced one misleading or false positive result after the next, you may want a ‘proper’ confirmatory study of your pet-therapy. For this purpose run the same RCT over again, and again, and again. Eventually you want a meta-analysis of all RCTs ever published. As you are the only person who ever conducted studies on the bogus treatment in question, this should be quite easy: you pool the data of all your trials and, bob’s your uncle: a nice little summary of the totality of the data that shows beyond doubt that your therapy works. Now even your narrow-minded boss will be impressed.
These nine lessons can and should be modified to suit your particular situation, of course. Nothing here is written in stone. The one skill any pseudo-scientist must have is flexibility.
Every now and then, some smart arse is bound to attack you and claim that this is not rigorous science, that independent replications are required, that you are biased etc. etc. blah, blah, blah. Do not panic: either you ignore that person completely, or (in case there is a whole gang of nasty sceptics after you) you might just point out that:
- your work follows a new paradigm; the one of your critics is now obsolete,
- your detractors fail to understand the complexity of the subject and their comments merely reveal their ridiculous incompetence,
- your critics are less than impartial, in fact, most are bought by BIG PHARMA,
- you have a paper ‘in press’ that fully deals with all the criticism and explains how inappropriate it really is.
In closing, allow me a final word about publishing. There are hundreds of alternative medicine journals out there to chose from. They will love your papers because they are uncompromising promotional. These journals all have one thing in common: they are run by apologists of alternative medicine who abhor to read anything negative about alternative medicine. Consequently hardly a critical word about alternative medicine will ever appear in these journals. If you want to make double sure that your paper does not get criticised during the peer-review process (this would require a revision, and you don’t need extra work of that nature), you can suggest a friend for peer-reviewing it. In turn, you can offer to him/her that you do the same to him/her the next time he/she has an article to submit. This is how pseudo-scientists make sure that the body of pseudo-evidence for their pseudo-treatments is growing at a steady pace.