I have said it so often that I hesitate to state it again: an uncritical researcher is a contradiction in terms. This begs the question as to how critical the researchers of alternative medicine truly are. In my experience, most tend to be uncritical in the extreme. But how would one go about providing evidence for this view? In a previous blog-post, I have suggested a fairly simple method: to calculate an index of negative conclusions drawn in the articles published by a specific researcher. This is what I wrote:
If we calculated the percentage of a researcher’s papers arriving at positive conclusions and divided this by the percentage of his papers drawing negative conclusions, we might have a useful measure. A realistic example might be the case of a clinical researcher who has published a total of 100 original articles. If 50% had positive and 50% negative conclusions about the efficacy of the therapy tested, his trustworthiness index (TI) would be 1.
Depending on what area of clinical medicine this person is working in, 1 might be a figure that is just about acceptable in terms of the trustworthiness of the author. If the TI goes beyond 1, we might get concerned; if it reaches 4 or more, we should get worried.
An example would be a researcher who has published 100 papers of which 80 are positive and 20 arrive at negative conclusions. His TI would consequently amount to 4. Most of us equipped with a healthy scepticism would consider this figure highly suspect.
So how would alternative medicine researchers do, if we applied this method for assessing their trustworthiness? Very poorly, I fear – but that is speculation! Let’s see some data. Let’s look at one prominent alternative medicine researcher and see. As an example, I have chosen Professor George Lewith (because his name is unique which avoids confusion with researchers), did a quick Medline search to identify the abstracts of his articles on alternative medicine, and extracted the crucial sentence from the conclusions of the most recent ones:
- The study design of registered TCM trials has improved in estimating sample size, use of blinding and placebos
- Real treatment was significantly different from sham demonstrating a moderate specific effect of PKP
- These findings highlight the importance of helping patients develop coherent illness representations about their LBP before trying to engage them in treatment-decisions, uptake, or adherence
- Existing theories of how context influences health outcomes could be expanded to better reflect the psychological components identified here, such as hope, desire, optimism and open-mindedness
- …mainstream science has moved on from the intellectual sterility and ad hominem attacks that characterise the sceptics’ movement
- Trustworthy and appropriate information about practitioners (e.g. from professional regulatory bodies) could empower patients to make confident choices when seeking individual complementary practitioners to consult
- Comparative effectiveness research is an emerging field and its development and impact must be reflected in future research strategies within complementary and integrative medicine
- The I-CAM-Q has low face validity and low acceptability, and is likely to produce biased estimates of CAM use if applied in England, Romania, Italy, The Netherlands or Spain
- Our main finding was of beta power decreases in primary somatosensory cortex and SFG, which opens up a line of future investigation regarding whether this contributes toward an underlying mechanism of acupuncture.
- …physiotherapy was appraised more negatively in the National Health Service than the private sector but osteopathy was appraised similarly within both health-care sectors
This is a bit tedious, I agree, so I stop after just 10 articles. But even this short list does clearly indicate the absence of negative conclusions. In fact, I see none at all – arguably a few neutral ones, but nothing negative. All is positive in the realm of alternative medicine research then? In case you don’t agree with that assumption, you might prefer to postulate that this particular alternative medicine researcher somehow avoids negative conclusions. And if you believe that, you are not far from considering that we are being misinformed.
Alternative medicine is not really a field where one might reasonably expect that rigorous research generates nothing but positive results; even to expect 50 or 40% of such findings would be quite optimistic. It follows, I think, that if researchers only find positives, something must be amiss. I have recently demonstrated that the most active research homeopathic group (Professor Witt from the Charite in Berlin) has published nothing but positive findings; even if the results were not quite positive, they managed to formulate a positive conclusion. Does anyone doubt that this amounts to misinformation?
So, I do have produced at least some tentative evidence for my suspicion that some alternative medicine researchers misinform us. But how precisely do they do it? I can think of several methods for avoiding publishing a negative result or conclusion, and I fear that all of them are popular with alternative medicine researchers:
- design the study in such a way that it cannot possibly give a negative result
- manipulate the data
- be inventive when it comes to statistics
- home in on to the one positive aspect your generally negative data might show
- do not write up your study; like this nobody will ever see your negative results
And why do they do it? My impression is that they use science not for testing their interventions but for proving them. Critical thinking is a skill that alternative medicine researchers do not seem to cultivate. Often they manage to hide this fact quite cleverly and for good reasons: no respectable funding body would give money for such an abuse of science! Nevertheless, the end-result is plain to see: no negative conclusions are being published!
There are at least two further implications of the fact that alternative medicine researchers misinform the public. The first concerns the academic centres in which these researchers are organised. If a prestigious university accommodates a research unit of alternative medicine, it gives considerable credence to alternative medicine itself. If the research that comes out of the unit is promotional pseudo-science, the result, in my view, amounts to misleading the public about the value of alternative medicine.
The second implication relates to the journals in which researchers of alternative medicine prefer to publish their articles. Today, there are several hundred journals specialised in alternative medicine. We have shown over and over again that these journals publish next to nothing in terms of negative results. In my view, this too amounts to systematic misinformation.
My conclusion from all this is depressing: the type of research that currently dominates alternative medicine is, in fact, pseudo-research aimed not at rigorously falsifying hypotheses but at promoting bogus treatments. In other words alternative medicine researchers crucially contribute to the ‘sea of misinformation’ in this area.
If my health insurance pays for this treatment, it must be scientifically tested and proven. The ‘appeal to authority’ is powerful indeed, and I imagine that many consumers fall for this argument. But it is a fallacy! Health insurances are misinforming us for commercial benefit.
In 2007, I published an analysis of German health insurance companies’ policies regarding bogus treatments (MMW 2006, 149: 55-56 [the paper is in German and unfortunately not Medline-listed]). For this purpose, I had selected three popular alternative modalities: Bach flower remedies, Schuessler salts, and kinesiology all of which are, of course, not supported by sound evidence nor by biological plausibility. What emerged from this evaluation was shocking: of the 13 companies analysed, 9 paid for Bach flower remedies, 7 for kinesiology and 9 for Schuessler salts.
If you now think ‘ah yes, those Germans are obsessed with alternative medicine’, think again. The situation in most other countries is not much better; health insurances go for alternative medicine as though there is no tomorrow. A review from the US concluded that the number of people using CAM insurance benefits was substantial; the effect on insurance expenditures was modest. Because the long-term trajectory of CAM cost under third-party payment is unknown, utilization of these services should be followed. And apparently this is by no means confined to human health; recently someone tweeted that he had a very hard time finding a pet-insurance which did not offer to cover woo.
A few years after the above-mentioned publication, I was invited to speak at an international meeting of health insurers. I told the delegates in no uncertain terms that most of what they were offering to their clients in terms of alternative medicine was either unproven or disproven. There was stunned silence during the official discussion period, and I asked myself whether I had impolitely embarrassed my hosts. Then came the tea break, and one high-level representative of an insurance company after the other came to me to chat. Essentially, they all said: “We are well aware of the facts and the evidence you reviewed in your lecture; most of these treatments are useless, of course. But we have to offer them to our customers because we need to be competitive.”
In other words, health insurers, who normally are keen to keep their costs down, do not mind to pay for treatments which they know are ineffective simply because they use it as some sort of an advertising gimmick. In doing so they say or imply that these treatments do work. I think this is not just wrong and short-sighted, it is unethical and it significantly contributes to the ‘sea of misinformation’.
It almost goes without saying that alternative practitioners contribute importantly to the ‘sea of misinformation’ about alternative medicine. Again, I could write books about this subject but have to refrain myself and therefore will merely put quick spotlights on several types of practitioners, mostly drawing from my own research on these subjects.
A survey of more than 9000 patients of U.K. non-medically trained acupuncturists showed that a considerable number had received advice from their therapists about prescribed medicines. Since these acupuncturists hold no medical qualifications, they are not qualified to issue such advice. It is therefore clear to me that the advice given is likely to be misleading. In 2000, we directly asked the U.K. acupuncturists’ advice about electro-acupuncture treatment for smoking cessation, a treatment which we previously had identified to be ineffective. The advice we received was frequently not based on current best evidence and some of it also raised serious safety concerns (Schmidt, K., & Ernst, E. Internet advice by acupuncturists—a risk factor for cardiovascular patients? Perfusion,2002, 15: 44-50. Article not Medline-listed).
Many chiropractors from the UK and other countries make unsustainable therapeutic claims on their websites. In 2002, at the height of the ‘‘MMR scare’’ in Britain, we conducted a study revealing that a sizable proportion of U.K. chiropractors advised mothers against having the measles-mumps-rubella (MMR) jab for their children. A survey of the U.K. chiropractors demonstrated that an alarming percentage of the U.K. chiropractors fail to provide advice about the risks of spinal manipulation before commencing treatment. As these risks are, in fact, considerable, this behaviour amounts to misinformation and is an obvious violation of medical ethics.
With osteopaths, it is a very similar story; the main difference is that there are far less investigations than for chiropractors. This may be due to the fact that, in the US, osteopaths are not alternative but conventional clinicians with much the same training and skills as proper doctors. But in Europe, they are strictly alternative and make as many bogus claims as chiropractors. Systematic investigations are rare, but I only need to remind us of my recent blog-post where I pointed out that:
Most osteopaths treat children for a wide range of conditions and claim that their interventions are helpful. They believe that children are prone to structural problems which can be corrected by their interventions. Here is an example from just one of the numerous promotional websites on this topic:
STRUCTURAL PROBLEMS, such as those affecting the proper mobility and function of the body’s framework, can lead to a range of problems. These may include:
- Postural – such as scoliosis
- Respiratory – such as asthma
- Manifestations of brain injury – such as cerebral palsy and spasticity
- Developmental – with delayed physical or intellectual progress, perhaps triggering learning behaviour difficulties
- Infections – such as ear and throat infections or urinary disturbances, which may be recurrent.
OSTEOPATHY can assist in the prevention of health problems, helping children to make a smooth transition into normal, healthy adult life.
Encouraging evidence exists for some specific herbs in the treatment of some specific conditions. Yet, virtually no good evidence exists to suggest that the prescriptions of individualized herbal mixtures by traditional herbalists across the globe generate more good than harm. Despite this lack of evidence, herbalists do not seem to offer this information voluntarily to his or her patients. When we directly asked the UK herbalists for advice on a clinical case, we found that it was ‘‘misleading at best and dangerous at worst’’ . In other words, herbalists misinform their patients and the public about the value of their treatments.
Many non-medically trained homeopaths advise their clients against the immunization of children. Instead, these practitioners often recommend using ‘‘homeopathic vaccinations’’ for which no good evidence exists. For instance, the vice-chair of the board of directors of ‘‘The Society of Homeopaths’’ had a site with the following statements: ‘‘Homeopathic alternatives to children’s immunisation are now available.’’ ‘‘Our clinic offers alternative immunisation programmes for the whole family.’’ Such statements amounts to misinformation which puts children’s health at risk.
Other alternative practitioners
I have chosen the above-listed professions almost at random and could have selected any other type as well. Arguably, all alternative practitioners who employ unproven treatments – and that must be the vast majority – misinform their patients to some extend. The only way to avoid this is to say: ‘look, I am going to give you a therapy for which there is no good evidence – I hope you don’t mind’. If they did that, they would be out of business in a flash. It follows, I think, that being in business is tantamount to misleading patients.
And there is, of course, another way of misinforming patients which is often forgotten yet very important: withholding essential information. In all of health care, informed consent is a ‘sine qua non’. Alternative practitioners very rarely obtain informed consent from their patients. The reason seems obvious (see above). I would argue that not informing people when they should be informed is a form of misinformation.
In this context, it is worth mentioning an investigation we did in 2009: We obtained the ethical codes of the following bodies: Association of Naturopathic Practitioners, Association of Traditional Chinese Medicine (UK), Ayurvedic Practitioners Association, British Acupuncture Council, Complementary and Natural Healthcare Council, European Herbal Practitioners Association, General Chiropractic Council, General Osteopathic Council, General Regulatory Council for Complementary Therapies, National Institute of Medical Herbalists, Register of Chinese Herbal Medicine, Society of Homeopaths, UK Healers, Unified Register of Herbal Practitioners. We then extracted the statements from these codes referring to evidence-based practice (EBP). The results showed that only the General Chiropractic Council, the General Osteopathic Council and the General Regulatory Council for Complementary Therapies oblige their members to adopt EBP.
It seems that misinformation is an alternative practitioner’s daily bread. Without it, alternative therapists would need to confine their practice to the few treatments/conditions for which the evidence is positive. If they ever followed this strategy, they would hardly be able to earn a living.
To include conventional health care professionals amongst those who significantly contribute to the ‘sea of misinformation’ on alternative medicine might come as a surprise. But sadly, they do deserve quite a prominent place in the list of contributors. In fact, I could write one entire book about each of the various professions’ ways to mislead patients about alternative medicine.
There are, of course, considerable national differences and other peculiarities which render each specific profession quite complex to evaluate. The material is huge – far to big to fit in a short comment. All I will therefore try to do with this post is to throw a quick spotlight on some of the mainstream professions mentioning just one or two relevant aspects in each instant.
Particularly in North America, many nurses seem to be besotted with ‘Therapeutic Touch’, an implausible and unproven ‘energy-therapy’. For instance, the College of Nurses of Ontario includes Therapeutic Touch as a therapy permitted for its members. In other regions, other alternative treatments might be more popular with nurses but, in general, many seem to have a weakness for this sector. Researchers from Aberdeen recently conducted a survey to establish the use of alternative medicine by registered nurses, as well as their knowledge-base and attitudes towards it. They sent a questionnaire to 621 nurses and achieved a remarkable response rate of 86%. Eighty per cent of the responders admitted to employ alternative medicine and 41% were using it currently. Only five nurses believed that alternative medicine was not effective and 74% would recommend it to others. In other words, there is a strong likelihood of patients being misinformed by nurses.
A recent article in the UK journal THE PRACTISING MIDWIFE (Sept 2013) by Valerie Smith (not Medline-listed) claimed that the Royal College of Midwives supports the use of homeopathic remedies during childbirth. This does come to no surprise to those who know that several surveys have suggested that midwives are particularly fond of un- or dis-proven therapies and that they employ them often without the knowledge of obstetricians. We investigated this question by conducting a systematic review of all surveys of alternative medicine use by midwives. In total,19 surveys met our inclusion criteria. Most were recent and many originated from the US. Prevalence data varied but were usually high, often close to 100%. Much of this practice was not supported by sound evidence for efficacy and some of the treatments employed had the potential to put patients at risk. It seems obvious that, in order to employ unproven treatment, midwives first need to misinform their patients.
Some physiotherapists promote and practise a range of unproven treatments, e.g. craniosacral therapy. I am not aware of statistics on this, but it is not difficult to find evidence on the Internet: One website boldly states that Physiotherapy & Craniosacral Therapy available with Charetred Physiotherapist with 20 years of experience in the NHS. Another one proudly announces: Our main methods of treatment are through Physiotherapy and Craniosacral Therapy. A third site claims that Craniosacral Therapy is attracting increasing interest for its gentle yet effective approach, working directly with the body’s natural capacity for self-repair to treat a wide range of conditions. And a final example: Catherine is a registered Cranio-Sacral Therapist, a Physiotherapist, and is a tutor at the London College of Cranio-Sacral Therapy. She is also qualified in acupuncture for pain relief and a member of the Craniosacral Therapy Association, the Chartered Society of Physiotherapy and Acupuncture Association for Chartered Physiotherapists.
If you go into any pharmacy in the UK, you do not need to search for long to find shelves full of homeopathic remedies, Bach flower remedies, aromatherapy-oils or useless herbal slimming aids, to mention just 4 of the many different bogus treatments on offer. If you do the same in Germany, France, Switzerland or other countries, the amount of bogus remedies and devices for sale might even be greater. Pharmacists, it seems to me, have long settled to be shopkeepers who have few scruples misleading their customers into believing that these useless products are worth buying. Their code of ethics invariably forbids them such promotion and trade, but most pharmacists seem to pay no or very little attention. The concern for profit has clearly won over the concern for customers or patients.
I have left my own profession for last – not because they are the least contributors to the ‘sea of misinformation, but because, in some respects, they are the most important ones. The general attitude amongst doctors today seems to be ‘I don’t care how it works, as long as it helps my patients’. I have dedicated a previous post on explaining that this is misleading nonsense; therefore there is no reason to not repeat myself. Instead, I might just mention how many doctors practice homeopathy thus misleading patients into believing that it is an effective therapy. Alternatively, I could refer to those charlatans with a medical degree who promote bogus cancer cures. In my view, misinformation by doctors is the most serious form of misinformation of them all: physicians involved in such activities violate their ethical code and betray patients who frequently trust doctors almost blindly.
It would be a misunderstanding to assume that, with this post, I am accusing all conventional health care professionals of misinforming us about alternative medicine. But some clearly do; and when they do abuse their positions of trust in this way, they do a serious disservice to us all. I hope that exposing this problem will contribute to conventional health care professionals behaving more responsibly in future.
Imagine: you consult your doctor and he says: “I am so sorry, but I have bad news: the tests have shown that you have cancer”. You go home and feel as though someone has hit you with a sledge hammer. You cry a lot and your thoughts go round in circles. A complete nightmare unfolds; you sometimes think you are dreaming but reality soon catches up with you.
A few days later, you have an appointment with the oncologist who explains the treatment plan. You feel there is no choice and you agree to it. After the first chemotherapy, you lose your hair, your well-being, your dignity, your control and your patience – time to investigate what else there is on offer. There must be an alternative!
By then lots of well-wishers will have mentioned to you that the conventional route is but one of many: there are, in fact, alternatives!!! You go on the internet and find not just a few, you find millions of website promoting hundreds of solutions – anything from diets to herbal remedies, from homeopathy to faith-healing. All are being promoted as cures for your cancer, and all are free of those nasty side-effects which make your life hell at the moment. You think “there is a choice after all”.
Who would not be tempted by these options advertised in the most glorious terms? Who would not begin to distrust the oncologists who kindly but firmly insist that ‘alternative cancer cures’ are bogus? Who would not want to get rid of the cancer and the side-effects in one genial master-stroke?
Cancer patients yearn for hope and are extremely vulnerable to such influences. I do not know a single one who, faced with the diagnosis and all it entails, has not looked at the ‘alternatives’. This is why it would be so very important that the websites informing patients and their carers convey accurate and responsible information. But do they?
One of our research projects at Exeter had been aimed at assessing the quality of the websites advising patients on alternative treatments for cancer. For this purpose, we evaluated a total of 32 sites which cancer patients were most likely to consult according to pre-defined criteria – in other words, we assessed the most frequented websites for cancer.
Our results were shocking: many of these sites were of poor quality and most of them recommended a plethora of unproven treatments for cancer, most frequently herbal remedies, diets and mind-body therapies. In our estimation, at least three of them were outright dangerous and had the potential to harm patients.
The level of misinformation in this area is sickening. Patients are being sold false hope by the truck-load. Yet they deserve better; they deserve impartial information on their illness and the best treatment for it – cancer patients especially so. What they get instead is a total disgrace: commercially driven lies about ‘treatments’ which are not just unproven but which would, if used as instructed, hasten their death. Some alternative therapies have potential for palliative and supporting care, BUT NONE OFFER A CURE OR A REDUCTION OF THE TUMOR BURDEN OR A CHANGE IN THE NATURAL HISTORY OF THE DISEASE.
The UK ‘Society of Homeopaths’ (SoH) is the largest professional organisation of UK non-doctor, so-called lay- homeopaths. On their website, the SoH made very specific claims about homeopathy; in particular, they listed conditions for which homeopathy had allegedly been proven to be effective. These claims have now thoroughly been debunked, and the evidence the SoH produced in support of their claims has been shown to be misleading, cherry-picked or misinterpreted.
I have no idea who conducted the above-named investigation and made a youtube video of it, but I think it is essentially correct and well worth watching. My own experiences with the SoH relate mainly to two encounters.
The first was a complaint I made about one of their high-ranking officers, Ralf Jeutter. He had been promotiong homeopathic vaccinations on his website (needless to stress, I think, that there is no evidence to support the notion that homeopathic vaccinations are effective). As I felt that the SoH dragged their feet pursuing my complaint, I had to send several reminders. Eventually, they considered it and concluded that Reuter had done nothing wrong. This, presumably, is the reason why, even today, he can state on his website that Homeopathy is used to help individuals in dealing better with kinds of infections such as leptospirosis, meningitis and cholera. All is fine, it seems as long as a disclaimer is added: Any information obtained here is not to be construed as medical OR legal advice. The decision to vaccinate and how you implement that decision is yours and yours alone. The evidence for the efficacy of homeopathic immunisation is ‘anecdotal’. That means it is based on individuals’ reports past and present.
My second encounter with the SoH relates to my 2010 analysis of the SoH code of ethics and their adherence to it. The code demanded that:
- ‘all speculative theories will be stated as such and clearly distinguished’
- ‘no advertising may be used which expressly or implicitly claims to cure named diseases’
- ‘Advertising shall not be false, fraudulent, misleading, deceptive, extravagant or sensational.’
Encouraged by these assurances, I decided to study the websites of some members of the SoH, and soon discovered numerous and very obvious violations of the above-mentioned imperatives. In an attempt to find the root of these transgressions, I scrutinised the SoH’s own website where I found a multitude violations on all levels of the SoH’s own code of ethics. Many of the violations related to claims which were not supported by evidence. In other words, the largest professional UK organisation of lay- homeopaths misled the public in several rather devious ways:
they pretended to adhere to a code of ethics which forbids members to mislead the public
SoH -members nevertheless did mislead the public in ways that public health at risk
and they did so not least because the SoH followed exactly the same strategy
thus the SoH violated its own code of ethics to the detriment of public health.
My analysis was conducted a while ago, and some might hope that the SoH has stopped systematically misleading the public. This hope, however, is harshly disappointed when you watch the brand-new video entitled TESTING HOMEOPATHY mentioned above. As the SoH is about to celebrate 35 years of wisdom, courage, knowledge and prosperity, I do wonder whether this should not be 35 years of dangerously misleading the public.
What do you think?
Quite a few people seem to be amazed about the amount of work and the enthusiasm I invest into this blog. To be honest, I am amazed too. Why do I do it? And why do I do it in the way I do it?
I am sure that my critics have a choice of answers at the ready; in one way or another, the following theories have been put forward:
- I was bought by the pharmaceutical industry.
- I have had a bad experience with one or several alternative practitioners.
- I want to get rich and/or famous.
- I feel that conventional medicine is flawless
- I don’t know what I am writing about.
None of these notions is remotely true, of course.
Big Pharma has never approached me, and so far nobody at all has ever tried to influence the contents of my posts. I never had a bad personal experience with alternative medicine; on the contrary, I had several quite positive ones. The blog does not earn me money; it even costs a little to run it, and nobody other than I pay for it. I do not think that fame is something one can achieve through doing a blog, and I certainly do not aim at doing so. Sadly, I know only too well about the many flaws in conventional medicine; it is often far from perfect but at least it makes progress; alternative medicine, by contrast, seems almost entirely static. As I have studied most aspects of alternative medicine for two decades and published more than any other person on this subject, I think the allegation of incompetence might be a bit far-fetched.
So, why then?
When I decided to become a doctor I, like most medical students, did so mainly to help suffering individuals. When I became a researcher, I felt more removed from this original ideal. Yet I told myself that, by conducting research, I might eventually contribute to a better health care of tomorrow. Helping suffering patients was still firmly on the agenda. But then I realised that my articles in peer-reviewed medical journals somehow missed an important target: in alternative medicine, one ought to speak not just to health care professionals but also to consumers and patients; after all, it is they who often make the therapeutic decisions in this area.
Once I had realised this, I started addressing the general public by writing for The Guardian and other newspapers, giving public lectures and publishing books for a lay audience, like TRICK OR TREATMENT…The more I did this sort of thing, the more I noticed how important this activity was. And when a friend offered to help me set up a blog, I did not hesitate for long.
So, the reason for my enthusiasm for this blog turns out to be the same as the one that enticed me to go into medicine in the first place. I do believe that it is helpful for consumers to know the truth about alternative medicine. Considering the thousands of sources of daily misinformation in this area, there is an urgent need for well-informed, critical information. By providing it, I am sure I can assist people to make better therapeutic decisions. In a way, I am back where I started all those years ago: hoping to help suffering patients in the most direct way my expertise allows.
Hot flushes are a big problem; they are not life-threatening, of course, but they do make life a misery for countless menopausal women. Hormone therapy is effective, but many women have gone off the idea since we know that hormone therapy might increase their risk of getting cancer and cardiovascular disease. So, what does work and is also risk-free? Acupuncture?
Together with researchers from Quebec, we wanted to determine whether acupuncture is effective for reducing hot flushes and for improving the quality of life of menopausal women. We decided to do this in form of a Cochrane review which was just published.
We searched 16 electronic databases in order to identify all relevant studies and included all RCTs comparing any type of acupuncture to no treatment/control or other treatments. Sixteen studies, with a total of 1155 women, were eligible for inclusion. Three review authors independently assessed trial eligibility and quality, and extracted data. We pooled data where appropriate.
Eight studies compared acupuncture versus sham acupuncture. No significant difference was found between the groups for hot flush frequency, but flushes were significantly less severe in the acupuncture group, with a small effect size. There was substantial heterogeneity for both these outcomes. In a post hoc sensitivity analysis excluding studies of women with breast cancer, heterogeneity was reduced to 0% for hot flush frequency and 34% for hot flush severity and there was no significant difference between the groups for either outcome. Three studies compared acupuncture with hormone therapy, and acupuncture turned out to be associated with significantly more frequent hot flushes. There was no significant difference between the groups for hot flush severity. One study compared electro-acupuncture with relaxation, and there was no significant difference between the groups for either hot flush frequency or hot flush severity. Four studies compared acupuncture with waiting list or no intervention. Traditional acupuncture was significantly more effective in reducing hot flush frequency, and was also significantly more effective in reducing hot flush severity. The effect size was moderate in both cases.
For quality of life measures, acupuncture was significantly less effective than HT, but traditional acupuncture was significantly more effective than no intervention. There was no significant difference between acupuncture and other comparators for quality of life. Data on adverse effects were lacking.
Our conclusion: We found insufficient evidence to determine whether acupuncture is effective for controlling menopausal vasomotor symptoms. When we compared acupuncture with sham acupuncture, there was no evidence of a significant difference in their effect on menopausal vasomotor symptoms. When we compared acupuncture with no treatment there appeared to be a benefit from acupuncture, but acupuncture appeared to be less effective than HT. These findings should be treated with great caution as the evidence was low or very low quality and the studies comparing acupuncture versus no treatment or HT were not controlled with sham acupuncture or placebo HT. Data on adverse effects were lacking.
I still have to meet an acupuncturist who is not convinced that acupuncture is not an effective treatment for hot flushes. You only need to go on the Internet to see the claims that are being made along those lines. Yet this review shows quite clearly that it is not better than placebo. It also demonstrates that studies which do suggest an effect do so because they fail to adequately control for a placebo response. This means that the benefit patients and therapists observe in routine clinical practice is not due to the acupuncture per se, but to the placebo-effect.
And what could be wrong with that? Quite a bit, is my answer; here are just 4 things that immediately spring into my mind:
1) Arguably, it is dishonest and unethical to use a placebo on ill patients in routine clinical practice and charge for it pretending it is a specific and effective treatment.
2) Placebo-effects are unreliable, small and usually of short duration.
3) In order to generate a placebo-effect, I don’t need a placebo-therapy; an effective one administered with compassion does that too (and generates specific effects on top of that).
4) Not all placebos are risk-free. Acupuncture, for instance, has been associated with serious complications.
The last point is interesting also in the context of our finding that the RCTs analysed failed to mention adverse-effects. This is a phenomenon we observe regularly in studies of alternative medicine: trialists tend to violate the most fundamental rules of research ethics by simply ignoring the need to report adverse-effects. In plain English, this is called ‘scientific misconduct’. Consequently, we find very little published evidence on this issue, and enthusiasts claim their treatment is risk-free, simply because no risks are being reported. Yet one wonders to what extend systematic under-reporting is the cause of that impression!
So, what about the legion of acupuncturists who earn a good part of their living by recommending to their patients acupuncture for hot flushes?
They may, of course, not know about the evidence which shows that it is not more than a placebo. Would this be ok then? No, emphatically no! All clinicians have a duty to be up to date regarding the scientific evidence in relation to the treatments they use. A therapist who does not abide by this fundamental rule of medical ethics is, in my view, a fraud. On the other hand, some acupuncturists might be well aware of the evidence and employ acupuncture nevertheless; after all, it brings good money! Well, I would say that such a therapist is a fraud too.
Indian homeopaths recently published a clinical trial aimed at evaluating homeopathic treatment in the management of diabetic polyneuropathy. The condition affects many diabetic patients; its symptoms include tingling, numbness, burning sensation in the feet and pain, particularly at night. The best treatment consists of adequate metabolic control of the underlying diabetes. The pain can be severe often does not respond adequately to conventional pain-killers. It is therefore obvious that any new, effective treatment would be more than welcome.
The new trial is a prospective observational study which was carried out from October 2005 to September 2009 by the Indian Central Council for Research in Homeopathy at its five Institutes. Patients suffering diabetic polyneuropathy (DPN) were screened and enrolled in the study, if they fulfilled the inclusion and exclusion criteria. The Diabetic Distal Symmetric Polyneuropathy Symptom Score (DDSPSS), developed by the Council, served as the primary outcome measure.
A total of 15 homeopathic medicines were identified after repertorizing the nosological symptoms and signs of the disease. The appropriate constitutional medicine was selected and prescribed in the 30, 200 and 1 M potencies on an individualized basis. Patients were followed up for 12 months.
Of 336 diabetics enrolled in the study, 247 patients who attended at least three follow-up appointments and baseline nerve conduction studies were included in the analysis. A statistically significant improvement in DDSPSS total score was found at 12 months. Most objective measures did not show significant improvements. Lycopodium clavatum (n = 132), Phosphorus (n = 27) and Sulphur (n = 26) were the most frequently prescribed homeopathic remedies.
From these results, the authors concluded that: “homeopathic medicines may be effective in managing the symptoms of DPN patients.”
Does this study tell us anything worth knowing? The short answer to this question, I am afraid, is NO.
Its weaknesses are all too obvious:
1) There was no control group.
2) Patients who did not come back to the follow-up appointments – presumably because they were not satisfied – were excluded from the analyses. The average benefit reported is thus likely to be a cherry-picked false positive result.
3) The primary outcome measure was not validated.
4) The observed positive effect on subjective symptoms could be due to several factors which are entirely unrelated to the homeopathic treatments’ e.g. better metabolic control, regression towards the mean, or social desirability.
Anyone who had seen the protocol of this study would have predicted the result; I see no way that such a study does not generate an apparently positive outcome. In other words, conducting the investigation was superfluous, which means that the patients’ participation was in vain; and this, in turn, means that the trial was arguably unethical.
This might sound a bit harsh, but I am entirely serious: deeply flawed research should not happen. It is a waste of scarce resources and patients’ tolerance; crucially, it has a powerful potential to mislead us and to set back our efforts to improve health care. All of this is unethical.
The problem of research which is so poor that it crosses the line into being unethical is, of course, not confined to homeopathy. In my view, it is an important issue in much of alternative medicine and quite possibly in conventional medicine as well. Over the years, several mechanisms have been put in place to prevent or at least minimize the problem, for instance, ethic committees and peer-review. The present study shows, I think, that these mechanisms are fragile and that, sometimes, they fail altogether.
In their article, the authors of the new homeopathic study suggest that more investigations of homeopathy for diabetic polyneuropathy should be done. However, I suggest almost precisely the opposite: unethical research of this nature should be prevented, and the existing mechanisms to achieve this aim must be strengthened.
The ‘Samueli Institute’ might be known to many readers of this blog; it is a wealthy institution that is almost entirely dedicated to promoting the more implausible fringe of alternative medicine. The official aim is “to create a flourishing society through the scientific exploration of wellness and whole-person healing“. Much of its activity seems to be focused on military medical research. Its co-workers include Harald Walach who recently was awarded a rare distinction for his relentless efforts in introducing esoteric pseudo-science into academia.
Now researchers from the Californian branch of the Samueli Institute have published an articles whic, in my view, is another landmark in nonsense.
Jain and colleagues conducted a randomized controlled trial to determine whether Healing Touch with Guided Imagery [HT+GI] reduced post-traumatic stress disorder (PTSD) compared to treatment as usual (TAU) in “returning combat-exposed active duty military with significant PTSD symptoms“. HT is a popular form of para-normal healing where the therapist channels “energy” into the patient’s body; GI is a self-hypnotic from of relaxation-therapy. While the latter approach might be seen as plausible and, at least to some degree, evidence-based, the former cannot.
123 soldiers were randomized to 6 sessions of HT+GI, while the control group had no such therapies. All patients also received standard conventional therapies, and the treatment period was three weeks. The results showed significant reductions in PTSD symptoms as well as depression for HT+GI compared to controls. HT+GI also showed significant improvements in mental quality of life and cynicism.
The authors concluded that HT+GI resulted in a clinically significant reduction in PTSD and related symptoms, and that further investigations of biofield therapies for mitigating PTSD in military populations are warranted.
The Samueli Institute claims to “support science grounded in observation, investigation, and analysis, and [to have] the courage to ask challenging questions within a framework of systematic, high-quality, research methods and the peer-review process“. I do not think that the above-named paper lives up to these standards.
As discussed in some detail in a previous post, this type of study-design is next to useless for determining whether any intervention does any good at all: A+B is always more than B alone! Moreover, if we test HT+GI as a package, how can we conclude about the effectiveness of one of the two interventions? Thus this trial tells us next to nothing about the effectiveness of HT, nor about the effectiveness of HT+GI.
Previously, I have argued that conducting a trial for which the result is already clear before the first patient has been recruited, is not ethical. Samueli Institute, however, claims that it “acts with the highest respect for the public it serves by ensuring transparency, responsible management and ethical practices from discovery to policy and application“. Am I the only one who senses a contradiction here?
Perhaps other research in this area might be more informative? Even the most superficial Medline-search brings to light a flurry of articles on HT and other biofield therapies that are relevant.
Several trials have indeed produces promissing evidence suggesting positive effects of such treatments on anxiety and other symptoms. But the data are far from uniform, and most investigations are wide open to bias. The more rigorous studies seem to suggest that these interventions are not effective beyond placebo. Our review demonstrated that “the evidence is insufficient” to suggest that reiki, another biofield therapy, is an effective treatment for any condition.
Another study showed that tactile touch led to significantly lower levels of anxiety. Conventional massage may even be better than HT, according to some trials. The conclusion from this body of evidence is, I think, fairly obvious: touch can be helpful (most clinicians knew that anyway) but this has nothing to do with energy, biofields, healing energy or any of the other implausible assumptions these treatments are based on.
I therefore disagree with the authors’ conclusion that “further investigation into biofield therapies… is warranted“. If we really want to help patients, let’s find out more about the benefits of touch and let’s not mislead the public about some mystical energies and implausible quackery. And if we truly want to improve heath care, as the Samueli Institute claims, let’s use our limited resources for research which meaningfully contributes to our knowledge.