There probably is no area in health care that produces more surveys than alternative medicine. I estimate that about 500 surveys are published every year; this amounts to about two every working day which is substantially more than the number of clinical trials in this field.
I have long been critical of this ‘survey-mania’. The reason is simple: most of these articles are of such poor quality that they tell us nothing of value.
The vast majority of these surveys attempts to evaluate the prevalence of use of alternative medicine, and it is this type of investigation that I intend to discuss here.
For a typical prevalence survey, a team of enthusiastic researchers might put together a few questions and design a questionnaire to find out what percentage of a group of individuals have tried alternative medicine in the past. Subsequently, the investigators might get one or two hundred responses. They then calculate simple descriptive statistics and demonstrate that xy% (let’s assume it is 45%) use alternative medicine. This finding eventually gets published in one of the many alternative medicine journals, and everyone is happy – well, almost everybody.
How can I be such a spoil-sport and claim that this result tells us nothing of value? At the very minimum, some might argue, it shows that enthusiasts of alternative medicine are interested in and capable of conducting research. I beg to differ: this is not research, it is pseudo-research which ignores most of the principles of survey-design.
The typical alternative medicine prevalence survey has none of the features that would render it a scientific investigation:
1) It lacks an accepted definition of what is being surveyed. There is no generally accepted definition of alternative medicine, and even if the researchers address specific therapies, they run into huge problems. Take prayer, for instance – some see this as alternative medicine, while others would, of course, argue that it is a religious pursuit. Or take herbal medicine – many consumers confuse it with homeopathy, some might think that drinking tea is herbal medicine, while others would probably disagree.
2) The questionnaires used for such surveys are almost never validated. Essentially, this means that we cannot be sure they evaluate what we think they evaluate. We all know that the way we formulate a question can determine the answer. There are many potential sources of bias here, and they are rarely taken into consideration.
3) Enthusiastic researchers of alternative medicine usually use a small convenience sample of participants for their surveys. This means they ask a few people who happen to be around to fill their questionnaire. As a consequence, there is no way the survey is representative of the population in question.
4) The typical survey has a low response rate; sometimes the response rate is not even provided or remains unknown even to the investigators. This means we do not know how the majority of patients/consumers who received but did not fill the questionnaire would have answered. Often there is good reason to suspect that those who have a certain attitude did respond, while those with a different opinion did not. This self-selection process is likely to produce misleading findings.
And why I am so sure about all of theses limitations? To my embarrassment, I know about them not least because I have made most these mistakes myself at some time in my career. You might also ask why this is important: what’s the harm in publishing a few flimsy surveys?
In my view, these investigations are regrettably counter-productive because:
they tend to grossly over-estimate the popularity of alternative medicine,
they distract money, manpower and attention from the truly important research questions in this field,
they give a false impression of a buoyant research activity,
and their results are constantly misused.
The last point is probably the most important one. The argument that is all too often spun around such survey data goes roughly as follows: a large percentage of the population uses alternative medicine; people pay out of their own pocket for these treatments; they are satisfied with them (if not, they would not pay for them). BUT THIS IS GROSSLY UNFAIR! Why should only those individuals who are rich enough to afford alternative medicine benefit from it? ALTERNATIVE MEDICINE SHOULD BE MADE AVAILABLE FOR ALL.
I rest my case.
The developed world is in the middle of a major obesity epidemic. It is predicted to cause millions of premature deaths and billions of dollars, money that would be badly needed elsewhere. The well-known method of eating less and moving more is most efficacious but sadly not very effective, that is to say people do not easily adopt and adhere to it. This is why many experts are searching for a treatment that works and is acceptable to all or at least most patients.
Entrepreneurs of alternative medicine have long jumped on this band waggon. They have learnt that the regulations are lax or non-existent, that consumers are keen to believe anything they tell them and that the opportunities to make a fast buck are thus enormous. Today, they are offering an endless array of treatments which are cleverly marketed, for instance via the Internet.
Since many years, my research team are involved in a programme of assessing the alternative slimming aids mostly through systematic reviews and occasionally also through conducting our own clinical trials. Our published analyses include the following treatments:
Supplements containing conjugated linoleic acid
There are, of course, many more but, for most, no evidence exist at all. The treatments listed above have all been submitted to clinical trials. The results show invariably that the outcomes were not convincingly positive: either there were too few data, or there were too many flaws in the studies, or the weight reduction achieved was too small to be clinically relevant.
Our latest systematic review is a good example; its aim was to evaluate the evidence from randomized controlled trials (RCTs) involving the use of the African Bush Mango, Irvingia gabonensis, for body weight reduction in obese and overweight individuals. Three RCTs were identified, and all had major methodological flaws. All RCTs reported statistically significant reductions in body weight and waist circumference favoring I. gabonensis over placebo. They also suggested positive effects of I. gabonensis on blood lipids. Adverse events included headache and insomnia. Despite these apparently positive findings, our conclusions had to be cautious: “Due to the paucity and poor reporting quality of the RCTs, the effect of I. gabonensis on body weight and related parameters are unproven. Therefore, I. gabonensis cannot be recommended as a weight loss aid. Future research in this area should be more rigorous and better reported.”
People who want to loose weight are often extremely desperate and ready to try anything. They are thus easy victims of the irresponsible promises that are being made on the Internet and elsewhere. Despite the overwhelmingly evidence to the contrary, consumers are led to believe that alternative slimming aids are effective. What is more, they are also misled to assume they are risks-free. This latter assumption is false too: apart from the harm done to the patient’s bank account, many alternative slimming aids are associated with side-effects which, in some cases, are serious and can even include death.
The conclusion from all this is short and simple: alternative slimming aids are bogus.
Several months ago, my co-workers and I once again re-visited the contentious issue of acupuncture’s safety. We published several articles on the topic none of which, I am afraid to say, was much appreciated by the slightly myopic world of acupuncture. The paper which created overt outrage and prompted an unprecedented amount of hate-mail was the one on deaths after acupuncture. This publication reported that around 90 fatalities associated with acupuncture had been documented in the medical literature.
The responses from acupuncturists ranged from disbelief to overt hostility. Acupuncturists the world over seemed to agree that there was something profoundly wrong with me personally and with my research; they all knew that acupuncture was entirely safe and that I was maliciously incorrect and merely out to destroy their livelihood.
So, am I alarmist or am I just doing my duty in reporting important facts? Two new articles might go some way towards answering this intriguing question.
The first is a review by Chinese acupuncturists who summarised all the adverse events published in the Chinese literature, a task which my article may have done only partially. The authors found 1038 cases of serious adverse events, including 35 fatalities. The most frequent non-fatal adverse events were syncope (468 cases), pneumothorax (307 cases), and subarachnoid hemorrhage (64 cases). To put this into context, we ought to know that the Chinese literature is hopelessly biased in favour of acupuncture. Thus the level of under-reporting can be assumed to be even larger than in English language publications.
The second new article is by a Swedish surgeon who aimed at systematically reviewing the literature specifically on vascular injuries caused by acupuncture. His literature searches found 31 such cases; the majority of these patients developed symptoms in direct connection with the acupuncture treatment. Three patients died, two from pericardial tamponade and one from an aortoduodenal fistula. There were 7 more tamponades, 8 pseudoaneurysms, two with ischemia, two with venous thrombosis, one with compartment syndrome and 7 with bleeding (5 in the central nervous system). The two patients with ischemia suffered lasting sequeleae.
The answer to the question asked above seems thus simple: the Chinese authors, the Swedish surgeon (none of whom I know personally or have collaborated with) and I are entirely correct and merely report the truth. And the truth is that acupuncture can cause severe complications through any of the following mechanisms:
1) puncturing the lungs resulting in a pneumothorax,
2) puncturing the heart causing a cardiac tamponade,
3) puncturing blood vessels causing haemorrhage,
4) injuring other vital structures in the body,
5) introducing bacteria or viruses resulting in infections.
Any of these complications can be severe and might, in dramatic cases, even lead to the death of the patient.
But we have to have the right perspective! These are extremely rare events! Most other treatments used in medicine are much much more risky! To keep banging on about such exotic events is not helpful! I can hear the acupuncture world shout in unison.
True, these are almost certainly rare events – but we have no good idea how rare they are. There is no adverse event reporting scheme in acupuncture, and the published cases are surely only the tip of the ice-berg. True, most other medical treatments carry much greater risks! And true, we need to have the right perspective in all of this!
So let’s put this in a reasonable perspective: with most other treatments, we know how effective they are. We can thus estimate whether the risks outweigh the benefit, and if we find that they do, we should (and usually do) stop using them. I am not at all sure that we can perform similar assessments in the case of acupuncture.
Clinical trials of acupuncture can be quite challenging. In particular, it is often difficult to make sure that any observed outcome is truly due to the treatment and not caused by some other factor(s). How tricky this can be, shows a recently published study.
A new RCT has all (well, almost all) the features of a rigorous study. It tested the effects of acupuncture in patients suffering from hay fever. The German investigators recruited 46 specialized physicians in 6 hospital clinics and 32 private outpatient clinics. In total, 422 patients with IgE sensitization to birch and grass pollen were randomized into three groups: 1) acupuncture plus rescue medication (RM) (n= 212), 2) sham acupuncture plus RM (n= 102), or 3) RM alone (n= 108). Twelve acupuncture sessions were provided in groups 1 and 2 over 8 weeks. The outcome measures included changes in the Rhinitis Quality of Life Questionnaire (RQLQ) overall score and the RM score (RMs) from baseline to weeks 7, 8 and 16 in the first year as well as week 8 in the second year after randomization.
Compared with sham acupuncture and with RM, acupuncture was associated with improvement in RQLQ score and RMS. There were no differences after 16 weeks in the first year. After the 8-week follow-up phase in the second year, small improvements favoring real acupuncture over sham were noted.
Based on these results, the authors concluded that “acupuncture led to statistically significant improvements in disease-specific quality of life and antihistamine use measures after 8 weeks of treatment compared with sham acupuncture and with RM alone, but the improvements may not be clinically significant.”
The popular media were full of claims that this study proves the efficacy of acupuncture. However, I am not at all convinced that this conclusion is not hopelessly over-optimistic.
It might not have been the acupuncture itself that led to the observed improvements; they could well have been caused by several factors unrelated to the treatment itself. To understand my concern, we need to look closer at the actual interventions employed by the investigators.
The real acupuncture was done on acupuncture points thought to be indicated for hay fever. The needling was performed as one would normally do it, and the acupuncturists were asked to treat the patients in group 1 in such a way that they were likely to experience the famous ‘de-qi’ feeling.
The sham acupuncture, by contrast, was performed on non-acupuncture points; acupuncturists were asked to use shallow needling only and they were instructed to try not to produce ‘de-qi’.
This means that the following factors in combination or alone could have caused [and in my view probably did cause] the observed differences in outcomes between the acupuncture and the sham group:
1) verbal or non-verbal communication between the acupuncturists and the patient [previous trials have shown this factor to be of crucial importance]
2) the visibly less deep needling in the sham-group
3) the lack of ‘de-qi’ experience in the sham-group.
Sham-treatments in clinical trials serve the purpose of a placebo. They are thus meant to be indistinguishable from the verum. If that is not the case [as in the present study], the trial cannot be accepted as being patient-blind. If a trial is not patient-blind, the expectations of patients will most certainly influence the results.
Therefore I believe that the marginal differences noted in this study were not due to the effects of acupuncture per se, but were an artifact caused through de-blinding of the patients. De facto, neither the patients nor the acupuncturists were blinded in this study.
If that is true, the effects were not just not clinically relevant, as noted by the authors, they also had nothing to do with acupuncture. In other words, acupuncture is not of proven efficacy for this condition – a verdict which is also supported by our systematic review of the subject which concluded that “the evidence for the effectiveness of acupuncture for the symptomatic treatment or prevention of allergic rhinitis is mixed. The results for seasonal allergic rhinitis failed to show specific effects of acupuncture…”
Once again, we have before us a study which looks impressive at first glance. At closer scrutiny, we find, however, that it had important design flaws which led to false positive results and conclusions. In my view, it would have been the responsibility of the authors to discuss these limitations in full detail and to draw conclusions that take them into account. Moreover, it would have been the duty of the peer-reviewers and journal editors to pick up on these points. Instead the editors even commissioned an accompanying editorial which displays an exemplary lack of critical thinking.
Having failed to do any of this, they are in my opinion all guilty of misleading the world media who reported extensively and often uncritically on this new study thus misleading us all. Sadly, the losers in this bonanza of incompetence are the many hay fever sufferers who will now be trying (and paying for) useless treatments.
Still in the spirit of ACUPUNCTURE AWARENESS WEEK, I have another critical look at a recent paper. If you trust some of the conclusions of this new article, you might think that acupuncture is an evidence-based treatment for coronary heart disease. I think this would be a recipe for disaster.
This condition affects millions and eventually kills a frighteningly large percentage of the population. Essentially, it is caused by the fact that, as we get older, the blood vessels supplying the heart also change, become narrower and get partially or even totally blocked. This causes lack of oxygen in the heart which causes pain known as angina pectoris. Angina is a most important warning sign indicating that a full blown heart attack might be not far.
The treatment of coronary heart disease consists in trying to let more blood flow through the narrowed coronaries, either by drugs or by surgery. At the same time, one attempts to reduce the oxygen demand of the heart, if possible. Normalisation of risk factors like hypertension and hypercholesterolaemia are key preventative strategies. It is not immediate clear to me how acupuncture might help in all this – but I have been wrong before!
The new meta-analysis included 16 individual randomised clinical trials. All had a high or moderate risk of bias. Acupuncture combined with conventional drugs (AC+CD) turned out to be superior to conventional drugs alone in reducing the incidence of acute myocardial infarction (AMI). AC+CD was superior to conventional drugs in reducing angina symptoms as well as in improving electrocardiography (ECG). Acupuncture by itself was also superior to conventional drugs for angina symptoms and ECG improvement. AC+CD was superior to conventional drugs in shortening the time to onset of angina relief. However, the time to onset was significantly longer for acupuncture treatment than for conventional treatment alone.
From these results, the authors [who are from the Chengdu University of Traditional Chinese Medicine in Sichuan, China] conclude that “AC+CD reduced the occurrence of AMI, and both acupuncture and AC+CD relieved angina symptoms and improved ECG. However, compared with conventional treatment, acupuncture showed a longer delay before its onset of action. This indicates that acupuncture is not suitable for emergency treatment of heart attack. Owing to the poor quality of the current evidence, the findings of this systematic review need to be verified by more RCTs to enhance statistical power.”
As in the meta-analysis discussed in my previous post, the studies are mostly Chinese, flawed, and not obtainable for an independent assessment. As in the previous article, I fail to see a plausible mechanism by which acupuncture might bring about the effects. This is not just a trivial or coincidental observation – I could cite dozens of systematic reviews for which the same criticism applies.
What is different, however, from the last post on gout is simple and important: if you treat gout with a therapy that is ineffective, you have more pain and eventually might opt for an effective one. If you treat coronary heart disease with a therapy that does not work, you might not have time to change, you might be dead.
Therefore I strongly disagree with the authors of this meta-analysis; “the findings of this systematic review need NOT to be verified by more RCTs to enhance statistical power” — foremost, I think, the findings need to be interpreted with much more caution and re-written. In fact, the findings show quite clearly that there is no good evidence to use acupuncture for coronary heart disease. To pretend otherwise is, in my view, not responsible.
There might be an important lesson here: A SEEMINGLY SLIGHT CORRECTION OF CONCLUSIONS OF SUCH SYSTEMATIC REVIEWS MIGHT SAVE LIVES.
This week is acupuncture awareness week, and I will use this occasion to continue focusing on this therapy. This first time ever event is supported by the British Acupuncture Council who state that it aims to “help better inform people about the ancient practice of traditional acupuncture. With 2.3 million acupuncture treatments carried out each year, acupuncture is one of the most popular complementary therapies practised in the UK today.“
Right, let’s inform people about acupuncture then! Let’s show them that there is often more to acupuncture research than meets the eye.
My team and I have done lots of research into acupuncture and probably published more papers on this than any other subject. We had prominent acupuncturists on board from the UK, Korea, China and Japan, we ran conferences, published books and are proud to have been innovative and productive in our multidisciplinary research. But here I do not intend to dwell on our own achievements, rather I will highlight several important new papers in this area.
Korean authors just published a meta-analysis to assess the effectiveness of acupuncture as therapy for gouty arthritis. Ten RCTs involving 852 gouty arthritis patients were included. Six studies of 512 patients reported a significant decrease in uric acid in the treatment group compared with a control group, while two studies of 120 patients reported no such effect. The remaining four studies of 380 patients reported a significant decrease in pain in the treatment group.
The authors conclude “that acupuncture is efficacious as complementary therapy for gouty arthritis patients”.
We should be delighted with such a positive and neat result! Why then do I hesitate and have doubts?
I believe that this paper reveals several important issues in relation to systematic reviews of Chinese acupuncture trials and studies of other TCM interventions. In fact, this is my main reason for discussing the new meta-analysis here. The following three points are crucial, in my view:
1) All the primary studies were from China, and 8 of the 10 were only available in Chinese.
2) All of them had major methodological flaws.
3) It has been shown repeatedly that all acupuncture-trials from China are positive.
Given this situation, the conclusions of any review for which there are only Chinese acupuncture studies might as well be written before the actual research has started. If the authors are pro-acupuncture, as the ones of the present article clearly are, they will conclude that “acupuncture is efficacious“. If the research team has some critical thinkers on board, the same evidence will lead to an entirely different conclusion, such as “due to the lack of rigorous trials, the evidence is less than compelling.“
Systematic reviews are supposed to be the best type of evidence we currently have; they are supposed to guide therapeutic decisions. I find it unacceptable that one and the same set of data could be systematically analysed to generate such dramatically different outcomes. This is confusing and counter-productive!
So what is there to do? How can we prevent being misled by such articles? I think that medical journals should refuse to publish systematic reviews which so clearly lack sufficient critical input. I also believe that reviewers of predominantly Chinese studies should provide English translations of these texts so that they can be independently assessed by those who are not able to read Chinese – and for the sake of transparency, journal editors should insist on this point.
And what about the value of acupuncture for gouty arthritis? I think I let the readers draw their own conclusion.
Everyone knows, I think, that smoking is bad for our health. Why then do so many of us still smoke? Because smoking is addictive – and addictions are, by definition, far from easy to get rid of. Many smokers try acupuncture, and acupuncturists are making a ‘pretty penny’ on the assumption that their treatment is an effective way to stop the habit. But what does the best evidence tell us?
A new randomized, double-blind, placebo-controlled clinical trial with 125 smokers was conducted to determine whether ear acupuncture with electrical stimulation (auriculotherapy) once a week for 5 consecutive weeks is more effective than sham treatment.
The results showed that there was no difference in the rate of smoking cessation between the two groups. After 6 weeks, the auriculotherapy group achieved a rate of 20.9% abstinence which was not significantly different from the 17.9% in the sham group.
The authors of this study concluded that “the results … do not support the use of auriculotherapy to assist with smoking cessation. It is possible that a longer treatment duration, more frequent sessions, or other modifications of the intervention protocol used in this study may result in a different outcome. However, based on the results of this study, there is no evidence that auriculotherapy is superior to placebo when offered once a week for 5 weeks, as described in previous uncontrolled studies.”
Of course, they are correct to state that, theoretically, a different treatment regimen might have generated different outcomes. But how likely is that in reality?
To answer this question, we might consult the Cochrane review on the subject (which incidentally is close to my heart: I initiated it many years ago and was its senior author until it was plagiarised by my former co-worker and my name was replaced by that of his new boss [never a dull day in alternative medicine research!]).
The latest version of this article concludes that “there is no consistent, bias-free evidence that acupuncture, acupressure, laser therapy or electrostimulation are effective for smoking cessation, but lack of evidence and methodological problems mean that no firm conclusions can be drawn. Further, well designed research into acupuncture, acupressure and laser stimulation is justified since these are popular interventions and safe when correctly applied, though these interventions alone are likely to be less effective than evidence-based interventions”
This is a very, very (yes, I meant very, very) odd conclusion, I think. If I had still been an author of this plagiarised paper, I would have suggested something a little more straightforward: 33 studies of various types of acupuncture for smoking cessation are currently available (if we include the new trial, the number is 34). The totality of this evidence fails to show that acupuncture is effective. Therefore acupuncture should NOT be considered a valid option for this indication.
Acupuncture remains a highly controversial treatment: its mechanism of action is less than clear and the clinical results are equally unconvincing. Of course, one ought to differentiate between different conditions; the notion that acupuncture is a panacea is most certainly nonsense.
In many countries, acupuncture is being employed mostly in the management of pain, and it is in this area where the evidence is perhaps most encouraging. Yet, even here the evidence from the most rigorous clinical trials seems to suggest that much, if not all of the effects of acupuncture might be due to placebo.
Moreover, we ought to be careful with generalisations and ask what type of pain? One very specific pain is that caused by aromatase inhibitors (AI), a medication frequently prescribed to women suffering from breast cancer. Around 50 % of these patients complain of AI-associated musculoskeletal symptoms (AIMSS) and 15 % discontinue treatment because of these complaints. So, can acupuncture help these women?
A recent randomised, sham-controlled trial tested whether acupuncture improves AIMSS. Postmenopausal women with early stage breast cancer, experiencing AIMSS were randomized to eight weekly real or sham acupuncture sessions. The investigators evaluated changes in the Health Assessment Questionnaire Disability Index (HAQ-DI) and pain visual analog scale (VAS). Serum estradiol, β-endorphin, and proinflammatory cytokine concentrations were also measured pre and post-intervention. In total, 51 women were enrolled of whom 47 were evaluable (23 randomized to real and 24 to sham acupuncture).
Baseline characteristics turned out to be balanced between groups with the exception of a higher HAQ-DI score in the real acupuncture group. The results failed to show a statistically significant difference in reduction of HAQ-DI or VAS between the two groups. Following eight weekly treatments, a significant reduction of IL-17 was noted in both groups. No significant modulation was seen in estradiol, β-endorphin, or other proinflammatory cytokine concentrations in either group. No difference in AIMSS changes between real and sham acupuncture was seen.
Even though this study was not large, it was rigorously executed and well-reported. As many acupuncturists claim that their treatment alleviates pain and as many women suffering from AM-induced pain experience benefit, acupuncture advocates will nevertheless claim that the findings of this study are wrong, misleading or irrelevant. The often remarkable discrepancy between experience and evidence will again be the subject of intense discussions. How can a tiny trial overturn the experience of so many?
The answer is: VERY EASILY! In fact, the simplest explanation is that both are correct. The trial was well-done and its findings are thus likely to be true. The experience of patients is equally true – yet it relies not on the effects of acupuncture per se, but on the context in which it is given. In simple language, the effects patients experience after acupuncture are due to a placebo-response.
This is the only simple explanation which tallies with both the evidence and the experience. Once we think about it carefully, we realise that acupuncture is highly placebo-genic:
It is exotic.
It is invasive.
It is slightly painful.
It involves time with a therapist.
It involves touch.
If anyone had the task to develop a treatment that maximises placebo-effects, he could not come up with a better intervention!
Ahhh, will acupuncture-fans say, this means that acupuncture is a helpful therapy. I don’t care how it works, as long as it does help. Did we not just cover this issues in some detail? Indeed we did – and I do not feel like re-visiting the three fallacies which underpin this sentence again.
According to Wikipedia, Gua sha involves repeated pressured strokes over lubricated skin with a smooth edge placed against the pre-oiled skin surface, pressed down firmly, and then moved downwards along muscles or meridians.This intervention causes bleeding from capillaries and sub-cutaneous blemishing which usually last for several days. According to a recent article on Gua Sha, it is a traditional healing technique popular in Asia and Asian immigrant communities involving unidirectional scraping and scratching of the skin until ‘Sha-blemishes’ appear.
Gua Sha paractitioners make far-reaching therapeutic claims, e.g.” Gua Sha is used whenever a patient has pain whether associated with an acute or chronic disorder… In addition to resolving musculo skeletal pain, Gua Sha is used to treat as well as prevent common cold, flu, bronchitis, asthma, as well as any chronic disorder involving pain, congestion of Qi and Blood“. Another source informs us that ” Gua Sha is performed to treat systemic toxicity, poor circulation, physical and emotional stress, and migraines. Gua Sha healing promotes the flow of Qi (energy) and blood throughout the body for overall health“.
Gua Sha “blemishes” can look frightful – more like the result of torture than of treatment. Yet with our current craze for all things exotic in medicine, Gua Sha is becoming popular also in Western countries. One German team has even published several RCTs of Gua Sha.
This group treated 40 patients with neck pain either with Gua Sha or locally applied heat packs. They found that, after one week, the pain was significantly reduced in the former compared to the latter group. The same team also published a study with 40 back or neck-pain patients who either received a single session of Gua Sha or were left untreated. The results indicate that one week later, the treated patients had less pain than the untreated ones.
My favoutite article on the subject must be a case report by the same German research team. It describes a woman suffering from chronic headaches. She was treated with a range of interventions, including Gua Sha – and her symptoms improved. From this course of events, the authors conclude that “this case provides first evidence that Gua Sha is effective in the treatment of headaches”
The truth, of course, is that neither this case nor the two RCTs provide any good evidence at all. The case-report is, in fact, a classic example of drawing hilariously over-optimistic conclusions from data that are everything but conclusive. And the two RCTs just show how remarkable placebo-effects can be, particularly if the treatment is exotic, impressive, involves physical touch, is slightly painful and raises high expectations.
My explanation for the observed effects after Gua Sha is quite simple: imagine you have a headache and accidentally injure yourself – say you fall off your bike and the tarmac scrapes off an area of skin on your thigh. This hurts quite a bit and distracts you from your headache, perhaps even to such an extend that you do not feel it any more. As the wound heals, it gets a bit infected and thus hurts for several days; chances are that your headache will be gone for that period of time. Of course, the Gua Sha- effect would be larger because the factors mentioned above (exotic treatment, expectation etc.) but essentially the accident and the treatment work via similar mechanisms, namely distraction and counter-irritation. And neither Gua Sha nor injuring yourself on the tarmac are truly recommendable therapies, in my view.
But surely, for the patient, it does not matter how she gets rid of her headache! The main point is that Gua Sha works! In a way, this attitude is understandable – except, we do not need the hocus pocus of meridians, qi, TCM, ancient wisdom etc. nor do we need to tolerate claims that Gua Sha is “serious medicine” and has any specific effects whatsoever. All we do need is to apply some common sense and then use any other method of therapeutic counter-irritation; that might be more honest, safer and would roughly do the same trick.
No, I am wrong! I forgot something important: it would not be nearly as lucrative for the TCM-practitioner.
Would it not be nice to have a world where everything is positive? No negative findings ever! A dream! No, it’s not a dream; it is reality, albeit a reality that exists mostly in the narrow realm of alternative medicine research. Quite a while ago, we have demonstrated that journals of alternative medicine never publish negative results. Meanwhile, my colleagues investigating acupuncture, homeopathy, chiropractic etc. seem to have perfected their strategy of avoiding the embarrassment of a negative finding.
Since several years, researchers in this field have adopted a study-design which is virtually sure to generate nothing but positive results. It is being employed widely by enthusiasts of placebo-therapies, and it is easy to understand why: it allows them to conduct seemingly rigorous trials which can impress decision-makers and invariably suggests even the most useless treatment to work wonders.
One of the latest examples of this type of approach is a trial where acupuncture was tested as a treatment of cancer-related fatigue. Most cancer patients suffer from this symptom which can seriously reduce their quality of life. Unfortunately there is little conventional oncologists can do about it, and therefore alternative practitioners have a field-day claiming that their interventions are effective. It goes without saying that desperate cancer victims fall for this.
In this new study, cancer patients who were suffering from fatigue were randomised to receive usual care or usual care plus regular acupuncture. The researchers then monitored the patients’ experience of fatigue and found that the acupuncture group did better than the control group. The effect was statistically significant, and an editorial in the journal where it was published called this evidence “compelling”.
Due to a cleverly over-stated press-release, news spread fast, and the study was celebrated worldwide as a major breakthrough in cancer-care. Finally, most commentators felt, research has identified an effective therapy for this debilitating symptom which affects so many of the most desperate patients. Few people seemed to realise that this trial tells us next to nothing about what effects acupuncture really has on cancer-related fatigue.
In order to understand my concern, we need to look at the trial-design a little closer. Imagine you have an amount of money A and your friend owns the same sum plus another amount B. Who has more money? Simple, it is, of course your friend: A+B will always be more than A [unless B is a negative amount]. For the same reason, such “pragmatic” trials will always generate positive results [unless the treatment in question does actual harm]. Treatment as usual plus acupuncture is more than treatment as usual, and the former is therefore moer than likely to produce a better result. This will be true, even if acupuncture is no more than a placebo – after all, a placebo is more than nothing, and the placebo effect will impact on the outcome, particularly if we are dealing with a highly subjective symptom such as fatigue.
I can be fairly confident that this is more than a theory because, some time ago, we analysed all acupuncture studies with such an “A+B versus B” design. Our hypothesis was that none of these trials would generate a negative result. I probably do not need to tell you that our hypothesis was confirmed by the findings of our analysis. Theory and fact are in perfect harmony.
You might say that the above-mentioned acupuncture trial does still provide important information. Its authors certainly think so and firmly conclude that “acupuncture is an effective intervention for managing the symptom of cancer-related fatigue and improving patients’ quality of life”. Authors of similarly designed trials will most likely arrive at similar conclusions. But, if they are true, they must be important!
Are they true? Such studies appear to be rigorous – e.g. they are randomised – and thus can fool a lot of people, but they do not allow conclusions about cause and effect; in other words, they fail to show that the therapy in question has led to the observed result.
Acupuncture might be utterly ineffective as a treatment of cancer-related fatigue, and the observed outcome might be due to the extra care, to a placebo-response or to other non-specific effects. And this is much more than a theoretical concern: rolling out acupuncture across all oncology centres at high cost to us all might be entirely the wrong solution. Providing good care and warm sympathy could be much more effective as well as less expensive. Adopting acupuncture on a grand scale would also stop us looking for a treatment that is truly effective beyond a placebo – and that surely would not be in the best interest of the patient.
I have seen far too many of those bogus studies to have much patience left. They do not represent an honest test of anything, simply because we know their result even before the trial has started. They are not science but thinly disguised promotion. They are not just a waste of money, they are dangerous – because they produce misleading results – and they are thus also unethical.