I have warned you before to be sceptical about Chinese studies. This is what I posted on this blog more than 2 years ago, for instance:
Imagine an area of therapeutics where 100% of all findings of hypothesis-testing research are positive, i.e. come to the conclusion that the treatment in question is effective. Theoretically, this could mean that the therapy is a miracle cure which is useful for every single condition in every single setting. But sadly, there are no miracle cures. Therefore something must be badly and worryingly amiss with the research in an area that generates 100% positive results.
Acupuncture is such an area; we and others have shown that Chinese trials of acupuncture hardly ever produce a negative finding. In other words, one does not need to read the paper, one already knows that it is positive – even more extreme: one does not need to conduct the study, one already knows the result before the research has started. But you might not believe my research nor that of others. We might be chauvinist bastards who want to discredit Chinese science. In this case, you might perhaps believe Chinese researchers.
In this systematic review, all randomized controlled trials (RCTs) of acupuncture published in Chinese journals were identified by a team of Chinese scientists. A total of 840 RCTs were found, including 727 RCTs comparing acupuncture with conventional treatment, 51 RCTs with no treatment controls, and 62 RCTs with sham-acupuncture controls. Among theses 840 RCTs, 838 studies (99.8%) reported positive results from primary outcomes and two trials (0.2%) reported negative results. The percentages of RCTs concealment of the information on withdraws or sample size calculations were 43.7%, 5.9%, 4.9%, 9.9%, and 1.7% respectively.
The authors concluded that publication bias might be major issue in RCTs on acupuncture published in Chinese journals reported, which is related to high risk of bias. We suggest that all trials should be prospectively registered in international trial registry in future.
END OF QUOTE
Now an even more compelling reason emerged for taking evidence from China with a pinch of salt:
A recent survey of clinical trials in China has revealed fraudulent practice on a massive scale. China’s food and drug regulator carried out a one-year review of clinical trials. They concluded that more than 80 percent of clinical data is “fabricated“. The review evaluated data from 1,622 clinical trial programs of new pharmaceutical drugs awaiting regulator approval for mass production. Officials are now warning that further evidence malpractice could still emerge in the scandal.
According to the report, much of the data gathered in clinical trials are incomplete, failed to meet analysis requirements or were untraceable. Some companies were suspected of deliberately hiding or deleting records of adverse effects, and tampering with data that did not meet expectations.
“Clinical data fabrication was an open secret even before the inspection,” the paper quoted an unnamed hospital chief as saying. Contract research organizations seem have become “accomplices in data fabrication due to cutthroat competition and economic motivation.”
A doctor at a top hospital in the northern city of Xian said the problem doesn’t lie with insufficient regulations governing clinical trials data, but with the failure to implement them. “There are national standards for clinical trials in the development of Western pharmaceuticals,” he said. “Clinical trials must be carried out in three phases, and they must be assessed at the very least for safety,” he said. “But I don’t know what happened here.”
Public safety problems in China aren’t limited to the pharmaceutical industry and the figure of 80 percent is unlikely to surprise many in a country where citizens routinely engage in the bulk-buying of overseas-made goods like infant formula powder. Guangdong-based rights activist Mai Ke said there is an all-pervasive culture of fakery across all products made in the country. “It’s not just the medicines,” Mai said. “In China, everything is fake, and if there’s a profit in pharmaceuticals, then someone’s going to fake them too.” He said the problem also extends to traditional Chinese medicines, which are widely used in conjunction with Western pharmaceuticals across the healthcare system.
“It’s just harder to regulate the fakes with traditional medicines than it is with Western pharmaceuticals, which have strict manufacturing guidelines,” he said.
According to Luo, academic ethics is an underdeveloped field in China, leading to an academic culture that is accepting of manipulation of data. “I don’t think that the 80 percent figure is overstated,” Luo said.
And what should we conclude from all this?
I find it very difficult to reach a verdict that does not sound hopelessly chauvinistic but feel that we have little choice but to distrust the evidence that originates from China. At the very minimum, I think, we must scrutinise it thoroughly; whenever it looks too good to be true, we ought to discard it as unreliable and await independent replications.
For some time now, the research activity in and around alternative medicine has been seemingly buoyant. In each of the last 4 years, Medline listed around 2 000 articles is the category of ‘complementary alternative medicine’. This will surely look impressive to many!
Why then did I write ‘seemingly’? To comprehend this a little better, we should have some comparisons. Here are numbers of Medline-listed articles published in 2015 for a few other areas:
- Surgery: 176 277
- Psychology: 65 679
- Internal medicine: 36 998
- Obstetrics/gynaecology: 13 818
- Pharmacology: 194 322
- Paediatrics: 30 646
Now you see, I hope, why the 2 049 Medline-listed articles in the category of ‘complementary alternative medicine’ are only seemingly impressive. But what about specific alternative therapies? Here are numbers of Medline-listed articles published in 2015 for some major alternative treatments:
- Homeopathy: 181
- Herbal medicine: 1 572
- Chiropractic: 314
- Acupuncture: 1 784
- Naturopathy: 45
- Dietary supplements: 5 199
These figures are perhaps interesting but not easy to interpret. They might indicate that certain sections of alternative medicine are more open to scientific scrutiny than others. Or do they show that for some areas there are more research funds and expertise than others? I am not sure I know the answer.
If we look a little closer at the research activity in defined alternative therapies, we are bound to get disappointed. I have recently done this for homeopathy and for acupuncture and reached rather gloomy conclusions.
In the case of homeopathy the were:
- The research activity into homeopathy is currently very subdued.
- Arguably the main research question of efficacy does not seem to concern researchers of homeopathy all that much.
- There is an almost irritating abundance of papers that are data-free and thrive on opinion (my category of ‘other papers’).
- Given all this, I find it hard to imagine that this area of investigation is going to generate much relevant new knowledge or clinical progress.
And in the case of acupuncture, I stated:
- Too little research is focussed on the two big questions: efficacy and safety.
- In relation to the meagre output in RCTs, there are too many systematic reviews.
- As long as we cannot be sure that acupuncture is more than a placebo, all these pre-clinical studies seem a bit out of place.
- The vast majority of the articles were in low or very low impact journals.
- There was only one paper that I would consider outstanding.
And what about the quality of the research into alternative medicine?
Well, this is a sad and depressing tale! If you doubt it, read my previous post or indeed any of the other ~500 which I have written on this particular subject in the past.
This is a post that I wanted to write for a while (I had done something similar on acupuncture moths ago); but I had to wait, and wait, and wait…until finally there were the awaited 100 Medline listed articles on homeopathy with a publication date of 2016. It took until the beginning of August to reach the 100 mark. To put this into perspective with other areas of alternative medicine, let me give you the figures for 3 other therapies:
- there are currently 1 413 articles from 2016 on herbal medicine;
- 875 on acupuncture;
- and 256 on chiropractic.
And to give you a flavour of the research activity in some areas of conventional medicine:
- there are currently almost 100 000 articles from 2016 on surgery;
- 1 410 on statins;
- and 33 033 on psychotherapy.
This suggests quite strongly, I think, that the research activity in homeopathy is relatively low (to put it mildly).
So, what do the first 100 Medline articles on homeopathy cover? Here are some of the findings of my mini-survey:
- there were 4 RCTs;
- 3 systematic reviews;
- 8 papers on observational-type data (case series, observational studies etc.);
- 9 animal studies;
- 14 other pre-clinical or basic research studies;
- 1 pilot study;
- 14 investigations of the quality of homeopathic preparations;
- 15 surveys;
- 2 investigations into the adverse effects of homeopathic treatments;
- 49 other papers (e. g. comments, opinion pieces, letters, perspective articles, editorials).
I should mention that, because I assessed 100 papers, the above numbers can be read both as absolute as well as percentage figures.
How should we interpret my findings?
As with my previous evaluation, I must caution not to draw generalizable conclusions from them. What follows should therefore be taken with a pinch of salt (or two):
- The research activity into homeopathy is currently very subdued.
- Arguably the main research question of efficacy does not seem to concern researchers of homeopathy all that much.
- There is an almost irritating abundance of papers that are data-free and thrive on opinion (my category of ‘other papers’).
- Given all this, I find it hard to imagine that this area of investigation is going to generate much relevant new knowledge or clinical progress.
On a good day, I get several emails from complete strangers; some are complimentary, others are critical, and others again are just strange. Few are stranger than the exchange I am about to disclose.
The author asked me twice to treat his/her emails with ‘trust and confidence’; after the second email, I nevertheless felt that I should not respect this wish but needed to share this brief exchange with my readers. I have, however, erased all the details that would allow an identification of the author.
INITIAL EMAIL of 18/7/2016
I am responding to you latest post regarding “Informed Consent”. I have decided to do so because my instincts suggest that we may in fact have an empathy in our individual objective to establish an evidence base for complementary medicine. However, I do not have any empathy with many of the contributors to your blog and especially with those that have a desire to “grind homeopathic vets and feed them to the pigs” Given that you moderate the site, I am surprised that you allowed such a post.
As you are aware, I obtained a copy of your book “A Scientist in Wonderland” which I have read with considerable interest and as you know, I have posted extracts on your blog. In this respect I make the following observations:
1. Your early experiences of homeopathy were positive and on this basis I find great difficulty in accepting that you are as anti-homeopathy as you publically state. From my own experience, this is not logical.
2. I am of the opinion that the sad loss of your Hungarian friend and colleague is an influencing factor, particularly as you avoided any mention of him receiving any form of alternative medicine.
3. I can empathise with your frustration at the lack of support from the alternative medicine community, as I have experienced this in my own efforts.
4. I am inclined to accept the possibility that you are using the blog to deliberately provoke the homeopathic community into action from a long standing but understandable state of complacency. (If you know that something works, then why is there a need to prove it).
5. I find difficulty to believe that you are at home surrounded by such closed minded individuals, because, historically, you have always moved on from such situations. However, I am not sure that you know how you can escape from the trap that you now find yourself in. Is this what you want for the rest of your life?
For a variety of reasons, I embarked on this … venture as a means of finding evidence that these therapies do work and have found that the homeopathy community is somewhat less than supportive in my efforts, so I do understand your potential frustration.
I appreciate that my observations are assumption based and may be wishful thinking on my part; however, if my assumptions have validity, please contact me, otherwise ignore this message.
If you do choose to pursue this conversation, then it must take place under the strict condition of TRUST & CONFIDENCE.
MY REPLY of 18/7/2016
thank you for your email. you say you read my memoir; may I suggest you read it again – because the answers to your questions seem to be all in there. your assumptions about me are quite wrong, and I think my book explains why.
THE RESPONSE of 21/7/2016
In Britain we have a saying “Don’t mention the war when speaking to a German”, so out of respect I refrained from mentioning the Nazi regime in my last message; however, as you have made an implied reference to it, I will now comment.
I have some six years of close working experience with a large German organisation … so that I am fully aware of the significant differences between the German and British mentality and approach to life. I am therefore able to appreciate many of the difficulties that you will have encountered when arriving in this country to take up the Exeter post, which by definition was designed to advise the UK alternative therapy community how to do things properly!
The Anglo/Germanic axis is a significant challenge under normal circumstances but for you to arrive in this country and make direct comparisons between alternative medicine and the Third Reich in a country that spearheaded the fight against the Nazi’s at a cost of nearly half a million British lives was a fatal mistake on your part.
Having spent some forty years in and around the alternative health world here in Britain, India and the USA I don’t think your view point can be further from the truth. What amazes me is that you do not moderate Nazi type comments such as “grinding homeopaths and feeding them to pigs” from your blog which is a complete contradiction to your reasoning.
Your blog purports to provide cautionary advice to would be patients choosing alternative health options but your band of followers seem to have no understanding whatsoever as to the importance of respect for others. They seem to believe that from the offset, respect has to be earned, which implies judgement. Any doctor or therapist that starts from this view point when dealing with a patient, should not be treating patients at all. Empathy and respect are key factors in the healing process and those that automatically practice this naturally operate under and accept a moral code of ethics which forms part of all training within the main alternative treatments. The fundamental ethic behind all medicine is “first do no harm”. How can this be achieved if you do not respect the patient, regardless of his views?
At a personal level, I am concerned that your early experiences have distorted your views and unfortunately you have managed to alienate yourself from the very form of healthcare that would best resolve these issues without the need for suppressive drugs.
I suggest that you re-read your book and honestly ask yourself if the “peaceful vantage point” referred to on page 170, in any way measures up to the “peaceful, happy time” you mention on page 36.
I again extend my offer of an exploratory conversation in an atmosphere of “trust and confidence”.
END OF QUOTE
I do not feel like adding any comments just now… perhaps just a few questions:
How is it possible that someone who has obviously read quite a bit of what I have published misunderstands so much of it? Deluded? Demented? Or worse?
What a silly question! At least this is what most sceptics would say: if we are not sure that it works, we do not need to spend any thoughts on a potential mechanism!
However, in the realm of acupuncture, the potential mode of action remains a hotly debated and fundamentally relevant issue.
The TCM folks, of course, ‘knew’ all along how acupuncture works: it re-balances the life-forces yin and yang. This is a nice theory – it has but one disadvantage: it has no bearing whatsoever on reality. Vitalistic ideas such as this one have long been proven to be nothing but fantasy.
Meanwhile, several more plausible hypotheses have been developed, and hundreds of papers have been published on the subject. One recent article, for instance, suggests a range of mechanisms including microinjury, increased local blood flow, facilitated healing, and analgesia. Acupuncture may trigger a somatic autonomic reflex, thereby affecting the gastric and cardiovascular functions. Acupuncture may also change the levels of neurotransmitters such as serotonin and dopamine, thereby affecting the emotional state and craving… By affecting other pain-modulating neurotransmitters such as met-enkephalin and substance P along the nociceptive pathway, acupuncture may relieve headache. Acupuncture may affect the hypothalamus pituitary axis and reduce the release of the luteinizing hormone…
Another article states that the Western explanation for acupuncture effectiveness is based upon more than half a century of basic and clinical research, which identified the activation of sensory system and the subsequent activity-dependent regulation of neurotransmitters, neurohormones, and several classes of neuromodulators as plausible mechanism for the acupuncture‘s therapeutic properties. The regulation of neurotrophins’ expression and activity is one of the possible neurophysiological mechanisms underlying acupuncture‘s effects on neuropathic pain, nerve injury, neurodegeneration, and even in the regulation of gonadal functions…
Recently Burnstock proposed that mechanical deformation of the skin by needles and application of heat or electrical current leads to release of large amounts of ATP from keratinocytes, fibroblasts and other cells in skin; the ATP then occupies specific receptor subtypes expressed on sensory nerve endings in the skin and tongue; the sensory nerves send impulses through ganglia to the spinal cord, the brain stem, hypothalamus and higher centres; the brain stem and hypothalamus contain neurons that control autonomic functions, including cardiovascular, gastrointestinal, respiratory, urinogenital and musculo-skeletal activity. Impulses generated in sensory fibres in the skin connect with interneurons to modulate (either inhibition or facilitation) the activities of the motoneurons in the brain stem and hypothalamus to change autonomic functions; specifically activated sensory nerves, via interneurons, also inhibit the neural pathways to the pain centres in the cortex.
A brand-new article in the journal SCIENTIFIC AMERICAN puts the hypothesis in perspective:
…scientists have been studying a roster of potential biological pathways by which needling might relieve pain. The most successful of these efforts has centered on adenosine, a chemical believed to ease pain by reducing inflammation. A 2010 mouse study found that acupuncture needles triggered a release of adenosine from the surrounding cells into the extracellular fluid that diminished the amount of pain the rodents experienced. The mice had been injected with a chemical that made them especially sensitive to heat and touch. The researchers reported a 24-fold increase in adenosine concentration in the blood of the animals after acupuncture, which corresponded to a two-thirds reduction in discomfort, as revealed by how quickly they recoiled from heat and touch. Injecting the mice with compounds similar to adenosine had the same effect as acupuncture needling. And injecting compounds that slowed the removal of adenosine from the body boosted the effects of acupuncture by making more adenosine available to the surrounding tissue for longer periods. Two years later a different group of researchers went on to show that an injection of PAP, an enzyme that breaks other compounds in the body down into adenosine, could relieve pain for an extended chunk of time by increasing the amount of adenosine in the surrounding tissue. They dubbed that experimental procedure “PAPupuncture.”
Both sets of findings have excited researchers—and for good reason. The current options for treating pain are limited and rely mostly on manipulating the body’s natural pain-management system, known as the opioid system. Opioid-based painkillers are problematic for several reasons. Not only does their efficacy tend to wane over time, but they have been linked to an epidemic of addiction and overdose deaths across the U.S.—so much so that the Centers for Disease Control and Prevention has recently advised doctors to seriously restrict their use. The available nonopioid pain treatments are few; many of them require multiple injections or catheterization to work; and they often come with side effects, such as impaired movement. Adenosine offers an entirely new mechanism to exploit for potential treatments—one that may come with fewer side effects and less potential for addiction. What is more, adenosine can be made to circulate in the body for prolonged stretches. Pharmaceutical companies are actively investigating adenosine-related compounds as potential drugs.
But however promising adenosine may be as a treatment, the findings from this research do not prove that acupuncture itself “works.” For one thing, the researchers did not show that the release of adenosine was specific to acupuncture. Acupuncture needles might cause adenosine to flood the surrounding tissue, but so might a hard pinch, or applied pressure, or any number of other physical insults. In fact, both of the studies found that when adenosine was turned on in mouse tissue by other mechanisms, the pain response was equal to or better than the response generated by acupuncture. For another thing, the study results offered no support for the use of acupuncture to treat any of the other conditions for which the procedure is often advertised. A localized adenosine response may mitigate localized pain. That does not mean it can also cure insomnia or infertility.
It may well be that the reams of research scientists have done on acupuncture have lit the path toward improved understanding of—and eventually better treatments for—intractable pain. But it may also be time to take whatever bread crumbs have been laid out by that work and move on.
END OF QUOTE
As we see, there is no shortage of potential explanations as to HOW acupuncture works. The most plausible theory still is that it works largely or even exclusively via a placebo effect.
Due to this type of mechanistic research, acupuncture has gained much credibility. The question is, does it deserve it? In my view, it would be much more fruitful to first make sure THAT acupuncture works (beyond a placebo response) and, if so, for what conditions. The question HOW it works is unquestionably interesting but in the final analysis it probably is secondary.
Yesterday, a press-release reached me announcing that a Chinese herbal medicine, ‘Phynova Joint and Muscle Relief Tablets’, containing the active ingredient Sigesbeckia, is now on sale in the UK for the first time in Boots The Chemist:
Sigesbeckia is the first traditional Chinese treatment granted a traditional herbal registration (THR) under the traditional herbal medicines product directive in the UK, by drug safety watchdog the Medicines and Healthcare Products Regulatory Agency (MHRA). Oxford based Phynova which manufactures the product was granted the UK licence last year.
Containing 500mg of the active ingredient, Phynova Joint and Muscle Relief Tablets are specially formulated for the relief of backache, arthritis, minor sports injuries, rheumatic or muscular pains and general aches and pains in muscles or joints. Two tablets are taken each day, one in the morning and one in the evening. They have no known side effects and are non-addictive. ..
The product, which retails at £19.99 for one month’s supply of 60 tablets, is available in 950 UK Boots outlets and online via Click and Collect from all stores. It will be sold both Over the Counter (OTC) by pharmacist staff and off the shelf as part of Boots’ pain relief fixture…
END OF QUOTE
What on earth is a ‘joint and muscle relief’? Personally I do not want to be relieved of my joints and muscles!!!
Yes, I know, they probably mean ‘joint and muscle pain relief’ but were not allowed to say so because this is a medical indication.
And what about the claim of ‘no side-effects’; is it possible that a pharmacological treatment has positive effects without any risks at all? This is not what they told me during my pharmacology course, if I remember correctly. And anyway, even placebos have side-effects!
I admit, I was puzzled.
The covering letter of the press-release provided more amazement: it informed me that “Phynova joint and muscle relief contains the active ingredient Sigesbeckia which has been through clinical trials and has been used for pain relief in China for hundreds of years…” It was the remark about clinical trials (PLURAL!!!) that caught my interest most.
So, I looked up ‘Sigesbeckia’ on Medline and found as good as nothing. This is mainly because the plant is spelled correctly ‘Siegesbeckia’ in honour of the famous botanist Siegesbeck.
Looking up ‘Siegesbeckia’, I found many pre-clinical studies but no clinical trials.
Next I searched for a comment from the MHRA and discovered that their account makes it very clear that a licence has been granted to this product “exclusively upon long standing use… and not upon data from clinical trials.”
So, who is right?
Are there clinical trials of this product or not? And, if there are any, where are they?
Perhaps someone from Phynova can enlighten us?
Nobody can doubt that, during the last 200 years, conventional medicine has made monumental progress. Homeopathy, however, has remained more or less like Hahnemann invented it. But now it seems as though homeopathy can celebrate an unprecedented step ahead. As so often in medicine, it originates from a commercial enterprise.
Genexa is a US firm that produces natural health products. On their website, they state that “At Genexa, we believe medicine should be free from unhealthy fillers and toxins”. They recently published a press-release introducing a line of homeopathic medicines certified organic by the U.S. Department of Agriculture and Non-GMO Project verified. They are keen to point out that these products “do not contain any genetically modified ingredients.” In fact, several of their remedies do not contain any active ingredients to speak of: they are homeopathic!
“We are extremely proud of our organic and non-GMO certifications – the seals are prominently featured on all our products and website for easy label reading and patient education,” stated David Johnson, CEO of Genexa, in their press-release. “Our quality standards are among the highest in the over-the-counter medicine industry.”
Genexa’s 11 homeopathic formulations are being advertised for the treatment of common health issues such as flu, cold, allergies, stress, pain, leg cramps, sleeplessness and jet lag. An entire line of products is, according to the press-release, specially formulated for children and includes treatments for cold, allergy and calming.
Genexa’s CMO proudly announced that “It’s important to us that our retail customers feel confident in the products and know they can trust they are purchasing medicines free from unhealthy fillers and toxins and simply focus on healing.” Presumably that trust must include the trust into the efficacy of the homeopathic remedies! Yes, I am pleased to report that, apparently it does; elsewhere they confirm this by stating that “Genexa holds itself to the highest standards in both quality and ethics.” The highest standards of ethics surely include that the remedies in question are demonstrably efficacious.
But how can we be sure? Are any of these homeopathic remedies supported by reasonably strong evidence? Oddly enough, despite all these affirmations, I did get my doubts when I tried to dig a bit deeper.
Take the homeopathic remedy called SLEEPOLOGY, for instance. The website informs us that “This homeopathic formulation consists of nine leading remedies designed to treat sleeplessness, inability to fall asleep, frequent waking, restless sleep and sleeplessness from stress, exhaustion, nervousness, excitability, restlessness, worries, irritability, and pain.” So, it’s a complex homeopathic remedy with 9 different ingredients. But is there any evidence of efficacy for this mixture? I am not aware of any clinical trials that have tested its efficacy. But I must be wrong, because on the website we are being told that “Clinical trials have demonstrated efficacy for treating sleeplessness for piper methysticum, and valeriana officinalis.” That may be so, but the trials were done with herbal extracts, not with homeopathic potencies! Could the statement therefore be more than a little misleading?
On the internet, I found all sorts of fascinating bits about the new homeopathic lines (my compliments to the PR firm that organised the launch!); for instance the revelation that: “The company’s proprietary medicines were created by and are regularly reviewed and enhanced by its chief medical officer, Dr. Todd Rowe*, a nationally respected physician with an expertise in homeopathic medicine formulation. Working with the Genexa team, Dr. Rowe and his team of chemists and pharmacists spent hundreds of hours meticulously formulating and testing the products. The result is a line of effective, potent medicines that are certified organic by the USDA and non-GMO verified by the Non-GMO Project. “Our formulations are based on tried and true principles for miasmatic and energetic balance, so that the remedies potentiate each other and promote the most positive patient outcomes,” said Dr. Rowe. “These powerful medicines work with your body to help it heal itself.”” However, I was unable to find out which potencies are being used for the Genexa homeopathic products. This information might not be that relevant: according to the homeopathic ‘like cures like’ principle, the effects of a substance are reversed through potentiation. This is why coffee, for instance, is potentised by homeopath to generate a sleeping remedy. Does it not follow then that, potentising two or more herbal ingredients that have hypnotic effects (as in SLEEPOLOGY), must generate a remedy for preventing sleep? A similarly puzzling lack of ‘homeopathic logic’ seems to apply to several other products in Genexa’s line of homeopathic remedies.
I have to admit, I am confused.
Could it be that the ‘breakthrough’ turns out to be a breakdown of ‘homeopathic logic’?
Let’s hope someone from Genexa reads these lines and can enlighten us.[*he is the President of the American Medical College of Homeopathy]
The ACUPUNCTURE NOW FOUNDATION (ANF) have recently published a document that is worth drawing your attention to. But first I should perhaps explain who the ANF are. They state that “The Acupuncture Now Foundation (ANF) was founded in 2014 by a diverse group of people from around the world who were concerned about common misunderstandings regarding acupuncture and wanted to help acupuncture reach its full potential. Our goal is to become recognized as a leader in the collection and dissemination of unbiased and authoritative information about all aspects of the practice of acupuncture.”
This, I have to admit, sounds like music to my ears! So, I studied the document in some detail – and the music quickly turned into musac.
The document which they call a ‘white paper’ promises ‘a review of the research’. Reading even just the very first sentence, my initial enthusiasm turned into bewilderment: “It is now widely accepted across health care disciplines throughout the world that acupuncture can be effective in treating such painful conditions as migraine headaches, and low back, neck and knee pain, as well as a range of painful musculoskeletal conditions.” Any review of research that starts with such a deeply uncritical and overtly promotional statement, must be peculiar (quite apart from the fact that the ANF do not seem to appreciate that back and neck pain are musculoskeletal by nature).
As I read on, my amazement grew into bewilderment. Allow me to present a few further statements from this review (together with a link to the article provided by the ANF in support and a very brief comment by myself) which I found more than a little over-optimistic, far-fetched or plainly wrong:
“Male fertility, especially sperm production and motility, has also been shown to improve with acupuncture. In a recent animal study, electro-acupuncture was found to enhance germ cell proliferation. This action is believed to facilitate the recovery of sperm production (spermatogenesis) and may restore normal semen parameters in subfertile patients.”
The article supplied as evidence for this statement refers to an animal experiment using a model where sperm are exposed to heat. This has almost no bearing on the clinical situation in humans and does not lend itself to any clinical conclusions regarding the treatment of sub-fertile men.
“In a recent meta-analysis, researchers concluded that the efficacy of acupuncture as a stand-alone therapy was comparable to antidepressants in improving clinical response and alleviating symptom severity of major depressive disorder (MDD). Also, acupuncture was superior to antidepressants and waitlist controls in improving both response and symptom severity of post-traumatic stress disorder (PTSD). The incidence of adverse events with acupuncture was significantly lower than antidepressants.”
The review provided as evidence is wide open to bias; it was criticised thus: “the authors’ findings did not reflect the evidence presented and limitations in study numbers, sample sizes and study pooling, particularly in some subgroup analyses, suggested that the conclusions are not reliable”. Moreover, we need to know that by no means all reviews of the subject confirm this positive conclusion, for instance, this, this, or this one; all of the latter reviews are more up-to-date than the one provided by ANF. Crucially, a Cochrane review concluded that “the evidence is inconclusive to allow us to make any recommendations for depression-specific acupuncture”.
“A randomized controlled trial of acupuncture and counseling for patients presenting with depression, after having consulted their general practitioner in primary care, showed that both interventions were associated with significantly reduced depression at three months when compared to usual care alone.”
We have discussed the trial in question on this blog. It follows the infamous ‘A+B versus B’ design which cannot possibly produce a negative result.
Now, please re-read the first paragraph of this post; but be careful not to fall off your chair laughing.
There would be more (much more) to criticise in the ANF report but, I think, these examples are ENOUGH!
Let me finish by quoting from the ANF’s view on the future as cited in their new ‘white paper’: “Looking ahead, it is clear that acupuncture is poised to make significant inroads into conventional medicine. It has the potential to become a part of every hospital’s standard of care and, in fact, this is already starting to take place not only in the U.S., but internationally. The treatment is a cost-effective and safe method of relieving pain in emergency rooms, during in-patient stays and after surgery. It can lessen post-operative nausea, constipation and urinary difficulties, and have a positive impact on conditions like hypertension, anxiety and insomnia…
Driven by popular demand and a growing body of scientific evidence, acupuncture is beginning to be taken seriously by mainstream conventional medicine, which is incorporating it into holistic health programs for the good of patients and the future of health care. In order for this transition to take place most effectively, misunderstandings about acupuncture need to be addressed. We hope this white paper has helped to clarify some of those misunderstandings and encourage anyone with questions to contact the Acupuncture Now Foundation.”
My question is short and simple: IGNORANCE OR FRAUD?
Guest post by Frank Van der Kooy
Some serious flaws in the scientific reporting of two acupuncture clinical trials, for the treatment of infertility and allergic rhinitis, were recently published on this blog. The overly positive way in which the researchers made their mostly negative results public, was also of concern. Both these studies were published by the researcher of the year, Prof Caroline Smith, of the National Institute of Complementary Medicine (NICM), Australia. The stream of comments and discussions that followed made me think of another commonly overlooked aspect when it comes to acupuncture clinical trials. Conflict of interest! In both these studies the authors declared to have no conflicts of interest and in other studies by this author this also seems to be the case. The question can be asked: If you are a practicing acupuncturist who runs a clinical trial of acupuncture, isn’t that, by default, a serious conflict of interest? The intention of this article is not an in-depth discussion of what a conflict of interest is, but rather to compare medical doctors with acupuncturists turned researchers. Let me explain.
Some medical doctors (GPs, surgeons etc.) decide to leave their practice after practicing 10-20 years to become full time researchers (and visa versa). Universities accept these people with open arms because they bring with them a wealth of knowledge regarding the practical side of medicine and healthcare in general. They are thus seen as an asset to any medical research project including clinical trials. Can the same be said about an acupuncturist? They also bring with them years of experience and thus they should also be a major asset to any acupuncture clinical trial. But I am afraid not!
Why? Medical doctors have a multitude of tools (drugs, surgical procedures, diagnostic tools etc.) at their disposal to treat all types of medical conditions. When will their background constitute a conflict of interest? When they publish a positive clinical trial of a specific medical intervention in which they have a vested interest. e.g owning shares in the company producing the medical intervention (financial interest) or if they have been staunch supporters of this intervention during their years of practice (emotional interest). Just imagine that you have prescribed a specific intervention to hundreds of patients over a long period of time, and you swore by it, and now you have to face them with a negative clinical trial result – that will be difficult. The former is easy to declare whilst the latter might be slightly more difficult.
Doctors also tend to focus on a specific disease e.g. cancer and will perform research with the existing tools at their disposal but also try to find new tools in order to improve the risk-benefit profile of the disease treatment. Thus, for a doctor there is the possibility that they might run into a conflict of interest, but due to the multitude of medical interventions out there this is by no means a given.
What about acupuncture practitioners turned researchers? An acupuncturist only has one tool at their disposal to treat all medical conditions. I can hear them say; but we stick needles in different places and depths etc. depending on the medical condition! Yes, but the fact remains that they can only stick needles into people – and that is a single intervention. So is this by default a conflict of interest? I would argue, yes, it is like having only one drug to treat all medical conditions. If you have treated hundreds of patients for various medical conditions with acupuncture and now suddenly you publish a negative clinical trial, you will not only be red faced when you run into your former patients – who paid for your evidence based acupuncture treatment – they might even sue you for misleading them. As an acupuncturist, you cannot allow the single tool that you have to be ineffective, otherwise people might start to question acupuncture. The fact that they have to protect acupuncture means that an acupuncturist will by default have a conflict of interest – no matter what medical condition they aim to treat.
If you have been emotionally and financially invested in acupuncture as a cure-all for 10-20 years, it will be very difficult, if not impossible, to publish a negative result as an acupuncture researcher.
Another aspect is that the acupuncture fraternity is a very tight knit community, where negative results are frowned upon because of everyone’s financial and emotional interests. Surely they will expel you from this community, if you publish negative results?
So how do acupuncture researchers go about running clinical trials? An example: Professors Smith and Bensoussan, both at the NICM, are currently registered as practicing acupuncturists. This means that they can legally practice acupuncture and, because they have been active for decades, they are also well known in the acupuncture fraternity. It is unknown, whether they are still actively practicing in their own practice or part-time in someone else’s practice, or if they have a financial stake in their former or someone else’s practice. Based on the fact that they are still registered as active acupuncturists, I can conclude that they do have an emotional and/or financial interest in the positive outcome of their acupuncture clinical trials.
Because of this inherent conflict of interest, and due to current strict clinical trial regulations, which makes it quite difficult (although not impossible) to fabricate or falsify data, they go for the next best thing – which is the design of their clinical trial e.g. the A+B versus A design. But it doesn’t stop there. As soon as a clinical trial fails to give a positive result, the results will be inflated to make it sound positive.
Why? Because they must prevent themselves from cognitive dissonance, they need to protect the single tool that they have, they must keep the acupuncture fraternity happy and they have to protect themselves against potential lawsuits from former (current) patients or a decrease in patient numbers (and thus financial income). On top of that – how would the media and the public react to an acupuncture clinical trial if the lead researcher declare that they have their own acupuncture clinic? Surely these factors together amount to a conflict of interest and should be declared as such?
So what, in this context, is the main difference between a doctor and an acupuncturist? A doctor has a multitude of medical interventions. He or she might have a conflict of interest, if they work on a specific intervention in which they have a vested interest. An acupuncturist only has one intervention and therefore they have a vested interest by default – which they never seem to declare!
Yesterday, I wrote about a new acupuncture trial. Amongst other things, I wanted to find out whether the author who had previously insisted I answer his questions about my view on the new NICE guideline would himself answer a few questions when asked politely. To remind you, this is what I wrote:
This new study was designed as a randomized, sham-controlled trial of acupuncture for persistent allergic rhinitis in adults investigated possible modulation of mucosal immune responses. A total of 151 individuals were randomized into real and sham acupuncture groups (who received twice-weekly treatments for 8 weeks) and a no acupuncture group. Various cytokines, neurotrophins, proinflammatory neuropeptides, and immunoglobulins were measured in saliva or plasma from baseline to 4-week follow-up.
Statistically significant reduction in allergen specific IgE for house dust mite was seen only in the real acupuncture group. A mean (SE) statistically significant down-regulation was also seen in pro-inflammatory neuropeptide substance P (SP) 18 to 24 hours after the first treatment. No significant changes were seen in the other neuropeptides, neurotrophins, or cytokines tested. Nasal obstruction, nasal itch, sneezing, runny nose, eye itch, and unrefreshed sleep improved significantly in the real acupuncture group (post-nasal drip and sinus pain did not) and continued to improve up to 4-week follow-up.
The authors concluded that acupuncture modulated mucosal immune response in the upper airway in adults with persistent allergic rhinitis. This modulation appears to be associated with down-regulation of allergen specific IgE for house dust mite, which this study is the first to report. Improvements in nasal itch, eye itch, and sneezing after acupuncture are suggestive of down-regulation of transient receptor potential vanilloid 1.
…Anyway, the trial itself raises a number of questions – unfortunately I have no access to the full paper – which I will post here in the hope that my acupuncture friend, who are clearly impressed by this paper, might provide the answers in the comments section below:
- Which was the primary outcome measure of this trial?
- What was the power of the study, and how was it calculated?
- For which outcome measures was the power calculated?
- How were the subjective endpoints quantified?
- Were validated instruments used for the subjective endpoints?
- What type of sham was used?
- Are the reported results the findings of comparisons between verum and sham, or verum and no acupuncture, or intra-group changes in the verum group?
- What other treatments did each group of patients receive?
- Does anyone really think that this trial shows that “acupuncture is a safe, effective and cost-effective treatment for allergic rhinitis”?
In the comments section, the author wrote: “after you have read the full text and answered most of your questions for yourself, it might then be a more appropriate time to engage in any meaningful discussion, if that is in fact your intent”, and I asked him to send me his paper. As he does not seem to have the intention to do so, I will answer the questions myself and encourage everyone to have a close look at the full paper [which I can supply on request].
- The myriad of lab tests were defined as primary outcome measures.
- Two sentences are offered, but they do not allow me to reconstruct how this was done.
- No details are provided.
- Most were quantified with a 3 point scale.
- Mostly not.
- Needle insertion at non-acupoints.
- The results are a mixture of inter- and intra-group differences.
- Patients were allowed to use conventional treatments and the frequency of this use was reported in patient diaries.
- I don’t think so.
So, here is my interpretation of this study:
- It lacked power for many outcome measures, certainly the clinical ones.
- There were hardly any differences between the real and the sham acupuncture group.
- Most of the relevant results were based on intra-group changes, rather than comparing sham with real acupuncture, a fact, which is obfuscated in the abstract.
- In a controlled trial fluctuations within one group must never be interpreted as caused by the treatment.
- There were dozens of tests for statistical significance, and there seems to be no correction for multiple testing.
- Thus the few significant results that emerged when comparing sham with real acupuncture might easily be false positives.
- Patient-blinding seems questionable.
- McDonald as the only therapist of the study might be suspected to have influenced his patients through verbal and non-verbal communications.
I am sure there are many more flaws, particularly in the stats, and I leave it to others to identify them. The ones I found are, however, already serious enough, in my view, to call for a withdrawal of this paper. Essentially, the authors seem to have presented a study with largely negative findings as a trial with positive results showing that acupuncture is an effective therapy for allergic rhinitis. Subsequently, McDonald went on social media to inflate his findings even more. One might easily ask: is this scientific misconduct or just poor science?
I would be most interested to hear what you think about it [if you want to see the full article, please send me an email].