MD, PhD, FMedSci, FRSB, FRCP, FRCPEd

acupuncture

If you want to scientifically investigate this question, it might be a good idea NOT to start with the following sentence: “Auricular acupuncture (AA) is effective in the treatment of preoperative anxiety”. Yet, this is exactly what the authors did in their recent publication.

The aim of this new study was to investigate whether AA can reduce exam anxiety as compared to placebo and no intervention. Forty-four medical students were randomized to receive AA, placebo, or no intervention in a crossover manner. Subsequently they completed three comparable oral anatomy exams with an interval of one month between the exams/interventions.

A licensed acupuncturist with more than five years of experience with this technique applied AA at the acupuncture points MA-IC1 (Lung), MA-TF1 (ear Shenmen), MA-SC (Kidney), MA-AT1 (Subcortex) and MA-TG (Adrenal gland) bilaterally. Indwelling fixed ‘New Pyonex’ needles embedded in a skin-coloured adhesive tape were used for AA. The participants were instructed by the acupuncturist to stimulate the auricular needles for 3–5 minutes, if they felt anxious. For the placebo procedure, ‘New Pyonex’ placebo needles were attached to five sites on the helix of the auricle bilaterally. ‘New Pyonex’ placebo needles have the same appearance as AA needles but consist of self-adhesive tape only. In order to avoid potential physiologic effects of acupressure, the participants were not instructed to stimulate the attached ‘New Pyonex’ placebo needles. AA and placebo needles were left in situ until the next day and were removed out of sight of the participants after the exam by the investigator, who was not involved in acupuncture procedure

Levels of anxiety were measured using a visual analogue scale before and after each intervention as well as before each exam. Additional measures included the State-Trait-Anxiety Inventory, duration of sleep at night, blood pressure, heart rate and the extent of participant blinding.

All included participants finished the study. Anxiety levels were reduced after AA and placebo intervention compared to baseline and the no intervention condition (p < 0.003). Moreover, AA was also better at reducing anxiety than placebo in the evening before the exam (p = 0.018). Participants were able to distinguish between AA and placebo intervention.

The authors concluded that both auricular acupuncture and placebo procedure were shown to be effective in reducing levels of exam anxiety in medical students. The superiority of verum AA over placebo AA and no intervention is considered to be due to stimulation of cranial nerves, but may have been increased in effect by insufficient participant blinding.

Here are just three of the major concerns I have about this study:

  • The trial design seems odd: a crossover study can only work well, if there is a stable baseline. This may not be the case with three consecutive exams; the anxiety experienced by students is bound to get less as time goes by. I think anyone who has passed a series of exams will confirm that there is a large degree of habituation.
  • It seems inadequate to employ just one acupuncturist; it means that the trial might end up testing not acupuncture per se but the skills of the therapist.
  • The placebo used for this study cannot possibly have fooled anyone into believing that it was real AA; volunteers were not even instructed to ‘stimulate’ the placebo devices. The difference to the ‘real thing’ must have been very clear to all involved. This means that the control for placebo-effects was woefully incomplete. In turn, this means that the observed outcomes are most likely due to residual bias.

In view of these concerns, allow me to re-phrase the authors’ conclusions:

THE RESULTS OF THIS POORLY-DESIGNED STUDY ARE DIFFICULT TO INTERPRET. MOST LIKELY THEY SHOW THAT ACUPUNCTURE IS NOT EFFECTIVE BUT MERELY WORKS THROUGH A PLACEBO-RESPONSE.

The ACUPUNCTURE NOW FOUNDATION (ANF) has featured on this blog before. Today I want to re-introduce them because I just came across one of their articles which I found remarkable. In it, they define what many of us have often wondered about: the most important myth about acupuncture.

Is it acupuncture’s current popularity, its long history, its mode of action, its efficacy, its safety?

No, here is the answer directly from the ANF:

The most important myth that needs to be put to rest is the idea promoted by a small group of vocal critics that acupuncture is nothing more than a placebo. Many cite the fact that studies showing acupuncture to be highly effective were of low quality and that several higher quality studies show that, while acupuncture was clinically effective, it usually does not outperform “sham” acupuncture. But those studies are dominated by the first quality issue cited above; studies with higher methodological rigor where the “real” acupuncture was so poorly done as to not be a legitimate comparison. Yet despite the tendency toward poor quality acupuncture in studies with higher methodological standards, a benchmark study was done that showed “real” acupuncture clearly outperforming “sham” acupuncture in four different chronic pain conditions.3 When you add this study together with the fact veterinary acupuncture is used successfully in many different animals, the idea of acupuncture only being placebo must now be considered finally disproven. This is further supported by studies which show that the underlying physiological pathways activated by acupuncture sometimes overlap, but can be clearly differentiated from, those activated by placebo responses.

Disappointed?

Yes, I was too.

The myth, according to the ANF, essentially is that sceptics do not understand the scientific evidence. And these blinkered sceptics even go as far as ignoring the findings from what the ANF consider to be a ‘benchmark study’! Ghosh, that’s nasty of them!!!

But, no – the benchmark study (actually, it was not a ‘study’ but a meta-analysis of studies) has been discussed fully on this blog (and in many other places too). Here is what I wrote in 2012 when it was first published:

An international team of acupuncture trialists published a meta-analysed of individual patient data to determine the analgesic effect of acupuncture compared to sham or non-acupuncture control for the following 4 chronic pain conditions: back and neck pain, osteoarthritis, headache, and shoulder pain. Data from 29 RCTs, with an impressive total of 17 922 patients, were included.

The results of this new evaluation suggest that acupuncture is superior to both sham and no-acupuncture controls for each of these conditions. Patients receiving acupuncture had less pain, with scores that were 0.23 (95% CI, 0.13-0.33), 0.16 (95% CI, 0.07-0.25), and 0.15 (95% CI, 0.07-0.24) SDs lower than those of sham controls for back and neck pain, osteoarthritis, and chronic headache, respectively; the effect sizes in comparison to no-acupuncture controls were 0.55 (95% CI, 0.51-0.58), 0.57 (95% CI, 0.50-0.64), and 0.42 (95% CI, 0.37-0.46) SDs.

Based on these findings, the authors reached the conclusion that “acupuncture is effective for the treatment of chronic pain and is therefore a reasonable referral option. Significant differences between true and sham acupuncture indicate that acupuncture is more than a placebo. However, these differences are relatively modest, suggesting that factors in addition to the specific effects of needling are important contributors to the therapeutic effects of acupuncture”.

… even the enthusiastic authors of this article admit that, when compared to sham, the effect size of real acupuncture is too small to be clinically relevant. Therefore one might argue that this meta-analysis confirms what critics have suggested all along: acupuncture is not a useful treatment for clinical routine.

Unsurprisingly, the authors of the meta-analysis do their very best to play down this aspect. They reason that, for clinical routine, the comparison between acupuncture and non-acupuncture controls is more relevant than the one between acupuncture and sham. But this comparison, of course, includes placebo- and other non-specific effects masquerading as effects of acupuncture – and with this little trick (which, by the way is very popular in alternative medicine), we can, of course, show that even sugar pills are effective.

I do not doubt that context effects are important in patient care; yet I do doubt that we need a placebo treatment for generating such benefit in our patients. If we administer treatments which are effective beyond placebo with kindness, time, compassion and empathy, our patients will benefit from both specific and non-specific effects. In other words, purely generating non-specific effects with acupuncture is far from optimal and certainly not in the interest of our patients. In my view, it cannot be regarded as not good medicine, and the authors’ conclusion referring to a “reasonable referral option” is more than a little surprising in my view.

Acupuncture-fans might argue that, at the very minimum, the new meta-analysis does demonstrate acupuncture to be statistically significantly better than a placebo. Yet I am not convinced that this notion holds water: the small residual effect-size in the comparison of acupuncture with sham might not be the result of a specific effect of acupuncture; it could be (and most likely is) due to residual bias in the analysed studies.

The meta-analysis is strongly driven by the large German trials which, for good reasons, were heavily and frequently criticised when first published. One of the most important potential drawbacks was that many participating patients were almost certainly de-blinded through the significant media coverage of the study while it was being conducted. Moreover, in none of these trials was the therapist blinded (the often-voiced notion that therapist-blinding is impossible is demonstrably false). Thus it is likely that patient-unblinding and the absence of therapist-blinding importantly influenced the clinical outcome of these trials thus generating false positive findings. As the German studies constitute by far the largest volume of patients in the meta-analysis, any of their flaws would strongly impact on the overall result of the meta-analysis.

So, has this new meta-analysis finally solved the decades-old question about the effectiveness of acupuncture? It might not have solved it, but we have certainly moved closer to a solution, particularly if we employ our faculties of critical thinking. In my view, this meta-analysis is the most compelling evidence yet to demonstrate the ineffectiveness of acupuncture for chronic pain.

END OF QUOTE

The ANF-text then goes from bad to worse. First they cite the evidence from veterinary acupuncture as further proof of the efficacy of their therapy. Well, the only systematic review in this are is, I think, by my team; and it concluded that there is no compelling evidence to recommend or reject acupuncture for any condition in domestic animals. Some encouraging data do exist that warrant further investigation in independent rigorous trials.

Lastly, the ANF mentions acupuncture’s mode of action which they seem to understand clearly and fully. Congratulations ANF! In this case, you are much better than the many experts in basic science or neurology who almost unanimously view these ‘explanations’ of how acupuncture might work as highly adventurous hypotheses or speculations.

So, what IS the most important myth about acupuncture? I am not sure and – unlike the ANF – I do not feel that I can speak for the rest of the world, but one of the biggest myths FOR ME is how acupuncture fans constantly manage to mislead the public.

Is acupuncture a pseudoscience? An interesting question! It was used as the title of a recent article. Knowing who authored it, the question unfortunately promised to be rhetorical. Dr Mike Cummings is (or was?) the ‘Medical Director at British Medical Acupuncture Society’ – hardly a source of critical or sceptical thinking about acupuncture, I’d say. The vast majority of his recent publications are in ‘ACUPUNCTURE IN MEDICINE’ and his blog post too is for that journal. Nevertheless, his thoughts might be worth considering, and therefore I present the essence of his post below [the footnotes refer to my comments following Cummings’ article]:

…Wikipedia has branded acupuncture as pseudoscience and its benefits as placebo [1]. ‘Acupuncture’ is clearly is not pseudoscience; however, the way in which it is used or portrayed by some may on occasion meet that definition. Acupuncture is a technique that predates the development of the scientific method [2] … so it is hardly fair to classify this ancient medical technique within that framework [3]. It would be better to use a less pejorative classification within the bracket of history when referring to acupuncture and other ancient East Asian medical techniques [4]. The contemporary use of acupuncture within modern healthcare is another matter entirely, and the fact that it can be associated with pre-scientific medicine does not make it a pseudoscience.

The Wikipedia acupuncture page is extensive and currently runs to 302 references. But how do we judge the quality or reliability of a text or its references? … I would generally look down on blogs, such as this, because they lack … hurdles prior to publication [5]. Open peer review was introduced relatively recently associated with immediate publication. But all this involves researchers and senior academics publishing and reviewing within their own fields of expertise. Wikipedia has a slightly different model built on five pillars. The second of those pillars reads:


Wikipedia is written from a neutral point of view: We strive for articles that document and explain major points of view, giving due weight with respect to their prominence in an impartial tone. We avoid advocacy and we characterize information and issues rather than debate them. In some areas there may be just one well-recognized point of view; in others, we describe multiple points of view, presenting each accurately and in context rather than as “the truth” or “the best view”. All articles must strive for verifiable accuracy, citing reliable, authoritative sources, especially when the topic is controversial or is on living persons. Editors’ personal experiences, interpretations, or opinions do not belong.


Experts within a field may be seen to have a certain POV (point of view), and are discouraged from editing pages directly because they cannot have the desired NPOV (neutral POV). This is a rather unique publication model in my experience, although the editing and comments are all visible and traceable, so there is no hiding… apart from the fact that editors are allowed to be entirely anonymous. Have a look at the talk page behind the main acupuncture page on Wikipedia. You may be shocked by the tone of much of the commentary. It certainly does not seem to comply with the fourth of the five pillars, which urges respect and civility, and in my opinion results primarily from the security of anonymity. I object to the latter, but there is always a balance to be found between freedom of expression (enhanced for some by the safety of anonymity) and cyber bullying (almost certainly fuelled in part by anonymity). That balance requires good moderation, and whilst there was some evidence of moderation on the talk page, it was inadequate to my mind… I might move to drop anonymity from Wikipedia if moderation is wanting.

Anyway my impression, for what it’s worth, is that the acupuncture page on Wikipedia is not written from an NPOV, but rather it appears to be controlled by semi professional anti-CAM pseudosceptics [6]. I have come across these characters [6] regularly since I was introduced to the value of needling in military general practice. I have a stereotypical mental image: plain or scary looking bespectacled geeks and science nuts [6], the worst are often particle physicists … Interacting with them is at first intense, but rapidly becomes tedious as they know little of the subject detail [6], fall back on the same rather simplistic arguments [6] and ultimately appear to be motivated by eristic discourse rather than the truth [6].

I am not surprised that they prefer to close the comments, because I imagine that some people might object rather strongly to many of the statements made in this text.

Here are my short comments:

[1] I should perhaps stress that I am not the author of nor a contributor to this Wiki (or any other) page.

[2] Is this an attempt to employ the ‘appeal to tradition’ fallacy?

[3] The Wiki page does by no means classify the ancient history of acupuncture as pseudoscience.

[4] I have always felt that classification of science or medicine according to geography is nonsensical; they should not be classified as Western or Asian but as sound or not, effective or not, etc.

[5] As we have often seen on this blog, the ‘hurdles’ (peer-review) are often laughable, particularly in the realm of alternative medicine.

[6] This article is essentially trying to show that the Wiki page is biased. Yet it ends with a bonanza of insults which essentially reveal the profound bias of the author.

IS ACUPUNCTURE PSEUDOSCIENCE? Cummings’ article promised to address this question. Sadly it did nothing of the sort. It  turned out to be an incompetent rant about a Wiki page. If anything, Cummings contributed to the neutral reader of his text getting convinced that, indeed, acupuncture IS a pseudoscience! At least Wiki used facts, arguments, evidence etc. and it went a lot further in finding a rational answer to this intriguing question.

The boom of alternative medicine in the US – and consequently in the rest of the developed world – is intimately connected with a NHI centre now called NCCIH (National Center for Complementary and Integrative Health). It was founded in the early 1990s because some politicians were bent on promoting quackery. Initially the institution had modest funding but, after more political interference, it had ample cash to pursue all sorts of activities, including sponsoring research into alternative therapies at US universities. A most interesting video summarising the history of the NCCIH can be seen here.

No other institution in the world had more funds for research into alternative medicine than the NCCIH, and it soon became the envy of alt med researchers globally. I have been invited by the NCCHI on several occasions and invariably was impressed by their apparent affluence. While we Europeans usually had to do our research on a shoe-string, our American colleagues seemed to be ‘rolling in it’.

I was often far less impressed with the research they sponsored. Not only it was invariably eye-wateringly expensive, but also its quality seemed often dismal. Sometimes, I even got the impression that research was used as a means of mainstreaming quackery for the unsuspecting American – and consequently world-wide – public.

An example of this mainstreaming is an article in JAMA published yesterday. Here is a short but telling excerpt:

Researchers led by Richard L. Nahin, PhD, MPH, lead epidemiologist at the NIH’s National Center for Complementary and Integrative Health (NCCIH), examined efficacy and safety evidence in 105 randomized controlled trials (RCTs) conducted between January 1966 and March 2016. The review—geared toward primary care physicians as part of the journal’s Symposium on Pain Medicine—focused on popular complementary approaches to common pain conditions.

Unlike a typical systematic review that assigns quality values to the studies, the investigators conducted a narrative review, in which they simply looked at the number of positive and negative trials. “If there were more positives than negatives then we generally felt the approach had some value,” Nahin explained. “If there were more negatives, we generally felt the approach had less value.” Trials that were conducted outside of the United States were excluded from the review.

Based on a “preponderance” of positive vs negative trials, complementary approaches that may offer pain relief include acupuncture and yoga for back pain; acupuncture and tai chi for osteoarthritis of the knee; massage therapy for neck pain; and relaxation techniques for severe headaches and migraine. Several other techniques had weaker evidence, according to the qualitative assessments, for specific pain conditions (see “Selected Complementary Health Approaches for Pain”). The treatments were generally safe, with no serious adverse events reported.

To me, this looks that NCCIH has now managed to persuade even the editors of JAMA to white-wash their dodgy science. The review referred to here is a paper we discussed some time ago on this blog. I then stated about it the following:

Reading the article carefully, it is impossible not to get troubled. Here are a few points that concern me most:

  • the safety of a therapy cannot be evaluated on the basis of data from RCTs (particularly as it has been shown repeatedly that trials of alternative therapies often fail to report adverse effects); much larger samples are needed for that; any statements about safety in the aims of the paper are therefore misplaced;
  • the authors talk about efficacy but seem to mean effectiveness;
  • the authors only included RCTs from the US which must result in a skewed and incomplete picture;
  • the article is from the National Center for Complementary and Integrative Health which is part of the NIH but which has been criticised repeatedly for being biased in favour of alternative medicine;
  • not all of the authors seem to be NIH staff, and I cannot find a declaration of conflicts of interest;
  • the discussion of the paper totally lacks any critical thinking;
  • there is no assessment of the quality of the trials included in this review.

My last point is by far the most important. A summary of this nature that fails to take into account the numerous limitations of the primary data is, I think, as good as worthless. As I know most of the RCTs included in the analyses, I predict that the overall picture generated by this review would have changed substantially, if the risks of bias in the primary studies had been accounted for.

I find it puzzling that the ‘lead epidemiologist at the NIH’s National Center for Complementary and Integrative Health’ would publish such dubious research. Why does he do it? If you have watched the video mentioned above, you are inclined to think that it might be because of political interference.

However, I suggest another, in a way much more damming reason or contributing factor: the NCCIH has so long indulged in such poor science that even its top people have forgotten what good science looks like. I know this is a bold hypothesis; so, let me try to support it with some data.

Several years ago, my team together with several other researches have looked at the NCCIH-sponsored research systematically according to 4 different subject areas. Here are the conclusions of our articles reporting the findings:

ACUPUNCTURE

Seven RCTs had a low risk of bias. Numerous methodological shortcomings were identified. Many NCCAM-funded RCTs of acupuncture have important limitations. These findings might improve future studies of acupuncture and could be considered in the ongoing debate regarding NCCAM-funding. [Focus on Alternative and Complementary Therapies Volume 17(1) March 2012 15–21]

HERBAL MEDICINE

This independent assessment revealed a plethora of serious concerns related to NCCAM studies of herbal medicine. [Perfusion 2011; 24: 89-102]

ENERGY MEDICINE

In conclusion, the NCCAM-funded RCTs of energy medicine are prime examples of misguided investments into research. In our opinion, NCCAM should not be funding poor-quality studies of implausible practices. The impact of any future studies of energy medicine would be negligible or even detrimental. [Focus on Alternative and Complementary Therapies Volume 16(2) June 2011 106–109 ]

CHIROPRACTIC

In conclusion, our review demonstrates that several RCTs of chiropractic have been funded by the NCCAM. It raises numerous concerns in relation to these studies; in particular, it suggests that many of these studies are seriously flawed. [https://www.ncbi.nlm.nih.gov/pubmed/21207089]

I think I can rest my case and urge you to watch the video mentioned above.

On 25 and 26 May of this year I wrote two posts about an acupuncture trial that, in my view, was dodgy. To refresh your memory, here is the relevant part of the 2nd post:

This new study was designed as a randomized, sham-controlled trial of acupuncture for persistent allergic rhinitis in adults investigated possible modulation of mucosal immune responses. A total of 151 individuals were randomized into real and sham acupuncture groups (who received twice-weekly treatments for 8 weeks) and a no acupuncture group. Various cytokines, neurotrophins, proinflammatory neuropeptides, and immunoglobulins were measured in saliva or plasma from baseline to 4-week follow-up.

Statistically significant reduction in allergen specific IgE for house dust mite was seen only in the real acupuncture group. A mean (SE) statistically significant down-regulation was also seen in pro-inflammatory neuropeptide substance P (SP) 18 to 24 hours after the first treatment. No significant changes were seen in the other neuropeptides, neurotrophins, or cytokines tested. Nasal obstruction, nasal itch, sneezing, runny nose, eye itch, and unrefreshed sleep improved significantly in the real acupuncture group (post-nasal drip and sinus pain did not) and continued to improve up to 4-week follow-up.

The authors concluded that acupuncture modulated mucosal immune response in the upper airway in adults with persistent allergic rhinitis. This modulation appears to be associated with down-regulation of allergen specific IgE for house dust mite, which this study is the first to report. Improvements in nasal itch, eye itch, and sneezing after acupuncture are suggestive of down-regulation of transient receptor potential vanilloid 1.

…the trial itself raises a number of questions:

  1. Which was the primary outcome measure of this trial?
  2. What was the power of the study, and how was it calculated?
  3. For which outcome measures was the power calculated?
  4. How were the subjective endpoints quantified?
  5. Were validated instruments used for the subjective endpoints?
  6. What type of sham was used?
  7. Are the reported results the findings of comparisons between verum and sham, or verum and no acupuncture, or intra-group changes in the verum group?
  8. What other treatments did each group of patients receive?
  9. Does anyone really think that this trial shows that “acupuncture is a safe, effective and cost-effective treatment for allergic rhinitis”?

In the comments section, the author wrote: “after you have read the full text and answered most of your questions for yourself, it might then be a more appropriate time to engage in any meaningful discussion, if that is in fact your intent”, and I asked him to send me his paper. As he does not seem to have the intention to do so, I will answer the questions myself and encourage everyone to have a close look at the full paper [which I can supply on request].

  1. The myriad of lab tests were defined as primary outcome measures.
  2. Two sentences are offered, but they do not allow me to reconstruct how this was done.
  3. No details are provided.
  4. Most were quantified with a 3 point scale.
  5. Mostly not.
  6. Needle insertion at non-acupoints.
  7. The results are a mixture of inter- and intra-group differences.
  8. Patients were allowed to use conventional treatments and the frequency of this use was reported in patient diaries.
  9. I don’t think so.

So, here is my interpretation of this study:

  • It lacked power for many outcome measures, certainly the clinical ones.
  • There were hardly any differences between the real and the sham acupuncture group.
  • Most of the relevant results were based on intra-group changes, rather than comparing sham with real acupuncture, a fact, which is obfuscated in the abstract.
  • In a controlled trial fluctuations within one group must never be interpreted as caused by the treatment.
  • There were dozens of tests for statistical significance, and there seems to be no correction for multiple testing.
  • Thus the few significant results that emerged when comparing sham with real acupuncture might easily be false positives.
  • Patient-blinding seems questionable.
  • McDonald as the only therapist of the study might be suspected to have influenced his patients through verbal and non-verbal communications.

I am sure there are many more flaws, particularly in the stats, and I leave it to others to identify them. The ones I found are, however, already serious enough, in my view, to call for a withdrawal of this paper. Essentially, the authors seem to have presented a study with largely negative findings as a trial with positive results showing that acupuncture is an effective therapy for allergic rhinitis. Subsequently, McDonald went on social media to inflate his findings even more. One might easily ask: is this scientific misconduct or just poor science?

END OF QUOTE

This and the previous post created lots of discussion and comments. However, the question whether the study in question amounted to scientific misconduct was never satisfactorily resolved. Therefore, I decided to write to the editor of ‘Ann Allergy Asthma Immunol‘ where the trial had been published. He answered by saying I would need to file an official complaint for him to address the issue. On 13 June, I therefore sent him the following email:

Thank you for your letter of 3/6/2016 suggesting I make a formal complaint about the paper entitled ‘EFFECT OF ACUPUNCTURE ON HOUSE DUST MITE…’ [ Ann Allergy Asthma Immunol 2016] by McDonald et al. I herewith wish to file such a complaint.

The article in question reports an RCT of acupuncture for persistent allergic rhinitis. It followed a parallel group design with 3 groups receiving the following interventions:

1.       Acupuncture

2.       Sham-acupuncture

3.       No treatment

There was a plethora of outcome measures and time points on which they were measured. A broad range of parameters was defined as primary endpoints.

The conclusion reached by the authors essentially was that acupuncture affected several outcome measures in a positive sense, thus supporting the notion that acupuncture is efficacious [“Symptoms and quality of life improved significantly and were still continuing to improve 4 weeks after treatment ceased.”] This conclusion, however, is misleading and needs correcting.

The main reasons for this are as follows:

·         Despite the fact that the authors did many dozens of statistical tests for significance, they did not correct for this multiplicity of tests. Consequently, some or most of the significant results are likely to be false positive.

·         Many of the positive results of this paper were not obtained by comparing one group to another but by doing before/after comparisons within one group. This approach defies the principle of a controlled clinical trial. For doing intra-group comparisons, we obviously do not need any control group at all. The findings from intra-group comparisons are prominently reported in the paper, for instance in the abstract, giving the impression that they originate from inter-group comparisons. One has to read the paper very carefully to find that, when inter-group comparisons were conducted, their results did NOT confirm the findings from the reported intra-group comparisons. As this is the case for most of the symptomatic endpoints, the impression given is seriously misleading and needs urgent correction.

On the whole, the article is a masterpiece of obfuscation and misrepresentation of the actual data. I urge you to consider the harm than can be done by such a misleading publication. In my view, the best way to address this problem is to withdraw the article.

I look forward to your decision.

Regards

E Ernst

END OF QUOTE

I had to send several reminders but my most recent one prompted the following response dated 7/11/ 2016:

Dear Professor Ernst,
Thank you for your patience while we worked through the process of considering your complaints levied on the article entitled ‘EFFECT OF ACUPUNCTURE ON HOUSE DUST MITE…’ [ Ann Allergy Asthma Immunol 2016] by McDonald et al. I considered the points that you made in your previous letter, sought input from our editorial team (including our biostatistics editor) , our publisher and the authors themselves. I sent the the  charges ( point by point) anonymously to the authors and allowed them time to respond which they did. I had their responses reviewed by selected editors and , as a result of this process, have decided not to pursue any corrective or punitive action based upon the following :
 
  1. Our editorial team recognizes that this is not the best clinical trial we have published in the Annals of Allergy, Asthma and Immunology. However, neither is is the worst. As in most published research studies, there are always things that could have been done better to make it a stronger paper. Never-the-less, the criticism falls fall short of any sort of remedy that would include withdrawal of the manuscript.
  2. Regarding your accusation that the multiple positive endpoint resulted in the authors making specific therapeutic claims, our assessment is that no specific therapeutic claim was made but rather the authors maintained that the data support the value of acupuncture in improving symptoms and quality of life in patients with AR. We do not believe there was overreach in those statements.
  3. The authors’ stated intent was to show immune changes associated with clinical markers of improvement in the active acupuncture group compared to controls. The authors maintain (and our editors agree) that their data assessments were primarily based upon three statistical tests not “dozens” (as stated in your original letter of complaint).  The power analysis and sample size calculations were presented to us and deemed adequate , making the probability of a type I error quite low.
  4. The authors acknowledge in their paper that there could be limitations to their data interpretation based upon potential disparities between intra- and intergroup comparisons. The editors felt their transparency was adequately disclosed.
In summary, as editor-in-chief of the Annals of Allergy, Asthma and Immunology, I did not find sufficient merit in your charges to initiate any corrective or punitive action for this manuscript. I understand you will strongly disagree with this decision and I regret that. However, in the final analysis, my primary intent is to preserve the objectivity, fairness, and integrity of our journal and its review process. I believe I have accomplished that in this instance.
 
Sincerely
END OF QUOTE
This seems to settle the issue: the study in question does not involve scientific misconduct!
Or does it?
I would be grateful for the view of the readers of this post.

I have warned you before to be sceptical about Chinese studies. This is what I posted on this blog more than 2 years ago, for instance:

Imagine an area of therapeutics where 100% of all findings of hypothesis-testing research are positive, i.e. come to the conclusion that the treatment in question is effective. Theoretically, this could mean that the therapy is a miracle cure which is useful for every single condition in every single setting. But sadly, there are no miracle cures. Therefore something must be badly and worryingly amiss with the research in an area that generates 100% positive results.

Acupuncture is such an area; we and others have shown that Chinese trials of acupuncture hardly ever produce a negative finding. In other words, one does not need to read the paper, one already knows that it is positive – even more extreme: one does not need to conduct the study, one already knows the result before the research has started. But you might not believe my research nor that of others. We might be chauvinist bastards who want to discredit Chinese science. In this case, you might perhaps believe Chinese researchers.

In this systematic review, all randomized controlled trials (RCTs) of acupuncture published in Chinese journals were identified by a team of Chinese scientists. A total of 840 RCTs were found, including 727 RCTs comparing acupuncture with conventional treatment, 51 RCTs with no treatment controls, and 62 RCTs with sham-acupuncture controls. Among theses 840 RCTs, 838 studies (99.8%) reported positive results from primary outcomes and two trials (0.2%) reported negative results. The percentages of RCTs concealment of the information on withdraws or sample size calculations were 43.7%, 5.9%, 4.9%, 9.9%, and 1.7% respectively.

The authors concluded that publication bias might be major issue in RCTs on acupuncture published in Chinese journals reported, which is related to high risk of bias. We suggest that all trials should be prospectively registered in international trial registry in future.

END OF QUOTE

Now an even more compelling reason emerged for taking evidence from China with a pinch of salt:

A recent survey of clinical trials in China has revealed fraudulent practice on a massive scale. China’s food and drug regulator carried out a one-year review of clinical trials. They concluded that more than 80 percent of clinical data is “fabricated“. The review evaluated data from 1,622 clinical trial programs of new pharmaceutical drugs awaiting regulator approval for mass production. Officials are now warning that further evidence malpractice could still emerge in the scandal.
According to the report, much of the data gathered in clinical trials are incomplete, failed to meet analysis requirements or were untraceable. Some companies were suspected of deliberately hiding or deleting records of adverse effects, and tampering with data that did not meet expectations.

“Clinical data fabrication was an open secret even before the inspection,” the paper quoted an unnamed hospital chief as saying. Contract research organizations seem have become “accomplices in data fabrication due to cutthroat competition and economic motivation.”

A doctor at a top hospital in the northern city of Xian said the problem doesn’t lie with insufficient regulations governing clinical trials data, but with the failure to implement them. “There are national standards for clinical trials in the development of Western pharmaceuticals,” he said. “Clinical trials must be carried out in three phases, and they must be assessed at the very least for safety,” he said. “But I don’t know what happened here.”

Public safety problems in China aren’t limited to the pharmaceutical industry and the figure of 80 percent is unlikely to surprise many in a country where citizens routinely engage in the bulk-buying of overseas-made goods like infant formula powder. Guangdong-based rights activist Mai Ke said there is an all-pervasive culture of fakery across all products made in the country. “It’s not just the medicines,” Mai said. “In China, everything is fake, and if there’s a profit in pharmaceuticals, then someone’s going to fake them too.” He said the problem also extends to traditional Chinese medicines, which are widely used in conjunction with Western pharmaceuticals across the healthcare system.
“It’s just harder to regulate the fakes with traditional medicines than it is with Western pharmaceuticals, which have strict manufacturing guidelines,” he said.

According to Luo, academic ethics is an underdeveloped field in China, leading to an academic culture that is accepting of manipulation of data. “I don’t think that the 80 percent figure is overstated,” Luo said.

And what should we conclude from all this?

I find it very difficult to reach a verdict that does not sound hopelessly chauvinistic but feel that we have little choice but to distrust the evidence that originates from China. At the very minimum, I think, we must scrutinise it thoroughly; whenever it looks too good to be true, we ought to discard it as unreliable and await independent replications.

A website I recently came across promised to teach me 7 things about acupuncture. This sort of thing is always of interest to me; so I read them with interest and found them so remarkable that I decided to reproduce them here:

1. Addiction recovery

Acupuncture calms and relaxes the mind making it easier for people to overcome addictions to drugs, cigarettes, and alcohol by reducing the anxiety and stress they feel when quitting.

2. Helps the body heal itself

The body contains natural pain relief chemicals, such as endorphins and has an amazing capacity for self-healing. Acupuncture helps stimulate the natural healing mechanisms and causes the body to manufacture pain relieving chemicals.

3. Builds a stronger immune system

The body’s immune system is negatively affected by stress, poor diet, illness and certain medical treatments, but acupuncture targets the underlying imbalances naturally and helps it to regain balance.

4. Eliminate that killer hangover

While it may not have been the best choice to finish off that bottle of wine, acupuncture can help the body detox and flush out the morning side effects.

5. Mood stabilizer

If you find yourself snapping at friends, family, or co-workers for unexplained reasons, acupuncture can get to the root of the problem, find the imbalance and help your body return to a healthier state of mind.

6. Chronic stomach problems

Some people suffer from stomach problems and never find the cause. Acupuncture targets your whole body, including the digestive tract and helps it to work in harmony with the rest of the body’s systems.

7. Coping with death

Grief can have an overwhelming effect on the body and manifest itself physically. Acupuncture helps reduce the anxiety of dealing with loss and help you cope with the stress.

END OF QUOTE

The ‘7 things’ are remarkably mislabelled – they should be called 7 lies! Let me explain:

  1. There are several Cochrane reviews on the subject of acupuncture for various addictions. Here are their conclusions: There is currently no evidence that auricular acupuncture is effective for the treatment of cocaine dependence. The evidence is not of high quality and is inconclusive. Further randomised trials of auricular acupuncture may be justified. There is no clear evidence that acupuncture is effective for smoking cessation.  There is currently no evidence that auricular acupuncture is effective for the treatment of cocaine dependence. The evidence is not of high quality and is inconclusive. Further randomised trials of auricular acupuncture may be justified.
  2. Even if the ‘endorphin story’ is true (in my view, it’s but a theory), there is no good evidence that acupuncture enhances our body’s self-healing mechanisms via endorphins or any other mechanism.
  3. Stronger immune system? My foot! I have no idea where this claim comes from, certainly not from anything resembling good evidence.
  4. Acupuncture for hangover or detox? This is just a stupid joke with no evidential support. I imagine, however, that it is superb marketing.
  5. The same applies to acupuncture to ‘stabilize’ your mood.
  6. Unexplained stomach problems? Go and see a doctor! Here is the conclusion of a Cochrane review related to IBS which is one of the more common unexplained stomach complaint: Sham-controlled RCTs have found no benefits of acupuncture relative to a credible sham acupuncture control for IBS symptom severity or IBS-related quality of life.
  7. I am not aware of any good evidence to show that acupuncture could ease the grieving process; I even doubt that this would be such a good or desirable thing: grieving is a necessary and essential process.

So, what we have here are essentially 7 fat lies. Yes, I know, the literature and the internet are full of them. And I suspect that they are a prominent reason why acupuncture is fairly popular today. Lies are a major marketing tool of acupuncturists – but that does not mean that we should let them get away with them!

Why?

Bogus claims may be good for the cash flow of alternative practitioners, but they are certainly not good for our health and well-being; in fact, they can cost lives!!!

IN THIS SPIRIT, LET ME ADD SEVEN THINGS YOU DO NEED TO KNOW ABOUT ACUPUNCTURE

  1. Traditional acupuncture is based on complete hocus pocus and is therefore implausible.
  2. ‘Western’ acupuncture is based on endorphin and other theories, which are little more than that and at best THEORIES.
  3. Acupuncture is often promoted as a ‘cure all’ which is implausible and not supported by evidence.
  4. Meridians, acupoints chi and all the other things acupuncturists claim to exist are pure fantasy.
  5. For a small list of symptoms, acupuncture is backed up by some evidence, but this is less than convincing and could well turn out to rely on little more than placebo.
  6. The claim of acupuncturists that acupuncture is entirely safe is false.
  7. Acupuncture studies from China cannot be trusted.

In alternative medicine, good evidence is like gold dust and good evidence showing that alternative therapies are efficacious is even rarer. Therefore, I was delighted to come across a brand-new article from an institution that should stand for reliable information: the NIH, no less.

According to its authors, this new article “examines the clinical trial evidence for the efficacy and safety of several specific approaches—acupuncture, manipulation, massage therapy, relaxation techniques including meditation, selected natural product supplements (chondroitin, glucosamine, methylsulfonylmethane, S-adenosylmethionine), tai chi, and yoga—as used to manage chronic pain and related disability associated with back pain, fibromyalgia, osteoarthritis, neck pain, and severe headaches or migraines.”

The results of this huge undertaking are complex, of course, but in a nutshell they are at least partly positive for alternative medicine. Specifically, the authors state that “based on a preponderance of positive trials vs negative trials, current evidence suggests that the following complementary approaches may help some patients manage their painful health conditions: acupuncture and yoga for back pain; acupuncture and tai chi for OA of the knee; massage therapy for neck pain with adequate doses and for short-term benefit; and relaxation techniques for severe headaches and migraine. Weaker evidence suggests that massage therapy, SM, and osteopathic manipulation might also be of some benefit to those with back pain, and relaxation approaches and tai chi might help those with fibromyalgia.”

This is excellent news! Finally, we have data from an authoritative source showing that some alternative treatments can be recommended for common pain conditions.

Hold on, not so fast! Yes, the NIH is a most respectable organisation, but we must not blindly accept anything of importance just because it appears to come form a reputable source. Let’s look a bit closer at the actual evidence provided by the authors of this paper.

Reading the article carefully, it is impossible not to get troubled. Here are a few points that concern me most:

  • the safety of a therapy cannot be evaluated on the basis of data from RCTs (particularly as it has been shown repeatedly that trials of alternative therapies often fail to report adverse effects); much larger samples are needed for that; any statements about safety in the aims of the paper are therefore misplaced;
  • the authors talk about efficacy but seem to mean effectiveness;
  • the authors only included RCTs from the US which must result in a skewed and incomplete picture;
  • the article is from the National Center for Complementary and Integrative Health which is part of the NIH but which has been criticised repeatedly for being biased in favour of alternative medicine;
  • not all of the authors seem to be NIH staff, and I cannot find a declaration of conflicts of interest;
  • the discussion of the paper totally lacks any critical thinking;
  • there is no assessment of the quality of the trials included in this review.

My last point is by far the most important. A summary of this nature that fails to take into account the numerous limitations of the primary data is, I think, as good as worthless. As I know most of the RCTs included in the analyses, I predict that the overall picture generated by this review would have changed substantially, if the risks of bias in the primary studies had been accounted for.

Personally, I find it lamentable that such a potentially worthy exercise ended up employing such lousy methodology. Perhaps even more lamentable is the fact that the NIH (or one of its Centers) can descend that low; to mislead the public in this way borders on scientific misconduct and is, in my view, unethical and unacceptable.

The aim of a new meta-analysis was to estimate the clinical effectiveness and safety of acupuncture for amnestic mild cognitive impairment (AMCI), the transitional stage between the normal memory loss of aging and dementia. Randomised controlled trials (RCTs) of acupuncture versus medical treatment for AMCI were identified using six electronic databases.

Five RCTs involving a total of 568 subjects were included. The methodological quality of the RCTs was generally poor. Participants receiving acupuncture had better outcomes than those receiving nimodipine with greater clinical efficacy rates (odds ratio (OR) 1.78, 95% CI 1.19 to 2.65; p<0.01), mini-mental state examination (MMSE) scores (mean difference (MD) 0.99, 95% CI 0.71 to 1.28; p<0.01), and picture recognition score (MD 2.12, 95% CI 1.48 to 2.75; p<0.01). Acupuncture used in conjunction with nimodipine significantly improved MMSE scores (MD 1.09, 95% CI 0.29 to 1.89; p<0.01) compared to nimodipine alone. Three trials reported adverse events.

The authors concluded that acupuncture appears effective for AMCI when used as an alternative or adjunctive treatment; however, caution must be exercised given the low methodological quality of included trials. Further, more rigorously designed studies are needed.

Meta-analyses like this one are, in my view, perfect examples for the ‘rubbish in, rubbish out’ principle of systematic reviews. This may seem like an unfair statement, so let me justify it by explaining the shortfalls of this specific paper.

The authors try to tell us that their aim was “to estimate the clinical effectiveness and safety of acupuncture…” While it might be possible to estimate the effectiveness of a therapy by pooling the data of a few RCTs, it is never possible to estimate its safety on such a basis. To conduct an assessment of therapeutic safety, one would need sample sizes that go two or three dimensions beyond those of RCTs. Thus safety assessments are best done by evaluating the evidence from all the available evidence, including case-reports, epidemiological investigations and observational studies.

The authors tell us that “two studies did not report whether any adverse events or side effects had occurred in the experimental or control groups.” This is a common and serious flaw of many acupuncture trials, and another important reason why RCTs cannot be used for evaluating the risks of acupuncture. Too many such studies simply don’t mention adverse effects at all. If they are then submitted to systematic reviews, they must generate a false positive picture about the safety of acupuncture. The absence of adverse effects reporting is a serious breach of research ethics. In the realm of acupuncture, it is so common, that many reviewers do not even bother to discuss this violation of medical ethics as a major issue.

The authors conclude that acupuncture is more effective than nimodipine. This sounds impressive – unless you happen to know that nimodipine is not supported by good evidence either. A Cochrane review provided no convincing evidence that nimodipine is a useful treatment for the symptoms of dementia, either unclassified or according to the major subtypes – Alzheimer’s disease, vascular, or mixed Alzheimer’s and vascular dementia.

The authors also conclude that acupuncture used in conjunction with nimodipine is better than nimodipine alone. This too might sound impressive – unless you realise that all the RCTs in question failed to control for the effects of placebo and the added attention given to the patients. This means that the findings reported here are consistent with acupuncture itself being totally devoid of therapeutic effects.

The authors are quite open about the paucity of RCTs and their mostly dismal methodological quality. Yet they arrive at fairly definitive conclusions regarding the therapeutic value of acupuncture. This is, in my view, a serious mistake: on the basis of a few poorly designed and poorly reported RCTs, one should never arrive at even tentatively positive conclusion. Any decent journal would not have published such misleading phraseology, and it is noteworthy that the paper in question appeared in a journal that has a long history of being hopelessly biased in favour of acupuncture.

Any of the above-mentioned flaws could already be fatal, but I have kept the most serious one for last. All the 5 RCTs that were included in the analyses were conducted in China by Chinese researchers and published in Chinese journals. It has been shown repeatedly that such studies hardly ever report anything other than positive results; no matter what conditions is being investigated, acupuncture turns out to be effective in the hands of Chinese trialists. This means that the result of such a study is clear even before the first patient has been recruited. Little wonder then that virtually all reviews of such trials – and there are dozens of then – arrive at conclusions similar to those formulated in the paper before us.

As I already said: rubbish in, rubbish out!

Acupuncture Today is a much-read online publication for people interested in acupuncture. It informs us that Chinese medicine is quite complex and can be difficult for some people to comprehend. This is because TCM is based, at least in part, on the Daoist belief that we live in a universe in which everything is interconnected. What happens to one part of the body affects every other part of the body. The mind and body are not viewed separately, but as part of an energetic system. Similarly, organs and organ systems are viewed as interconnected structures that work together to keep the body functioning.

To me, this sounds suspiciously woolly. Do they think that conventional healthcare professionals view the various body-parts as separate entities? Do they feel that conventional practitioners see the mind entirely separate from the body? Do they believe others fail to realize that what affects the brain does not affect the rest of the body? These common preconceptions have always puzzled me. Intrigued, I read on.

Elsewhere we learn that Acupuncture Today and acupuncturetoday.com are the only complete news sources in the profession and we don’t take this honor lightly. The acupuncture and Oriental medicine profession is a blend of ancient traditions, healing styles and modern therapies. We provide content that is comprehensive enough to appeal to each of the profession’s diverse groups. In addition, we provide a complete suite of additional products including newsletters, calendars and classifieds that provide our advertisers with the contextual platform they need to communicate with our readers, their customers.

Acupuncture Today seems to reflect a lot of what many acupuncturists want to hear – and thus it might provide us with an important insight into the mind-set of acupuncturists. On their website, I found an article which fascinated me:

START OF QUOTE

A more efficient method for diagnosis and treatment by remote medical dowsing has been found and used in acupuncture with great success. The procedure involves a pendulum, a picture of the patient, an anatomy book, a steel pointer, and a very thin bamboo pointer.

Being a dentist, orthodontist, acupuncturist and dowser, I like to take the liberty of treating a person affected with lockjaw or temporal-mandibular joint ailments via remote dowsing…

…When the mandible cannot open due to a spasm, the chief symptom is pain. Until energy is restored, the muscle cannot lengthen and pain cannot be eliminated. Acupuncture is a good way to correct this condition without the use of a dental appliance. Dentists specializing in treating TMJ use a computerized equipment scan (electrosonography), surface electromyography and the myomonitor to relax the muscles.

Another procedure to treat TMJ is using dowsing. At this point, I will talk about dowsing procedures and information needed to successfully carry out the procedures. Remote dowsing requires the use of the pendulum, a slender bamboo pointer, an anatomy book, a picture of the patient and a steel pointer.

To treat a TMJ patient, the picture of the patient is dowsed holding a pendulum in the right hand while the left hand uses a bamboo pointer to touch the closing and opening muscles individually in the anatomy book. The closing muscles will have good energy (as evidenced by the circular movement of the pendulum) while the lower head of the lateral pterygoid will have no energy (as evidenced by little or no movement of the pendulum). Having advance information on TMJ acupuncture points helps, but these points will have to be tested if needling will supply energy. Master Tong has suggested a point between Liver 2 and Liver 3. I find Spleen 2, a distal point related to the lower head of the lateral pterygoid, to be more effective. This can be checked by having the patient hold the point of the steel pointer so it touches Spleen 2 on the large toe.

To treat a TMJ patient, the picture of the patient is dowsed holding a pendulum in the right hand while the left hand uses a bamboo pointer to touch the closing and opening muscles individually in the anatomy book. The closing muscles will have good energy (as evidenced by the circular movement of the pendulum) while the lower head of the lateral pterygoid will have no energy (as evidenced by little or no movement of the pendulum). Having advance information on TMJ acupuncture points helps, but these points will have to be tested if needling will supply energy. Master Tong has suggested a point between Liver 2 and Liver 3. I find Spleen 2, a distal point related to the lower head of the lateral pterygoid, to be more effective. This can be checked by having the patient hold the point of the steel pointer so it touches Spleen 2 on the large toe.

By dowsing the picture of the patient with the right hand and using a bamboo pointer to touch the lower head of the pterygoid muscle in the anatomy book with the left hand, it will be evident by the circular movement of the pendulum that these muscles now have good energy. This is done before the needle is inserted. In this manner all points can be checked for ailments such as TMJ, stroke, backaches, and neck and shoulder problems before needling. When the needles are placed and after the needling procedure, energy can be checked using the pendulum. By being very accurate on the location of acupuncture points, less treatments will be needed to obtain results. Another point is Small Intestine 19, a local point which is also very effective. Good results are obtained by careful and accurate needling. Therefore, the number of visits are few…

Dowsing is a diagnostic aid that has been used for other situations and can be very helpful to acupuncturists. In conclusion, I feel that remote dowsing is a great approach to diagnosis and treatment.

END OF QUOTE

If I had not seen alternative practitioners doing this procedure with my own eyes, I might have thought the article is a hoax. Sadly, this is the ‘real world’ of alternative medicine.

I tried to find some acupuncturists who had objected to this intense nonsense, but I was not successful in this endeavour. The article was published 6 years ago (no, not on 1 April!), yet so far, nobody has objected.

I have also tried to see whether articles promoting quackery of this nature are rare exceptions in the realm of acupuncture, or whether they are regular occurrences. My impression is that the latter is the case.

What can be concluded from all this?

In a previous post about quackery in chiropractic, I have argued that the tolerance of quackery must be one of the most important hallmarks of a quack profession. As I still believe this to be true, I have to ask to which extend THE TOLERANCE OF SUCH EXTREME QUACKERY MAKES ACUPUNCTURISTS QUACKS?

[I would be most interested to have my readers’ views on this question]

Gravityscan Badge

Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted.


Click here for a comprehensive list of recent comments.

Categories