Yesterday, I wrote about a new acupuncture trial. Amongst other things, I wanted to find out whether the author who had previously insisted I answer his questions about my view on the new NICE guideline would himself answer a few questions when asked politely. To remind you, this is what I wrote:
This new study was designed as a randomized, sham-controlled trial of acupuncture for persistent allergic rhinitis in adults investigated possible modulation of mucosal immune responses. A total of 151 individuals were randomized into real and sham acupuncture groups (who received twice-weekly treatments for 8 weeks) and a no acupuncture group. Various cytokines, neurotrophins, proinflammatory neuropeptides, and immunoglobulins were measured in saliva or plasma from baseline to 4-week follow-up.
Statistically significant reduction in allergen specific IgE for house dust mite was seen only in the real acupuncture group. A mean (SE) statistically significant down-regulation was also seen in pro-inflammatory neuropeptide substance P (SP) 18 to 24 hours after the first treatment. No significant changes were seen in the other neuropeptides, neurotrophins, or cytokines tested. Nasal obstruction, nasal itch, sneezing, runny nose, eye itch, and unrefreshed sleep improved significantly in the real acupuncture group (post-nasal drip and sinus pain did not) and continued to improve up to 4-week follow-up.
The authors concluded that acupuncture modulated mucosal immune response in the upper airway in adults with persistent allergic rhinitis. This modulation appears to be associated with down-regulation of allergen specific IgE for house dust mite, which this study is the first to report. Improvements in nasal itch, eye itch, and sneezing after acupuncture are suggestive of down-regulation of transient receptor potential vanilloid 1.
…Anyway, the trial itself raises a number of questions – unfortunately I have no access to the full paper – which I will post here in the hope that my acupuncture friend, who are clearly impressed by this paper, might provide the answers in the comments section below:
- Which was the primary outcome measure of this trial?
- What was the power of the study, and how was it calculated?
- For which outcome measures was the power calculated?
- How were the subjective endpoints quantified?
- Were validated instruments used for the subjective endpoints?
- What type of sham was used?
- Are the reported results the findings of comparisons between verum and sham, or verum and no acupuncture, or intra-group changes in the verum group?
- What other treatments did each group of patients receive?
- Does anyone really think that this trial shows that “acupuncture is a safe, effective and cost-effective treatment for allergic rhinitis”?
In the comments section, the author wrote: “after you have read the full text and answered most of your questions for yourself, it might then be a more appropriate time to engage in any meaningful discussion, if that is in fact your intent”, and I asked him to send me his paper. As he does not seem to have the intention to do so, I will answer the questions myself and encourage everyone to have a close look at the full paper [which I can supply on request].
- The myriad of lab tests were defined as primary outcome measures.
- Two sentences are offered, but they do not allow me to reconstruct how this was done.
- No details are provided.
- Most were quantified with a 3 point scale.
- Mostly not.
- Needle insertion at non-acupoints.
- The results are a mixture of inter- and intra-group differences.
- Patients were allowed to use conventional treatments and the frequency of this use was reported in patient diaries.
- I don’t think so.
So, here is my interpretation of this study:
- It lacked power for many outcome measures, certainly the clinical ones.
- There were hardly any differences between the real and the sham acupuncture group.
- Most of the relevant results were based on intra-group changes, rather than comparing sham with real acupuncture, a fact, which is obfuscated in the abstract.
- In a controlled trial fluctuations within one group must never be interpreted as caused by the treatment.
- There were dozens of tests for statistical significance, and there seems to be no correction for multiple testing.
- Thus the few significant results that emerged when comparing sham with real acupuncture might easily be false positives.
- Patient-blinding seems questionable.
- McDonald as the only therapist of the study might be suspected to have influenced his patients through verbal and non-verbal communications.
I am sure there are many more flaws, particularly in the stats, and I leave it to others to identify them. The ones I found are, however, already serious enough, in my view, to call for a withdrawal of this paper. Essentially, the authors seem to have presented a study with largely negative findings as a trial with positive results showing that acupuncture is an effective therapy for allergic rhinitis. Subsequently, McDonald went on social media to inflate his findings even more. One might easily ask: is this scientific misconduct or just poor science?
I would be most interested to hear what you think about it [if you want to see the full article, please send me an email].