The aim of this pragmatic study was “to investigate the effectiveness of acupuncture in addition to routine care in patients with allergic asthma compared to treatment with routine care alone.”
Patients with allergic asthma were included in a controlled trial and randomized to receive up to 15 acupuncture sessions over 3 months plus routine care, or to a control group receiving routine care alone. Patients who did not consent to randomization received acupuncture treatment for the first 3 months and were followed as a cohort. All trial patients were allowed to receive routine care in addition to study treatment. The primary endpoint was the asthma quality of life questionnaire (AQLQ, range: 1–7) at 3 months. Secondary endpoints included general health related to quality of life (Short-Form-36, SF-36, range 0–100). Outcome parameters were assessed at baseline and at 3 and 6 months.
A total of 1,445 patients were randomized and included in the analysis (184 patients randomized to acupuncture plus routine care and 173 to routine care alone, and 1,088 in the nonrandomized acupuncture plus routine care group). In the randomized part, acupuncture was associated with an improvement in the AQLQ score compared to the control group (difference acupuncture vs. control group 0.7 [95% confidence interval (CI) 0.5–1.0]) as well as in the physical component scale and the mental component scale of the SF-36 (physical: 2.5 [1.0–4.0]; mental 4.0 [2.1–6.0]) after 3 months. Treatment success was maintained throughout 6 months. Patients not consenting to randomization showed similar improvements as the randomized acupuncture group.
The authors concluded that in patients with allergic asthma, additional acupuncture treatment to routine care was associated with increased disease-specific and health-related quality of life compared to treatment with routine care alone.
We have been over this so many times (see for instance here, here and here) that I am almost a little embarrassed to explain it again: it is fairly easy to design an RCT such that it can only produce a positive result. The currently most popular way to achieve this aim in alternative medicine research is to do a ‘A+B versus B’ study, where A = the experimental treatment, and B = routine care. As A always amounts to more than nothing – in the above trial acupuncture would have placebo effects and the extra attention would also amount to something – A+B must always be more than B alone. The easiest way of thinking of this is to imagine that A and B are both finite amounts of money; everyone can understand that A+B must always be more than B!
Why then do acupuncture researchers not get the point? Are they that stupid? I happen to know some of the authors of the above paper personally, and I can assure you, they are not stupid!
So, why?
I am afraid there is only one reason I can think of: they know perfectly well that such an RCT can only produce a positive finding, and precisely that is their reason for conducting such a study. In other words, they are not using science to test a hypothesis, they deliberately abuse it to promote their pet therapy or hypothesis.
As I stated above, it is fairly easy to design an RCT such that it can only produce a positive result. Yet, it is arguably also unethical, perhaps even fraudulent, to do this. In my view, such RCTs amount to pseudoscience and scientific misconduct.
22 Responses to If researchers commit scientific misconduct, acupuncture may appear effective
Leave a Reply to Edzard Cancel reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
It’s a disgraceful scam perpetrated by the shameless.
Stupid and stupidity are relative terms. Relative to his brilliant achievements in chemistry and activism for peace, Linus Pauling’s late-life obsession with Vitamin C being a panacea constituted stupidity and one is not altogether wrong to call him stupid in that context.
Many eminent “alternativists” started out as academics.
I can’t believe someone is so stupid! Acupuncture is not a drug. Why don’t you use RCT for all the gastric bypass studies? Oh, I see! because stupid people don’t use RCT! Hahaha, you are a funny old man.
https://www.ncbi.nlm.nih.gov/pubmed/26983924
https://www.ncbi.nlm.nih.gov/pubmed/27377635
https://www.ncbi.nlm.nih.gov/pubmed/27736851
https://www.ncbi.nlm.nih.gov/pubmed/25697125
I can’t believe someone is so stupid!
Are people actually believing you? Your argument relies on your A + B vs B study. But that study assumes that acupuncture does not work. Stop using circular reasoning.
in real medicine one has to provide proof that a treatment work for it to be used, not proof that it does not work.
In real science, you do not make a claim which is not justified by scientific investigation. You cannot say whether something works or doesn’t work unless the direction is justified. Right now the results are mixed.
But your issue is more basic than that. You are not committing a scientific error. You are committing a basic logical error. You are using a result which is based on the assumption that what you are currently trying to show is true. That’s circular reasoning, and is a far more fundamental flaw.
You are trying to show P. You are using R to show P. But you assume P to show R. Understand?
“You are using a result which is based on the assumption that what you are currently trying to show is true.”
CAN YOU BE MORE PRECISE?
GIVE A CONCRETE EXAMPLE WHERE I HAVE DONE THIS, PLEASE.
> GIVE A CONCRETE EXAMPLE WHERE I HAVE DONE THIS, PLEASE.
Hey; no need to yell. I already explained how you’re doing this. You are relying on your study which knocks down A + B vs B studies, in your argument against the efficacy of acupuncture. But you assume that acupuncture does not work, in order to reach the result that you did, in your paper.
I am not sure I get you – the default position in responsible healthcare must always be that a therapy is ineffective until proven otherwise.
this blog is about critically evaluating claims made for alt med; and that’s what I am doing.
perhaps you need to reflect about one or two things before commenting?
> I am not sure I get you – the default position in responsible healthcare must always be that a therapy is ineffective until proven otherwise.
That might be a reasonable default position for a baseline assumption but not for a claim. You cannot claim that something does not work until you show that it does not work. Regardless, your error is far more fundamental: your evidence for your claim relies on the assumption that your claim is true.
Let’s say that I assumed that pain killers did not work, and then based on that assumption I reached the conclusion that a lot of medical research was flawed. After that I go around saying that pain killers have not been shown to work, and I use my study as justification for my claim. Is that valid? No. Why? Because I have assumed that which I am now trying to show, in a result that I am using to justify what I am trying to show.
> perhaps you need to reflect about one or two things before commenting?
No; but you do.
Here’s what you have. Previously you assumed P (acupuncture does not work) and reached a conclusion Q (A + B vs B studies are biased towards positive results). In these blog articles, you are trying to SHOW P. The problem is, you’re doing that by using Q, which relies on the assumption that P is true.
You’re not even committing a soft fallacy in scientific investigation. You’re committing a hard fallacy: circular reasoning, and it seems that you’re not getting that.
you are wrong:
I do not claim that xy does not work.
I doubt that it does not work based on not one single trial but on the totality of the reliable evidence.
You specifically use the assumption that acupuncture is no better than a placebo to conclude, in your study, that A + B vs B studies give false positives. Now you are using your conclusion that A + B vs B studies give false positives to reject evidence justifying the position that acupuncture does work better than a placebo.
You are assuming, in a prior result, that which you are trying to show here, and yet you are using that prior result to justify your position here. You cannot cite your study as part of this argument because it assumes that what you are trying to show.
I think you are talking rubbish.
I make no assumption about acupuncture. I use the evidence from reliable systematic reviews.
> I make no assumption about acupuncture. I use the evidence from reliable systematic reviews.
Your paper specifically looks at A + B vs B studies involving acupuncture treatment. You specifically admit that your result works on the assumption that acupuncture is not effective. You cannot use a paper that assumes that which you’re now trying to show, as justification for what you’re trying to show.
You can’t get this. Oh well. Good bye.
I assume Daniel is saying that using this study as a proof for the “A + B vs. B = positive results”-concept is circular reasoning, because we cannot be sure if acupuncture is the reason for the positive results.
But here is the catch: Even if acupuncture would have a positive effect, it wouldn’t change the argument Edzard is making. Acupuncture could be working and the design and the conclusions drawn would still be just as flawed. If you use the “A + B vs. B” design, you will get positive results (maybe with the exception that your intervention is actually harming the patient). Therefore it is a poorly designed study, with results that should better be ignored, no matter if the intervention is actually effective or not – it just does not count as evidence.
I don’t really see any circular reasoning here. Just good logical, critical thinking.
Edzard’s argument is:
– A + B vs. B = positive results
– This study uses this design
– Therefore the design is the reason for the positive results and the study shouldn’t be used to make claims about the efficacy of the intervention
What Daniel seems to be thinking his argument is:
– This study uses the A + B vs. B design
– This study has positive results
– Therefore A + B vs. B = positive results
This study is just an example of CAM proponents using this flawed design to publish conclusion that they shouldn’t be drawing, unless they’ve thrown critical thinking out of the window.
> If you use the “A + B vs. B” design, you will get positive results (maybe with the exception that your intervention is actually harming the patient).
The only empirical justification for this result that has been presented is a study which relies on the assumption that acupuncture does not work any better than a placebo. That is why his use of the argument is circular reasoning. If his paper did not rely on the assumption about the efficacy of acupuncture, the story would be different.
Can you provide to me a paper that provides empirical justification for the claim that A + B vs B studies give false positives, and which does not rely on the assumption that the claim being made here is true?
“The only empirical justification for this result that has been presented is a study which relies on the assumption that acupuncture does not work any better than a placebo.”
By Jove!
you ought to read a little bit more of what I have posted on this blog!
“Previously you assumed P (acupuncture does not work) and reached a conclusion Q (A + B vs B studies are biased towards positive results). In these blog articles, you are trying to SHOW P. The problem is, you’re doing that by using Q, which relies on the assumption that P is true.”
I think you are just misinformed here. To use your own language: Conclusion Q clearly wasn’t reached just because of assumption P. But because of piles of other evidence for conclusion Q. Seeing that acupuncture studies often use this design, you can use Q to argue that the quality of these studies is so low, that you cannot make claims about the efficacy of acupuncture, without committing a logical error.
“Can you provide to me a paper that provides empirical justification for the claim that A + B vs B studies give false positives, and which does not rely on the assumption that the claim being made here is true?”
I personally have enough empirical justification without a specific paper. You just need to exercise some critical thinking:
Comparing an intervention A and intervention B against just intervention B ist plain bad science, to put it bluntly. We are well aware of the dynamics of the placebo effects and it is a well established fact, that the only way to minimize those effects is to do a similar (double blinded) placebo intervention in the control group.
In this particular study a balanced design would’ve been “Acupuncture + usual care” (A + B) vs. “sham-acupuncture + usual care” (Placebo-A + B). That is the only way to make sure that the patients receive similar care, have similar amounts of contact with the medical team etc. Even the authors of the study acknowledge that: “The study is further limited by the fact that without a sham acupuncture group, non-specific (placebo) effects could not be ascertained.” (Albeit to only to ignore this limitation and continue to draw some wild conclusions!)
So my argument is basically: We are very aware of a mechanism (placebo effects) that could, and most likely would, affect the results in a “positive” way, resulting in unwanted bias. There is not a single drop of evidence showing that this design would be well balanced, on the contrary. If it would be so, we could just scrap the placebo controls! That would save loads of money. For some reason, we haven’t done that…
Do I really need to publish a paper about this to make you understand the logic too?
https://www.ncbi.nlm.nih.gov/pubmed/?term=ernst%2C+lee%2C+j+postgrad+med
this paper might come close to showing that A+B v B generates only positive results.