Many proponents of alternative medicine seem somewhat suspicious of research; they have obviously understood that it might not produce the positive result they had hoped for; after all, good research tests hypotheses and does not necessarily confirm beliefs. At the same time, they are often tempted to conduct research: this is perceived as being good for the image and, provided the findings are positive, also good for business.
Therefore they seem to be tirelessly looking for a study design that cannot ‘fail’, i.e. one that avoids the risk of negative results but looks respectable enough to be accepted by ‘the establishment’. For these enthusiasts, I have good news: here is the study design that cannot fail.
It is perhaps best outlined as a concrete example; for reasons that will become clear very shortly, I have chosen reflexology as a treatment of diabetic neuropathy, but you can, of course, replace both the treatment and the condition as it suits your needs. Here is the outline:
- recruit a group of patients suffering from diabetic neuropathy – say 58, that will do nicely,
- randomly allocate them to two groups,
- the experimental group receives regular treatments by a motivated reflexologist,
- the controls get no such therapy,
- both groups also receive conventional treatments for their neuropathy,
- the follow-up is 6 months,
- the following outcome measures are used: pain reduction, glycemic control, nerve conductivity, and thermal and vibration sensitivities,
- the results show that the reflexology group experience more improvements in all outcome measures than those of control subjects,
- your conclusion: This study exhibited the efficient utility of reflexology therapy integrated with conventional medicines in managing diabetic neuropathy.
Mission accomplished!
This method is fool-proof, trust me, I have seen it often enough being tested, and never has it generated disappointment. It cannot fail because it follows the notorious A+B versus B design (I know, I have mentioned this several times before on this blog, but it is really important, I think): both patient groups receive the essential mainstream treatment, and the experimental group receives a useless but pleasant alternative treatment in addition. The alternative treatment involves touch, time, compassion, empathy, expectations, etc. All of these elements will inevitably have positive effects, and they can even be used to increase the patients’ compliance with the conventional treatments that is being applied in parallel. Thus all outcome measures will be better in the experimental compared to the control group.
The overall effect is pure magic: even an utterly ineffective treatment will appear as being effective – the perfect method for producing false-positive results.
And now we hopefully all understand why this study design is so very popular in alternative medicine. It looks solid – after all, it’s an RCT!!! – and it thus convinces even mildly critical experts of the notion that the useless treatment is something worth while. Consequently the useless treatment will become accepted as ‘evidence-based’, will be used more widely and perhaps even reimbursed from the public purse. Business will be thriving!
And why did I employ reflexology for diabetic neuropathy? Is that example not a far-fetched? Not a bit! I used it because it describes precisely a study that has just been published. Of course, I could also have taken the chiropractic trial from my last post, or dozens of other studies following the A+B versus B design – it is so brilliantly suited for misleading us all.
23 Responses to Want to legitimise your bogus therapy? Here is how
Leave a Reply to jm Cancel reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
Oh, clever. Now how do I use that for my engineering research? I really could do with proving that coal is good for you.
Coal is used to provide electricity, which heats our homes and powers our lighting and communications devices.
Now, conduct a study to compare A+B versus B, where:
A+B = people having homes with coal-powered electricity.
B = people having homes without electricity.
Select participants who are living in a cold climate. The study will prove that coal significantly improves quality of life.
How about in Sahara desert?
Jorma Kärtsy’s question reveals the truth of sCAM: it doesn’t solve any real world problems. The studies it churns out are crafted to produce false positive results.
Let’s face it, to sell snake oil one needs to master crafty sales techniques; one doesn’t require an effective product. Theatrical distractions are a very effective technique. A+B versus B studies are just one of the many forms of theatrical distraction used in the sCAM industry.
Entirely agree, but there’s another lesson to be drawn from such studies: “touch, time, compassion, empathy, expectations, etc.” are probably valuable. I can’t prove it of course (and it needs to be proved, of course!) but I have a strong feeling that many people are attracted to ‘alternative’ (yuck, hate that word) therapists precisely because they sound caring and sympathetic, seem to have plenty of time and work in pleasant surroundings – in contrast to many GPs, through no fault of their own. So maybe another model for a trial would be Alternative therapy plus conventional versus Conventional therapy with the same feel-good factors. Do I remember reading about a trial of acupuncture which attempted to do that?
“…through no fault of their own…”
So whose fault would it be then?
No fault of their own because the economics of an NHS GP do not run to half hour consultations in plush surroundings etc. Not saying some GPs couldn’t do better in the ‘care and concern’ department, but let’s give them some benefit of the doubt, for the sake of charity.
Are you really saying that the economics of an NHS GP precludes compassion? Seriously? No benefit of the doubt or any amount of charity can shift the fault from GPs to economics. Sorry. Compassion takes no time or money at all.
If adding treatment B indeed invariably leads to better outcomes, then shouldn’t B always be offered as a part of standard care? Provided of course B was shown not only effective as a complementary treatment, but also to be safe?
yes, provided the effects are caused by B per se and not by its context.
and for finding out whether B has specific effects you need studies that control for context effects – the type of study that alternativists tend to avoid [probably because they fear the result]
Thanks very much for your reply. I see what you mean. Even though one could be tempted to just add treatment B to standard care regardless of its specific effects, there would be no end to the adding-on of complementary treatments. As soon as A + B becomes the standard of care, CAM-researchers would show that adding C to A + B leads to even better outcomes…
Next thing you know, compassion and empathy will become part of standard care…THEN where will we be? I’ll tell you where – our comapssion and empathy reserves will run out, and we’ll be dependent on the Buddhists. If we all agree to label inevitable positive effects useless…we might just stand a chance.
“and the experimental group receives a useless but pleasant alternative treatment in addition. The alternative treatment involves touch, time, compassion, empathy, expectations, etc. All of these elements will inevitably have positive effects, and they can even be used to increase the patients’ compliance with the conventional treatments that is being applied in parallel”
Make B a million dollar, less pressure in life, and healthier food provided by a personal cook. That would improve the outcome in most patients. Still chances are low that we make it a standard treatment. Why?
As far as an insurance/public health system pays it, it’s a question of efficiency, as resources are limited. Studies with the A vs A+B design create “evidence” that potentially leads to misalloation of scarce resources in healthcare.
very true!
The comments from Joyce Beck and Bryan raise a very important, but frequently overlooked, issue. Let’s examine this issue…
There is more than adequate evidence to indicate that things such as “touch, time, compassion, empathy, expectations, etc.” are highly valuable for a wide range of health conditions. Sadly, GPs are not allocated nearly enough time to provide these complementary services.
So, is there a problem with offering complementary treatment as per Bryan’s question? Specifically, if A=complementary treatment that is safe, B=medical treatment, and A+B versus B is demonstrably more effective than B, then why not offer A as an adjunct to B?
Suppose the complementary treatment A is provided by a homeopath belonging to a register vetted and approved by the Professional Standards Authority for Health and Social Care (PSA). One would expect that treatment A was being delivered by a healthcare professional just as one expects that treatment B is being delivered by a healthcare professional (e.g. a GP who’s a licenced MD).
To be a registered professional with the PSA, does a homeopath need to be qualified and licensed to practice in any of the following areas of healthcare: medicine; clinical psychology; diet and nutrition; social work? No! A homeopath doesn’t even need to obtain a certificate in Counselling & Listening Skills Level I.
In conclusion, complementary treatment can be a useful adjunct to medical treatment, but ONLY when the treatment is delivered by an appropriately qualified professional. sCAM quacks (irrespective of them being registered with the PSA) are not professionally qualified to provide this useful adjunct — they are charlatans masquerading as healthcare providers.
Not allocated enough time for compassion, eh? Exactly how much time does compassion take?
And, when did compassion become a “complementary service”?
Wow.
It depends on what you mean by the word compassion. In the provision of healthcare, compassion is not sympathetic pity, it is concern for the suffering of others — an emotion, not an action per se.
During a typical 5 to 10 minute GP appointment there isn’t time for a cup of tea and a sympathetic chat. Many patients expect at least a little sympathy when they are ill, but fail to realize that sympathy is provided by their support network of family and friends. It is not the remit of primary health care to provide sympathetic pity. Indeed, sympathetic pity can be highly detrimental to patient recovery. There’s a huge difference between being a professionally trained compassionate listener and an untrained sympathetic ear.
UK patients who need time to talk through their problems have access to talking therapy services (and other psychological services) provided by the NHS. They don’t need to waste their time and money on an alt-med quack.
Of course, if the patient wants sympathy rather than compassionate healthcare then paying a quack might be a better option than draining their family and friends.
Further reading:
http://en.wikipedia.org/wiki/Compassion#Medicine
http://en.wikipedia.org/wiki/Compassion_fatigue
Who said anything about sympathetic pity? I think you should reread the post and the comments.
It’s interesting that you bring up “a typical 5 to 10 minute GP appointment” and “quack” in the same comment…
But not in the same paragraph or sentence. Do you have any idea about context?
Presumably this can be confirmed by having group B go for a nice chat and a cup of tea with a friendly face with and an invented role after their regular treatment…?
There have been some excellent comments and broad generalizations, assumptions made above. Overall good debate!