Many proponents of alternative medicine seem somewhat suspicious of research; they have obviously understood that it might not produce the positive result they had hoped for; after all, good research tests hypotheses and does not necessarily confirm beliefs. At the same time, they are often tempted to conduct research: this is perceived as being good for the image and, provided the findings are positive, also good for business.
Therefore they seem to be tirelessly looking for a study design that cannot ‘fail’, i.e. one that avoids the risk of negative results but looks respectable enough to be accepted by ‘the establishment’. For these enthusiasts, I have good news: here is the study design that cannot fail.
It is perhaps best outlined as a concrete example; for reasons that will become clear very shortly, I have chosen reflexology as a treatment of diabetic neuropathy, but you can, of course, replace both the treatment and the condition as it suits your needs. Here is the outline:
- recruit a group of patients suffering from diabetic neuropathy – say 58, that will do nicely,
- randomly allocate them to two groups,
- the experimental group receives regular treatments by a motivated reflexologist,
- the controls get no such therapy,
- both groups also receive conventional treatments for their neuropathy,
- the follow-up is 6 months,
- the following outcome measures are used: pain reduction, glycemic control, nerve conductivity, and thermal and vibration sensitivities,
- the results show that the reflexology group experience more improvements in all outcome measures than those of control subjects,
- your conclusion: This study exhibited the efficient utility of reflexology therapy integrated with conventional medicines in managing diabetic neuropathy.
This method is fool-proof, trust me, I have seen it often enough being tested, and never has it generated disappointment. It cannot fail because it follows the notorious A+B versus B design (I know, I have mentioned this several times before on this blog, but it is really important, I think): both patient groups receive the essential mainstream treatment, and the experimental group receives a useless but pleasant alternative treatment in addition. The alternative treatment involves touch, time, compassion, empathy, expectations, etc. All of these elements will inevitably have positive effects, and they can even be used to increase the patients’ compliance with the conventional treatments that is being applied in parallel. Thus all outcome measures will be better in the experimental compared to the control group.
The overall effect is pure magic: even an utterly ineffective treatment will appear as being effective – the perfect method for producing false-positive results.
And now we hopefully all understand why this study design is so very popular in alternative medicine. It looks solid – after all, it’s an RCT!!! – and it thus convinces even mildly critical experts of the notion that the useless treatment is something worth while. Consequently the useless treatment will become accepted as ‘evidence-based’, will be used more widely and perhaps even reimbursed from the public purse. Business will be thriving!
And why did I employ reflexology for diabetic neuropathy? Is that example not a far-fetched? Not a bit! I used it because it describes precisely a study that has just been published. Of course, I could also have taken the chiropractic trial from my last post, or dozens of other studies following the A+B versus B design – it is so brilliantly suited for misleading us all.
Reflexology is one of the most popular of all alternative therapies. Anyone who has ever had a session knows why: it is a strangely pleasant and oddly agreeable experience. Reflexologists massage your feet which can be mildly painful but usually is quite relaxing. They look for and subsequently focus on areas of tenderness believing they correspond to specific organs or whole organ systems. Even though few reflexologists would admit to it, they tend to make vague and unreliable diagnoses: if they feel something unusual at a certain point of the sole of your foot, they assume that a certain inner organ is in trouble. Reflexologists even have maps where the sole of a foot is depicted showing which area corresponds to which organ.
The treatment might be enjoyable but the assumptions that underpin it are nonsensical for at least two reasons: firstly, there are no nerve or other connections between a specific area on the sole of a foot and a certain organ. Secondly, the maps which reflexologists employ differ and fail to agree which area corresponds to which organ. Thus there are inconsistencies within the realm of reflexology and there are inconsistencies in relation to the known facts regarding physiology, anatomy etc.
Proponents of reflexology are quite undisturbed by these problems and seem to believe that not their assumptions but science must be wrong. After all, reflexology does work! That is to say that patients perceive benefit from it, pay out of their own pocket for the experience and tend to come back for more.
Several years ago, we asked 8 UK professional organisations of reflexology which conditions they thought could be treated effectively with reflexology. We gave them a list of 25 conditions to chose from, many of which were serious, e.g. cancer and AIDS. Collectively, the organisations felt that 22 of these illnesses would respond to reflexology.
But this is opinion, not evidence! What do the trial data tell us? Is reflexology more than a placebo?
As with many other areas of alternative medicine, controlled clinical trials are scarce; but this is not to say that none at all are available. Our own trial of reflexology for menopausal symptoms failed to show that this therapy has any effects beyond placebo. More recently, we published a systematic review to evaluate all of the 23 studies that had been published at that stage. They related to a wide range of medical conditions and their methodological quality was often poor. Nine high quality randomised clinical trials (RCTs) generated negative findings. Eight RCTs suggested that reflexology is effective for the following conditions: diabetes, premenstrual syndrome, cancer patients, multiple sclerosis, symptomatic idiopathic detrusor over-activity and dementia. These studies, however, were wide open to bias. Therefore, our conclusions had to be cautious: the best clinical evidence does not demonstrate convincingly reflexology to be an effective treatment for any medical condition.
For you and me, this simply means that there is currently no good evidence to suggest that reflexology works. But the story does not end here. There will be more studies and enthusiasts are most likely to concede that our conclusions were incorrect. In fact, a further trial has just become available.
This new single-blind, randomized and placebo controlled study included 20 moderately to severely affected multiple sclerosis patients. Each participant received for 8 weeks, 1 hour per week of either reflexology or sham reflexology. The primary outcome measure was the Multiple Sclerosis Impact Scale at baseline, 8 weeks and 16 weeks. The results revealed improvements in both groups but no statistically significant differences between the two groups at either 8 or 16 weeks. The conclusions of the investigators were clear: The results do not support the use of reflexology for symptom relief in a more disabled multiple sclerosis population and are strongly suggestive of a placebo response.
There is, of course, nothing wrong with a relaxing foot-massage; it is agreeable, no doubt, and if someone wants to pay for the luxury, why not? By contrast, there is a lot wrong with reflexology, I think. A foot-massage is not administered under the pretence of generating any specific therapeutic effects. Reflexologists, however, claim they can exert highly specific effects on inner organs, influence the natural history of a wide range of diseases, and provide reliable diagnoses. They thus mislead their clients. This is not just wrong, it also has the potential to do serious harm. I believe it is time to end this nonsense.