“… more research is needed …” – what does this conclusion really mean?

In recent years, I have found myself getting irritated with researchers finishing their evaluation of a so-called alternative medicine (SCAM) with the sentence ‘MORE RESEARCH IS NEEDED’ (or similar). It is irritating because it fails to draw a line under assessments of even the most hopelessly implausible treatment. And, because it leaves things open, it seems to imply that, until further research is available, things can go on as before.

When I realised that plenty of my own papers ended with this statement, I was first taken aback and then even more irritated. How could I have been guilty of repeatedly publishing such nonsense?

Here are just 5 examples of my blundering:

… further trials of high methodological quality with sufficient sample size and follow-up are needed …

Future rigorous randomised clinical trials with larger sample sizes will be necessary …

Future investigations in this area should overcome the multiple methodological weaknesses of the previous research.

More and larger long-term, high-quality trials are needed.

Larger and more rigorous trials are needed to objectively assess the effects of this herbal supplement.

But subsequently I re-considered and asked myself: what does ‘MORE RESEARCH IS NEEDED’, a phrase used by so many researchers, really mean?

Contrary to how it seems often to be understood in SCAM, it cannot (should not) mean that, until there is more evidence, we are all free to employ the treatment in question.

Let’s take my first two of my articles quoted above as examples. The first was an assessment of qigong for the primary prevention of cardiovascular disease, and the second an evaluation of acupuncture as a treatment of ankle sprains. When concluding that, in both cases, more research is needed, I did certainly not mean to issue a ‘carte blanche’ to clinicians for carrying on using an evidently unproven SCAM!

What the sentence ‘MORE RESEARCH IS NEEDED’ actually means is almost the opposite:

1. at present, the evidence is insufficient;
2. more research is needed for a firm verdict;
3. currently, the effectiveness of the treatment is unproven;
4. it is unwise and possibly even unethical to employ unproven treatments in clinical routine, particularly in situations for which evidence-based therapies are available.

And, if this is so, one also needs to express that NO MORE RESEARCH IS NEEDED, whenever this applies. In the realm of SCAM, this would be the case, if a therapy is hopelessly implausible, for instance. I am glad to say that, occasionally, I did do just that:

… There are hundreds of different homeopathic remedies which can be prescribed for thousands of symptoms in dozens of different dilutions. Thus we would probably need to work flat out for several lifetimes in order to arrive at a conclusion that fully substantiates my opening statement*.

This seems neither possible nor desirable. Perhaps it is preferable to simply combine common sense with the best existing knowledge. These two tell us that 1) homeopathy is biologically implausible, 2) its own predictions seem to be incorrect and 3) the clinical evidence is largely negative…

… the conundrum of homeopathy seems to be solved. ‘Heavens!’ I hear the homeopathic fraternity shout. ‘We need more research!’ But are they correct? How much research is enough to show that any treatment does not work (sorry, is not superior to placebo)? Here we go full circle: should we really spend several lifetimes in order to arrive at a more robust conclusion?

*homeopathy is not better than placebo