In recent years, I have found myself getting irritated with researchers finishing their evaluation of a so-called alternative medicine (SCAM) with the sentence ‘MORE RESEARCH IS NEEDED’ (or similar). It is irritating because it fails to draw a line under assessments of even the most hopelessly implausible treatment. And, because it leaves things open, it seems to imply that, until further research is available, things can go on as before.

When I realised that plenty of my own papers ended with this statement, I was first taken aback and then even more irritated. How could I have been guilty of repeatedly publishing such nonsense?

Here are just 5 examples of my blundering:

further trials of high methodological quality with sufficient sample size and follow-up are needed

Future rigorous randomised clinical trials with larger sample sizes will be necessary

Future investigations in this area should overcome the multiple methodological weaknesses of the previous research.

More and larger long-term, high-quality trials are needed.

Larger and more rigorous trials are needed to objectively assess the effects of this herbal supplement.

But subsequently I re-considered and asked myself: what does ‘MORE RESEARCH IS NEEDED’, a phrase used by so many researchers, really mean?

Contrary to how it seems often to be understood in SCAM, it cannot (should not) mean that, until there is more evidence, we are all free to employ the treatment in question.

Let’s take my first two of my articles quoted above as examples. The first was an assessment of qigong for the primary prevention of cardiovascular disease, and the second an evaluation of acupuncture as a treatment of ankle sprains. When concluding that, in both cases, more research is needed, I did certainly not mean to issue a ‘carte blanche’ to clinicians for carrying on using an evidently unproven SCAM!

What the sentence ‘MORE RESEARCH IS NEEDED’ actually means is almost the opposite:

  1. at present, the evidence is insufficient;
  2. more research is needed for a firm verdict;
  3. currently, the effectiveness of the treatment is unproven;
  4. it is unwise and possibly even unethical to employ unproven treatments in clinical routine, particularly in situations for which evidence-based therapies are available.

And, if this is so, one also needs to express that NO MORE RESEARCH IS NEEDED, whenever this applies. In the realm of SCAM, this would be the case, if a therapy is hopelessly implausible, for instance. I am glad to say that, occasionally, I did do just that:

… There are hundreds of different homeopathic remedies which can be prescribed for thousands of symptoms in dozens of different dilutions. Thus we would probably need to work flat out for several lifetimes in order to arrive at a conclusion that fully substantiates my opening statement*.

This seems neither possible nor desirable. Perhaps it is preferable to simply combine common sense with the best existing knowledge. These two tell us that 1) homeopathy is biologically implausible, 2) its own predictions seem to be incorrect and 3) the clinical evidence is largely negative…

… the conundrum of homeopathy seems to be solved. ‘Heavens!’ I hear the homeopathic fraternity shout. ‘We need more research!’ But are they correct? How much research is enough to show that any treatment does not work (sorry, is not superior to placebo)? Here we go full circle: should we really spend several lifetimes in order to arrive at a more robust conclusion?

*homeopathy is not better than placebo


7 Responses to “… more research is needed …” – what does this conclusion really mean?

  • “1. at present, the evidence is insufficient;
    “2. more research is needed for a firm verdict;
    “3. currently, the effectiveness of the treatment is unproven;
    “4. it is unwise and possibly even unethical to employ unproven treatments in clinical routine, particularly in situations for which evidence-based therapies are available”

    I know a few surgeons who ought to take note of this.

  • Of course it is worth noting: “beyond a reasonable doubt” has still sent many an innocent man to the gas chamber.

    • It is just as well, then, that the standard of evidence required by the scientific process is considerably higher than that accepted by a criminal Court.

  • Isnt that better than burying all the unfavorable studies and only publishing the few favorable ghostwritten ones, like big pharma does when they roll out a new drug with serious/fatal side effects? To be honest their articles should end with “No further research wanted. There is nothing bad we want you to see here.”

    • you haven’t heard of such safeguards a trial registers, have you?

      • ” Not all listed studies are regulated and/or reviewed by the U.S. Food and Drug Administration or other governmental entities. Information on is provided by study sponsors and investigators, and they are responsible for ensuring that the studies follow all applicable laws and regulations. NLM staff do not verify the scientific validity or relevance of the submitted information beyond a limited quality control review for apparent errors, deficiencies, or inconsistencies. “

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted but you must tick the box: “Save my name, email, and website in this browser for the next time I comment.”

The most recent comments from all posts can be seen here.