Not sure about a news item? … That’s what I suggest you could do

In so-called alternative medicine (SCAM) – but certainly not just there – we regularly encouter reports about new research results that sound odd, implausible, too good to be true, or outright fantastic (like borne out of fantasy). What should one do with such news? Keep an open mind, yes sure, but what if the news leads us up the garden path? Here is what I usually do and what I recommend you might do as well:

1. Check who published the story; some sources are clearly more trustworthy than others (think of ‘Natural News‘, or WDDTY, for instance).
2. Try to find other outlets confirming the news; if none can be located, be extra sceptical.
3. Identify the origin of the new research; an academic might be more trustworthy that a SCAM practitioner or a commercial firm.
4. Find out where the study was originally published; some SCAM journals publish virtually any rubbish (think of EBCAM or JCAM).
5. If you are still in doubt and continue to be interested, go on Medline and obtain the original article.
6. If it’s behind a pay-wall, email the authors for a copy.
7. Check the validity of the paper; this can be rather a big task for someone not trained in critical assessment of scientific papers, but there are certain pointers: in case of a clinical trial, for instance, check whether it was large or small, randomised or not, placebo-controlled or not, blind or not.
8. If the findings look suspicious to you, find out more about the researchers: for example, do they have a track record of publishing results that look false-positive (think of M Frass or other members of the ‘ALTERNATIVE MEDICINE HALL OF FAME‘, for instance)?
9. Identify studies by other researchers addressing the same research question; have similar findings been published, or do most of the previous investigations contradict the results of the new study?
10. Find out who sponsored the new study.
11. Look up what the authors declare in terms of conflicts of interest.
12. If all of this leaves important questions unanswered, don’t be shy, write to the authors and ask.

When I have gone through all these steps, I usually have a fairly clear impression whether I can trust the research or not. Obviously keeping an open mind about new discoveries is sensible. But please. do remember that charlatans might (and often do) put a lot of BS in your mind, if you open it too wide for too long.