It’s again the season for nine lessons, I suppose. So, on the occasion of Christmas Eve, let me rephrase the nine lessons I once gave (with my tongue firmly lodged in my cheek) to those who want to make a pseudo-scientific career in so-called alternative medicine (SCAM) research.
- Throw yourself into qualitative research. For instance, focus groups are a safe bet. They are not difficult to do: you gather 5 -10 people, let them express their opinions, record them, extract from the diversity of views what you recognize as your own opinion and call it a ‘common theme’, and write the whole thing up, and – BINGO! – you have a publication. The beauty of this approach is manifold:
-
- you can repeat this exercise ad nauseam until your publication list is of respectable length;
- there are plenty of SCAM journals that will publish your articles;
- you can manipulate your findings at will;
- you will never produce a paper that displeases the likes of King Charles;
- you might even increase your chances of obtaining funding for future research.
- Conduct surveys. They are very popular and highly respected/publishable projects in SCAM. Do not get deterred by the fact that thousands of similar investigations are already available. If, for instance, there already is one describing the SCAM usage by leg-amputated policemen in North Devon, you can conduct a survey of leg-amputated policemen in North Devon with a medical history of diabetes. As long as you conclude that your participants used a lot of SCAMs, were very satisfied with it, did not experience any adverse effects, thought it was value for money, and would recommend it to their neighbour, you have secured another publication in a SCAM journal.
- In case this does not appeal to you, how about taking a sociological, anthropological or psychological approach? How about studying, for example, the differences in worldviews, the different belief systems, the different ways of knowing, the different concepts about illness, the different expectations, the unique spiritual dimensions, the amazing views on holism – all in different cultures, settings or countries? Invariably, you must, of course, conclude that one truth is at least as good as the next. This will make you popular with all the post-modernists who use SCAM as a playground for enlarging their publication lists. This approach also has the advantage to allow you to travel extensively and generally have a good time.
- If, eventually, your boss demands that you start doing what (in his narrow mind) constitutes ‘real science’, do not despair! There are plenty of possibilities to remain true to your pseudo-scientific principles. Study the safety of your favourite SCAM with a survey of its users. You simply evaluate their experiences and opinions regarding adverse effects. But be careful, you are on thin ice here; you don’t want to upset anyone by generating alarming findings. Make sure your sample is small enough for a false negative result, and that all participants are well-pleased with their SCAM. This might be merely a question of selecting your patients wisely. The main thing is that your conclusions do not reveal any risks.
- If your boss insists you tackle the daunting issue of SCAM’s efficacy, you must find patients who happened to have recovered spectacularly well from a life-threatening disease after receiving your favourite form of SCAM. Once you have identified such a person, you detail her experience and publish this as a ‘case report’. It requires a little skill to brush over the fact that the patient also had lots of conventional treatments, or that her diagnosis was never properly verified. As a pseudo-scientist, you will have to learn how to discretely make such details vanish so that, in the final paper, they are no longer recognisable.
- Your boss might eventually point out that case reports are not really very conclusive. The antidote to this argument is simple: you do a large case series along the same lines. Here you can even show off your excellent statistical skills by calculating the statistical significance of the difference between the severity of the condition before the treatment and the one after it. As long as this reveals marked improvements, ignores all the many other factors involved in the outcome and concludes that these changes are the result of the treatment, all should be tickety-boo.
- Your boss might one day insist you conduct what he narrow-mindedly calls a ‘proper’ study; in other words, you might be forced to bite the bullet and learn how to do an RCT. As your particular SCAM is not really effective, this could lead to serious embarrassment in the form of a negative result, something that must be avoided at all costs. I, therefore, recommend you join for a few months a research group that has a proven track record in doing RCTs of utterly useless treatments without ever failing to conclude that it is highly effective. In other words, join a member of my ALTERNATIVE MEDICINE HALL OF FAME. They will teach you how to incorporate all the right design features into your study without the slightest risk of generating a negative result. A particularly popular solution is to conduct a ‘pragmatic’ trial that never fails to produce anything but cheerfully positive findings.
- But even the most cunningly designed study of your SCAM might one day deliver a negative result. In such a case, I recommend taking your data and running as many different statistical tests as you can find; chances are that one of them will produce something vaguely positive. If even this method fails (and it hardly ever does), you can always focus your paper on the fact that, in your study, not a single patient died. Who would be able to dispute that this is a positive outcome?
- Now that you have grown into an experienced pseudo-scientist who has published several misleading papers, you may want to publish irrefutable evidence of your SCAM. For this purpose run the same RCT over again, and again, and again. Eventually, you want a meta-analysis of all RCTs ever published (see examples here and here). As you are the only person who conducted studies on the SCAM in question, this should be quite easy: you pool the data of all your dodgy trials and, bob’s your uncle: a nice little summary of the totality of the data that shows beyond doubt that your SCAM works and is safe.
3 Responses to Nine lessons but no carols
Leave a Reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
I guess there is too much money in medicine now to trust anybody. This from Dr Healy’s book Pharmageddon:
“There can be a few better symbols of Pharmageddon then prescription only drugs becoming among the most consumed drugs in pregnancy in the face of strengthening warnings that they cause birth defects. The answer to how this could happen lie in great part in how the pharmaceutical companies have managed to capitalize on the way the very protections put in place by Senator Kefauver in his 1962 Bill and in the reforms that defeated him. Prescription only status has made doctors the targets of a marketing exercise that is far more sophisticated than placing even billions of pages of advertisements in medical journals and bribing doctors to use drugs. As outlined in chapter 1, the patent status of drugs has given companies an incentive to chase blockbuster profits – doing so regardless of patient welfare. Controlled trials have given the companies a means to persuade doctors that snake oil works so well that withholding it in pregnancy would be unethical, and also a means to make problems consequent on treatments vanish. But all of these things hinge on the fact that these drugs are available by prescription only.”
“Similar conflicts of interest and biases exist in virtually every field of medicine, particularly those that rely heavily on drugs or devices. It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of The New England Journal of Medicine .” Dr Marcia Angell, 2009
@ stan
Marcia Angell, MD, also wrote:
“If it is found to be reasonably safe and effective, it will be accepted.”
Until it’s not, maybe, sometimes or it depends.
“Low-value medical practices are medical practices that are either ineffective or that cost more than other options but only offer similar effectiveness (Prasad et al., 2013; Prasad et al., 2011; Schpero, 2014). Such practices can result in physical and emotional harm, undermine public trust in medicine, and have both an opportunity cost (Korenstein et al., 2018) and a financial cost (Reid et al., 2016; Beaudin-Seiler, 2016). Identifying and eliminating low-value medical practices will, therefore, reduce costs and improve care.”
https://elifesciences.org/articles/45183
But yes, credit to the medical sciences in their efforts for trying to improve their approaches. But sometimes that improvement means admitting, “oops, we were wrong”.