Recently, I had the pleasure to give a lecture about bias in research to medical students at my former medical school in Vienna. This led to interesting discussions with the audience. They prompted me to think more than usual about ‘the biased researcher’, a phenomenon that, in my opinion, seriously plagues the field of so-called alternative medicine (SCAM). The way I see it, we can differentiate 4 overlapping categories of researchers (male or female; for simplicity, I here use only the masculine form) investigating the effectiveness of various forms of SCAM:
THE TRUE SCIENTIST
The true scientist is adequaately trained in all aspects of his work. Therefore, he knows that he research consists of testing hypotheses. He does his job without emotional or ideological baggage. All he aims at doing is answering the research question at hand in the most rigorous fashion. He is not influenced by outside pressures, does not care about the direction of his results, and merely wants to conduct the best science possible that his particular situation allows. In other words, he does what he can to minimize all sources of bias.
THE SLOPPY RESEARCHER
The sloppy researcher is either less well trained or he is less focussed and somewhat careless. He tends to cut corners, and is thus prone to make mistakes. His miskakes can introduce bias in his research which is unintentional because he has no axe to grind. In other words, the sloppy researcher is not biased but might easily produce biased results. As the sloppyness is unintentional, the resulting bias can go in either direction; the sloppy researcher might therefore generate false-positive and false-negative findings at random.
THE BIASED RESEARCHER
The biased researcher does have an axe to grind. Typically, he has a strong positive opinion about the treatment he is testing. For him, the concept of falsifying his beloved hypothesis is an abomination – he might know that this is how science out to work, but he can simply not bring himself to doing it. His mission is to confirm his prior conviction that the therapy in question is effective. This conviction is so strong that he does not feel that he is doing anything wrong. Obviously, the biased researcher would introduce bias into his research at multible levels. His bias will then compell him to hide the flaws in his research as much as he can. Consequently, his published papers will not easily disclose his bias and will therefore have the power to mislead the public.
THE DISHONEST RESEARCHER
The dishonest researcher is out to cheat. He wants to generate resuts of a certain type, usually showing that the therapy in question is effective. He is usually motivated by money and/or ambition. He may be sufficiently well trained to be able to hide his dishonesty from detection. Like the biased researcher’s papers, his fraudulent publications will not disclose his fabrications and will therefore have the power to mislead the public.
You will, of course, realize that, in my attempt to create these 4 categories, I have exaggerated and created caricatures of the real-life situations. However, I feel the the distinction between the 4 categories might be helpful to understand medical research and its pitfalls. As I pointed out in the introduction, the categories overlap. In reality, most researchers are hybrids of two or more categories. For instance, nobody can entirely be free of bias and everybody makes mistakes occasionally.
The question arises as to which type of category might dominate SCAM. I am not aware of reliable research that would answer it. However, my experience tells me that, in SCAM, we have a regrettable void of true scientists combined with an abundance of biased researchers (see, for instance, the growing list of researchers in my ALTERNATIVE MEDICINE HALL OF FAME). What is worse, the latter category is bringing SCAM research more and more into disrepute which, in turn, demotivates true scientists to consider SCAM as a serious subject.