research
The last few days, I spent much of my time answering questions from journalists on the subject of Charles lll. [interestingly, almost exclusively journalists NOT writing for UK newspapers]. Unsurprisingly, they all wanted to know about the way Charles managed to close down my research department at Exeter University some 10 years ago.
The story is old and I am a bit tired of repeating it. So, nowadays I often refer people to Wikipedia where a short paragraph sums it up:
Ernst was accused by Prince Charles’ private secretary of having breached a confidentiality agreement regarding the 2005 Smallwood report. After being subjected to a “very unpleasant” investigation by the University of Exeter, the university “accepted his innocence but continued, in his view, to treat him as ‘persona non grata’. All fundraising for his unit ceased, forcing him to use up its core funding and allow its 15 staff to drift away.”[15] He retired in 2011, two years ahead of his official retirement.[10][25] In July 2011, a Reuters article described his “long-running dispute with the Prince about the merits of alternative therapies” and stated that he “accused Britain’s heir-to-the-throne Prince Charles and other backers of alternative therapies on Monday of being ‘snake-oil salesmen’ who promote products with no scientific basis”, and that the dispute “had cost him his job – a claim Prince Charles’s office denied”.[14][26] Ernst is a republican, and has supported Republic, an organisation which campaigns for the abolition of the British monarchy.[27]
Re-reading it yesterday, I noticed that the text is not entirely correct (a full account can be found here). Let me explain:
- There never was a formal confidentiality agreement with signature etc. But I did feel bound to keep the contents of the Smallwood report confidential.
- The investigation by my University was not just ‘very unpleasant’, it was also far too long. It lasted 13 months! I had to take lawyers against my own University!
- In addition, it was unnecessary, not least because a University should simply establish the facts and, if reasonable, defend its professor from outside attacks. The facts could have been established over a cup of tea with the Vice Chancellor in less than half an hour.
- When my department had been destroyed in the process, I retired voluntarily and was subsequently re-employed for half a year to help find a successor. In retrospect, I see this move as a smart ploy by the University to keep me sweet and prevent me from going to the press.
- A successor was never hired; one good candidate was found but he was told that he had to find 100% of the funds to do the job. Nobody of high repute would have found this acceptable, and thus the only good candidate was not even tempted to accept the position.
- The snake oil salesman story is an entirely separate issue (see here) that happened years later.
- It is true that Charles’s office denied that Charles knew about his 1st private secretary writing to my Vice Chancellor asking him to investigate my alleged breach of confidence. However, as Sir Michael Peat started his letter with the words “I AM WRITING … AS THE PRINCE OF WALES’ PRIVATE SECRETARY…, I find this exceedingly hard to believe.
- Even though Charles did a sterling job in trying, I did not become a republican. I do have considerable doubts that Charles will be a good King (his reign might even be the end of the monarchy), and I did help the republican cause on several occasions but I never formally joined any such group (in general, I am not a joiner of parties, clubs or interest groups).
To one of the journalists who recently interviewed me, I explained that I do not in the slightest feel sore, bitter, or angry on a personal level. Going into early retirement suited me perfectly fine, and thanks to that decision I enjoy life to the full. The significance of this story lies elsewhere: Charles’ intervention managed to permanently close the then worldwide-only department that systematically and critically investigated so-called alternative medicine. If you know another, please let me know.
It is not often that I publish a paper with a philosopher in a leading journal of philosophy. In fact, it is the first time, and I am rather proud of it – so much so that I must show my readers (the article is freely available via the link below and I encourage everyone to read the full text) the abstract of our article entitled WHY HOMOEOPATHY IS PSEUDOSCIENCE (Synthese (2022) 200:394):
Homoeopathy is commonly recognised as pseudoscience. However, there is, to date, no systematic discussion that seeks to establish this view. In this paper, we try to fill this gap. We explain the nature of homoeopathy, discuss the notion of pseudoscience, and provide illustrative examples from the literature indicating why homoeopathy fits the
bill. Our argument contains a conceptual and an empirical part.
In the conceptual part, we introduce the premise that a doctrine qualifies as a pseudoscience if, firstly, its proponents claim scientific standing for it and, secondly, if they produce bullshit to defend it, such that, unlike science, it cannot be viewed as the most reliable knowledge on its topic. In the empirical part, we provide evidence that homoeopathy fulfils both criteria. The first is quickly established since homoeopaths often explicitly claim scientificity.
To establish the second, we dive into the pseudo-academic literature on homoeopathy to provide evidence of bullshit in the arguments of homoeopaths. Specifically, we show that they make bizarre ontological claims incompatible with natural science, illegitimately shift the burden of proof to sceptics, and mischaracterise, cherry-pick, and misreport the evidence. Furthermore, we demonstrate that they reject essential parts of established scientific methodology and use epistemically unfair strategies to immunise their doctrine against recalcitrant evidence.
And here is our conclusion:
At the beginning of the paper, we noted that homoeopathy is commonly named one of the prototypical pseudosciences. However, there has been, to date, no comprehensive discussion as to what makes it a pseudoscience. Moreover, the problem is not trivial since the most well-known and influential demarcation criteria, such as Popper’s falsifiability criterion and Kuhn’s problem-solving criterion, cannot account for it, as we have shown. We have tried to fill this research gap using a novel bullshitology-based approach to the demarcation problem. Following this approach, we have argued that homoeopathy should be regarded as pseudoscience because its proponents claim scientific standing for it and produce argumentative bullshit to defend it, thus violating important epistemic standards central to science.
Earlier this year, I started the ‘WORST PAPER OF 2022 COMPETITION’. As a competition without a prize is no fun, I am offering the winner (that is the lead author of the winning paper) one of my books that best fits his/her subject. I am sure this will overjoy him or her.
And how do we identify the winner? I will continue blogging about nominated papers (I hope to identify about 10 in total), and towards the end of the year, I let my readers decide democratically.
In this spirit of democratic voting, let me suggest to you ENTRY No 8 (it is so impressive that I must show you the unadulterated abstract):
Introduction
Female sexual dysfunction (FSD) seriously affects the quality of life of women. However, most women do not have access to effective treatment.
Aim
This study aimed to determine the feasibility and effectiveness of the use of acupuncture in FSD treatment based on existing clear acupuncture protocol and experience-supported face-to-face therapy.
Methods
A retrospective analysis was performed on 24 patients with FSD who received acupuncture from October 2018 to February 2022. The Chinese version of the female sexual function index , subjective sensation, sexual desire, sexual arousal, vaginal lubrication, orgasm, sexual satisfaction, and dyspareunia scores were compared before and after the treatment in all 24 patients.
Main Outcome Measure
A specific female sexual function index questionnaire was used to assess changes in female sexual function before and after the acupuncture treatment.
Results
In this study, the overall treatment improvement rate of FSD was 100%. The Chinese version of the female sexual function index total score, sexual desire score, sexual arousal score, vaginal lubrication score, orgasm score, sexual satisfaction score, and dyspareunia score during intercourse were significantly different before and after the treatment (P < .05). Consequently, participants reported high levels of satisfaction with acupuncture. This study indicates that acupuncture could be a new and effective technique for treating FSD. The main advantages of this study are its design and efficacy in treating FSD. To the best of our knowledge, this is the first study to evaluate the efficacy of acupuncture in the treatment of FSD using the female sexual function index scale from 6 dimensions. The second advantage is that the method used (ie, the nonpharmacological method) is simple, readily available, highly safe with few side effects, and relatively inexpensive with high patient satisfaction. However, limitations include small sample size and lack of further detailed grouping, pre and post control study of patients, blank control group, and pre and post control study of sex hormones.
Conclusion
Acupuncture can effectively treat FSD from all dimensions with high safety, good satisfaction, and definite curative effect, and thus, it is worthy of promotion and application.
My conclusion is very different: acupuncture can effectively kill any ability for critical thinking.
I hardly need to list the flaws of this paper – they are all too obvious, e.g.:
- there is no control group; the results might therefore be due to a host of factors that are unrelated to acupuncture,
- the trial was too small to allow far-reaching conclusions,
- the study does not tell us anything about the safety of acupuncture.
The authors call their investigation a ‘pilot study’. Does that excuse the flimsiness of their effort? No! A pilot study cannot draw conclusions such as the above.
What’s the harm? you might ask; nobody will ever read such rubbish and nobody will have the bizarre idea to use acupuncture for treating FSD. I’m afraid you would be wrong to argue in this way. The paper already got picked up by THE DAILY MAIL in an article entitled “Flailing libido? Acupuncture could help boost sex drive, scientists say” which was as devoid of critical thinking as the original study. Thus we can expect that hundreds of desperate women are already getting needled and ripped off as we speak. And in any case, offensively poor science is always harmful; it undermines public trust in research (and it renders acupuncture research the laughing stock of serious scientists).
This study aimed to evaluate the efficacy of Persian barley water in controlling the clinical outcomes of hospitalized COVID-19 patients. It was designed as a single-blind, add-on therapy, randomized controlled clinical trial and conducted in Shiraz, Iran, from January to March 2021. One hundred hospitalized COVID-19 patients with moderate disease severity were randomly allocated to receive routine treatment (per local protocols) with or without 250 ml of Persian barley water (PBW) daily for two weeks. Clinical outcomes and blood tests were recorded before and after the study period. Multivariable modeling was applied using Stata software for data analysis.
The length of hospital stay (LHS) was 4.5 days shorter in the intervention group than the control group regardless of history of cigarette smoking (95% confidence interval: -7.22, -1.79 days). Also, body temperature, erythrocyte sedimentation rate (ESR), C-reactive protein (CRP), and creatinine significantly dropped in the intervention group compared to the control group. No adverse events related to PBW occurred.
The authors from the Department of Traditional Medicine, Shiraz University of Medical Sciences, Shiraz, Iran, concluded that this clinical trial demonstrated the efficacy of PBW in minimizing the LHS, fever, and levels of ESR, CRP, and creatinine among hospitalized COVID-19 patients with moderate disease severity. More robust trials can help find safe and effective herbal formulations as treatments for COVID-19.
I must admit, I did not know about PBW. The authors explain that PBW is manufactured from Hordeum vulgare via a specific procedure. According to recent studies, barley is rich in constituents such as selenium, tocotrienols, phytic acid, catechin, lutein, vitamin E, and vitamin C; these compounds are responsible for their antioxidant and anti-inflammatory properties. Barley grains also have immune-stimulating effects, antioxidant properties, protective effects on the liver and digestive systems, anti-cancer effects, and act to reduce uric acid levels.
But even if these effects would constitute a plausible mechanism for explaining the observed effects (which I do not think they do), the study itself is more than flimsy.
I do not understand why researchers investigating an important issue do not make sure that their study is as rigorous as possible.
- Why not use an adequately large sample size?
- Why not employ a placebo?
- Why not double-blind?
- Why not report the most important outcome, i.e. mortality?
As it stands, nobody will take this study seriously. Perhaps this is a good thing – but perhaps PBW does have positive effects (I know it’s a long shot) and, in this case, a poor-quality study would only prevent an effective therapy come to light.
It has been reported that a recent inspection from the Care Quality Commission (CQC) found that the diagnostic imaging service at AECC University College in Parkwood Road, Bournemouth, requires improvement in three out of four areas – including patient safety. This is surprising not least because the AECC prides itself on being “a leading higher education institution in healthcare disciplines, nationally and internationally recognised for quality and excellence.” 
The unannounced inspection in May this year resulted in several demands for the service to improve upon. For example, the CQC report said staff “did not receive all of the training they needed to keep patients safe” and that patient chaperones “did not receive chaperone training”. Moreover, managers were reported as not always ensuring staff were competent to operate certain equipment. In fact, there was no record of staff competencies which meant inspectors “could not tell if staff had been trained to use equipment”. General cleanliness was also found lacking in relation to certain procedures, namely no sink in any of the site’s nine ultrasound rooms (including those for transvaginal scans) – meaning staff carrying out ultrasound scanning did not have access to a clinical handwashing facility.
The CQC states on its website that it “is the independent regulator of health and adult social care in England. We make sure health and social care services provide people with safe, effective, compassionate, high-quality care and we encourage care services to improve. We monitor, inspect and regulate services. Then we publish what we find, including performance ratings, to help people choose care. Where we find poor care, we will use our powers to take action.”
No doubt, these are laudable aims. What I find, however, disappointing is that the CQC’s inspection of the AECC did not question the nature of some of the courses taught by the AECC. Earlier this year, I reported in a blog post that the AECC has announced a new MSc ‘Musculoskeletal Paediatric Health‘. This motivated me to look into the evidence for such a course. This is what I found with several Medline searches (date of the review on chiropractic for any pediatric conditions, followed by its conclusion + link [so that the reader can look up the evidence]):
2008
I am unable to find convincing evidence for any of the above-named conditions.
2009
Previous research has shown that professional chiropractic organisations ‘make claims for the clinical art of chiropractic that are not currently available scientific evidence…’. The claim to effectively treat otitis seems to
be one of them. It is time now, I think, that chiropractors either produce the evidence or abandon the claim.
2009
The … evidence is neither complete nor, in my view, “substantial.”
2010
2018
What seems to emerge is rather disappointing:
- There are no really new reviews.
- Most of the existing reviews are not on musculoskeletal conditions.
- All of the reviews cast considerable doubt on the notion that chiropractors should go anywhere near children.
But perhaps I was too ambitious. Perhaps there are some new rigorous clinical trials of chiropractic for musculoskeletal conditions. A few further searches found this (again year and conclusion):
2019
2018
I might have missed one or two trials because I only conducted rather ‘rough and ready’ searches, but even if I did: would this amount to convincing evidence? Would it be good science?
No! and No!
So, why does the AECC offer a Master of Science in ‘Musculoskeletal Paediatric Health’?
____________________
Isn’t that a question the CQC should have asked?
Olivia Newton-John, actress, singer, and advocate of so-called alternative medicine (SCAM) has died following a lengthy battle with breast cancer. Her husband announced her death yesterday: “Dame Olivia Newton-John (73) passed away peacefully at her Ranch in Southern California this morning, surrounded by family and friends,” the post read. “We ask that everyone please respect the family’s privacy during this very difficult time. Olivia has been a symbol of triumphs and hope for over 30 years sharing her journey with breast cancer. Her healing inspiration and pioneering experience with plant medicine continues with the Olivia Newton-John Foundation Fund, dedicated to researching plant medicine and cancer.” 
Olivia was born on 26 September 1948 in Cambridge, UK. She came from a remarkable family. Her maternal grandfather was the Nobel Prize-winning physicist Max Born. She was thus the niece of my late friend Gustav Born. Newton-John’s father was an MI5 officer on the Enigma project at Bletchley Park who took Rudolf Hess into custody during World War II. After the war, he became the headmaster of the Cambridgeshire High School for Boys. He then took up a post in Australia, and young Olivia grew up down under. After starting out as a singer, she had her breakthrough with the film ‘Grease’ which brought her world fame.
Olivia was first diagnosed with breast cancer over 30 years ago and became an outspoken advocate of SCAM. Her cancer came back twice, and in 2017, she was diagnosed to have bone metastases. Meanwhile, she had married John Easterling, the boss of a natural remedy company, in an Incan spiritual ceremony in Peru.
In 2017, she said, “I decided on my direction of therapies after consultation with my doctors and natural therapists and the medical team at my Olivia Newton-John Cancer Wellness and Research Centre in Melbourne”. The Olivia Newton-John Cancer Wellness & Research Centre is a treatment centre of Austin Health, an Australian public hospital. They say that “anyone with a referral from their doctor can be treated here, regardless of the stage of their treatment or insurance status. At the ONJ Centre your care is built around your individual needs. This includes your physical, psychological and emotional health. Every patient is surrounded by a multidisciplinary team of cancer specialists, allied health and wellbeing therapists. Your dedicated treatment team work together to guide you through your optimal treatment pathway. Learn more about the cancer treatments we deliver at the ONJ Centre, how we support you through your care, and find answers to commonly asked questions.”
Their therapies include acupuncture and several other alternatives used for palliation, but the site seems refreshingly free of false claims and quackery. On their website, they say that “palliative care assists patients who have a life limiting illness to be as symptom free as possible. We work with you to meet your emotional, spiritual and practical needs in a holistic way. Our support is also extended to your family and carers.”
Olivia Newton-John’s history with SCAM is revealing. It seems that, by initially using SCAM instead of conventional treatments for her breast cancer in 1992, she worsened her prognosis. When the cancer returned, she opted for the best conventional oncology on offer. Yet, her liking for SCAM had not disappeared. Since 2017, she seems to have used cannabis and other SCAMs as add-ons to conventional medicine. Sadly, she had learned her lesson too late: alternative cancer treatments are a dangerous myth.
Earlier this year, I started the ‘WORST PAPER OF 2022 COMPETITION’. You will ask what is there to win in this competition? I agree: a competition without a prize is no fun. Therefore, I suggest offering the winner (that is the author of the winning paper) one of my books that best fits his/her subject. I am sure this will over-joy him or her. And how do we identify the winner? I suggest that I continue blogging about nominated papers (I hope to identify about 10 in total), and towards the end of the year, I let my readers decide democratically.
In this spirit of democratic voting, let me suggest to you ENTRY No 6:
This study was to ascertain the efficacy of dry cupping therapy (DCT) and optimal cup application time duration for cervical spondylosis (CS). It was designed as a randomized clinical trial involving 45 participants with clinically diagnosed CS. The eligible subjects were randomly allocated into three groups, each having 15 participants. Each of the three groups, i.e., A, B, and C, received DCT daily for 15 days for 8 min, 10 min, and 12 min, respectively. All the participants were evaluated at the baseline, 7th, and 15th day of the trial using the neck disability index (NDI) as well as the visual analog scale (VAS).
The baseline means ± SD of NDI and VAS scores were significantly reduced in all three groups at the end of the trial. Although all three groups were statistically equal in terms of NDI, group C demonstrated greater efficacy in terms of VAS.
The authors concluded that the per-protocol analysis showed that dry cupping effectively alleviated neck pain across all treatment groups. Although this effect on neck disability index was statistically equal in all three groups, the 12-min protocol was more successful in reducing pain.
Who would design such a study and why?
- The authors claim they wanted to ascertain the efficacy of DCT. A trial is for testing, not ascertaining. And this study does certainly not test for efficacy.
- The groups were too small to generate a meaningful result of what, in fact, was an equivalence study.
- Intra-group changes in symptoms between baseline and time points during treatment are irrelevant in a controlled trial.
- The slightly better results of group C are most likely due to chance or non-specific effects (a longer application of a placebo would generate better outcomes that a shorter one).
- The study participants had cervical spondylosis, yet the conclusion is about neck pain. The two are not identical.
- The title of the paper promises that we learn something about the safety of DCT. Sadly, a trial with just 45 patients has no chance in hell to pick up adverse effects in a reliable way.
- As there is no control group, the study cannot tell us anything about possible specific effects of DCT.
The authors of the study have impressive affiliations:
- Department of Ilaj bil Tadbir, Luqman Unani Medical College Hospital and Research Center, Bijapur, India.
- Department of Ilaj bil Tadbir, National Institute of Unani Medicine, Bengaluru, India.
- Department of Moalajat, Luqman Unani Medical College Hospital and Research Center, Bijapur, India.
I would have hoped that researchers from national institutions and medical colleges should be able to design a trial that has at least a small chance to produce a meaningful finding. As it turns out, my hope was badly disappointed.
I have been warning the public about the indirect dangers of so-called alternative medicine (SCAM) for a very long time. It is now 25 years ago, for instance, that I published an article in the ‘European Journal of Pediatrics’ entitled “The attitude against immunisation within some branches of complementary medicine“. Here is the discussion section of this paper:
… certain groupings within COMPLEMENTARY MEDICINE (CM) may advise their patients against immunisation. Within these groupings, there is, of course, a considerable diversity of attitudes towards immunisation. Therefore
generalisations are difficult and more detailed investigations are required to clarify the issue.
The question arises whether the level of advice against immunisation as it exists today represents a real or only a potential risk. One study from the U.K. demonstrates homoeopathy to be the most prevalent reason for non-compliance with immunisation [30]. The problem may not be confined to naturopathy, chiropractic and homoeopathy. Books relating to CM in general [e.g. 19] also strongly advise against immunisation: “Vaccination may provoke the illness which it is supposed to prevent. People who are vaccinated can transmit the illness, even if they are not ill themselves. The vaccine can make the person more susceptible to the illness … The vaccinated child is a contaminated child”.
At present, our data is insufficient to de®ne which proportion of which complementary practitioners share this
attitude. The origin of this stance against vaccination is largely unknown. For instance, there is nothing in Hahnemann’s writings against immunisation [14]. It may therefore stem from a general antipathy toward modern medicine which seems to be prevalent within CM [7, 19, 23]. A more specific reason is that immunisation is viewed as detrimental, burdened with long-term side effects. It is also felt that it is not fully effective and unnecessary because
better methods of protection exist within CM [16].
Anti-immunisation activists are often unable to argue their case rationally, yet they place advertisements in the daily press warning about immunisation. In Britain, one tragic case has recently been publicised. A physician advised parents against measles vaccination for their child who was suspected of suffering from convulsions. Five years later, the child suffered severe brain damage after contracting measles. The doctor was sued by the parents and found guilty of negligence and ordered to pay £825,000 in damages [1].
In medicine we must, of course, always be vigilant about the risks of our interventions. Each form of immunisation should therefore be continuously scrutinised for its possible risks and benefits. Most forms of immunisation are clearly not entirely free of risk [e.g. 22] – in fact, no effective intervention will ever be entirely risk-free. Therefore the risks have to be discounted against the benefits. It follows that any blanket rejection of immunisation, in general, must be misleading. It endangers not only the individual patient but (if prevalent) also the herd immunity of the community at large. Such unreflected rejection of immunisation, in general, will inevitably do more harm than good.
It is concluded that the advice of some, by no means all complementary practitioners in relation to immunisation represents an area for concern, which requires further research. Complementary practitioners and patients alike should be educated about the risks and benefits of immunisation. Paediatricians should be informed about the present negative attitude of some complementary practitioners and discuss the issue openly with their patients.
_____________________________
I suspect that, had we heeded my caution, researched the subject more thoroughly, and taken appropriate action, the current pandemic might have produced fewer and less vocal anti-vaxxers, and fewer patients might have died.
Crystal healing is the treatment of all types of illnesses via the ‘healing energy’ of gemstones. It is as implausible as so-called alternative medicine (SCAM) gets. In my recent analysis of 150 SCAMs, I concluded that “there are no rigorous trials testing the therapeutic value of crystal healing”. This assumption is further confirmed by published papers like this one: 
Recently, crystal healing and gemstone therapy, also known as litho- or gemmotherapy, is extensively promoted in the media, newspapers and the internet. There is also a growing interest of cancer patients in this unconventional treatment, resulting in the need for oncologists to give informed advice to their patients and to prevent them from wasting hope, time and money in an ineffective treatment, and at worst to postpone the necessary treatment of this life threatening disease. In the context of the currently ever-growing New-Age wave, believing in crystal healing has spread widely in the population. It is a historical belief similar to that of charmstones, rather than one based on modern scientific practices and advances. Pleasant feelings or seeming successes of crystal healing can be attributed to the strong placebo effect, or the believers wanting it to be true and seeing only things that back that up: cognitive bias. A scientific proof of any positive effect beyond a placebo effect does not exist. Even though this treatment can be generally regarded as harmless and without toxicity, it should not be recommended to cancer patients. Thereby we will help prevent our patients from wasting hope, time and money in an ineffective treatment, and at worst to postpone the necessary treatment of this life threatening disease, resulting in a worsened prognosis.
Yet, it seems that we were not entirely correct. Recently, I came across an article that mentions such a study:
A study conducted in 2001 by British psychologist Christopher French challenged 80 volunteers to differentiate between real and fake crystals after holding them in their hands for five minutes and meditating. Six people felt nothing at all, and the rest reported feeling some energy, whether in the form of tingling in the body or an improved sense of wellbeing. Both groups, though—whether holding the fake crystals or the real ones—reported similar impressions, suggesting the placebo effect could be at play.
“When scientists conduct robust clinical trials, they want to strip the intervention of these placebo effects to figure out if it has a specific benefit,” Jarry explains. “Alternative medicine’s reputation benefits strongly from these non-specific placebo effects. Enough people will start to feel better after using crystals (because of regression to the mean, self-limiting illness, misremembering, etc.), and they will publicly testify to their improvement, giving the illusion that crystals work. What they don’t know is what would have happened had they not used the crystals.”
So, if you want to keep a hunk of amethyst at your desk to alleviate your grief, or a Tiger’s Eye stone to clear the mind, go ahead: they may not be manipulating a sacred energy field around your body to heal you, but they can certainly manipulate your mind.
Unfortunately, the link provided does not lead directly to the study but to the publication list of Chris French. This in turn leads us to the reference in question:
French, C. C., O’Donnell, H., & Williams, L. (2001). Hypnotic susceptibility, paranormal belief and reports of ‘crystal power’. British Psychological Society Centenary Annual Conference, Glasgow, 28-31 March 2001. (Abstract published in Proceedings of the British Psychological Society, 9(2), 186).
Sadly, I cannot find the paper online and I suspect it exists only as an abstract in a conference book (I have emailed Chris and asked him). In any case, his study did not test the therapeutic value; so, my above statement is not entirely false.
A study by an international team of researchers estimated the proportion of healthcare interventions tested within Cochrane Reviews that are effective according to high-quality evidence.
They selected a random sample of 2428 (35%) of all Cochrane Reviews published between 1 January 2008 and 5 March 2021 and extracted data about interventions within these reviews that were compared with placebo, or no treatment, and whose outcome quality was rated using Grading of Recommendations Assessment, Development and Evaluation (GRADE). They then calculated the proportion of interventions whose effectiveness was based on high-quality evidence according to GRADE, had statistically significant positive effects and were judged as beneficial by the review authors. They also calculated the proportion of interventions that suggested harm.
Of 1567 eligible interventions, 87 (5.6%) had high-quality evidence on first-listed primary outcomes, positive, statistically significant results, and were rated by review authors as beneficial. Harms were measured for 577 (36.8%) interventions, 127 of which (8.1%) had statistically significant evidence of harm. Our dependence on the reliability of Cochrane author assessments (including their GRADE assessments) was a potential limitation of our study. 
The authors drew the following conclusions: While many healthcare interventions may be beneficial, very few have high-quality evidence to support their effectiveness and safety. This problem can be remedied by high-quality studies in priority areas. These studies should measure harms more frequently and more rigorously. Practitioners and the public should be aware that many frequently used interventions are not supported by high-quality evidence.
Proponents of so-called alternative medicine (SCAM) are fond of the ‘strawman’ fallacy; meaning they like to present a picture of conventional medicine that is overtly negative in order for SCAM to appear more convincing (Prince Charles, for instance, uses this trick every single time he speaks about SCAM). Therefore I am amazed that this paper has not been exploited in that way by them. I was expecting headlines such as
Evidence-based medicine is not supported by evidence
or
Less than 6% of all conventional treatments are supported by sound evidence.
etc.
Why did they not have a field day with this new paper then?
As the article is behind a paywall, it took me a while to get the full paper (thanks Paul). Now that I have read it, I think I understand the reason.
In the article, the authors provide figures for specific types of treatments. Let me show you some of the percentages of interventions that met the primary outcome (high quality, statistically significant effect, and authors interpret as effective):
- pharmacological 73.8%
- surgical 4.6%
- exercise 5.8%
- diet 1.2%
- alternative 0.0%
- manual therapies 0.0%
So, maybe the headlines should not be any of the above but:
No good evidence to support SCAM?
or
SCAM is destroying the evidence base of medicine.
