Some consumers believe that research is by definition reliable, and patients are even more prone to this error. When they read or hear that ‘RESEARCH HAS SHOWN…’ or that ‘A RECENT STUDY HAS DEMONSTRATED…’, they usually trust the statements that follow. But is this trust in research and researchers justified? During 25 years that I have been involved in so-called alternative medicine (SCAM), I have encountered numerous instances which make me doubt. In this post, I will briefly discuss some the many ways in which consumers can be mislead by apparently sound evidence (for an explanation as to what is and what isn’t evidence, see here).
ABSENCE OF EVIDENCE
I have just finished reading a book by a German Heilpraktiker that is entirely dedicated to SCAM. In it, the author makes hundreds of statements and presents them as evidence-based facts. To many lay people or consumers, this will look convincing, I am sure. Yet, it has one fatal defect: the author fails to offer any real evidence that would back up his statements. The only references provided were those of other books which are equally evidence-free. This popular technique of making unsupported claims allows the author to make assertions without any checks and balances. A lay person is usually unable or unwilling to differentiate such fabulations from evidence, and this technique is thus easy and poular for misleading us about SCAM.
On this blog, we have encountered this phenomenon ad nauseam: a commentator makes a claim and supports it with some seemingly sound evidence, often from well-respected sources. The few of us who bother to read the referenced articles quickly discover that they do not say what the commentator claimed. This method relies on the reader beeing easily bowled over by some pretend-evidence. As many consumers cannot be bothered to look beyond the smokescreen supplied by such pretenders, the method usually works surprisingly well.
An example: Vidatox is a homeopathic cancer ‘cure’ from Cuba. The Vidatox website clains that it is effective for many cancers. Considering how sensational this claim is, one would expect to find plenty of published articles on Vidatox. However, a Medline search resulted in one paper on the subject. Its authors drew the following conclusion: Our results suggest that the concentration of Vidatox used in the present study has not anti-neoplastic effects and care must be taken in hiring Vidatox in patients with HCC.
The question one often has to ask is this: where is the line between misleading research and fraud?
There is no area in healthcare that produces more surveys than SCAM. About 500 surveys are published every year! This ‘survey-mania’ has a purpose: it promotes a positive message about SCAM which hypothesis-testing research rarely does.
For a typical SCAM survey, a team of enthusiastic researchers might put together a few questions and design a questionnaire to find out what percentage of a group of individuals have tried SCAM in the past. Subsequently, the investigators might get one or two hundred responses. They then calculate simple descriptive statistics and demonstrate that xy % use SCAM. This finding eventually gets published in one of the many third-rate SCAM journals. The implication then is that, if SCAM is so popular, it must be good, and if it’s good, the public purse should pay for it. Few consumers would realise that this conclusion is little more that a fallacious appeal to popularity.
AVOIDING THE QUESTION
Another popular way of SCAM researchers to mislead the public is to avoid the research questions that matter. For instance, few experts would deny that one of the most urgent issues in chiropractic relates to the risk of spinal manipulations. One would therefore expect that a sizable proportion of the currently published chiropractic research is dedicated to it. Yet, the opposite is the case. Medline currently lists more than 3 000 papers on ‘chiropractic’, but only 17 on ‘chiropractic, harm’.
A pilot study is a small scale preliminary study conducted in order to evaluate feasibility, time, cost, adverse events, and improve upon the study design prior to performance of a full-scale research project. Yet, the elementary preconditions are not fulfilled by the plethora of SCAM pilot studies that are currently being published. True pilot studies of SCAM are, in fact, very rare. The reason for the abundance of pseudo-pilots is obvious: they can easily be interpreted as showing encouragingly positive results for whatever SCAM is being tested. Subsequently, SCAM proponents can mislead the public by claiming that there are plenty of positive studies and therefore their SCAM is supported by sound evidence.
As regularly mentioned on this blog, there are several ways to design a study such that the risk of producing a negative result is minimal. The most popular one in SCAM research is the ‘A+B versus B’ design. In this study, for instance, cancer patients who were suffering from fatigue were randomised to receive usual care or usual care plus regular acupuncture. The researchers then monitored the patients’ experience of fatigue and found that the acupuncture group did better than the control group. The effect was statistically significant, and an editorial in the journal where it was published called this evidence “compelling”. Due to a cleverly over-stated press-release, news spread fast, and the study was celebrated worldwide as a major breakthrough in cancer-care.
Imagine you have an amount of money A and your friend owns the same sum plus another amount B. Who has more money? Simple, it is, of course your friend: A+B will always be more than A [unless B is a negative amount]. For the same reason, such “pragmatic” trials will always generate positive results [unless the treatment in question does actual harm]. Treatment as usual plus acupuncture is more than treatment as usual alone, and the former is therefore more than likely to produce a better result. This will be true, even if acupuncture is a pure placebo – after all, a placebo is more than nothing, and the placebo effect will impact on the outcome, particularly if we are dealing with a highly subjective symptom such as fatigue.
A more obvious method for generating false positive results is to omit blinding. The purpose of blinding the patient, the therapist and the evaluator of the group allocation in clinical trials is to make sure that expectation is not a contributor to the result. Expectation might not move mountains, but it can certainly influence the result of a clinical trial. Patients who hope for a cure regularly do get better, even if the therapy they receive is useless, and therapists as well as evaluators of the outcomes tend to view the results through rose-tinted spectacles, if they have preconceived ideas about the experimental treatment.
Failure to randomise is another source of bias which can mislead us. If we allow patients or trialists to select or chose which patients receive the experimental and which get the control-treatment, it is likely that the two groups differ in a number of variables. Some of these variables might, in turn, impact on the outcome. If, for instance, doctors allocate their patients to the experimental and control groups, they might select those who will respond to the former and those who don’t to the latter. This may not happen with intent but through intuition or instinct: responsible health care professionals want those patients who, in their experience, have the best chances to benefit from a given treatment to receive that treatment. Only randomisation can, when done properly, make sure we are comparing comparable groups of patients. Non-randomisation can easily generate false-positive findings.
It is also possible to mislead people with studies which do not test whether an experimental treatment is superior to another one (often called superiority trials), but which assess whether it is equivalent to a therapy that is generally accepted to be effective. The idea is that, if both treatments produce similarly positive results, both must be effective. Such trials are called non-superiority or equivalence trials, and they offer a wide range of possibilities for misleading us. If, for example, such a trial has not enough patients, it might show no difference where, in fact, there is one. Let’s consider a simple, hypothetical example: someone comes up with the idea to compare antibiotics to acupuncture as treatments of bacterial pneumonia in elderly patients. The researchers recruit 10 patients for each group, and the results reveal that, in one group, 2 patients died, while, in the other, the number was 3. The statistical tests show that the difference of just one patient is not statistically significant, and the authors therefore conclude that acupuncture is just as good for bacterial infections as antibiotics.
Even trickier is the option to under-dose the treatment given to the control group in an equivalence trial. In the above example, the investigators might subsequently recruit hundreds of patients in an attempt to overcome the criticism of their first study; they then decide to administer a sub-therapeutic dose of the antibiotic in the control group. The results would then seemingly confirm the researchers’ initial finding, namely that acupuncture is as good as the antibiotic for pneumonia. Acupuncturists might then claim that their treatment has been proven in a very large randomised clinical trial to be effective for treating this condition. People who do not happen to know the correct dose of the antibiotic could easily be fooled into believing them.
Obviously, the results would be more impressive, if the control group in an equivalence trial received a therapy which is not just ineffective but actually harmful. In such a scenario, even the most useless SCAM would appear to be effective simply because it is less harmful than the comparator.
A variation of this theme is the plethora of controlled clinical trials in SCAM which compare one unproven therapy to another unproven treatment. Perdicatbly, the results would often indicate that there is no difference in the clinical outcome experienced by the patients in the two groups. Enthusiastic SCAM researchers then tend to conclude that this proves both treatments to be equally effective. The more likely conclusion, however, is that both are equally useless.
Another technique for misleading the public is to draw conclusions which are not supported by the data. Imagine you have generated squarely negative data with a trial of homeopathy. As an enthusiast of homeopathy, you are far from happy with your own findings; in addition you might have a sponsor who puts pressure on you. What can you do? The solution is simple: you only need to highlight at least one positive message in the published article. In the case of homeopathy, you could, for instance, make a major issue about the fact that the treatment was remarkably safe and cheap: not a single patient died, most were very pleased with the treatment which was not even very expensive.
A further popular method for misleading the public is the outright omission findings that SCAM researchers do not like. If the aim is that the public believe the myth that all SCAM is free of side-effects, SCAM researchers only need to omit reporting them in clinical trials. On this blog, I have alerted my readers time and time again to this common phenomenon. We even assessed it in a systematic review. Sixty RCTs of chiropractic were included. Twenty-nine RCTs did not mention adverse effects at all. Sixteen RCTs reported that no adverse effects had occurred. Complete information on incidence, severity, duration, frequency and method of reporting of adverse effects was included in only one RCT.
Most trails have many outcome measures; for instance, a study of acupuncture for pain-control might quantify pain in half a dozen different ways, it might also measure the length of the treatment until pain has subsided, the amount of medication the patients took in addition to receiving acupuncture, the days off work because of pain, the partner’s impression of the patient’s health status, the quality of life of the patient, the frequency of sleep being disrupted by pain etc. If the researchers then evaluate all the results, they are likely to find that one or two of them have changed in the direction they wanted (especially, if they also include half a dozen different time points at which these variables are quatified). This can well be a chance finding: with the typical statistical tests, one in 20 outcome measures would produce a significant result purely by chance. In order to mislead us, the researchers only need to “forget” about all the negative results and focus their publication on the ones which by chance have come out as they had hoped.
When it come to fraud, there is more to chose from than one would have ever wished for. We and others have, for example, shown that Chinese trials of acupuncture hardly ever produce a negative finding. In other words, one does not need to read the paper, one already knows that it is positive – even more extreme: one does not need to conduct the study, one already knows the result before the research has started. This strange phenomenon indicates that something is amiss with Chinese acupuncture research. This suspicion was even confirmed by a team of Chinese scientists. In this systematic review, all randomized controlled trials (RCTs) of acupuncture published in Chinese journals were identified by a team of Chinese scientists. A total of 840 RCTs were found, including 727 RCTs comparing acupuncture with conventional treatment, 51 RCTs with no treatment controls, and 62 RCTs with sham-acupuncture controls. Among theses 840 RCTs, 838 studies (99.8%) reported positive results from primary outcomes and two trials (0.2%) reported negative results. The percentages of RCTs concealment of the information on withdraws or sample size calculations were 43.7%, 5.9%, 4.9%, 9.9%, and 1.7% respectively. The authors concluded that publication bias might be major issue in RCTs on acupuncture published in Chinese journals reported, which is related to high risk of bias. We suggest that all trials should be prospectively registered in international trial registry in future.
A survey of clinical trials in China has revealed fraudulent practice on a massive scale. China’s food and drug regulator carried out a one-year review of clinical trials. They concluded that more than 80 percent of clinical data is “fabricated“. The review evaluated data from 1,622 clinical trial programs of new pharmaceutical drugs awaiting regulator approval for mass production. Officials are now warning that further evidence malpractice could still emerge in the scandal.
I hasten to add that fraud in SCAM research is certainly not confined to China. On this blog, you will find plenty of evidence for this statement, I am sure.
Research is obviously necessary, if we want to answer the many open questions in SCAM. But sadly, not all research is reliable and much of SCAM research is misleading. Therefore, it is always necessary to be on the alert and apply all the skills of critical evaluation we can muster.
The amount of different so-called alternative medicines (SCAMs) that are being tried or promoted against COVID-19 is legion. Anything really from vitamins to herbal remedies, homeopathics to chiropractic. In fact, it is hard these days to find a SCAM that is not touted for COVID-19.
This study aimed to evaluate if a dietary supplement of quercetin (a polyphenol contained in many fruit and vegetables), vitamin C and bromelain (a proteolytic enzyme contained in pineapple) could be protective against coronavirus infections.
In the verum group, a supplement containing
- 500mg of quercetin,
- 500mg of vitamin C,
- 50mg of bromelain (QCB)
was administered daily in 2 divided doses for 71 healthcare workers working in areas with high risk of COVID-19, whereas 42 were the control group who received no supplements. The maximum period of follow-up was 120 days. Termination of QCB use prematurely or having a coronavirus infection was the end of a volunteer’s study participation. A rapid diagnostic test was used to detect immunoglobulin positivity.
A total of 113 persons were included. No significant difference were detected between groups at baseline. Mean age of QCB group was 39.0 ± 8.8 years and control group was 32.9 ± 8.7. Average follow-up period for the QCB group was 113 days, and for the control group, 118 days. During the follow-up period, 1 healthcare worker in the QCB group and 9 out in 42 in control group contracted COVID-19. One case was asymptomatic, while others were not. Transmission risk hazard ratio of participants who did not receive QCB was 12.04 (95% Confidence interval= 1.26-115.06, P = 0.031). No significant effect of gender, smoking, antihypertensive medication exposure and having chronic disease on rate of transmission. The authors concluded that this study revealed that QCB was protective for healthcare workers.
The sudy is so poorly written and reported that I had trouble making sense of it. In fact, I first thought it was a fake. Then I saw this note:
Preprints with The Lancet is part of SSRN´s First Look, a place where journals identify content of interest prior to publication. Authors have opted in at submission to The Lancet family of journals to post their preprints on Preprints with The Lancet. The usual SSRN checks and a Lancet-specific check for appropriateness and transparency have been applied. Preprints available here are not Lancet publications or necessarily under review with a Lancet journal. These preprints are early stage research papers that have not been peer-reviewed. The findings should not be used for clinical or public health decision making and should not be presented to a lay audience without highlighting that they are preliminary and have not been peer-reviewed.
If the results are for real (because of the small sample size, the lack of a placebo-control, dozens of potential confounders, etc., the findings could easily false-positive), they would merit urgent replication in a larger, more rigorous trial.
Meanwhile I would be very sceptical about the validity of the results. The paper (it really is just a submission for publication in the Lancet; I am not even sure that it will be officially published and I don’t quite see why it is being made available to the public in this way) is too flimsy for words. Despite these warnings, it is likely that many consumers will fall for the claim that QCB was protective for healthcare workers.
In these pre-Xmas days, many homes will smell of cinnamon. It’s certainly a wonderful spice for creating an atmosphere. But ther are also other uses for ciannamon.
Current treatments for overactive bladder (OAB) have limited efficacy, low persistence and a high rate of adverse events commonly leading to treatment cessation in clinical practice. Clinicians in Asia commonly use traditional Chinese medicine as an alternative for OAB treatment despite it having uncertain efficacy and safety. To evaluate the efficacy and safety of cinnamon patch (CP) treatment for alleviating symptoms of OAB, this double-blind randomized, placebo-controlled trial was conducted.
The 6-week study was conducted in an outpatient setting; 66 subjects diagnosed as having OAB were enrolled and treated with a placebo (n=33) or CP (n=33). The OAB symptom score (OABSS) was selected as the primary end point, and a patient perception of bladder condition (PPBC), an urgency severity scale (USS), and post-voiding residual urine (PVR) volume were selected as secondary end points.
In total, 66 participants (40 women and 26 men), 60 years of age, were included in the intention-to-treat analyses. Baseline characteristics were comparable between the CP and placebo groups. Treatment with a CP showed statistically significant differences in reductions in OABSS scores, PPBC scores, and USS scores.
The authors concluded that compared to a placebo, treatment with CP might be considered an effective and safe complementary therapy for OAB. Further studies employing a positive control, different dosage forms, larger sample sizes, and longer treatment periods are warranted.
Cinnamon (Cinnamomum zeylanicum and Cinnamon cassia)belongs to the Lauraceae family. It contains manganese, iron, dietary fiber, and calcium as well as cinnamaldehyde, cinnamic acid, cinnamate, and numerous other components such as polyphenols and antioxidant, anti-inflammatory, antidiabetic, antimicrobial, anticancer effects. Several reports have dealt with the numerous properties of cinnamon in the forms of bark, essential oils, bark powder, and phenolic compounds, and each of these properties can play a key role in human health.
The new study is interesting and prompts me to ponder:
- Do the pharmacologically active ingredients of cinnamon pass the skin barrier in sufficient amounts to have any effect at all? Or perhaps it was the scent? In which case, this would have been a study of aromatherapy.
- Considering the typical scent of cinnamon, I find it hard to imagine that this study was truly double blind.
- Cinnamon is alleged to have antimicrobial, antiviral, antifungal, antioxidant, antitumor, antihypertensive, antilipemic, antidiabetic, gastroprotective, and immunomodulatory effects. I do wonder which, if any, of these are responsible for the observed clinical results of this trial.
- Cinnamon is known to sometimes lead to allergic reactions. I wonder whether this could be a problem when it is applied in patches.
So, for the time being, I think, I prefere cinnamon, the spice, to cinnamon, the medicine.
There is some encouraging evidence regarding the positive influence of vitamin D on COVID-19. But is it convincing? Is it causal? As always, it is worth looking at the totality of the reliable evidence.
In this systematic review and meta-analysis, the researchers analyze the association between vitamin D deficiency and COVID-19 severity. They conducted an analysis of the prevalence of vitamin D deficiency and insufficiency in people with the disease. Five online databases—Embase, PubMed, Scopus, Web of Science, ScienceDirect and pre-print Medrevix were searched. The inclusion criteria were observational studies measuring serum vitamin D in adult and elderly subjects with COVID-19. The main outcome was the prevalence of vitamin D deficiency in severe cases of COVID-19.
The researchers identified 1542 articles and 27 met their inclusion criteria. The results show that
- vitamin D deficiency was not associated with a higher chance of infection by COVID-19,
- severe cases of COVID-19 present 64% more vitamin D deficiency compared with mild cases,
- vitamin D concentration insufficiency increased hospitalization and mortality rates,
- There was a positive association between vitamin D deficiency and the severity of the disease.
The authors concluded that the results of the meta-analysis confirm the high prevalence of vitamin D deficiency in people with COVID-19, especially the elderly. We should add that vitamin D deficiency was not associated with COVID-19 infection. However, we observed a positive association between vitamin D deficiency and the severity of the disease. From this perspective, evaluating blood vitamin D levels could be considered in the clinical practice of health professionals. Moreover, vitamin D supplementation could be considered in patients with vitamin D deficiency and insufficiency, if they have COVID-19. However, there is no support for supplementation among groups with normal blood vitamin D values with the aim of prevention, prophylaxis or reducing the severity of the disease.
These are interesting findings, no doubt. They relate to associations, as the authors repeatedly stress in the text of the paper. They do not, however, signify cause and effect relationships. The principal outcome of this research should be a hypothesis that subsequently needs testing in clinical trials.
So, why on earth did the authors chose that seriously misleading title of their paper? It clearly implies a causal effect; and this can only be verified by conducting clinical trials. One such study has been published (as discussed here) and it concluded that administration of calcifediol may improve the clinical outcome of subjects requiring hospitalization for COVID-19.
My conclusion: it seems well worth conducting more and more rigorous clinical trials.
Despite reported widespread use of dietary supplements by cancer patients, few empirical data with regard to their safety or efficacy exist. Because of concerns that antioxidants could reduce the cytotoxicity of chemotherapy, a prospective study was carried out to evaluate associations between supplement use and breast cancer outcomes.
Patients with breast cancer randomly assigned to an intergroup metronomic trial of cyclophosphamide, doxorubicin, and paclitaxel were queried on their use of supplements at registration and during treatment (n =1,134). Cancer recurrence and survival were indexed at 6 months after enrollment.
There were indications that use of any antioxidant supplement (vitamins A, C, and E; carotenoids; coenzyme Q10) both before and during treatment was associated with an increased hazard of recurrence and, to a lesser extent, death. Relationships with individual antioxidants were weaker perhaps because of small numbers. For non-antioxidants, vitamin B12 use both before and during chemotherapy was significantly associated with poorer disease-free survival and overall survival. Use of iron during chemotherapy was significantly associated with recurrence as was use both before and during treatment. Results were similar for overall survival. Multivitamin use was not associated with survival outcomes.
The authors concluded that associations between survival outcomes and use of antioxidant and other dietary supplements both before and during chemotherapy are consistent with recommendations for caution among patients when considering the use of supplements, other than a multivitamin, during chemotherapy.
These data are interesting but, for a range of reasons, not compelling. There might have been several important confounding factors distorting the findings. Even though clinical and life-style variables were statistically adjusted for in this study, it might still be possible that supplement users and non-users were not comparable in impotant prognostic variables. Simply put, sicker patients might be more likely to use supplements and would then have worse outcomes not because of the supplements but their disease severity.
Moreover, it seems important to note that other research showed the opposite effects. For instance, a study prospectively examined the associations between antioxidant use after breast cancer (BC) diagnosis and BC outcomes in 2264 women. The cohort included women who were diagnosed with early stage, primary BC from 1997 to 2000 who enrolled, on average, 2 years postdiagnosis. Baseline data were collected on antioxidant supplement use since diagnosis and other factors. BC recurrence and mortality were ascertained, and hazard ratios (HRs) and 95% confidence intervals (CIs) were estimated.
Antioxidant supplement use after diagnosis was reported by 81% of women. Among antioxidant users, frequent use of vitamin C and vitamin E was associated with a decreased risk of BC recurrence. Vitamin E use was associated with a decreased risk of all-cause mortality. Conversely, frequent use of combination carotenoids was associated with increased risk of death from BC and all-cause mortality.
The authors concluded that frequent use of vitamin C and vitamin E in the period after BC diagnosis was associated with a decreased likelihood of recurrence, whereas frequent use of combination carotenoids was associated with increased mortality. The effects of antioxidant supplement use after diagnosis likely differ by type of antioxidant.
Yet another study provided limited support for the hypothesis that antioxidant supplements may reduce the risk of breast cancer recurrence or breast cancer-related mortality.
What is needed, it seems, is a systematic review of all these contradicting studies. A 2009 review is available of the associations between antioxidant supplement use during breast cancer treatment and patient outcomes.
Inclusion criteria were: two or more subjects; clinical trial or observational study design; use of antioxidant supplements (vitamin C, vitamin E, antioxidant combinations, multivitamins, glutamine, glutathione, melatonin, or soy isoflavones) during chemotherapy, radiation therapy, and/or hormonal therapy for breast cancer as exposures; treatment toxicities, tumor response, recurrence, or survival as outcomes.
A total of 22 articles met the criteria. Their findings did not support any conclusions regarding the effects of individual antioxidant supplements during conventional breast cancer treatment on toxicities, tumor response, recurrence, or survival. A few studies suggested that antioxidant supplements might decrease side effects associated with treatment, including vitamin E for hot flashes due to hormonal therapy and glutamine for oral mucositis during chemotherapy. Underpowered trials suggest that melatonin may enhance tumor response during treatment.
The authors concluded that the evidence is currently insufficient to inform clinician and patient guidelines on the use of antioxidant supplements during breast cancer treatment. Thus, well designed clinical trials and observational studies are needed to determine the short- and long-term effects of such agents.
Antioxidants seem to have evolved as parts of elaborate networks in which each substance plays slightly different roles. This means that each antioxidant has a different spectrum of actions. And this means that it is probably not very constructive to lump them all together and excect to see uniform effects. What we would need to create more clarity is a series of RCTs on single antioxidants. But who is going to fund them? We might be waiting a long time for more clarity. Meanwhile, consuming a healthy and well-balanced diet might be the best advice for cancer patients and everyone else.
Many experts doubt that acupuncture generates the many positive health effects that are being claimed by enthusiasts. Yet, few consider that acupuncture might not be merely useless but could even make things worse. Here is a trial that seems to suggest exactly that.
This study evaluated whether combining two so-called alternative medicines (SCAMs), acupuncture and massage, reduce postoperative stress, pain, anxiety, muscle tension, and fatigue more than massage alone.
Patients undergoing autologous tissue breast reconstruction were randomly assigned to one of two postoperative SCAMs for three consecutive days. All participants were observed for up to 3 months. Forty-two participants were recruited from January 29, 2016 to July 11, 2018. Twenty-one participants were randomly assigned to massage alone and 21 to massage and acupuncture. Stress, anxiety, relaxation, nausea, fatigue, pain, and mood (score 0-10) were measured at enrollment before surgery and postoperative days 1, 2, and 3 before and after the intervention. Patient satisfaction was evaluated.
Stress decreased from baseline for both Massage-Only Group and Massage+Acupuncture Group after each treatment intervention. Change in stress score from baseline decreased significantly more in the Massage-Only Group at pretreatment and posttreatment. After adjustment for baseline values, change in fatigue, anxiety, relaxation, nausea, pain, and mood scores did not differ between groups. When patients were asked whether they would recommend the study, 100% (19/19) of Massage-Only Group and 94% (17/18) of Massage+Acupuncture Group responded yes.
The authors concluded tha no additive beneficial effects were observed with addition of acupuncture to massage for pain, anxiety, relaxation, nausea, fatigue, and mood. Combined massage and acupuncture was not as effective in reducing stress as massage alone, although both groups had significant stress reduction. These findings indicate a need for larger studies to explore these therapies further.
I recently went to the supermarket to find out whether combining two bank notes (£10 + £5) can buy more goods than one £10 note alone. What I found was interesting: the former did indeed purchase more than the latter. Because I am a scientist, I did not stop there; I went to a total of 10 shops and my initial finding was confirmed each time: A+B results in more than A alone.
It stands to reason that the same thing happens with clinical trials. We even tested this hypothesis in a systematic review entitled ‘A trial design that generates only ”positive” results‘. Here is our abstract:
In this article, we test the hypothesis that randomized clinical trials of acupuncture for pain with certain design features (A + B versus B) are likely to generate false positive results. Based on electronic searches in six databases, 13 studies were found that met our inclusion criteria. They all suggested that acupuncture is effective (one only showing a positive trend, all others had significant results). We conclude that the ‘A + B versus B’ design is prone to false positive results and discuss the design features that might prevent or exacerbate this problem.
But why is this not so with the above-mentioned study?
Why is, in this instance, A even more that A+B?
There are, of course, several possible answers. To use my supermarket example again, the most obvious one is that B is not a £5 note but a negative amount, a dept note, in other words: A + B can only be less than A alone, if B is a minus number. In the context of the clinical trail, this means acupuncture must have caused a negative effect.
But is that possible? Evidently yes! Many patients don’t like needles and experience stress at the idea of a therapist sticking one into their body. Thus acupuncture would cause stress, and stress would have a negative effect on all the other parameters quantified in the study (pain, anxiety, muscle tension, and fatigue).
My conclusion: in certain situations, acupuncture is more than just useless; it makes things worse.
Alzheimer is a devastating condition. Despite much research, we are still far from being able to effectively prevent or treat it. Some claim that relatively simple dietary interventions might work. What does the evidence tell us?
The aim of this systematic review was to evaluate the effect of dietary interventions on the cognitive performance of individuals with Alzheimer’s disease (AD). Thirty-two RCT could be included.
The findings show that a wide range of supplements have been submitted to testing in RCTs. Most of the supplements seem to be less than useful. However, some seem to show some promise:
- Omega-3 fatty acid has positive effects at different doses.
- ‘Fortasyn Connect’ (a multi-nutrient mixture) seems to be effective in the early stages of the disease.
- Probiotic, Ginseng, Inositol and specialized nutritional formulas seem to have a positive effect on cognition.
Most of the primary studies had poor methodological quality, included patients with mild AD, small samples, and did not obtain significative results for all the cognitive outcomes.
The authors concluded that the effect of most dietary interventions on cognition in AD patients remains inconclusive, however, several nutrients, isolated or not, show potential to improve cognitive function in AD, especially in its early stages.
I am relieved that the authors of this thoroughly-researched review phrased their conclusions as cautiously as they did. The thing is, most of the primary trials are truly not worth writing home about. Some are just 4 weeks long, others include merely 30 odd patients. Many look more like marketing excercises than science.
The authors also stated that better quality studies are urgently needed to confirm the therapeutic potential of the diet so that a dietary recommendation in AD that contributes to the quality of life of patients and relatives can be established. This has become almost a standard sentence for ending a scientific paper. In this instance, however, it seems very true.
Some of us got used to the idea that acupuncture might be effective for pain. But could it work for infections? Unlikely! Well, let’s not rely on gut feelings; let’s have a fair and critical look at the evidence.
This systematic review assessed the evidence for acupuncture for uncomplicated recurrent urinary tract infections (rUTI) women. Five randomised controlled trials (RCTs) evaluating the effects of acupuncture and related therapies for prophylaxis or treatment of uncomplicated rUTI in women were included. The methodological quality of the studies and the strength of the evidence were low to moderate. The chance of achieving a composite cure with acupuncture therapies was greater than that with antibiotics (three studies, 170 participants, RR 1.92, 95% CI 1.31‐2.81, I2 = 38%). The risk of UTI recurrence was lower with acupuncture than with no treatment (two studies, 135 participants, RR 0.39, 95% CI 0.26–0.58, I2 = 0%) and sham acupuncture (one study, 53 participants, RR 0.45, 95% CI 0.22–0.92).
The authors concluded that acupuncture showed promising results compared to no treatment and sham acupuncture in reducing recurrence, based on low to moderate certainty evidence. Low certainty evidence found acupuncture increased the chance of achieving a composite cure compared to antibiotics. Findings from this review should be interpreted with caution, taking into consideration the biases identified and small sample size of the included trials. Included studies suggest acupuncture has a good safety profile for women with UTI, and may be considered as a therapeutic option in the treatment and prevention of rUTI in women, particularly those who are unresponsive to, or intolerant of, antibiotics. Rigorously designed research is needed to inform clinical decisionmaking about the use of acupuncture for women with UTIs.
The authors of this review are affiliated to the Second Affiliated Hospital of Guangzhou University of Chinese Medicine (Guangdong Provincial Hospital of Chinese Medicine), Guangdong Provincial Academy of Chinese Medical Sciences and The Second Clinical College, Guangzhou University of Chinese Medicine, Guangzhou, China and the China-Australia International Research Centre for Chinese Medicine, School of Health and Biomedical Sciences, RMIT University, Melbourne, VIC, Australia. The review was funded by the China-Australia International Research Centre for Chinese Medicine (CAIRCCM) (International Cooperation Project, Grant Number 2012DFA31760), and the National Natural Science Foundation of China (NSFC) (Grant Number 81873261). In view of these facts, it is strange, I think, that the authors declared no conflicts of interest.
The 5 primary studies included in this review are the following:
- Alraek T, Soedal LI, Fagerheim SU, Digranes A, Baerheim A. Acupuncture treatment in the
prevention of uncomplicated recurrent lower urinary tract infections in adult women. American journal of
public health. 2002;92(10):1609-11.
- Aune A, Alraek T, LiHua H, Baerheim A. Acupuncture in the prophylaxis of recurrent lower urinary
tract infection in adult women. Scandinavian journal of primary health care. 1998;16(1):37-9.
- Hong JY, Li F, Liang XQ, Hou Z. [Efficacy observation on female chronic pyelonephritis treated with
abdominal cluster-needling therapy]. Zhongguo zhen jiu = Chinese acupuncture & moxibustion.
- Yu SM, Guo DD. Moxibustion combined with antibiotics was used to treat 30 cases of chronic
urinary tract infection in adult women. Shandong Journal of Traditional Chinese Medicine.
- Liu JL, Luo Q, Liu XH, Lin L. Observation on the clinical effect of external treatment of strong renal
moxibustion on recurrent urinary tract infection. China Modern Doctor. 2018;56(29):116-8.
- Alraek T, Soedal LI, Fagerheim SU, Digranes A, Baerheim A. Acupuncture treatment in the
As always, it is worth checking these studies for reliability.
In the trial by Alraek et al patients were randomised patients to receive either acupuncture or no treatment. Is anyone surprised that the former group fared better than the latter? (I am not!)
The trial by Aune et al is the only study that attempted to control for placebo effects by using a sham control group. This is what they used as a sham treatment: Sham acupuncture was given using six needles superficially inserted in the calves, thighs or abdomen outside known acupuncture points or meridians. Needles were not manipulated in the sham group. Sham controls have the purpose of rendering patients unaware whether they receive the real or the sham treatment. The method used here cannot achieve this aim; patients were easily able to determine that they were in the control group.
The last three trials are all not Medline-listed studies authored by Chinese investigators published in inaccessible journals in Chinese. We know that such studies invariably report positive outcomes which are often fabricated and thus have a reliability close to zero. But even if we ignore these facts for a moment, from what I see in the results table of the review, these studies are invalid. All three are equivalence trials of acupuncture versus antibiotics; with a sample size of merely around 30, they must be woefully underpowered and thus unable to generate a reliable result.
The authors of this review claim that the risk of bias of trials was generally high or unclear. This is an understatement to put it mildly. In fact, the quality of the studies was mostly dismal.
In view of all this, I take the liberty to re-formulate the conclusions drawn by the review authors as follows:
Due to the lack of reliable RCTs, the effectiveness of acupuncture as a treatment or prevention of rUTIs remains unproven. Due to the implausibility of the therapy, its effectiveness seems highly unlikely.
The BJOG should never have published such a deeply misleading paper.
Vitamin D and Omega-3 supplements help the elderly avoid Covid-19 infection by boosting their immune systems, study claims. Yes, that was the headline in the DAILY MAIL on 11/11/2020. Naturally, I found this interesting. So, I looked up the original paper. Here is its abstract:
Importance: The benefits of vitamin D, omega-3 fatty acids, and exercise in disease prevention remain unclear.
Objective: To test whether vitamin D, omega-3s, and a strength-training exercise program, alone or in combination, improved 6 health outcomes among older adults.
Design, setting, and participants: Double-blind, placebo-controlled, 2 × 2 × 2 factorial randomized clinical trial among 2157 adults aged 70 years or older who had no major health events in the 5 years prior to enrollment and had sufficient mobility and good cognitive status. Patients were recruited between December 2012 and November 2014, and final follow-up was in November 2017.
Interventions: Participants were randomized to 3 years of intervention in 1 of the following 8 groups: 2000 IU/d of vitamin D3, 1 g/d of omega-3s, and a strength-training exercise program (n = 264); vitamin D3 and omega-3s (n = 265); vitamin D3 and exercise (n = 275); vitamin D3 alone (n = 272); omega-3s and exercise (n = 275); omega-3s alone (n = 269); exercise alone (n = 267); or placebo (n = 270).
Main outcomes and measures: The 6 primary outcomes were change in systolic and diastolic blood pressure (BP), Short Physical Performance Battery (SPPB), Montreal Cognitive Assessment (MoCA), and incidence rates (IRs) of nonvertebral fractures and infections over 3 years. Based on multiple comparisons of 6 primary end points, 99% confidence intervals are presented and P < .01 was required for statistical significance.
Results: Among 2157 randomized participants (mean age, 74.9 years; 61.7% women), 1900 (88%) completed the study. Median follow-up was 2.99 years. Overall, there were no statistically significant benefits of any intervention individually or in combination for the 6 end points at 3 years. For instance, the differences in mean change in systolic BP with vitamin D vs no vitamin D and with omega-3s vs no omega-3s were both -0.8 (99% CI, -2.1 to 0.5) mm Hg, with P < .13 and P < .11, respectively; the difference in mean change in diastolic BP with omega-3s vs no omega-3s was -0.5 (99% CI, -1.2 to 0.2) mm Hg; P = .06); and the difference in mean change in IR of infections with omega-3s vs no omega-3s was -0.13 (99% CI, -0.23 to -0.03), with an IR ratio of 0.89 (99% CI, 0.78-1.01; P = .02). No effects were found on the outcomes of SPPB, MoCA, and incidence of nonvertebral fractures). A total of 25 deaths were reported, with similar numbers in all treatment groups.
Conclusions and relevance: Among adults without major comorbidities aged 70 years or older, treatment with vitamin D3, omega-3s, or a strength-training exercise program did not result in statistically significant differences in improvement in systolic or diastolic blood pressure, nonvertebral fractures, physical performance, infection rates, or cognitive function. These findings do not support the effectiveness of these 3 interventions for these clinical outcomes.
The study has noting to do with COVID-19 and very little with infections. The bit about infections shows almost the opposite of what the MAIL claims. So, where does the notion stipulated in the headline come from?
The MAIL article gives the answer: Professor Heike Bischoff-Ferrari from Zurich University in Switzerland, who led the latest study, said: ‘Our findings suggest supplementation of vitamin D and omega-3s in adults aged 70 or older who lead an active lifestyle and have no pre-existing conditions does not provide any benefits when it comes to bone health, memory and muscle function. ‘However, we believe there is an effect on infections – such as Covid-19.’
I would not be surprised, if the last sentence in the quote was taken out of context.
I would not be surprised, if this is the worst health related article in the DAIL MAIL this year.
And, by Jove, there are plenty to choose from.
And why do I report all this?
As I have pointed out before, I believe that journalists have a lot to answer for when it comes to misleading the public about so-called alternative medicine (SCAM):
- “Scientists have shown how homeopathy works” – journalists’ obsession with ‘balance’
- ACUPUNCTURE: journalists, be aware of your responsibility not to mislead the public
- Drowning in a sea of misinformation. Part 10: Journalists
My hope is that, by reminding them of their ‘errors’ every now and then, I might contribute to some progress.
Yes, I know, I am an incurable optimist!
This challenge for all homeopaths of the world was inspired by an avid commentator to this blog who, at every fitting and unfitting occasion, insists that those who doubt homeopathy must do a homeopathic proving.
A homeopathic ‘proving’ (Arzneimittelpruefung in Hahnamann’s less confusing terminology) is a test where a healthy person takes a (usually potentised) homeopathic remedy and then carefully notes all the symptoms and sensations which appear subsequently. When Hahnemann ‘discovered’ homeopathy, he took some cinchona and thought to experience the symptoms of malaria. This was the reason why he, after further such experiments, postulated that LIKE CURES LIKE.
To the present day, homeopathy relies on such provings. If we cannot sleep after drinking coffee, it is not unlike a proving of coffee, and homeopaths conclude that potentised coffee is a remedy for insomnia. I have done several provings many years ago, but they never worked the way homeopaths expect. We also investigared whether a related phenomenon, homeopathic aggravations (the worsening of the presenting symptom after taking the a well-chosen homeopathic remedy), claimed by homeopaths do exist at all; the answer was simple: no! In fact, the only people who believe in provings and aggravations are the homeopaths.
All this inspired me to now issue
A challenge for all homeopaths of the world
Here is the deal:
- you, the convinced homeopath, name the 6 homeopathic remedies that you cannot possibly miss when doing a proving on yourself;
- I order them in the potency you wish (only condition: it must be higher than C12) from a reputable source;
- I have the bottles delivered unopened to a notary where I live;
- the notary fills them into containers marked 1-6 (if you wish, you can send the notary empty containers for that ppurpose);
- the notary keeps the code under lock and key that links the name of the remedies to the numbers 1-6;
- he then mails the coded 6 remedies to you;
- you can use the proving method which you consider best and do as many provings as you like (the only limiting factors are the number of globuli in the containers and the time you have to crack the code);
- I give you 100 days for conducting the provings;
- once you are ready, you send your verdicts to the notary (e.g. 1 = rhus, tox, 2 = sulfur, 3 = arsenic, etc., etc.);
- the notary looks up the code and lets us both know the result.
I am happy to pay all the costs involved in the experiment (notary, remedies, postage, etc.). We can also discuss some of the details of this challenge, in case they run counter to your views on provings, rigorous science, etc.
To make sure we both ‘mean business’, once we both accept these conditions (you can flesh out the missing details as you wish), we both transfer a sum Euro 2 000 to an account with the notary. If you want to increase the sum, please let me know; as I said, we can discuss most of the details of my challenge to suit your needs. If you manage to ‘crack the code’ 1-6, the notary will transfer the sum of Euro 4 000 (your deposit and mine) to your account. If you fail, he will transfer the same amount to my account.
The entry into the challenge closes at the end of the year 2020.
Why should you take on this challenge? I can see several reasons:
- You want to prove that provings are valid.
- You want to teach me, and all other critics of homeopathy, a lesson.
- You want to earn Euro 2 000 quickly and without much work.
- You want the sceptics of the world to know that homeopathy is valid (we will report about our experiment fairly and to publish the report not just on this blog, but anywhere you want [provided the editors accept the paper for publication]).
Why do I take on the risk of losing a significant amount of money? Here too, I see more than one reason:
- I do not consider it a great risk; as I said, I did several provings myself and am quite certain they don’t work.
- I know about the implausiblity of the assumption that a remedy which contains nothing has any effects beyond expectation.
- I could do with the extra Euro 2 000.
- If no homeopath takes on the challenge, I shall henceforce declare that homeopaths were unable to prove that their provings are valid.