case-control study
On this blog, we have some people who continue to promote conspiracy theories about Covid and Covid vaccinations. It is, therefore, time, I feel, to present them with some solid evidence on the subject (even though it means departing from our usual focus on SCAM).
This Cochrane review assessed the efficacy and safety of COVID‐19 vaccines (as a full primary vaccination series or a booster dose) against SARS‐CoV‐2. An impressive team of investigators searched the Cochrane COVID‐19 Study Register and the COVID‐19 L·OVE platform (last search date 5 November 2021). They also searched the WHO International Clinical Trials Registry Platform, regulatory agency websites, and Retraction Watch. They included randomized controlled trials (RCTs) comparing COVID‐19 vaccines to placebo, no vaccine, other active vaccines, or other vaccine schedules.
A total of 41 RCTs could be included and analyzed assessing 12 different vaccines, including homologous and heterologous vaccine schedules and the effect of booster doses. Thirty‐two RCTs were multicentre and five were multinational. The sample sizes of RCTs were 60 to 44,325 participants. Participants were aged: 18 years or older in 36 RCTs; 12 years or older in one RCT; 12 to 17 years in two RCTs; and three to 17 years in two RCTs. Twenty‐nine RCTs provided results for individuals aged over 60 years, and three RCTs included immunocompromised patients. No trials included pregnant women. Sixteen RCTs had two‐month follow-ups or less, 20 RCTs had two to six months, and five RCTs had greater than six to 12 months or less. Eighteen reports were based on preplanned interim analyses. The overall risk of bias was low for all outcomes in eight RCTs, while 33 had concerns for at least one outcome. 343 registered RCTs with results not yet available were identified.The evidence for mortality was generally sparse and of low or very low certainty for all WHO‐approved vaccines, except AD26.COV2.S (Janssen), which probably reduces the risk of all‐cause mortality (risk ratio (RR) 0.25, 95% CI 0.09 to 0.67; 1 RCT, 43,783 participants; high‐certainty evidence).High‐certainty evidence was found that BNT162b2 (BioNtech/Fosun Pharma/Pfizer), mRNA‐1273 (ModernaTx), ChAdOx1 (Oxford/AstraZeneca), Ad26.COV2.S, BBIBP‐CorV (Sinopharm‐Beijing), and BBV152 (Bharat Biotect) reduce the incidence of symptomatic COVID‐19 compared to placebo (vaccine efficacy (VE): BNT162b2: 97.84%, 95% CI 44.25% to 99.92%; 2 RCTs, 44,077 participants; mRNA‐1273: 93.20%, 95% CI 91.06% to 94.83%; 2 RCTs, 31,632 participants; ChAdOx1: 70.23%, 95% CI 62.10% to 76.62%; 2 RCTs, 43,390 participants; Ad26.COV2.S: 66.90%, 95% CI 59.10% to 73.40%; 1 RCT, 39,058 participants; BBIBP‐CorV: 78.10%, 95% CI 64.80% to 86.30%; 1 RCT, 25,463 participants; BBV152: 77.80%, 95% CI 65.20% to 86.40%; 1 RCT, 16,973 participants).Moderate‐certainty evidence was found that NVX‐CoV2373 (Novavax) probably reduces the incidence of symptomatic COVID‐19 compared to placebo (VE 82.91%, 95% CI 50.49% to 94.10%; 3 RCTs, 42,175 participants).There is low‐certainty evidence for CoronaVac (Sinovac) for this outcome (VE 69.81%, 95% CI 12.27% to 89.61%; 2 RCTs, 19,852 participants).High‐certainty evidence was found that BNT162b2, mRNA‐1273, Ad26.COV2.S, and BBV152 result in a large reduction in the incidence of severe or critical disease due to COVID‐19 compared to placebo (VE: BNT162b2: 95.70%, 95% CI 73.90% to 99.90%; 1 RCT, 46,077 participants; mRNA‐1273: 98.20%, 95% CI 92.80% to 99.60%; 1 RCT, 28,451 participants; AD26.COV2.S: 76.30%, 95% CI 57.90% to 87.50%; 1 RCT, 39,058 participants; BBV152: 93.40%, 95% CI 57.10% to 99.80%; 1 RCT, 16,976 participants).
Moderate‐certainty evidence was found that NVX‐CoV2373 probably reduces the incidence of severe or critical COVID‐19 (VE 100.00%, 95% CI 86.99% to 100.00%; 1 RCT, 25,452 participants).
Two trials reported high efficacy of CoronaVac for severe or critical disease with wide CIs, but these results could not be pooled.
mRNA‐1273, ChAdOx1 (Oxford‐AstraZeneca)/SII‐ChAdOx1 (Serum Institute of India), Ad26.COV2.S, and BBV152 probably result in little or no difference in serious adverse events (SAEs) compared to placebo (RR: mRNA‐1273: 0.92, 95% CI 0.78 to 1.08; 2 RCTs, 34,072 participants; ChAdOx1/SII‐ChAdOx1: 0.88, 95% CI 0.72 to 1.07; 7 RCTs, 58,182 participants; Ad26.COV2.S: 0.92, 95% CI 0.69 to 1.22; 1 RCT, 43,783 participants); BBV152: 0.65, 95% CI 0.43 to 0.97; 1 RCT, 25,928 participants). In each of these, the likely absolute difference in effects was fewer than 5/1000 participants.
Evidence for SAEs is uncertain for BNT162b2, CoronaVac, BBIBP‐CorV, and NVX‐CoV2373 compared to placebo (RR: BNT162b2: 1.30, 95% CI 0.55 to 3.07; 2 RCTs, 46,107 participants; CoronaVac: 0.97, 95% CI 0.62 to 1.51; 4 RCTs, 23,139 participants; BBIBP‐CorV: 0.76, 95% CI 0.54 to 1.06; 1 RCT, 26,924 participants; NVX‐CoV2373: 0.92, 95% CI 0.74 to 1.14; 4 RCTs, 38,802 participants).
The authors’ conclusions were as follows: Compared to placebo, most vaccines reduce, or likely reduce, the proportion of participants with confirmed symptomatic COVID‐19, and for some, there is high‐certainty evidence that they reduce severe or critical disease. There is probably little or no difference between most vaccines and placebo for serious adverse events. Over 300 registered RCTs are evaluating the efficacy of COVID‐19 vaccines, and this review is updated regularly on the COVID‐NMA platform (covid-nma.com).
_____________________
As some conspiratorial loons will undoubtedly claim that this review is deeply biased; it might be relevant to add the conflicts of interest of its authors:
- Carolina Graña: none known.
- Lina Ghosn: none known.
- Theodoros Evrenoglou: none known.
- Alexander Jarde: none known.
- Silvia Minozzi: no relevant interests; Joint Co‐ordinating Editor and Method editor of the Drugs and Alcohol Group.
- Hanna Bergman: Cochrane Response – consultant; WHO – grant/contract (Cochrane Response was commissioned by the WHO to perform review tasks that contribute to this publication).
- Brian Buckley: none known.
- Katrin Probyn: Cochrane Response – consultant; WHO – consultant (Cochrane Response was commissioned to perform review tasks that contribute to this publication).
- Gemma Villanueva: Cochrane Response – employment (Cochrane Response has been commissioned by WHO to perform parts of this systematic review).
- Nicholas Henschke: Cochrane Response – consultant; WHO – consultant (Cochrane Response was commissioned by the WHO to perform review tasks that contributed to this publication).
- Hillary Bonnet: none known.
- Rouba Assi: none known.
- Sonia Menon: P95 – consultant.
- Melanie Marti: no relevant interests; Medical Officer at WHO.
- Declan Devane: Health Research Board (HRB) – grant/contract; registered nurse and registered midwife but no longer in clinical practice; Editor, Cochrane Pregnancy and Childbirth Group.
- Patrick Mallon: AstraZeneca – Advisory Board; spoken of vaccine effectiveness to media (print, online, and live); works as a consultant in a hospital that provides vaccinations; employed by St Vincent’s University Hospital.
- Jean‐Daniel Lelievre: no relevant interests; published numerous interviews in the national press on the subject of COVID vaccination; Head of the Department of Infectious Diseases and Clinical Immunology CHU Henri Mondor APHP, Créteil; WHO (IVRI‐AC): expert Vaccelarate (European project on COVID19 Vaccine): head of WP; involved with COVICOMPARE P et M Studies (APHP, INSERM) (public fundings).
- Lisa Askie: no relevant interests; Co‐convenor, Cochrane Prospective Meta‐analysis Methods Group.
- Tamara Kredo: no relevant interests; Medical Officer in an Infectious Diseases Clinic at Tygerberg Hospital, Stellenbosch University.
- Gabriel Ferrand: none known.
- Mauricia Davidson: none known.
- Carolina Riveros: no relevant interests; works as an epidemiologist.
- David Tovey: no relevant interests; Emeritus Editor in Chief, Feedback Editors for 2 Cochrane review groups.
- Joerg J Meerpohl: no relevant interests; member of the German Standing Vaccination Committee (STIKO).
- Giacomo Grasselli: Pfizer – speaking engagement.
- Gabriel Rada: none known.
- Asbjørn Hróbjartsson: no relevant interests; Cochrane Methodology Review Group Editor.
- Philippe Ravaud: no relevant interests; involved with Mariette CORIMUNO‐19 Collaborative 2021, the Ministry of Health, Programme Hospitalier de Recherche Clinique, Foundation for Medical Research, and AP‐HP Foundation.
- Anna Chaimani: none known.
- Isabelle Boutron: no relevant interests; member of Cochrane Editorial Board.
___________________________
And as some might say this analysis is not new, here are two further papers just out:
Objectives To determine the association between covid-19 vaccination types and doses with adverse outcomes of severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) infection during the periods of delta (B.1.617.2) and omicron (B.1.1.529) variant predominance.
Design Retrospective cohort.
Setting US Veterans Affairs healthcare system.
Participants Adults (≥18 years) who are affiliated to Veterans Affairs with a first documented SARS-CoV-2 infection during the periods of delta (1 July-30 November 2021) or omicron (1 January-30 June 2022) variant predominance. The combined cohorts had a mean age of 59.4 (standard deviation 16.3) and 87% were male.
Interventions Covid-19 vaccination with mRNA vaccines (BNT162b2 (Pfizer-BioNTech) and mRNA-1273 (Moderna)) and adenovirus vector vaccine (Ad26.COV2.S (Janssen/Johnson & Johnson)).
Main outcome measures Stay in hospital, intensive care unit admission, use of ventilation, and mortality measured 30 days after a positive test result for SARS-CoV-2.
Results In the delta period, 95 336 patients had infections with 47.6% having at least one vaccine dose, compared with 184 653 patients in the omicron period, with 72.6% vaccinated. After adjustment for patient demographic and clinical characteristics, in the delta period, two doses of the mRNA vaccines were associated with lower odds of hospital admission (adjusted odds ratio 0.41 (95% confidence interval 0.39 to 0.43)), intensive care unit admission (0.33 (0.31 to 0.36)), ventilation (0.27 (0.24 to 0.30)), and death (0.21 (0.19 to 0.23)), compared with no vaccination. In the omicron period, receipt of two mRNA doses were associated with lower odds of hospital admission (0.60 (0.57 to 0.63)), intensive care unit admission (0.57 (0.53 to 0.62)), ventilation (0.59 (0.51 to 0.67)), and death (0.43 (0.39 to 0.48)). Additionally, a third mRNA dose was associated with lower odds of all outcomes compared with two doses: hospital admission (0.65 (0.63 to 0.69)), intensive care unit admission (0.65 (0.59 to 0.70)), ventilation (0.70 (0.61 to 0.80)), and death (0.51 (0.46 to 0.57)). The Ad26.COV2.S vaccination was associated with better outcomes relative to no vaccination, but higher odds of hospital stay and intensive care unit admission than with two mRNA doses. BNT162b2 was generally associated with worse outcomes than mRNA-1273 (adjusted odds ratios between 0.97 and 1.42).
Conclusions In veterans with recent healthcare use and high occurrence of multimorbidity, vaccination was robustly associated with lower odds of 30 day morbidity and mortality compared with no vaccination among patients infected with covid-19. The vaccination type and number of doses had a significant association with outcomes.
SECOND EXAMPLE Long COVID, or complications arising from COVID-19 weeks after infection, has become a central concern for public health experts. The United States National Institutes of Health founded the RECOVER initiative to better understand long COVID. We used electronic health records available through the National COVID Cohort Collaborative to characterize the association between SARS-CoV-2 vaccination and long COVID diagnosis. Among patients with a COVID-19 infection between August 1, 2021 and January 31, 2022, we defined two cohorts using distinct definitions of long COVID—a clinical diagnosis (n = 47,404) or a previously described computational phenotype (n = 198,514)—to compare unvaccinated individuals to those with a complete vaccine series prior to infection. Evidence of long COVID was monitored through June or July of 2022, depending on patients’ data availability. We found that vaccination was consistently associated with lower odds and rates of long COVID clinical diagnosis and high-confidence computationally derived diagnosis after adjusting for sex, demographics, and medical history.
_______________________________________
There are, of course, many more articles on the subject for anyone keen to see the evidence. Sadly, I have little hope that the COVID loons will be convinced by any of them. Yet, I thought I should give it nevertheless a try.
A team of French researchers assessed whether a conflict of interest (COI) might be associated with the direction of the results of meta-analyses of homoeopathy trials. Their analysis (published as a ‘letter to the editor) is complex, therefore, I present here only their main finding.
The team conducted a literature search until July 2022 on PubMed and Embase to identify meta-analyses of randomized clinical trials assessing the efficacy of homoeopathy. They then assessed the existence of potential COI, defined by the presence of at least one of the following criteria:
- affiliation of one or more authors to an academic homoeopathy research or care facility, or to the homoeopathy industry;
- research sponsored or funded by the homoeopathy industry;
- COI declared by the authors.
The researchers also evaluated and classified any spin in meta-analyses conclusions into three categories (misleading reporting, misleading interpretation and inappropriate extrapolation). Two reviewers assessed the quality of meta-analyses and the risk of bias based. Publication bias was evaluated by the funnel plot method. For all the studies included in these meta-analyses, the researchers checked whether they reported a statistically significant result in favour of homoeopathy. Further details about the methods are provided on OSF (https://osf.io/nqw7r/) and in the preregistered protocol (CRD42020206242).
Twenty meta-analyses were included in the analysis (list of references available at https://osf.io/nqw7r/).
- Among the 13 meta-analyses with COI, a significantly positive effect of homoeopathy emerged (OR=0.60 (95% CI 0.50 to 0.70)).
- There was no such effect for meta-analyses without COI (OR=0.96 (95% CI 0.75 to 1.23)).
The authors concluded that in the presence of COI, meta-analyses of homoeopathy trials are more likely
to have favourable results. This is consistent with recent research suggesting that systematic reviews with financial COI are associated with more positive outcomes.
Meta-analyses are systematic reviews (critical assessments of the totality of the available evidence) where the data from the included studies are pooled. For a range of reasons, this may not always be possible. Therefore the number of meta-analyses (20) is substantially lower than that of the existing systematic reviews (>50).
Both systematic reviews and meta-analyses are theoretically the most reliable evidence regarding the value of any intervention. I said ‘theoretically’ because, like any human endeavour, they need to be done in an unbiased fashion to produce reliable results. People with a conflict of interest by definition struggle to be free of bias. As we have seen many times, this would include homoeopaths.
This new analysis confirms what many of us have feared. If proponents of homeopathy with an overt conflict of interest conduct a meta-analysis of studies of homeopathy, the results tend to be more positive than when independent researchers do it. The question that emerges from this is the following:
Are the findings of those researchers who have an interest in producing a positive result closer to the truth than the findings of researchers who have no such conflict?
I let you decide.
The well-known Dr. Chris van Tulleken recently joined forces with Professor Michael Heinrich and Dr. Anthony Booker from the University College London School of Pharmacy to test a range of herbal products on sale in the UK. They bought over 70 herbal products from various high street stores and internet retailers. Some of the products were ‘THR’ (traditional herbal registration) herbal medicines, and some were marketed as food supplements. They then analyzed their chemistry to see whether each one really contained what the label says. The three popular herbal remedies we tested were:-
- Milk thistle (Silybum marianum),
- Ginkgo (Ginkgo biloba),
- Evening primrose (Oenothera).
The team at UCL used two different methods of analysis to verify the identity of these herbal products and extracts. High-performance thin-layer chromatography (HPTLC) is a sophisticated technique for the analysis of herbal products and is one of the most commonly used methods in the industry. HPTLC analysis creates a chemical fingerprint of the product which the researchers can then compare to an accepted reference standard for the herb. They look for a broad spectrum of ‘marker compounds’ these are the pharmacologically active and/or chemical constituents within a plant that can be used to verify its potency or identity. For complex samples or where additional confirmation is required, researchers often turn to ¹H nuclear magnetic resonance spectroscopy (¹H-NMR) which allows individual samples to be compared in detail against other samples or to the whole group.
In every THR product tested, the product contained what was claimed on the label. However, the food supplements showed a wide range of quality.
- Of the food supplement products labeled as Ginkgo, 8 out of 30 (27%) contained little or no ginkgo extract.
- 36% of the food supplement milk thistle products contained no detectable milk thistle. Although this is quite a small sample size it is still a startling result. Furthermore, in one case of milk thistle, unidentified adulterants suspected to be synthetic compounds were present in place of milk thistle.
- All of the evening primrose food products we tested did contain what the packet claimed.
The researchers concluded that their investigation shows that a regulatory system for herbal products, like the THR scheme, ensures that people have access to safe herbal medicine products. So, if you are considering buying herbal products then do look out for the THR mark– otherwise, you might not just be wasting your money, you might be consuming other, potentially dangerous, ingredients.
_______________________________
This is an interesting investigation. The researchers should be commended for it! However, I disagree with some of their conclusions. Here is why:
- The investigation merely tested the quality of the products and NOT THEIR SAFETY! To claim that the THR ensures access to safe herbal medicines is incorrect. A product might be of adequate quality but can still be unsafe. The THR only implies safety because the herbal has been used for years without problems being noted. This is not the same as ensuring that it is safe. A direct test of safety is usually not available.
- The recommendation to buy a product with a THR mark is also somewhat misleading. It implies that these products are effective. I fail to see convincing evidence that either MILK THISTLE, GINKGO, or EVENING PRIMROSE are effective for any disease or condition. Thus the responsible recommendation should, in my view, be to NOT buy them regardless of whether they are of good quality or not.
Imagine you have caught a cold. You think it is not necessary to see a doctor, but you want to take something that helps your body to get better. What is your choice of remedy? There are many options provided by conventional medicine as well as by so-called alternative medicine (SCAM).
Many people opt for SCAM to address health issues or prevent diseases. Yet, the evidence for SCAMs is either lacking or controversial due to methodological weaknesses. Thus, practitioners and patients primarily rely on subjective references rather than credible evidence from systematic research.
This study investigated whether cognitive and personality factors explain the differences in belief in SCAM and homeopathy. The researchers investigated the robustness of 21 predictors when examined together to obtain insights into some key determinants of such beliefs in a sample of 599 participants (60% female, 18-81 years). A combination of predictors explained 20% of the variance in SCAM belief. These predictors were:
- ontological confusions,
- spiritual epistemology,
- agreeableness,
- death anxiety,
- gender.
Approximately 21% of the variance in belief in homeopathy was explained by the following predictors:
- ontological confusions,
- illusory pattern perception,
- need for cognitive closure,
- need for cognition,
- honesty-humility,
- death anxiety,
- gender,
- age.
The authors concluded that some of the predictors from previous research replicated whereas others did not. Demographics and certain cognitive variables seem to be key determinants associated with beliefs in SCAM and homeopathy. Those individual differences and cognitive biases might result in a different perception of the world. However, variables related to abilities did not predict the beliefs. Thus, they might not be a result of inability but rather of ignorance.
Previous studies have shown that SCAM believers tend to believe in paranormal phenomena and conspiracies. I think that, in the discussion sections of this blog, we have ample evidence for this to be true. Paranormal beliefs are usually built on a magical worldview without reasoned review, which is shared by SCAM proponents. Such beliefs advocate emotional criteria for truth instead of data and logical considerations. Another belief, namely spirituality, is closely related to paranormal beliefs and religiosity and also associated with being a SCAM user. Lindeman found that SCAM belief could be best explained by intuitive reasoning, paranormal beliefs, and ontological confusions, defined as category mistakes in which properties of living and lifeless entities are mixed.
The authors point out that their results do not replicate previous findings that showed predictive value of certain cognitive variables such as cognitive style. An explanation could be that rather inattention to accuracy than the inability to consider empirical evidence fosters the beliefs. People might simply not be aware of the absence of evidence. Another possibility is that people are aware of the absence of evidence but are reluctant to engage with it. Practitioners and patients often claim “whatever works is good” or “the main thing is that it works”. Thus, it is ignorance rather than a lack of capacity to appropriately process the evidence.
The authors of this study are well aware of the limitations of their research:
“As with most cross-sectional studies using questionnaires, our results are based on self-reports. Additionally, single items were used for measuring belief strength. Even if multi-item measures often have advantages, single items can be advantageous in terms of practical benefits, e.g., adapting to subjects’ limited attention and time resources. There are several single item measures successfully used to measure diverse concepts including attitudes. Also, the variance on those items in our sample shows that participants were able to reflect their beliefs and rank them on the scale provided. Another limitation is that the findings are based on regression analyses, which do not provide insight into causality. Thus, the relationship remains correlational. Even if our sample was broader than in many other psychological studies—it was slightly unbalanced, especially in comparison to the German population. It over-represented educated individuals which may lead to an inadequate variation of the cognitive variables if we consider the relationship between cognition and education. However, education and the cognitive variables are only weakly correlated. Thus, it can be assumed that the unbalanced sample did not affect the distribution of cognitive variables to a great extent.”
The ‘keto diet’ is a currently popular high-fat, low-carbohydrate diet; it limits the intake of glucose which results in the production of ketones by the liver and their uptake as an alternative energy source by the brain. It is said to be an effective treatment for intractable epilepsy. In addition, it is being promoted as a so-called alternative medicine (SCAM) for a wide range of conditions, including:
- weight loss,
- cognitive and memory enhancement,
- type II diabetes,
- cancer,
- neurological and psychiatric disorders.
Now, it has been reported that the ‘keto diet’ may be linked to higher levels of cholesterol and double the risk of cardiovascular events. In the study, researchers defined a low-carb, high-fat (LCHF) diet as 45% of total daily calories coming from fat and 25% coming from carbohydrates. The study, which has so far not been peer-reviewed, was presented Sunday at the American College of Cardiology’s Annual Scientific Session Together With the World Congress of Cardiology.
“Our study rationale came from the fact that we would see patients in our cardiovascular prevention clinic with severe hypercholesterolemia following this diet,” said Dr. Iulia Iatan from the Healthy Heart Program Prevention Clinic, St. Paul’s Hospital, and University of British Columbia’s Centre for Heart Lung Innovation in Vancouver, Canada, during a presentation at the session. “This led us to wonder about the relationship between these low-carb, high-fat diets, lipid levels, and cardiovascular disease. And so, despite this, there’s limited data on this relationship.”
The researchers compared the diets of 305 people eating an LCHF diet with about 1,200 people eating a standard diet, using health information from the United Kingdom database UK Biobank, which followed people for at least a decade. They found that people on the LCHF diet had higher levels of low-density lipoprotein and apolipoprotein B. Apolipoprotein B is a protein that coats LDL cholesterol proteins and can predict heart disease better than elevated levels of LDL cholesterol can. The researchers also noticed that the LCHF diet participants’ total fat intake was higher in saturated fat and had double the consumption of animal sources (33%) compared to those in the control group (16%). “After an average of 11.8 years of follow-up – and after adjustment for other risk factors for heart disease, such as diabetes, high blood pressure, obesity, and smoking – people on an LCHF diet had more than two times higher risk of having several major cardiovascular events, such as blockages in the arteries that needed to be opened with stenting procedures, heart attack, stroke, and peripheral arterial disease.” Their press release also cautioned that their study “can only show an association between the diet and an increased risk for major cardiac events, not a causal relationship,” because it was an observational study, but their findings are worth further investigation, “especially when approximately 1 in 5 Americans report being on a low-carb, keto-like or full keto diet.”
I have to say that I find these findings not in the slightest bit surprising and would fully expect the relationship to be causal. The current craze for this diet is concerning and we need to warn consumers that they might be doing themselves considerable harm.
Other authors have recently pointed out that, within the first 6-12 months of initiating the keto diet, transient decreases in blood pressure, triglycerides, and glycosylated hemoglobin, as well as increases in HDL and weight loss may be observed. However, the aforementioned effects are generally not seen after 12 months of therapy. Despite the diet’s favorable effect on HDL-C, the concomitant increases in LDL-C and very-low-density lipoproteins (VLDL) may lead to increased cardiovascular risks. And another team of researchers has warned that “given often-temporary improvements, unfavorable effects on dietary intake, and inadequate data demonstrating long-term safety, for most individuals, the risks of ketogenic diets may outweigh the benefits.”
In this retrospective matched-cohort study, Chinese researchers investigated the association of acupuncture treatment for insomnia with the risk of dementia. They collected data from the National Health Insurance Research Database (NHIRD) of Taiwan to analyze the incidence of dementia in patients with insomnia who received acupuncture treatment.
The study included 152,585 patients, selected from the NHIRD, who were newly diagnosed with insomnia between 2000 and 2010. The follow-up period ranged from the index date to the date of dementia diagnosis, date of withdrawal from the insurance program, or December 31, 2013. A 1:1 propensity score method was used to match an equal number of patients (N = 18,782) in the acupuncture and non-acupuncture cohorts. The researchers employed Cox proportional hazards models to evaluate the risk of dementia. The cumulative incidence of dementia in both cohorts was estimated using the Kaplan–Meier method, and the difference between them was assessed through a log-rank test.
Patients with insomnia who received acupuncture treatment were observed to have a lower risk of dementia (adjusted hazard ratio = 0.54, 95% confidence interval = 0.50–0.60) than those who did not undergo acupuncture treatment. The cumulative incidence of dementia was significantly lower in the acupuncture cohort than in the non-acupuncture cohort (log-rank test, p < 0.001).
The researchers concluded that acupuncture treatment significantly reduced or slowed the development of dementia in patients with insomnia.
They could be correct, of course. But, then again, they might not be. Nobody can tell!
As many who are reading these lines know: CORRELATION IS NOT CAUSATION.
But if acupuncture was not the cause for the observed differences, what could it be? After all, the authors used clever statistics to make sure the two groups were comparable!
The problem here is, of course, that they can only make the groups comparable for variables that were measured. These were about 20 parameters mostly related to medication intake and concomitant diseases. This leaves a few hundred potentially relevant variables that were not quantified and could thus not be accounted for.
My bet would be lifestyle: it is conceivable that the acupuncture group had acupuncture because they were generally more health-conscious. Living a relatively healthy life might reduce the dementia risk entirely unrelated to acupuncture. According to Occam’s razor, this explanation is miles more likely that the one about acupuncture.
So, what this study really demonstrates or implies is, I think, this:
- The propensity score method can never be perfect in generating completely comparable groups.
- The JTCM publishes rubbish.
- Correlation is not causation.
- To establish causation in clinical medicine, RCTs are usually the best option.
- Occam’s razor can be useful when interpreting research findings.
This prospective study aimed to identify an optimal lifestyle profile to protect against memory loss in older individuals from areas representative of the north, south, and west of China. Individuals aged 60 years or older who had normal cognition and underwent apolipoprotein E (APOE) genotyping at baseline in 2009 were included. Participants were followed up until death, discontinuation, or 26 December 2019.
Six lifestyle factors were assessed:
- a healthy diet (adherence to the recommended intake of at least 7 of 12 eligible food items),
- regular physical exercise (≥150 min of moderate intensity or ≥75 min of vigorous intensity, per week),
- active social contact (≥twice per week),
- active cognitive activity (≥twice per week),
- never or previously smoked,
- never drinking alcohol.
Participants were categorised into the favourable group if they had 4-6 healthy lifestyle factors, into the average group for two to three factors, and into the unfavourable group for zero to one factor.
Memory function was assessed using the World Health Organization/University of California-Los Angeles Auditory Verbal Learning Test, and global cognition was assessed via the Mini-Mental State Examination. Linear mixed models were used to explore the impact of lifestyle factors on memory in the study sample.
A total of 29 072 participants were included (mean age of 72.23 years; 48.54% (n=14 113) were women; and 20.43% (n=5939) were APOE ε4 carriers). Over the 10-year follow-up period (2009-19), participants in the favourable group had slower memory decline than those in the unfavourable group (by 0.028 points/year, 95% confidence interval 0.023 to 0.032, P<0.001). APOE ε4 carriers with favourable (0.027, 95% confidence interval 0.023 to 0.031) and average (0.014, 0.010 to 0.019) lifestyles exhibited a slower memory decline than those with unfavourable lifestyles. Among people who were not carriers of APOE ε4, similar results were observed among participants in the favourable (0.029 points/year, 95% confidence interval 0.019 to 0.039) and average (0.019, 0.011 to 0.027) groups compared with those in the unfavourable group. APOE ε4 status and lifestyle profiles did not show a significant interaction effect on memory decline (P=0.52).
The authors concluded that a healthy lifestyle is associated with slower memory decline, even in the presence of the APOE ε4 allele. This study might offer important information to protect older adults against memory decline.
This is an important and meticulously reported study. It is the first large-scale investigation that assesses the effects of different lifestyle profiles, APOE ε4 status, and their interactions on longitudinal memory trajectories over a 10-year follow-up period. The results show that lifestyle is associated with the rate of memory decline in cognitively normal older individuals, including in people who are genetically susceptible to memory decline. The authors are rightly careful to avoid causal inferences between lifestyle and memory decline. To demonstrate causality beyond doubt, we would need different study designs.
The authors also discuss several weaknesses of the study:
- Firstly, the assessments of lifestyle factors were based on self-reports and are, therefore, prone to measurement errors.
- Secondly, several participants were excluded due to missing data or not returning for follow-up evaluations, which might have led to selection bias.
- Thirdly, the proportion of individuals with an unhealthy lifestyle might have been underestimated in the study because people with poor health were less likely to have participated in the study.
- Fourthly, given the nature of the study design, it could not assess whether maintaining a healthy lifestyle had already started influencing memory by the time of enrolment in the study.
- Fifthly, the evaluation of memory using a single neuropsychological test that does not comprehensively reflect overall memory function. However, the Auditory Verbal Learning Test is an effective instrument for memory assessment, and a composite score was used based on four Auditory Verbal Learning Test subscales to represent memory conditions to the greatest extent possible.
- Sixthly, as participants might become familiar with repeated cognitive testing, a learning effect could have influenced the results.
- Finally, memory decline was studied solely among older adults; however, memory problems commonly affect young individuals as well.
The authors, therefore, state that further studies should be conducted to facilitate a more extensive investigation into the effects of a healthy lifestyle on memory decline across the lifespan. This approach would help to elucidate the crucial age window during which a healthy lifestyle can exert the most favourable effect.
Migraines are common headache disorders and risk factors for subsequent strokes. Acupuncture has been widely used in the treatment of migraines; however, few studies have examined whether its use reduces the risk of strokes in migraineurs. This study explored the long-term effects of acupuncture treatment on stroke risk in migraineurs using national real-world data.
A team of Taiwanese researchers collected new migraine patients from the Taiwan National Health Insurance Research Database (NHIRD) from 1 January 2000 to 31 December 2017. Using 1:1 propensity-score matching, they assigned patients to either an acupuncture or non-acupuncture cohort and followed up until the end of 2018. The incidence of stroke in the two cohorts was compared using the Cox proportional hazards regression analysis. Each cohort was composed of 1354 newly diagnosed migraineurs with similar baseline characteristics. Compared with the non-acupuncture cohort, the acupuncture cohort had a significantly reduced risk of stroke (adjusted hazard ratio, 0.4; 95% confidence interval, 0.35–0.46). The Kaplan–Meier model showed a significantly lower cumulative incidence of stroke in migraine patients who received acupuncture during the 19-year follow-up (log-rank test, p < 0.001).
The authors concluded that acupuncture confers protective benefits on migraineurs by reducing the risk of stroke. Our results provide new insights for clinicians and public health experts.
After merely 10 minutes of critical analysis, ‘real-world data’ turn out to be real-bias data, I am afraid.
The first question to ask is, were the groups at all comparable? The answer is, NO; the acupuncture group had
- more young individuals;
- fewer laborers;
- fewer wealthy people;
- fewer people with coronary heart disease;
- fewer individuals with chronic kidney disease;
- fewer people with mental disorders;
- more individuals taking multiple medications.
And that are just the variables that were known to the researcher! There will be dozens that are unknown but might nevertheless impact on a stroke prognosis.
But let’s not be petty and let’s forget (for a minute) about all these inequalities that render the two groups difficult to compare. The potentially more important flaw in this study lies elsewhere.
Imagine a group of people who receive some extra medical attention – such as acupuncture – over a long period of time, administered by a kind and caring therapist; imagine you were one of them. Don’t you think that it is likely that, compared to other people who do not receive this attention, you might feel encouraged to look better after your health? Consequently, you might do more exercise, eat more healthily, smoke less, etc., etc. As a result of such behavioral changes, you would be less likely to suffer a stroke, never mind the acupuncture.
SIMPLE!
I am not saying that such studies are totally useless. What often renders them worthless or even dangerous is the fact that the authors are not more self-critical and don’t draw more cautious conclusions. In the present case, already the title of the article says it all:
Acupuncture Is Effective at Reducing the Risk of Stroke in Patients with Migraines: A Real-World, Large-Scale Cohort Study with 19-Years of Follow-Up
My advice to researchers of so-called alternative medicine (SCAM) and journal editors publishing their papers is this: get your act together, learn about the pitfalls of flawed science (most of my books might assist you in this process), and stop misleading the public. Do it sooner rather than later!
Chronic kidney disease is common, often progressive, and difficult to treat or prevent. Effective interventions would therefore be more than welcome. This paper explored the relation of habitual fish oil use with the risk of chronic kidney diseases (CKD).
A total of 408,023 participants (54.2% female) without prior CKD and with completed information regarding their consumption of major food groups and fish oil in the UK Biobank were enrolled. Fish oil use and dietary intakes were assessed by touch screen questionnaire and food frequency questionnaire, respectively. Incident CKD was recorded from hospital inpatient records.
At baseline, 128,843 (31.6%) participants reported taking fish oil supplements. During a median follow-up period of 12.0 years, a total of 10,782 (2.6%) participants developed CKD. With adjustments for important confounders, habitual fish oil use was associated with a significantly lower hazard of incident CKD (hazard ratio [HR], 0.90; 95% confidence interval [CI], 0.87-0.95), compared with non-use. Consistently, participants reporting ≥2 servings/week of oily fish (HR, 0.86; 95% CI, 0.79-0.94) and nonoily fish (HR, 0.86; 95% CI, 0.77-0.97) consumption had a lower hazard of incident CKD compared to those reporting no consumption ever. Additionally, among the 97,914 participants with data on plasma fatty acid, there were significant inverse relationships of plasma omega-3 polyunsaturated fatty acid (PUFA) (per SD increment, HR, 0.89, 95% CI, 0.84-0.94) and eicosatetraenoic acid (per SD increment, HR, 0.91, 95% CI, 0.87-0.96) with incident CKD.
The authors concluded that habitual fish oil use was associated with a lower hazard of CKD, which was further confirmed by the consistent inverse relations between fish consumption and circulating omega-3 PUFA concentration with incident CKD.
I like this paper! It shows in an exemplary fashion how to interpret an association between two variables: fish oil consumption does not necessarily CAUSE the lower risk, it is merely associated with it and there might be a number of non-causal explanations for the link. Whether there is a true cause-effect relationship needs to be investigated in further, differently designed studies. The present paper does not overstate its conclusions but it is nevertheless important, as it hopefully will prompt others to clarify the crucial issue of causality.
Wouldn’t it be nice, if researchers of so-called alternative medicine (SCAM) finally learned this simple lesson?
The impact of drug-induced liver injury (DILI) on patients with chronic liver disease (CLD) is unclear. There are few reports comparing DILI in CLD and non-CLD patients. In this study, the researchers aimed to determine the incidence and outcomes of DILI in patients with and without CLD.
They collected data on eligible individuals with suspected DILI between 2018 and 2020 who were evaluated systematically for other etiologies, causes, and the severity of DILI. They compared the causative agents, clinical features, and outcomes of DILI among subjects with and without CLD who were enrolled in the Thai Association for the Study of the Liver DILI registry. Subjects with definite, or highly likely DILI were included in the analysis.
The researchers evaluated the causal relationship between the clinical pattern of liver injury and the suspected drugs or SCAM products with the Roussel Uclaf Causality Assessment Method (RUCAM) system. RUCAM is a validated and established tool to quantitatively assess causality in cases of suspected DILI and/or SCAM product-induced liver injury. They also used the Clinical Assessment of Causality Scale to assess the association as definite (>95% likelihood), highly likely (75–95%), probable (50–74%), possible (25–49%) or unlikely (<25%).
A total of 200 subjects diagnosed with DILI were found in the registry. Of those, 41 had CLD and 159 had no evidence of CLD. So-called alternative medicine (SCAM) products were identified as the most common class of DILI agents. Approximately 59% of DILI in the CLD and 40% in non-CLD group were associated with SCAM use. Individuals with pre-existing CLD had similar severity including mortality. Twelve patients (6%) developed adverse outcomes related to DILI including seven (3.5%) deaths and five (2.5%) with liver failure. Mortality was 4.88% in CLD and 3.14% in non-CLD subjects over median periods of 58 (8-106) days and 22 (1-65) days, respectively.
The authors concluded that, in this liver disease registry, the causes, clinical presentation, and outcomes of DILI in subjects with CLD and without CLD patients were not different. Further study is required to confirm our findings.
Consumers often prefer SCAM to conventional medicine because SCAM is viewed as gentle and safe. The notions are that they
- are natural and therefore harmless;
- have been in use for ages and thus have stood the test of time.
Readers of this blog will appreciate that both notions are, in fact, fallacies:
- appeal to nature;
- appeal to tradition.
This new paper is an impressive reminder that SCAM’s reputation as a safe option is not justified, and that SCAM relies more on fallacies than on facts.
