study design
Much of the discussions on this blog are directly or indirectly related to the subject of research integrity. Research integrity refers to the ethical and professional standards that researchers must adhere to while conducting research. It involves conducting research in a way that allows others to have confidence and trust in the methods and findings of the research. Research integrity includes honesty, rigour, transparency, open communication, care and respect for all participants, and accountability. According to the ‘Concordat To Support Research Integrity‘, the core elements of research integrity are:
- Honesty in all aspects of research, including in the presentation of research goals, intentions and findings; in reporting on research methods and procedures; in gathering data; in using and acknowledging the work of other researchers; and in conveying valid interpretations and making justifiable claims based on research findings.
- Rigour, in line with prevailing disciplinary norms and standards, and in performing research and using appropriate methods; in adhering to an agreed protocol where appropriate; in drawing interpretations and conclusions from the research; and in communicating the results.
- Transparency and open communication in declaring potential competing interests in the reporting of research data collection methods; in the analysis and interpretation of data; in making research findings widely available, which includes publishing or otherwise sharing negative or null results to recognise their value as part of the research process; and in presenting the work to other researchers and to the public.
- Care and respect for all participants in research, and for the subjects, users and beneficiaries of research, including humans, animals, the environment and cultural objects. Those engaged with research must also show care and respect for the integrity of the research record.
- Accountability of funders, employers and researchers to collectively create a research environment in which individuals and organisations are empowered and enabled to own the research process. Those engaged with research must also ensure that individuals and organisations are held to account when behaviour falls short of the standards set by this concordat.
These points apply to all types and aspects of research and to all individuals involved in it. Obviously, they apply also to research of so-called alternative medicine (SCAM). I will therefore briefly discuss each point in respect of SCAM research.
Ad 1
Honesty in research is an important ethical prerequisite. In SCAM, we regularly see that the honesty seems to be in short supply. There are many ways in which lack of honesty shows itself. For me, the most impressive one is when SCAM researchers disclose – as they often do – that their aim is to demostrate that their assumptions/theories/hypotheses are correct. In such cases, they abuse the tools of science which are not for confirming hypotheses but for testing, i.e. trying to falsifying them. This blog offers plenty of examples for this phenomenon (e.g. here or here)
Ad 2
The lack of rigor in much of SCAM research is legend and has become a constant theme on this blog. How often have we seen useless and unnecessary observational studies or pilot studies on subjects where controlled trials already exist? How often have we lamented over misinterpretation of a study’s findings? How often have we reported research that was ill-conceived from the outset, unable to answer any meaningful research question? My all time favourite examples of drawing a wrong conclusions are from the area of homeopathy, e.g. here and here.
Ad 3
Much of SCAM research is being conducted by SCAM enthusiasts who have no qualitification or experience in research or science. Inevitably, this must lead to bias. Just think of the fact that some countries, e.g. China, or some SCAM journals, or some SCAM researchers (e.g. THE ALTERNATIVE MEDICINE HALL OF FAME) publish as good as no negative findings
Ad 4
Lack of respect for research subjects has many guises. For instance, if clinical trials are being conducted of treatments that are utterly implausible. This, in my view, constitutes an abuse of the willingness of volunteers to make a contribution to research. Another example is the incessant flow of untrustworthy research by SCAM enthusiasts which eventually can only erode the public’s trust in research and is bound to render the essential cooperation of researchers and volunteers more and more problematic.
Ad 5
When I think of accountability, I think yet again of the men and women in my ALTERNATIVE MEDICINE HALL OF FAME. As any of its members ever been held accountable for misleading the public through their pseudo-research? Sadly, the answer, as far as I know, is NO.
This study investigated whether Tongxinluo,a traditional Chinese medicine compound that has shown promise in in vitro, animal, and small human studies for myocardial infarction, could improve clinical outcomes in patients with ST-segment elevation myocardial infarction (STEMI). The randomized, double-blind, placebo-controlled clinical trial was conducted among patients with STEMI within 24 hours of symptom onset from 124 hospitals in China. Patients were enrolled from May 2019 to December 2020; the last date of follow-up was December 15, 2021.
Patients were randomized 1:1 to receive either Tongxinluo or placebo orally for 12 months. A loading dose of 2.08 g was given after randomization, followed by the maintenance dose of 1.04 g, 3 times a day, in addition to STEMI guideline-directed treatments. The primary end point was 30-day major adverse cardiac and cerebrovascular events (MACCEs), a composite of cardiac death, myocardial reinfarction, emergent coronary revascularization, and stroke. Follow-up for MACCEs occurred every 3 months to 1 year.
Among 3797 patients who were randomized, 3777 (Tongxinluo: 1889 and placebo: 1888; mean age, 61 years; 76.9% male) were included in the primary analysis. Thirty-day MACCEs occurred in 64 patients (3.4%) in the Tongxinluo group vs 99 patients (5.2%) in the control group. Individual components of 30-day MACCEs, including cardiac death, were also significantly lower in the Tongxinluo group than the placebo group. By 1 year, the Tongxinluo group continued to have lower rates of MACCEs and cardiac death. There were no significant differences in other secondary end points including 30-day stroke; major bleeding at 30 days and 1 year; 1-year all-cause mortality; and in-stent thrombosis. More adverse drug reactions occurred in the Tongxinluo group than the placebo group, mainly driven by gastrointestinal symptoms.
The authors concluded that in patients with STEMI, the Chinese patent medicine Tongxinluo, as an adjunctive therapy in addition to STEMI guideline-directed treatments, significantly improved both 30-day and 1-year clinical outcomes. Further research is needed to determine the mechanism of action of Tongxinluo in STEMI.
Tongxinluo is mixture of various active ingredients, including
- ginseng,
- leech,
- scorpion,
- Paeonia lactiflora,
- cicada slough,
- woodlouse bug,
- centipede,
- sandalwood.
With chaotic mixtures of this type, it is impossible to name all the potentially active ingredients, list their actions, or identify the ones that are truly relevant. According to the thinking of TCM proponents, this would also be the wrong way to go about it – such mixtures work as a whole, they would insist.
Tongxinluo is by no means a mixture that has not been studied before.
A previous systematic review of 12 studies found that Tongxinluo capsule is superior to conventional treatment in improving clinical overall response rate and hemorheological indexes and is relatively safe. Due to the deficiencies of the existing studies, more high-quality studies with rigorous design are required for further verification.
A 2022 meta-analysis indicated that the mixture had beneficial effects on the prevention of cardiovascular adverse events, especially in TVR or ISR after coronary revascularization and may possibly lower the incidence of first or recurrent MI and HF within 12 months in patients with CHD, while insufficient sample size implied that these results lacked certain stability. And the effects of TXLC on cardiovascular mortality, cerebrovascular events, and unscheduled readmission for CVDs could not be confirmed due to insufficient cases. Clinical trials with large-sample sizes and extended follow-up time are of interest in the future researches.
A further meta-analysis suggested beneficial effects on reducing the adverse cardiovascular events without compromising safety for CHD patients after PCI on the 6-month course.
Finally, a systematic review of 10 studies found that the remedy is an effective and safe therapy for CHD patients after percutaneous coronary interventions.
So, should we believe the new study with its remarkable findings? On the one hand, the trial seems rigorous and is reported in much detail. On the other hand, the study (as all previous trials of this mixture) originates from China. We know how important TCM is for that country as an export item, and we know how notoriously unreliable Chinese research sadly has become. In view of this, I would like to see an independent replication of this study by an established research group outside China before I recommend Tongxinluo to anyone.
Let’s not forget:
if it sound too good to be true, it probably is!
So-called alternative medicine (SCAM) interventions are growing in popularity and are even advocated as treatments for long COVID symptoms. However, comprehensive analysis of current evidence in this setting is still lacking. This study aims to review existing published studies on the use of SCAM interventions for patients experiencing long COVID through a systematic review of randomized controlled trials (RCTs).
A comprehensive electronic literature search was performed in multiple databases and clinical trial registries from September 2019 to January 2023. RCTs evaluating efficacy and safety of SCAM for long COVID were included. Methodological quality of each included trial
was appraised with the Cochrane ‘risk of bias’ tool. A qualitative analysis was conducted due to heterogeneity of included studies.
A total of 14 RCTs with 1195 participants were included in this review. Study findings demonstrated that SCAM interventions could benefit patients with long COVID, especially those suffering from
- neuropsychiatric disorders,
- olfactory dysfunction,
- cognitive impairment,
- fatigue,
- breathlessness,
- mild-to-moderate lung fibrosis.
The main interventions reported were:
- self-administered transcutaneous auricular vagus nerve stimulation,
- neuro-meditation,
- dietary supplements,
- olfactory training,
- aromatherapy,
- inspiratory muscle training,
- concurrent training,
- online breathing programs,
- online well-being programs.
The authors concluded that SCAM interventions may be effective, safe, and acceptable to patients with symptoms of long COVID. However, the findings from this systematic review should be interpreted with caution due to various methodological limitations. More rigorous trials focused on CAM for long COVID are warranted in the future.
Such wishy-washy conclusions seem to be popular in the fantasy land of SCAM. Yet, they are, in my view, most ojectionable because:
- they tell us nothing of value;
- that something “MAY BE EFFECTIVE” has been known before and cannot be the result of but is the reason for a systematic review;
- a review of 14 RCTs of almost as many interventions cannot possibly tell us anything about the SAFETY of these treatments;
- it also does not provide evidence of effectiveness and merely indicates a lack of independent replications;
- if the abstract mentions an assessment of the study rigor, one expects that it also informs us about this important aspect.
Once we do come around looking at the methodological quality of the primary studies we realize that it is mostly miserable. This means that the conclusions of the review are not just irritating but plainly misleading. Responsible researchers should have concluded along the following lines:
The quantity and the quality of the evidence are both low. Therefore, the effectiveness and safety of SCAM interventions for long COVID remains unproven.
PS
This project was financially supported by The HEAD Foundation, Singapore and in part by the grant from the NIH R61 AT01218.
Shame on the authors, journal editors, peer-reviewers, and funders of this dangerous nonsense!
This review evaluated the magnitude of the placebo response of sham acupuncture in trials of acupuncture for nonspecific LBP, and assessed whether different types of sham acupuncture are associated with different responses. Four databases including PubMed, EMBASE, MEDLINE, and the Cochrane Library were searched through April 15, 2023, and randomized controlled trials (RCTs) were included if they randomized patients with LBP to receive acupuncture or sham acupuncture intervention. The main outcomes included the placebo response in pain intensity, back-specific function and quality of life. Placebo response was defined as the change in these outcome measures from baseline to the end of treatment. Random-effects models were used to synthesize the results, standardized mean differences (SMDs, Hedges’g) were applied to estimate the effect size.
A total of 18 RCTs with 3,321 patients were included. Sham acupuncture showed a noteworthy pooled placebo response in pain intensity in patients with LBP [SMD −1.43, 95% confidence interval (CI) −1.95 to −0.91, I2=89%]. A significant placebo response was also shown in back-specific functional status (SMD −0.49, 95% CI −0.70 to −0.29, I2=73%), but not in quality of life (SMD 0.34, 95% CI −0.20 to 0.88, I2=84%). Trials in which the sham acupuncture penetrated the skin or performed with regular needles had a significantly higher placebo response in pain intensity reduction, but other factors such as the location of sham acupuncture did not have a significant impact on the placebo response.
The authors concluded that sham acupuncture is associated with a large placebo response in pain intensity among patients with LBP. Researchers should also be aware that the types of sham acupuncture applied may potentially impact the evaluation of the efficacy of acupuncture. Nonetheless, considering the nature of placebo response, the effect of other contextual factors cannot be ruled out in this study.
As the authors stated in their conclusion: the effect of other contextual factors cannot be ruled out. I would go much further and say that the outcomes noted here are mostly due to effects other than placebo. Obvious candidates are:
- regression towards the mean;
- natural history of the condition;
- success of patient blinding;
- social desirability.
To define the placebo effect in acupuncture trials as the change in the outcome measures from baseline to the end of treatment – as the authors of the review do – is not just naive, it is plainly wrong. I would not be surprised, if different sham acupuncture treatments have different effects. To me this would be an expected, plausible finding. But such differences just cannot be estimated in the way the authors suggest. For that, we would need an RCT in which patients are randomized to be treated in the same setting with a range of different types of sham acupuncture. The results of such a study might be revealing but I doubt that many ethics committees would be happy to grant their approval for it.
In the absence of such data, the best we can do is to design trials such that the verum is tested against a credible placebo which, for patients, is indistinguishable from the verum, while demonstrating that blinding is successful.
Since 1997, several meta-analyses (MAs) of placebo-controlled randomised efficacy trials of homoeopathy for any indication (PRETHAIs) have been published with different methods, results and conclusions. To date, a formal assessment of these MAs has not been performed. The main objective of this systematic review of MAs of PRETHAIs was to evaluate the efficacy of homoeopathic treatment.
The inclusion criteria were as follows: MAs of PRETHAIs in humans; all ages, countries, settings, publication languages; and MAs published from 1 Jan. 1990 to 30 Apr. 2023. The exclusion criteria were as follows: systematic reviews without MAs; MAs restricted to age or gender groups, specific indications, or specific homoeopathic treatments; and MAs that did not assess efficacy. We searched 8 electronic databases up to 14 Dec. 2020, with an update search in 6 databases up to 30 April 2023.
The primary outcome was the effect estimate for all included trials in each MA and after restricting the sample to trials with high methodological quality, according to predefined criteria. The risk of bias for each MA was assessed by the ROBIS (Risk Of Bias In Systematic reviews) tool. The quality of evidence was assessed by the GRADE framework. Statistical analyses were performed to determine the proportion of MAs showing a significant positive effect of homoeopathy vs. no significant difference.
Six MAs were included, covering individualised homoeopathy (I-HOM, n = 2), nonindividualised homoeopathy (NI-HOM, n = 1) and all homoeopathy types (ALL-HOM = I-HOM + NI-HOM, n = 3). The MAs comprised between 16 and 110 trials, and the included trials were published from 1943–2014. The median trial sample size ranged from 45 to 97 patients. The risk of bias (low/unclear/high) was rated as low for three MAs and high for three MAs.
Effect estimates for all trials in each MA showed a significant positive effect of homoeopathy compared to placebo (5 of 5 MAs, no data in 1 MA). Sensitivity analyses with sample restriction to high-quality trials were available from 4 MAs; the effect remained significant in 3 of the MAs (2 MAs assessed ALL-HOM, 1 MA assessed I-HOM) and was no longer significant in 1 MA (which assessed NI-HOM).
The authors concluded that the quality of evidence for positive effects of homoeopathy beyond placebo (high/moderate/low/very low) was high for I-HOM and moderate for ALL-HOM and NI-HOM. There was no support for the alternative hypothesis of no outcome difference between homoeopathy and placebo. The available MAs of PRETHAIs reveal significant positive effects of homoeopathy beyond placebo. This is in accordance with laboratory experiments showing partially replicable effects of homoeopathically potentised preparations in physico-chemical, in vitro, plant-based and animal-based test systems.
Reading this SR, I got the impression that it was designed to generate a positive result. The 6 included MAs are marginally positive (mainly due to publication bias and other artefacts) and thus very well known to fans of homeopathy. The authors of this paper must therefore have expected that combining them in a review would generate an overall positive finding.
The first question I asked myself while studying this paper was: who would want to do an SR of MAs (a most peculiar exercise); why not at least an SR of SRs which is already an unusual project but would make at least some sense. (An SR is a review that includes all studies that match a set of pre-definied criteria. A MA is a special form of SR where statistical pooling was possible.) The answer is, I fear, simple: this would not have generated a positive result: here are now dozens of SRs of homeopathy and most are not positive (as discussed regularly on this blog)
The authors themselves provide no real justification for their bizarre approach. All they tell us about it is this:
One approach is to focus on a specifc indication (e.g., depression [4], acute respiratory tract infections in children [5]) while often including open-label trials and observational studies. In this approach, data synthesis is grouped by design, thus yielding information about homoeopathy in patient care. Te opposite approach is to include all indications while restricting study designs to placebo-controlled trials and aggregating results in an MAs, thus yielding information about the specifc efects of homoeopathy beyond those of placebo. A major reason for using this approach has been the claim that ‘homoeopathy violates natural laws and thus any efect must be a placebo efect’ [6].
Since 1997, at least six MAs of placebo-controlled homoeopathy trials for any condition have been published [6–11]. These MAs have difered in their methods for trial inclusion, data synthesis and assessment of risk of bias; furthermore, their results and conclusions have been inconsistent. During this period, there have been substantial advancements in methodology and quality standards for MAs and other SRs [12–15], including SRs of SRs (also called overviews or umbrella reviews) [16–18]. To our knowledge, a formal SR of MAs of randomised placebo-controlled homoeopathy trials for any condition has not been performed. Herein, we report such an SR.
What the authors actually did is this: they knew of the 6 MA; they also knew that they arrived at cautiously positive conclusions; finally they also were aware of the fact that, obviously, the 6 MA included more or less almost the same primary studies. So, by pooling the MAs, they generated a positive result which was no longer marginally positive but strongly. Anyone looking through this strategy (which in effect multiplies the results of many primary studies by factor 6) must realize that this methods creates a false-positive impression.
My suspicion that this paper is a deliberate attempt at misleading us about the ‘efficacy’ (actually it should be effectiveness) is strengthened by further facts:
- One of the two MAs by Mathie et al excluded primary studies that reported positive findings (i.e. mine and the one by Walach et al)
- Funding: Open Access funding enabled and organized by Projekt DEAL. Funding specifcally for this SR was provided by Christophorus-Stiftung (No. 393 CST), Stiftung Marion Meyenburg (Date 24.09.2020), Dr. Hauschka Stiftung (Date 16.11.2020) and Gesellschaft für Pluralität im Gesundheitswesen (Dates 11.06.2021, 22.06.2021). General funding for IFAEMM was provided by the Software-AG Stiftung (SE-P 13544). The funders had no infuence on the writing of the protocol or on the planning, conduct and publication of this SR.
- Competing interests: In the past 3 years, HJH has received research grants from two manufacturers of anthroposophic medicinal products (Wala Heilmittel GmbH, Bad Boll/ Eckwälden, Germany; Weleda AG, Arlesheim Switzerland). Anthroposophic medicine is not based on the homoeopathic simile principle or on drug provings, but some anthroposophic medicinal products are potentized. The two manufacturers had no involvement with the present SR. Anthroposophic medicinal products were not part of the intervention in any of the trials evaluated in the MAs of this SR (Suppl. Table 15). DSR has received a development grant from Heel GmbH (manufacturer of homoeopathic products) for online training in case report writing. AG, KvA and HK declare that they have no competing interests.
- Author details: 1) Institute for Applied Epistemology and Medical Methodology at Witten/ Herdecke University (IFAEMM), Freiburg, Germany. 2) Faculty of Health, Department of Medicine, Chair of Medical Theory, Integrative and Anthroposophic Medicine, Witten/Herdecke University, Witten, Germany. 3) Maryland University of Integrative Health (MUIH), Laurel, MD, USA. 4) Homeopathic Pharmacopoeia Convention of the United States (HPCUS), Southeastern, PA, USA.
Personally, I do not find it surprising that these authors bend over backwards to publish something positive about homeopathy (such things happen in homeopathy all the time). However, I do find it astonishing that an allegedly decent journal passes such pseudoscience for publication as though it is serious science.
Yes, that would be nice!
You want to lose weight?
Just take a few pills an Bob’s your uncle!
There is, of course no shortage of such pills – but do they work?
This study aimed at quantifying and ranking the effects of different nutraceuticals on weight loss. PubMed, Scopus, and Web of Science to November 2022 were searched and all randomized trials (RCTs) evaluating the comparative effects of two or more nutraceuticals, or comparing a nutraceutical against a placebo for weight loss in adults with overweight or obesity were included. A random-effects network meta-analysis was conducted with a Frequentist framework to estimate mean difference [MD] and 95% confidence interval [CI] of the effect of nutraceuticals on weight loss.
One hundred and eleven RCTs with 6171 participants that investigated the effects of 18 nutraceuticals on body weight were eligible. In the main analysis incorporating all trials, there was high certainty of evidence for supplementation of spirulina (MD: -1.77 kg, 95% CI: -2.77, -0.78) and moderate certainty of evidence that supplementation of curcumin (MD: -0.82 kg, 95% CI: -1.33, -0.30), psyllium (MD: -3.70 kg, 95% CI: -5.18, -2.22), chitosan (MD: -1.70 kg, 95% CI: -2.62, -0.78), and Nigella sativa (MD: -2.09 kg, 95%CI: -2.92, -1.26) could result in a small improvement in body weight. Supplementations with green tea (MD: -1.25 kg, 95%CI: -1.68, -0.82) and glucomannan (MD: -1.36 kg, 95%CI: -2.17, -0.54) demonstrated small weight loss, also the certainty of evidence was rated low.
The authors concluded that supplementations with nutraceuticals can result in a small weight loss in adults with overweight or obesity.
The authors tell us little about the methodological quality of the studies. All they did report was this:
Among trials with a low risk of bias, only chitosan (mean difference: −1.72 kg, 95%CI: −3.37, −0.06) and green tea (mean difference: −1.61 kg, 95%CI: −3.14, −0.09) were effective for weight loss compared with placebo. There was no significant weight loss following increased consumption of other nutraceuticals in trials with a low risk of bias.
In view of the lack of reliability of the primary studies, I feel that the conclusions drawn by the authors are not justified. Even though far from recent, I much prefer our own conclusion of a similar data set:
The evidence for most dietary supplements as aids in reducing body weight is not convincing. None of the reviewed dietary supplements can be recommended for over-the-counter use.
In other words, if you want to lose weight, don’t rely on dietary supplements!
This study evaluated the effect of ear acupressure (auriculotherapy) on the weight-gaining pattern of overweight women during pregnancy. It was a single-blinded randomized clinical trial conducted between January and September 2022 and took place in health centers of Qom University of Medical Sciences in Iran.
One-hundred thirty overweight pregnant women were selected by a purposeful sampling method and then divided into two groups by block randomization method. In the intervention group, two seeds were placed in the left ear on the metabolism and stomach points, while two seeds were placed in the right ear on the mouth and appetite points. Participants in the intervention group were instructed to press the seeds six times a day, 20 minutes before a meal for five weeks. For the placebo group, the Vaccaria seedless label was placed at the same points as the intervention group.
A digital scale with an accuracy of 0.1 kg was used to weigh the pregnant women during each visit. Descriptive statistics, independent T-test, chi-square, and repeated measure ANOVA (analysis of variance) test were used to check the research objectives.
There was a statistically significant difference between the auriculotherapy and placebo groups immediately after completing the study (1120.68 ± 425.83 vs. 2704.09 ± 344.96 (g); = 0.018), respectively. Also, there was a substantial difference in the weight gain of women two weeks (793.10 ± 278.38 vs. 1090.32 ± 330.31 (g); < 0.001) and four weeks after the intervention (729.31 ± 241.52 vs. 964.51 ± 348.35 (g); < 0.001) between the auriculotherapy and placebo groups.
The authors concluded that the results of the present study indicated the effectiveness of auriculotherapy in controlling the weight gain of overweight pregnant women. This treatment could be used as a safe method, with easy access, and low cost in low-risk pregnancies.
In order to understand these findings, it is worth reading the methods section of the paper. It explains what actually happened with the two groups:
After providing explanations to familiarize the participants with the working method and answering their questions, the participants were requested to be comfortable. The first author who has an auriculotherapy certificate did the intervention. The intervention began by disinfecting both ears with a 70% alcohol solution. After determining the location of metabolism and stomach points in the left ear and mouth and appetite points in the right ear related to weight and appetite control, the researcher placed the seeds on the desired points… The intervention lasted for a total of 5 weeks. The seeds were changed twice a week (once every three days) by the researcher. The participants in the intervention group were taught to press the seeds 6 times a day for one minute each time. The pressure method was to use moderate stimulation with continuous pressure. In the first session, the researcher fully taught the participants the amount of pressure and the duration of it in a practical way and asked them to do this once in her presence to ensure that it was correct. Participants were recommended to do this preferably 20 minutes before eating. The researcher reminded the participants in the intervention group of their daily interventions by phone or text message. Each night, they were asked to check if they had followed the instructions and completed the daily registration checklist. In each seed replacement session, which was performed every three days, the checklist of the previous session was viewed and checked, and a checklist was received every week at the same time as the participants were weighed. Subjects were also emphasized in case of any symptoms of allergies or infections and pain as soon as possible through the contact number provided to them to discuss the issue with the researcher to remove the seeds.
In the placebo group, instead of real seeds, a label without Vaccaria seed (waterproof fabric adhesive) was placed by the researcher at the desired points in both ears, and the participants did not receive training to compress the points. They also did not receive the list of daily pressing points. All follow-ups and replacement of labels were performed in the same way as the intervention group in the placebo group. Finally, all participants were requested to notify the researcher if any seeds or labels were removed for any reason. It should be noted that pregnant mothers were unaware of the nature of the group to which they belonged.
It seems clear, therefore, that the patients were NOT blinded and that the verum patients received different care and more attention/encouragement than the placebo group. This means firstly that the trial was NOT single-blind, as the authors claim. Secondly, it means that the outcomes were most likely NOT due to ear acupressure at all – they were caused by the non-specific effects of expectation, extra attention, etc. which, in turn, motivated the women to better control their weight. Consequently, the conclusions of this study should be re-phrased:
The results of the present study fail to indicate the effectiveness of auriculotherapy in controlling the weight gain of overweight pregnant women.
In addition, I feel that the researchers, supervisors, peer-reviewers, editors should all bow their heads in shame for trying to mislead us.
This case report aims to describe the effects of craniosacral therapy and acupuncture in a patient with chronic migraine.
A 33-year-old man with chronic migraine was treated with 20 sessions of craniosacral therapy and acupuncture for 8 weeks. The number of migraine and headache days were monitored every month. The pain intensity of headache was measured on the visual analog scale (VAS). Korean Headache Impact Test-6 (HIT-6) and Migraine Specific Quality of Life (MSQoL) were also used.
The number of headache days per month reduced from 28 to 7 after 8 weeks of treatment and to 3 after 3 months of treatment. The pain intensity of headache based on VAS reduced from 7.5 to 3 after 8 weeks and further to < 1 after 3 months of treatment. Furthermore, the patient’s HIT-6 and MSQoL scores improved during the treatment period, which was maintained or further improved at the 3 month follow-up. No side effects were observed during or after the treatment.
The authors concluded that this case indicates that craniosacral therapy and acupuncture could be effective treatments for chronic
migraine. Further studies are required to validate the efficacy of craniosacral therapy for chronic migraine.
So, was the treatment period 8 weeks long or was it 3 months?
No, I am not discussing this article merely for making a fairly petty point. The reason I mention it is diffteren. I think it is time to discuss the relevance of case reports.
What is the purpose of a case report in medicine/healthcare. Here is the abstract of an article entitled “The Importance of Writing and Publishing Case Reports During Medical Training“:
Case reports are valuable resources of unusual information that may lead to new research and advances in clinical practice. Many journals and medical databases recognize the time-honored importance of case reports as a valuable source of new ideas and information in clinical medicine. There are published editorials available on the continued importance of open-access case reports in our modern information-flowing world. Writing case reports is an academic duty with an artistic element.
An article in the BMJ is, I think, more informative:
It is common practice in medicine that when we come across an interesting case with an unusual presentation or a surprise twist, we must tell the rest of the medical world. This is how we continue our lifelong learning and aid faster diagnosis and treatment for patients.
It usually falls to the junior to write up the case, so here are a few simple tips to get you started.
First steps
Begin by sitting down with your medical team to discuss the interesting aspects of the case and the learning points to highlight. Ideally, a registrar or middle grade will mentor you and give you guidance. Another junior doctor or medical student may also be keen to be involved. Allocate jobs to split the workload, set a deadline and work timeframe, and discuss the order in which the authors will be listed. All listed authors should contribute substantially, with the person doing most of the work put first and the guarantor (usually the most senior team member) at the end.
Getting consent
Gain permission and written consent to write up the case from the patient or parents, if your patient is a child, and keep a copy because you will need it later for submission to journals.
Information gathering
Gather all the information from the medical notes and the hospital’s electronic systems, including copies of blood results and imaging, as medical notes often disappear when the patient is discharged and are notoriously difficult to find again. Remember to anonymise the data according to your local hospital policy.
Writing up
Write up the case emphasising the interesting points of the presentation, investigations leading to diagnosis, and management of the disease/pathology. Get input on the case from all members of the team, highlighting their involvement. Also include the prognosis of the patient, if known, as the reader will want to know the outcome.
Coming up with a title
Discuss a title with your supervisor and other members of the team, as this provides the focus for your article. The title should be concise and interesting but should also enable people to find it in medical literature search engines. Also think about how you will present your case study—for example, a poster presentation or scientific paper—and consider potential journals or conferences, as you may need to write in a particular style or format.
Background research
Research the disease/pathology that is the focus of your article and write a background paragraph or two, highlighting the relevance of your case report in relation to this. If you are struggling, seek the opinion of a specialist who may know of relevant articles or texts. Another good resource is your hospital library, where staff are often more than happy to help with literature searches.
How your case is different
Move on to explore how the case presented differently to the admitting team. Alternatively, if your report is focused on management, explore the difficulties the team came across and alternative options for treatment.
Conclusion
Finish by explaining why your case report adds to the medical literature and highlight any learning points.
Writing an abstract
The abstract should be no longer than 100-200 words and should highlight all your key points concisely. This can be harder than writing the full article and needs special care as it will be used to judge whether your case is accepted for presentation or publication.
What next
Discuss with your supervisor or team about options for presenting or publishing your case report. At the very least, you should present your article locally within a departmental or team meeting or at a hospital grand round. Well done!
Both papers agree that case reports can be important. They may provide valuable resources of unusual information that may lead to new research and advances in clinical practice and should offer an interesting case with an unusual presentation or a surprise twist.
I agree!
But perhaps it is more constructive to consider what a case report cannot do.
It cannot provide evidence about the effectiveness of a therapy. To publish something like:
- I had a patient with the common condition xy;
- I treated her with therapy yz;
- this was followed by patient feeling better;
is totally bonkers – even more so if the outcome was subjective and the therapy consisted of more than one intervention, as in the article above. We have no means of telling whether it was treatment A, or treatment B, or a placebo effect, or the regression towards the mean, or the natural history of the condition that caused the outcome. The authors might just as well just have reported:
WE RECENTLY TREATED A PATIENT WHO GOT BETTER
full stop.
Sadly – and this is the reason why I spend some time on this subject – this sort of thing happens very often in the realm of SCAM.
Case reports are particularly valuable if they enable and stimulate others to do more research on a defined and under-researched issue (e.g. an adverse effect of a therapy). Case reports like the one above do not do this. They are a waste of space and tend to be abused as some sort of indication that the treatments in question might be valuable.
This systematic review and meta-analysis was aimed at analyzing the effectiveness of craniosacral therapy in improving pain and disability among patients with headache disorders.
PubMed, Physiotherapy Evidence Database, Scopus, Cochrane Library, Web of Science, and Osteopathic Medicine Digital Library databases were searched in March 2023. Two independent reviewers searched the databases and extracted data from randomized clinical trials comparing craniosacral therapy with control or sham interventions. The same reviewers assessed the methodological quality and the risk of bias using the PEDro scale and the Cochrane Collaboration tool, respectively. Grading of recommendations, assessment, development, and evaluations was used to rate the certainty of the evidence. Meta-analyses were conducted using random effects models using RevMan 5.4 software.
The searches retrieved 735 papers, and 4 studies were finally included. The craniosacral therapy provided statistically significant but clinically unimportant change on pain intensity (Mean difference = –1.10; 95% CI: –1.85, –0.35; I2: 44%), and no change on disability or headache effect (Standardized Mean Difference = –0.34; 95% CI –0.70, 0.01; I2: 26%). The certainty of the evidence was downgraded to very low.
The authors concluded that very low certainty of evidence suggests that craniosacral therapy produces clinically unimportant effects on pain intensity, whereas no significant effects were observed in disability or headache effect.
I find it strange that researchers seem so frequently unable to formulate their conclusions clearly. Is it political correctness? Or are they somehow favorably inclined (i.e. biased) towards the treatment that they pretend to critically evaluate?
Let’s look at the facts related to this review:
- Craniosacral therapy (CST) is utterly implausible.
- Only 4 RCTs were found.
- They were of poor quality.
- They were published mostly by people who want to promote CST.
- Therefore the overall statistically significant effect is most likely a false-positive result.
- This means that the conclusion should be much more straight forward.
I suggest something along the following lines:
A critical evaluation of the existing RCTs failed to find convincing evidence that CST is an effective treatment for headache disorders.
This randomised, double blind controlled trial compared the efficacy of curcumin versus omeprazole in improving patient reported outcomes in people with dyspepsia.
The interventions were:
- curcumin alone (C),
- omeprazole alone (O),
- curcumin plus omeprazole (C+O).
Patients in the combination group received two capsules of 250 mg curcumin, four times daily, and one capsule of 20 mg omeprazole once daily for 28 days.
Main outcome measure was unctional dyspepsia symptoms on days 28 and 56, assessed using the Severity of Dyspepsia Assessment (SODA) score. Secondary outcomes were the occurrence of adverse events and serious adverse events.
A total of 206 patients were enrolled in the study and randomly assigned to one of the three groups; 151 patients completed the study. Demographic data (age 49.7±11.9 years; women 73.4%), clinical characteristics and baseline dyspepsia scores were comparable between the three groups. Significant improvements were observed in SODA scores on day 28 in the pain (−4.83, –5.46 and −6.22), non-pain (−2.22, –2.32 and −2.31) and satisfaction (0.39, 0.79 and 0.60) categories for the C+O, C, and O groups, respectively. These improvements were enhanced on day 56 in the pain (−7.19, –8.07 and −8.85), non-pain (−4.09, –4.12 and −3.71) and satisfaction (0.78, 1.07, and 0.81) categories in the C+O, C, and O groups, respectively. No significant differences were observed among the three groups and no serious adverse events occurred.
The authors concluded that curcumin and omeprazole had comparable efficacy for functional dyspepsia with no obvious synergistic effect.
This study, which was funded by the Thai Traditional and Alternative Medicine Fund, has been picked up by the press and is being lauded as a solid proof of efficacy. Its authors too are not half proud of their splendid trial:
This multicentre randomised controlled trial provides highly reliable evidence for the treatment of functional dyspepsia. PPIs, widely used and approved for over-the-counter use, were compared with curcumin, a popular herbal remedy. The study design, including double blind randomisation, minimised biases. Participants met strict criteria, underwent endoscopy and were tested for H pylori infection. Furthermore, we implemented measures to minimise biases by ensuring that the individuals administering the drugs, participants receiving the drugs and individuals conducting the assessment remained blinded to the type of medications administered to the participants. The trial was carried out in hospitals, and certified individuals used standardised questionnaires for assessments. Statistical methods were appropriate and followed accepted principles.
Two follow-up appointments were scheduled, and blood tests showed no abnormal symptoms or liver function abnormalities. However, participants with high body mass index indicated a trend towards liver function impairment in the curcumin group, suggesting the need for larger studies. Some participants did not provide follow-up information, which is a study weakness. However, the number of participants who provided this information was sufficient for statistical analysis and the majority of the participants attended the follow-up visit. Therefore, it can be deduced from the results that even if the number of participants followed after drug administration increased, the study findings would not be significantly different. Another limitation of this study was the absence of long term follow-up data for all patients after treatment. This is a question that will require further investigation.
The strength of the study lies in its relevance to daily clinical practice, providing additional drug options in addition to PPIs alone, without added side effects. The study was unbiased, partially funded by government organisations and the first well designed trial comparing curcumin with PPI for functional dyspepsia, with confirmation through endoscopy and ruling out H pylori infection. Limitations of this study included the small number of patients who were lost to follow-up and the lack of long term follow-up data.
However, I am far less impressed.
Why?
Curcumin is bright yellow and has a very distinct taste/smell. Even though curumin was given in capsules, patients can easily tell what they are taking. I therefore doubt that they were adequately blinded. In fact, the authors seem to agree when they state the following:
We observed that despite improvements in pain and non-pain scores, there was no significant improvement in the SODA satisfaction scores in the O and C+O groups (table 3). A possible explanation for this observation could be related to the taste and/or smell of curcumin, which might have caused reduced pleasantness for the participants while ingesting it. This potential discomfort could offset the improvements in pain and non-pain symptoms, leading to the non-significant change in satisfaction score. Further studies may be needed to explore this hypothesis as well as to improve the palatability of curcumin.
Sadly, the success of blinding (which under such circumstances should always be tested) was not reported and probably not even quantified. If many patients were de-blinded, it seems inevitable that their expectation influenced the results. In other words, the much-lauded effect of curcumin might just be due to placebo and curcumin might be entirely useless. Or, to put it bluntly, the trial was not nearly as good as many made it out to be.
PS
Sad to see that the reviewers of a reputable journal failed to pick up on this significant flaw.
