study design
It’s again the season for nine lessons, I suppose. So, on the occasion of Christmas Eve, let me rephrase the nine lessons I once gave (with my tongue firmly lodged in my cheek) to those who want to make a pseudo-scientific career in so-called alternative medicine (SCAM) research.
- Throw yourself into qualitative research. For instance, focus groups are a safe bet. They are not difficult to do: you gather 5 -10 people, let them express their opinions, record them, extract from the diversity of views what you recognize as your own opinion and call it a ‘common theme’, and write the whole thing up, and – BINGO! – you have a publication. The beauty of this approach is manifold:
-
- you can repeat this exercise ad nauseam until your publication list is of respectable length;
- there are plenty of SCAM journals that will publish your articles;
- you can manipulate your findings at will;
- you will never produce a paper that displeases the likes of King Charles;
- you might even increase your chances of obtaining funding for future research.
- Conduct surveys. They are very popular and highly respected/publishable projects in SCAM. Do not get deterred by the fact that thousands of similar investigations are already available. If, for instance, there already is one describing the SCAM usage by leg-amputated policemen in North Devon, you can conduct a survey of leg-amputated policemen in North Devon with a medical history of diabetes. As long as you conclude that your participants used a lot of SCAMs, were very satisfied with it, did not experience any adverse effects, thought it was value for money, and would recommend it to their neighbour, you have secured another publication in a SCAM journal.
- In case this does not appeal to you, how about taking a sociological, anthropological or psychological approach? How about studying, for example, the differences in worldviews, the different belief systems, the different ways of knowing, the different concepts about illness, the different expectations, the unique spiritual dimensions, the amazing views on holism – all in different cultures, settings or countries? Invariably, you must, of course, conclude that one truth is at least as good as the next. This will make you popular with all the post-modernists who use SCAM as a playground for enlarging their publication lists. This approach also has the advantage to allow you to travel extensively and generally have a good time.
- If, eventually, your boss demands that you start doing what (in his narrow mind) constitutes ‘real science’, do not despair! There are plenty of possibilities to remain true to your pseudo-scientific principles. Study the safety of your favourite SCAM with a survey of its users. You simply evaluate their experiences and opinions regarding adverse effects. But be careful, you are on thin ice here; you don’t want to upset anyone by generating alarming findings. Make sure your sample is small enough for a false negative result, and that all participants are well-pleased with their SCAM. This might be merely a question of selecting your patients wisely. The main thing is that your conclusions do not reveal any risks.
- If your boss insists you tackle the daunting issue of SCAM’s efficacy, you must find patients who happened to have recovered spectacularly well from a life-threatening disease after receiving your favourite form of SCAM. Once you have identified such a person, you detail her experience and publish this as a ‘case report’. It requires a little skill to brush over the fact that the patient also had lots of conventional treatments, or that her diagnosis was never properly verified. As a pseudo-scientist, you will have to learn how to discretely make such details vanish so that, in the final paper, they are no longer recognisable.
- Your boss might eventually point out that case reports are not really very conclusive. The antidote to this argument is simple: you do a large case series along the same lines. Here you can even show off your excellent statistical skills by calculating the statistical significance of the difference between the severity of the condition before the treatment and the one after it. As long as this reveals marked improvements, ignores all the many other factors involved in the outcome and concludes that these changes are the result of the treatment, all should be tickety-boo.
- Your boss might one day insist you conduct what he narrow-mindedly calls a ‘proper’ study; in other words, you might be forced to bite the bullet and learn how to do an RCT. As your particular SCAM is not really effective, this could lead to serious embarrassment in the form of a negative result, something that must be avoided at all costs. I, therefore, recommend you join for a few months a research group that has a proven track record in doing RCTs of utterly useless treatments without ever failing to conclude that it is highly effective. In other words, join a member of my ALTERNATIVE MEDICINE HALL OF FAME. They will teach you how to incorporate all the right design features into your study without the slightest risk of generating a negative result. A particularly popular solution is to conduct a ‘pragmatic’ trial that never fails to produce anything but cheerfully positive findings.
- But even the most cunningly designed study of your SCAM might one day deliver a negative result. In such a case, I recommend taking your data and running as many different statistical tests as you can find; chances are that one of them will produce something vaguely positive. If even this method fails (and it hardly ever does), you can always focus your paper on the fact that, in your study, not a single patient died. Who would be able to dispute that this is a positive outcome?
- Now that you have grown into an experienced pseudo-scientist who has published several misleading papers, you may want to publish irrefutable evidence of your SCAM. For this purpose run the same RCT over again, and again, and again. Eventually, you want a meta-analysis of all RCTs ever published (see examples here and here). As you are the only person who conducted studies on the SCAM in question, this should be quite easy: you pool the data of all your dodgy trials and, bob’s your uncle: a nice little summary of the totality of the data that shows beyond doubt that your SCAM works and is safe.
Our ‘Memorandum Integrative Medicine‘ seems to be causing ripples. A German website that claims to aim at informing consumers objectively posted a rebuttal. Here is my translation (together with comments by myself inserted via reference numbers in brackets and added below):
With drastic words and narrow-mindedness bordering on ideology (1), the Münster Circle, an association of opponents to complementary therapies such as homeopathy (2), takes issue with the treatment concept of integrative medicine in a memorandum (3). By integrative medicine physicians understand the combination of doctor-led medicine and doctor-led complementary medicine to a meaningful total concept with the goal of reducing side effects and to treating patients individually and optimally (4). Integrative medicine focuses primarily on chronic diseases, where conventional acute medicine often reaches its limits (5)In the memorandum of the Münsteraner Kreis, general practitioner Dr. Claudia Novak criticizes integrative medicine as “guru-like self-dramatization” (6) by physicians and therapists, which undermines evidence-based medicine and leads to a deterioration in patient care. She is joined by Prof. Dr. Edzard Ernst, Professor Emeritus of Alternative Medicine, who has changed from Paul to Saul with regard to homeopathy (7) and is leading a veritable media campaign against proponents of treatment procedures that have not been able to prove their evidence in randomized placebo-controlled studies (8). The professor ignores the fact that this involves a large number of drugs that are used as a matter of course in everyday medicine (9) – for example, beta-blockers or other cardiological drugs (10). “Like the devil fears the holy water” (11), the Münsteraner Kreis seems to fear the concept of integrative medicine (12). The vehemence coupled with fear with which they warn against the treatment concept makes one sit up and take notice (13). “As an experienced gynecologist who has successfully worked with biological medicine as an adjunct in his practice for decades, I can only shake my head at such narrow-mindedness”, points out Fred-Holger Ludwig, MD (14). Science does not set limits for itself, but the plurality of methods is immanent (15). “Why doesn’t Prof. Ernst actually give up his professorial title for alternative medicine? That would have to be the logical consequence of its overloud criticism of established treatment concepts from homeopathy to to integrative medicine”, questions Dr. Ludwig (16).
The concept of integrative medicine is about infiltrating alternative procedures into medicine, claim the critics of the concept, without mentioning that many naturopathic procedures have been used for centuries with good results (17) and that healthcare research gives them top marks (18). “Incidentally, the scientists among the representatives of the Münster Circle should know that it is difficult to capture individualized treatment concepts with the standardized procedures of randomized, placebo-controlled studies (19). Anyone who declares the highest level of evidence to be the criterion for approval makes medicine impossible and deprives patients in oncology or with rare diseases, for example, of chances of successful treatment (20). Even there, drugs are used that cannot be based on high evidence, tested in placebo-controlled studies, because the number of cases is too low (21),” notes Dr. Ludwig .
- Ideology? Evidence is not ideology, in my view.
- We are an association of multidisciplinary experts advocating a level playing field with sound evidence in all areas of healthcare.
- The actual memorandum is not linked in this text; does the author not want his readers to form the own opinion?
- In our memorandum, we offer various definitions of integrative medicine (IM), none of which is remotely similar to this one.
- No, IM is usually being promoted in a much wider sense.
- This term does not appear in our memorandum.
- I am not aware that I changed from Paul to Saul with regard to homeopathy; I know that I was led mostly by the evidence.
- I feel flattered but don’t think that my humble work is a ‘media campaign’.
- True, I do not pretend to understand all areas of medicine and tend to be silent in the ones that I lack up-to-date expertise.
- Is he really saying that beta-blockers are not evidence-based?
- The holy water comparison from a homeopath, who arguably makes a living from dishing out ‘holy water’, made me laugh!
- It is most revealing, I think, that he thinks our motivation is fear.
- Splendid!
- FHL is the author of the article, and it is thus charmingly naive that he cites himself in this way
- I somehow doubt that he understands what he is expressing here.
- I find this rather a bizarre idea but I’ll think about it.
- Argumentum ad traditionem.
- Those that get ‘top marks’ belong to evidence-based medicine and not to IM.
- Here the author reveals that he does not understand the RCT methodology and even fails to know the trial evidence on homeopathy – RCTs of individualised homeopathy are possible and have been published (e.g. this one).
- If he really believes this, I fear for his patients.
- Pity that he does not provide an example.
To understand FHL better, it is worth knowing that he claims to treat cancer patients with conventional and homeopathic medicine. He states that this approach reduces side effects – without providing evidence, of course.
Altogether, FHL does not dispute a single fact or argument from our memorandum. In fact, I get the impression that he never actually read it. To me, it feels as though he merely read an article ABOUT the document. In any case, his critique is revealing and important, in my view. It demonstrates that there are no good arguments to defend IM.
So, thank you FHL!
Hardly a day goes by that I am not asked by someone – a friend, colleague, practitioner, journalist, etc. – about the evidence for this or that so-called alternative medicine (SCAM). I always try my best to give a truthful answer, and often it amounts to something like this: TO THE BEST OF MY KNOWLEDGE, THERE IS NO GOOD EVIDENCE TO SHOW THAT IT WORKS.
The reactions to this news vary, e.g.:
- Some ignore it and seem to think ‘what does he know?’.
- Some thank me and make their decisions accordingly.
- Some feel they better do a fact-check.
The latter reaction is perhaps the most interesting because often the person, clearly an enthusiast of that particular SCAM, later comes back to me and triumphantly shows me evidence that contradicts my statement.
This means I now must have a look at what evidence he/she has found.
It can fall into several categories:
- Opinion articles published by proponents of the SCAM in question.
- Papers that are not truly relevant to the SCAM.
- Research that provides data about the SCAM that does not relate to its effectiveness, e.g. surveys, or qualitative studies.
- Studies of the SCAM in question.
It is usually easy to explain why the three first-named categories are irrelevant. Yet, the actual studies can be a problem. Remember, I told that person that no good evidence exists, and now he (let’s assume I am dealing with a man) proudly shows me a study of it suggesting the opposite. There might be the following explanations:
- I did not know this high-quality study (e.g. because it is new) and my dismissive statement was thus questionable or wrong.
- The study draws a positive conclusion about the SCAM but this conclusion is not justified.
In the first instance, do I need to change my mind and apologize for my wrong statement? Perhaps! But I also need to explain that, even with a rigorous study, we really ought to have one (better more than one) independent replication before we start changing our clinical routine.
In the second instance, I need to explain why the conclusion is not justified. The realm of SCAM is plagued by studies with misleading conclusions (as regular readers of this blog know only too well). Therefore, this situation arises with some regularity. There are numerous reasons why a study can generate unreliable findings (as regular readers of this blog know only too well). Some of them are easy to understand others might be more difficult for non-scientists to comprehend. This means that the discussions with the man who proudly brought the ‘evidence’ to my attention can be tedious.
Often he feels that I am unfair to his favorite SCAM. He might argue that:
- I am biased;
- I lack an open mind;
- I am not qualified;
- I am changing the goalpost;
- I am applying double standards because much of the research into conventional medicine is also not flawless.
In such cases, we are likely to eventually end our discussions by agreeing to disagree. He will be convinced of his point of view and I will be convinced of mine. Essentially, we are more or less where we started, and the whole palaver was for nothing.
… a bit like this post?
I hope not!
What I have been trying to demonstrate is that:
- SCAM enthusiasts are often difficult, sometimes impossible to convince;
- research is not always easy to understand and requires a minimum of education and know-how.
Earlier this year, I started the ‘WORST PAPER OF 2022 COMPETITION’. As a prize, I am offering the winner (that is the lead author of the winning paper) one of my books that best fits his/her subject. I am sure this will overjoy him or her. I hope to identify about 10 candidates for the prize, and towards the end of the year, I let my readers decide democratically on who should be the winner. In this spirit of democratic voting, let me suggest to you entry No 9. Here is the unadulterated abstract:
Background
With the increasing popularity of traditional Chinese medicine (TCM) by the global community, how to teach basic knowledge of TCM to international students and improve the teaching quality are important issues for teachers of TCM. The present study was to analyze the perceptions from both students and teachers on how to improve TCM learning internationally.
Methods
A cross-sectional national survey was conducted at 23 universities/colleges across China. A structured, self-reported on-line questionnaire was administered to 34 Chinese teachers who taught TCM course in English and to 1016 international undergraduates who were enrolled in the TCM course in China between 2017 and 2021.
Results
Thirty-three (97.1%) teachers and 900 (88.6%) undergraduates agreed Chinese culture should be fully integrated into TCM courses. All teachers and 944 (92.9%) undergraduates thought that TCM had important significance in the clinical practice. All teachers and 995 (97.9%) undergraduates agreed that modern research of TCM is valuable. Thirty-three (97.1%) teachers and 959 (94.4%) undergraduates thought comparing traditional medicine in different countries with TCM can help the students better understand TCM. Thirty-two (94.1%) teachers and 962 (94.7%) undergraduates agreed on the use of practical teaching method with case reports. From the perceptions of the undergraduates, the top three beneficial learning styles were practice (34.3%), teacher’s lectures (32.5%), case studies (10.4%). The first choice of learning mode was attending to face-to-face teaching (82.3%). The top three interesting contents were acupuncture (75.5%), Chinese herbal medicine (63.8%), and massage (55.0%).
Conclusion
To improve TCM learning among international undergraduates majoring in conventional medicine, integration of Chinese culture into TCM course, comparison of traditional medicine in different countries with TCM, application of the teaching method with case reports, and emphasization of clinical practice as well as modern research on TCM should be fully considered.
I am impressed with this paper mainly because to me it does not make any sense at all. To be blunt, I find it farcically nonsensical. What precisely? Everything:
- the research question,
- the methodology,
- the conclusion
- the write-up,
- the list of authors and their affiliations: Department of Chinese Integrative Medicine, Women’s Hospital, School of Medicine, Zhejiang University, Hangzhou, China, Department of Traditional Chinese Medicine, School of Basic Medicine, Qingdao University, Qingdao, China, Department of Chinese Integrative Medicine, The Second Affiliated Hospital of Kunming Medical University, Kunming, China, Department of Traditional Chinese Medicine, The Affiliated Hospital of Xuzhou Medical University, Xuzhou, China, Department of Traditional Chinese Medicine, Medical College, China Three Gorges University, Yichang, China, Basic Teaching and Research Department of Acupuncture and Moxibustion, College of Traditional Chinese Medicine, Xinjiang Medical University, Urumqi, China, Institute of Integrative Medicine, Dalian Medical University, Dalian, China, Department of Chinese and Western Medicine, Chongqing Medical University, Chongqing, China, Department of Chinese and Western Medicine, North Sichuan Medical College, Nanchong, China, Department of Chinese and Western Medicine, School of Medicine, Xiamen University, Xiamen, China, School of Traditional Chinese Medicine, Capital Medical University, Beijing, China, School of Traditional Chinese Medicine, Southern Medical University, Guangzhou, China, Department of Traditional Chinese Medicine, The First Affiliated Hospital of Soochow University, Suzhou, China, Department of Traditional Chinese Medicine, School of Medicine, Xiamen University, Xiamen, China, Department of Chinese Medicine/Department of Chinese Integrative Medicine, The First Affiliated Hospital of Anhui Medical University, Hefei, China, Department of Traditional Chinese Medicine, Shengjing Hospital Affiliated to China Medical University, Shenyang, China, Department of Acupuncture, Affiliated Hospital of Jiangsu University, Zhenjiang, China, Teaching and Research Section of Traditional Chinese Medicine, The Second Affiliated Hospital of Soochow University, Suzhou, China, Department of Traditional Chinese Medicine, The Second Affiliated Hospital of Harbin Medical University, Harbin, China, Department of Chinese Medicine, The First Affiliated Hospital of Anhui Medical University, Hefei, China, Department of Chinese Medicine, The First Affiliated Hospital of Kunming Medical University, Kunming, China, Department of Traditional Chinese Medicine, Shengli Clinical Medical College of Fujian Medical University, Fuzhou, China, Department of Chinese Medicine, The First Affiliated Hospital of Jinzhou Medicine University, Jinzhou, China, Department of Integrated Traditional and Western Medicine, The First Affiliated Hospital of Harbin Medical University, Harbin, China, Department of Chinese Medicine, The Second Affiliated Hospital of Guangzhou Medical University, Guangzhou, China, Department of Traditional Chinese Medicine, The First Affiliated Hospital of Fujian Medical University, Fuzhou, China.
- the journal that had this paper peer-reviewed and published.
But what impressed me most with this paper is the way the authors managed to avoid even the slightest hint of critical thinking. They even included a short paragraph in the discussion section where they elaborate on the limitations of their work without ever discussing the true flaws in the conception and execution of this extraordinary example of pseudoscience.
Today, you cannot read a newspaper or listen to the radio without learning that there has been a significant, sensational, momentous, unprecedented, etc. breakthrough in the treatment of Alzheimer’s disease. The reason for all this excitement (or is it hype?) is this study just out in the NEJM:
BACKGROUND
The accumulation of soluble and insoluble aggregated amyloid-beta (Aβ) may initiate or potentiate pathologic processes in Alzheimer’s disease. Lecanemab, a humanized IgG1 monoclonal antibody that binds with high affinity to Aβ soluble protofibrils, is being tested in persons with early Alzheimer’s disease.
METHODS
We conducted an 18-month, multicenter, double-blind, phase 3 trial involving persons 50 to 90 years of age with early Alzheimer’s disease (mild cognitive impairment or mild dementia due to Alzheimer’s disease) with evidence of amyloid on positron-emission tomography (PET) or by cerebrospinal fluid testing. Participants were randomly assigned in a 1:1 ratio to receive intravenous lecanemab (10 mg per kilogram of body weight every 2 weeks) or placebo. The primary end point was the change from baseline at 18 months in the score on the Clinical Dementia Rating–Sum of Boxes (CDR-SB; range, 0 to 18, with higher scores indicating greater impairment). Key secondary end points were the change in amyloid burden on PET, the score on the 14-item cognitive subscale of the Alzheimer’s Disease Assessment Scale (ADAS-cog14; range, 0 to 90; higher scores indicate greater impairment), the Alzheimer’s Disease Composite Score (ADCOMS; range, 0 to 1.97; higher scores indicate greater impairment), and the score on the Alzheimer’s Disease Cooperative Study–Activities of Daily Living Scale for Mild Cognitive Impairment (ADCS-MCI-ADL; range, 0 to 53; lower scores indicate greater impairment).
RESULTS
A total of 1795 participants were enrolled, with 898 assigned to receive lecanemab and 897 to receive placebo. The mean CDR-SB score at baseline was approximately 3.2 in both groups. The adjusted least-squares mean change from baseline at 18 months was 1.21 with lecanemab and 1.66 with placebo (difference, −0.45; 95% confidence interval [CI], −0.67 to −0.23; P<0.001). In a substudy involving 698 participants, there were greater reductions in brain amyloid burden with lecanemab than with placebo (difference, −59.1 centiloids; 95% CI, −62.6 to −55.6). Other mean differences between the two groups in the change from baseline favoring lecanemab were as follows: for the ADAS-cog14 score, −1.44 (95% CI, −2.27 to −0.61; P<0.001); for the ADCOMS, −0.050 (95% CI, −0.074 to −0.027; P<0.001); and for the ADCS-MCI-ADL score, 2.0 (95% CI, 1.2 to 2.8; P<0.001). Lecanemab resulted in infusion-related reactions in 26.4% of the participants and amyloid-related imaging abnormalities with edema or effusions in 12.6%.
CONCLUSIONS
Lecanemab reduced markers of amyloid in early Alzheimer’s disease and resulted in moderately less decline on measures of cognition and function than placebo at 18 months but was associated with adverse events. Longer trials are warranted to determine the efficacy and safety of lecanemab in early Alzheimer’s disease. (Funded by Eisai and Biogen; Clarity AD ClinicalTrials.gov number, NCT03887455. opens in new tab.)
It’s a good study, and I (like everyone else) hope that it will mean tangible progress in the management of that devastating disease. Most media outlets are announcing the news with the claim that it is the FIRST TIME that any treatment has been shown to delay the cognitive decline of Alzheimer’s disease patients.
But is this true?
I think not!
There have been several studies showing that the herbal remedy GINKGO BILOBA slows the Alzheimer-related decline. Here is the latest systematic review of the subject:
Background: Ginkgo biloba is a natural medicine used for cognitive impairment and Alzheimer’s disease. The objective of this review is to explore the effectiveness and safety of Ginkgo biloba in treating mild cognitive impairment and Alzheimer’s disease.
Methods: Electronic search was conducted from PubMed, Cochrane Library, and four major Chinese databases from their inception up to 1(st) December, 2014 for randomized clinical trials on Ginkgo biloba in treating mild cognitive impairment or Alzheimer’s disease. Meta-analyses were performed by RevMan 5.2 software.
Results: 21 trials with 2608 patients met the inclusion criteria. The general methodological quality of included trials was moderate to poor. Compared with conventional medicine alone, Ginkgo biboba in combination with conventional medicine was superior in improving Mini-Mental State Examination (MMSE) scores at 24 weeks for patients with Alzheimer’s disease (MD 2.39, 95% CI 1.28 to 3.50, P<0.0001) and mild cognitive impairment (MD 1.90, 95% CI 1.41 to 2.39, P<0.00001), and Activity of Daily Living (ADL) scores at 24 weeks for Alzheimer’s disease (MD -3.72, 95% CI -5.68 to -1.76, P=0.0002). When compared with placebo or conventional medicine in individual trials, Ginkgo biboba demonstrated similar but inconsistent findings. Adverse events were mild.
Conclusion: Ginkgo biloba is potentially beneficial for the improvement of cognitive function, activities of daily living, and global clinical assessment in patients with mild cognitive impairment or Alzheimer’s disease. However, due to limited sample size, inconsistent findings and methodological quality of included trials, more research are warranted to confirm the effectiveness and safety of ginkgo biloba in treating mild cognitive impairment and Alzheimer’s disease.
I know, the science is not nearly as good as that of the NEJM trial. I also know that the trial data for ginkgo biloba are not uniformly positive. And I know that, after several studies showed good results, later trials tended not to confirm them.
But this is what very often happens in clinical research: studies are initially promising, only to be disappointing as more studies emerge. I sincerely hope that this will not happen with the new drug ‘Lecanemab’ and that today’s excitement will not turn out to be hype.
Electroacupuncture (EA) is often advocated for depression and sleep disorders but its efficacy remains uncertain. The aim of this study was, therefore, to “assess the efficacy and safety of EA as an alternative therapy in improving sleep quality and mental state for patients with insomnia and depression.”
A 32-week patient- and assessor-blinded, randomized, sham-controlled clinical trial (8-week intervention plus 24-week follow-up) was conducted from September 1, 2016, to July 30, 2019, at 3 tertiary hospitals in Shanghai, China. Patients were randomized to receive
- EA treatment and standard care,
- sham acupuncture (SA) treatment and standard care,
- standard care only as control.
Patients in the EA or SA groups received a 30-minute treatment 3 times per week (usually every other day except Sunday) for 8 consecutive weeks. All treatments were performed by licensed acupuncturists with at least 5 years of clinical experience. A total of 6 acupuncturists (2 at each center; including X.Y. and S.Z.) performed EA and SA, and they received standardized training on the intervention method before the trial. The regular acupuncture method was applied at the Baihui (GV20), Shenting (GV24), Yintang (GV29), Anmian (EX-HN22), Shenmen (HT7), Neiguan (PC6), and SanYinjiao (SP6) acupuncture points, with 0.25 × 25-mm and 0.30 × 40-mm real needles (Wuxi Jiajian Medical Device Co, Ltd), or 0.30 × 30-mm sham needles (Streitberger sham device [Asia-med GmbH]).
For patients in the EA group, rotating or lifting-thrusting manipulation was applied for deqi sensation after needle insertion. The 2 electrodes of the electrostimulator (CMNS6-1 [Wuxi Jiajian Medical Device Co, Ltd]) were connected to the needles at GV20 and GV29, delivering a continuous wave based on the patient’s tolerance. Patients in the SA group felt a pricking sensation when the blunt needle tip touched the skin, but without needle insertion. All indicators of the nearby electrostimulator were set to 0, with the light switched on. Standard care (also known as treatment as usual or routine care) was used in the control group. Patients receiving standard care were recommended by the researchers to get regular exercise, eat a healthy diet, and manage their stress level during the trial. They were asked to keep the regular administration of antidepressants, sedatives, or hypnotics as well. Psychiatrists in the Shanghai Mental Health Center (including X.L.) guided all patients’ standard care treatment and provided professional advice when a patient’s condition changed.
The primary outcome was change in Pittsburgh Sleep Quality Index (PSQI) from baseline to week 8. Secondary outcomes included PSQI at 12, 20, and 32 weeks of follow-up; sleep parameters recorded in actigraphy; Insomnia Severity Index; 17-item Hamilton Depression Rating Scale score; and Self-rating Anxiety Scale score.
Among the 270 patients (194 women [71.9%] and 76 men [28.1%]; mean [SD] age, 50.3 [14.2] years) included in the intention-to-treat analysis, 247 (91.5%) completed all outcome measurements at week 32, and 23 (8.5%) dropped out of the trial. The mean difference in PSQI from baseline to week 8 within the EA group was -6.2 (95% CI, -6.9 to -5.6). At week 8, the difference in PSQI score was -3.6 (95% CI, -4.4 to -2.8; P < .001) between the EA and SA groups and -5.1 (95% CI, -6.0 to -4.2; P < .001) between the EA and control groups. The efficacy of EA in treating insomnia was sustained during the 24-week postintervention follow-up. Significant improvement in the 17-item Hamilton Depression Rating Scale (-10.7 [95% CI, -11.8 to -9.7]), Insomnia Severity Index (-7.6 [95% CI, -8.5 to -6.7]), and Self-rating Anxiety Scale (-2.9 [95% CI, -4.1 to -1.7]) scores and the total sleep time recorded in the actigraphy (29.1 [95% CI, 21.5-36.7] minutes) was observed in the EA group during the 8-week intervention period (P < .001 for all). No between-group differences were found in the frequency of sleep awakenings. No serious adverse events were reported.
The result of the blinding assessment showed that 56 patients (62.2%) in the SA group guessed wrongly about their group assignment (Bang blinding index, −0.4 [95% CI, −0.6 to −0.3]), whereas 15 (16.7%) in the EA group also guessed wrongly (Bang blinding index, 0.5 [95% CI, 0.4-0.7]). This indicated a relatively higher degree of blinding in the SA group.
The authors concluded that, in this randomized clinical trial of EA treatment for insomnia in patients with depression, quality of sleep improved significantly in the EA group compared with the SA or control group at week 8 and was sustained at week 32.
This trial seems rigorous, it has a sizable sample size, uses a credible placebo procedure, and is reported in sufficient detail. Why then am I skeptical?
- Perhaps because we have often discussed how untrustworthy acupuncture studies from China are?
- Perhaps because I fail to see a plausible mechanism of action?
- Perhaps because the acupuncturists could not be blinded and thus might have influenced the outcome?
- Perhaps because the effects of sham acupuncture seem unreasonably small?
- Perhaps because I cannot be sure whether the acupuncture or the electrical current is supposed to have caused the effects?
- Perhaps because the authors of the study are from institutions such as the Shanghai Municipal Hospital of Traditional Chinese Medicine, the Department of Acupuncture and Moxibustion, Huadong Hospital, Fudan University, Shanghai,
- Perhaps because the results seem too good to be true?
If you have other and better reasons, I’d be most interested to hear them.
This double-blind, randomized study assessed the effectiveness of physiotherapy instrument mobilization (PIM) in patients with low back pain (LBP) and compared it with the effectiveness of manual mobilization.
Thirty-two participants with LBP were randomly assigned to one of two groups:
- The PIM group received lumbar mobilization using an activator instrument, stabilization exercises, and education.
- The manual group received lumbar mobilization using a pisiform grip, stabilization exercises, and education.
Both groups had 4 treatment sessions over 2-3 weeks. The following outcomes were measured before the intervention, and after the first and fourth sessions:
- Numeric Pain Rating Scale (NPRS),
- Oswestry Disability Index (ODI) scale,
- Pressure pain threshold (PPT),
- lumbar spine range of motion (ROM),
- lumbar multifidus muscle activation.
There were no differences between the PIM and manual groups in any outcome measures. However, over the period of study, there were improvements in both groups in NPRS (PIM: 3.23, Manual: 3.64 points), ODI (PIM: 17.34%, Manual: 14.23%), PPT (PIM: ⩽ 1.25, Manual: ⩽ 0.85 kg.cm2), lumbar spine ROM (PIM: ⩽ 9.49∘, Manual: ⩽ 0.88∘), and/or lumbar multifidus muscle activation (percentage thickness change: PIM: ⩽ 4.71, Manual: ⩽ 4.74 cm; activation ratio: PIM: ⩽ 1.17, Manual: ⩽ 1.15 cm).
The authors concluded that both methods of lumbar spine mobilization demonstrated comparable improvements in pain and disability in patients with LBP, with neither method exhibiting superiority over the other.
If this conclusion is meant to tell us that both treatments were equally effective, I beg to differ. The improvements documented here are consistent with improvements caused by the natural history of the condition, regression towards the mean, and placebo effects. The data do not prove that they are due to the treatments. On the contrary, they seem to imply that patients get better no matter what therapy is used. Thus, I feel that the results are entirely in keeping with the hypothesis that spinal mobilization is a placebo treatment.
So, allow me to re-phrase the authors’ conclusion as follows:
Lumbar mobilizations do not seem to have specific therapeutic effects and might therefore be considered to be ineffective for LBP.
Acupuncture is emerging as a potential therapy for relieving pain, but the effectiveness of acupuncture for relieving low back and/or pelvic pain (LBPP) during pregnancy remains controversial. This meta-analysis aimed to investigate the effects of acupuncture on pain, functional status, and quality of life for women with LBPP pain during pregnancy.
The authors included all RCTs evaluating the effects of acupuncture on LBPP during pregnancy. Data extraction and study quality assessments were independently performed by three reviewers. The mean differences (MDs) with 95% CIs for pooled data were calculated. The primary outcomes were pain, functional status, and quality of life. The secondary outcomes were overall effects (a questionnaire at a post-treatment visit within a week after the last treatment to determine the number of people who received good or excellent help), analgesic consumption, Apgar scores >7 at 5 min, adverse events, gestational age at birth, induction of labor and mode of birth.
Ten studies, reporting on a total of 1040 women, were included. Overall, acupuncture
- relieved pain during pregnancy (MD=1.70, 95% CI: (0.95 to 2.45), p<0.00001, I2=90%),
- improved functional status (MD=12.44, 95% CI: (3.32 to 21.55), p=0.007, I2=94%),
- improved quality of life (MD=−8.89, 95% CI: (−11.90 to –5.88), p<0.00001, I2 = 57%).
There was a significant difference in overall effects (OR=0.13, 95% CI: (0.07 to 0.23), p<0.00001, I2 = 7%). However, there was no significant difference in analgesic consumption during the study period (OR=2.49, 95% CI: (0.08 to 80.25), p=0.61, I2=61%) and Apgar scores of newborns (OR=1.02, 95% CI: (0.37 to 2.83), p=0.97, I2 = 0%). Preterm birth from acupuncture during the study period was reported in two studies. Although preterm contractions were reported in two studies, all infants were in good health at birth. In terms of gestational age at birth, induction of labor, and mode of birth, only one study reported the gestational age at birth (mean gestation 40 weeks).
The authors concluded that acupuncture significantly improved pain, functional status and quality of life in women with LBPP during the pregnancy. Additionally, acupuncture had no observable severe adverse influences on the newborns. More large-scale and well-designed RCTs are still needed to further confirm these results.
What should we make of this paper? 
In case you are in a hurry: NOT A LOT!
In case you need more, here are a few points:
- many trials were of poor quality;
- there was evidence of publication bias;
- there was considerable heterogeneity within the studies.
The most important issue is one studiously avoided in the paper: the treatment of the control groups. One has to dig deep into this paper to find that the control groups could be treated with “other treatments, no intervention, and placebo acupuncture”. Trials comparing acupuncture combined plus other treatments with other treatments were also considered to be eligible. In other words, the analyses included studies that compared acupuncture to no treatment at all as well as studies that followed the infamous ‘A+Bversus B’ design. Seven studies used no intervention or standard of care in the control group thus not controlling for placebo effects.
Nobody can thus be in the slightest surprised that the overall result of the meta-analysis was positive – false positive, that is! And the worst is that this glaring limitation was not discussed as a feature that prevents firm conclusions.
Dishonest researchers?
Biased reviewers?
Incompetent editors?
Truly unbelievable!!!
In consideration of these points, let me rephrase the conclusions:
The well-documented placebo (and other non-specific) effects of aacupuncture improved pain, functional status and quality of life in women with LBPP during the pregnancy. Unsurprisingly, acupuncture had no observable severe adverse influences on the newborns. More large-scale and well-designed RCTs are not needed to further confirm these results.
PS
I find it exasperating to see that more and more (formerly) reputable journals are misleading us with such rubbish!!!
The aim of this evaluator-blinded randomized clinical trial was to determine if manual therapy added to a therapeutic exercise program produced greater improvements than a sham manual therapy added to the same exercise program in patients with non-specific shoulder pain. 
Forty-five subjects were randomly allocated into one of three groups:
- manual therapy (glenohumeral mobilization technique and rib-cage technique);
- thoracic sham manual therapy (glenohumeral mobilization technique and rib-cage sham technique);
- sham manual therapy (sham glenohumeral mobilization technique and rib-cage sham technique).
All groups also received a therapeutic exercise program. Pain intensity, disability, and pain-free active shoulder range of motion were measured post-treatment and at 4-week and 12-week follow-ups. Mixed-model analyses of variance and post hoc pairwise comparisons with Bonferroni corrections were constructed for the analysis of the outcome measures.
All groups reported improved pain intensity, disability, and pain-free active shoulder range of motion. However, there were no between-group differences in these outcome measures.
The authors concluded that the addition of the manual therapy techniques applied in the present study to a therapeutic exercise protocol did not seem to add benefits to the management of subjects with non-specific shoulder pain.
What does that mean?
I think it means that the improvements observed in this study were due to 1) exercise and 2) a range of non-specific effects, and that they were not due to the manual techniques tested.
I cannot say that I find this enormously surprising. But I would also find it unsurprising if fans of these methods would claim that the results show that the physios applied the techniques not correctly.
In any case, I feel this is an interesting study, not least because of its use of sham therapy. But I somehow doubt that the patients were unable to distinguish sham from verum. If so, the study was not patient-blind which obviously is difficult to achieve with manual treatments.
The aim of this study was to evaluate the effectiveness of craniosacral therapy on different features in migraine patients.
Fifty individuals with migraine were randomly divided into two groups (n = 25 per group):
- craniosacral therapy group (CTG),
- sham control group (SCG).
The interventions were carried out with the patient in the supine position. The CTG received a manual therapy treatment focused on the craniosacral region including five techniques, and the SCG received a hands-on placebo intervention. After the intervention, individuals remained supine with a neutral neck and head position for 10 min, to relax and diminish tension after treatment. The techniques were executed by the same experienced physiotherapist in both groups.
The analyzed variables were pain, migraine severity, and frequency of episodes, functional, emotional, and overall disability, medication intake, and self-reported perceived changes, at baseline, after a 4-week intervention, and at an 8-week follow-up.
After the intervention, the CTG significantly reduced pain (p = 0.01), frequency of episodes (p = 0.001), functional (p = 0.001) and overall disability (p = 0.02), and medication intake (p = 0.01), as well as led to a significantly higher self-reported perception of change (p = 0.01), when compared to SCG. The results were maintained at follow-up evaluation in all variables.
The authors concluded that a protocol based on craniosacral therapy is effective in improving pain, frequency of episodes, functional and overall disability, and medication intake in migraineurs. This protocol may be considered as a therapeutic approach in migraine patients.
Sorry, but I disagree!
And I have several reasons for it:
- The study was far too small for such strong conclusions.
- For considering any treatment as a therapeutic approach in migraine patients, we would need at least one independent replication.
- There is no plausible rationale for craniosacral therapy to work for migraine.
- The blinding of patients was not checked, and it is likely that some patients knew what group they belonged to.
- There could have been a considerable influence of the non-blinded therapists on the outcomes.
- There was a near-total absence of a placebo response in the control group.
Altogether, the findings seem far too good to be true.
