MD, PhD, FMedSci, FRSB, FRCP, FRCPEd.

critical thinking

Wellness?

Yes, it’s a new buzz-word in the realm of alternative medicine – actually, not so new; it’s been around for years and seems to attract charlatans of all imaginable types.

But what precisely is it?

The authors of this paper explain: “While the concept of wellness is still evolving, it is generally recognized that wellness is a holistic concept best represented as a continuum, with sickness, premature death, disability, and reactive approaches to health on one side and high-level wellness, enhanced health, and proactive approaches to health and well-being on the other. It is further acknowledged that wellness is multidimensional and includes physiologic, psychological, social, ecologic, and economic dimensions. These multiple dimensions make wellness difficult to accurately assess as multiple subjective and objective measures are required to account for the different dimensions. Thus, the assessment of wellness in individuals may include a variety of factors, including assessment of physiologic functioning, anthropometry, happiness, depression, anxiety, mood, sleep, health symptoms, toxic load, neurocognitive function, socioeconomic status, social connectivity, and perceived self-efficacy.”

Sounds a bit woolly?

I agree! It sounds like a gimmick for getting at the cash of the gullible public.

Is there money to be made with ‘wellness’?

Sure! Lots!

For instance, with so-called ‘wellness retreats’.

Wellness retreats are all the rage. They use all sorts of bogus therapies within luxurious holiday settings for the ‘well to do’ end of our societies.

But is there any science behind this approach?

Few studies have evaluated the effect of retreat experiences, and no published studies have reported health outcomes. The objective of this new study therefore was to assess the effect of a week-long wellness-retreat experience in wellness tourists. The study was designed as a longitudinal observational study without a control group. Outcomes were assessed upon arrival and departure and 6 weeks after the retreat. The intervention was a ‘holistic, 1-week, residential, retreat experience that included many educational, therapeutic, and leisure activities and an organic, mostly plant-based diet’.

The outcome measures included anthropometric measures, urinary pesticide metabolites, a food and health symptom questionnaire, the Five Factor Wellness Inventory, the General Self Efficacy questionnaire, the Pittsburgh Insomnia Rating Scale, the Depression Anxiety Stress Scale, the Profile of Mood States, and the Cogstate cognitive function test battery.

Statistically significant improvements were seen in almost all measures after 1 week. Many of these improvements were also sustained at 6 weeks. There were statistically significant improvements in all anthropometric measures after 1 week, with reductions in abdominal girth, weight, and average systolic and diastolic pressure. Statistically significant improvements were also noted in psychological and health symptom measures. Urinary pesticide metabolites were detected in pooled urine samples before the retreat and were undetectable after the retreat.

The authors concluded that “the retreat experiences can lead to substantial improvements in multiple dimensions of health and well-being that are maintained for 6 weeks. Further research that includes objective biomarkers and economic measures in different populations is required to determine the mechanisms of these effects and assess the value and relevance of retreat experiences to clinicians and health insurers.”

IS THIS GOOD OR BAD RESEARCH?

Let’s apply my checklist from the previous post:

  • published in one of the many dodgy CAM journals? YES
  • single author? NO
  • authors are known to be proponents of the treatment tested? YES
  • author has previously published only positive studies of the therapy in question? YES
  • lack of plausible rationale for the study? YES
  • lack of plausible rationale for the therapy that is being tested? YES
  • stated aim of the study is ‘to demonstrate the effectiveness of…’ ? NO
  • stated aim ‘to establish the effectiveness AND SAFETY of…’? NO
  • text full of mistakes, e. g. spelling, grammar, etc.? NO
  • sample size is tiny? YES
  • pilot study reporting anything other than the feasibility of a definitive trial? NO
  • methods not described in sufficient detail? YES
  • mismatch between aim, method, and conclusions of the study? YES
  • results presented only as a graph? NO
  • statistical approach inadequate or not sufficiently detailed? NO
  • discussion without critical input? NO
  • lack of disclosures of ethics, funding or conflicts of interest? NO
  • conclusions which are not based on the results? YES

To me, this rough and ready assessment indicates that there are too many warning signals for characterising this as a rigorous study. It looks a lot like pseudo-science, I fear.

But these are at best formal markers. More important is the fact that the whole idea of measuring the effects of a ‘wellness retreat’ makes little sense, particularly in the absence of a control group. If we take a few people out of their usual, stressful work-environment and put them into a nice and luxurious holiday atmosphere where they get papered, eat better food, exercise more, sleep better and relax a lot – what would we expect after one week?

Yes, precisely! We would expect that almost anything measurable has changed for the better!

In fact, this result is so predictable that it is hardly worth documenting. Crucially, the outcome has very little to do with wellness, holism, or alternative medicine.

My conclusion: wellness not only attracts charlatans, entrepreneurs and windbags, it also is firmly steeped in pseudoscience.

 

 

 

One of the questions I hear frequently is ‘HOW CAN I BE SURE THIS STUDY IS SOUND’? Even though I have spent much of my professional life on this issue, I am invariably struggling to provide an answer. Firstly, because a comprehensive reply must inevitably have the size of a book, perhaps even several books. And secondly, to most lay people, the reply would be intensely boring, I am afraid.

Yet many readers of this blog evidently search for some guidance – so, let me try to provide a few indicators – indicators, not more!!! – as to what might signify a good and a poor clinical trial (other types of research would need different criteria).

INDICATORS SUGGESTIVE OF A GOOD CLINICAL TRIAL

  • Author from a respected institution.
  • Article published in a respected journal.
  • A clear research question.
  • Full description of the methods used such that an independent researcher could repeat the study.
  • Randomisation of study participants into experimental and control groups.
  • Use of a placebo in the control group where possible.
  • Blinding of patients.
  • Blinding of investigators, including clinicians administering the treatments.
  • Clear definition of a primary outcome measure.
  • Sufficiently large sample size demonstrated with a power calculation.
  • Adequate statistical analyses.
  • Clear presentation of the data such that an independent assessor can check them.
  • Understandable write-up of the entire study.
  • A discussion that puts the study into the context of all the important previous work in this area.
  • Self-critical analysis of the study design, conduct and interpretation of the results.
  • Cautious conclusion which are strictly based on the data presented.
  • Full disclosure of ethics approval and informed consent,
  • Full disclosure of funding sources.
  • Full disclosure of conflicts of interest.
  • List of references is up-to-date and includes also studies that contradict the authors’ findings.

I told you this would be boring! Not only that, but each bullet point is far too short to make real sense, and any full explanation would be even more boring to a lay person, I am sure.

What might be a little more fun is to list features of a clinical trial that might signify a poor study. So, let’s try that.

WARNIG SIGNALS INDICATING A POOR CLINICAL TRIAL

  • published in one of the many dodgy CAM journals (or in a book, blog or similar),
  • single author,
  • authors are known to be proponents of the treatment tested,
  • author has previously published only positive studies of the therapy in question (or member of my ‘ALT MED HALL OF FAME’),
  • lack of plausible rationale for the study,
  • lack of plausible rationale for the therapy that is being tested,
  • stated aim of the study is ‘to demonstrate the effectiveness of…’ (clinical trials are for testing, not demonstrating effectiveness or efficacy),
  • stated aim ‘to establish the effectiveness AND SAFETY of…’ (even large trials are usually far too small for establishing the safety of an intervention),
  • text full of mistakes, e. g. spelling, grammar, etc.
  • sample size is tiny,
  • pilot study reporting anything other than the feasibility of a definitive trial,
  • methods not described in sufficient detail,
  • mismatch between aim, method, and conclusions of the study,
  • results presented only as a graph (rather than figures which others can re-calculate),
  • statistical approach inadequate or not sufficiently detailed,
  • discussion without critical input,
  • lack of disclosures of ethics, funding or conflicts of interest,
  • conclusions which are not based on the results.

The problem here (as above) is that one would need to write at least an entire chapter on each point to render it comprehensible. Without further detailed explanations, the issues raised remain rather abstract or nebulous. Another problem is that both of the above lists are, of course, far from complete; they are merely an expression of my own experience in assessing clinical trials.

Despite these caveats, I hope that those readers who are not complete novices to the critical evaluation of clinical trials might be able to use my ‘warning signals’ as a form of check list that helps them to tell the chaff from the wheat.

Nobody really likes to be criticised; it can be painful. Painful but often necessary! Criticism produces progress. Criticism is therefore important. So, let’s think about criticism for a moment.

Obviously I am not talking of criticism such as ‘YOU ARE AN IDIOT’. In fact, that’s not criticism at all; it’s an insult. I am also not thinking about criticism like ‘YOUR ARGUMENT IS IDIOTIC’. I prefer to focus on criticism that is constructive, well-argued and based on evidence.

In healthcare, there is plenty of that type of criticism – luckily, I hasten to add. Its aim is to improve healthcare of the future. We need criticism to make progress. Without it, things come to a standstill or regress. This is why all the major medical journals are full of it, and many medical conferences are entirely or partly focussed on such aspects . For instance, frequently-cited papers in the BMJ, Lancet, NEJM, etc. point out that:

  • much of the current medical research is unreliable,
  • many therapies in current use have severe adverse effects,
  • patients frequently do not get the optimal treatment in a timely fashion,
  • modern medicine is too often inhumane.

The hope is that by disclosing these and many other deficits, appropriate actions can be found and taken to improve the situation and make progress. This process is hardly ever straight forward. All too often it is slow, inadequate and impeded by logistic and other obstacles. Therefore, it is crucial that constructive criticism continues to be voiced. Many clinicians, researchers and other experts have dedicated their lives to this very task.

Now, let’s look at the realm of alternative medicine.

There is certainly not less to criticise here than in conventional medicine. So, are all the journals of alternative medicine full of criticism of alternative medicine? Are there regular conferences focussed on criticism? Are alternative practitioners keen to hear about the weaknesses of their beliefs, practice, etc.?

The short answer is, no!

Yet, advocates of alternative medicine are, of course, not adverse to voicing criticism. In fact, they criticise almost non stop – at least this is the impression I get from reading their comments on this blog and from continually discussing with them since 1993.

But there is a fundamental difference: they criticise (often rightly) conventional medicine, and they criticise those (sometimes rightly) who criticise alternative medicine. When it comes to criticising their own practices, however, there is an almost deafening silence.

In my view, these differences between alternative and conventional medicine are far from trivial. In conventional medicine:

  • There is a long tradition of criticism.
  • Criticism is published and discussed prominently.
  • Criticism is usually well-accepted.
  • Criticism is often taken on board and appropriate action follows.
  • Criticism thus can and often does lead to progress.

By contrast, in alternative medicine almost nothing of the above ever happens. Criticism is directed almost exclusively towards those who are outside the realm. Criticism from the inside is as good as non-existent.

The consequences of this situation are easy to see for everyone, and they can be dramatic:

  • The journals of alternative medicine publish nothing that could be perceived to be negative for the practice of alternative medicine.
  • Self-critical thinking has no tradition and has remained an almost alien concept.
  • The very few people from the ‘inside’ who dare to criticise alternative practices are ousted and/or declared to be incompetent or worse.
  • No action is taken to initiate change.
  • The assumptions of alternative medicine remain unaltered for centuries.
  • Progress is all but absent.

It is time that the world of alternative medicine finally understands that constructive criticism is a necessary step towards progress!

Hyperthyroidism is, so I am told, a frequent veterinary problem, particularly in elderly cats. Homeopathic treatment is sometimes used to treat this condition. One article even provided encouraging details based on 4 case-reports. All 4 cats showed resolution of clinical signs; three attained normal thyroid hormone levels.  The authors concluded that homeopathic and complementary therapies avoid the potential side effects of methimazole and surgical thyroidectomy, they are less costly than radioactive iodine treatment, and they provide an option for clients who decline conventional therapies.

Yes, you guessed correctly: such a paper can only be published in the journal ‘HOMEOPATHY‘, respectable journals would not allow such conclusions based on 4 case-reports. They don’t permit inferences as to cause and effect. We have no idea what would have happened to these animals without homeopathy – perhaps they would have fared even better!

What we need is a proper controlled trial. The good news is that such a study has just been published. This double-blinded, placebo-controlled randomised trial was aimed at testing the efficacy of individualised homeopathy in the treatment of feline hyperthyroidism. Cats were randomised into two treatment arms. Either a placebo or a homeopathic treatment was given to each cat blindly.

After 21 days, the T4 levels, weight (Wt) and heart rate (HR) were compared with pre-treatment values. There were no statistically significant differences in the changes seen between the two treatment arms following placebo or homeopathic treatment, or between the means of each parameter for either treatment arm before and after placebo or homeopathic treatment. In a second phase of the study, patients in both treatment arms were given methimazole treatment for 21 days and T4, Wt and HR determined again. Subsequently, statistically significant reductions were noted in T4 (P<0.0001) and HR (P=0.02), and a statistically significant increase was observed in Wt (P=0.004).

The authors concluded that the results of this study failed to provide any evidence of the efficacy of homeopathic treatment of feline hyperthyroidism.

So, homeopathy does not work – not in humans nor in animals. This statement, backed by solid facts, proves all those wrong who cannot resist uttering the notion that HOMEOPATHY CANNOT BE A PLACEBO BECAUSE IT WORKS IN ANIMALS.

It doesn’t!

And we have seen the evidence for the correctness of this fact so often (for instance here, here, here and here) that I feel embarrassed to say it again: highly diluted homeopathic remedies are placebos. As soon as we adequately control for placebo and other non-specific effects in properly controlled studies, the alleged effects, reported in anecdotes and other uncontrolled studies, simply disappear.

 

We have repeatedly discussed the risks of chiropractic spinal manipulation (see, for instance here, here and here). Some chiropractors seem to believe that using a hand-held manipulator, called ‘activator’, better controls the forces used on the spine and therefore is safer. This recent paper raises doubts on this hypothesis.

A neurosurgeon from Florida published the case-report of a 75-year-old active woman who presented to a local hospital emergency room with a 3-day history of the acute onset of severe left temporal headache, initially self-treated with non-steroidals, to which they were resistant. Additional complaints included some vague right eye blurring of vision and a mild speech disturbance. Her primary-care physician had ordered an outpatient MRI, which was interpreted as showing a small sub-acute left posterior temporal lobe haemorrhage. He then referred her to the emergency room where she was categorized as a “stroke alert” and evaluated according to the hospital “stroke-alert” protocol.

There was no prior history of migraine, but some mild treated hypertension. The patient subsequently gave a history of chronic neck and back pain, but no headache, for which she had intermittently received chiropractic adjustments. Her current problem started after an activator treatment to the base of the left side at the junction of the skull with the upper cervical spine. She became concerned enough a few days later, because of the persistence of unremitting headache, to contact her primary-care physician. The patient was not taking any anticoagulants or antiplatelet agents and had a relatively unremarkable past medical and surgical history. Although she did not have a formal visual field examination or an ophthalmology consultation, she was found to have an incomplete right homonymous hemi-anopsia on clinical exam by the neurologist.

Based on MRI characteristics, the haemorrhage was determined to be primarily subarachnoid and displacing but not involving any brain parenchyma, and without any extra-axial component. After a 4-day hospitalization for evaluation and observation, the patient was discharged, neurologically improved in terms of visual and speech symptoms as well as headache complaints, to outpatient follow-up. She has remained well with resolution of imaging abnormalities and no reoccurrence of symptoms.

The authors explain how difficult it is to prove specific causation in such cases. It is frequently inferred by epidemiological reasoning or evidence. While there are other potential causes of the haemorrhage that occurred in this case, none is as or more likely than the activator stimulus. In support of the activator as the cause of the haemorrhage, the symptoms began almost immediately after the activator treatment (a temporal relationship), the area to which the activator was applied is almost directly superficial to the area of haemorrhage (a spatial relationship), the anatomic location of this haemorrhage is statistically unusual for any underlying and/or preexisting conditions, including stroke. The MRI confirmed that there was no infarction underlying the area of haemorrhage. The MRA disclosed no dissections or vascular lesions present. The only mechanisms left are trauma or cryptic vascular lesion that ruptured, obliterated itself, and occurred coincident to the activator stimulus. Although Activator stimulus is not high energy, it nonetheless was targeted to the cervico-occipital junction, an area where neural tissue is among the most vulnerable and least protected and closest to the skin (as opposed to the lower cervical or any of the thoracic or lumbar spine). There are many articles that make reference to minor or trivial head injury as a likely cause of intracranial haemorrhage.

The author concluded that he was unable to find a single documented case in which a brain hemorrhage in any location was reported from activator treatment. As such, this case appears to represent the first well-documented and reported brain hemorrhage plausibly a consequence of activator treatment. In the absence of any relevant information in the chiropractic or medical literature regarding cerebral hemorrhage as a consequence of activator treatment, this case should be instructive to the clinician who is faced with a diagnostic dilemma and should not forget to inquire about activator treatment as a potential cause of this complication. Our case had a benign course, but we do not rule out a more serious or potentially dangerous clinical course or adverse outcome. This is of heightened concern in the elderly and/or those with treatment-induced coagulopathy or platelet inhibition.

In light of all of the difficulties inherent in linking chiropractic treatments, including activator treatments, with serious neurological events, it is very possible that intracranial hemorrhage is far more frequent than reported. Several articles comment on the likelihood that complications of this type are almost certainly underreported. Most of the incidents mentioned in case series or surveys had never been previously reported. Neurologists, neurosurgeons, and chiropractors should be more vigilant both in the application and evaluation of these methods in all patients who report new neurologic-type symptoms following a manipulation (including an activator application) to the occiput or the cranio-cervical junction.

I think that case-report speaks for itself.

Chiropractors will, of course, argue (yet again) that:

  • conventional treatments cause much more harm,
  • spinal manipulation is highly effective,
  • such complications are extreme rarities,
  • the risk/benefit profile of spinal manipulation is positive,
  • some studies have failed to show a risk of spinal manipulation,
  • case-reports cannot establish causality.

We have rehearsed these arguments ad nauseam on this blog. The bottom line is well-expressed in the above conclusions: it is very possible that intracranial hemorrhage is far more frequent than reported. And that obviously applies to all other types of complications after chiropractic treatments.

Acupuncture is often recommended as a treatment for shoulder pain, but its effectiveness is far from proven. A new study has just been published; but does it change this uncertainty?

A total of 227 patients with subacromial pain syndrome were recruited to this RCT. The patients were allocated to three groups who received either A) group exercise, B) group exercise plus acupuncture or C) group exercise plus electro-acupuncture. The primary outcome measure was the Oxford Shoulder Score. Follow-up was post treatment, and at 6 and 12 months. Data were analysed on intention-to-treat principles with imputation of missing values.

Treatment groups were similar at baseline. All treatment groups demonstrated improvements over time. Between-group estimates were, however, small and non-significant.

The authors concluded that neither acupuncture nor electro-acupuncture were found to be more beneficial than exercise alone in the treatment of subacromial pain syndrome. 

Well, that was to be expected!… I hear the rationalists amongst us exclaim.

Actually, I am not so sure.

One could easily have expected that the acupuncture groups (B and C) show a significant advantage over group A.

Why?

Because acupuncture is a ‘theatrical placebo’, a ritual that impresses patients and thus impacts on results, particularly on subjective outcomes like pain. If the results had shown a benefit for acupuncture + exercise (groups B and C) versus exercise alone (group A), what would we have made of it? Acupuncture fans would surely have claimed that it is evidence confirming acupuncture’s effectiveness. Sceptics, on the other hand, would have rightly insisted that it demonstrates nothing of the sort – it merely confirms that placebo effects can affect clinical outcomes such as pain.

As it turned out, however, this trial results happened to indicate that these placebo-effects can be so small that they fail to reach the level of statistical significance.

I think there is one noteworthy message here: RCTs with such a design (no adequate control for placebo effects) can easily generate false-positive results (in this case, this did not happen, but it was nevertheless a possible outcome). Such studies are popular but utterly useless: they don’t advance our knowledge one single iota. If that is so, we should not waste our resources on them because, in the final analysis, this is not ethical. In other words, we must stop funding research that has little or no chance of advancing our knowledge.

First she promoted vaginal steam baths and now Gwyneth Paltrow claims that putting a ball of jade (which you can order from her online-business, if you happen to have the cash) in their vaginas is good for women.

Yes, I kid you not; this is what she states on her website:

The strictly guarded secret of Chinese royalty in antiquity—queens and concubines used them to stay in shape for emperors—jade eggs harness the power of energy work, crystal healing, and a Kegel-like physical practice. Fans say regular use increases chi, orgasms, vaginal muscle tone, hormonal balance, and feminine energy in general. Shiva Rose has been practicing with them for about seven years, and raves about the results; we tried them, too, and were so convinced we put them into the goop shop. Jade eggs’ power to cleanse and clear make them ideal for detox…

But if you think that Gwyneth is somehow fixated on her feminine parts, you are probably mistaken. She is much more versatile than that and seems to employ her vagina merely for drumming up publicity for her business. If you browse her site, you find no end of baffling, vagina-unrelated wonders and purchasable products from the world of alternative medicine.

Here are just two further examples.

Flower  remedies

A flower essence is a bioenergetic preparation. Through the use of sunlight and water, we are able to capture the energy of a flower and use it for healing purposes: A freshly harvested flower is placed on the surface of water for a specific length of time and exposed to sunlight, resulting in the vibrational imprint of the flower in water. The flower essence is then used as an energetic remedy, with each flower having its own range of unique therapeutic benefits.

Vibrational imprint?

Unique therapeutic benefit?

Pull the other one! The truth about (Bach) flower remedies is much simpler: they are expensive placebos.

A method for getting rid of the parasites we allegedly all suffer from

…an eight-day, mono-diet goat-milk cleanse—accompanied by a specific vermifuge made of anti-parasitic herbs—is the most successful treatment. Parasites primarily live in the mucus lining of the gut system, where they feed on nutrients before they enter the body. Think of the goat milk as bait—parasites come out of the gut lining to drink the milk, which they love, but they also consume the vermifuge, which will eventually eradicate them. On top of being highly effective, this method is a much more gentle medicine than bombarding them—and your body—with a harsh drug.

Are they for real?

This is pure and potentially very dangerous, unethical nonsense!

Oh sorry – I forgot: we now must call it differently now: we are obviously dealing with Gwyeneth’s ‘alternative facts’.

The Committees of Advertising Practice (CAP) write and maintain the UK Advertising Codes, which are administered by the Advertising Standards Authority. On their website, the CAP recently published an updated advertising code for naturopathy. As we have regularly discussed the fact that the public is being frequently misled in this area, I consider the code important in the context of this blog. I therefore take the liberty of repeating it here – not least in the hope that this helps preventing misinformation in the future [the numbers in square brackets refer to me footnotes below].

START OF QUOTE

What is Naturopathy?

Naturopathy is a holistic [1] approach to healthcare that uses a combination of one or more different disciplines (for example herbal medicine or hydrotherapy) and a healthy lifestyle [2] in order to gain and maintain a healthy body [3].

What claims are likely to be acceptable?

The promotion of a healthy [4] lifestyle is likely to acceptable as are claims that go no further than those commonly accepted for healthy [4] eating, sleeping well, taking exercise and the like.

What claims are likely to be problematic?

The ASA and CAP have not yet been provided with evidence which demonstrates that Naturopathy can be used to treat medical conditions (Rule 12.1).  Therefore, any claims that go beyond accepted claims for a healthy [4] lifestyle are likely to be problematic [5] unless they are supported by a robust body of evidence.  In 2013, the ASA ruled against claims on a marketer’s website which said that Naturopathy could be used to treat acute and chronic illness and disease because the marketer had not provided any evidence in support of their claims (CNM The College of Naturopathic Medicine Ltd, 13 March 2013).

What about serious medical conditions?

Claims to offer treatment on conditions for which medical supervision should be sought [6] are likely to be considered to discourage essential treatment unless that treatment is carried out under the supervision of a suitably qualified health professional (Rule 12.2).

END OF QUOTE

Naturopathy has been the subject of my posts before – see for instance here, here, here, here and here. Naturopathy can be dangerous to the point where it can kill the patient – see for instance here and here. Therefore it is important that advertising gets regulated. To make it very clear: the above statement by the CAP is, in my view, a step in the right direction, and I encourage alternative practitioners to look up the equivalent CAP documents for their specific therapy.

Having said that, I still feel the need to make a few comments:

  1. It is misleading to call naturopathy ‘holistic’. This is often factually incorrect and also gives the impression that conventional medicine is not holistic – see also here.
  2. Are we sure that all lifestyles promoted by naturopaths are, in fact, healthy?
  3. Maintaining a healthy body is naturopathy speak for DISEASE PREVENTION. Who decides what is effective prevention? On what evidence? How come many naturopaths are against the most effective means of prevention of all times – vaccination?
  4. Who decides what is ‘healthy’? On what evidence?
  5. Why ‘problematic’? Are they not wrong or bogus or false or fraudulent or criminal?
  6. Are there conditions for which medical supervision should not be sought? Which are they?

Whenever a level-headed person discloses that a specific alternative therapy is not based on good evidence, you can bet your last shirt that a proponent of the said treatment responds by claiming that conventional medicine is not much better.

There are several variations to this theme. Today I want to focus on just one of them, namely the counter-claim that, only a short while ago, conventional medicine was not much better than the said alternative therapy (the implication is that it must be unfair to demand evidence from alternative medicine, while accepting a similar state of affairs in conventional medicine). The argument has recently been formulated by one commentator on this blog as follows:

“Trepanation, leeches for UTI’s, and bloodletting are all historical treatments of medical doctors…It’s hypocritical… to impute mainstream chiropractice to the profession’s beginnings and yet not admit that medicine’s founding and evolution was inbued with consistently scientific rigor.”

Sadly, some people seem to be convinced by such words, and this is why they are being repeated ad nauseam by interested parties. Yet the argument is fallacious for a range of reasons.

  • Firstly, it is based on the classical ‘tu quoque’ fallacy (appeal to hypocrisy).
  • Secondly – unless we happen to be historians – it is not the healthcare of the past that is relevant to our discussions. The question cannot be what this or that group of clinicians used to do; the question is HOW DO THEY TREAT THEIR PATIENTS TODAY?

As soon as we focus on this issue, it is impossible to deny that conventional medicine has made lots of progress and moved light years away from treatments such as trepanation, leeches, bloodletting and many others.

Why?

Why did we make such huge progress?

Because research showed that many of the traditional treatments were ineffective, unsafe and/or implausible (thus demonstrating that hundreds of years of experience – which alternative therapists rate so very highly – is of more than dubious value), and because we consequently developed and tested new therapies and subsequently used those treatments that passed these tests and were proven to do more good than harm.

By contrast, in the last decades, centuries and millennia, homeopathy, chiropractic, acupuncture, paranormal healing etc. did make no (or very little) progress. So much so that Hahnemann, for instance, would pass any exam for  homeopathy today. (If you disagree with this statement, please post a list of those treatments that have been given up by alternative therapists in the last 100 years or so.) Come to think of it, it is a hallmark of alternative medicine that it does not progress in the way conventional medicine does. It is almost completely static, a fact, that renders it akin to a dogma or a cult.

But why? Why is there no real progress in alternative medicine?

Don’t tell me that there is no research, research funding, etc. There are now hundreds of studies of homeopathy or chiropractic, thousands of acupuncture, and dozens of paranormal healing, for instance. The trouble is not the paucity of such research but its findings! The totality of the evidence in each of these areas fails to show that the therapy in question is efficacious.

And there we have, I think, another hallmark of alternative medicine: it is an area where research is only acted upon, if its findings are in line with the preconceptions and aspirations of its proponents.

I find this interesting!

It means, amongst other things, that research into alternative medicine tends not to be used for finding the truth or establishing new knowledge; it is mainly employed for the promotion of the therapy in question, regardless of what the truth about it might be (this would disqualify this exercise from being research and qualify it as PSEUDO-RESEARCH). If the research findings are such that they cannot be used for promotion, they are simply ignored or defamed as inadequate.

This new RCT was embargoed until today; so, I had to wait until I was able to publish my comments. Here are the essentials of the study:

The Swedish investigators compared the effect of two types of acupuncture versus no acupuncture in infants with colic in public child health centres (CHCs). The study was designed as a multicentre, randomised controlled, single-blind, three-armed trial (ACU-COL) comparing two styles of acupuncture with no acupuncture, as an adjunct to standard care. Among 426 infants whose parents sought help for colic and registered their child’s fussing/crying in a diary, 157 fulfilled the criteria for colic and 147 started the intervention.

Parallel to usual care, study participants visited the study CHC twice a week for 2 weeks. Thus, all infants received usual care plus 4 extra visits to a CHC, during which parents met a nurse for 20–30 min and were able to discuss their infant’s symptoms. Together these were considered to represent gold standard care. The nurse listened, and gave evidence-based advice and calming reassurance. Breastfeeding mothers were encouraged to continue breastfeeding. At each visit, the study nurse carried the infant to a separate treatment room where they were left alone with the acupuncturist for 5 min.

The acupuncturist treated the baby according to group allocation and recorded the treatment procedures and any adverse events. Disposable stainless steel 0.20×13 mm Vinco needles (Helio, Jiangsu Province, China) were used. Infants allocated to group A received standardised MA at LI4. One needle was inserted to a depth of approximately 3 mm unilaterally for 2–5 s and then withdrawn without stimulation. Infants allocated to group B received semi-standardised individualised acupuncture, mimicking clinical TCM practice. Following a manual, the acupuncturists were able to choose one point, or any combination of Sifeng, LI4 and ST36, depending on the infant’s symptoms, as reported in the diary. A maximum of five insertions were allowed per treatment. Needling at Sifeng consisted of 4 insertions, each to a depth of approximately 1 mm for 1 s. At LI4 and ST36, needles were inserted to a depth of approximately 3 mm, uni- or bilaterally. Needles could be retained for 30 seconds. De qi was not sought, therefore stimulation was similarly minimal in groups A and B. Infants in group C spent 5 min alone with the acupuncturist without receiving acupuncture.

The effect of the two types of acupuncture was similar and both were superior to gold standard care alone. Relative to baseline, there was a greater relative reduction in time spent crying and colicky crying by the second intervention week (p=0.050) and follow-up period (p=0.031), respectively, in infants receiving either type of acupuncture. More infants receiving acupuncture cried <3 hours/day, and thereby no longer fulfilled criteria for colic, in the first (p=0.040) and second (p=0.006) intervention weeks. No serious adverse events were reported.

The authors concluded that acupuncture appears to reduce crying in infants with colic safely.

Notice that the investigators are cautious and state in the abstract that “acupuncture appears to reduce crying…” Their conclusions from the actual article are, however, quite different; here they state the following:

Among those initially experiencing excessive infant crying, the majority of parents reported normal values once the infant’s crying had been evaluated in a diary and a diet free of cow’s milk had been introduced. Therefore, objective measurement of crying and exclusion of cow’s milk protein are recommended as first steps, to avoid unnecessary treatment. For those infants that continue to cry >3 hours/day, acupuncture may be an effective treatment option. The two styles of MA tested in ACU-COL had similar effects; both reduced crying in infants with colic and had no serious side effects. However, there is a need for further research to find the optimal needling locations, stimulation and treatment intervals.

Such phraseology is much more assertive and seems to assume acupuncture caused specific therapeutic effects. Yet, I think, this assumption is not warranted.

In fact, I believe, the study shows almost the opposite of what the authors conclude. Both minimal and TCM acupuncture seemed to reduce the symptoms of colic compared to no acupuncture at all. I think, this confirms previous research showing that acupuncture is a ‘theatrical placebo’. The study was designed without an adequate placebo group. It would have been easy to use some form of sham acupuncture in the control group. Why did the authors not do that? Heaven knows, but one might speculate that they were aiming for a positive result – and what better way to ensure it than with a ‘no treatment’ control group?

There are, of course, numerous other flaws. For instance, Prof David Colquhoun FRS, Professor of Pharmacology at University College London, criticised the study because of its lousy statistics:

START OF QUOTE

“It is truly astonishing that, in the 21st century, the BMJ still publishes a journal devoted to a form of pre-scientific medicine which after more than 3000 trials has still not been able to produce convincing evidence of efficacy1. Like most forms of alternative medicine, acupuncture has been advocated for a vast range of problems, and there is little evidence that it works for any of them. Colic has not been prominent in these claims. What parent would think that sticking needles into their baby would stop it crying? The idea sounds bizarre. It is. This paper certainly doesn’t show that it works.

“The statistical analysis in the paper is incompetent. This should have been detected by the referees, but wasn’t.  For a start, the opening statement, ‘A two-sided P value ≤0.05 was considered statistically significant’ is simply unacceptable in the light of all recent work about reproducibility.  Still worse, Table 1 uses the description ‘statistical tendency towards significance (p=0.051–0.1)’.

“Worst of all, Table 1 reports 24 different P values, of which three are (just) below 0.05. Yet no correction has been used for multiple comparisons. This is very bad practice. It’s highly unlikely that, if the proper correction had been done, any of the results would have given a type 1 error rate below 5%.

“Even were it not for this, most of the ‘significant’ P values are marginal (only slightly less than 0.05).  It is now well known that the type 1 error rate gives an optimistic view. What matters is the false positive rate – the chance that a ‘significant’ result is a false positive.  A p-value close to 0.05 implies that there is at least a 30% chance that they are false positives.  If one thought, a priori, that the chance of colic being cured by sticking needles into a baby was less than 50%, the false positive rate could easily be greater than 80%2.  It is now recognised that this misinterpretation of p-values is a major contributor to the crisis of reproducibility.

“Other problems concern the power calculation.  A priori calculations of power are well-known to be overoptimistic, because small trials usually overestimate the effect size.  In this case the initial estimated sample size was not attained, and a rather mysterious recalculation of power was used.

“Another small problem: the discussion points out that ‘the majority of infants in this cohort did not have colic’.

“The nature of the control group is not very clear. An appropriate control might have been to cuddle the baby – this was used in a study in which another implausible treatment, chiropractic, was shown not to work.  This appears not to have been done.

“Lastly, p-values are reported in the text without mention of effect sizes. This is contrary to all statistical advice.

“In conclusion, the design of the trial is reasonable (apart from the control group) but the statistical analysis is appalling.  It’s very likely that there aren’t any real effects of acupuncture at all. This paper serves more to muddy the waters than to add useful information. It’s a model for the sort of mistakes that have led to the crisis in reproducibility.  The BMJ should not be publishing this sort of stuff, and the referees seem to have no understanding of statistics.”

END OF QUOTE

Despite these rather obvious – some would say fatal – flaws, the editor of ACUPUNCTURE IN MEDICINE (AIM) thought this trial to be so impressively rigorous that he issued a press-release about it. This, I think, is particularly telling, perhaps even humorous: it shows what kind of a journal AIM is, and also provides an insight into the state of acupuncture research in general.

The long and short of it is that conclusions about specific therapeutic effects of acupuncture are not permissible. We know that colicky babies respond even to minimal attention, and this trial confirms that even a little additional TLC in the form of acupuncture will generate an effect. The observed outcome is most likely unrelated to acupuncture.

Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted but you must tick the box: “Save my name, email, and website in this browser for the next time I comment.”

The most recent comments from all posts can be seen here.

Archives
Categories