MD, PhD, FMedSci, FRSB, FRCP, FRCPEd

study design

1 2 3 9

THE CONVERSATION recently carried an article shamelessly promoting osteopathy. It seems to originate from the University of Swansea, UK, and is full of bizarre notions. Here is an excerpt:

To find out more about how osteopathy could potentially affect mental health, at our university health and well-being academy, we have recently conducted one of the first studies on the psychological impact of OMT – with positive results.

For the last five years, therapists at the academy have been using OMT to treat members of the public who suffer from a variety of musculoskeletal disorders which have led to chronic pain. To find out more about the mental health impacts of the treatment, we looked at three points in time – before OMT treatment, after the first week of treatment, and after the second week of treatment – and asked patients how they felt using mental health questionnaires.

This data has shown that OMT is effective for reducing anxiety and psychological distress, as well as improving patient self-care. But it may not be suitable for all mental illnesses associated with chronic pain. For instance, we found that OMT was less effective for depression and fear avoidance.

All is not lost, though. Our results also suggested that the positive psychological effects of OMT could be further optimised by combining it with therapy approaches like acceptance and commitment therapy (ACT). Some research indicates that psychological problems such as anxiety and depression are associated with inflexibility, and lead to experiential avoidance. ACT has a positive effect at reducing experiential avoidance, so may be useful with reducing the fear avoidance and depression (which OMT did not significantly reduce).

Other researchers have also suggested that this combined approach may be useful for some subgroups receiving OMT where they may accept this treatment. And, further backing this idea up, there has already been at least one pilot clinical trial and a feasibility study which have used ACT and OMT with some success.

Looking to build on our positive results, we have now begun to develop our ACT treatment in the academy, to be combined with the osteopathic therapy already on offer. Though there will be a different range of options, one of these ACT therapies is psychoeducational in nature. It does not require an active therapist to work with the patient, and can be delivered through internet instruction videos and homework exercises, for example.

Looking to the future, this kind of low cost, broad healthcare could not only save the health service money if rolled out nationwide but would also mean that patients only have to undergo one treatment.

END OF QUOTE

So, they recruited a few patients who had come to receive osteopathic treatments (a self-selected population full of expectation and in favour of osteopathy), let them fill a few questionnaires and found some positive changes. From that, they conclude that OMT (osteopathic manipulative therapy) is effective. Not only that, they advocate that OMT is rolled out nationwide to save NHS funds.

Vis a vis so much nonsense, I am (almost) speechless!

As this comes not from some commercial enterprise but from a UK university, the nonsense is intolerable, I find.

Do I even need to point out what is wrong with it?

Not really, it’s too obvious.

But, just in case some readers struggle to find the fatal flaws of this ‘study’, let me mention just the most obvious one. There was no control group! That means the observed outcome could be due to many factors that are totally unrelated to OMT – such as placebo-effect, regression towards the mean, natural history of the condition, concomitant treatments, etc. In turn, this also means that the nationwide rolling out of their approach would most likely be a costly mistake.

The general adoption of OMT would of course please osteopaths a lot; it could even reduce anxiety – but only that of the osteopaths and their bank-managers, I am afraid.

One thing one cannot say about George Vithoulkas, the ueber-guru of homeopathy, is that he is not as good as his word. Last year, he announced that he would focus on publishing case reports that would convince us all that homeopathy is effective:

…the only evidence that homeopathy can present to the scientific world at this moment are these thousands of cured cases. It is a waste of time, money, and energy to attempt to demonstrate the effectiveness of homeopathy through double blind trials.

… the international “scientific” community, which has neither direct perception nor personal experience of the beneficial effects of homeopathy, is forced to repeat the same old mantra: “Where is the evidence? Show us the evidence!” … the successes of homeopathy have remained hidden in the offices of hardworking homeopaths – and thus go largely ignored by the world’s medical authorities, governments, and the whole international scientific community…

… simple questions that are usually asked by the “gnorant”, for example, “Can homeopathy cure cancer, multiple sclerosis, ulcerative colitis, etc.?” are invalid and cannot elicit a direct answer because the reality is that many such cases can be ameliorated significantly, and a number can be cured…

And focussing on successful cases is just what the great Vithoulkas now does.

Together with homeopaths from the  Centre for Classical Homeopathy, Vijayanagar, Bangalore, India, Vithoulkas has recently published a retrospective case series of 10 Indian patients who were diagnosed with dengue fever and treated exclusively with homeopathic remedies at Bangalore, India. This case series demonstrates with evidence of laboratory reports that even when the platelets dropped considerably there was good result without resorting to any other means.

The homeopaths concluded that a need for further, larger studies is indicated by this evidence, to precisely define the role of homeopathy in treating dengue fever. This study also emphasises the importance of individualised treatment during an epidemic for favourable results with homeopathy.

Bravo!

Keeping one’s promise must be a good thing.

But how meaningful are these 10 cases?

Dengue is a viral infection which, in the vast majority of cases, takes a benign course. After about two weeks, patients tend to be back to normal, even if they receive no treatment at all. In other words, the above-quoted case series is an exact description of the natural history of the condition. To put it even more bluntly: if these patients would have been treated with kind attention and good general care, the outcome would not have been one iota different.

To me, this means that “to precisely define the role of homeopathy in treating dengue fever” would be a waste of resources. It’s role is already clear: there is no role of homeopathy in the treatment of this (or any other) condition.

Sorry George.

Yesterday, it was announced that homeopaths can easily and quickly earn a sizable amount of money.

The announcement was made during the German sceptics conference ‘Skepkon‘ in Koeln. As I could not be present, I obtained the photo via Twitter.

So, if you are a homeopath or a fan of homeopathy, all you have to do – as the above slide says – is to reproducibly identify homeopathic remedies in high potency. The procedure for obtaining the money has to follow three pre-defined steps:

  1. Identification of three homeopathic preparations in high potency according to a proscribed protocol.
  2. Documentation of a method enabling a third party to identify the remedies.
  3. Verification of the experiment by repeating it.

Anyone interested must adhere to the full instructions published by the German sceptics GWUP:

1. Review of test protocol

Together with a representative of GWUP, the applicants review and agree on this protocol prior to the start of the test. Minor changes may be applied if justified, provided they are mutually agreed to in advance and do not affect the validity of the test, especially the blinding and randomization of the samples. In any case we want to avoid that the results get compromised or their credibility impeached by modifications of the protocol while the test is already under way. After mutual confirmation, the test protocol is binding for the whole duration of the test and its evaluation.

2. Selection of drugs

The applicant proposes which three drugs should be used in the trial. This gives them the opportunity to select substances that they think they could distinguish particularly well as homeopathic remedies. The potency may be selected freely as well, whereby the following conditions must be observed:

– all drugs must be available as sugar globules of the same grade (“Globuli” in German);
– the same potency, namely D- or C-potency above D24 / C12, is used for all three drugs;
– all drugs can be procured from the same producer.

3. Procurement of samples

The samples will be purchased by GWUP and shipped from the vendor directly to the notary who will perform the randomization. GWUP will purchase sufficient numbers of packages to complete the series of 12 samples according to the randomization list. The procurement will ensure that the samples derive from different batches of production as follows.

3.1. Common remedies

Common remedies, i.e. remedies sold in high numbers, will be procured from randomly selected pharmacies from the biggest cities in Germany (Berlin, Hamburg, Munich, Cologne, Frankfurt, Stuttgart…). Each pharmacy supplies a bottle for each of the three selected remedies and ships it directly to the notary in charge of randomization. If the applicants need a sample of known content for calibration, then this will be procured from yet another pharmacy in another German city.

3.2. Special remedies

If due to low sales it is possible that the above procedure is not sufficient to have all samples from different batches, a randomly selected pharmacy will be appointed to produce all the samples from raw materials purchased from the producer. GWUP will procure the mother tinctures, the raw sugar pills, and bottles and packages, to be shipped directly to the respective pharmacy who then will do the potentization, label the bottles and send them to the notary. If there are extra samples of known content required for calibration, then an extra set of samples will be produced. One set of samples will be kept in a sealed package for future reference.

The applicant and GWUP mutually agree on which procedure is used before the start of procurement. If more than 10 grams of globules per sample are required for the procedure used for inentification, the applicant has to indicate this in advance, and GWUP will take this into account.

4. Randomization / blinding

The randomization and blinding is done by a sworn-in public notary in Würzburg, Germany, who is selected by a random procedure. Würzburg is chosen because the first part of the task is to be evaluated at the University of Würzburg, for all participants based in Europe. For overseas applicants, the location will be mutually agreed on.

The notary receives a coding list showing how the three drugs A, B and C are to be distributed among the twelve samples. This list is compiled by the GWUP representative by throwing dice. The notary also determines which drug is assigned to which letter by throwing dice. Note that the drugs may not be present in the set in equal numbers.

The notary completely removes the original label from the bottle and replaces it with the number without opening the bottle. The randomization protocol is deposited in a sealed envelope with the notary public without a copy being made beforehand. The notary disposes of surplus packs. If special remedies are processed, one set of marked samples is sealed and forwarded to GWUP for later reference in a sealed package.

The coded bottles are sent from the notary to the applicant without individual packaging and documentation. The applicant confirms receipt of the samples.

5. Identification

The applicant identifies which of the 12 bottles contains which drug, using any method and procedure of his choice. There is no limit as to the method used for identification, and this well may be a procedure not currently recognized by modern science. However, GWUP at the start requires a short and rough outline of how the applicant wants to proceed, and GWUP reserves the right to reject applications whose sincerity for scientific work seems questionable.

The applicant is also required to specify a period of time within which they will be able to produce their results. This period may not exceed six months. If it expires without the applicant being able to show their results, the outcome will be considered negative. However, the candidate may apply for an extension in good time before the deadline, if they can provide a reasonable explanation and is not caused by the respective identification process as such.

The applicant is explicitly advised to observe ethics standards, and to procure the consent of an appropriate ethics committee if their method involves testing on humans or animals.

6. Result Pt. 1

If reasonable, the applicant may present their findings as part of the PSI-Tests held annually by GWUP at the University of Würzburg. The applicant’s result will be compared to the coding protocol from the notary. The number of bottles in which the notary’s record corresponds to the applicant’s details is determined. The result includes a description of the method used, if possible with meaningful intermediate data such as measurement protocols or symptom lists of drug provings.

The first part of the test is considered a success if the content of no more than one bottle is identified incorrectly and a description of the procedure is produced.

7. Result Pt. 2 and 3: Replication and Verification

Replication of the test is to ensure that a successful first result was not caused by chance alone. In addition, the procedure explained by the applicant is to be verified in a way depending on its nature. The objective is to verify that the identification was indeed performed by using this very method, and that the description is complete and suitable for a third party to achieve the same outcome.

For replication, steps 2 to 5 will be repeated. Applicants may select to use the same drugs as before. In this case they will then procured from another manufacturer or prepared by a different pharmacy with raw material from a different supplier. Alternatively, the candidate may indicate three new drugs which then can be obtained from the original vendor.

For a successful replication the same precision as before is required, that is, that only one out of 12 bottles may be identified incorrectly.

The evaluation and presentation of these results may take place at any location, press or other media may be invited to the event as agreed to by the applicant and GWUP.


Is anyone going to take up this challenge?

Personally, I don’t hold my breath.

Many years ago (at a time when homeopaths still saw me as one of their own), I had plans to do a similar but slightly less rigorous test as part of a doctoral thesis for one of my students.

Our investigation was straight forward: we approached several of the world’s leading/most famous homeopaths and asked them to participate. Their task was to tell us which homeopathic remedy they thought was easiest to differentiate from a placebo. Subsequently we would post them several vials – I think the number was 10 – and ask them to tell us which contained the remedy of their choice (in a C30 potency), and which the placebo (the distribution was 50:50, and the authenticity of each vial was to be confirmed by a notary). The experimental method for identifying which was which was entirely left to each participating homeopath; they were even allowed to use multiple, different tests. Based on the results, we would then calculate whether their identification skills were better than pure chance.

Sadly, the trial never happened. Initially, we had a positive response from some homeopaths who were interested in participating. However, when they then saw the exact protocol, they all pulled out.

But times may have changed; perhaps today there are some homeopaths out there who actually believe in homeopathy?

Perhaps our strategy to work only with ‘the best’ homeopaths was wrong?

Perhaps there are some homeopaths who are less risk-adverse?

I sure hope that lots of enthusiastic homeopaths will take up this challenge.

GOOD LUCK! And watch this space.

We recently discussed the deplorable case of Larry Nassar and the fact that the ‘American Osteopathic Association’ stated that intravaginal manipulations are indeed an approved osteopathic treatment. At the time, I thought this was a shocking claim. So, imagine my surprise when I was alerted to a German trial of osteopathic intravaginal manipulations.

Here is the full and unaltered abstract of the study:

Introduction: 50 to 80% of pregnant women suffer from low back pain (LBP) or pelvic pain (Sabino und Grauer, 2008). There is evidence for the effectiveness of manual therapy like osteopathy, chiropractic and physiotherapy in pregnant women with LBP or pelvic pain (Liccardione et al., 2010). Anatomical, functional and neural connections support the relationship between intrapelvic dysfunctions and lumbar and pelvic pain (Kanakaris et al., 2011). Strain, pressure and stretch of visceral and parietal peritoneum, bladder, urethra, rectum and fascial tissue can result in pain and secondary in muscle spasm. Visceral mobility, especially of the uterus and rectum, can induce tension on the inferior hypogastric plexus, which may influence its function. Thus, stretching the broad ligament of the uterus and the intrapelvic fascia tissue during pregnancy can reinforce the influence of the inferior hypogastric plexus. Based on above facts an additional intravaginal treatment seems to be a considerable approach in the treatment of low back pain in pregnant women.
Objective: The purpose of this study was to compare the effect of osteopathic treatment including intravaginal techniques versus osteopathic treatment only in females with pregnancy-related low back pain.
Methods: Design: The study was performed as a randomized controlled trial. The participants were randomized by drawing lots, either into the intervention group including osteopathic and additional intravaginal treatment (IV) or a control group with osteopathic treatment only (OI). Setting: Medical practice in south of Germany.
Participants 46 patients were recruited between the 30th and 36th week of pregnancy suffering from low back pain.
Intervention Both groups received three treatments within a period of three weeks. Both groups were treated with visceral, mobilization, and myofascial techniques in the cervical, thoracic and lumbar spine, the pelvic and the abdominal region (American Osteopathic Association Guidelines, 2010). The IV group received an additional treatment with intravaginal techniques in supine position. This included myofascial techniques of the M. levator ani and the internal obturator muscles, the vaginal tissue, the pubovesical and uterosacral ligaments as well as the inferior hypogastric plexus.
Main outcome measures As primary outcome the back pain intensity was measured by Visual Analogue Scale (VAS). Secondary outcome was the disability index assessed by Oswestry-Low-Back-Pain-Disability-Index (ODI), and Pregnancy-Mobility-Index (PMI).
Results: 46 participants were randomly assigned into the intervention group (IV; n = 23; age: 29.0 ±4.8 years; height: 170.1 ±5.8 cm; weight: 64.2 ±10.3 kg; BMI: 21.9 ±2.6 kg/m2) and the control group (OI; n = 23; age: 32.0 ±3.9 years; height: 168.1 ±3.5 cm; weight: 62.3 ±7.9 kg; BMI: 22.1 ±3.2 kg/m2). Data from 42 patients were included in the final analyses (IV: n=20; OI: n=22), whereas four patients dropped out due to general pregnancy complications. Back pain intensity (VAS) changed significantly in both groups: in the intervention group (IV) from 59.8 ±14.8 to 19.6 ±8.4 (p<0.05) and in the control group (OI) from 57.4 ±11.3 to 24.7 ±12.8. The difference between groups of 7.5 (95%CI: -16.3 to 1.3) failed to demonstrate statistical significance (p=0.93). Pregnancy-Mobility-Index (PMI) changed significantly in both groups, too. IV group: from 33.4 ±8.9 to 29.6 ±6.6 (p<0.05), control group (OI): from 36.3 ±5.2 to 29.7 ±6.8. The difference between groups of 2.6 (95%CI: -5.9 to 0.6) was not statistically significant (p=0.109). Oswestry-Low-Back-Pain-Disability-Index (ODI) changed significantly in the intervention group (IV) from 15.1 ±7.8 to 9.2 ±3.6 (p<0.05) and also significantly in the control group (OI) from 13.8 ±4.9 to 9.2 ±3.0. Between-groups difference of 1.3 (95%CI: -1.5 to 4.1) was not statistically significant (p=0.357).
Conclusions: In this sample a series of osteopathic treatments showed significant effects in reducing pain and increasing the lumbar range of motion in pregnant women with low back pain. Both groups attained clinically significant improvement in functional disability, activity and quality of life. Furthermore, no benefit of additional intravaginal treatment was observed.

END OF QUOTE

My first thoughts after reading this were: how on earth did the investigators get this past an ethics committee? It cannot be ethical, in my view, to allow osteopaths (in Germany, they have no relevant training to speak of) to manipulate women intravaginally. How deluded must an osteopath be to plan and conduct such a trial? What were the patients told before giving informed consent? Surely not the truth!

My second thoughts were about the scientific validity of this study: the hypothesis which this trial claims to be testing is a far-fetched extrapolation, to put it mildly; in fact, it is not a hypothesis, it’s a very daft idea. The control-intervention is inadequate in that it cannot control for the (probably large) placebo effects of intravaginal manipulations. The observed outcomes are based on within-group comparisons and are therefore most likely unrelated to the treatments applied. The conclusion is as barmy as it gets; a proper conclusion should clearly and openly state that the results did not show any effects of the intravaginal manipulations.

In summary, this is a breathtakingly idiotic trial, and everyone involved in it (ethics committee, funding body, investigators, statistician, reviewers, journal editor) should be deeply ashamed and apologise to the poor women who were abused in a most deplorable fashion.

Amongst all the implausible treatments to be found under the umbrella of ‘alternative medicine’, Reiki might be one of the worst, i. e. least plausible and outright bizarre (see for instance here, here and here). But this has never stopped enthusiasts from playing scientists and conducting some more pseudo-science.

This new study examined the immediate symptom relief from a single reiki or massage session in a hospitalized population at a rural academic medical centre. It was designed as a retrospective analysis of prospectively collected data on demographic, clinical, process, and quality of life for hospitalized patients receiving massage therapy or reiki. Hospitalized patients requesting or referred to the healing arts team received either a massage or reiki session and completed pre- and post-therapy symptom questionnaires. Differences between pre- and post-sessions in pain, nausea, fatigue, anxiety, depression, and overall well-being were recorded using an 11-point Likert scale.

Patients reported symptom relief with both reiki and massage therapy. Reiki improved fatigue and anxiety  more than massage. Pain, nausea, depression, and well being changes were not different between reiki and massage encounters. Immediate symptom relief was similar for cancer and non-cancer patients for both reiki and massage therapy and did not vary based on age, gender, length of session, and baseline symptoms.

The authors concluded that reiki and massage clinically provide similar improvements in pain, nausea, fatigue, anxiety, depression, and overall well-being while reiki improved fatigue and anxiety more than massage therapy in a heterogeneous hospitalized patient population. Controlled trials should be considered to validate the data.

Don’t I just adore this little addendum to the conclusions, “controlled trials should be considered to validate the data” ?

The thing is, there is nothing to validate here!

The outcomes are not due to the specific effects of Reiki or massage; they are almost certainly caused by:

  • the extra attention,
  • the expectation of patients,
  • the verbal or non-verbal suggestions of the therapists,
  • the regression towards the mean,
  • the natural history of the condition,
  • the concomitant therapies administered in parallel,
  • the placebo effect,
  • social desirability.

Such pseudo-research only can only serve one purpose: to mislead (some of) us into thinking that treatments such as Reiki might work.

What journal would be so utterly devoid of critical analysis to publish such unethical nonsense?

Ahh … it’s our old friend the Journal of Alternative and Complementary Medicine

Say no more!

Since many months, I have noticed a proliferation of so-called pilot studies of alternative therapies. A pilot study (also called feasibility study) is defined as a small scale preliminary study conducted in order to evaluate feasibility, time, cost, adverse events, and improve upon the study design prior to performance of a full-scale research project. Here I submit that most of the pilot studies of alternative therapies are, in fact, bogus.

To qualify as a pilot study, an investigation needs to have an aim that is in line with the above-mentioned definition. Another obvious hallmark must be that its conclusions are in line with this aim. We do not need to conduct much research to find that even these two elementary preconditions are not fulfilled by the plethora of pilot studies that are currently being published, and that proper pilot studies of alternative medicine are very rare.

Three recent examples of dodgy pilot studies will have to suffice (but rest assured, there are many, many more).

Foot Reflexotherapy Induces Analgesia in Elderly Individuals with Low Back Pain: A Randomized, Double-Blind, Controlled Pilot Study

The aim of this study was to evaluate the effects of foot reflexotherapy on pain and postural balance in elderly individuals with low back pain. And the conclusions drawn by its authors were that this study demonstrated that foot reflexotherapy induced analgesia but did not affect postural balance in elderly individuals with low back pain.

Effect of Tai Chi Training on Dual-Tasking Performance That Involves Stepping Down among Stroke Survivors: A Pilot Study.

The aim of this study was to investigate the effect of Tai Chi training on dual-tasking performance that involved stepping down and compared it with that of conventional exercise among stroke survivors. And the conclusions read: These results suggest a beneficial effect of Tai Chi training on cognition among stroke survivors without compromising physical task performance in dual-tasking.

The Efficacy of Acupuncture on Anthropometric Measures and the Biochemical Markers for Metabolic Syndrome: A Randomized Controlled Pilot Study.

The aim of this study was to evaluate the efficacy [of acupuncture] over 12 weeks of treatment and 12 weeks of follow-up. And the conclusion: Acupuncture decreases WC, HC, HbA1c, TG, and TC values and blood pressure in MetS.

It is almost painfully obvious that these studies are not ‘pilot’ studies as defined above.

So, what are they, and why are they so popular in alternative medicine?

The way I see it, they are the result of amateur researchers conducting pseudo-research for publication in lamentable journals in an attempt to promote their pet therapies (I have yet to find such a study that reports a negative finding). The sequence of events that lead to the publication of such pilot studies is usually as follows:

  • An enthusiast or a team of enthusiasts of alternative medicine decide that they will do some research.
  • They have no or very little know-how in conducting a clinical trial.
  • They nevertheless feel that such a study would be nice as it promotes both their careers and their pet therapy.
  • They design some sort of a plan and start recruiting patients for their trial.
  • At this point they notice that things are not as easy as they had imagined.
  • They have too few funds and too little time to do anything properly.
  • This does, however, not stop them to continue.
  • The trial progresses slowly, and patient numbers remain low.
  • After a while the would-be researchers get fed up and decide that their study has enough patients to stop the trial.
  • They improvise some statistical analyses with their results.
  • They write up the results the best they can.
  • They submit it for publication in a 3rd class journal and, in order to get it accepted, they call it a ‘pilot study’.
  • They feel that this title is an excuse for even the most obvious flaws in their work.
  • The journal’s reviewers and editors are all proponents of alternative medicine who welcome any study that seems to confirm their belief.
  • Thus the study does get published despite the fact that it is worthless.

Some might say ‘so what? no harm done!’

But I beg to differ: these studies pollute the medical literature and misguide people who are unable or unwilling to look behind the smoke-screen. Enthusiasts of alternative medicine popularise these bogus trials, while hiding the fact that their results are unreliable. Journalists report about them, and many consumers assume they are being told the truth – after all it was published in a ‘peer-reviewed’ medical journal!

My conclusions are as simple as they are severe:

  • Such pilot studies are the result of gross incompetence on many levels (researchers, funders, ethics committees, reviewers, journal editors).
  • They can cause considerable harm, because they mislead many people.
  • In more than one way, they represent a violation of medical ethics.
  • The could be considered scientific misconduct.
  • We should think of stopping this increasingly common form of scientific misconduct.

In recent days, journalists across the world had a field day (mis)reporting that doctors practising integrative medicine were doing something positive after all. I think that the paper shows nothing of the kind – but please judge for yourself.

The authors of this article wanted to determine differences in antibiotic prescription rates between conventional General Practice (GP) surgeries and GP surgeries employing general practitioners (GPs) additionally trained in integrative medicine (IM) or complementary and alternative medicine (CAM) (referred to as IM GPs) working within National Health Service (NHS) England.

They conducted a retrospective study on antibiotic prescription rates per STAR-PU (Specific Therapeutic group Age–sex weighting Related Prescribing Unit) using NHS Digital data over 2016. Publicly available data were used on prevalence of relevant comorbidities, demographics of patient populations and deprivation scores. setting Primary Care. Participants were 7283 NHS GP surgeries in England. The association between IM GPs and antibiotic prescribing rates per STAR-PU with the number of antibiotic prescriptions (total, and for respiratory tract infection (RTI) and urinary tract infection (UTI) separately) as outcome. results IM GP surgeries (n=9) were comparable to conventional GP surgeries in terms of list sizes, demographics, deprivation scores and comorbidity prevalence.

Statistically significant fewer total antibiotics  were prescribed at NHS IM GP surgeries compared with conventional NHS GP surgeries. In contrast, the number of antibiotics prescribed for UTI were similar between both practices.

The authors concluded that NHS England GP surgeries employing GPs additionally trained in IM/CAM have lower antibiotic prescribing rates. Accessibility of IM/CAM within NHS England primary care is limited. Main study limitation is the lack of consultation data. Future research should include the differences in consultation behaviour of patients self-selecting to consult an IM GP or conventional surgery, and its effect on antibiotic prescription. Additional treatment strategies for common primary care infections used by IM GPs should be explored to see if they could be used to assist in the fight against antimicrobial resistance.

The study was flimsy to say the least:

  • It was retrospective and is therefore open to no end of confounders.
  • There were only 9 surgeries in the IM group.

Moreover, the results were far from impressive. The differences in antibiotic prescribing between the two groups of GP surgeries were minimal or non-existent. Finally, the study was financed via an unrestricted grant of WALA Heilmittel GmbH, Germany (“approx. 900 different remedies conforming to the anthroposophic understanding of man and nature”) and its senior author has a long track record of publishing papers promotional for anthroposophic medicine.

Such pseudo-research seems to be popular in the realm of CAM, and I have commented before on similarly futile projects. The comparison, I sometimes use is that of a Hamburger restaurant:

Employees by a large Hamburger chain set out to study the association between utilization of Hamburger restaurant services and vegetarianism. The authors used a retrospective cohort design. The study population comprised New Hampshire residents aged 18-99 years, who had entered the premises of a Hamburger restaurant within 90 days for a primary purpose of eating. The authors excluded subjects with a diagnosis of cancer. They measured the likelihood of  vegetarianism among recipients of services delivered by Hamburger restaurants compared with a control group of individuals not using meat-dispensing facilities. They also compared the cohorts with regard to the money spent in Hamburger restaurants. The adjusted likelihood of being a vegetarian was 55% lower among the experimental group compared to controls. The average money spent per person in Hamburger restaurants were also significantly lower among the Hamburger group.

To me, it is obvious that such analyses must produce a seemingly favourable result for CAM. In the present case, there are several reasons for this:

  1. GPs who volunteer to be trained in CAM tend to be in favour of ‘natural’ treatments and oppose synthetic drugs such as antibiotics.
  2. Education in CAM would only re-inforce this notion.
  3. Similarly, patients electing to consult IM GPs tend to be in favour of ‘natural’ treatments and oppose synthetic drugs such as antibiotics.
  4. Such patients might be less severely ill that the rest of the patient population (the data from the present study do in fact imply this to be true).
  5. These phenomena work in concert to generate less antibiotic prescribing in the IM group.

In the final analysis, all this finding amounts to is a self-fulfilling prophecy: grocery shops sell less meat than butchers! You don’t believe me? Perhaps you need to read a previous post then; it concluded that physicians practicing integrative medicine (the 80% who did not respond to the survey were most likely even worse) not only use and promote much quackery, they also tend to endanger public health by their bizarre, irrational and irresponsible attitudes towards vaccination.

What is upsetting with the present paper, in my view, are the facts that:

  • a reputable journal published this junk,
  • the international press has a field-day reporting this study implying that CAM is a good thing.

The fact is that it shows nothing of the kind. Imagine we send GPs on a course where they are taught to treat all their patients with blood-letting. This too would result in less prescription of antibiotics, wouldn’t it? But would it be a good thing? Of course not!

True, we prescribe too much antibiotics. Nobody doubts that. And nobody doubts that it is a big problem. The solution to this problem is not more CAM, but less antibiotics. To realise the solution we do not need to teach GPs CAM but we need to remind them of the principles of evidence-based practice. And the two are clearly not the same; in fact, they are opposites.

 

As I often said, I find it regrettable that sceptics often say THERE IS NOT A SINGLE STUDY THAT SHOWS HOMEOPATHY TO BE EFFECTIVE (or something to that extent). This is quite simply not true, and it gives homeopathy-fans the occasion to suggest sceptics wrong. The truth is that THE TOTALITY OF THE MOST RELIABLE EVIDENCE FAILS TO SUGGEST THAT HIGHLY DILUTED HOMEOPATHIC REMEDIES ARE EFFECTIVE BEYOND PLACEBO. As a message for consumers, this is a little more complex, but I believe that it’s worth being well-informed and truthful.

And that also means admitting that a few apparently rigorous trials of homeopathy exist and some of them show positive results. Today, I want to focus on this small set of studies.

How can a rigorous trial of a highly diluted homeopathic remedy yield a positive result? As far as I can see, there are several possibilities:

  1. Homeopathy does work after all, and we have not fully understood the laws of physics, chemistry etc. Homeopaths favour this option, of course, but I find it extremely unlikely, and most rational thinkers would discard this possibility outright. It is not that we don’t quite understand homeopathy’s mechanism; the fact is that we understand that there cannot be a mechanism that is in line with the laws of nature.
  2. The trial in question is the victim of some undetected error.
  3. The result has come about by chance. Of 100 trials, 5 would produce a positive result at the 5% probability level purely by chance.
  4. The researchers have cheated.

When we critically assess any given trial, we attempt, in a way, to determine which of the 4 solutions apply. But unfortunately we always have to contend with what the authors of the trial tell us. Publications never provide all the details we need for this purpose, and we are often left speculating which of the explanations might apply. Whatever it is, we assume the result is false-positive.

Naturally, this assumption is hard to accept for homeopaths; they merely conclude that we are biased against homeopathy and conclude that, however, rigorous a study of homeopathy is, sceptics will not accept its result, if it turns out to be positive.

But there might be a way to settle the argument and get some more objective verdict, I think. We only need to remind ourselves of a crucially important principle in all science: INDEPENDENT REPLICATIONTo be convincing, a scientific paper needs to provide evidence that the results are reproducible. In medicine, it unquestionably is wise to accept a new finding only after it has been confirmed by other, independent researchers. Only if we have at least one (better several) independent replications, can we be reasonably sure that the result in question is true and not false-positive due to bias, chance, error or fraud.

And this is, I believe, the extremely odd phenomenon about the ‘positive’ and apparently rigorous studies of homeopathic remedies. Let’s look at the recent meta-analysis of Mathie et al. The authors found several studies that were both positive and fairly rigorous. These trials differ in many respects (e. g. remedies used, conditions treated) but they have, as far as I can see, one important feature in common: THEY HAVE NOT BEEN INDEPENDENTLY REPLICATED.

If that is not astounding, I don’t know what is!

Think of it: faced with a finding that flies in the face of science and would, if true, revolutionise much of medicine, scientists should jump with excitement. Yet, in reality, nobody seems to take the trouble to check whether it is the truth or an error.

To explain this absurdity more fully, let’s take just one of these trials as an example, one related to a common and serious condition: COPD

The study is by Prof Frass and was published in 2005 – surely long enough ago for plenty of independent replications to emerge. Its results showed that potentized (C30) potassium dichromate decreases the amount of tracheal secretions was reduced, extubation could be performed significantly earlier, and the length of stay was significantly shorter. This is a scientific as well as clinical sensation, if there ever was one!

The RCT was published in one of the leading journals on this subject (Chest) which is read by most specialists in the field, and it was at the time widely reported. Even today, there is hardly an interview with Prof Frass in which he does not boast about this trial with truly sensational results (only last week, I saw one). If Frass is correct, his findings would revolutionise the lives of thousands of seriously suffering patients at the very brink of death. In other words, it is inconceivable that Frass’ result has not been replicated!

But it hasn’t; at least there is nothing in Medline.

Why not? A risk-free, cheap, universally available and easy to administer treatment for such a severe, life-threatening condition would normally be picked up instantly. There should not be one, but dozens of independent replications by now. There should be several RCTs testing Frass’ therapy and at least one systematic review of these studies telling us clearly what is what.

But instead there is a deafening silence.

Why?

For heaven sakes, why?

The only logical explanation is that many centres around the world did try Frass’ therapy. Most likely they found it does not work and soon dismissed it. Others might even have gone to the trouble of conducting a formal study of Frass’ ‘sensational’ therapy and found it to be ineffective. Subsequently they felt too silly to submit it for publication – who would not laugh at them, if they said they trailed a remedy that was diluted 1: 1000000000000000000000000000000000000000000000000000000000000 and found it to be worthless? Others might have written up their study and submitted it for publication, but got rejected by all reputable journals in the field because the editors felt that comparing one placebo to another placebo is not real science.

And this is roughly, how it went with the other ‘positive’ and seemingly rigorous studies of homeopathy as well, I suspect.

Regardless of whether I am correct or not, the fact is that there are no independent replications (if readers know any, please let me know).

Once a sufficiently long period of time has lapsed and no replications of a ‘sensational’ finding did not emerge, the finding becomes unbelievable or bogus – no rational thinker can possibly believe such a results (I for one have not yet met an intensive care specialist who believes Frass’ findings, for instance). Subsequently, it is quietly dropped into the waste-basket of science where it no longer obstructs progress.

The absence of independent replications is therefore a most useful mechanism by which science rids itself of falsehoods.

It seems that homeopathy is such a falsehood.

 

 

The plethora of dodgy meta-analyses in alternative medicine has been the subject of a recent post – so this one is a mere update of a regular lament.

This new meta-analysis was to evaluate evidence for the effectiveness of acupuncture in the treatment of lumbar disc herniation (LDH). (Call me pedantic, but I prefer meta-analyses that evaluate the evidence FOR AND AGAINST a therapy.) Electronic databases were searched to identify RCTs of acupuncture for LDH, and 30 RCTs involving 3503 participants were included; 29 were published in Chinese and one in English, and all trialists were Chinese.

The results showed that acupuncture had a higher total effective rate than lumbar traction, ibuprofen, diclofenac sodium and meloxicam. Acupuncture was also superior to lumbar traction and diclofenac sodium in terms of pain measured with visual analogue scales (VAS). The total effective rate in 5 trials was greater for acupuncture than for mannitol plus dexamethasone and mecobalamin, ibuprofen plus fugui gutong capsule, loxoprofen, mannitol plus dexamethasone and huoxue zhitong decoction, respectively. Two trials showed a superior effect of acupuncture in VAS scores compared with ibuprofen or mannitol plus dexamethasone, respectively.

The authors from the College of Traditional Chinese Medicine, Jinan University, Guangzhou, Guangdong, China, concluded that acupuncture showed a more favourable effect in the treatment of LDH than lumbar traction, ibuprofen, diclofenac sodium, meloxicam, mannitol plus dexamethasone and mecobalamin, fugui gutong capsule plus ibuprofen, mannitol plus dexamethasone, loxoprofen and huoxue zhitong decoction. However, further rigorously designed, large-scale RCTs are needed to confirm these findings.

Why do I call this meta-analysis ‘dodgy’? I have several reasons, 10 to be exact:

  1. There is no plausible mechanism by which acupuncture might cure LDH.
  2. The types of acupuncture used in these trials was far from uniform and  included manual acupuncture (MA) in 13 studies, electro-acupuncture (EA) in 10 studies, and warm needle acupuncture (WNA) in 7 studies. Arguably, these are different interventions that cannot be lumped together.
  3. The trials were mostly of very poor quality, as depicted in the table above. For instance, 18 studies failed to mention the methods used for randomisation. I have previously shown that some Chinese studies use the terms ‘randomisation’ and ‘RCT’ even in the absence of a control group.
  4. None of the trials made any attempt to control for placebo effects.
  5. None of the trials were conducted against sham acupuncture.
  6. Only 10 studies 10 trials reported dropouts or withdrawals.
  7. Only two trials reported adverse reactions.
  8. None of these shortcomings were critically discussed in the paper.
  9. Despite their affiliation, the authors state that they have no conflicts of interest.
  10. All trials were conducted in China, and, on this blog, we have discussed repeatedly that acupuncture trials from China never report negative results.

And why do I find the journal ‘dodgy’?

Because any journal that publishes such a paper is likely to be sub-standard. In the case of ‘Acupuncture in Medicine’, the official journal of the British Medical Acupuncture Society, I see such appalling articles published far too frequently to believe that the present paper is just a regrettable, one-off mistake. What makes this issue particularly embarrassing is, of course, the fact that the journal belongs to the BMJ group.

… but we never really thought that science publishing was about anything other than money, did we?

A few days ago, the German TV ‘FACT’ broadcast a film (it is in German, the bit on homeopathy starts at ~min 20) about a young woman who had her breast cancer first operated but then decided to forfeit subsequent conventional treatments. Instead she chose homeopathy which she received from Dr Jens Wurster at the ‘Clinica Sta Croce‘ in Lucano/Switzerland.

Elsewhere Dr Wurster stated this: Contrary to chemotherapy and radiation, we offer a therapy with homeopathy that supports the patient’s immune system. The basic approach of orthodox medicine is to consider the tumor as a local disease and to treat it aggressively, what leads to a weakening of the immune system. However, when analyzing all studies on cured cancer cases it becomes evident that the immune system is always the decisive factor. When the immune system is enabled to recognize tumor cells, it will also be able to combat them… When homeopathic treatment is successful in rebuilding the immune system and reestablishing the basic regulation of the organism then tumors can disappear again. I’ve treated more than 1000 cancer patients homeopathically and we could even cure or considerably ameliorate the quality of life for several years in some, advanced and metastasizing cases.

The recent TV programme showed a doctor at this establishment confirming that homeopathy alone can cure cancer. Dr Wurster (who currently seems to be a star amongst European homeopaths) is seen lecturing at the 2017 World Congress of Homeopathic Physicians in Leipzig and stating that a ‘particularly rigorous study’ conducted by conventional scientists (the senior author is Harald Walach!, hardly a conventional scientist in my book) proved homeopathy to be effective for cancer. Specifically, he stated that this study showed that ‘homeopathy offers a great advantage in terms of quality of life even for patients suffering from advanced cancers’.

This study did, of course, interest me. So, I located it and had a look. Here is the abstract:

BACKGROUND:

Many cancer patients seek homeopathy as a complementary therapy. It has rarely been studied systematically, whether homeopathic care is of benefit for cancer patients.

METHODS:

We conducted a prospective observational study with cancer patients in two differently treated cohorts: one cohort with patients under complementary homeopathic treatment (HG; n = 259), and one cohort with conventionally treated cancer patients (CG; n = 380). For a direct comparison, matched pairs with patients of the same tumour entity and comparable prognosis were to be formed. Main outcome parameter: change of quality of life (FACT-G, FACIT-Sp) after 3 months. Secondary outcome parameters: change of quality of life (FACT-G, FACIT-Sp) after a year, as well as impairment by fatigue (MFI) and by anxiety and depression (HADS).

RESULTS:

HG: FACT-G, or FACIT-Sp, respectively improved statistically significantly in the first three months, from 75.6 (SD 14.6) to 81.1 (SD 16.9), or from 32.1 (SD 8.2) to 34.9 (SD 8.32), respectively. After 12 months, a further increase to 84.1 (SD 15.5) or 35.2 (SD 8.6) was found. Fatigue (MFI) decreased; anxiety and depression (HADS) did not change. CG: FACT-G remained constant in the first three months: 75.3 (SD 17.3) at t0, and 76.6 (SD 16.6) at t1. After 12 months, there was a slight increase to 78.9 (SD 18.1). FACIT-Sp scores improved significantly from t0 (31.0 – SD 8.9) to t1 (32.1 – SD 8.9) and declined again after a year (31.6 – SD 9.4). For fatigue, anxiety, and depression, no relevant changes were found. 120 patients of HG and 206 patients of CG met our criteria for matched-pairs selection. Due to large differences between the two patient populations, however, only 11 matched pairs could be formed. This is not sufficient for a comparative study.

CONCLUSION:

In our prospective study, we observed an improvement of quality of life as well as a tendency of fatigue symptoms to decrease in cancer patients under complementary homeopathic treatment. It would take considerably larger samples to find matched pairs suitable for comparison in order to establish a definite causal relation between these effects and homeopathic treatment.

_________________________________________________________________

Even the abstract makes several points very clear, and the full text confirms further embarrassing details:

  • The patients in this study received homeopathy in addition to standard care (the patient shown in the film only had homeopathy until it was too late, and she subsequently died, aged 33).
  • The study compared A+B with B alone (A=homeopathy, B= standard care). It is hardly surprising that the additional attention of A leads to an improvement in quality of life. It is arguably even unethical to conduct a clinical trial to demonstrate such an obvious outcome.
  • The authors of this paper caution that it is not possible to conclude that a causal relationship between homeopathy and the outcome exists.
  • This is true not just because of the small sample size, but also because of the fact that the two groups had not been allocated randomly and therefore are bound to differ in a whole host of variables that have not or cannot be measured.
  • Harald Walach, the senior author of this paper, held a position which was funded by Heel, Baden-Baden, one of Germany’s largest manufacturer of homeopathics.
  • The H.W.& J.Hector Foundation, Germany, and the Samueli Institute, provided the funding for this study.

In the film, one of the co-authors of this paper, the oncologist HH Bartsch from Freiburg, states that Dr Wurster’s interpretation of this study is ‘dishonest’.

I am inclined to agree.

1 2 3 9
Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted.


Click here for a comprehensive list of recent comments.

Categories