MD, PhD, FMedSci, FSB, FRCP, FRCPEd

economic evaluation

The dismal state of chiropractic research is no secret. But is anything being done about it? One important step would be to come up with a research strategy to fill the many embarrassing gaps in our knowledge about the validity of the concepts underlying chiropractic.

A brand-new article might be a step in the right direction. The aim of this survey was to identify chiropractors’ priorities for future research in order to best channel the available resources and facilitate advancement of the profession. The researchers recruited 60 academic and clinician chiropractors who had attended any of the annual European Chiropractors’ Union/European Academy of Chiropractic Researchers’ Day meetings since 2008. A Delphi process was used to identify a list of potential research priorities. Initially, 70 research priorities were identified, and 19 of them reached consensus as priorities for future research. The following three items were thought to be most important:

  1.  cost-effectiveness/economic evaluations,
  2.  identification of subgroups likely to respond to treatment,
  3.  initiation and promotion of collaborative research activities.

The authors state that this is the first formal and systematic attempt to develop a research agenda for the chiropractic profession in Europe. Future discussion and study is necessary to determine whether the themes identified in this survey should be broadly implemented.

Am I the only one who finds these findings extraordinary?

The chiropractic profession only recently lost the libel case against Simon Singh who had disclosed that chiropractors HAPPILY PROMOTE BOGUS TREATMENTS. One would have thought that this debacle might prompt the need for rigorous research testing the many unsubstantiated claims chiropractors still make. Alas, the collective chiropractic wisdom does not consider such research as a priority!

Similarly, I would have hoped that chiropractors perceive an urgency to investigate the safety of their treatments. Serious complications after spinal manipulation are well documented, and I would have thought that any responsible health care profession would consider it essential to generate reliable evidence on the incidence of such events.

The fact that these two areas are not considered to be priorities is revealing. In my view, it suggests that chiropractic is still very far from becoming a mature and responsible profession. It seems that chiropractors have not learned the most important lessons from recent events; on the contrary, they continue to bury their heads in the sand and carry on seeing research as a tool for marketing.

As mentioned several times on this blog, homeopathy lacks a solid evidence base (to put it mildly). There are powerful organisations which attempt to mislead the public about this fact, but most homeopathy-fans know this only too well, in my opinion. Some try to bypass this vexing fact by trying to convince us that homeopathy is value for money, never mind the hard science of experimental proof of its principles or the complexity of the clinical data. They might feel that politicans would take notice, if homeopathy would be appreciated as a cheap form of health care. In this context, it is worth mentioning that researchers from Sheffield have just published a systematic review of economic evaluations of homeopathy

They included 14 published assessments in their review. Eight studies found cost savings associated with the use of homeopathy. Four investigations suggested that improvements in homeopathy patients were at least as good as in control group patients, at comparable costs. Two studies found improvements similar to conventional treatment, but at higher costs. The researchers also noted that studies were highly heterogeneous and had numerous methodological weaknesses.

The authors concluded that “although the identified evidence of the costs and potential benefits of homeopathy seemed promising, studies were highly heterogeneous and had several methodological weaknesses. It is therefore not possible to draw firm conclusions based on existing economic evaluations of homeopathy“.

Thre are, of course, several types of economic evaluations of medical interventions; the most basic of these simply compares the cost of one medication with those of another. In such an analysis, homeopathy would normally win against conventional tratment, as homeopathic remedies are generally inexpensive. If one adds the treatment time into the equation, things become a little more complex; homeopathic consultations tend to be considerably longer that conventional ones, and if the homeopaths’ time is costed at the same rate as the time of conventional doctors, it is uncertain whether homeopathy would still be cheaper.

Much more relevant, in my view, are cost-effective analyses which compare the relative costs and outcomes of two or more treatments. The results of such evaluations are often expressed in terms of a ratio where the denominator is a gain in health from a treatment and the numerator is the cost associated with the health gain. The most common measure used to express this is the QUALY.

Any cost-effective analysis can only produce meaningfully positive results, if the treatment in question supported by sound evidence for effectivenes. A treatment that is not demonstrably effective cannot be cost-effective! And this is where the principal problem with any cost-effectiveness analysis of homeopathy lies. Homeopathic remedies are placebos and thus can be neither effective nor cost-effective. Arguments to the contrary are in my view fallacious.

The authors of the new article say they have  identified evidence of the potential benefits of homeopathy. How can this be? They based this conclusion only on the 14 studies included in their review. But this is only about 5% of the total available data. Reliable estimates of effectiveness should be based on the totality of the available evidence and not on a selection thereof.

I therefore think it is wise to focus on the part of the authors’ conclusion that does make sense: ” It is… not possible to draw firm conclusions based on existing economic evaluations of homeopathy“. In plain English: economic evaluations of  homeopathy fail to show that it is value for money.

Categories