The UK ‘Society of Homeopaths’ (SoH) is the largest professional organisation of UK non-doctor, so-called lay- homeopaths. On their website, the SoH made very specific claims about homeopathy; in particular, they listed conditions for which homeopathy had allegedly been proven to be effective. These claims have now thoroughly been debunked, and the evidence the SoH produced in support of their claims has been shown to be misleading, cherry-picked or misinterpreted.
I have no idea who conducted the above-named investigation and made a youtube video of it, but I think it is essentially correct and well worth watching. My own experiences with the SoH relate mainly to two encounters.
The first was a complaint I made about one of their high-ranking officers, Ralf Jeutter. He had been promotiong homeopathic vaccinations on his website (needless to stress, I think, that there is no evidence to support the notion that homeopathic vaccinations are effective). As I felt that the SoH dragged their feet pursuing my complaint, I had to send several reminders. Eventually, they considered it and concluded that Reuter had done nothing wrong. This, presumably, is the reason why, even today, he can state on his website that Homeopathy is used to help individuals in dealing better with kinds of infections such as leptospirosis, meningitis and cholera. All is fine, it seems as long as a disclaimer is added: Any information obtained here is not to be construed as medical OR legal advice. The decision to vaccinate and how you implement that decision is yours and yours alone. The evidence for the efficacy of homeopathic immunisation is ‘anecdotal’. That means it is based on individuals’ reports past and present.
My second encounter with the SoH relates to my 2010 analysis of the SoH code of ethics and their adherence to it. The code demanded that:
- ‘all speculative theories will be stated as such and clearly distinguished’
- ‘no advertising may be used which expressly or implicitly claims to cure named diseases’
- ‘Advertising shall not be false, fraudulent, misleading, deceptive, extravagant or sensational.’
Encouraged by these assurances, I decided to study the websites of some members of the SoH, and soon discovered numerous and very obvious violations of the above-mentioned imperatives. In an attempt to find the root of these transgressions, I scrutinised the SoH’s own website where I found a multitude violations on all levels of the SoH’s own code of ethics. Many of the violations related to claims which were not supported by evidence. In other words, the largest professional UK organisation of lay- homeopaths misled the public in several rather devious ways:
they pretended to adhere to a code of ethics which forbids members to mislead the public
SoH -members nevertheless did mislead the public in ways that public health at risk
and they did so not least because the SoH followed exactly the same strategy
thus the SoH violated its own code of ethics to the detriment of public health.
My analysis was conducted a while ago, and some might hope that the SoH has stopped systematically misleading the public. This hope, however, is harshly disappointed when you watch the brand-new video entitled TESTING HOMEOPATHY mentioned above. As the SoH is about to celebrate 35 years of wisdom, courage, knowledge and prosperity, I do wonder whether this should not be 35 years of dangerously misleading the public.
What do you think?
In my last post and several others before, I have stated that consumers are incessantly being mislead about the value of alternative medicine. This statement requires evidence, and I intend to provide it – not just in one post but in a series of posts following in fast succession.
I start with an investigation we did over a decade ago. Its primary aim was to determine which complementary therapies are believed by their respective representing UK professional organizations to be suited for which medical conditions.
For this purpose, we sent out 223 questionnaires to CAM organizations representing a single CAM therapy (yes, amazingly that many such institutions exist just in the UK!). They were asked to list the 15 conditions which they felt benefited most from their specific CAM therapy, as well as the 15 most important contra-indications, the typical costs of initial and any subsequent treatments and the average length of training required to become a fully qualified practitioner. The conditions and contra-indications quoted by responding CAM organizations were recorded and the top five of each were determined. Treatment costs and hours of training were expressed as ranges.
Only 66 questionnaires were returned. Taking undelivered questionnaires into account, the response rate was 34%. Two or more responses were received from CAM organizations representing twelve therapies: aromatherapy, Bach flower remedies, Bowen technique, chiropractic, homoeopathy, hypnotherapy, magnet therapy, massage, nutrition, reflexology, Reiki and yoga.
The top seven common conditions deemed to benefit from all twelve therapies, in order of frequency, were: stress/anxiety, headaches/migraine, back pain, respiratory problems (including asthma), insomnia, cardiovascular problems and musculoskeletal problems. It is perhaps important at this stage to point out that some of these conditions are serious, even life-threatening. Aromatherapy, Bach flower remedies, hypnotherapy, massage, nutrition, reflexology, Reiki and yoga were all recommended as suitable treatments for stress/anxiety. Aromatherapy, Bowen technique, chiropractic, hypnotherapy, massage, nutrition, reflexology, Reiki and yoga were all recommended for headache/migraine. Bowen technique, chiropractic, magnet therapy, massage, reflexology and yoga were recommended for back pain. None of the therapies cost more than £60 for an initial consultation and treatment. No correlation between length of training and treatment cost was noted.
I think, this article provides ample evidence to show that, at least in the UK, professional organisations of alternative medicine readily issue statements about the effectiveness of specific alternative therapies which are not supported by evidence. Several years later, Simon Singh noted that phenomenon in a Guardian-comment and wrote about the British Chiropractic Association “they happily promote bogus claims”. He was famously sued for libel but won the case. Simon had picked the BCA merely by chance. The frightening thought is that he could have targeted any other of the 66 organisations from our investigation: they all seem to promote bogus claims quite happily.
Several findings from our study stood out for being particularly worrying: according to the respective professional organisation, Bach Flower Remedies were deemed to be effective for cancer and AIDS, for instance. If their peers put out such irresponsible nonsense, we should not be amazed at the claims made by the practitioners. And if the practitioners tell such ‘tall tales’ to their clients, to journalists and to everyone else, how can we be amazed that we seem to be drowning in a sea of misinformation?
Can one design a clinical study in such a way that it looks highly scientific but, at the same time, has zero chances of generating a finding that the investigators do not want? In other words, can one create false positive findings at will and get away with it? I think it is possible; what is more, I believe that, in alternative medicine, this sort of thing happens all the time. Let me show you how it is done; four main points usually suffice:
- The first rule is that it ought to be an RCT, if not, critics will say the result was due to selection bias. Only RCTs have the reputation of being ‘top notch’.
- Once we are clear about this design feature, we need to define the patient population. Here the trick is to select individuals with an illness that cannot be quantified objectively. Depression, stress, fatigue…the choice is vast. The aim must be to employ an outcome measure that is well-accepted, validated etc. but which nevertheless is entirely subjective.
- Now we need to consider the treatment to be “tested” in our study. Obviously we take the one we are fond of and want to “prove”. It helps tremendously, if this intervention has an exotic name and involves some exotic activity; this raises our patients’ expectations which will affect the result. And it is important that the treatment is a pleasant experience; patients must like it. Finally it should involve not just one but several sessions in which the patient can be persuaded that our treatment is the best thing since sliced bread – even if, in fact, it is entirely bogus.
- We also need to make sure that, for our particular therapy, no universally accepted placebo exists which would allow patient-blinding. That would be fairly disastrous. And we certainly do not want to be innovative and create such a placebo either; we just pretend that controlling for placebo-effects is impossible or undesirable. By far the best solution would be to give the control group no treatment at all. Like this, they are bound to be disappointed for missing out a pleasant experience which, in turn, will contribute to unfavourable outcomes in the control group. This little trick will, of course, make the results in the experimental group look even better.
That’s about it! No matter how ineffective our treatment is, there is no conceivable way our study can generate a negative result; we are in the pink!
Now we only need to run the trial and publish the positive results. It might be advisable to recruit several co-authors for the publication – that looks more serious and is not too difficult: people are only too keen to prolong their publication-list. And we might want to publish our study in one of the many CAM-journals that are not too critical, as long as the result is positive.
Once our article is in print, we can legitimately claim that our bogus treatment is evidence-based. With a bit of luck, other research groups will proceed in the same way and soon we will have not just one but several positive studies. If not, we need to do two or three more trials along the same lines. The aim is to eventually do a meta-analysis that yields a convincingly positive verdict on our phony intervention.
You might think that I am exaggerating beyond measure. Perhaps a bit, I admit, but I am not all that far from the truth, believe me. You want proof? What about this one?
Researchers from the Charite in Berlin just published an RCT to investigate the effectiveness of a mindful walking program in patients with high levels of perceived psychological distress.
To prevent allegations of exaggeration, selective reporting, spin etc. I take the liberty of reproducing the abstract of this study unaltered:
Participants aged between 18 and 65 years with moderate to high levels of perceived psychological distress were randomized to 8 sessions of mindful walking in 4 weeks (each 40 minutes walking, 10 minutes mindful walking, 10 minutes discussion) or to no study intervention (waiting group). Primary outcome parameter was the difference to baseline on Cohen’s Perceived Stress Scale (CPSS) after 4 weeks between intervention and control.
Seventy-four participants were randomized in the study; 36 (32 female, 52.3 ± 8.6 years) were allocated to the intervention and 38 (35 female, 49.5 ± 8.8 years) to the control group. Adjusted CPSS differences after 4 weeks were -8.8 [95% CI: -10.8; -6.8] (mean 24.2 [22.2; 26.2]) in the intervention group and -1.0 [-2.9; 0.9] (mean 32.0 [30.1; 33.9]) in the control group, resulting in a highly significant group difference (P < 0.001).
Conclusion. Patients participating in a mindful walking program showed reduced psychological stress symptoms and improved quality of life compared to no study intervention. Further studies should include an active treatment group and a long-term follow-up
This whole thing could just be a bit of innocent fun, but I am afraid it is neither innocent nor fun, it is, in fact, quite serious. If we accept manipulated trials as evidence, we do a disservice to science, medicine and, most importantly, to patients. If the result of a trial is knowable before the study has even started, it is unethical to run the study. If the trial is not a true test but a simple promotional exercise, research degenerates into a farcical pseudo-science. If we abuse our patients’ willingness to participate in research, we jeopardise more serious investigations for the benefit of us all. If we misuse the scarce funds available for research, we will not have the money to conduct much needed investigations. If we tarnish the reputation of clinical research, we hinder progress.
“Wer heilt hat recht”. Every German knows this saying and far too many believe it. Literally translated, it means THE ONE WHO HEALS IS RIGHT, and indicates that, in health care, the proof of efficacy of a treatment is self-evident: if a clinician administers a treatment and the patient improves, she was right in prescribing it and the treatment must have been efficacious. The only English saying which is vaguely similar (but rarely used for therapies) is THE PROOF OF THE PUDDING IS IN THE EATING, translated into a medical context: the proof of the treatment is in the clinical outcome.
The saying is German but the sentiment behind it is amazingly widespread across the world, particularly the alternative one. If I had a fiver for each time a German journalist has asked me to comment on this ‘argument’ I could probably invite all my readers for a beer in the pub. The notion seems to be irresistibly appealing and journalists, consumers, patients, politicians etc. fall for it like flies. It is popular foremost as a counter-argument against scientists’ objections to homeopathy and similar placebo-treatments. If the homeopath cured her patient, then she and her treatments are evidently fine!
It is time, I think, that I scrutinise the argument and refute it once and for all.
The very first thing to note is that placebos never cure a condition. They might alleviate symptoms, but cure? No!
The next issue relates to causality. The saying assumes that the sole reason for the clinical outcome is the treatment. Yet, if a patient’s symptoms improve, the reason might have been the prescribed treatment, but this is just one of a multitude of different options, e.g.:
- the placebo-effect
- the regression towards the mean
- the natural history of the condition
- the Hawthorne effect
- the compassion of the clinician
- other treatments that might have been administered in parallel
Often it is a complex mixture of these and possibly other phenomena that is responsible and, unless we run a proper clinical trial, we cannot even guess the relative importance of each factor. To claim in such a messy situation that the treatment given by the clinician was the cause of the improvement, is ridiculously simplistic and overtly wrong.
But that is precisely what the saying WER HEILT HAT RECHT does. It assumes a simple mono-causal relationship that never exists in clinical settings. And, annoyingly, it somewhat arrogantly dismisses any scientific evidence by implying that the anecdotal observation is so much more accurate and relevant.
The true monstrosity of the saying can be easily disclosed with a little thought experiment. Let’s assume the saying is correct and we adopt it as a major axiom in health care. This would have all sorts of terrible consequences. For instance, any pharmaceutical company would be allowed to produce colourful placebos and sell them for a premium; they would only need to show that some patients do experience some relief after taking it. THE ONE WHO HEALS IS RIGHT!
The saying is a dangerously misleading platitude. That it happens to be German and that the Germans remain so frightfully fond of it disturbs me. That the notion, in one way or another, is deeply ingrained in the mind of charlatans across the world is worrying but hardly surprising – after all, it is said to have been coined by Samuel Hahnemann.
A lengthy article posted by THE HOMEOPATHIC COLLEGE recently advocated treating cancer with homeopathy. Since I doubt that many readers access this publication, I take the liberty of reproducing here their (also fairly lengthy) CONCLUSIONS in full:
Laboratory studies in vitro and in vivo show that homeopathic drugs, in addition to having the capacity to reduce the size of tumors and to induce apoptosis, can induce protective and restorative effects. Additionally homeopathic treatment has shown effects when used as a complementary therapy for the effects of conventional cancer treatment. This confirms observations from our own clinical experience as well as that of others that when suitable remedies are selected according to individual indications as well as according to pathology and to cell-line indications and administered in the appropriate doses according to the standard principles of homeopathic posology, homeopathic treatment of cancer can be a highly effective therapy for all kinds of cancers and leukemia as well as for the harmful side effects of conventional treatment. More research is needed to corroborate these clinical observations.
Homeopathy over almost two decades of its existence has developed more than four hundred remedies for cancer treatment. Only a small fraction have been subjected to scientific study so far. More homeopathic remedies need to be studied to establish if they have any significant action in cancer. Undoubtedly the next big step in homeopathic cancer research must be multiple comprehensive double-blinded, placebo-controlled, randomized clinical trials. To assess the effect of homeopathic treatment in clinical settings, volunteer adult patients who prefer to try homeopathic treatment instead of conventional therapy could be recruited, especially in cases for which no conventional therapy has been shown to be effective.
Many of the researchers conducting studies — cited here but not discussed — on the growing interest in homeopathic cancer treatment have observed that patients are driving the demand for access to homeopathic and other alternative modes of cancer treatment. So long as existing cancer treatment is fraught with danger and low efficacy, it is urgent that the research on and the provision of quality homeopathic cancer treatment be made available for those who wish to try it.
When I report about nonsense like that, I find it hard not to go into a fuming rage. But doing that would not be very constructive – so let me instead highlight (in random order) eight simple techniques that seem to be so common when unsubstantiated claims are being promoted for alternative treatments:
1) cherry pick the data
2) use all sorts of ‘evidence’ regardless how flimsy or irrelevant it might be
3) give yourself the flair of being highly scientific and totally impartial
4) point out how dangerous and ineffective all the conventional treatments are
5) do not shy away from overt lies
6) do not forget to stress that the science is in full agreement with your exhaustive clinical experience
7) stress that patients want what you are offering
8) ignore the biological plausibility of the underlying concepts
Provided we adhere to these simple rules, we can convince the unsuspecting public of just about anything – even of the notion that homeopathy is a cure for cancer!
In Europe, we have chiropractors, homeopaths, naturopaths and anthroposophical physicians who recommend to their patients not to vaccinate their children. In the US, they have all this plus some of the clergy to jeopardize herd immunity.
An outbreak of measles infections has been reported in Tarrant County, Texas, US where at least 21 people have been affected this month at the Eagle Mountain International Church. The ministers of this church have been critical of vaccination and advised to use alternative treatments. Several more cases of infections with fever and rash have been noted, but so far remain unconfirmed.
Before the measles vaccine was introduced in 1963, between 3 million and 4 million people in the U.S. were infected each year, 48,000 of them needed hospitalisation and 400 to 500 died. Another 1,000 developed chronic disabilities. In the US measles were considered eradicated in 2000, but outbreaks continue because of imported infections brought back by travellers from areas where measles remains common.
The Texas outbreak was caused by a non-vaccinated visitor who had been infected in Indonesia and then returned to expose unvaccinated church members, staff and children in a day-care centre. In the wider community, more than 98 per cent of kids are immunized and less than 1 per cent are exempt. But the congregation of unvaccinated people allowed the disease to catch hold. Church leaders, including Kenneth Copeland and his daughter, Terri Pearsons, senior pastor at Eagle Mountain, have advocated faith-healing and questioned vaccines in the past.
And what can faith-healing achieve? Where is the evidence that it prevents or cures infections or any other diseases? You probably guessed: there is none.
Swiss chiropractors have just published a clinical trial to investigate outcomes of patients with radiculopathy due to cervical disk herniation (CDH). All patients had neck pain and dermatomal arm pain; sensory, motor, or reflex changes corresponding to the involved nerve root and at least one positive orthopaedic test for cervical radiculopathy were included. CDH was confirmed by magnetic resonance imaging. All patients received regular neck manipulations.
Baseline data included two pain numeric rating scales (NRSs), for neck and arm, and the Neck Disability Index (NDI). At two, four and twelve weeks after the initial consultation, patients were contacted by telephone, and the data for NDI, NRSs, and patient’s global impression of change were collected. High-velocity, low-amplitude thrusts were administered by experienced chiropractors. The proportion of patients reporting to feel “better” or “much better” on the patient’s global impression of change scale was calculated. Pre-treatment and post-treatment NRSs and NDIs were analysed.
Fifty patients were included. At two weeks, 55.3% were “improved,” 68.9% at four and 85.7% at twelve weeks. Statistically significant decreases in neck pain, arm pain, and NDI scores were noted at 1 and 3 months compared with baseline scores. 76.2% of all sub-acute/chronic patients were improved at 3 months.
The authors concluded that most patients in this study, including sub-acute/chronic patients, with symptomatic magnetic resonance imaging-confirmed CDH treated with spinal manipulative therapy, reported significant improvement with no adverse events.
In the presence of disc herniation, chiropractic manipulations have been described to cause serious complications. Some experts therefore believe that CDH is a contra-indication for spinal manipulation. The authors of this study imply, however, that it is not – on the contrary, they think it is an effective intervention for CDH.
One does not need to be a sceptic to notice that the basis for this assumption is less than solid. The study had no control group. This means that the observed effect could have been due to:
a placebo response,
the regression towards the mean,
the natural history of the condition,
or other factors which have nothing to do with the chiropractic intervention per se.
And what about the interesting finding that no adverse-effects were noted? Does that mean that the treatment is safe? Sorry, but it most certainly does not! In order to generate reliable results about possibly rare complications, the study would have needed to include not 50 but well over 50 000 patients.
So what does the study really tell us? I have pondered over this question for some time and arrived at the following answer: NOTHING!
Is that a bit harsh? Well, perhaps yes. And I will revise my verdict slightly: the study does tell us something, after all – chiropractors tend to confuse research with the promotion of very doubtful concepts at the expense of their patients. I think, there is a name for this phenomenon: PSEUDO-SCIENCE.
The aim of this retrospective chart-review was to identify the percentage of non-musculoskeletal and musculoskeletal conditions treated by interns in the NUHS Student Clinic. The information was taken from the charts of patients treated in the fall trimester of 2011.
The results show that 52% of all patients were treated only for musculoskeletal conditions, and 48% were treated for non-musculoskeletal conditions, or musculoskeletal plus non-musculoskeletal conditions.
The authors draw the following conclusions: The NUHS Student Clinic interns are treating a greater percentage of non-musculoskeletal conditions and a lesser percentage of musculoskeletal conditions than practicing chiropractic physicians. The student interns also treat a lesser percentage of non-musculoskeletal and a greater percentage of musculoskeletal conditions than allopathic practitioners. This comparison would suggest that NUHS is nearing its institutional goal of training its student interns as primary care practitioners.
The very last sentence of the conclusions is particularly surprising, in my view. Do these findings really imply that the NUHS is training competent primary care practitioners? I fail to see that the data demonstrate this. On the contrary, I think they show that some US chiropractic schools want to promote the notion that chiropractors are, in fact, primary care physicians. More worryingly, I fear that this article demonstrates how, through the diligent work of chiropractic schools, the myth is being kept alive that chiropractic is effective for all sorts of non-musculoskeletal conditions. In other words, I think we might here have a fine example of unsubstantiated beliefs being handed from one to the next generation of chiropractors.
Evidence-based chiropractic my foot! They continue to “happily promote bogus claims”.
A cult can be defined not just in a religious context, but also as a” usually nonscientific method or regimen claimed by its originator to have exclusive or exceptional power in curing a particular disease.” After ~20 years of researching this area, I have come to suspect that much of alternative medicine resembles a cult – a bold statement, so I better explain.
One characteristic of a cult is the unquestioning commitment of its members to the bizarre ideas of their iconic leader. This, I think, chimes with several forms alternative medicine. Homeopaths, for instance, very rarely question the implausible doctrines of Hahnemann who, to them, is some sort of a semi-god. Similarly, few chiropractors doubt even the most ridiculous assumptions of their founding father, D D Palmer who, despite of having been a somewhat pathetic figure, is uncritically worshipped. By definition, a cult-leader is idealised and thus not accountable to anyone; he (yes, it is almost invariably a male person) cannot be proven wrong by logic arguments nor by scientific facts. He is quite simply immune to any form of scrutiny. Those who dare to disagree with his dogma are expelled, punished, defamed or all of the above.
Cults tend to brain-wash their members into unconditional submission and belief. Likewise, fanatics of alternative medicine tend to be brain-washed, i.e. systematically misinformed to the extend that reality becomes invisible. They unquestioningly believe in what they have been told, in what they have read in their cult-texts, and in what they have learnt from their cult-peers. The effects of this phenomenon can be dramatic: the powers of discrimination of the cult-member are reduced, critical questions are discouraged, and no amount of evidence can dissuade the cult-member from abandoning even the most indefensible concepts. Internal criticism is thus by definition non-existent.
Like religious cults, many forms of alternative medicine promote an elitist concept. Cult-members become convinced of their superiority, based not on rational considerations but on irrational beliefs. This phenomenon has a range of consequences. It leads to the isolation of the cult-member from the rest of the world. By definition, critics of the cult do not belong to the elite; they are viewed as not being able to comprehend the subtleties of the issues at hand and are thus ignored or not taken seriously. For cult-members, external criticism is thus non-existent or invalid.
Cult-members tend to be on a mission, and so are many enthusiasts of alternative medicine. They use any conceivable means to recruit new converts. For instance, they try to convince family, friends and acquaintances of their belief in their particular alternative therapy at every conceivable occasion. They also try to operate on a political level to popularize their cult. They cherry pick data, often argue emotionally rather than rationally, and ignore all arguments which contradict their belief system.
Cult-members, in their isolation from society, tend to be assume that there is little worthy of their consideration outside the cult. Similarly, enthusiasts of alternative medicine tend to think that their treatment is the only true method of healing. Therapies, concepts and facts which are not cult-approved are systematically defamed. An example is the notion of BIG PHARMA which is employed regularly in alternative medicine. No reasonable person assumes that the pharmaceutical industry smells of roses. However, the exaggerated and systematic denunciation of this industry and its achievements is a characteristic of virtually all branches of alternative medicine. Such behaviour usually tells us more about the accuser than the accused.
There are many other parallels between a cult and alternative medicine, I am sure. In my view, the most striking one must be the fact that any spark of cognitive dissonance in the cult-victim is being extinguished by highly effective and incessant flow of misinformation which often amounts to a form of brain-washing.
Whenever we consider alternative medicine, we think of therapeutic interventions and tend to forget that alternative practitioners frequently employ diagnostic methods which are alien to mainstream health care. Acupuncturists, iridologists, spiritual healers, massage therapists, reflexologists, applied kinesiologists, homeopaths, chiropractors, osteopaths and many other types of alternative practitioners all have their very own ways of diagnosing what might be wrong with their patients.
The purpose of a diagnostic test or technique is, of course, to establish the presence or absence of an abnormality, condition or disease. Conventional doctors use all sorts of validated diagnostic methods, from physical examination to laboratory tests, from blood pressure measurements to X-rays. Alternative practitioners use mostly alternative methods for arriving at a diagnosis, and we should ask: how reliable are these techniques?
Anyone trying to answer this question, will be surprised to find how very little reliable information on this topic exists. Scientific tests of the validity of alternative diagnostic tests are a bit like gold dust. And this is why a recently published article is, in my view, of particular importance and value.
The aim of this study was to evaluate the inter-rater reliability of pulse-diagnosis as performed by Traditional Korean Medicine (TKM) clinicians. A total 658 patients with stroke who were admitted into Korean oriental medical university hospitals were included. Each patient was seen by two TKM-experts for an examination of the pulse signs – pulse diagnosis is regularly used by practitioners of TKM and Traditional Chinese Medicine (TCM), and is entirely different from what conventional doctors do when they feel the pulse of a patient. Inter-observer reliability was assessed using three methods: simple percentage agreement, the kappa value, and the AC(1) statistic. The kappa value indicated that the inter-observer reliability in evaluating the pulse signs ranged from poor to moderate, whereas the AC(1) analysis suggested that agreement between the two experts was generally high (with the exception of ‘slippery pulse’). The kappa value indicated that the inter-observer reliability was generally moderate to good (with the exceptions of ‘rough pulse’ and ‘sunken pulse’) and that the AC(1) measure of agreement between the two experts was generally high.
Based on these findings, the authors drew the following conclusion: “Pulse diagnosis is regarded as one of the most important procedures in TKM… This study reveals that the inter-observer reliability in making a pulse diagnosis in stroke patients is not particularly high when objectively quantified. Additional research is needed to help reduce this lack of reliability for various portions of the pulse diagnosis.”
This indicates, I think, that the researchers (who are themselves practitioners of TCM!) are not impressed with the inter-rater reliability of the most commonly used diagnostic tool in TCM/TKM. Imagine this to be true for a commonly used test in conventional medicine; imagine, for instance, that one doctor measuring your blood pressure produces entirely different readings than the next one. Hardly acceptable, don’t you think?
And, of course, inter-rater reliability would be only one of several preconditions for their diagnostic methods to be valid. Other essential preconditions for diagnostic tests to be of value are their specificity and their sensitivity; do they discriminate between healthy and unhealthy, and are they capable of differentiating between severely abnormal findings and those that are just a little out of the normal range?
Until we have answers to all the open questions about each specific alternative diagnostic method, it would be unwise to pretend these tests are valid. Imagine a doctor prescribing a life-long anti-hypertensive therapy on the basis of a blood pressure reading that is little more than guess-work!
Since non-validated diagnostic tests can generate both false positive and false negative results, the danger of using them should not be under-estimated. In a way, invalid diagnostic tests are akin to bogus bomb-detectors (which made headlines recently): both are techniques to identify a problem. If the method generates a false positive result, an alert will be issued in vain, people will get anxious for nothing, time and money will be lost, etc. If the method generates a false negative result, we will assume to be safe while, in fact, we are not. In extreme cases, such an error will cost lives.
It is difficult to call those ‘experts’ who advocate using such tests anything else than irresponsible, I’d say. And it is even more difficult to have any confidence in the treatments that might be administered on the basis of such diagnostic methods, wouldn’t you agree?