MD, PhD, FMedSci, FSB, FRCP, FRCPEd

survey

I have mentioned it before, I know, but it seems important, so please bear with me as I revisit the subject: there is no other area of health care that is more plagued by surveys than alternative medicine. They are usually conducted on a small convenience sample of consumers and try to tell us that many of them use and like alternative medicine (or a specific alternative treatment). And why is this important? Because this information is subsequently employed to convince us, politicians, journalists, heirs to the throne etc. that thousands of consumers cannot be wrong and that alternative medicine must therefore be a good thing.
Sceptics know, of course, that this argumentum ad populum is a classical fallacy. Recently, we published an article which provides fairly hard evidence to substantiate this fact.

The main aim of our systematic review was to estimate the prevalence of use of alternative medicine (AM) in the UK. Five databases were searched for peer-reviewed surveys published between 1 January 2000 and 7 October 2011. In addition, relevant book chapters and files from our own departmental records were searched by hand. Eighty-nine surveys were included, with a total of 97,222 participants. Surely, fact that this large amount of UK surveys had emerged in only about one decade, speaks for itself.

Most studies turned out to be of poor methodological quality. Across all surveys, the average one-year prevalence of AM-use was 41.1%, and the average lifetime prevalence was 51.8%. However, many of these investigations were flimsy. According to methodologically sound surveys, the equivalent rates were 26.3% and 44%, respectively. In surveys with response rates >70%, the average one-year prevalence was nearly threefold lower than in surveys with response rates below 50%. Herbal medicine was the most popular CAM, followed by homeopathy, aromatherapy, massage and reflexology.

To the best of my knowledge, this is the first time that four crucial points about such surveys have been clearly documented:

1) The amount of surveys in AM is staggering.

2) They contribute very little worthwhile knowledge and mostly seem to be exercises in AM-promotion.

3) Their methodological quality is usually low.

4) The poor quality surveys systematically over-estimate the prevalence of AM-use.

I think it is time that AM investigators focus on real research answering important questions which advance out knowledge, that AM-journal editors stop publishing meaningless nonsense, and that decision-makers understand the difference between promotion dressed up as science and real research.

 

 

A team of Swiss and UK chiropractors just published a survey to determine which management options their colleagues would choose in response to several clinical case scenarios. In order to avoid the accusations of citing out of context, or misreporting the findings in other ways, the wording of the following post is close to the original text of the article.

PART ONE

The clinical scenarios refer to treatments which appear not to be successful, not indicated, possibly harmful or where a patient might be suffering from a treatment-induced complication:

Scenario 1. A patient with non-specific low back pain has not improved at all after 4–6 treatments.

Scenario 2. A patient, who has a simple neck problem with no previous long-term problems, has now improved at least 80% and stayed at this level for a couple of weeks.

Scenario 3. A patient returns from the last treatment with a new distal pain (e.g. sciatica when treated only for localized LBP, or brachialgia when treated only for local neck pain).

Scenario 4. An elderly woman complains about immediate chest pain on inspiration after manual treatment directed to her thoracic spine.

It is worth noting that scenario 4 is the most dramatic but it is by far not the worst case scenario; this would have been the case of a patient who develops signs of a stroke after neck manipulation. It is telling, I think, that this possibility has been excluded in the survey.

The following 9 management options were provided:

• I would re-evaluate the patient with a view to establishing a better diagnosis

• I would send the patient for diagnostic imaging

• I would change my treatment approach and use another technique

• I would send the patient for a second opinion to another healthcare professional but keep on monitoring their condition

• I would try a few times more

• I would encourage the patient to continue the treatment until their spine is subluxation-free

• I would stop treatment and monitor the patient regularly

• I would stop the treatment, apologise and report the event to the chiropractic reporting and learning system

• I would stop the treatment, but tell the patient that s/he is welcome to return if they feel the need

To each of these options, the chiropractors could answer by ticking: ‘never’, ‘unlikely’, ‘likely’ and ‘most likely’.

PART TWO

In a second part of the questionnaire the researchers assessed the chiropractors’ general attitude towards safety issues by seeking the level of agreement on a five-point scale, with the responses ‘strongly disagree’, ‘disagree’, ‘neither agree nor disagree’, ‘agree’ and ‘strongly agree’, with 23 statements relating to six different safety dimensions, as follows:

• Teamwork – helping out, relationships, respect, teamwork-emphasis

• Work pressure – rushing, overwork, staff contingent, patient numbers

• Staff training – in response to new processes, on-the-job, appropriateness of tasks

• Process and standardisation – organisation, procedures, workflow, processes

• Communication openness – ideas for improvement, alternative views, asking questions, voicing disagreement

• Patient tracking/follow-up – reminders, documentation, reports, monitoring

260 Swiss and 1258 UK chiropractors were invited to complete the questionnaire. Responses were received from 76% of the Swiss and from 31% of the UK chiropractors. The dismal response rate for UK chiropractors seems to speak volumes.

The results of this survey indicate that both Swiss and UK chiropractors tend to manage clinical scenarios where treatment appears not to be successful, not indicated, possibly harmful or where a serious complication might have occurred, by re-evaluating their care. Stopping treatment and/or incident reporting to a safety incident reporting and learning system were generally found to be unlikely courses of action. The authors believe that this unlikeliness of safety incident reporting is due to a range of recognised barriers, although Swiss and UK chiropractors are positive about local communication and openness which are important tenets for safety incident reporting. The observed positivity towards key aspects of clinic safety indicates a developing safety culture within the Swiss and UK chiropractic professions.

In this context, scenario 4 is the most dramatic and therefore the most relevant scenario -but, as noted above, not a worst case scenario. It suggested a rib fracture as a result of chiropractic manipulation, with osteoporosis as a possible risk factor. The authors state that there is a strong argument for such an incident to be reported because patient injury occurred and because reflection on the detailed circumstances of the case, shared with colleagues, might serve to minimise the risk of such an occurrence happening elsewhere. However, incident reporting was found to be an unlikely option and comments revealed that this may be due to a perceived connection of reporting with guilt and error, as has been identified with other healthcare reporting initiatives, or only warranted in extreme cases.

The survey also showed that 33% of UK and 48% of Swiss chiropractors seem to work alone. In the eyes of the authors, this is limiting opportunities for fostering a safety culture through activities such as teamwork.

The authors draw the following conclusions:

• This study prompted chiropractors to reflect on aspects of clinical risk.

• Swiss and UK chiropractors tend to manage potentially risky clinical scenarios by reevaluating their care and changing their approach

• Safety incident reporting to an online system is currently an unlikely course of action, probably due to previously recognised barriers, although Swiss and UK chiropractors are positive about local communication and openness which are important tenets for safety incident reporting.

• Barriers to the use of safety incident reporting systems need to be addressed in order to encourage wider use of the existing systems.

• A significant proportion of Swiss and UK chiropractors practice in a single-handed environment. We suggest that single-handed practitioners have most to gain from participation in a national safety incident reporting and learning system.

• Female chiropractors appear to be more risk-averse than male chiropractors.

• Positivity towards key aspects of clinic safety indicate a developing safety culture within the Swiss and UK chiropractic professions.

In my view, the findings of this survey are deeply worrying and the interpretation of the authors is not far from an attempt to ‘white-wash’ the results. Like with most investigations of this nature, the results are wide open to selection bias; particularly the dismal UK response rate begs many questions. In all likelihood, reality is much worse than implied by the results of this investigation. And these results clearly show that, even with a fairly dramatic safety incident, chiropractors fail to respond adequately. There is no doubt in my mind: chiropractors put patients at risk.

If we believe homeopaths, we might get the impression that homeopathy is firmly established in mainstream health care. “They would say that, wouldn’t they?” To convince skeptics, we might want to have a bit more than wishful thinking.

We have just published a systematic review in order to instill some evidence into this debate. Our aim was to evaluate all the data from recent surveys of GPs and assess their involvement with and attitudes towards homeopathy. We searched 7 electronic databases to identify all relevant articles. Data extraction was conducted by three independent reviewers. Thirteen surveys met the inclusion criteria. Their findings suggest that less than 10% of GPs treated patients with homeopathy; referral rates varied hugely and ranged from 4.6% to 73%.

Two surveys also assessed GPs’ endorsement of homeopathy; they suggested that less than 15% of GPs were endorsing homeopathy. One survey asked about GPs’ personal usage of homeopathy and reported less than 10% had used this form of therapy on themselves. 

Three surveys investigated adverse events (AEs) from homeopathic treatments. One was solely focussed on AEs which were classified as “serious” (either life threatening or likely to cause disability or sever morbidity) or non-serious. In total, 21 “indirect” serious AEs were reported (e.g., stopping medication, refusing immunisation, refusing cancer treatment, delaying diagnosis). Another survey found that 14% of GPs reported AEs following homeopathic treatment within a year. Other authors reported that the discontinuation of conventional asthma treatment in favour of a homeopathic remedy had led to cardiovascular arrest.

These data shed a much more sober light on the use of homeopathy in the UK. They fail to show that homeopathy is well-accepted by British GPs. More importantly perhaps they disclose serious problems with the use of homeopathy.

Having previously criticised the abundance of mostly rather meaningless surveys in alternative medicine, I now should perhaps admit to having published my fair share of such investigations. The most recent one was only just published.

The aim of this survey was to conduct a follow-up of a previous, identical investigation and to  thus ascertain changes in usage, referral rate, beliefs and attitudes towards alternative medicine during the last decade. A questionnaire was posted in 2009 to all GPs registered with the Liverpool Primary Care Trust asking them whether they treat, refer, endorse or discuss eight common alternative therapies and about their views on National Health Service (NHS) funding, effectiveness, training needs and theoretical validity of each therapy. Comparisons were made between these results and those collected 10 years ago.

The response rate was unfortunately low (32%) compared with the 1999 survey (52%). The main findings were similar as 10 years before: the most popular therapies were still acupuncture, hypnotherapy and chiropractic and the least popular were aromatherapy, reflexology and medical herbalism. GPs felt most comfortable with acupuncture and had greater belief in its theoretical validity, a stronger desire for training in acupuncture and more support for acupuncture to receive NHS funding than for the other alternative therapies. Opinions about homeopathy had become less supportive during the last 10 years. Overall, GPs were less likely to endorse alternative treatments than previously shown (38% versus 19%).

I think these findings speak for themselves. They suggest that British GPs have become more skeptical about alternative medicine in general and about homeopathy in particular. It would, of course, be interesting to know why this is so. Unfortunately we are merely able to speculate here: might it be the increasingly obvious lack of evidence and biological plausibility that matter? As a rationalist, I would hope this to be true but our data do not allow any firm conclusion.

Speaking about the data, I have to admit that they are rather soft. This was just a very small survey in one specific part of the UK. More importantly, the flaws in our investigation are fairly obvious. The most important limitation probably is the low response rate. It may be caused by a general ‘survey-fatigue’ that many GPs suffer from. Whatever the reason, it severely limits the usefulness of our paper.

So why publish the survey at all then? The answer is simple: we certainly do already have an abundance of surveys, but we have a dearth of longitudinal data. Because we employed the same methodology as 10 years ago, this investigation does provide a unique insight into what might have been happening over time – albeit with more than just a pinch of salt.

Today, one day after a homeopathic retailer made headlines for advocating homeopathy as a treatment of measles, is the start of WORLD HOMEOPATHY AWARENESS WEEK. This is an ideal occasion, I think, for raising awareness of the often lamentably poor research that is being conducted in this area.

We have already on this blog discussed some rather meaningless research by Boiron, the world’s largest manufacturer of homeopathic preparations. I concluded my post by asking: “what can possibly be concluded from this article that is relevant to anyone? I did think hard about this question, and here is my considered answer: nothing (other than perhaps the suspicion that homeopathy-research is in a dire state)”. Now a new article has become available which sheds more light on those issues.

With this prospective observational study, the Boiron researchers wanted to determine the “characteristics and management of patients in France consulting allopathic general practitioners (AGPs) and homeopathic general practitioners (HGPs) for influenza-like illness (ILI)”. The investigation was conducted in Paris during the 2009-2010 influenza season. Sixty-five HGPs and 124 AGPs recruited a total of 461 patients with ILI. All the physicians and patients completed questionnaires recording demographic characteristics as well as patients’ symptoms.

Most AGPs (86%), and most patients consulting them (58%) were men; whereas most HGPs (57%), and most patients visiting them (56%) were women. Patients consulting AGPs were seen sooner after the onset of symptoms, and they self-treated more frequently with cough suppressants or expectorants. Patients visiting HGPs were seen later after the onset of symptoms and they self-treated with homeopathic medications more frequently.

At enrollment, headaches, cough, muscle/joint pain, chills/shivering, and nasal discharge/congestion were more common in patients visiting AGPs. 37.1% of all patients consulting AGPs were prescribed at least one homeopathic remedy, and 59.6% of patients visiting HGPs were prescribed at least one conventional medication. Patients’ satisfaction with their treatments did not differ between AGPs and HGPs; it was highest for the sub-group of patients who had been treated exclusively with homeopathy.

The authors draw the following conclusions from these data: In France, homeopathy is widely accepted for the treatment of ILI and does not preclude the use of allopathic medications. However, patients treated with homeopathic medications only are more satisfied with their treatment than other patients.

This  type of article, I think, falls into the category of promotion rather than science; it seems to me as though the investigation was designed not by scientists but by Boiron’s marketing team. The stated aim was to determine the “characteristics and management of patients…“, yet the thinly disguised true purpose is, I fear, to show that patients who receive homeopathic treatment are satisfied with this approach. I have previously pointed out that such findings are akin to demonstrating that people who elect to frequent a vegetarian restaurant tend to not like eating meat. Patients who want to consult a homeopath also want homeopathy; consequently they are happy when they get what they wanted. This is not rocket science, in fact, it is not science at all.

But what about the impressive acceptance of homeopathic remedies by French non-homeopathic doctors? It would, of course, be an ‘argumentum ad populum’ fallacy [which implies that ‘generally accepted’ equals ‘effective’] to assume that this proves the value of homeopathy. Yet this finding nevertheless requires an explanation: why did these doctors chose to employ homeopathy? Was it because they knew it worked? I doubt it! In my view, there are other, more plausible reasons: perhaps their patients asked for or even insisted on it; perhaps they felt that this is better than causing bacterial resistance by prescribing an antibiotic for a viral infection?

While I find this study as useless as the one I previously discussed on this blog, and while I fear that it confirms the all too often doubtful quality of research in this area, it might nevertheless contain a tiny item of interest. The authors report that “at enrollment, headaches, cough, muscle/joint pain, chills/shivering, and nasal discharge/congestion were more common in patients visiting AGPs”. In plain English, this strongly suggests that patients who decide to consult a homeopath are less ill than those who go to see a conventional doctor.

Does that mean that a certain group of individuals frequent homeopaths only when they are not really very sick? Does that indicate that even enthusiasts do not trust homeopathy all that far? Is that perhaps similar to out Royal family who seem to consult real doctors and surgeons when they are truly ill, while keeping a homeopath on stand-by for the rest of the time? These might be relevant research questions for the future; somehow I doubt, however, that the guys in charge of Boiron will ever address them.

The UK General Chiropractic Council has commissioned a survey of chiropractic patients’ views of chiropractic. Initially, 600 chiropractors were approached to recruit patients, but only 47 volunteered to participate. Eventually, 70 chiropractors consented and recruited a total of 544 patients who completed the questionnaire in 2012. The final report of this exercise has just become available.

I have to admit, I found it intensely boring. This is mainly because the questions asked avoided contentious issues. One has to dig deep to find nuggets of interest. Here are some of the findings that I thought were perhaps mildly intriguing:

15% of all patients did not receive information about possible adverse effects (AEs) of their treatment.

20% received no explanations why investigations such as X-rays were necessary and what risks they carried.

17% were not told how much their treatment would cost during the initial consultation.

38% were not informed about complaint procedures.

9% were not told about further treatment options for their condition.

18% said they were not referred to another health care professional when the condition failed to improve.

20% noted that the chiropractor did not liaise with the patient’s GP.

I think, one has to take such surveys with more than just a pinch of salt. At best, they give a vague impression of what patients believe. At worst, they are not worth the paper they are printed on.

Perhaps the most remarkable finding from the report is the unwillingness of chiropractors to co-operate with the GCC which, after all, is their regulating body. To recruit only ~10% of all UK chiropractors is more than disappointing. This low response rate will inevitably impact on the validity of the results and the conclusions.

It can be assumed that those practitioners who did volunteer are a self-selected sample and thus not representative of the UK chiropractic profession; they might be especially good, correct or obedient. This, in turn, also applies to the sample of patients recruited for this research. If that is so, the picture that emerged from the survey is likely to be be far too positive.

In any case, with a response rate of only ~10%, any survey is next to useless. I would therefore put it in the category of ‘not worth the paper it is printed on’.

 

Gravityscan Badge

Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted.


Click here for a comprehensive list of recent comments.

Categories