A paper entitled ‘Real world research: a complementary method to establish the effectiveness of acupuncture’ caught my attention recently. I find it quite remarkable and think it might stimulate some discussion on this blog. Here is its abstract:
Acupuncture has been widely used in the management of a variety of diseases for thousands of years, and many relevant randomized controlled trials have been published. In recent years, many randomized controlled trials have provided controversial or less-than-convincing evidence that supports the efficacy of acupuncture. The clinical effectiveness of acupuncture in Western countries remains controversial.
Acupuncture is a complex intervention involving needling components, specific non-needling components, and generic components. Common problems that have contributed to the equivocal findings in acupuncture randomized controlled trials were imperfections regarding acupuncture treatment and inappropriate placebo/sham controls. In addition, some inherent limitations were also present in the design and implementation of current acupuncture randomized controlled trials such as weak external validity. The current designs of randomized controlled trials of acupuncture need to be further developed. In contrast to examining efficacy and adverse reaction in a “sterilized” environment in a narrowly defined population, real world research assesses the effectiveness and safety of an intervention in a much wider population in real world practice. For this reason, real world research might be a feasible and meaningful method for acupuncture assessment. Randomized controlled trials are important in verifying the efficacy of acupuncture treatment, but the authors believe that real world research, if designed and conducted appropriately, can complement randomized controlled trials to establish the effectiveness of acupuncture. Furthermore, the integrative model that can incorporate randomized controlled trial and real world research which can complement each other and potentially provide more objective and persuasive evidence.
In the article itself, the authors list seven criteria for what they consider good research into acupuncture:
- Acupuncture should be regarded as complex and individualized treatment;
- The study aim (whether to assess the efficacy of acupuncture needling or the effectiveness of acupuncture treatment) should be clearly defined and differentiated;
- Pattern identification should be clearly specified, and non-needling components should also be considered;
- The treatment protocol should have some degree of flexibility to allow for individualization;
- The placebo or sham acupuncture should be appropriate: knowing “what to avoid” and “what to mimic” in placebos/shams;
- In addition to “hard evidence”, one should consider patient-reported outcomes, economic evaluations, patient preferences and the effect of expectancy;
- The use of qualitative research (e.g., interview) to explore some missing areas (e.g., experience of practitioners and patient-practitioner relationship) in acupuncture research.
Furthermore, the authors list the advantages of their RWR-concept:
- In RWR, interventions are tailored to the patients’ specific conditions, in contrast to standardized treatment. As a result, conclusions based on RWR consider all aspects of acupuncture that affect the effectiveness.
- At an operational level, patients’ choice of the treatment(s) decreases the difficulties in recruiting and retaining patients during the data collection period.
- The study sample in RWR is much more representative of the real world situation (similar to the section of the population that receives the treatment). The study, therefore, has higher external validity.
- RWR tends to have a larger sample size and longer follow-up period than RCT, and thus is more appropriate for assessing the safety of acupuncture.
The authors make much of their notion that acupuncture is a COMPLEX INTERVENTION; specifically they claim the following: Acupuncture treatment includes three aspects: needling, specific non-needling components drove by acupuncture theory, and generic components not unique to acupuncture treatment. In addition, acupuncture treatment should be performed on the basis of the patient condition and traditional Chinese medicine (TCM) theory.
There is so much BS here that it is hard to decide where to begin refuting. As the assumption of acupuncture or other alternative therapies being COMPLEX INTERVENTIONS (and therefore exempt from rigorous tests) is highly prevalent in this field, let me try to just briefly tackle this one.
The last time I saw a patient and prescribed a drug treatment I did all of the following:
- I greeted her, asked her to sit down and tried to make her feel relaxed.
- I first had a quick chat about something trivial.
- I then asked why she had come to see me.
- I started to take notes.
- I inquired about the exact nature and the history of her problem.
- I then asked her about her general medical history, family history and her life-style.
- I also asked about any psychological problems that might relate to her symptoms.
- I then conducted a physical examination.
- Subsequently we discussed what her diagnosis might be.
- I told her what my working diagnosis was.
- I ordered a few tests to either confirm or refute it and explained them to her.
- We decided that she should come back and see me in a few days when her tests had come back.
- In order to ease her symptoms in the meanwhile, I gave her a prescription for a drug.
- We discussed this treatment, how and when she should take it, adverse effects etc.
- We also discussed other therapeutic options, in case the prescribed treatment was in any way unsatisfactory.
- I reassured her by telling her that her condition did not seem to be serious and stressed that I was confident to be able to help her.
- She left my office.
The point I am trying to make is: prescribing an entirely straight forward drug treatment is also a COMPLEX INTERVENTION. In fact, I know of no treatment that is NOT complex.
Does that mean that drugs and all other interventions are exempt from being tested in rigorous RCTs? Should we allow drug companies to adopt the RWR too? Any old placebo would pass that test and could be made to look effective using RWR. In the example above, my compassion, care and reassurance would alleviate my patient’s symptoms, even if the prescription I gave her was complete rubbish.
So why should acupuncture (or any other alternative therapy) not be tested in proper RCTs? I fear, the reason is that RCTs might show that it is not as effective as its proponents had hoped. The conclusion about the RWR is thus embarrassingly simple: proponents of alternative medicine want double standards because single standards would risk to disclose the truth.
13 Responses to ‘Real World Research’: the paper that broke my BS-detector!
Leave a Reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
The “real-world research” claim is neither novel nor unique to sCAM. I have encountered it in my own narrowly specialized field of (life-threatening) fungal diseases. Clinical trials of systemic antifungal agents tend to have patient admission criteria that dramatically narrow down the types of individual to be treated — to the point that many of my clinical colleagues complain the trials show only what happens in a very small group of people, few of whom resemble the types of patient they encounter in the “real world”. (In particular, patients who have been treated with another agent within the past couple of weeks are often excluded.)
My usual response is that it’s better to have a robust quantitative idea of (new) drug efficacy and safety in a tightly defined patient group than to have equivocal data that leave you unable to predict even vaguely how the new agent might be expected to work.
I’d make the same response to any acupuncturist who suggested “real world” trials would be a better approach to research. They’re trying to put the cart before the horse. Stimulated by this blog, I’ve troubled to look at a few of the papers purporting to be clinical research in acupuncture and other forms of witchcraft. Most of them are typically too small to be powered statistically for anything, and they include patients with diverse forms of disease characterized by approximately similar symptoms. They’re already “real world” studies, as far as they go. At least in the trials where the patients included do represent a small and well defined subset of the real world, you get a clear answer: acupuncture and its snakeoil bedfellows show no effect.
Your description of the complexity of a consultation in medicine may partially miss the point. An altmed consultation typically lasts a lot longer than one in real medicine, because the placebo effect is enhanced by the practitioner’s repeatedly emphasizing what a fascinatingly complex thing their brand of witchcraft comprises. In the manner of a poor performer in, say, music or the circus, the snakeoil salesman has to ensure that what he does looks complicated and difficult. You probably avoid burdening your patients with technical terms, explaining as much as possible very simply. The pseudomedics revel in spouting lots of “technical” terms (most of which are bollocks): impressing the patient with the nonsense is part of the cure.
‘An altmed consultation typically lasts a lot longer than one in real medicine’.
I would concede that charlatans might well embellish their procedures by drawing out a consultation but a protracted consultation certainly does not always imply charlatanism. Short consultations were, until relatively recently, the hallmark of the visit to the GP who appeared to have very little time for discussion culminating in a somewhat aggrieved patient and not infrequently an incorrect diagnosis. Thankfully this has changed and GP’s seem to me anyway to have more time to offer. Acquiring and collating information in a consultation is both an art and a skill which is honed throughout a clinical career. Some clinicians, perhaps like yourself, are more astute than others and quickly arrive at a correct diagnosis whilst others might need more time. And in taking time perhaps they appear to patients to be more thorough which no doubt enhances the placebo effect of the process.
There is obviously good reason to bash altmeds but not, I feel, because they take more time.
@Bill Kuslar
“There is obviously good reason to bash altmeds but not, I feel, because they take more time.” That’s fair enough at face value, and I can only agree that, in the UK, GP visits have often become ridiculously constrained to conform to a fixed 10- or 15-minute appointment system.
We are repeatedly told that the more extensive, one-to-one feel of consultation and genuine interest offered as part of the CAM package contributes enormously to the apparent success of the treatment: it’s a theatrical placebo effect. Clinicians who deal with real diseases as well as the “worried well” need to balance a dispassionate approach (important for minimizing inaccuracies of diagnosis) with empathy for the patient. An altmed practitioner can afford a much higher empathy component: it keeps the patient coming back — something no physician particularly wants in a state-funded healthcare system. And heavy doses of empathy require more time.
I don’t disagree with you but I do believe a degree of empathy and is required in the clinical encounter. If not then perhaps we could dissociate ourselves entirely from the face-to-face clinical encounter and ask appropriate questions via a computer terminal.
Do you mean this empathy “the ability to understand how someone feels because you can imagine what it is like to be them”? Because it can be useful in diagnostics – there are many patients who are reluctant to complain, do not know what is significant (and may focus on trivialities) etc., but it is not what alternatives use.
Oh. Complementary medicine backed up by complementary research. Sounds logical. 😉
IMO trying to bring into play the idea of complex interventions is probably a misstep on the part of the Acupuncture researcher(s). It redirects attention away from pulses/needling/moxi/herbs onto vague procedural aspects.
I don’t think Edzard’s comparison with his last consultation is entirely valid – I got all that last time I visited my GP for sure. But I experienced no empathy, no sense of “care”. The only sign of interest I saw in him was when I mentioned a practice partner who had retired early, and he made abundantly clear his own desire to not be GP-ing any longer than he had to 🙂
My point is that while all the steps in the consultation might be the same on a flowchart they aren’t necessarily subjectively comparable.
I can see what these researchers are trying to do though: since RCT are so problematic for Acupuncture they want to find another approach that might be somewhat acceptable in the direction of “evidence”.
There are several objective problems that aren’t considered though:
– different acupuncture schools take different approaches, and would undoubtedly give different treatments to the same patient at any moment.
– practitioners develop in practice, and most add new skills from time to time
– if one treats (at least in part) based on pulses, or skin tone, or iris characteristics… then one treats every patient differently. There *is* no specific treatment for e.g. eneuresis.
RCTs are so problematic because IME you can’t fake a needle. Edzard’s own try with a collapsing needle would fool nobody, and shows his lack of direct experience of acupuncture. When the needle goes in you feel it: all the way. And – let’s allow the Acu model of points/meridians etc. for a moment – when it hits the spot there is often a feeling of a strong surge of energy. If you really want to do RCT then you have to get over not only the objective problem of deciding what or who you’re treating, but also the subjective issue that it’s all-but impossible to fool the placebo recipient.
” Edzard’s own try with a collapsing needle would fool nobody, and shows his lack of direct experience of acupuncture.”
2x wrong: 1) we have properly validated out sham-needle, and acupuncture-naïve patient ARE fooled.2) I do have ‘direct experience’.
Well colour me astonished.
Do you have any references? Obv needs full protocols or whatever to know how experimenters know patients were fooled etc. (preferably not behind a paywall)
http://www.ncbi.nlm.nih.gov/pubmed/12512790
happy?
@Rich Lee
“when it [acupuncture] hits the spot there is often a feeling of a strong surge of energy.” Would that be a sensation of a prick, or pain? Why do so many people commenting on this site misuse the word ‘energy’? If I experienced a strong surge of energy with someone sticking needles into me, it would be to direct my fist in the direction of the needle sticker.
May i just offer my eleven years of monthly acupuncture visits.
I have never felt pain yet at his needles, cannot really ever feel them go in (never seen them either as i always have my eyes closed) he calls me the ‘King of Reaction’ …. generally never feel anything except release if its something tense such as tense shoulder, calf, hamstring, quad, lower back. My work is physical, I push myself hard ie self employed craftsman generally on ‘price-work’ in my own workshop, high end ‘clever’ work, theres no sick pay, no safety net.
When there is a release of something tight that has been bothering me say piriformis, the relief is immediate and enjoyable, the limb might jump and no amount of stretching or exercise could have shifted that tightness. My own pet theory is that he is ‘running to earth’ some sort of electrical charge and yes in something that is very noticably tight there is an associated flash or jolt of pain, but the relief is wonderful.
There were lots of exercises to learn in the first few years and yes I did work at them, in fact showing some to my father in law was the only time I seemed to definitely have impressed him!
I first went for tennis/ or golfer? in both elbows, in that period I would say it took a long time to solve the problem and the exercises were as much a part as the needles, this he freely admitted. i would say three months had it much reduced, half that time would be weekly, the remainder fortnightly.
What I didn’t like is how he would play with energy, up down/ up down each month… i didn’t know where the hell I was. Also on the first visit he said “I’m going to calm your mind” … that again on reflection i did not want. I am creative self employed, need to work intensely, read lots … i like being me !!
I had about five real hellish angry episodes over eleven years (on reflection now I count in another three) that I feel were a result of being heightened energy, all his responses were not adequete, the latter ones being quite awful. I could have ended up in prison!! I started to ‘join the dots’ and realised I’d rather not attend, familiarity breeds contempt the old saying goes, yes seems both sides worked that way.
Luckily I do react well to it, also for sprained pulled muscles it seems to aid fast recovery. Lots of dietary advice / lifestyle/ exercises yes he was concerned with the whole ‘me’.
But its funny how these complementaries attracts unusual people, was time for me to call a halt.
I always have the same nurse give me the infuenza vaccine. She has this wonderfully neat way of doing it absolutely painlessly. Like your acupuncture, Richard, my yearly flu shots work absolute wonders. Apart from strengthening the immune system and preventing both the flu and many other pests, I have overcome all kinds of bodily ailments during the years. I have overcome golfers elbows, shoulder impingements, sprains, broken ribs, lumbago etc. etc. Sometimes in only a few days, sometimes longer. I warmly recommend taking the flu jab every year.