It was 20 years ago today that I started my job as ’Professor of Complementary Medicine’ at the University of Exeter and became a full-time researcher of all matters related to alternative medicine. One issue that was discussed endlessly during these early days was the question whether alternative medicine can be investigated scientifically. There were many vociferous proponents of the view that it was too subtle, too individualised, too special for that and that it defied science in principle. Alternative medicine, they claimed, needed an alternative to science to be validated. I spent my time arguing the opposite, of course, and today there finally seems to be a consensus that alternative medicine can and should be submitted to scientific tests much like any other branch of health care.
Looking back at those debates, I think it is rather obvious why apologists of alternative medicine were so vehement about opposing scientific investigations: they suspected, perhaps even knew, that the results of such research would be mostly negative. Once the anti-scientists saw that they were fighting a lost battle, they changed their tune and adopted science – well sort of: they became pseudo-scientists (‘if you cannot beat them, join them’). Their aim was to prevent disaster, namely the documentation of alternative medicine’s uselessness by scientists. Meanwhile many of these ‘anti-scientists turned pseudo-scientists’ have made rather surprising careers out of their cunning role-change; professorships at respectable universities have mushroomed. Yes, pseudo-scientists have splendid prospects these days in the realm of alternative medicine.
The term ’pseudo-scientist’ as I understand it describes a person who thinks he/she knows the truth about his/her subject well before he/she has done the actual research. A pseudo-scientist is keen to understand the rules of science in order to corrupt science; he/she aims at using the tools of science not to test his/her assumptions and hypotheses, but to prove that his/her preconceived ideas were correct.
So, how does one become a top pseudo-scientist? During the last 20 years, I have observed some of the careers with interest and think I know how it is done. Here are nine lessons which, if followed rigorously, will lead to success (… oh yes, in case I again have someone thick enough to complain about me misleading my readers: THIS POST IS SLIGHTLY TONGUE IN CHEEK).
- Throw yourself into qualitative research. For instance, focus groups are a safe bet. This type of pseudo-research is not really difficult to do: you assemble about 5 -10 people, let them express their opinions, record them, extract from the diversity of views what you recognise as your own opinion and call it a ‘common theme’, write the whole thing up, and - BINGO! – you have a publication. The beauty of this approach is manifold: 1) you can repeat this exercise ad nauseam until your publication list is of respectable length; there are plenty of alternative medicine journals who will hurry to publish your pseudo-research; 2) you can manipulate your findings at will, for instance, by selecting your sample (if you recruit people outside a health food shop, for instance, and direct your group wisely, you will find everything alternative medicine journals love to print); 3) you will never produce a paper that displeases the likes of Prince Charles (this is more important than you may think: even pseudo-science needs a sponsor [or would that be a pseudo-sponsor?]).
- Conduct surveys. These are very popular and highly respected/publishable projects in alternative medicine – and they are almost as quick and easy as focus groups. Do not get deterred by the fact that thousands of very similar investigations are already available. If, for instance, there already is one describing the alternative medicine usage by leg-amputated police-men in North Devon, and you nevertheless feel the urge of going into this area, you can safely follow your instinct: do a survey of leg-amputated police men in North Devon with a medical history of diabetes. There are no limits, and as long as you conclude that your participants used a lot of alternative medicine, were very satisfied with it, did not experience any adverse effects, thought it was value for money, and would recommend it to their neighbour, you have secured another publication in an alternative medicine journal.
- If, for some reason, this should not appeal to you, how about taking a sociological, anthropological or psychological approach? How about studying, for example, the differences in worldviews, the different belief systems, the different ways of knowing, the different concepts about illness, the different expectations, the unique spiritual dimensions, the amazing views on holism – all in different cultures, settings or countries? Invariably, you will, of course, conclude that one truth is at least as good as the next. This will make you popular with all the post-modernists who use alternative medicine as a playground for getting a few publications out. This approach will allow you to travel extensively and generally have a good time. Your papers might not win you a Nobel prize, but one cannot have everything.
- It could well be that, at one stage, your boss has a serious talk with you demanding that you start doing what (in his narrow mind) constitutes ’real science’. He might be keen to get some brownie-points at the next RAE and could thus want you to actually test alternative treatments in terms of their safety and efficacy. Do not despair! Even then, there are plenty of possibilities to remain true to your pseudo-scientific principles. By now you are good at running surveys, and you could, for instance, take up your boss’ suggestion of studying the safety of your favourite alternative medicine with a survey of its users. You simply evaluate their experiences and opinions regarding adverse effects. But be careful, you are on somewhat thinner ice here; you don’t want to upset anyone by generating alarming findings. Make sure your sample is small enough for a false negative result, and that all participants are well-pleased with their alternative medicine. This might be merely a question of selecting your patients cleverly. The main thing is that your conclusion is positive. If you want to go the extra pseudo-scientific mile, mention in the discussion of your paper that your participants all felt that conventional drugs were very harmful.
- If your boss insists you tackle the daunting issue of therapeutic efficacy, there is no reason to give up pseudo-science either. You can always find patients who happened to have recovered spectacularly well from a life-threatening disease after receiving your favourite form of alternative medicine. Once you have identified such a person, you write up her experience in much detail and call it a ‘case report’. It requires a little skill to brush over the fact that the patient also had lots of conventional treatments, or that her diagnosis was assumed but never properly verified. As a pseudo-scientist, you will have to learn how to discretely make such irritating details vanish so that, in the final paper, they are no longer recognisable. Once you are familiar with this methodology, you can try to find a couple more such cases and publish them as a ‘best case series’ – I can guarantee that you will be all other pseudo-scientists’ hero!
- Your boss might point out, after you have published half a dozen such articles, that single cases are not really very conclusive. The antidote to this argument is simple: you do a large case series along the same lines. Here you can even show off your excellent statistical skills by calculating the statistical significance of the difference between the severity of the condition before the treatment and the one after it. As long as you show marked improvements, ignore all the many other factors involved in the outcome and conclude that these changes are undeniably the result of the treatment, you will be able to publish your paper without problems.
- As your boss seems to be obsessed with the RAE and all that, he might one day insist you conduct what he narrow-mindedly calls a ‘proper’ study; in other words, you might be forced to bite the bullet and learn how to plan and run an RCT. As your particular alternative therapy is not really effective, this could lead to serious embarrassment in form of a negative result, something that must be avoided at all cost. I therefore recommend you join for a few months a research group that has a proven track record in doing RCTs of utterly useless treatments without ever failing to conclude that it is highly effective. There are several of those units both in the UK and elsewhere, and their expertise is remarkable. They will teach you how to incorporate all the right design features into your study without there being the slightest risk of generating a negative result. A particularly popular solution is to conduct what they call a ‘pragmatic’ trial, I suggest you focus on this splendid innovation that never fails to produce anything but cheerfully positive findings.
- It is hardly possible that this strategy fails – but once every blue moon, all precautions turn out to be in vain, and even the most cunningly designed study of your bogus therapy might deliver a negative result. This is a challenge to any pseudo-scientist, but you can master it, provided you don’t lose your head. In such a rare case I recommend to run as many different statistical tests as you can find; chances are that one of them will nevertheless produce something vaguely positive. If even this method fails (and it hardly ever does), you can always home in on the fact that, in your efficacy study of your bogus treatment, not a single patient died. Who would be able to doubt that this is a positive outcome? Stress it clearly, select it as the main feature of your conclusions, and thus make the more disappointing findings disappear.
- Now that you are a fully-fledged pseudo-scientist who has produced one misleading or false positive result after the next, you may want a ‘proper’ confirmatory study of your pet-therapy. For this purpose run the same RCT over again, and again, and again. Eventually you want a meta-analysis of all RCTs ever published. As you are the only person who ever conducted studies on the bogus treatment in question, this should be quite easy: you pool the data of all your trials and, bob’s your uncle: a nice little summary of the totality of the data that shows beyond doubt that your therapy works. Now even your narrow-minded boss will be impressed.
These nine lessons can and should be modified to suit your particular situation, of course. Nothing here is written in stone. The one skill any pseudo-scientist must have is flexibility.
Every now and then, some smart arse is bound to attack you and claim that this is not rigorous science, that independent replications are required, that you are biased etc. etc. blah, blah, blah. Do not panic: either you ignore that person completely, or (in case there is a whole gang of nasty sceptics after you) you might just point out that:
- your work follows a new paradigm; the one of your critics is now obsolete,
- your detractors fail to understand the complexity of the subject and their comments merely reveal their ridiculous incompetence,
- your critics are less than impartial, in fact, most are bought by BIG PHARMA,
- you have a paper ‘in press’ that fully deals with all the criticism and explains how inappropriate it really is.
In closing, allow me a final word about publishing. There are hundreds of alternative medicine journals out there to chose from. They will love your papers because they are uncompromising promotional. These journals all have one thing in common: they are run by apologists of alternative medicine who abhor to read anything negative about alternative medicine. Consequently hardly a critical word about alternative medicine will ever appear in these journals. If you want to make double sure that your paper does not get criticised during the peer-review process (this would require a revision, and you don’t need extra work of that nature), you can suggest a friend for peer-reviewing it. In turn, you can offer to him/her that you do the same to him/her the next time he/she has an article to submit. This is how pseudo-scientists make sure that the body of pseudo-evidence for their pseudo-treatments is growing at a steady pace.
The main aim of our systematic review was to estimate the prevalence of use of alternative medicine (AM) in the UK. Five databases were searched for peer-reviewed surveys published between 1 January 2000 and 7 October 2011. In addition, relevant book chapters and files from our own departmental records were searched by hand. Eighty-nine surveys were included, with a total of 97,222 participants. Surely, fact that this large amount of UK surveys had emerged in only about one decade, speaks for itself.
Most studies turned out to be of poor methodological quality. Across all surveys, the average one-year prevalence of AM-use was 41.1%, and the average lifetime prevalence was 51.8%. However, many of these investigations were flimsy. According to methodologically sound surveys, the equivalent rates were 26.3% and 44%, respectively. In surveys with response rates >70%, the average one-year prevalence was nearly threefold lower than in surveys with response rates below 50%. Herbal medicine was the most popular CAM, followed by homeopathy, aromatherapy, massage and reflexology.
To the best of my knowledge, this is the first time that four crucial points about such surveys have been clearly documented:
1) The amount of surveys in AM is staggering.
2) They contribute very little worthwhile knowledge and mostly seem to be exercises in AM-promotion.
3) Their methodological quality is usually low.
4) The poor quality surveys systematically over-estimate the prevalence of AM-use.
I think it is time that AM investigators focus on real research answering important questions which advance out knowledge, that AM-journal editors stop publishing meaningless nonsense, and that decision-makers understand the difference between promotion dressed up as science and real research.
A team of Swiss and UK chiropractors just published a survey to determine which management options their colleagues would choose in response to several clinical case scenarios. In order to avoid the accusations of citing out of context, or misreporting the findings in other ways, the wording of the following post is close to the original text of the article.
The clinical scenarios refer to treatments which appear not to be successful, not indicated, possibly harmful or where a patient might be suffering from a treatment-induced complication:
Scenario 1. A patient with non-specific low back pain has not improved at all after 4–6 treatments.
Scenario 2. A patient, who has a simple neck problem with no previous long-term problems, has now improved at least 80% and stayed at this level for a couple of weeks.
Scenario 3. A patient returns from the last treatment with a new distal pain (e.g. sciatica when treated only for localized LBP, or brachialgia when treated only for local neck pain).
Scenario 4. An elderly woman complains about immediate chest pain on inspiration after manual treatment directed to her thoracic spine.
It is worth noting that scenario 4 is the most dramatic but it is by far not the worst case scenario; this would have been the case of a patient who develops signs of a stroke after neck manipulation. It is telling, I think, that this possibility has been excluded in the survey.
The following 9 management options were provided:
• I would re-evaluate the patient with a view to establishing a better diagnosis
• I would send the patient for diagnostic imaging
• I would change my treatment approach and use another technique
• I would send the patient for a second opinion to another healthcare professional but keep on monitoring their condition
• I would try a few times more
• I would encourage the patient to continue the treatment until their spine is subluxation-free
• I would stop treatment and monitor the patient regularly
• I would stop the treatment, apologise and report the event to the chiropractic reporting and learning system
• I would stop the treatment, but tell the patient that s/he is welcome to return if they feel the need
To each of these options, the chiropractors could answer by ticking: ‘never’, ‘unlikely’, ‘likely’ and ‘most likely’.
In a second part of the questionnaire the researchers assessed the chiropractors’ general attitude towards safety issues by seeking the level of agreement on a five-point scale, with the responses ‘strongly disagree’, ‘disagree’, ‘neither agree nor disagree’, ‘agree’ and ‘strongly agree’, with 23 statements relating to six different safety dimensions, as follows:
• Teamwork – helping out, relationships, respect, teamwork-emphasis
• Work pressure – rushing, overwork, staff contingent, patient numbers
• Staff training – in response to new processes, on-the-job, appropriateness of tasks
• Process and standardisation – organisation, procedures, workflow, processes
• Communication openness – ideas for improvement, alternative views, asking questions, voicing disagreement
• Patient tracking/follow-up – reminders, documentation, reports, monitoring
260 Swiss and 1258 UK chiropractors were invited to complete the questionnaire. Responses were received from 76% of the Swiss and from 31% of the UK chiropractors. The dismal response rate for UK chiropractors seems to speak volumes.
The results of this survey indicate that both Swiss and UK chiropractors tend to manage clinical scenarios where treatment appears not to be successful, not indicated, possibly harmful or where a serious complication might have occurred, by re-evaluating their care. Stopping treatment and/or incident reporting to a safety incident reporting and learning system were generally found to be unlikely courses of action. The authors believe that this unlikeliness of safety incident reporting is due to a range of recognised barriers, although Swiss and UK chiropractors are positive about local communication and openness which are important tenets for safety incident reporting. The observed positivity towards key aspects of clinic safety indicates a developing safety culture within the Swiss and UK chiropractic professions.
In this context, scenario 4 is the most dramatic and therefore the most relevant scenario -but, as noted above, not a worst case scenario. It suggested a rib fracture as a result of chiropractic manipulation, with osteoporosis as a possible risk factor. The authors state that there is a strong argument for such an incident to be reported because patient injury occurred and because reflection on the detailed circumstances of the case, shared with colleagues, might serve to minimise the risk of such an occurrence happening elsewhere. However, incident reporting was found to be an unlikely option and comments revealed that this may be due to a perceived connection of reporting with guilt and error, as has been identified with other healthcare reporting initiatives, or only warranted in extreme cases.
The survey also showed that 33% of UK and 48% of Swiss chiropractors seem to work alone. In the eyes of the authors, this is limiting opportunities for fostering a safety culture through activities such as teamwork.
The authors draw the following conclusions:
• This study prompted chiropractors to reflect on aspects of clinical risk.
• Swiss and UK chiropractors tend to manage potentially risky clinical scenarios by reevaluating their care and changing their approach
• Safety incident reporting to an online system is currently an unlikely course of action, probably due to previously recognised barriers, although Swiss and UK chiropractors are positive about local communication and openness which are important tenets for safety incident reporting.
• Barriers to the use of safety incident reporting systems need to be addressed in order to encourage wider use of the existing systems.
• A significant proportion of Swiss and UK chiropractors practice in a single-handed environment. We suggest that single-handed practitioners have most to gain from participation in a national safety incident reporting and learning system.
• Female chiropractors appear to be more risk-averse than male chiropractors.
• Positivity towards key aspects of clinic safety indicate a developing safety culture within the Swiss and UK chiropractic professions.
In my view, the findings of this survey are deeply worrying and the interpretation of the authors is not far from an attempt to ’white-wash’ the results. Like with most investigations of this nature, the results are wide open to selection bias; particularly the dismal UK response rate begs many questions. In all likelihood, reality is much worse than implied by the results of this investigation. And these results clearly show that, even with a fairly dramatic safety incident, chiropractors fail to respond adequately. There is no doubt in my mind: chiropractors put patients at risk.
If we believe homeopaths, we might get the impression that homeopathy is firmly established in mainstream health care. “They would say that, wouldn’t they?” To convince skeptics, we might want to have a bit more than wishful thinking.
We have just published a systematic review in order to instill some evidence into this debate. Our aim was to evaluate all the data from recent surveys of GPs and assess their involvement with and attitudes towards homeopathy. We searched 7 electronic databases to identify all relevant articles. Data extraction was conducted by three independent reviewers. Thirteen surveys met the inclusion criteria. Their findings suggest that less than 10% of GPs treated patients with homeopathy; referral rates varied hugely and ranged from 4.6% to 73%.
Two surveys also assessed GPs’ endorsement of homeopathy; they suggested that less than 15% of GPs were endorsing homeopathy. One survey asked about GPs’ personal usage of homeopathy and reported less than 10% had used this form of therapy on themselves.
Three surveys investigated adverse events (AEs) from homeopathic treatments. One was solely focussed on AEs which were classified as “serious” (either life threatening or likely to cause disability or sever morbidity) or non-serious. In total, 21 “indirect” serious AEs were reported (e.g., stopping medication, refusing immunisation, refusing cancer treatment, delaying diagnosis). Another survey found that 14% of GPs reported AEs following homeopathic treatment within a year. Other authors reported that the discontinuation of conventional asthma treatment in favour of a homeopathic remedy had led to cardiovascular arrest.
These data shed a much more sober light on the use of homeopathy in the UK. They fail to show that homeopathy is well-accepted by British GPs. More importantly perhaps they disclose serious problems with the use of homeopathy.
Having previously criticised the abundance of mostly rather meaningless surveys in alternative medicine, I now should perhaps admit to having published my fair share of such investigations. The most recent one was only just published.
The aim of this survey was to conduct a follow-up of a previous, identical investigation and to thus ascertain changes in usage, referral rate, beliefs and attitudes towards alternative medicine during the last decade. A questionnaire was posted in 2009 to all GPs registered with the Liverpool Primary Care Trust asking them whether they treat, refer, endorse or discuss eight common alternative therapies and about their views on National Health Service (NHS) funding, effectiveness, training needs and theoretical validity of each therapy. Comparisons were made between these results and those collected 10 years ago.
The response rate was unfortunately low (32%) compared with the 1999 survey (52%). The main findings were similar as 10 years before: the most popular therapies were still acupuncture, hypnotherapy and chiropractic and the least popular were aromatherapy, reflexology and medical herbalism. GPs felt most comfortable with acupuncture and had greater belief in its theoretical validity, a stronger desire for training in acupuncture and more support for acupuncture to receive NHS funding than for the other alternative therapies. Opinions about homeopathy had become less supportive during the last 10 years. Overall, GPs were less likely to endorse alternative treatments than previously shown (38% versus 19%).
I think these findings speak for themselves. They suggest that British GPs have become more skeptical about alternative medicine in general and about homeopathy in particular. It would, of course, be interesting to know why this is so. Unfortunately we are merely able to speculate here: might it be the increasingly obvious lack of evidence and biological plausibility that matter? As a rationalist, I would hope this to be true but our data do not allow any firm conclusion.
Speaking about the data, I have to admit that they are rather soft. This was just a very small survey in one specific part of the UK. More importantly, the flaws in our investigation are fairly obvious. The most important limitation probably is the low response rate. It may be caused by a general ‘survey-fatigue’ that many GPs suffer from. Whatever the reason, it severely limits the usefulness of our paper.
So why publish the survey at all then? The answer is simple: we certainly do already have an abundance of surveys, but we have a dearth of longitudinal data. Because we employed the same methodology as 10 years ago, this investigation does provide a unique insight into what might have been happening over time – albeit with more than just a pinch of salt.
Today, one day after a homeopathic retailer made headlines for advocating homeopathy as a treatment of measles, is the start of WORLD HOMEOPATHY AWARENESS WEEK. This is an ideal occasion, I think, for raising awareness of the often lamentably poor research that is being conducted in this area.
We have already on this blog discussed some rather meaningless research by Boiron, the world’s largest manufacturer of homeopathic preparations. I concluded my post by asking: “what can possibly be concluded from this article that is relevant to anyone? I did think hard about this question, and here is my considered answer: nothing (other than perhaps the suspicion that homeopathy-research is in a dire state)”. Now a new article has become available which sheds more light on those issues.
With this prospective observational study, the Boiron researchers wanted to determine the ”characteristics and management of patients in France consulting allopathic general practitioners (AGPs) and homeopathic general practitioners (HGPs) for influenza-like illness (ILI)”. The investigation was conducted in Paris during the 2009-2010 influenza season. Sixty-five HGPs and 124 AGPs recruited a total of 461 patients with ILI. All the physicians and patients completed questionnaires recording demographic characteristics as well as patients’ symptoms.
Most AGPs (86%), and most patients consulting them (58%) were men; whereas most HGPs (57%), and most patients visiting them (56%) were women. Patients consulting AGPs were seen sooner after the onset of symptoms, and they self-treated more frequently with cough suppressants or expectorants. Patients visiting HGPs were seen later after the onset of symptoms and they self-treated with homeopathic medications more frequently.
At enrollment, headaches, cough, muscle/joint pain, chills/shivering, and nasal discharge/congestion were more common in patients visiting AGPs. 37.1% of all patients consulting AGPs were prescribed at least one homeopathic remedy, and 59.6% of patients visiting HGPs were prescribed at least one conventional medication. Patients’ satisfaction with their treatments did not differ between AGPs and HGPs; it was highest for the sub-group of patients who had been treated exclusively with homeopathy.
The authors draw the following conclusions from these data: In France, homeopathy is widely accepted for the treatment of ILI and does not preclude the use of allopathic medications. However, patients treated with homeopathic medications only are more satisfied with their treatment than other patients.
This type of article, I think, falls into the category of promotion rather than science; it seems to me as though the investigation was designed not by scientists but by Boiron’s marketing team. The stated aim was to determine the “characteristics and management of patients…“, yet the thinly disguised true purpose is, I fear, to show that patients who receive homeopathic treatment are satisfied with this approach. I have previously pointed out that such findings are akin to demonstrating that people who elect to frequent a vegetarian restaurant tend to not like eating meat. Patients who want to consult a homeopath also want homeopathy; consequently they are happy when they get what they wanted. This is not rocket science, in fact, it is not science at all.
But what about the impressive acceptance of homeopathic remedies by French non-homeopathic doctors? It would, of course, be an ’argumentum ad populum’ fallacy [which implies that 'generally accepted' equals 'effective'] to assume that this proves the value of homeopathy. Yet this finding nevertheless requires an explanation: why did these doctors chose to employ homeopathy? Was it because they knew it worked? I doubt it! In my view, there are other, more plausible reasons: perhaps their patients asked for or even insisted on it; perhaps they felt that this is better than causing bacterial resistance by prescribing an antibiotic for a viral infection?
While I find this study as useless as the one I previously discussed on this blog, and while I fear that it confirms the all too often doubtful quality of research in this area, it might nevertheless contain a tiny item of interest. The authors report that “at enrollment, headaches, cough, muscle/joint pain, chills/shivering, and nasal discharge/congestion were more common in patients visiting AGPs”. In plain English, this strongly suggests that patients who decide to consult a homeopath are less ill than those who go to see a conventional doctor.
Does that mean that a certain group of individuals frequent homeopaths only when they are not really very sick? Does that indicate that even enthusiasts do not trust homeopathy all that far? Is that perhaps similar to out Royal family who seem to consult real doctors and surgeons when they are truly ill, while keeping a homeopath on stand-by for the rest of the time? These might be relevant research questions for the future; somehow I doubt, however, that the guys in charge of Boiron will ever address them.
The UK General Chiropractic Council has commissioned a survey of chiropractic patients’ views of chiropractic. Initially, 600 chiropractors were approached to recruit patients, but only 47 volunteered to participate. Eventually, 70 chiropractors consented and recruited a total of 544 patients who completed the questionnaire in 2012. The final report of this exercise has just become available.
I have to admit, I found it intensely boring. This is mainly because the questions asked avoided contentious issues. One has to dig deep to find nuggets of interest. Here are some of the findings that I thought were perhaps mildly intriguing:
15% of all patients did not receive information about possible adverse effects (AEs) of their treatment.
20% received no explanations why investigations such as X-rays were necessary and what risks they carried.
17% were not told how much their treatment would cost during the initial consultation.
38% were not informed about complaint procedures.
9% were not told about further treatment options for their condition.
18% said they were not referred to another health care professional when the condition failed to improve.
20% noted that the chiropractor did not liaise with the patient’s GP.
I think, one has to take such surveys with more than just a pinch of salt. At best, they give a vague impression of what patients believe. At worst, they are not worth the paper they are printed on.
Perhaps the most remarkable finding from the report is the unwillingness of chiropractors to co-operate with the GCC which, after all, is their regulating body. To recruit only ~10% of all UK chiropractors is more than disappointing. This low response rate will inevitably impact on the validity of the results and the conclusions.
It can be assumed that those practitioners who did volunteer are a self-selected sample and thus not representative of the UK chiropractic profession; they might be especially good, correct or obedient. This, in turn, also applies to the sample of patients recruited for this research. If that is so, the picture that emerged from the survey is likely to be be far too positive.
In any case, with a response rate of only ~10%, any survey is next to useless. I would therefore put it in the category of ‘not worth the paper it is printed on’.