MD, PhD, MAE, FMedSci, FRSB, FRCP, FRCPEd.

education

2020 was certainly a difficult year (please note, I am trying a British understatement here). From the point of view of running this blog, it was sad to lose James Randi (1928 – 2020) who had been the hero of so many sceptics worldwide, and to learn of the passing of Frank Odds (1945-2020) who was a regular, thoughtful commentator here.

Reviewing the topics we tackled, I could mention dozens. But let me pick out just a few themes that I feel might be important.

HOMEOPATHY

Homeopathy continued to have a rough time; the German medical profession has finally realised that homeopathy is treatment with placebos and the German Green Party no longer backs homeopathy. In India, the Supreme Court ruled: Homeopathy must not be sold as a cure of Covid-19, and in the US improved labeling on homeopathic products were introduced. To make matters worse I issued A CHALLENGE FOR ALL HOMEOPATHS OF THE WORLD.

NOVEL SCAMs

On this blog, I like to write about new so-called alternative medicines (SCAMs) that I come accross. Blood letting is not exactly new, but Oh look! Bloodletting is back! Many other ‘innovations’ were equally noteworthy. Here is merely a very short selection of modalities that were new to me:

COVID-19

Unquestionably the BIG subject (not just) in SCAM was – is and will be for a while – the pandemic. It prompted quacks of any type to crawl out of the woodwork misleading the public about their offerings. On 24 January, I wrote for the first time about it: Coronavirus epidemic: Why don’t they ask the homeopaths for help? Thereafter, every charlatan seemed to jump on the COVID bandwaggon, even Trump: Trump seems to think that UV might be the answer to the corona-pandemic – could he mean “ultraviolet blood irradiation”?  It became difficult to decide who was making a greater fool of themselves, Trump or the homeopaths (Is this the crown of the Corona-idiocy? Nosodes In Prevention And Management Of COVID -19). Few SCAM entrepreneurs (Eight new products aimed at mitigating COVID-19. But do they really work?) were able to resist the opportunity. Snakeoil salesmen were out in force and view COVID-19 as an ‘opportunity’. It is impossible to calculate what impact all this COVID-quackery had, but I fear that many people lost their lives at least in part because of it.

VACCINATION

The unavoidable consequence of the pandemic was that the anti-vaxx brigade sensed that their moment had arrived. Ex-doctor Andrew Wakefield: “Better to die as a free man than live as a slave” (and get vaccinated against Covid-19). Again the ‘charlatan in chief’ made his influence felt through the ‘Trump-Effect’ on vaccination attitudes. Unsurprisingly, the UK ‘Society of Homeopath’ turned out to be an anti-vaxx hub that endangers public health. And where there is anti-vaxx, chiropractors are seldom far: Ever wondered why so many chiropractors are profoundly anti-vax?. All this could be just amusing, but sadly it has the potential to cost lives through Vaccine hesitancy due to so-called alternative medicine (SCAM).

ETHICS

I happen to believe that ethics in SCAM are an important, yet much neglected topic. It is easy to understand why this should be so: adhering to the rules of medical ethics would all but put an end to SCAM. This applies to chiropractic (The lack of chiropractic ethics: “valid consent was not obtained in a single case”), to homeopaths (Ethical homeopathy) and to most other SCAM professions. If I had a wish for the next year(s), it would be that funding agencies focus on research into the many ethical problems posed by the current popularity of SCAM.

CONCLUSION

If I had another wish, it would be that critical thinking becomes a key subject in schools, universities and adult education. Why do so many people make irrational choices? One answer to this question is, because we fail to give this subject the importance it demands. The lack of critical thinking is the reason why we elect leaders who are compulsory lyers, make wrong choices about healthcare, and continue to destroy the planet as though there is no tomorrow. It is high time that we, as a society, realise how fundamentally important critical thinking truly is.

OUTLOOK

Yes, 2020 was difficult: Brexit, COVID-19, anti-vaxx, etc. But it was not all bad (certainly not for me personally), and there is good reason for hope: the globally malign influence of Trump is about to disappear, and we now have several effective vaccines. Common sense, decency and science might triumph after all.

HEALTHY NEW YEAR EVERYONE!

I often hear that my ambitions to inform the public and inspire critical thinking are hopeless: there are simply too many quacks trumpeting nonsense, and their collective influence is surely bigger than mine. This can be depressing, of course. And because I often feel that I am fighting an unwinnable battle, stories like this are so importand and up-lifting.

Denby Royal was a ‘holistic nutritionist’, then she became a critic of so-called alternative medicine (SCAM). Here is the story about her transformation:

… I had gone to holistic nutrition school. I was running my own nutrition consulting business. And suddenly I didn’t believe in any of it anymore. How did this flip flop come to pass?

… As a holistic nutritionist, I was an active participant in what I now consider alternative medicine tomfoolery, specifically pushing supplements on a clientele of the “worried well” who often mistook wellness enthusiasts like me for medical experts. I want to be clear that I wasn’t knowingly deceiving anyone—I really did believe in the solutions I was offering my clients… To holistic nutrition enthusiasts and people who believe in a certain kind of alt wellness, these “natural” and “holistic” products seem more trustworthy than what mainstream medicine offers. The truth is, they often lack sufficient, peer-reviewed, reliable scientific evidence of their supposed effectiveness.

Did I have rock-solid evidence that these products would do what their labels promised they would do? Not really. Sure, I read studies here and there that found specific health benefits for some of the products. But I rarely mentioned the fine print (if I knew it at all)—that the sample sizes of many of these studies often were so small that the results couldn’t be generalized to a larger population, that the studies’ authors sometimes noted that more research was needed to support any findings on the effects they found, or that systematic reviews later found that many studies were poorly constructed or at risk for bias, making their findings even less compelling than they seemed initially. And in some cases, study authors themselves note that their findings are merely jumping off points, and that more long-term studies are needed in order to draw more solid conclusions…

Was I relying on strong, valid evidence? Nah, not really. But at the time, I thought what I had was better than strong evidence: Faith in a lifestyle and a dogmatic belief that all things traditional and mainstream were unhealthy or harmful, and therefore, that all things unconventional and alternative were curative and would bring about “wellness.”

In an effort to expand my product knowledge I researched a lot of the different supplements available. I was using all the best bias-confirming websites where other homeopathic medicine enthusiasts evangelized their favorite remedies, their enthusiasm and insistence, and anecdotal evidence standing in for what typically shows us that a product is safe and effective—clinical trials and FDA approval.

When their arguments and reasoning started to sink in, I realized that my faith in the healing powers of supplements may have been overzealous at best, unfounded at worst. My world crumbled like a piece of raw gluten-free paleo cheesecake. It started to sink in: Where there was a morsel of convincing medical information blended with enough compelling nonsense and communicated with enough conviction, I believed it, hook, line, and sinker.

When I started to notice the holes in the fabric of holistic nutrition, the fabric looked, well, pretty threadbare. I subsequently disconnected from social media and distanced myself from the entire culture. I took a good look at how I was personally and publicly communicating my relationships with food and wellness. After spending my twenties experimenting with all kinds of specialty diets, I was left feeling exhausted, anxious, underweight, overweight, and fed-up.

And so that last domino fell when I took away the thing that was propping it up for me: social media. Instagram is a playground for wellness influencers, including, at the time, me. My Instagram account was the best way to advertise my nutrition consulting business, so maintaining a certain persona there felt completely crucial to my success, and eventually, my identity. It was a world full of beautifully curated accounts of thin yogis gathering wild herbs in nature or making raw desserts with ingredients that cost more than my entire monthly food budget. I started to feel like the alternative wellness community I was part of—myself included—was an echo chamber, where we stockpiled likes and positive comments to build a wall that would keep out ideas that challenged our status quo. In fact, the more reassurance I received from my online community, the harder I believed in our gospel.

As I was disentangling my beliefs from everything I was learning by looking at the actual evidence, I realized that my education to become a holistic nutritionist hadn’t prepared me to understand health and wellness as completely and comprehensively as I’d once thought. Sure, I’d spent a some time studying the pathology of disease, and a little longer learning about how each bodily system works to get your human suit from point A to point B, but I am only slightly closer to being a medical professional than I am to becoming a professional cricket player. First of all, in total, my entire formal education as a holistic nutritionist was 10 months long. Second of all, that education was intended to complement—not replace—traditional medical treatment. But as soon as I finished the program, I could immediately start taking on clients. And lots of potential clients out there are just like the way I used to be—wishing they looked or felt different and in search of the panacea, willing (if not eager) to defer to an expert.

There may have been many people willing to look to me as an expert, but here’s the thing: in my school, there were no residency or clinical hours required to prepare us for the real world or to take on clients—unlike dietitians here in Canada, who must obtain a bachelor’s degree in Nutritional and Food Sciences, qualify to complete a rigorous post-degree internship program and register with a provincial dietetics organization, or get a master’s degree. We received a certificate, and that was that. It was a credential that wholeheartedly fell short of resembling anything close to making me an authority on the subject of health as it relates to food and diet. But most people in the general public can’t be expected to understand the ins and outs of how experts are credentialed and licensed—many of us assume that someone calling themselves something that we associate with authority is, in fact, an authority we can trust.

The brief education that I received to become a holistic nutritionist did provide me with valuable stepping stones and a general understanding of how the body works. My program discouraged students from saying “treat,” “heal,” “prevent,” or “cure.” Generally speaking holistic nutrition programs don’t provide the training and medical education that registered dietitians receive, which enables them to give sound, ethical medical nutrition advice, nor are they required by law, the way dietitian programs are, to provide it. In fact, in 2015 graduates of the Canadian School of Natural Nutrition were barred from identifying as Registered Holistic Nutritionists, and since then must use the title “Holistic Nutritional Consultant.”

… With what I do have from my classroom education, I can analyze a lifestyle that needs some fine-tuning and provide guidance on how to structure a solid meal plan. That’s about it. After years of self-diagnosis and hashtagging all my fad-diet escapades (for this, I greatly apologize to all those I have alienated with my profuse self-righteousness), I can at least say I have a deep appreciation for those who are actually on the front lines in the fight against unproven medical remedies and the potential damage it may do to those who use it to the exclusion of traditional medicine.

The influence of these remedies is not harmless, and I have seen firsthand in many different examples and situations how it can lure people away from real, evidence-based help in their times of need. I am fortunate enough that within my practice I had enough foresight to turn away individuals who required more guidance than I was capable of giving. But along the way I made many embarrassing and conjectural recommendations. Like I said, I was far from knowingly deceiving anyone. I firmly held the belief that alternative medicine, no matter the cost, was an investment in a healthful future. My own medicine cabinet, an arsenal full of supplements, tincture, and powders, was a personal testament to how deeply I was devoted to holistic nutrition.

This essay is a firm farewell from a world I disconnected from long ago. The person that over years I let myself become through naiveté, not doing my own research, and a misguided desire to be different. So here I am now, officially having left the church of woo, bidding the world of alternative health adieu.

_________________________________

Reading Denby’s account, I was reminded of many themes we have previously discussed on this blog. One issue that perhaps needs more focus is this notion:

I was far from knowingly deceiving anyone.”

I have not yet met a SCAM practitioner who says:

“I am in the business of  deceiving my patients.”

The reasons for this are simple:

  1. if they knowingly deceive, they would not tell us,
  2. and if they don’t know that they are deciving their patients, they cannot possibly admit to it.

The way Denby repeatedly assures us that she was far from knowingly deceiving anyone sounds charmingly naive and is, in my experience, very typical for SCAM practitioners. It depicts them as honorable people. Yet, in actual fact, it is neither charming nor honorable. It merely demonstrates the fact that they were perhaps not ruthlessly dishonest but all the more dangerous.

Let me explain this with a deliberately extreme example:

  • A man with a chronic condition – say type 2 diabetes – consults a SCAM practitioner who is knowingly deceiving him claiming that her SCAM effectively treats his condition. The patient follows the advice but, since he is not totally convinced (deception is rarely perfect), consults his doctor who puts him straight. This patient will therefore survive.
  • The same chap consults a SCAM practitioner who is deeply convinced of the effectiveness of her SCAM and thus not knowingly deceiving her patient when she claims that it is effective for his diabetes. Her conviction is so strong that the patient blindly believes her. Thus he stops his conventional medication and hopes for the best. This patient could easily die.

In a nutshell:

‘Honest’ conviction might render a quack more socially acceptable but also more dangerous to her patients.

 

On this blog we have seen just about every variation of misdemeanors by practitioners of so-called alternative medicine (SCAM). Today, I will propose a scale and rank order of these lamentable behaviours. As we regularly discuss chiropractic and homeopathy here, I decided to use these two professions as examples (but I could, of course, have chosen almost any other SCAM).

  1. Treating conditions which are not indicated: SCAM practitioners of all types are often asked by their patients to treat conditions which their particular SCAM cannot not affect. Instead of honestly saying so, they frequently apply their SCAM, wait for the natural history of the condition to do its bit, and subsequently claim that their SCAM was effective.
  2. Over-charging: asking too much money for services or goods is common (not just) in SCAM. It raises the question, what is the right price? There is, of course, no easy answer to it. Over-charging is therefore mostly a judgement call and not something absolute.
  3. Misleading a patient: there are numerous ways in which patients can be misled by their SCAM practitioners. A chiropractor who uses the Dr title, without explaining that it is not a medical title, is misleading his/her patients. A homeopath who implies that the remedy he/she is selling is a proven treatment is also misleading his/her patients.
  4. Being economical with the truth: the line between lying (see below) and being economical with the truth is often blurred. In my view, a chiropractor who does not volunteer the information that acute back pain, in most cases, resolves within a few days regardless of whether he/she mapipulates the patient’s spine or not, is economical with the truth. Similarly, a homeopath who does not explain up front that the remedy he/she prescribes does not contain a single active molecule is economical with the truth.
  5. Employing unreasonably long series of therapy: A chiropractor or homeopath, who treats a patient for months without any improvement in the patient’s condition, should suggest to call it a day. Patients should be given a treatment plan at the first consultation which includes the information when it would be reasonable to stop the SCAM.
  6. Failing to refer: A chiropractor or homeopath, who treats a patient for months without any improvement in the patient’s condition should refer the patient to another, better suited healthcare provider. Failing to do so is a serious disservice to the patient.
  7. Unethical behaviour: there are numerous ways SCAM practitioners regularly violate healthcare ethics. The most obvious one, as discussed often before on this blog, is to cut corners around informed consent. A chiropractor might, for instance, not tell his/her patient before sarting the treatment that spinal manipulation is not supported by sound evidence for efficacy or safety. A homeopathy might not explain that homeopathy is generally considered to be implausible and not evidence-based.
  8. Neglect: medical neglect occurs when patients are harmed or placed at significant risk of harm by gaps in their medical care. If a chiropractor or a homeopath, for instance, claim to be able to effectively treat asthma and fail to insist that all prescribed asthma medications must nevertheless be continued – as both often do – they are guilty of neglect, in my view. Medical neglect can be a reason for starting legal proceedings.
  9. Lying: knowingly not telling the truth can also be a serious legal issue. An example would be a chiropractor who, after beeing asked by a patient whether neck manipulation can cause harm, answers that it is an entirely safe procedure which has never injured anyone. Similarly, if a homeopaths informs his/her patient that the remedy he/she is prescribing has been extensively tested and found to be effective for the patient’s condition, he must be lying. If these practitioners believe what they tell the patient to be true, they might not technically be lying, but they would be neglecting their ethical duty to be adequately informed and they would therefore present an even greater danger to thier patients.
  10. Abuse: means to use something for the wrong purpose in a way that is harmful or morally wrong. A chiropractor who tells the mother of a healty child that they need maintenance care in order to prevent them falling ill in the future is abusing her and the child, in my view. Equally, I think that a homeopath, who homeopathically treats a disease that would otherwise be curable with conventional treatments, abuses his patient.
  11. Fraud: fraud is a legal term referring to dishonest acts that intentionally use deception to illegally deprive another person or entity of money, property, or legal rights. It relies on the use of intentional misrepresentation of fact to accomplish the taking. Arguably, most of the examples listed above are fraud by this definition.
  12. Sexual misconduct: the term refers to any behaviour which is sexual in nature and which is unwelcome and engaged in without consent. It ranges from unwanted groping to rape. There is, for instance, evidence that sexual misconduct is not a rarety in the realm of chiropractic. I have personally served once as an expert witness against a SCAM practitioner is a court case at the Exeter Crown Court.

The 12 categories listed above are not nearly as clearly defined as one would wish, and there is plenty of overlap. I am not claiming that my suggested ‘scale of misdemeanors by SCAM practitioners‘ or the proposed rank order are as yet optimal or even adequate. I am, however, hoping that readers will help me with their suggestions to improve them. Perhaps your input might then generate a scale of practical use for the future.

Who does not like a nice fragrance?

Who would object to aromatherapy?

Nobody, I suppose.

But, if its called THERAPY, we surely must ask whether it is therapeutic. And is aromatherapy therapeutic? Let’s see:

This randomized, placebo-controlled clinical trial tested whether patients with post-dural puncture headache (PDPH) caused by spinal anesthesia would benefit from aromatherapy. A total od 50 patients received 15-minute inhalations of either lavender oil or liquid paraffin as placebo. The severity of headache was scored before (baseline) and after the intervention – immediately, 30, 60, 90, and 120 minutes after – using a visual analog scale. In addition, the dosage and frequency of the pain killers as well as adverse effects of the intervention were recorded.

Both groups showed a reduction in headache scores post intervention. However, the headache scores between the groups was significantly different immediately after the intervention in favor of lavender oil (difference: 1.60 ± 0.63, P = .015). Furthermore, it was observed that the mean changes of the headache scores compared to the baseline were significant at each time interval in favor of the placebo group (P < .05), except immediately after the intervention. No significant difference was observed in Diclofenac intake between groups (P = .440), and no adverse effects were noted.

The authors concluded that aromatherapy with lavender oil was observed to reduce the severity of PDPH only immediately after the intervention, while only minimal effects were observed at successive time intervals. However, it is noted that the study was likely underpowered and further studies are recommended to better understand the effects of lavender oil on PDPH and compare its effects to other herbal products or pharmacological agents commonly used for managing headaches.

I find it laudable that some researchers conduct clinical trials even of so-called alternative medicines (SCAMs) which many of us might view as trivial. I find it more laudable that they try to do this rigorously by adding a placebo control group to the study. And I would find it even more laudable, if they did this adequately.

Considering parafin oil to be a placebo in a study of lavender oil inhalation can hardly be called adequate. Placebos are used in clinical trials mostly to account for the expectation of patients. This means that, whenever possible, patients need to be blinded to the group aloocation and the placebo must be indistinguishable from the verum. In the present trial, the patients could obviously tell the difference between the smell of lavender and the absence of any smell in the control group. Thus, their expectation could easily suffice to bring about the findings observed in the study. This means that the trial does not neccessarily demonstrate the effects of armoatherapy, but might (and most probably does) merely show the power of expectation.

How can one design such a trial more rigorously? you will ask.

There are several options. For instance, for the control group, one could use an artificial fragrance not made from natural lavender. Alternatively, one could include only patients who are unfamiliar with the smell of lavender and use a similaryly pleasant fragrance from a different plant as the control intervention.

As it stands, the study – even though aimed at testing the hypothesis that aromatherpy with lavender has specific effects on pain – tells us next to nothing.

… except, of course that it is always worth thinking very carefully about the adequate way to conduct a clinical trial.

In the last few years, several individuals in Germany have, from entirely different angles, taken a fresh look at the evidence on homeopathy and found it to be desperately wanting. Independent of each other, they published articles and books about their research and insights. Here are 5 examples:

Die Homöopathie-Lüge: So gefährlich ist die Lehre von den weißen KügelchenChristian WeymayrNicole Heißmann, 2012

In Sachen Homöopathie: Eine Beweisaufnahme, Norbert Aust, 2013

Homöopathie neu gedacht: Was Patienten wirklich hilft, Natalie Grams, 2015

Der Glaube an die Globuli: Die Verheißungen der HomöopathieNorbert SchmackeBernd Hontschik, 2015

Der wahrscheinlich teuerste Zucker der Welt: Was Sie über Homöopathie und Alternativmedizin wissen sollten, Oliver Grunau, 2017

Inevitably, these individuals came into contact with each other and subsequently founded several working-groups to discuss their concerns and coordinate their activities. Thus the INH and the Muensteraner Kreis were born. So, now we have at least three overlapping groups of enthusiastic, multidisciplinary experts who voluntarily work towards informing the German public that paying for homeopathy out of public funds is unethical, nonsensical and not in the interest of progress:

  • the GWUP,
  • the INH
  • and the Muensteraner Kreis.

No wonder then, that the German homeopathic industry and other interested parties got worried. When they realised that (presumably due to the work of these altruistic enthusiasts) the sales figures of homeopathics in Germany had, for the first time since many years, started declining, they panicked.

Their reaction was, as far as I can see, similar to their previous response to criticism: they started a media campaign in an attempt to sway public opinion. And just like before, they have taken to employing PR-people who currently spend their time defaming all individuals voicing criticism of homeopathy in Germany. Their prime targets are those experts who are most exposed to activities of responsibly informing the public about homeopathy via lectures, publications social media, etc. All of us currently receive floods of attack, insults and libellous defamations. As before (innovation does not seem to be a hallmark of homeopathy), these attacks relate to claims that:

  • we are incompetent,
  • we do not care about the welfare of patients,
  • we are habitual liars,
  • we are on the payroll of the pharmaceutical industry,
  • we aim at limiting patient choice,
  • we do what we do because we crave the limelight.

So, what is going to happen?

I cannot read tea leaves but am nevertheless sure of a few things:

  • The German homeopathy lobby will not easily give up; after all, they have half a billion Euros per year to lose.
  • They will not argue on the basis of science or evidence, because they know that neither are in their favour.
  • They will fight dirty and try to defame everyone who stands in their way.
  • They will use their political influence and their considerable financial power.

AND YET THEY WILL LOSE!

Not because we are so well organised or have great resources – in fact, as far as I can see, we have none – but because, in medicine, the evidence is invincible and will eventually prevail. Progress might be delayed, but it cannot be halted by those who cling to an obsolete dogma.

______________________________________

The above text is from a blog post I published 17 moths ago. It seems that my prediction was not wrong. Today, I saw two tweets which suggest that, in Germany, homeopathy is continuing to disappear from the realm of conventional medicine:

  1. Soeben erreicht mich die Nachricht, dass der Vorstand der Bayerischen Landesärztekammer in der Sitzung am 5.12. einstimmig beschlossen hat, die Zusatzweiterbildung Homöopathie aus der ärztlichen Weiterbildungsordnung zu entfernen. EINSTIMMIG!!!
  2. 11 von 17 Ärztekammern haben die Homöopathie gekickt. Bleiben noch 6. Die nächste Entscheidung steht in Berlin an.

I’ll translate them for you:

  1. I have just received the news that the Board of the Bavarian Medical Association has unanimously decided in the meeting on 5.12. to remove the additional training in homeopathy from the medical training regulations. UNANIMOUSLY!!!
  2. 11 of 17 medical associations have kicked homeopathy. That leaves 6. The next decision is due in Berlin.

This means that in most German counties (there is hope that, sooner or later, the other 6 will follow suit), doctors will no longer be able to train in homeopathy and use the title ‘homeopath’.

Yes, it took a while – about 200 years – but it seems that the German medical profession is finally realising that homeopathy is treatment with placebos.

Some consumers believe that research is by definition reliable, and patients are even more prone to this error. When they read or hear that ‘RESEARCH HAS SHOWN…’ or that ‘A RECENT STUDY HAS DEMONSTRATED…’, they usually trust the statements that follow. But is this trust in research and researchers justified? During 25 years that I have been involved in so-called alternative medicine (SCAM), I have encountered numerous instances which make me doubt. In this post, I will briefly discuss some the many ways in which consumers can be mislead by apparently sound evidence (for an explanation as to what is and what isn’t evidence, see here).

ABSENCE OF EVIDENCE

I have just finished reading a book by a German Heilpraktiker that is entirely dedicated to SCAM. In it, the author makes hundreds of statements and presents them as evidence-based facts. To many lay people or consumers, this will look convincing, I am sure. Yet, it has one fatal defect: the author fails to offer any real evidence that would back up his statements. The only references provided were those of other books which are equally evidence-free. This popular technique of making unsupported claims allows the author to make assertions without any checks and balances. A lay person is usually unable or unwilling to differentiate such fabulations from evidence, and this technique is thus easy and poular for misleading us about SCAM.

FAKE-EVIDENCE

On this blog, we have encountered this phenomenon ad nauseam: a commentator makes a claim and supports it with some seemingly sound evidence, often from well-respected sources. The few of us who bother to read the referenced articles quickly discover that they do not say what the commentator claimed. This method relies on the reader beeing easily bowled over by some pretend-evidence. As many consumers cannot be bothered to look beyond the smokescreen supplied by such pretenders, the method usually works surprisingly well.

An example: Vidatox is a homeopathic cancer ‘cure’ from Cuba. The Vidatox website clains that it is effective for many cancers. Considering how sensational this claim is, one would expect to find plenty of published articles on Vidatox. However, a Medline search resulted in one paper on the subject. Its authors drew the following conclusion: Our results suggest that the concentration of Vidatox used in the present study has not anti-neoplastic effects and care must be taken in hiring Vidatox in patients with HCC. 

The question one often has to ask is this: where is the line between misleading research and fraud?

SURVEYS

There is no area in healthcare that produces more surveys than SCAM. About 500 surveys are published every year! This ‘survey-mania’ has a purpose: it promotes a positive message about SCAM which hypothesis-testing research rarely does.

For a typical SCAM survey, a team of enthusiastic researchers might put together a few questions and design a questionnaire to find out what percentage of a group of individuals have tried SCAM in the past. Subsequently, the investigators might get one or two hundred responses. They then calculate simple descriptive statistics and demonstrate that xy % use SCAM. This finding eventually gets published in one of the many third-rate SCAM journals. The implication then is that, if SCAM is so popular, it must be good, and if it’s good, the public purse should pay for it. Few consumers would realise that this conclusion is little more that a fallacious appeal to popularity.

AVOIDING THE QUESTION

Another popular way of SCAM researchers to mislead the public is to avoid the research questions that matter. For instance, few experts would deny that one of the most urgent issues in chiropractic relates to the risk of spinal manipulations. One would therefore expect that a sizable proportion of the currently published chiropractic research is dedicated to it. Yet, the opposite is the case. Medline currently lists more than 3 000 papers on ‘chiropractic’, but only 17 on ‘chiropractic, harm’.

PILOT-STUDIES

A pilot study is a small scale preliminary study conducted in order to evaluate feasibility, time, cost, adverse events, and improve upon the study design prior to performance of a full-scale research project. Yet, the elementary preconditions are not fulfilled by the plethora of SCAM pilot studies that are currently being published. True pilot studies of SCAM are, in fact, very rare. The reason for the abundance of pseudo-pilots is obvious: they can easily be interpreted as showing encouragingly positive results for whatever SCAM is being tested. Subsequently, SCAM proponents can mislead the public by claiming that there are plenty of positive studies and therefore their SCAM is supported by sound evidence.

‘SAFE‘ STUDY-DESIGNS

As regularly mentioned on this blog, there are several ways to design a study such that the risk of producing a negative result is minimal. The most popular one in SCAM research is the ‘A+B versus B’ design. In this study, for instance, cancer patients who were suffering from fatigue were randomised to receive usual care or usual care plus regular acupuncture. The researchers then monitored the patients’ experience of fatigue and found that the acupuncture group did better than the control group. The effect was statistically significant, and an editorial in the journal where it was published called this evidence “compelling”. Due to a cleverly over-stated press-release, news spread fast, and the study was celebrated worldwide as a major breakthrough in cancer-care.

Imagine you have an amount of money A and your friend owns the same sum plus another amount B. Who has more money? Simple, it is, of course your friend: A+B will always be more than A [unless B is a negative amount]. For the same reason, such “pragmatic” trials will always generate positive results [unless the treatment in question does actual harm]. Treatment as usual plus acupuncture is more than treatment as usual alone, and the former is therefore more than likely to produce a better result. This will be true, even if acupuncture is a pure placebo – after all, a placebo is more than nothing, and the placebo effect will impact on the outcome, particularly if we are dealing with a highly subjective symptom such as fatigue.

A more obvious method for generating false positive results is to omit blinding. The purpose of blinding the patient, the therapist and the evaluator of the group allocation in clinical trials is to make sure that expectation is not a contributor to the result. Expectation might not move mountains, but it can certainly influence the result of a clinical trial. Patients who hope for a cure regularly do get better, even if the therapy they receive is useless, and therapists as well as evaluators of the outcomes tend to view the results through rose-tinted spectacles, if they have preconceived ideas about the experimental treatment.

Failure to randomise is another source of bias which can mislead us. If we allow patients or trialists to select or chose which patients receive the experimental and which get the control-treatment, it is likely that the two groups differ in a number of variables. Some of these variables might, in turn, impact on the outcome. If, for instance, doctors allocate their patients to the experimental and control groups, they might select those who will respond to the former and those who don’t to the latter. This may not happen with intent but through intuition or instinct: responsible health care professionals want those patients who, in their experience, have the best chances to benefit from a given treatment to receive that treatment. Only randomisation can, when done properly, make sure we are comparing comparable groups of patients. Non-randomisation can easily generate false-positive findings.

It is also possible to mislead people with studies which do not test whether an experimental treatment is superior to another one (often called superiority trials), but which assess whether it is equivalent to a therapy that is generally accepted to be effective. The idea is that, if both treatments produce similarly positive results, both must be effective.  Such trials are called non-superiority or equivalence trials, and they offer a wide range of possibilities for misleading us. If, for example, such a trial has not enough patients, it might show no difference where, in fact, there is one. Let’s consider a simple, hypothetical example: someone comes up with the idea to compare antibiotics to acupuncture as treatments of bacterial pneumonia in elderly patients. The researchers recruit 10 patients for each group, and the results reveal that, in one group, 2 patients died, while, in the other, the number was 3. The statistical tests show that the difference of just one patient is not statistically significant, and the authors therefore conclude that acupuncture is just as good for bacterial infections as antibiotics.

Even trickier is the option to under-dose the treatment given to the control group in an equivalence trial. In the above example, the investigators might subsequently recruit hundreds of patients in an attempt to overcome the criticism of their first study; they then decide to administer a sub-therapeutic dose of the antibiotic in the control group. The results would then seemingly confirm the researchers’ initial finding, namely that acupuncture is as good as the antibiotic for pneumonia. Acupuncturists might then claim that their treatment has been proven in a very large randomised clinical trial to be effective for treating this condition. People who do not happen to know the correct dose of the antibiotic could easily be fooled into believing them.

Obviously, the results would be more impressive, if the control group in an equivalence trial received a therapy which is not just ineffective but actually harmful. In such a scenario, even the most useless SCAM would appear to be effective simply because it is less harmful than the comparator.

A variation of this theme is the plethora of controlled clinical trials in SCAM which compare one unproven therapy to another unproven treatment. Perdicatbly, the results would often indicate that there is no difference in the clinical outcome experienced by the patients in the two groups. Enthusiastic SCAM researchers then tend to conclude that this proves both treatments to be equally effective. The more likely conclusion, however, is that both are equally useless.

Another technique for misleading the public is to draw conclusions which are not supported by the data. Imagine you have generated squarely negative data with a trial of homeopathy. As an enthusiast of homeopathy, you are far from happy with your own findings; in addition you might have a sponsor who puts pressure on you. What can you do? The solution is simple: you only need to highlight at least one positive message in the published article. In the case of homeopathy, you could, for instance, make a major issue about the fact that the treatment was remarkably safe and cheap: not a single patient died, most were very pleased with the treatment which was not even very expensive.

OMISSION

A further popular method for misleading the public is the outright omission findings that SCAM researchers do not like. If the aim is that the public believe the myth that all SCAM is free of side-effects, SCAM researchers only need to omit reporting them in clinical trials. On this blog, I have alerted my readers time and time again to this common phenomenon. We even assessed it in a systematic review. Sixty RCTs of chiropractic were included. Twenty-nine RCTs did not mention adverse effects at all. Sixteen RCTs reported that no adverse effects had occurred. Complete information on incidence, severity, duration, frequency and method of reporting of adverse effects was included in only one RCT.

Most trails have many outcome measures; for instance, a study of acupuncture for pain-control might quantify pain in half a dozen different ways, it might also measure the length of the treatment until pain has subsided, the amount of medication the patients took in addition to receiving acupuncture, the days off work because of pain, the partner’s impression of the patient’s health status, the quality of life of the patient, the frequency of sleep being disrupted by pain etc. If the researchers then evaluate all the results, they are likely to find that one or two of them have changed in the direction they wanted (especially, if they also include half a dozen different time points at which these variables are quatified). This can well be a chance finding: with the typical statistical tests, one in 20 outcome measures would produce a significant result purely by chance. In order to mislead us, the researchers only need to “forget” about all the negative results and focus their publication on the ones which by chance have come out as they had hoped.

FRAUD

When it come to fraud, there is more to chose from than one would have ever wished for. We and others have, for example, shown that Chinese trials of acupuncture hardly ever produce a negative finding. In other words, one does not need to read the paper, one already knows that it is positive – even more extreme: one does not need to conduct the study, one already knows the result before the research has started. This strange phenomenon indicates that something is amiss with Chinese acupuncture research. This suspicion was even confirmed by a team of Chinese scientists. In this systematic review, all randomized controlled trials (RCTs) of acupuncture published in Chinese journals were identified by a team of Chinese scientists. A total of 840 RCTs were found, including 727 RCTs comparing acupuncture with conventional treatment, 51 RCTs with no treatment controls, and 62 RCTs with sham-acupuncture controls. Among theses 840 RCTs, 838 studies (99.8%) reported positive results from primary outcomes and two trials (0.2%) reported negative results. The percentages of RCTs concealment of the information on withdraws or sample size calculations were 43.7%, 5.9%, 4.9%, 9.9%, and 1.7% respectively. The authors concluded that publication bias might be major issue in RCTs on acupuncture published in Chinese journals reported, which is related to high risk of bias. We suggest that all trials should be prospectively registered in international trial registry in future.

A survey of clinical trials in China has revealed fraudulent practice on a massive scale. China’s food and drug regulator carried out a one-year review of clinical trials. They concluded that more than 80 percent of clinical data is “fabricated“. The review evaluated data from 1,622 clinical trial programs of new pharmaceutical drugs awaiting regulator approval for mass production. Officials are now warning that further evidence malpractice could still emerge in the scandal.

I hasten to add that fraud in SCAM research is certainly not confined to China. On this blog, you will find plenty of evidence for this statement, I am sure.

CONCLUSION

Research is obviously necessary, if we want to answer the many open questions in SCAM. But sadly, not all research is reliable and much of SCAM research is misleading. Therefore, it is always necessary to be on the alert and apply all the skills of critical evaluation we can muster.

 

In so-called alternative medicine (SCAM), we have numerous diets that are either promoted for specific conditions or that are popular in a more general sense. Meat-free forms of nutrition can perhaps not be characterised as SCAM, but they are certainly popular with consumers as well as practitioners of SCAM. It is often said that meat-free diets are healthy, but few entusiasts like to consider that it might be associated with health risks.

This study tested whether vegetarianism and veganism are risk factors for bone factures in a prospective cohort with a large proportion of non-meat eaters.

In EPIC-Oxford, dietary information was collected at baseline (1993–2001) and at follow-up (≈ 2010). Participants were categorised into four diet groups at both time points (with 29,380 meat eaters, 8037 fish eaters, 15,499 vegetarians, and 1982 vegans at baseline in analyses of total fractures). Outcomes were identified through linkage to hospital records or death certificates until mid-2016. The risks were calculated of total (n = 3941) and site-specific fractures (arm, n = 566; wrist, n = 889; hip, n = 945; leg, n = 366; ankle, n = 520; other main sites, i.e. clavicle, rib, and vertebra, n = 467) by diet group over an average of 17.6 years of follow-up.

Compared with meat eaters and after adjustment for socio-economic factors, lifestyle confounders, and body mass index (BMI), the risks of hip fracture were higher in

  • fish eaters (hazard ratio 1.26; 95% CI 1.02–1.54),
  • vegetarians (1.25; 1.04–1.50),
  • vegans (2.31; 1.66–3.22).

This was equivalent to rate differences of 2.9 (0.6–5.7), 2.9 (0.9–5.2), and 14.9 (7.9–24.5) more cases for every 1000 people over 10 years, respectively. The vegans also had higher risks of total (1.43; 1.20–1.70), leg (2.05; 1.23–3.41), and other main site fractures (1.59; 1.02–2.50) than meat eaters. Overall, the significant associations appeared to be stronger without adjustment for BMI and were slightly attenuated but remained significant with additional adjustment for dietary calcium and/or total protein. No significant differences were observed in risks of wrist or ankle fractures by diet group with or without BMI adjustment, nor for arm fractures after BMI adjustment.

The authors concluded that non-meat eaters, especially vegans, had higher risks of either total or some site-specific fractures, particularly hip fractures. This is the first prospective study of diet group with both total and multiple specific fracture sites in vegetarians and vegans, and the findings suggest that bone health in vegans requires further research.

The effect is by no means large, and I doubt that it will persuade many non-meat eaters to change their diet. However, perhaps the study can serve as a reminder that people who avoid meat might some lack essential nutrients and that they should therefore consider making sure that no deficiencies can develop.

Out of the blue I received an email infroming me that Wellness consultancy and online health boutique Conscious Spaces is marking the Black Friday sale season with 12% off its hugely popular Qi tech EMF protection devices. Shop Black EMFriday at consciousspaces.com/collections/black-emfriday…

I must be a sucker for such stuff, so I had to have a look.

Most impressive!

The ‘Qi-Max Cell™ 5G / WIFI / EMF Protection For Home & Business’ for instance is for sale at £4,399.00 Sale Price (normally it costs £600 more!!!).

Naturally, I was fascinated and had to know more. Luckily, the email told me all I needed to know:

What are EMFs?

EMFs, or electromagnetic fields, are invisible fields of energy, or radiation waves. There are many different types of electromagnetic fields in the world around us. They come from both natural sources (like sunlight) and man-made sources (like mobile phones). Over the last century, exposure to man-made EMFs has been steadily increasing in line with the growing demand for electricity and the more recent explosion of wireless technology, including smart phones, laptops and tablets.

Where’s the harm?

Exposure to EMFs of the kind emitted by mobile technologies has been found to be harmful to health by a growing number independent, non-industry funded scientists and doctors. With thousands of papers, the extent of scientific research into the health impacts of EMF radiation exposure is too vast to list, but a cohesive body of evidence exists surrounding the damage caused to DNA, cells, organ systems, fertility, brain function, liver and memory.

How do Qi tech devices work?

WaveGuard’s Qi technology provides a sanctuary from EMFs by creating a protective shield of negatively charged electrons. These devices come in a variety of sizes to provide different size torus fields of protection, from the Qi-Me, for personal protection on the go, through to the Qi-Max, providing a protective field with a 50m radius.

The Qi-Me device uses the same technology as the larger Qi-Shield device which has been scientifically proven to provide EMF Protection tested using a double-blind study at the BION Institute. Priced at £399 (£350 during Black Friday), it provides a 1m radius (2m diameter) of EMF protection and is available in Walnut, Maple, Olive and Yew.

The Qi-Shield provides an EMF protection field of 2.5m radius (5m diameter). Perfect for your office, bedroom, vehicle or air travel, it is priced at £899 (£790 during Black Friday) and is available in walnut.

The Qi-Home provides the relief of being protected from harmful and damaging EMFs while at home, with an EMF protection field of 7.5m radius (15m diameter). It is priced at £2750 (£2,420 during Black Friday) it is available in Swiss pine, oak and beech.

The Qi-Max Cell is the largest and most powerful EMF protection device, creating an EMF protection field of 50m radius (100m diameter). Available in Swiss pine, it is priced at £4999 (£4,399 during Black Friday).

Tara Williams, founder of Conscious Spaces, says: ‘The calming effect I felt when I first held a Qi-Shield in a high EMF environment was a revelation. My heart rate is usually up in those sorts of settings, but this had an immediate positive effect. I now carry this or the Qi-Me with me wherever I go and have noticed a real improvement in my EHS (electromagnetic hypersensitivity) symptoms.’

________________________________________

As I said, I am most impressed by the ‘Qi-Max Cell’ (it creates an EMF protection field of 50m radius (100m diameter) in width and 35m radius (70m diameter) in height, protecting your family, workplace and business against mobile phone radiation, WiFi, electrical frequencies, electro-magnetic frequencies) and, of course by the prospect of saving £600!

But seriously! Would it not be illuminating to get such a device and take it apart to see what technology it actually contains? Or does one of my readers already know?

Misinformation by chiropractors is unfortunately nothing new and has been discussed ad nauseam on this blog. It is tempting to ask whether chiropractors have lost (or more likely never had) the ability to ditinguish real information from misinformation or substantiated from unsubstantiated claims. During the pandemic, the phenomenon of chiropractic misinformation has become even more embarrassingly obvious, as this new article highlights.

Chiropractors made statements on social media claiming that chiropractic treatment can prevent or impact COVID-19. The rationale for these claims is that spinal manipulation can impact the nervous system and thus improve immunity. These beliefs often stem from nineteenth-century chiropractic concepts. The authors of the paper are aware of no clinically relevant scientific evidence to support such statements.

The investigators explored the internet and social media to collect examples of misinformation from Europe, North America, Australia and New Zealand regarding the impact of chiropractic treatment on immune function. They discussed the potential harm resulting from these claims and explore the role of chiropractors, teaching institutions, accrediting agencies, and legislative bodies.

The authors conclude as follows: In this search of public media in Europe, North America, New Zealand, and Australia, we discovered many cases of misinformation. Claims of chiropractic treatment improving immunity conflict with the advice from authorities and the scientific consensus. The science referenced by these claims is missing, flawed or has no clinical relevance. Consequently, their claims about clinical effectiveness are spurious at best and misleading at worst. However, our examples cannot be used to make statements about the magnitude of the problem among practitioners as our samples were not intended to be representative. For that reason, we also did not include an analysis of the arguments provided in the various postings. In view of the seriousness of the topic, it would be relevant to conduct a systematic study on a representative sample of public statements, to better understand these issues. Our search illustrates the possible danger to public health of misinformation posted on social media and the internet. This situation provides an opportunity for growth and maturation for the chiropractic profession. We hope that individual chiropractors will reflect on and improve their communication and practices. Further, we hope that the chiropractic teaching institutions, regulators, and professional organisations will always demonstrate responsible leadership in their respective domains by acting to ensure that all chiropractors understand and uphold their fiduciary duties.

Several previous papers have found similar things, e.g.: Twitter activity about SMT and immunity increased during the COVID-19 crisis. Results from this work have the potential to help policy makers and others understand the impact of SMT misinformation and devise strategies to mitigate its impact.

The pandemic has crystallised the embarrassment about chiropractic false claims. Yet, the phenomenon of chiropractors misleading the public has long been known and arguably is even more important when it relates to matters other than COVID-19. Ten years ago, we published this paper:

Background: Some chiropractors and their associations claim that chiropractic is effective for conditions that lack sound supporting evidence or scientific rationale. This study therefore sought to determine the frequency of World Wide Web claims of chiropractors and their associations to treat, asthma, headache/migraine, infant colic, colic, ear infection/earache/otitis media, neck pain, whiplash (not supported by sound evidence), and lower back pain (supported by some evidence).

Methods: A review of 200 chiropractor websites and 9 chiropractic associations’ World Wide Web claims in Australia, Canada, New Zealand, the United Kingdom, and the United States was conducted between 1 October 2008 and 26 November 2008. The outcome measure was claims (either direct or indirect) regarding the eight reviewed conditions, made in the context of chiropractic treatment.

Results: We found evidence that 190 (95%) chiropractor websites made unsubstantiated claims regarding at least one of the conditions. When colic and infant colic data were collapsed into one heading, there was evidence that 76 (38%) chiropractor websites made unsubstantiated claims about all the conditions not supported by sound evidence. Fifty-six (28%) websites and 4 of the 9 (44%) associations made claims about lower back pain, whereas 179 (90%) websites and all 9 associations made unsubstantiated claims about headache/migraine. Unsubstantiated claims were made about asthma, ear infection/earache/otitis media, neck pain,

Conclusions: The majority of chiropractors and their associations in the English-speaking world seem to make therapeutic claims that are not supported by sound evidence, whilst only 28% of chiropractor websites promote lower back pain, which is supported by some evidence. We suggest the ubiquity of the unsubstantiated claims constitutes an ethical and public health issue.

It makes it clear that the misleading information of chiropractors is a serious problem. And I find it disappointing to see that so little has been done about it, and that progress seems so ellusive.

This, of course, begs the question, where does all this misinformation come from? The authors of the new paper stated that beliefs often stem from nineteenth-century chiropractic concepts. This, I believe, is very true and it gives us an important clue. It suggests that, because it is good for business, chiro schools are still steeped in obsolete notions of pseudo- and anti-science. Thus, year after year, they seem to churn out new generations of naively willing victims of the Dunning Kruger effect.

We have often heard it said on this blog and elsewhere that chiropractors are making great strides towards reforming themselves and becoming an evidence-based profession. In view of the data cited above, this does not ring all that true, I am afraid. Is the picture that emerges not one of a profession deeply embroiled in BS with but a few fighting a lost battle to clean up the act?

Vitamin D and Omega-3 supplements help the elderly avoid Covid-19 infection by boosting their immune systems, study claims. Yes, that was the headline in the DAILY MAIL on 11/11/2020. Naturally, I found this interesting. So, I looked up the original paper. Here is its abstract:

Importance: The benefits of vitamin D, omega-3 fatty acids, and exercise in disease prevention remain unclear.

Objective: To test whether vitamin D, omega-3s, and a strength-training exercise program, alone or in combination, improved 6 health outcomes among older adults.

Design, setting, and participants: Double-blind, placebo-controlled, 2 × 2 × 2 factorial randomized clinical trial among 2157 adults aged 70 years or older who had no major health events in the 5 years prior to enrollment and had sufficient mobility and good cognitive status. Patients were recruited between December 2012 and November 2014, and final follow-up was in November 2017.

Interventions: Participants were randomized to 3 years of intervention in 1 of the following 8 groups: 2000 IU/d of vitamin D3, 1 g/d of omega-3s, and a strength-training exercise program (n = 264); vitamin D3 and omega-3s (n = 265); vitamin D3 and exercise (n = 275); vitamin D3 alone (n = 272); omega-3s and exercise (n = 275); omega-3s alone (n = 269); exercise alone (n = 267); or placebo (n = 270).

Main outcomes and measures: The 6 primary outcomes were change in systolic and diastolic blood pressure (BP), Short Physical Performance Battery (SPPB), Montreal Cognitive Assessment (MoCA), and incidence rates (IRs) of nonvertebral fractures and infections over 3 years. Based on multiple comparisons of 6 primary end points, 99% confidence intervals are presented and P < .01 was required for statistical significance.

Results: Among 2157 randomized participants (mean age, 74.9 years; 61.7% women), 1900 (88%) completed the study. Median follow-up was 2.99 years. Overall, there were no statistically significant benefits of any intervention individually or in combination for the 6 end points at 3 years. For instance, the differences in mean change in systolic BP with vitamin D vs no vitamin D and with omega-3s vs no omega-3s were both -0.8 (99% CI, -2.1 to 0.5) mm Hg, with P < .13 and P < .11, respectively; the difference in mean change in diastolic BP with omega-3s vs no omega-3s was -0.5 (99% CI, -1.2 to 0.2) mm Hg; P = .06); and the difference in mean change in IR of infections with omega-3s vs no omega-3s was -0.13 (99% CI, -0.23 to -0.03), with an IR ratio of 0.89 (99% CI, 0.78-1.01; P = .02). No effects were found on the outcomes of SPPB, MoCA, and incidence of nonvertebral fractures). A total of 25 deaths were reported, with similar numbers in all treatment groups.

Conclusions and relevance: Among adults without major comorbidities aged 70 years or older, treatment with vitamin D3, omega-3s, or a strength-training exercise program did not result in statistically significant differences in improvement in systolic or diastolic blood pressure, nonvertebral fractures, physical performance, infection rates, or cognitive function. These findings do not support the effectiveness of these 3 interventions for these clinical outcomes.

Speachless?

Me too!

The study has noting to do with COVID-19 and very little with infections. The bit about infections shows almost the opposite of what the MAIL claims. So, where does the notion stipulated in the headline come from?

The MAIL article gives the answer: Professor Heike Bischoff-Ferrari from Zurich University in Switzerland, who led the latest study, said: ‘Our findings suggest supplementation of vitamin D and omega-3s in adults aged 70 or older who lead an active lifestyle and have no pre-existing conditions does not provide any benefits when it comes to bone health, memory and muscle function. ‘However, we believe there is an effect on infections – such as Covid-19.’  

I would not be surprised, if the last sentence in the quote was taken out of context.

I would not be surprised, if this is the worst health related article in the DAIL MAIL this year.

And, by Jove, there are plenty to choose from.

And why do I report all this?

As I have pointed out before, I believe that journalists have a lot to answer for when it comes to misleading the public about so-called alternative medicine (SCAM):

My hope is that, by reminding them of their ‘errors’ every now and then, I might contribute to some progress.

Yes, I know, I am an incurable optimist!

Subscribe via email

Enter your email address to receive notifications of new blog posts by email.

Recent Comments

Note that comments can be edited for up to five minutes after they are first submitted but you must tick the box: “Save my name, email, and website in this browser for the next time I comment.”

The most recent comments from all posts can be seen here.

Archives
Categories