There is little evidence that it doesn’t work

When Maggie Dunn and Maggy Wallace of the Complementary and Natural Healthcare Council spoke at Leicester Skeptics in the Pub earlier this week, I certainly got the impression that they were, at least to a certain extent, able to be swayed by rational argument.

I think they genuinely took something away from the Q&A session, yet there was one thing that they repeatedly said both in the Q&A and during our dinner beforehand that was (a) important and (b) not responded to. I wish to address this point here.

When faced with points made about the fact that there was no evidence for the claims made by a lot of the practices they regulate, their response was words to the effect of “but there is little evidence that it doesn’t work”.

The argument offered in opposition to this was simply that the onus of evidence is on the person making the claim. While I agree with this, it is more of a custom in argument rather than a valid point. However, there are reasons why this custom is observed that I believe are more influential than simply stating it.

There are two points that are implicitly made when someone points out that there is “no evidence that it doesn’t work either”:

1. That in the absence of knowledge, the probability of being right or wrong is 50/50.
2. That in the absence of knowledge, it is ethical to take a position and communicate it authoritatively.

Both of these points are incorrect.

The human body is an incredibly complex organism, and there are potentially billions of possible medical interventions, only a small handful of which are likely to work for any given ailment.

Even if we find that a given intervention is indeed useful, the probability of it being useful for any particular disease is still small. I can think of no intervention that works for most diseases.

If you were to make one reasonable and thought-through assumption about a drug’s possible effects from extensive knowledge of chemistry and biology, there is a good chance you’re going to be wrong when you apply it to the complexities of the human body.

But if you were to make an assumption based on no knowledge whatsoever, it would be highly likely that you are wrong. What’s more, the principle of Occam’s razor dictates that the chances of you being right will diminish with the number of assumptions made.

For instance, take reflexology. The first assumption is that various parts of the body are somehow connected with pathways to various parts of the foot. The second assumption is that massaging near one end of a pathway will produce an effect at the other. The third assumption is that this effect will be clinically beneficial. The forth assumption is that reflexologists have correctly mapped which positions on the foot are connected to which organs.

The likelihood of any of these single assumptions being correct in the absence of any evidence is miniscule. But for reflexology to be effective, all of these assumptions would have to be correct.

Even if we assume that the chance of each being correct is 10%, a ludicrous overestimate, then the chances of the therapy working would be a tiny 1 in 10,000.

But, for the purposes of argument, let’s imagine we live in a strange universe where the probability of any intervention being efficacious for any disease was the same as a coin toss landing heads.

Would it then be ethical to make claims of efficacy for an untested intervention?

I think not.

By making an authoritative claim that the intervention works, you are implying that you have a greater knowledge of the intervention’s efficacy than someone who is ignorant on the subject. In the mind of a person hearing your claim, the probability of efficacy will now be significantly higher than 50/50. After all, they heard it from someone presenting themselves as an expert.

If you were to ask a person who has never studied the efficacy of reflexology if it works for arthritis, the only honest answer they would be able to give would be “I’ve never studied it but in the absence of evidence it is unlikely to be effective”. For reflexologists, having never tested their treatment’s efficacy, any other reply is dishonest.

At Boots, it's 3 for the price of 2 on quackery

At 2pm on the 14th July, Skeptic Kash Farooq alerted me via Twitter to a quack medicine product on the Boots web site that he was going to make an Advertising Standards Authority complaint about.

The only problem was that the ASA does not regulate web site content. And this has been a problem stopping us from tackling Boots for some time. If Boots has false or unjustifiable claims to make about a product, they only seem to make them in areas where the ASA can’t touch them. They use packaging, point of sale materials and their web site to make their claims – all outside of the ASA’s remit. I’ve personally never seen widespread false claims made by Boots on posters, leaflets or in the press where the ASA can start issuing adjudications against them.

Whether this is a deliberate strategy by Boots, or just through chance alone I cannot be sure. But yesterday, they slipped up.

Kash had noticed that Boots had a 3 for the price of 2 offer “across all vitamins, complementary medicines and herbal products”. There are a couple of exceptions where the ASA will regulate claims made on the web. One is:

“We regulate sales promotions, such as special offers, prize draws and competitions wherever they appear.”

Boots appeared to have put their entire range of alternative health products – the products for which they regularly make unjustified claims of health benefits – fully within the remit of the Advertising Standards Authority.

I had no idea of how long this promotion would last, and with the ASA sometimes taking over a week to look at a case, I did not want the claims to fall out of remit before my complaint went in. Boots may have seen the tweets about them and realised they needed to withdraw the promotion. For the best chance of success, the complaint had to go in by the start of business the next morning.

I started going through the claims and realised that there was no possible way I could get through them by myself. Boots had 679 products in the range, many of which were making clearly unjustifiable claims. And in comes the power of Twitter. With a couple of Tweets, I suddenly had a small army of helpers.

I created a shared Google Spreadsheet in which a team of 9 or 10 people started adding URLs from the Boots web site and copying and pasting next to them the unjustifiable claims made about the product. With a little help from technical wizard @tommorris answering my call for help, I found a program that would automatically download the large number of web pages and print them to a local PDF to hold as evidence.

Watching what was happening on the Google Spreadsheet was awe-inspiring. When I started letting people into the document, there were 80 URLs copied and pasted into the list. By the time I got 15 more URLs into it, @the_beacon, @richardtomsett, @HelenaThomas, @dellybean, @kashfarooq, @nwoolhouseuk, @cherryblack, @RoisinThomas and @kingmuskar had pretty much copied and pasted all of the claims and were now waiting on me.

By the end of the evening, we’d sent off complaints to the Advertising Standards Authority about 107 Boots products.

The claims they were making varied from shocking – claiming that a homeopathic remedy is “to relieve the pain of teething.” To the bizarre – a magnet which you put in your knickers which they claimed “helps to reduce or completely eliminate the symptoms of menopause” – something one of my helpers described as a “Fanny Magnet”. There were some less serious claims such as listing “30c Aconitum napellus” as an active ingredient on a product when I can say with 99.999999999999999999999999999999999994% certainty that if manufactured carefully contains no Aconitum napellus (and I worked that number out, it’s not just a guess).

But my helpers continued after I finished. @nwoolhouseuk, @ScepticLetters,
@GDLockUK, @kashfarooq and @the_beacon together sent in a second complaint with another 133 products listed. @nwoolhouseuk was still going at 1:30 in the morning, and @ScepticLetters finally finished it off at 4am.

Boots will now hopefully be held to account. For years, whether accidentally or by design, they been keeping the misleading claims they make about their products just beyond the remit of the ASA. One slip up, and with excellent teamwork we caught them out in one night with a total of 240 complaints.

Letters to the Gideons

I did my Skeptical Activism & The Quacklash talk at Westminster Skeptics in the pub last night. My opening gag for the talk is related to a letter I wrote some years ago.

Several years ago I was inspired by the hilarious Timewaster letters and went through a period of writing daft letters to various organisations for no other reason than my own personal amusement. Last night, someone suggested I should publish these letters on my blog. While I don't want to ruin my opening gag by publishing the one I refer to in the talk, here's a series of letters I wrote to The Gideons, the people that leave Bibles in hotel rooms.