There is little evidence that it doesn’t work

When Maggie Dunn and Maggy Wallace of the Complementary and Natural Healthcare Council spoke at Leicester Skeptics in the Pub earlier this week, I certainly got the impression that they were, at least to a certain extent, able to be swayed by rational argument.

I think they genuinely took something away from the Q&A session, yet there was one thing that they repeatedly said both in the Q&A and during our dinner beforehand that was (a) important and (b) not responded to. I wish to address this point here.

When faced with points made about the fact that there was no evidence for the claims made by a lot of the practices they regulate, their response was words to the effect of “but there is little evidence that it doesn’t work”.

The argument offered in opposition to this was simply that the onus of evidence is on the person making the claim. While I agree with this, it is more of a custom in argument rather than a valid point. However, there are reasons why this custom is observed that I believe are more influential than simply stating it.

There are two points that are implicitly made when someone points out that there is “no evidence that it doesn’t work either”:

1. That in the absence of knowledge, the probability of being right or wrong is 50/50.
2. That in the absence of knowledge, it is ethical to take a position and communicate it authoritatively.

Both of these points are incorrect.

The human body is an incredibly complex organism, and there are potentially billions of possible medical interventions, only a small handful of which are likely to work for any given ailment.

Even if we find that a given intervention is indeed useful, the probability of it being useful for any particular disease is still small. I can think of no intervention that works for most diseases.

If you were to make one reasonable and thought-through assumption about a drug’s possible effects from extensive knowledge of chemistry and biology, there is a good chance you’re going to be wrong when you apply it to the complexities of the human body.

But if you were to make an assumption based on no knowledge whatsoever, it would be highly likely that you are wrong. What’s more, the principle of Occam’s razor dictates that the chances of you being right will diminish with the number of assumptions made.

For instance, take reflexology. The first assumption is that various parts of the body are somehow connected with pathways to various parts of the foot. The second assumption is that massaging near one end of a pathway will produce an effect at the other. The third assumption is that this effect will be clinically beneficial. The forth assumption is that reflexologists have correctly mapped which positions on the foot are connected to which organs.

The likelihood of any of these single assumptions being correct in the absence of any evidence is miniscule. But for reflexology to be effective, all of these assumptions would have to be correct.

Even if we assume that the chance of each being correct is 10%, a ludicrous overestimate, then the chances of the therapy working would be a tiny 1 in 10,000.

But, for the purposes of argument, let’s imagine we live in a strange universe where the probability of any intervention being efficacious for any disease was the same as a coin toss landing heads.

Would it then be ethical to make claims of efficacy for an untested intervention?

I think not.

By making an authoritative claim that the intervention works, you are implying that you have a greater knowledge of the intervention’s efficacy than someone who is ignorant on the subject. In the mind of a person hearing your claim, the probability of efficacy will now be significantly higher than 50/50. After all, they heard it from someone presenting themselves as an expert.

If you were to ask a person who has never studied the efficacy of reflexology if it works for arthritis, the only honest answer they would be able to give would be “I’ve never studied it but in the absence of evidence it is unlikely to be effective”. For reflexologists, having never tested their treatment’s efficacy, any other reply is dishonest.

The CNHC won't be publishing my complaints

Following my complaints to the CNHC about 14 reflexologists claiming to treat specific diseases without any credible evidence, you may have noticed that nothing has yet appeared on the CNHC’s decisions page.

I contacted Maggie Dunn about this matter. It appears that because my complaints were dealt with before it got through to the conduct and competence committee, they do not intend to publish the details on the web site, nor do they intend on ever naming the offending practitioners.

I do not believe that they intend to go any further than this press release published in December 2009 congratulating themselves on a job well done.

With still no clarification from the CNHC on what practitioners are allowed to claim, it seems likely that there are many more CNHC members continuing to do exactly the same with no worry about regulation.

Maybe it’s time to submit a few more complaints.

The CNHC wishes to place on formal record their thanks to Simon Perry

When the Complementary and Natural Healthcare Council (CNHC) decided to start letting in reflexologists, I started sending in complaints about them on the same day that they joined for claiming to treat disease with magic foot massages.

The CNHC has now upheld my complaints, asked the reflexologists to stop making unjustifiable claims, and informed me of their intention to contact all registered members to issue advice to all registered practitioners.

Oh, and they formally placed on record their thanks to me for "bringing this matter to their attention".

The CNHC’s Code of Conduct, Performance and Ethics states:

15. You must follow CNHC guidelines in relation to advertising your services
Any advertising you undertake in relation to your professional activities must be accurate. Advertisements must not be misleading, false, unfair or exaggerated. You must not claim that your personal skills, equipment or facilities are better than anyone else’s.
If you are involved in advertising or promoting any other product or service, you must make sure that you use your knowledge, healthcare skills, qualifications and experience in an accurate and professionally responsible way. You must not make or support unjustifiable statements relating to particular products or services. Any potential financial rewards to you should be made explicit and play no part at all in your advice or recommendations of products and services that you give to patients, clients and users.

The key line in that paragraph is:

You must not make or support unjustifiable statements relating to particular products or services.

Now have I missed something here? The whole point of Complementary health is that you can make claims of efficacy without justification. Don’t get me wrong, I personally believe justification to be extremely important. My point is simply that the regulatory body is exclusively for practitioners who don't share this view.

So I started my little project. By searching for practitioners on the CNHC web site each day I could find out who joined. I actually wrote a computer program to do this as it takes multiple searches to get all the results. I would then manually check their web sites to see what was being claimed. If they made any unjustifiable claims, I reported them via the CNHC’s complaints procedure. I knocked these complaints out in a few minutes – quick enough to get the point across but so hastily that I failed on my first complaint to correct a copied & pasted spelling error and even added my own grammar hiccup at the end.

The ad suggests that reflexology is suitable for treating babies with colic, IBS and arthritus. She also claims to have experience in treating fertility issues. There is no reliable evidence to suggest that reflexology is capable of treating this issues.

But even with the bad English, it was sufficient to make the point: if the CNHC is going to regulate healthcare then it needs to deal with the question of efficacy.

Yesterday I received a call from the Maggie Dunn, the CNHC’s Chief Executive Officer to update me on the status of my complaints.

The CNHC had received evidence from the practitioners to defend their claims. They had also been in touch with the Advertising Standards Agency. The investigative committee met on Tuesday 24th November to discuss the cases.

They decided that my complaints were indeed covered by paragraph 15 of the code of conduct. All fourteen of my complaints were upheld. The practitioners will now be told to stop making these claims.

Sadly however, the CNHC decided that fitness to practice was not impaired because the unjustifiable claims were not made deliberately. The CNHC has a number of “Profession Specific Boards”. Someone from the board relating to reflexology had advised the investigative committee that it is likely the practitioner had been trained to believe that they could treat these diseases.

Now while I do not find it at all surprising that the practitioner was trained this way, I still take issue with this decision. If a practitioner has undergone training that tells them they can treat specific diseases for which there is no evidence then the practitioner has not undergone proper training. An untrained or poorly trained practitioner is surely unfit to practice.

What’s more positive however, is what else the CNHC now plans to do. It makes no sense for the CNHC to allow new registrants to join, only to be immediately reported by me. Instead, I’m told, the CNHC will now look at the evidence and provide advice in advance.

The evidence review will go beyond reflexology; it will extend to all of the disciplines covered by the CNHC.

The CNHC will then be contacting all registrants to explain what they are allowed, and not allowed, to claim.

I confirmed with Maggie that this advice will not solely relate to advertising. I was told that the advice will make clear that practitioners are not permitted to make these claims within the day to day interactions with their clients.

I also expressed a concern to Maggie over the training received by the practitioners. There are undoubtedly many courses out there teaching that reflexology is an efficacious treatment for many diseases. Would the CNHC be doing anything about these courses?

Maggie told me that as a regulator, the CNHC sees it as their duty to get in contact with alternative health course providers and authors. Given the nature of my original complaint, I expect this will enforce the view that claims must be justifiable.

What would a course on reflexology consisting only of justifiable claims cover exactly? How to spell reflexology?

This is so important, and so surprising I feel I need summarise in bullet points:

• CNHC will tell practitioners to remove claims they cannot justify.
• CNHC will conduct a review of evidence base for regulated therapies.
• CNHC will contact all registrants to instruct them not to make claims without justification.
• CNHC will contact complementary health course providers and authors to instruct them not to make claims without justification.

It is my view that adhering to the CNHC’s guidelines will make it impossible to practice complementary medicine.

Could this the end of the CNHC? It would be hugely ironic if forcing its members to act ethically became the cause of its demise.

Finally, Maggie told me that the investigations committee had decided that they wished to “place on formal record their thanks to Simon Perry for bringing this matter to their attention.”

You’re welcome.