The SMILE Trial Lightning Process for Children with CFS: Results too good to be true?

The SMILE trial holds many anomalies and leaves us with more questions than answers.

keith'ds pouting girl

A guest post by Dr. Keith Geraghty

Honorary Research Fellow at the University of Manchester, Centre for Primary Care, Division of Population Health and Health Services Research

censored
ASA ruling left some awkward moments in Phil Parker’s videos promoting his Lightning Process.

The Advertising Standards Authority previously ruled that the Lightning Process (LP) should not be advertised as a treatment for CFS/ME. So how then, did LP end up getting tested as a treatment in a clinical trial involving adolescents with CFS/ME? Publication of the trial sparked controversy after it was claimed that LP, in addition to specialist medical care, out-performed specialist medical care alone. This blog attempts to shed light on just how a quack alternative online teaching programme, ended up in a costly clinical trial and discusses how the SMILE trial exemplifies all that is wrong with contemporary psycho-behavioural trials; that are clearly vulnerable to bias and spin.

The SMILE trial compared LP plus specialist medical care (SMC) to SMC alone (commonly a mix of cognitive behavioural therapy and graded exercise therapy). LP is a trademarked training programme created by Phil Parker from osteopathy, life coaching and neuro-linguistic programming. It costs over £600 and after assessment and telephone briefings, clients attend group sessions over three days. While there is much secrecy about what exactly these sessions involve, a cursory search online shows us that past clients were told to ‘block out all negative thoughts’ and to consider themselves well, not sick. A person with an illness is said to be ‘doing illness’ (LP spells doing as duing, to signify LP means more than just doing). LP appears to attempt to get a participant to ‘stop doing’ by blocking negative thoughts and making positive affirmations.

Leading psychologists have raised concerns. Professor James Coyne called LP “quackery” and said neuro-linguistic programming “…has been thoroughly debunked for its pseudoscience”. In an expert reaction to the SMILE trial for the Science Media Centre, Professor Dorothy Bishop of Oxford University stated: “the intervention that was assessed is commercial and associated with a number of warning signs. The Lightning Process appears based on neuro-linguistic programming, which, despite its scientific-sounding name, has long been recognised as pseudoscience“.

The first and most obvious question is why did the SMILE trial take place? Trial lead Professor Esther Crawley, who runs an NHS paediatric CFS/ME clinic, says she undertook the trial after many of her patients and their parents asked about LP. Patients with CFS/ME often report a lack of support from doctors and health care providers and some turn to the internet seeking help; some are drawn to try alternative approaches, such as LP. But is that justification enough for spending over £160,000 on testing LP on children? I think not. Should we test every quack approach peddled online: herbs, crystals, spiritual healing – particularly when funding in CFS/ME research is so limited currently? There must also be a compelling scientific plausibility to justify a trial. Simply wanting to see if something helps, does not merit adequate justification.

The SMILE trial has a fundamental design flaw. The trial compared specialist medical care alone (SMC) against SMC plus LP (SMC&LP). To the novice observer this may appear acceptable, but clinical trials are used to test item x against item y. For example, imagine trying to see which drug works better, drug A or drug B, you would not give drug A to one group and both drugs A and B to another group – yet this is exactly what happened in SMILE. In seeking to test LP, Prof. Crawley gave LP&SMC together – rendering any findings from this trial arm as pretty meaningless. The proper controls were missing. In addition, a trial of this magnitude would normally have a third arm, a do-nothing or usual care group, or another talk therapy control – yet such controls were missing.

Next we turn to the trial’s primary outcome measures. These were subjective self-reports of changes in physical function (using SF-36). Secondary outcomes were quality of life, anxiety and school attendance. These outcomes were assessed at 6 months with a follow-up at 12 months. It is reported that SMC+LP outperformed SMC alone on these measures at 6 and maintained at 12 months. However, there is no way to determine whether any claimed improvements came from LP alone, given LP was mixed with SMC. We could assume that LP+SMC meant more support, positive expectations and increased contact time. Here we see how farcical SMILE is as a trial. We have one group getting two treatments (possible double help) and one group getting one treatment (possible half help).

Of particular concern is how few of the available patients enrolled in and completed the trial: 637 children aged 12-18 attended screening or appointment at a specialist CFS/ME clinic; fewer than half (310) were deemed eligible; just 136 consented to receiving trial information and then only 100 were randomised (less than 1/3 of the eligible group). 49 had SMC and 51 had SMC+LP. Overall 207 patients either declined to participate or were not sufficiently interested to return the consent form. Were patients self-selecting? Were those less likely to respond to nonspecific factors choosing not to participate, and were we left with a group interested in LP – give Prof. Crawley said many patients asked about LP?

As the trial progressed, patients dropped out: of the 51 participants allocated to SMC+LP, only 39 received full SMC+LP. At 6-month assessment just 38 of the 48 allocated to SMC and 46 of the 51 in SMC+LP are fully recorded. At 12 months there are further losses to follow-up in both cohorts: 14% in LP and 24% in SMC.  The reasons for participant loss are not fully clear, though the paper reports 5 adverse events (3 in the SMC+LP arm). It is worth noting that physical function at 6 months deteriorated in 9 participants (roughly 10% overall), 8 in the SMC arm, with 5 participants having a fall of ≤10 on the SF-36 physical function subscale (deemed not clinically important). Again questions are raised as to whether some degree of self-selection took place? The fact 3 of the participants assigned to SMC alone appear to have received LP reflects possible contamination of research cohorts that are meant to be kept apart.

 Seven problems stand out in SMILE:

  1. The use of the SF-36 physical function test was questionable. This self-report instrument is not designed or adequately validated for use in children.
  2. Many of the participants appear to have had symptoms of anxiety and depression at the start of the trial. SMILE defined anxiety and depression as a score of ≥12 out of 22 on the self-report HADS. Usually a score of 8 or above is considered positive for mild anxiety and depression, and of above 12 for moderate anxiety and depression[1]. The average mean HADS score at trial entry was 9.6 (meaning using standard cut-offs, most participants met a criteria for anxiety and depression). On the Spence Anxiety Scale (SCAS) the average entry score was 35, with above 33 indicative of anxiety in this age group. Such mild to moderate elevations in depression and anxiety symptoms are very responsive to nonspecific support.
  3. There is an anomaly in the data on improvement: in the physical function test, the average base level of the children at entry into the trial was 54.5 (n=99), considered severely physically impaired. Only 52.5% of participants had been able to attend at least 3 days of school in the week prior to their entry into the study. Yet those assigned to SMC+LP were well enough to attend 3 consecutive days of sessions lasting 4 hours. The reports of severe physical disablement do not match the capabilities of those who participated in the course. Were the children’s self-reported poor physical abilities exaggerated to justify enrolment in the trial? Were the children’s elevated depression and anxiety symptoms responsive to the nonspecific elements in extra time of being assigned to LP plus standard care?
  4. If the subjective self-report is accepted as a recovery criterion, in LP, just 12 hours of talk therapy, added to SMC would cure the majority of children with CFS. Such an effect would be astonishing, if true. In randomized controlled trials in adults with CFS/ME, such dramatic restoration of physical function (a wholesale return to near normal) is universally not seen. The SMILE Trial is clearly unbelievable.
  5. SMILE’s reliance on the broad NICE criteria means there is a clear risk patients were included in the trial who would not have met stricter definitions of the illness. There is a growing concern that loose entry criteria in clinical trials in ME/CFS allow enrolments of many participants who do not in fact have ME/CFS. A detailed study of CFS prevalence found many children are wrongly diagnosed with CFS, when they may just be suffering from general fatigue and/or mental health complaints (Jones et al., 2004). SMILE uses NICE guidelines to diagnose CFS: fatigue must be present for at least 3 months with one or more of four other symptoms, which can be as general as sleep disturbance[2]. In contrast, Jones et al. showed that using the Centre for Disease Control criteria of at least four specific symptoms alongside detailed clinical examination, many children believed to have CFS are diagnosed with other exclusionary disorders, often general fatigue, mental health complaints, drug and alcohol abuse or eating disorders (that are often not readily disclosed to parents or doctors)[3].
  6. LP involves attempting to coerce clients into thinking that they have control over their symptoms and to block out symptoms. This alone would distort any response by a participant in a follow-on questionnaire about symptoms.
  7. LP was delivered by people from the Lightning Process Company. Phil Parker and his employees held a clear financial interest in a positive outcome in SMILE. Such an obvious conflict of interest is hard to disentangle and totally nullifies any outcomes from this trial.

Final Thoughts

The SMILE trial holds many anomalies and leaves us with more questions than answers.

It is not clear whether the children enrolled in the trial, diagnosed with CFS using NICE criteria, might of been deemed non-CFS using more stringent clinical screening (e.g. CDC or IOM Criteria).

There is no way of determining whether any effect following SMC+LP was anything more than the result of non-specific factors, psychological tricks and persuasion.

The fact LP+SMC appears to have cured the majority of participants with as little as 12 hours talk therapy is a big flashing red light that this trial is clearly fundamentally flawed.

There is a very real danger of promoting LP as a treatment for CFS/ME: The UK ME Association conducted a survey of members (4,217 members) and found that 20% of those who tried LP reported feeling worse (7.9% slightly worse,12.9% much worse). SMILE cannot be, and should not be, used to justify LP as a treatment for CFS/ME.

The Lightning Process has no scientific credibility and this trial highlights a fundamental flaw in contemporary clinical trials: they are susceptible to suggestion, bias and spin. The SMILE trial appears to draw paediatric CFS/ME clinical care for children into a swamp of pseudoscience and mysticism. This is a clear step backward. There is little to smile about after reviewing the SMILE trial.

Dr. Geraghty is currently an Honorary Research Fellow within the Centre for Primary Care, Division of Population Health and Health Services Research at the University of Manchester. He previously worked as a research associate at Cardiff University and Imperial College London. He left a career in clinical medicine after becoming ill with ME/CFS. The main themes of his work are doctor-patient relationships, medically unexplained symptoms, quality and safety in health care delivery, physician well-being and evidence-based medicine. He has a special interest in medically unexplained symptoms (MUS), and Myalgic Encephalomyelitis/Chronic Fatigue Syndrome. 

Although only recently published, his recent ‘PACE-Gate’: When clinical trial evidence meets open data access is already ranked #2 out of 1,350 papers in altmetics in Journal of Health Psychology.

A recent Times article cited Dr Geraghty on reasons why NICE need to update their recommendations for ME/CFS

Special thanks to John Peters and David Marks for their feedback.

References:
Coyne, J. (2017) Mind the Brain Blog, https://www.coyneoftherealm.com/blogs/mind-the-brain/embargo-broken-bristol-university-professor-to-discuss-trial-of-quack-chronic-fatigue-syndrome-treatment
Dorothy Bishop andExpert Commentary to the SMC (2017) http://www.sciencemediacentre.org/expert-reaction-to-controversial-treatment-for-cfsme/

1. Crawley, E., et al., Chronic disabling fatigue at age 13 and association with family adversity. Pediatrics, 2012. 130(1): p. e71-e79.
2. Crawley, E.M., et al., Clinical and cost-effectiveness of the Lightning Process in addition to specialist medical care for paediatric chronic fatigue syndrome: randomised controlled trial. Archives of Disease in Childhood, 2017.
3. Jones, J.F., et al., Chronic fatigue syndrome and other fatiguing illnesses in adolescents: a population-based study. Journal of Adolescent Health, 2004. 35(1): p. 34-40.

Embargo broken: Bristol University Professor to discuss trial of quack chronic fatigue syndrome treatment.

An alternative press briefing to compare and contrast with what is being provided by the Science Media Centre for a press conference on Wednesday September 20, 2017.

mind the brain logo

This blog post provides an alternative press briefing to compare and contrast with what was provided by the Science Media Centre for a press conference on Wednesday September 20, 2017.

The press release attached at the bottom of the post announces the publication of results of highly controversial trial that many would argue should never have occurred. The trial exposed children to an untested treatment with a quack explanation delivered by unqualified persons. Lots of money was earned from the trial by the promoters of the quack treatment beyond the boost in credibility for their quack treatment.

Note to journalists and the media: for further information email jcoynester@Gmail.com

This trial involved quackery delivered by unqualified practitioners who are otherwise untrained and insensitive to any harm to patients.

The UK Advertising Standards Authority had previously ruled that Lightning Process could not be advertised as a treatment. [ 1 ]

The Lightning is billed as mixing elements from osteopathy, life coaching and neuro-linguistic programming. That is far from having a mechanism of action based in science or evidence. [2] Neuro-linguistic programming (NLP) has been thoroughly debunked for its pseudoscientific references to brain science and ceased to be discussed in the scientific literature. [3]

Many experts would consider the trial unethical. It involved exposing children and adolescents to an unproven treatment with no prior evidence of effectiveness or safety nor any scientific basis for the mechanism by which it is claimed to work.

 As an American who has decades served on of experience with Committees for the Protection of Human Subjects and Data Safety and Monitoring Boards, I don’t understand how this trial was approved to recruit human subjects, and particularly children and adolescents.

I don’t understand why a physician who cared about her patients would seek approval to conduct such a trial.

Participation in the trial violated patients’ trust that medical settings and personnel will protect them from such risks.

Participation in the trial is time-consuming and involves loss of opportunity to obtain less risky treatment or simply not endure the inconvenience and burden of a treatment for which there is no scientific basis to expect would work.

Esther Crawley has said “If the Lightning Process is dangerous, as they say, we need to find out. They should want to find it out, not prevent research.”  I would like to see her try out that rationale in some of the patient safety and human subjects committee meetings I have attended. The response would not likely be very polite.

Patients and their parents should have been informed of an undisclosed conflict of interest.

phil parker NHSThis trial served as basis for advertising Lightning Process on the Web as being offered in NHS clinics and as being evaluated in a randomized controlled trial. [4]

Promoters of the Lightning Process received substantial payments from this trial. Although a promoter of the treatment was listed on the application for the project, she was not among the paper’s authors, so there will probably be no conflict of interest declared.

The providers were not qualified medical personnel, but were working for an organization that would financially benefit from positive findings.

It is expected that children who received the treatment as part of the trial would continue to receive it from providers who were trained and certified by promoters of the Lightning Process,

By analogy, think of a pharmaceutical trial in which the influence of drug company and that it would profit from positive results was not indicated in patient consent forms. There would be a public outcry and likely legal action.

astonishingWhy might the SMILE create the illusion that Lightning Process is effective for chronic fatigue syndrome?

There were multiple weaknesses in the trial design that would likely generate a false impression that the Lightning Process works. Under similar conditions, homeopathy and sham acupuncture appear effective [5]. Experts know to reject such results because (1) more rigorous designs are required to evaluate efficacy of treatment in order to rule out placebo effects; and (b) there must be a scientific basis for the mechanism of change claimed for how the treatment works. 

Indoctrination of parents and patients with pseudoscientific information. Advertisements for the Lightning Process on the Internet, including YouTube videos, and created a demand for this treatment among patients but it’s cost (£620) is prohibitive for many.

Selection Bias. Participation in the trial involved a 50% probability the treatment would be received for free. (Promoters of the Lightning Process received £567 for each patient who received the treatment in the trial). Parents who believed in the power of the the Lightning Process would be motived to enroll in the trial in order to obtain the treatment free for their children.

The trial was unblinded. Patients and treatment providers knew to which group patients were assigned. Not only with patients getting the Lightning Process be exposed to the providers’ positive expectations and encouragement, those assigned to the control group could register the disappointment when completing outcome measures.

The self-report subjective outcomes of this trial are susceptible to nonspecific factors (placebo effects). These include positive expectations, increased contact and support, and a rationale for what was being done, even if scientifically unsound. These nonspecific factors were concentrated in the group receiving the Lightning Process intervention. This serves to stack the deck in any evaluation of the Lightning Process and inflate differences with the patients who didn’t get into this group.

There were no objective measures of outcome. The one measure with a semblance of objectivity, school attendance, was eliminated in a pilot study. Objective measures would have provided a check on the likely exaggerated effects obtained with subjective seif-report measures.

The providers were not qualified medical, but were working for an organization that would financially benefit from positive findings. The providers were highly motivated to obtain positive results.

During treatment, the  Lightning Process further indoctrinates child and adolescent patients with pseudoscience [ 6 ] and involves coercion to fake that they are getting well [7 ]. Such coercion can interfere with the patients getting appropriate help when they need it, their establishing appropriate expectations with parental and school authorities, and even their responding honestly to outcome assessments.

 It’s not just patients and patient family members activists who object to the trial. As professionals have gotten more informed, there’s been increasing international concern about the ethics and safety of this trial.

The Science Media Centre has consistently portrayed critics of Esther Crawley’s work as being a disturbed minority of patients and patients’ family members. Smearing and vilification of patients and parents who object to the trial is unprecedented.

Particularly with the international controversy over the PACE trial of cognitive behavior therapy  and graded exercise therapy for chronic fatigue syndrome, the patients have been joined by non-patient scientists and clinicians in their concerns.

Really, if you were a fully informed parent of a child who was being pressured to participate in the trial with false claims of the potential benefits, wouldn’t you object?

embargoed news briefing

Notes

[1] “To date, neither the ASA nor CAP [Committee of Advertising Practice] has seen robust evidence for the health benefits of LP. Advertisers should take care not to make implied claims about the health benefits of the three-day course and must not refer to conditions for which medical supervision should be sought.”

[2] The respected Skeptics Dictionary offers a scathing critique of Phil Parker’s Lightning Process. The critique specifically cites concerns that Crawley’s SMILE trial switched outcomes to increase the likelihood of obtaining evidence of effectiveness.

[3] The entry for Neuro-linguistic programming (NLP) inWikipedia states:

There is no scientific evidence supporting the claims made by NLP advocates and it has been discredited as a pseudoscience by experts.[1][12] Scientific reviews state that NLP is based on outdated metaphors of how the brain works that are inconsistent with current neurological theory and contain numerous factual errors.[13][14

[4] NHS and LP    Phil Parker’s webpage announces the collaboration with Bristol University and provides a link to the officialSMILE  trial website.

{5] A provocative New England Journal of Medicine article, Active Albuterol or Placebo, Sham Acupuncture, or No Intervention in Asthma study showed that sham acupuncture as effective as an established medical treatment – an albuterol inhaler – for asthma when judged with subjective measures, but there was a large superiority for the established medical treatment obtained with objective measures.

[6] Instructional materials that patient are required to read during treatment include:

LP trains individuals to recognize when they are stimulating or triggering unhelpful physiological responses and to avoid these, using a set of standardized questions, new language patterns and physical movements with the aim of improving a more appropriate response to situations.

* Learn about the detailed science and research behind the Lightning Process and how it can help you resolve your issues.

* Start your training in recognising when you’re using your body, nervous system and specific language patterns in a damaging way

What if you could learn to reset your body’s health systems back to normal by using the well researched connection that exists between the brain and body?

The Lightning Process does this by teaching you how to spot when the PER is happening and how you can calm this response down, allowing your body to re-balance itself.

The Lightning Process will teach you how to use Neuroplasticity to break out of any destructive unconscious patterns that are keeping you stuck, and learn to use new, life and health enhancing ones instead.

The Lightning Process is a training programme which has had huge success with people who want to improve their health and wellbeing.

[7] Responsibility of patients:

Believe that Lightning Process will heal you. Tell everyone that you have been healed. Perform magic rituals like standing in circles drawn on paper with positive Keywords stated on them. Learn to render short rhyme when you feel symptoms, no matter where you are, as many times as required for the symptoms to disappear. Speak only in positive terms and think only positive thoughts. If symptoms or negative thoughts come, you must stretch forth your arms with palms facing outward and shout “Stop!” You are solely responsible for ME. You can choose to have ME. But you are free to choose a life without ME if you wish. If the method does not work, it is you who are doing something wrong.

skeptical-cat-is-fraught-with-skepticism-300x225Special thanks to the Skeptical Cat who provided me with an advance copy of the press release from Science Media Centre.