ABSTRACT:
OBJECTIVE: Acceptance and commitment therapy (ACT) is a third-wave psychological intervention
that has attracted considerable clinical and research attention. A previous meta-analysis
of ACT trials in psychosis reported a large effect size of ACT against overall psychotic
symptomatology. However, there were critical methodological issues in the review that
justify replication.
METHODS: Systematic review and meta-analysis of randomized controlled trials (RCTs) testing
ACT vs. any comparator condition in a sample of adults with psychosis. The outcome
of interest was overall psychotic symptomatology.
RESULTS: The search identified seven published and eight unpublished trials (of which we were
able to obtain data from one). Data on symptomatology were extracted from six trials
that involved 274 participants. The summary effect size (Hedge's G) for overall symptomatology
was small and not significant (-0.21, 95%CI -0.60-0.18). Trials were generally rated
as having a high risk of bias. Safety reporting was inadequate across included trials.
CONCLUSIONS: Our observed effect size contrasted with that reported in a previous meta-analysis;
differences were likely explained by errors in data extraction. The findings of this
review suggest that there is currently inadequate evidence to conclude that ACT is
a safe and effective treatment against psychotic symptomatology.
SYSTEMATIC REVIEW REGISTRATION: CRD42018097200
Keywords: acceptance and commitment therapy; ACT; systematic review; meta-analysis; randomized controlled trial; RCT.
FIGURES
| Received: March 8 2020; Accepted: April 22 2020 |
INTRODUCTION
There has been considerable interest in acceptance and commitment therapy (ACT) in the treatment of various psychiatric disorders, including psychosis.1 Belonging to the third wave of cognitive behavioral therapy (CBT),2 ACT was first defined and described by Hayes et al.,2 who set out six basic principles - cognitive diffusion, expansion and acceptance, contact and connection with the present moment, the observing self, values clarification, and committed action - that work in conjunction with one another towards the overarching goal of effectively handling painful thoughts and experiences and creating a vibrant, vital life.3 ACT has been manualized as a specific treatment for psychosis.4 However, whether it is safe and effective for this purpose remains unclear. To date, there have been two systematic reviews5,6 and two systematic reviews and meta-analysis7,8 of ACT for psychosis.
Ost5 reported a review and meta-analysis of 60 ACT trials, involving 4,234 participants across a range of mental and physical disorders (as well as stress at work). Of note, four trials testing ACT against psychotic symptoms were included.9-12 A small effect size (0.42) across all conditions was reported. However, Ost5 did not report a meta-analysis of ACT against psychotic symptoms, and notes that only one trial9 reported a significant difference in favor of ACT. Methodological issues with the included trials were identified, leading the authors to conclude that ACT was possibly effective for psychotic symptoms. This review has been heavily criticized13 for potential bias and lack of methodological rigor; for example, screening, data extraction, and risk of bias analysis (using a measure developed by the review author) was undertaken by only one researcher.5
Wakefield et al.6 reported a systematic review - but not a meta-analysis - of ACT for psychosis. The authors included 13 controlled and uncontrolled trials and concluded that ACT was beneficial for patients with first-episode or chronic psychosis. However, the authors included papers that reported reanalyses of combined datasets. For example, Bach et al.14 examined the mediating role of symptom believability on hospitalization outcomes by combining datasets from two previous trials.9,10 As a consequence, Wakefield et al.6 conflated the number of ACT studies because the same datasets were included more than once.
Two meta-analyses of the safety and efficacy of acceptance and mindfulness-based interventions for psychosis have been reported.7,15 Neither review was pre-registered. Authors of both meta-analyses included trials that reported against several different outcomes, including overall symptomatology, positive/negative symptoms, hospitalization/days in hospital, depressive symptoms, social functioning, mindfulness skills, and acceptance. As a consequence, there is a risk of selective outcome reporting bias, with preference given to outcomes with significant associations. Four ACT trials9-12 were included in the original review7 and two10,11 in their meta-analysis. A small and non-significant effect size (-0.16) was reported against psychotic symptomatology. Low risk of bias was reported for the included studies. Jansen et al.15 identified two additional trials (for a total of six) that evaluated ACT in people with psychosis. Their meta-analysis of ACT against overall symptomatology reported a large effect size (-1.07).15 The effect size for one of the included trials in particular16 is considerable (-4.45) and may be incorrect (we wrote to the editor of Schizophrenia Research on March 9, 2020, informing them of this potential error). Seemingly, the authors extracted the standard error rather than the standard deviation (SD), and used this in their meta-analysis. The authors conclude that ACT is a safe intervention and state that no study reported any serious adverse events that were directly related to the intervention. Safety data were only reported in two10,12 of seven included ACT trials. Using the Cochrane risk of bias tool,17 the authors again concluded that the studies had a low risk of bias.15
To date, there have been four previous systematic reviews of ACT against psychotic symptomatology that generally affirm the use of ACT in this population. As we have discussed, there are potential errors in two6,15 and a lack of methodological rigor in one.5 There is a pressing need for a rigorous systematic review and meta-analysis testing the safety and efficacy of ACT against psychosis symptoms.
METHODS
Review design
This was a systematic review and meta-analysis of randomized controlled trials (RCT) of ACT in the treatment of patients diagnosed with psychosis. The main outcome was psychotic symptomatology at the end of treatment; we note that while this outcome may be dissonant with the intended aims of ACT, it is the primary measure used in previous reviews.5 We focused on symptoms at the end of treatment, as this was more likely to have been reported consistently in included studies. It was anticipated that there might be considerable variation in timings of follow-up measures, making these difficult to include in a meta-analysis. We have published a protocol18 for this review, which includes consideration of the issues related to outcome selection and a detailed description of our methodology. The review was registered on PROSPERO on May 21, 2018 (registration number: CRD42018097200; web address:
https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=97200). This report complies with Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.19
Inclusion and exclusion criteria
Trials were included that:
- Compared full-length, therapist-delivered ACT, regardless of medium (e.g., individual, group, telephone, online), with any comparator condition, using a randomized controlled design;
- Included adult participants (age 18 years or older) with a diagnosis of schizophrenia or other psychotic disorder determined using any standard diagnostic criteria;
- Reported psychotic symptomatology as an outcome;
- Were written in English and published after 1999 (the year ACT was first described).
- Trials were excluded if:
- Participants had developmental impairment, intellectual disability, affective psychosis, organic psychosis, primary diagnosis of major depression with psychotic symptoms, primary drug or alcohol addiction, or any clinically significant medical diseases;
- Authors reported a follow-up or extension of a previously published trial (to avoid the risk of counting the same participants twice).
Data sources and search strategy
The following databases were searched for eligible studies: Cochrane Central Register of Controlled Trials (CENTRAL), Education Resources Information Center (ERIC), Excerpta Medica Database (Embase), MEDLINE via Ovid, and PsycINFO via Ovid. The following search strategy was employed (keywords are italicized; subject headings are emboldened; Boolean operators are capitalized): 'acceptance commitment therapy' OR act OR 'clinic* behavi?r analy*', psychos?s OR 'psychotic disorder*' OR schizophrenia, exp Cognitive Therapy/, exp Psychotic Disorders/ OR exp Schizophrenia/, 1 OR 3, 2 OR 4, 5 AND 6, and limit 7 to RCT.
Schizophrenia Research on subject headings in a highly sensitive syntax. The search strategy was initially executed on May 20, 2018 and updated on February 23, 2020.
As possible publication bias has been noted in some reviews of ACT research,5 we sought to identify unpublished trials. This may provide critical insight into the current state of the evidence, as publication bias typically exaggerates the magnitude of the treatment effect.20 On May 21, 2018, we searched the following trial registries with psychosis and schizophrenia as condition and acceptance and commitment therapy and ACT as intervention: ClinicalTrials.gov, the Australian New Zealand Clinical Trials Registry (ANZCTR), and the International Standard Randomized Controlled Trial Number (ISRCTN) Registry.
Review management software
We used the online Covidence software package to manage title and abstract and full-text screening.
Screening
Titles and abstracts of studies retrieved using the search strategy were screened independently by two researchers to identify trials that met the inclusion/exclusion criteria. A third researcher resolved any disagreements. Full texts of these potentially eligible studies were retrieved and independently assessed for inclusion by two researchers, with any further disagreements resolved by the third researcher.
Data extraction
We extracted the following data from included studies: study citation, country, setting, dose of intervention, delivery medium of intervention, primary endpoint, number of participants allocated to intervention and control groups, number of analyzed participants allocated to intervention and control groups, calculation of sample size, pre- and post-intervention/control summary statistics, harms and registration status (an International Committee of Medical Journal Editors [ICMJE] requirement for trials that started recruiting after July 200521), and country of origin. Extracted data were entered directly into a spreadsheet. We calculated effect sizes using any reported, post-treatment standard measure of psychotic pathology that included, but was not limited to, the Positive and Negative Syndrome Scale (PANSS)22 and the Brief Psychiatric Rating Scale (BPRS),23 even where this was not the stated primary outcome. Data on hospitalization as a surrogate marker of psychotic symptoms were also extracted.
Risk of bias
Study quality was determined using version 2 of the Cochrane risk of bias tool (RoB 2).24 The updated tool was produced because of developments in the understanding of bias in RCTs and feedback from researchers that used version 1 of the measure. Bias was rated high, low, or some concerns, in five domains: 1) Risk of bias arising from the randomization process; 2) Risk of bias due to deviations from the intended interventions (effect of assignment to intervention and effect of adhering to intervention); 3) Missing outcome data; 4) Risk of bias in the measurement of the outcome; 5) Risk of bias in the selection of the reported result. The overall default rating of risk of bias is high risk if one of the domains is rated high. Ratings were completed by two researchers, with discrepancies resolved by a third.
Data synthesis
We used Review Manager version 5 (http://community.cochrane.org/help/tools-and-software/revman-5/revman-5-download) to compute the meta-analyses. The most common summary statistic across studies was the standardized mean difference. These were calculated to determine the effect of ACT against comparator treatments. Because four of the included studies had small group sizes (< 20), Hedge's G was considered a more appropriate metric of effect size.25 Where applicable, an intent-to-treat analysis was used.
Heterogeneity across the studies was assessed using the Cochrane chi-square (?2) and quantified with theI 2statistic.26 I 2is the proportion of total variation provided by between-study variation, andI 2values of 25, 50 and 75% were considered to represent low, moderate, and high heterogeneity, respectively.27 We checked for potential publication bias by visual inspection of a funnel plot.
RESULTS
Search results
Figure 1 shows the flow of papers through the review. The initial search identified 1,305 articles; this was reduced to 1,152 following removal of duplicates. Title and abstract and full-text screening resulted in seven trials that met our inclusion criteria.
Unpublished trials
The search of trial registries identified a further eight studies, involving at least 126 participants, which met the inclusion criteria. Table 1 summarizes the results of this search. Principal investigators were contacted on May 22, 2018, with a request to release data for our meta-analysis. Two authors representing three trials responded. The corresponding author for NCT02336581 stated that analysis was incomplete, but that the aim of the trial was primarily feasibility and acceptability. The corresponding author for ISRCTN 68540929 and ISRCTN 73327972 - trials that ended in December 2013 and August 2018 respectively - stated that the authors were not in a position to share data. As of February 25, 2020, data from these trials had not yet been published.
| Principal investigator(s) | Study title | Registry | Registration status | Registration number | Start date | End date | Recruitment status | Study population | Number enrolled | Arms | Primary outcome | Availability of data |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gaudiano | Researching the effectiveness of acceptance-based coping during hospitalization (REACH) | ClinicalTrials.gov | Prospective | NCT02336581 | March 2015 | September 2017 | Completed | Psychosis | 62 | ACT/enhanced treatment as usual | Symptoms determined using the BPRS | May 22, 2018: response received, data not available. Rechecked Feb 26, 2020: data not published. |
| Lundgren | Acceptance and commitment therapy for psychotic inpatients | ClinicalTrials.gov | Retrospective | NCT02458105 | April 2015 | December 2017 | Withdrawn | Psychosis | Not reported | ACT/attentional control | Rehospitalization | May 22, 2018: no response received (as of Feb 26, 2020). Rechecked Feb 26, 2020: data not published. |
| Fatouros-Bergman | Cognitive behavior therapy for patients with psychoses | ClinicalTrials.gov | Retrospective | NCT03501888 | October 2015 | August 2016 | Completed | Psychosis | 16 | ACT/cognitive behavior therapy | Symptoms determined using the BPRS | May 22, 2018: no response received (as of Feb 26, 2020). Rechecked Feb 26, 2020: data not published. |
| Landa | Acceptance and commitment therapy for delusions | ClinicalTrials.gov | Prospective | NCT00657631 | April 2008 | June 2009 | Completed | Schizophrenia/ delusional disorder | 4 | Unable to determine from the information provided in the registry | Distress (measure not stated) | May 22, 2018: no response received (as of Feb 26, 2020). Rechecked Feb 26, 2020: data not published. |
| Ftanou | Treating acute psychotic symptoms in an inpatient service using acceptance and commitment therapy, measuring acceptance-based attitudes and actions, health outcomes and level of disability | Australian and New Zealand Clinical Trials Registry | Retrospective | ACTRN12609000603213 | July 2009 | Withdrawn | Withdrawn | Acute psychotic symptoms | 0 | ACT/treatment as usual | Voices actions and acceptance scale | May 22, 2018: no response received (as of Feb 26, 2020). Rechecked Feb 26, 2020: data not published. |
| Johns & Jolley | Acceptance and commitment therapy for recovery | ISRCTN | Retrospective | ISRCTN68540929 | February 2013 | December 2013 | Completed | Any service user or caregiver within the Promoting Recovery service | Not reported | ACT/waitlist control | WEMWBS, CORE-10 (psychological distress) | May 22, 2018: Response received. Data not available ahead of publication. Results not published as of Feb 26, 2020. |
| O'Donoghue, & Jolley | Balancing ACT: evaluating the effectiveness of psychoeducation and acceptance and commitment therapy groups for people with bipolar disorder | ISRCTN | Prospective | ISRCTN73327972 | May 2017 | August 2018 | Completed | Psychosis with bipolar disorder/symptoms | 36 | ACT/waitlist control | Quality of life determined using the Brief Quality of Life in Bipolar Disorder scale | May 22, 2018: Response received. Data not available ahead of publication. Results not published as of Feb 26, 2020. |
| Boden | Acceptance and commitment therapy for the inpatient treatment of psychosis | ClinicalTrials.gov | Prospective | NCT01981356 | September 2014 | April 2015 | Completed | Psychotic disorders | 18 | ACT/treatment as usual | Symptoms determined using the BPRS | Data were reported in registry entry. Results extracted and included in the review. |
ACT = acceptance and commitment therapy; BPRS = Brief Psychiatric Rating Scale; ISRCTN = International Standard Randomized Controlled Trial Number; WEMWBS = Warwick-Edinburgh Mental Well-being Scale.
Usable data were available from one unpublished trial.28 The authors published a summary of the feasibility of doing an ACT trial based on their experiences from the study, but did report results in the manuscript.29
The final number of trials included in this review and meta-analysis was therefore eight.
Characteristics on included trials
Table 2 shows the characteristics of the eight included trials. Trials were published between 2002 and 2018. Three were conducted in the United States, two in Australia, and one each in Sweden, Canada, and the United Kingdom.
| Citation | Country | Registration status | Population (mean age [years] male/ schizophrenia [%]) | Setting | ACT | Control condition | Number of participants allocated (ACT, control) | Number of participants included in analysis (ACT, control) | Sample size calculation | Symptom measure | Baseline symptoms measure mean (SD) ACT, control | End point symptoms measures mean (SD) ACT, control | Reported harms ACT, control |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Bach & Hayes9 | United States | Not registered | 39, 63/54 | Psychiatric inpatients | Individual, four sessions, lasting 45-50 min | Treatment as usual | 40, 40 | 35, 35 | No | Hospitalization | Not applicable | 4 months following discharge from inpatient care, 7, 14 participants rehospitalized | Psychiatric hospitalization reported as an outcome |
| Gaudiano & Herbert10 | United States | Not registered | Not possible to extract data from the manuscript. Data not reported. | Psychiatric inpatients | Individual, one to five sessions (average of three), lasting 60 min | Enhanced treatment as usual | 19, 21 | 19, 21 | No | BPRS: total scoreHospitalization | 58.4 (8.6),59.0 (8.0) | Prior to discharge from hospital41.6 (7.8), 45.6 (8.5) Nine control and five ACT patients hospitalized | Psychiatric hospitalization reported as an outcome. Authors report "no adverse events were observed from participation in the study." |
| White12 | United Kingdom | The trial was retrospectively registered (NCT01003132), but the registration number was not reported in the manuscript. | 34, 78/48 | Community-dwelling | Individual, 10 sessions | Waitlist | 14, 13 | 14, 10 | No | Positive symptoms determined using the positive symptoms subscale of PANSS | 11.36 (2.62), 12.75 (4.16) | 3 months post baseline9.75 (3.60), 11.70 (4.72) | Participant in the trial did have contact with crisis contacts that would constitute an adverse event, but numbers not reported. Authors stated that "There were no suspected unexpected serious adverse reactions over the course of the trial." |
| Shawyer11 | Australia | Retrospectively registered (after the first participant had been enrolled). Reason for retrospective registration not given. | 40, 56/74 | Community-dwelling | Individual, 12 sessions, weekly (plus three assessment and engagement sessions) | Befriending (n=17 participants were initially randomized to a waiting list and then to ACT or befriending) | 21, 22 | The authors reported that between 18-20 participants were followed up in each group, depending on the measure | Yes | PANSS: total score | 67.9 (13.6), 58.1 (8.4) | Completion of therapy60.1 (15.9), 57.5 (10.4) | Harms not reported |
| Boden29* | United States | Prospectively registered | Not possible to extract data from the manuscript. Data not reported. | Inpatients | Individual, four sessions | The control condition is treatment as usual in registry. In the manuscript, the authors describe attentional control (not described in detail). | 12, 6 | 5, 4 | No | Symptoms determined using the BPRS | Data are not reported in the registry | Discharge from hospital-8.0 (13.8),-11.6 (5.8) | No harms reported |
| Shawyer30 | Australia | Prospectively registered | 34, 61/75 | Community-dwelling | Individual, eight sessions, 50 min, weekly to fortnightly, over 3 months, delivered by a clinical psychologist | 8, 50 min sessions of befriending delivered by a clinical psychologist | 49, 47 | 49, 47 | Yes | PANSS: total score | 79.2 (14.1),78.6 (14.9) | Post-therapy74.6 (16.6), 72.6 (14.7) | The authors state similar rates of hospital admissions in both groups (serious adverse event), but do not report numbers. No other harms reported. |
| Spidel16 | Canada | Not registered | Not possible to extract data from the manuscript. Data not reported. Authors state that participants had psychosis and childhood trauma. | Not specified | Group, eight sessions (up to eight participants), lasting 90 min | Waitlist (treatment as usual) | 30, 20 | 30, 15 | No | BPRS (expanded) total score | 50.2 (1.6),51.0 (1.9)┼ | End of treatment 43.1 (1.6), 51.1 (1.9) | Not reported |
| Tyrberg31 | Sweden | Not registered | 41, 62/29 | Inpatients | Individual, one to four sessions (average of two), lasting 45 min | Treatment as usual | 12, 10 | 11, 10 | No | Rehospitalization | Not applicable | 4-month follow-up from trial entry 1, 4 | Not reported |
ACT = ACT = acceptance and commitment therapy; BPRS = Brief Psychiatric Rating Scale; PANSS = Positive and Negative Syndrome Scale; SD = standard deviation.
* Data were published on ClinicalTrials.gov. The registry entry was last updated on April 14, 2016 (checked on July 30, 2019). A short feasibility report29 summarizes the feasibility of ACT in an inpatient setting, but does not report raw data.
┼ Standard error (SE) reported in the manuscript. SD was calculated using the formula SD = SE.Vn.
Disorders of interest and measurement
As per our inclusion criteria, trials focused on patients experiencing psychosis. Of these, Spidel et al.16 also required participants to have experienced childhood trauma. The demographics and clinical profile of participants were not reported across all trials; notably, we were not able to extract necessary information from three studies.10,16,29 Where demographic and clinical data were reported, participants were typically in their 30s, there was a slight preponderance of males. Psychotic symptoms were reported as an outcome in six published trials: three used the PANSS and three the BPRS. The authors of three trials reported hospitalization as an outcome. Primary outcomes were specified in two of seven published trials11,30; the remaining studies merely listed outcomes used in the trial.
Trial arms and participants
Four of the trials were conducted in inpatient settings and three in the community. Overall, there were 197 and 179 participants in the ACT and comparator groups, respectively. The smallest trial involved 18 participants,29 and the largest, 96.30
In the Gaudiano & Herbert10 trial, we identified an error in the results table (which was confirmed by the corresponding author): the number of participants in the experimental and control groups is reversed. The authors published a corrigendum confirming this.32 We have reported the correct numbers in our data extraction (i.e., control = 21, experimental = 19), as shown in Table 1.
ACT
Seven trials9-12,29-31 tested individual ACT, and one tested group ACT.16 Individual treatment comprised one31 to 1511 sessions, typically lasting between 45 minutes and 1 hour. Group ACT16 comprised eight sessions lasting 90 minutes, with eight participants in each session. The ACT approach used varied between studies. Four authors based their intervention on the original2 Hayes manual,9,10,29,31 while in the other four studies,11,12,16,30 ACT manuals were developed or adapted especially for the trial. The Spidel16 trial was the only study that explicitly stated metaphors were not used because of the cognitive problems associated with psychosis. In other studies (e.g., Gaudiano & Herbert10 und Tyrberg et al.31), metaphors were applied as part of the ACT approach. In three studies,10,30,31 participants were given homework tasks to complete between sessions. The psychology background of clinicians providing ACT was stated in two studies.11,30 That ACT was provided by a psychologist could be worked out or inferred (e.g., the affiliation of the author that provided the ACT) in other studies, but was not explicitly stated. In all but two trials,12,31 the clinician providing treatment received additional training and supervision to deliver ACT.
Comparator interventions
The comparator intervention was treatment as usual or waiting list control in four9,10,12,16 of the trials. Befriending was used as a comparator intervention in two trials.11,30 Boden states in their registry entry that there was an attentional control component in their trial, whereby ACT facilitators met with participants for 15 minutes every other day to provide additional support and answer questions, ensuring not to discuss ACT techniques; however, this is not reported in the published feasibility report.29
Sample size calculations
Two trials reported sample size calculations.11,30 In both cases, researchers failed to recruit the required number of participants.
Numbers completing treatment and included in the analysis
A CONSORT diagram33 of participant flow through the trial was provided in four studies.10-12,30 Of 328 participants invited to take part, 215 (66%) consented, suggesting that around two patients needed to be asked to get one to consent to participate in the trial. In the ACT and comparison groups, respectively, 183 (93%) and 162 (91%) participants were included in the analysis. Data on the feasibility of delivering ACT to patients with psychosis (e.g., number of sessions completed, duration of sessions) was inconsistently reported. Shawyer et al.30 reported that participants completed seven of eight sessions in both the ACT and befriending groups.
Harms reporting
Adverse and serious adverse events that occurred during the trial were not reported in adequate detail in any of the included trials. Shawyer et al.30 reported admission to hospital as a serious adverse event. Gaudiano & Herbert10 stated that no harms occurred. White et al.12 stated that there were no serious adverse reactions over the course of the trial.
Meta-analyses
Figure 2 is a forest plot showing the effect sizes for each study individually and pooled as an aggregate for psychotic symptoms at the end of ACT treatment. An effect size below 0 (left of the line of no effect) favors ACT; one above 0 (right of the line of no effect) favors the comparator interventions(s). The pooled effect size for the six included trials was -0.21 (95% confidence interval [95%CI] -0.60 to 0.18), suggesting a small effect size in favor of ACT. Moderate statistical heterogeneity (I 2 > 50%) was observed.

Three trials9,10,31 reported rehospitalization as a surrogate measure of psychotic relapse. A relative risk (RR) less than 1 favors ACT; above 1, the comparator condition. The RR was 0.37 (95%CI 0.17 to 0.80), suggesting that ACT is associated with a significant reduction in the risk of admission (Figure 3).

Subgroup analyses
Post-hoc, we conducted subgroup analyses to examine differences in post-treatment symptom outcomes between individually delivered and group-delivered ACT. Five trials testing individual ACT and one16 testing group ACT reported psychotic symptoms at the end of treatment as an outcome. In a separate meta-analysis (Figure 4), a medium to large effect size for group ACT was observed (standard mean difference [SMD] = -0.91, 95%CI -1.50 to -0.31). Individual ACT was seemingly not effective (SMD = -0.04, 95%CI -0.33 to 0.26).

Sensitivity analyses
Of the six trials included in primary meta-analysis, three11,29,30 had a comparator intervention (e.g., befriending) that was intended to control for the nonspecific effects of treatment, while three10,12,16 used treatment as usual. In post-hoc sensitivity analyses (Figure 5), there was a medium effect size (SMD= -0.64, 95%CI -1.02 to -0.27) where treatment-as-usual was the comparator. In trials with active control, no effect of ACT on post-treatment psychosis symptoms was observed; conversely, there was a nonsignificant trend toward the control (SMD = 0.16, 95%CI -0.15 to 0.48).

Prospective trial registration
Of the eight trials included in this review, two were registered prospectively and one retrospectively. Four were not registered. White et al.12 registered their trial, but did not report the registration number in the manuscript.
Risk of bias
Table 3 shows the risk of bias for the eight included trials. One trial was considered to have a low risk of bias, and one as some concerns. Six trials were rated as having a high risk of bias. This mostly related to domain 5 (bias in selection of the reported result), i.e., results were selected from multiple outcomes. For example, Spidel et al.16 listed six outcomes - emotional regulation-acceptance, psychiatric symptoms, trauma symptoms, anxiety symptoms, service engagement, and feedback interview - and no prespecified plan for analysis was available. Tyrberg31 was rated as having a high risk of bias arising from the randomization process. The authors state that they used pairwise randomization to ensure an even flow of participants; consequently, it seems reasonable to suspect that the enrolling researcher would therefore have knowledge of the upcoming group allocation. In the Cochrane risk of bias algorithm34 (page 6), this requires a high risk of bias rating. Three studies9,16,29 were rated some concerns against domain 1, because essentially no information was given about the randomization process. For example, Bach & Hayes9 state that "those who agreed to participate were randomly assigned" (page 1130); no other information is provided. We rated four studies some concern against domain 4 (risk of bias in measurement of the outcome), due to potential for outcome assessment to have been influenced by knowledge of the assigned intervention.
| Study | Domain 1: Risk of bias arising from the randomization process | Domain 2: Risk of bias due to deviations from the intended interventions (effect of assignment to intervention) | Domain 2: Risk of bias due to deviations from the intended interventions (effect of adhering to intervention) | Domain 3: Missing outcome data | Domaine 4: Risk of bias in measurement of the outcome | Domain 5: Risk of bias in selection of the reported result | Overall risk of bias* |
|---|---|---|---|---|---|---|---|
| Bach & Hayes9 | Some concerns | Low | Low | Low | Low | High | High |
| Gaudiano & Herbert10 | Low | Low | Low | Low | Some concerns | High | High |
| White12 | Low | Low | Low | Low | Low | High | High |
| Shawyer11 | Low | Low | Low | Low | Low | High | High |
| Boden unpublished | Some concerns | Low | Low | Low | Some concerns | Some concerns | Some concerns |
| Shawyer30 | Low | Low | Low | Low | Low | Low | Low |
| Spidel16 | Some concerns | Low | Low | Low | Some concerns | High | High |
| Tyrberg31 | High* | Low | Low | Low | Some concerns | High | High |
* The default overall rating of risk of bias is high risk if even one of the domains is judged as such.24

Publication bias
Figure 6 shows a funnel plot for the six studies included in the meta-analysis of ACT against psychotic symptoms. The modest number of trials means that it is challenging to determine asymmetry. Visual interpretation of the plot is, however, suggestive of possible publication bias. This publication bias is supported by our search of trial registries, with seven trials remaining unpublished, despite considerable time elapsing since they were registered.

DISCUSSION
This systematic review aimed to establish the safety and efficacy of ACT in the treatment of schizophrenia-spectrum and other psychotic disorders. Our meta-analysis suggests ACT has a small, nonsignificant effect size vs. comparator interventions against psychotic symptoms. We observed a significant reduction in the risk of relapse, but only three trials were included in the meta-analysis. Sub-group analysis provided limited evidence that group ACT may be more effective than individual ACT. Sensitivity analysis suggested that, in more methodologically robust studies (with an attentional control), ACT was not an effective treatment against psychosis symptoms. The safety of ACT was not adequately reported in the included trials, and this is of notable concern. We identified seven completed trials where researchers would not release data to include in our meta-analysis. We therefore advise caution in applying ACT in clinical practice.
Our cautious conclusion about the safety and efficacy of ACT in psychosis is consistent with the review by Ost,5 but discrepant with the three other reviews of ACT in psychosis.6,7,15 As we have already noted, Wakefield et al.6 included some data sets twice in their qualitative synthesis, which may have influenced their generally positive conclusion. Jansen et al.15 made critical errors in data extraction; consequently, we believe the effect size they report against overall symptomatology in their paper is an overestimate. We note that the authors may have also made the same error when extracting data on acceptance and anxiety.
Risk of bias
Six trials were rated as having a high risk of bias. Our rating is notably discordant from that of the recent Jansen et al.15 review, which stated that ACT trials had a low risk of bias. In part, the discrepancy between the two reviews may be related to the version of the Cochrane risk of bias measure that was used; Jansen et al.15 used the original version,17 while we used the revised (RoB 2) tool.24 Specifically, Jansen et al.15 reported that included studies had a low risk of selective reporting bias - an item that has been revised in version 2. A judgment of low risk of bias in selection of the reported result now requires that the trial was analyzed following a prespecified plan that was finalized before unblinded outcome data were available for analysis. Only one trial met these criteria.30
Trial registration
Only two trials included in this review were prospectively registered. Pre-registration is intended to minimize the risk of selective reporting bias.35 Previous systematic reviews of ACT have not noted trial registration as an issue.
Harms reporting
Authors have previously noted the need for better harms reporting in clinical trials.36 We found that there was virtually no discussion of potential harms associated with ACT in the included studies. Authors that did mention adverse events simply stated that no harm occurred during the conduct of the trial. Jansen et al.15 concluded their systematic review by stating that ACT appears to be a safe intervention. We disagree with this statement. There was little evidence that researchers in included trials actively monitored participants for adverse events, including serious adverse events. According to Good Clinical Practice (GCP) guidance, an adverse event is "any untoward medical occurrence in a participant, which does not necessarily have a causal relationship with the trial intervention."37 It seems extremely unlikely that no events that met this definition occurred during the conduct of the included trials. In our view, there is a need for investigators to be more proactive in monitoring harms in future ACT trials.
Risks with underpowered trials
The sample size of the included trials was modest, and included trials were potentially underpowered. The sample size calculation in the Shawyer et al.11 trial, for example, required 60 participants, yet only 44 were recruited; a limitation the authors acknowledge. Intuitively, researchers are most concerned with an underpowered trial resulting in a type II error (where the hypothesis is true, but not rejected). However, should underpowered trials deliver a positive result, there may be an increased risk of type I error (where the hypothesis is false but accepted38). Small sample sizes can undermine the reliability of trials; this should be borne in mind when interpreting results.
Limitations of this review
Seven completed yet unpublished trials were identified in our search, and our funnel plot suggested the possibility of important publication bias. The identification of unpublished work is a strength of this review. This finding also raises concerns about the validity of our - indeed anyone's - observations. It is possible that further unregistered and unpublished ACT trials have been completed, but not reported. Authors of unpublished trials either did not respond to our e-mails or were unwilling to share their data. It is understandable that research groups typically first publish their work in a peer-reviewed journal before making data publicly available; that said, one of the unpublished trials (ISRCTN68540929) was - according to the registry entry - completed in 2013, and another in 2009 (NCT00657631), yet their data are still not publicly available.
Several further limitations of this review warrant mention. Firstly, we used an inclusive definition of ACT, to an extent this was unavoidable given the small number of trials currently published. Consequently, there was considerable variation in the duration and mode of delivery; ideally, we would have preferred trials that tested a prescribed ACT treatment program (e.g., 12 weekly sessions of individual therapy delivered by a psychologist following a standard treatment manual).
A further limitation pertains to heterogeneity. For example, Shawyer et al.11 only included participants experiencing problematic command hallucinations, while Spidel et al.16 focused on individuals with psychosis who had experienced childhood trauma. Similarly, comparators varied across studies as well (e.g., treatment as usual, befriending). We only included studies published in English. There may be relevant studies that were published in other languages. Finally, we did not extract follow-up data and can consequently not comment on the long-term benefits of ACT.
CONCLUSION
Our systematic review and meta-analysis provide limited evidence that ACT is an effective treatment for people experiencing psychosis. Included trials were modest in scale and had a high risk of bias. In our view, there is a need for methodologically rigorous, appropriately powered, multicenter trials, with an appropriate comparator. Additionally, we encourage researchers to ensure that their trials are registered, and that data are made available promptly. Our advice to clinicians is that caution be exercised when applying a treatment whose efficacy has not yet been established, and where there is a paucity of data concerning the potential for harm.
Disclosure
The authors report no conflicts of interest.










PDF
Print
Send this article by email
How to cite this article
Submit a comment
Mendeley
Pocket




