Article Text

Download PDFPDF

Process evaluation and the development of behavioural interventions to improve psychological distress among survivors of critical illness
  1. Ann M. Parker1,2,
  2. Albahi Malik1,2,
  3. Megan Hosey2,3
  1. 1 Division of Pulmonary and Critical Care Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA
  2. 2 Outcomes After Critical Illness and Surgery (OACIS) Group, Johns Hopkins University, Baltimore, MD, USA
  3. 3 Department of Physical Medicine and Rehabilitation, Johns Hopkins University School of Medicine, Baltimore, MD, USA
  1. Correspondence to Dr Ann M. Parker, Division of Pulmonary and Critical Care Medicine, Johns Hopkins University School of Medicine, Baltimore MD 21205, USA ; aparke36{at}

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Depressive, anxiety and post-traumatic stress disorder (PTSD) symptoms are common, long-lasting and associated with worse quality of life among intensive care unit (ICU) survivors.1–4 There are few clinical trials evaluating interventions to improve these symptoms. Cox and colleagues evaluated the feasibility of a mobile mindfulness training application (App) to address psychological stress after critical illness.5 Mindfulness promotes non-judgmental awareness of the present and acceptance of emotions, thoughts (eg, ‘I’m broken’, ‘I’m dependent’) and physiological states (eg, pain, shortness of breath), thus improving engagement with thoughts and behaviours related to illness along with coping and medical management of disease.6

This multisite pilot randomised controlled trial (RCT) randomised 80 participants to one of the three groups: mobile App, telephone mindfulness training or education control. Eligibility criteria included adults in the ICU ≥24 hours with cardiorespiratory failure. Participants randomised to mobile mindfulness intervention interacted with the App over 4 weekly mindfulness training sessions. Participants in the telephone group completed 4 weekly telephone mindfulness sessions (~30 min each) with a trained psychologist. The education control group received two telephone calls and a self-directed web-based programme focused on the nature and treatment of critical illness. Primary outcomes included feasibility, acceptability and usability. Secondary outcomes included depressive, anxiety and PTSD symptoms.

Approximately 83% (n=66) of patients completed the study. Loss to follow-up was 29%, 10% and 11% in the mobile mindfulness, telephone mindfulness and education groups, respectively. The percentage of participants who were not lost to follow-up completing all intervention sessions was ≥92% in each group. The results demonstrated ‘strong’ acceptability and usability with qualitative feedback considered overall ‘positive’ regarding the App. At 3-month follow-up, preliminary data demonstrated no significant improvement from baseline (before hospital discharge) in anxiety and depressive symptoms in the education control group while there was a trend toward improvements in these symptoms in the mobile and telephone mindfulness groups. There was no such trend for PTSD symptoms. In the mobile mindfulness group, the frequency and duration of App interaction correlated with improvement in depressive symptoms.

The success of this study is attributable, in part, to the authors’ extensive prior work in the field of postdischarge interventions to improve psychological stress symptoms in ICU survivors (table 1).7–9 While not explicitly described, they incorporated aspects of a process evaluation when refining the complex behavioural intervention (ie, mobile mindfulness training). Process evaluation allows for better understanding of an intervention to allow for adaptation, more effective implementation or application to a different population and is necessary for each phase of intervention assessment (ie, pilot feasibility, effectiveness and postevaluation implementation).10 In 2015, the UK Medical Research Council published guidelines for the process evaluation of complex interventions, with a focus on four key components: (1) theoretical framework and causal hypothesis, (2) implementation, (3) mechanisms of impact and 4) context.11 Process evaluation is often overlooked but is especially important in studying complex interventions in ICU survivors given the population’s heterogeneity in illness severity, comorbidities, exposure to ICU therapies and experiences of recovery. In the following paragraphs, we will discuss these four key components of process evaluation in the context of this study.

Table 1

Summary of key components of process evaluation highlighted by each study conducted by the authors

Theoretical Framework

Process evaluation is founded in a theoretical framework, often based on hypothesis-generating qualitative research in the population of interest.11 In this case, the authors developed the theoretical framework for a mindfulness training intervention in ICU survivors (see figure 1 for our interpretation) using an initial qualitative study (n=23 ICU survivors and caregivers) demonstrating that symptoms of anxiety and depression are common among ICU survivors, who also reported difficulty coping with their illness.12 In a subsequent pilot study aimed at developing and evaluating a coping skills training programme in ICU survivors, the authors identified several mindfulness-related skills that were highly valued but used infrequently.9

Figure 1

Conceptual framework for mindfulness training in survivors of critical illness. ICU, intensive care unit; PTSD, post-traumatic stress disorder.


Both quantitative and qualitative measures are used to examine the next aspect of process evaluation: implementation, or the ‘what’ and ‘how’ of intervention delivery.11 13 The authors conducted a prior pilot study (n=8 ICU survivors and caregivers) demonstrating that a telephone mindfulness programme was acceptable and feasible. This study identified several unresolved implementation issues: dose (number and duration of sessions); access to the intervention (perhaps via more advanced technology) and patient preference for treatment modality (eg, self-management vs therapist-based).8 While each of these issues was considered when designing the current RCT (table 1), several issues remain unsolved, including the optimal dose of mindfulness sessions (frequency and duration). Studies of mindfulness training (eg, Mindfulness Based Stress Reduction) in other populations suggest that 6–12 sessions are sufficient to understand and gain mastery of mindfulness skills.14

Next, the investigators conducted an RCT (n=175) evaluating the effects of a telephone-based and web-based coping skills training intervention versus an education control on depressive symptoms in ICU survivors.7 When this RCT did not demonstrate significant reductions in depressive symptoms, the authors noted that adherence to the protocols was low in both groups, with ‘medical illness’ being the most common explanation.7 Hence, by considering this aspect of implementation, that is, adherence or uptake, the authors refined the eligibility criteria for the current RCT.7 While adherence was improved in the current RCT, twice as many participants randomised to the mobile versus telephone mindfulness group or education control were lost to follow-up. The majority were lost to follow-up before initiating the App. Such differential attrition might lead to bias if participants lost to follow-up versus those completing the study were more likely to rate the App as having low acceptability or usability or were more likely to have higher, more persistent psychological stress symptoms. Including use of a rating scale, such as the Mobile Application Rating Scale (MARS), in the process evaluation would provide a greater understanding of App quality, a factor that may have impacted participant engagement.15

Two additional aspects of implementation are intervention training and fidelity monitoring. Understanding ‘what’ is delivered requires a detailed description of the credentials, background and training provided for the interventionist, as done via publication of the protocol for the current RCT.5 The authors conducted fidelity monitoring using audio recording (telephone group) and electronic reporting (mobile group) of treatment sessions to ensure the intervention was being delivered as intended.11 Intervention training and fidelity monitoring are especially important aspects of implementation in multisite studies, given increased potential for variability.16

Mechanisms of impact

Another important aspect of process evaluation is a mechanism of impact, that is, a description of how the participants interact with the intervention, mediators and potential unexpected pathways. Mindfulness training is an effective intervention to reduce psychological distress in populations with chronic medical problems,6 though its mechanisms are poorly understood. Importantly, feasibility studies, such as the current study,5 are not powered to perform an analysis of mediators. Nonetheless, a crucial aspect of process evaluation is a comprehensive understanding of intervention theory that allows for targeted exploration of causal mechanisms using quantitative, when possible, and qualitative techniques.10 By failing to understand why an intervention is successful, investigators run the risk of drawing incorrect conclusions regarding causal pathways and overlooking key intervention components or mediators.10 Importantly, ICU survivors experience impairments in physical, cognitive and mental health, with prior studies demonstrating the close interplay between physical and mental health impairments.17 18 Hence, it is important to understand how a complex intervention affecting one of these outcomes (eg, depressive symptoms) might impact others (eg, physical function), as well as the role of mediators (eg, coping skills) that may affect the likelihood of treatment uptake and efficacy. A qualitative analysis of potential mediators could strengthen insights gained from the current study.


Process evaluation promotes an understanding of the context in which participants interact with complex interventions. Context includes demographics, socioeconomic factors, cultural systems and beliefs, and characteristics inherent to a given medical condition.10 A thorough understanding of the intervention’s acceptance, reach and effectiveness among subgroups of ICU survivors is key to informing future RCTs and broader application of the intervention. The coping skills training RCT7 demonstrated that patients with higher baseline distress were more likely to benefit from an intervention aimed at improving coping skills and self-efficacy, thus highlighting the need for studies of future interventions to target subgroups of ICU patients most likely to benefit.7

Ongoing work and future steps

The extended body of work leading to the current RCT is unique within the field of postdischarge interventions to improve psychological stress among ICU survivors. The ICU diary, which is associated with reduced psychological stress in this population, is similarly a complex intervention.19 20 Such complex behavioural interventions show promise in improving the multifaceted long-term sequelae of critical illness. By involving behavioural health experts in the design of such interventions and incorporating formal process evaluations, future studies can describe key components of behavioural interventions and their generalisability.

In summary, the current RCT of a mobile mindfulness training intervention provides important insights into the implementation of a complex behavioural intervention aimed at improving psychological stress among ICU survivors. Future research in this area should include further evaluation, using a larger sample size, of the optimal dose of mindfulness training; potential mediators of the association between mindfulness training and improved psychological stress symptoms; and the quality of future mobile apps.



  • Contributors AP: contributed to the manuscript as the author primarily responsible for drafting the manuscript and incorporating coauthors' edits. MH and AM: contributed by making important additions to the content, editing and reviewing for completeness/accuracy.

  • Funding AMP receives salary support from an NIH grant (K23HL138206).

  • Competing interests None declared.

  • Provenance and peer review Commissioned; externally peer reviewed.

Linked Articles