Theoretical Framework

Process evaluation is founded in a theoretical framework, often based on hypothesis-generating qualitative research in the population of interest.11 In this case, the authors developed the theoretical framework for a mindfulness training intervention in ICU survivors (see figure 1 for our interpretation) using an initial qualitative study (n=23 ICU survivors and caregivers) demonstrating that symptoms of anxiety and depression are common among ICU survivors, who also reported difficulty coping with their illness.12 In a subsequent pilot study aimed at developing and evaluating a coping skills training programme in ICU survivors, the authors identified several mindfulness-related skills that were highly valued but used infrequently.9

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Conceptual framework for mindfulness training in survivors of critical illness. ICU, intensive care unit; PTSD, post-traumatic stress disorder.

Implementation

Both quantitative and qualitative measures are used to examine the next aspect of process evaluation: implementation, or the ‘what’ and ‘how’ of intervention delivery.11 13 The authors conducted a prior pilot study (n=8 ICU survivors and caregivers) demonstrating that a telephone mindfulness programme was acceptable and feasible. This study identified several unresolved implementation issues: dose (number and duration of sessions); access to the intervention (perhaps via more advanced technology) and patient preference for treatment modality (eg, self-management vs therapist-based).8 While each of these issues was considered when designing the current RCT (table 1), several issues remain unsolved, including the optimal dose of mindfulness sessions (frequency and duration). Studies of mindfulness training (eg, Mindfulness Based Stress Reduction) in other populations suggest that 6–12 sessions are sufficient to understand and gain mastery of mindfulness skills.14

Next, the investigators conducted an RCT (n=175) evaluating the effects of a telephone-based and web-based coping skills training intervention versus an education control on depressive symptoms in ICU survivors.7 When this RCT did not demonstrate significant reductions in depressive symptoms, the authors noted that adherence to the protocols was low in both groups, with ‘medical illness’ being the most common explanation.7 Hence, by considering this aspect of implementation, that is, adherence or uptake, the authors refined the eligibility criteria for the current RCT.7 While adherence was improved in the current RCT, twice as many participants randomised to the mobile versus telephone mindfulness group or education control were lost to follow-up. The majority were lost to follow-up before initiating the App. Such differential attrition might lead to bias if participants lost to follow-up versus those completing the study were more likely to rate the App as having low acceptability or usability or were more likely to have higher, more persistent psychological stress symptoms. Including use of a rating scale, such as the Mobile Application Rating Scale (MARS), in the process evaluation would provide a greater understanding of App quality, a factor that may have impacted participant engagement.15

Two additional aspects of implementation are intervention training and fidelity monitoring. Understanding ‘what’ is delivered requires a detailed description of the credentials, background and training provided for the interventionist, as done via publication of the protocol for the current RCT.5 The authors conducted fidelity monitoring using audio recording (telephone group) and electronic reporting (mobile group) of treatment sessions to ensure the intervention was being delivered as intended.11 Intervention training and fidelity monitoring are especially important aspects of implementation in multisite studies, given increased potential for variability.16

Mechanisms of impact

Another important aspect of process evaluation is a mechanism of impact, that is, a description of how the participants interact with the intervention, mediators and potential unexpected pathways. Mindfulness training is an effective intervention to reduce psychological distress in populations with chronic medical problems,6 though its mechanisms are poorly understood. Importantly, feasibility studies, such as the current study,5 are not powered to perform an analysis of mediators. Nonetheless, a crucial aspect of process evaluation is a comprehensive understanding of intervention theory that allows for targeted exploration of causal mechanisms using quantitative, when possible, and qualitative techniques.10 By failing to understand why an intervention is successful, investigators run the risk of drawing incorrect conclusions regarding causal pathways and overlooking key intervention components or mediators.10 Importantly, ICU survivors experience impairments in physical, cognitive and mental health, with prior studies demonstrating the close interplay between physical and mental health impairments.17 18 Hence, it is important to understand how a complex intervention affecting one of these outcomes (eg, depressive symptoms) might impact others (eg, physical function), as well as the role of mediators (eg, coping skills) that may affect the likelihood of treatment uptake and efficacy. A qualitative analysis of potential mediators could strengthen insights gained from the current study.

Context

Process evaluation promotes an understanding of the context in which participants interact with complex interventions. Context includes demographics, socioeconomic factors, cultural systems and beliefs, and characteristics inherent to a given medical condition.10 A thorough understanding of the intervention’s acceptance, reach and effectiveness among subgroups of ICU survivors is key to informing future RCTs and broader application of the intervention. The coping skills training RCT7 demonstrated that patients with higher baseline distress were more likely to benefit from an intervention aimed at improving coping skills and self-efficacy, thus highlighting the need for studies of future interventions to target subgroups of ICU patients most likely to benefit.7

Ongoing work and future steps

The extended body of work leading to the current RCT is unique within the field of postdischarge interventions to improve psychological stress among ICU survivors. The ICU diary, which is associated with reduced psychological stress in this population, is similarly a complex intervention.19 20 Such complex behavioural interventions show promise in improving the multifaceted long-term sequelae of critical illness. By involving behavioural health experts in the design of such interventions and incorporating formal process evaluations, future studies can describe key components of behavioural interventions and their generalisability.

In summary, the current RCT of a mobile mindfulness training intervention provides important insights into the implementation of a complex behavioural intervention aimed at improving psychological stress among ICU survivors. Future research in this area should include further evaluation, using a larger sample size, of the optimal dose of mindfulness training; potential mediators of the association between mindfulness training and improved psychological stress symptoms; and the quality of future mobile apps.