Introduction
Prolonged elevated stress often has devastating negative long-term effects on physical and mental health (Cohen, Reference Cohen2000; Haight et al., Reference Haight, Peddie, Crosswell, Hives, Almeida and Puterman2023; O’Connor et al., Reference O’Connor, Thayer and Vedhara2021). Moreover, stress imposes a substantial financial burden on society, mainly due to loss of productivity and increased demand for healthcare services (Hassard et al., Reference Hassard, Teoh, Visockaite, Dewe and Cox2018). For some occupations, such as healthcare professionals, acute stress seems to be an inherent part of daily work (Mingote Adán et al., Reference Mingote Adán, Moreno Jiménez and Gálvez Herrer2004). Substantial evidence shows that stress-management interventions can effectively reduce subjective stress, with multi-modal, relaxation-based, and cognitive interventions showing the most beneficial effects in reducing stress symptoms (Estevez Cores et al., Reference Estevez Cores, Sayed, Tracy and Kempton2021).
The increasing demand for psychological services (e.g. American Psychological Association, 2021) and the rapid development of secure information technologies have encouraged the growth of internet-delivered psychological interventions (Andersson, Reference Andersson2018). In the face of the burden brought on the world by the COVID-19 pandemic, the world has faced the reality that, at times, internet interventions might be the only available option when addressing mental health issues when face-to-face contact is not recommended (Wind et al., Reference Wind, Rijkeboer, Andersson and Riper2020). In general, internet-delivered interventions are seen as a more cost-effective alternative to face-to-face therapy (Andersson et al., Reference Andersson, Cuijpers, Carlbring, Riper and Hedman2014). Being also more accessible, internet interventions are seen as particularly valuable when addressing the lack of trained therapists, especially in rural or remote locations (Lattie et al., Reference Lattie, Stiles-Shields and Graham2022).
Numerous randomised controlled trials (RCTs) have demonstrated the efficacy of internet-delivered interventions on various mental health outcomes, such as anxiety and mood disorders (Andersson et al., Reference Andersson, Carlbring, Titov and Lindefors2019). A meta-analysis of 13 RCTs published between 2010 and 2021 confirmed the efficacy of internet interventions targeting elevated stress as a primary outcome and also documented secondary effects on measures of anxiety and depression (Svärdman et al., Reference Svärdman, Sjöwall and Lindsäter2022). Moreover, in nine out of 10 studies with follow-up measures, the effects of internet interventions on stress sustained (n=4) or even slightly increased (n=5) over time (Svärdman et al., Reference Svärdman, Sjöwall and Lindsäter2022). Nevertheless, most existing studies have focused on the effectiveness of interventions without examining how the effects are sustained or how different mental health symptoms interact. Moreover, it has been suggested that alongside a reduction of symptoms, an increase in well-being should be confirmed to establish intervention effects (Cloninger, Reference Cloninger2006). However, the interplay between the increase in psychological well-being and the reduction of symptoms is rarely addressed in intervention research, including research on the effects of CBT-based internet interventions.
To address these gaps in previous research, the current study aimed to investigate longitudinal links between symptoms of stress, anxiety, and depression, as well as psychological well-being, in a help-seeking sample of medical nurses who participated in the internet-delivered CBT-based intervention FOREST, targeting stress reduction (Jovarauskaite et al., Reference Jovarauskaite, Dumarkaite, Truskauskaite-Kuneviciene, Jovaisiene, Andersson and Kazlauskas2021; the description of the intervention FOREST is provided in the Method section). In the current study, we defined stress as the subjective feeling of lack of control and confidence to handle difficulties in important areas of life (Cohen and Williamson, Reference Cohen, Williamson, Spacapan and Oskamp1988). Anxiety was defined as a feeling of nervousness and inability to control worrying; and depression as a lack of interest or pleasure when doing things and feeling down or hopeless (Kroenke et al., Reference Kroenke, Spitzer, Williams and Löwe2009). Psychological well-being was defined as the subjective experience of positive emotional states, vitality, and sense of purpose (Bech, Reference Bech2004).
As was found in the previously conducted RCT, FOREST had immediate and follow-up effects, decreasing stress and depression and increasing well-being (Dumarkaite et al., Reference Dumarkaite, Truskauskaite, Andersson, Jovarauskaite, Jovaisiene, Nomeikaite and Kazlauskas2023). Symptoms of depression and anxiety tend to be interlinked over time (Jacobson and Newman, Reference Jacobson and Newman2017). Therefore, we expected that anxiety and depression would predict each other over time. There is abundant literature linking exposure to stressful life events and mental health. Some longitudinal research shows that stressful events precede mental health deterioration (e.g. Schneider et al., Reference Schneider, Long, Arch and Hankin2021), and some report bi-directionality of associations between exposure to stress and mental health (e.g. Schönfeld et al., Reference Schönfeld, Brailovskaia, Zhang and Margraf2019). However, there has been less research on longitudinal links between subjective stress and mental health. Some evidence shows that subjective stress is an even stronger predictor of overall health than exposure to stressful events (Shields et al., Reference Shields, Fassett-Carman, Gray, Gonzales, Snyder and Slavich2023). Also, the evidence from prospective longitudinal research showed that higher levels of subjective stress were a risk factor for future mental health problems in healthcare workers (Van Steenkiste, Reference Van Steenkiste, Schoofs, Gilis and Messiaen2022). Keeping this in mind and also considering that the current study was conducted in the context of stress recovery intervention, we expected that lower levels of subjective stress would predict a decrease in depression and anxiety symptoms and an increase in well-being over time. Nevertheless, most studies that inform intervention research are conducted in non-intervention settings. Addressing longitudinal links between stress, anxiety, depression, and well-being in the help-seeking sample that reported intervention-induced changes in mental health indicators would allow a better understanding of what further outcomes we might expect after fostering mental health.
Method
Participants
The participants of the internet-delivered stress recovery program FOREST (Dumarkaite et al., Reference Dumarkaite, Truskauskaite, Andersson, Jovarauskaite, Jovaisiene, Nomeikaite and Kazlauskas2023; Jovarauskaite et al., Reference Jovarauskaite, Dumarkaite, Truskauskaite-Kuneviciene, Jovaisiene, Andersson and Kazlauskas2021) were included in the current study. Participants were nurses and assistant nurses from both study arms of the RCT: an intervention group (IG) and a delayed treatment group (DTG), which received the intervention after a 3-month follow-up assessment. The inclusion criteria were as follows: (1) being a licensed nurse actively working within the national healthcare system; (2) providing written informed consent electronically; (3) completing a baseline assessment prior to randomization; (4) sufficient proficiency in Lithuanian to understand the intervention content and instructions; (5) possessing a computer, tablet, smartphone, or similar device with internet access. Exclusion criteria were as follows: (1) experiencing an acute psychiatric crisis; (2) a high risk of suicide; (3) alcohol or drug addiction; (4) current experience of interpersonal violence. Overall, 208 applicants registered for participation, and 184 met the inclusion and did not meet the exclusion criteria (93 were assigned to the IG and 91 to the DTG). Of these, 73 participants were excluded from the analysis because (1) they were from the DTG group and had not completed pre-test measurement questionnaires (n=29); (2) they had never logged in to the program (n=23); (3) they had not completed the post-test and/or follow-up assessment (n=21). The final sample of the current study consisted of 111 participants (65 from the IG and 46 from the DTG). The study flowchart is presented in Fig. S1 in the Supplementary material. Demographic characteristics of the current study sample (n=111) collected at enrolment are presented in Table 1.
Table 1. Demographic characteristics of the participants at the enrolment (n=111)
We found no differences between participants of the current study (n=111) and excluded participants (n=73) at the enrolment in terms of demographic characteristics, including gender (χ2(1)=0.89, p=.35), age (t 182=1.21, p=.23), job position (χ2(1)=0.00, p=.98), education (χ2(2)=0.98, p=.62), working status (χ2(2)=1.99, p=.37), work experience (χ2(3)=3.86, p=.28), subjective income (χ2(2)=0.13, p=.94), partnership status (χ2(1)=0.22, p=.64), seeing a psychologist (χ2(1)=2.84, p=.09), and usage of psychopharmacological medicine due to mental health difficulties (χ2(1)=1.08, p=.30). Also, there were no differences between included and excluded participants regarding any outcome measures at the enrolment: perceived stress (M (SD)=7.86 (2.33) vs M (SD)=7.89 (3.20); t 121.58=0.06, p=.95), anxiety (M (SD)=2.88 (1.56) vs M (SD)=2.63 (1.95); t 130.36=–0.93, p=.35), depression (M (SD)=2.56 (1.49) vs M (SD)=2.56 (1.75); t 182=0.13, p=.99), and psychological well-being (M (SD)=9.41 (4.65) vs M (SD)=9.84 (5.22); t 182=0.59, p=.56).
Procedures
The participants were all from Lithuania and were enrolled via disseminating information through press releases to media, healthcare institutions, and specialised social networks. Detailed enrolment procedures and participant eligibility criteria have been reported previously (Dumarkaite et al., Reference Dumarkaite, Truskauskaite, Andersson, Jovarauskaite, Jovaisiene, Nomeikaite and Kazlauskas2023; Jovarauskaite et al., Reference Jovarauskaite, Dumarkaite, Truskauskaite-Kuneviciene, Jovaisiene, Andersson and Kazlauskas2021). Informed consent was obtained from all participants before data collection. In the current study, the IG data collected at T1, T2 and T3 and the DTG data collected at T3, T4 and T5 were merged into a single data pool and represented pre-test, post-test, and follow-up, respectively. Thus, the current study presents the secondary analysis of the IG data collected at T1 and T2, as well as IG and DTG data collected at T3, while the data collected at T4 and T5 has not been analysed previously.
Program FOREST
The online cognitive behavior therapy-based stress recovery program used in the study FOREST aimed to enhance the healthcare staff’s stress recovery skills. FOREST also included mindfulness principles that have been shown to assist in the modulation of brain–behaviour interaction involved in recovery from stress (Johnson et al., Reference Johnson, Thom, Stanley, Haase, Simmons, Shih, Thompson, Potterat, Minor and Paulus2014). FOREST consisted of six sessions and lasted six weeks. Each session was unlocked on a weekly basis and covered the following topics: (1) Introduction; (2) Relaxation; (3) Psychological detachment; (4) Mastery; (5) Control; (6) Keeping the change alive. The topics of sessions 2 to 5 represented the four elements of the stress recovery experiences (Sonnentag and Fritz, Reference Sonnentag and Fritz2007). FOREST consisted of three main elements that can be found throughout the whole program: psychoeducation (text and video recordings), exercises (written and audio recordings), and communication with a psychologist. The psychologist’s role included providing feedback to the participants once they completed exercises and replying to personal messages from program participants. FOREST was developed specifically for healthcare staff experiencing stress by clinical psychologists and researchers at Vilnius University. More details on the program and its entire content can be found in the study protocol (Jovarauskaite et al., Reference Jovarauskaite, Dumarkaite, Truskauskaite-Kuneviciene, Jovaisiene, Andersson and Kazlauskas2021) and the FOREST RCT study (Dumarkaite et al., Reference Dumarkaite, Truskauskaite, Andersson, Jovarauskaite, Jovaisiene, Nomeikaite and Kazlauskas2023).
Measures
Stress
The Perceived Stress Scale (PSS-4) (Cohen and Williamson, Reference Cohen, Williamson, Spacapan and Oskamp1988) was used to evaluate the changes in self-reported perceived stress. The PSS-4 consists of four questions (e.g. ‘In the last month, how often have you felt that you were unable to control the important things in your life?’); each question is ranked on a 5-point Likert scale ranging from 0 (‘never’) to 4 (‘very often’). A higher score indicates more pronounced perceived stress. In the current study, Cronbach’s alpha for the PSS-4 at pre-test was α=.65.
Symptoms of anxiety and depression
The Patient Health Questionnaire (PHQ-4) (Kroenke et al., Reference Kroenke, Spitzer, Williams and Löwe2009) was used to evaluate the changes in self-reported anxiety and depression symptoms. The PHQ-4 consists of four items and two subscales: the anxiety subscale (e.g. ‘Feeling nervous, anxious or on edge’) and depression subscale (e.g. ‘Little interest or pleasure in doing things’); each item is ranked on a 4-point Likert scale ranging from 0 (‘not at all’) to 3 (‘nearly every day’). A higher score indicates more pronounced anxiety and depression symptoms. In the current study, Cronbach’s alpha for the PHQ-4 anxiety subscale at pre-test was α=.83, and for the PHQ-4 depression subscale at pre-test was α=.75.
Psychological well-being
The World Health Organization Well-being Index (WHO-5) (Bech, Reference Bech2004) was used to evaluate the changes in self-reported psychological well-being. The WHO-5 consists of five items (e.g. ‘I have felt cheerful and in good spirits’); each item is ranked on a 6-point Likert scale ranging from 0 (‘at no time’) to 5 (‘all of the time’). A higher score indicates more pronounced psychological well-being. In the current study, Cronbach’s alpha for the WHO-5 at pre-test was α=.87.
Data analyses
The current study aimed to investigate the longitudinal interplay between mental health indicators when participating in the internet-based stress-reduction intervention. As a preliminary analysis, we assessed the change on four indicators: symptoms of stress, anxiety, and depression, as well as the level of psychological well-being using the latent change modelling approach (Duncan et al., Reference Duncan, Duncan and Strycker2013). We ran a series of latent change models, indicating the estimated baseline levels and the estimated change in variables from pre- to post-test and from pre-test to follow-up.
To analyse the interplay between mental health indicators over time, we conducted a cross-lagged panel analysis (Selig and Little, Reference Selig and Little2012), with four variables representing symptoms of stress, anxiety, and depression, as well as the level of psychological well-being measured at three time-points, i.e. pre-test, post-test, and 3-month follow-up. When running the autoregressive cross-lagged model, the cross-lagged effects were estimated by controlling for the stability paths between the pre-test and post-test and between post-test and follow-up, as well as within-time correlations among all variables at all measurement points. All analyses were conducted using the sum scores of the variables with the robust maximum likelihood estimator in Mplus 8.2 (Muthén and Muthén, Reference Muthén and Muthén1998–2017). The model fit of the cross-lagged model was evaluated using the comparative fit index (CFI), the Tucker–Lewis index (TLI), the root mean square error of approximation (RMSEA), and χ2 test (Kline, Reference Kline2011). The CFI/TLI values higher than .90 and RMSEA values below .08 indicate an acceptable fit; CFI/TLI values higher than .95, RMSEA values below .06, and insignificant χ2 test represent a good fit. The full information maximum likelihood (FIML) algorithm was used for handling missing data in the current study.
Results
The latent change analysis indicated statistically significant reductions in stress, anxiety, and depression symptoms and a significant increase in well-being from pre- to post-test (see Fig. 1). At follow-up, the decreases in stress and depression symptoms and the increase in well-being were sustained, while anxiety levels returned to pre-intervention levels. The results of the autoregressive cross-lagged panel analysis indicated that the model fitted data well (χ2(16)=17.43, p=.358, CFI/TLI=.997/.988, RMSEA [90% CI]=.028 [.000, .094]). Significant stability paths and cross-lagged effects are presented in Fig. 2. The within-time correlations are reported in Table 2. As shown in Fig. 2, the levels of anxiety symptoms at the post-test positively predicted the levels of stress and depression symptoms at follow-up. Additionally, the levels of psychological well-being at the post-test negatively predicted stress symptoms at follow-up. No cross-lagged effects were observed from the pre-test to the post-test.
Figure 1. The univariate trajectories of stress, anxiety, depression, and psychological well-being over the study period (n=111). The variables were measured in different scales and are incomparable with one another; stars next to the estimated means represent the difference at post-test or follow-up in comparison with pre-test: **p<.01, ***p<.001.
Figure 2. Standardised significant estimates of the cross-lagged panel model (n=111). Numbers on the lines represent standardised beta coefficients; numbers within rectangles represent standardised R 2 coefficients; *p<.05, **p<.01, ***p<.001.
Table 2. Within-time cross-lagged panel correlations between study variables
All correlations are significant at p<.001.
Discussion
The current study aimed to investigate the longitudinal links between symptoms of stress, anxiety, and depression and the psychological well-being of the participants of the CBT-based internet intervention FOREST, targeting stress recovery. Our previous RCT study (Dumarkaite et al., Reference Dumarkaite, Truskauskaite, Andersson, Jovarauskaite, Jovaisiene, Nomeikaite and Kazlauskas2023) confirmed that the intervention positively affected nurses’ mental health and the effects on stress and depression symptoms, as well as well-being, sustained at a 3-month follow-up. The same within-group effects were confirmed in the current study using the combined sample of intervention participants who received intervention right after enrolment and delayed treatment group data. Overall, the results demonstrate that reduced anxiety and increased well-being predict the sustainability of the effects of reduced stress. Additionally, we found that reduced anxiety appears to contribute to sustaining intervention effects on reduced depression symptoms.
In contrast to our expectations, we did not find that reduced subjective stress was linked with decreased symptoms of poor mental health. In contrast, increased mental health contributed to sustaining lower subjective stress levels achieved through participation in the intervention. Even though there is strong evidence that exposure to stressful events precedes the symptoms of anxiety and depression (Schneider et al., Reference Schneider, Long, Arch and Hankin2021), our results suggest that when addressing elevated subjective stress, we need to target other mental health outcomes, such as anxiety and psychological well-being if we want effects to be sustained. Also, contrary to our expectations, we did not find bi-directional longitudinal links between anxiety and depression symptoms. The results suggest that reduced anxiety plays a role in sustaining intervention effects on depression symptoms but not vice versa.
Previous research has demonstrated that one of the primary mechanisms behind improved mental health following mindfulness-based stress-reduction interventions is decreased repetitive negative thinking (Gu et al., Reference Gu, Strauss, Bond and Cavanagh2015). It is also known that the reduction of repetitive negative thinking is usually paired with reduced anxiety (Monteregge et al., Reference Monteregge, Tsagkalidou, Cuijpers and Spinhoven2020). Although the design of the current study did not allow testing mediation mechanisms of intervention effects, it has been shown that reduced anxiety is one of the predictors of sustained intervention effects. Therefore, in stress reduction intervention research, more attention should be paid to targeting decreased anxiety through reduced repetitive negative thinking or other potential mechanisms.
The results also highlighted the importance of improved psychological well-being when targeting elevated stress and promoting mental health. There are many mechanisms for tackling well-being (Diener and Biswas-Diener, Reference Diener and Biswas-Diener2019). Some of them, such as CBT and mindfulness, were also used in the FOREST intervention. The results suggest that increased well-being contributes to sustaining reduced stress. From the perspective of the experiences of recovery from stress model (Sonnentag and Fritz, Reference Sonnentag and Fritz2007), positive emotionality, which is an internal part of psychological well-being (Bech, Reference Bech2004), might be seen as internal resources, providing a sense of self-esteem and self-confidence (Philippe et al., Reference Philippe, Dobbin, Ross and Houle2018) and thus might assist in coping with everyday challenges and protecting from stress (Cohen and Williamson, Reference Cohen, Williamson, Spacapan and Oskamp1988). Therefore, future intervention research should consider measuring psychological well-being as a secondary intervention outcome.
In summary, based on the findings of the current study, when applying internet-delivered interventions targeted at sustained stress reduction in clinical research and practice settings, specialists should consider incorporating more anxiety-reduction strategies. Moreover, interventions targeting stress reduction should consider focusing not only on immediate relief but also on longer-term strategies that could potentially increase overall well-being. Also, based on the findings of the current study, researchers and clinicians should consider monitoring anxiety and well-being as early indicators of whether the stress-reduction intervention has the potential to have long-lasting results. Finally, when personalising the interventions targeting stress, the baseline level of anxiety and well-being could be used as indicators when offering different levels of support or booster sessions to individuals with higher levels of anxiety or lower levels of well-being.
The current study’s findings should be seen in light of its limitations. The study’s main limitation is the combined study sample, resulting in different measurement times for the intervention group (IG) and delayed treatment group (DTG). Even though we did not find any differences between groups at pre-test and post-test and, in most cases, at follow-up, stress levels were higher at follow-up in the DTG, possibly indicating that long-term effects on stress were more sustainable in the IG, compared with DTG. Even though the waiting conditions do not seem to have negative effects on intervention outcomes when participating in stress reduction interventions (Elliott and Brown, Reference Elliott and Brown2002), in help-seeking samples (this was also the case in the current study), waiting for the intervention might introduce some dissatisfaction and, in turn, the final intervention effects might be less beneficial (Gunnarsson et al., Reference Gunnarsson, McCambridge and Bendtsen2023). Additionally, the study was conducted with a sample of self-referred nurses, indicating the risk of volunteering bias and the results should be generalised to other samples with caution. Moreover, the very short versions of stress, anxiety, and depression measures were used in the study and could limit the comparison of the results with other studies. Finally, the follow-up effects of the study were relatively short-term. Future studies should test the predictors of the sustained intervention effects with longer follow-ups.
Conclusion
The results of the current study provide evidence of the importance of reducing anxiety and promoting psychological well-being when targeting stress in internet interventions, as the post-intervention levels of anxiety and psychological well-being are associated with the sustainability of the intervention effects on reduced stress over time.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S1352465824000456
Data availability statement
The data that support the findings of this study are available from the corresponding author, upon reasonable request.
Acknowledgements
None.
Author contributions
Inga Truskauskaite: Conceptualization-Lead, Data curation-Equal, Formal analysis-Lead, Funding acquisition-Equal, Investigation-Equal, Methodology-Equal, Project administration-Supporting, Visualization-Lead, Writing - original draft-Lead, Writing - review & editing-Lead; Austeja Dumarkaite: Conceptualization-Supporting, Data curation-Equal, Investigation-Lead, Methodology-Supporting, Project administration-Equal, Writing - review & editing-Supporting; Auguste Nomeikaite: Investigation-Supporting, Project administration-Supporting, Writing - review & editing-Supporting; Gerhard Andersson: Funding acquisition-Supporting, Methodology-Supporting, Resources-Equal, Supervision-Supporting, Validation-Supporting, Writing - review & editing-Supporting; Evaldas Kazlauskas: Conceptualization-Supporting, Funding acquisition-Lead, Investigation-Equal, Methodology-Equal, Project administration-Equal, Resources-Equal, Supervision-Lead, Validation-Lead, Writing - review & editing-Equal.
Financial support
This work was supported by the European Regional Development Fund according to the supported activity ‘Research Projects Implemented by World-class Researcher Groups’ under grant no. 01.2.2-LMT-K-718 (project no. 01.2.2-LMT-K-718-03-0072).
Competing interests
All authors declare no competing interests.
Ethical standards
The study was approved by the Psychology Research Ethics Committee of Vilnius University (reference no. 2021-03-22/61). All participants gave informed consent to participate in the study and for the results to be published. The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees and with the Helsinki Declaration of 1975, as revised in 2008. The trial was registered at www.clinicaltrials.gov (NCT04817995: https://clinicaltrials.gov/ct2/show/NCT04817995, 30 March 2021).
Open access
Comments
No Comments have been published for this article.