A Feasibility Study Investigating Mechanisms of Change in Public Mental Health Dialectical Behaviour Therapy Programmes

Abstract

Few studies have investigated the feasibility of researching brief forms of dialectical behaviour therapy (DBT) for borderline personality disorder (BPD) in public mental health settings. This study aimed to provide preliminary evidence for the feasibility of implementing DBT over a 6-month period for BPD symptoms within Australian public mental health services. Of the 79 participants with BPD recruited, 62 commenced and 24 participants completed the therapy. The participants attended one of three outpatient DBT programmes and completed measures of BPD symptoms, DBT skills-use, and difficulties with emotion regulation at baseline and after 6 months of treatment. A major challenge with feasibility identified was the high attrition rate (61%). However, for completers there were significant improvements in BPD symptoms, DBT skills-use, and difficulties with emotion regulation. These effect sizes were used to estimate the sample sizes needed by future larger trials of brief DBT for BPD in public health settings. The implementation of brief DBT for BPD patients within a public mental health outpatient setting, appears to result in significant reductions in BPD symptoms. However, further exploration of strategies to reduce drop-out rates are required.

Introduction

Borderline personality disorder (BPD) is a complex mental disorder which involves a pervasive instability of interpersonal relationships, self-image, affect, and marked impulsivity (American Psychiatric Association, 2013). Marsha Linehan (1993) developed dialectical behaviour therapy (DBT) by combining cognitive behaviour therapy (CBT) and Zen Buddhist practices to treat individuals with personality disorders and interpersonal difficulties by improving emotion regulation and adaptive coping strategies. Since its development, DBT has accumulated the most empirical support for the treatment of BPD, when compared with alternative approaches such as CBT, schema therapy, and mentalisation-based therapy (Cristea et al., 2017; Stoffers et al., 2012).

An important emerging question in the literature concerns dosage. Standard DBT involves a combination of weekly 1-h individual therapy, weekly 2–2.5-h group skills training, as needed between session telephone consultations, as well as weekly team meetings of clinicians over a 12-month period (Linehan, 1993). Over 40 randomised controlled trials have established the efficacy of this standard dosage of DBT over 12 months (e.g., Linehan et al., 2015; McMain et al., 2009; van den Bosch, Koeter, Stijnen, Verheul, & van den Brink, 2005; Verheul, Van den Bosch, Koeter, Ridder, et al., 2003). There is also evidence of the effectiveness of this standard 12-month protocol in north American public mental health settings (Comtois, Elwood, Holdcraft, Smith, & Simpson, 2007 (N = 24); Harley, Baity, Blais, & Jacobo, 2007 (N = 24)) and a 14-month protocol in an Australian public mental health setting (Walton, Bendit, Baker, Carter, & Lewin, 2020). However, public mental health services do not have the resources or clinician time to adhere to the strict treatment protocols which form the foundation of randomised controlled trial (RCT) methodology (Shafran et al., 2009). Factors, such as high staff turnover, lack of funding for administration, and lower levels of training and supervision, may result in failures of public health DBT programmes to achieve the expected results (Swales, Taylor, & Hibbs, 2012). There is a need therefore to explore whether shorter protocols with lower dosages of treatment may still be effective within public health services.

While the efficacy of DBT over the first 6 months of treatment has not been as thoroughly investigated, there is some evidence to suggest that this length of time is sufficient to improve symptoms of BPD in randomised controlled trials (Koons et al., 2001). There is even some preliminary evidence that 6 months of DBT may result in significant reductions in a range of psychiatric symptoms, suicidal behaviour and non-suicidal self-injury, visits to the emergency department, psychiatric admissions, and days in hospital in a New Zealand public mental health service (Brassington & Krawitz, 2006 (N = 10)), and Australian public mental health services (Prendergast & McCausland, 2007 (N = 11); Pasieczny & Connor, 2011 (N = 41)). Such preliminary evidence has resulted in a recent increase research focusing upon establishing the efficacy of DBT over a 6-month period (McMain et al., 2018).

While evidence of the effectiveness of DBT for BPD with a 6 month, rather than a 12-month dosage, is an important research question, a preliminary question that first needs to be answered is the feasibility of introducing a 6-month DBT for BPD protocol within an Australian public health system? There is a need for more translational research examining the challenges and feasibility of implementation in real-world settings. If the challenges of implementation are too burdensome, then an intervention becomes irrelevant no matter how strong the literature is for the efficacy of the intervention. The question about the feasibility of implementation of DBT for BPD is particularly salient. A team of clinicians must be trained (usually 10 intensive training days, 6 months of practice, and an exam) and provided with weekly supervision. Each clinician must be released to devote approximately 4 h per week to the programme, for a minimum of 12 months. Additional clinical hours (approximately 20) are necessary for programme coordination. These features limit the extent to which comprehensive DBT can be implemented with fidelity in routine clinical settings, where limited funding and high staff turnover are common, and adherence to treatment models are not routinely monitored (Feigenbaum et al., 2012). The lack of organisational and administrative support, together with the limited funding and low prioritisation of introducing new, evidence-based practice in mental health, have all been cited as common barriers to implementation of DBT in public mental health at a global level (Carmel, Rose, & Fruzzetti, 2013; Flynn et al., 2020; King, Hibbs, Saville, & Swales, 2018; Swales et al., 2012).

There is a need therefore for research into the feasibility of implementing a 6-month DBT intervention for adults with BPD within an Australian adult public health service. Feasibility studies are critical to the preparation of large, randomised control trials by helping to identify potential challenges and solutions to implementation as well as identifying potential causal mechanisms in the change process (Pearson et al., 2020). Typically, feasibilities studies measure the ease or effectiveness of recruitment strategies, the willingness of participants to be randomised, adherence/compliance rates, as well as the standard deviation of the outcome measure to estimate sample sizes needed for future studies (Arain, Campbell, Cooper, & Lancaster, 2010).

While there is recent preliminary evidence of the feasibility of DBT for BPD within a rural adolescent Australian public health service in partnership with a non-government organisation (NGO) (Lakeman, Emeleus, Davies, & Anderson, 2021), it is unknown what the similarities and differences are to introducing a DBT for BPD service within a metropolitan adult public health service with a typical service delivery that does not involve a partnership with an NGO. Moreover, while the Lakeman et al. study did well in recording the attrition rate and reasons for drop-outs, it did not assess the recruitment rate and challenges with recruitment. This, of course, is crucial information for mental health services considering introducing a DBT programme as significant challenges in recruitment may make implementation unfeasible. Furthermore, the Lakeman et al. study did not include a measure of treatment adherence. Training and supervision of therapists is resource intensive and so an important consideration in feasibility is how well this translates to treatment adherence. Given these limitations, there is a need for a feasibility study of the implementation of a 6-month DBT programme within a metropolitan public mental health service.

The present study, therefore, used data from a pre–post trial of a 6-month DBT for BPD implemented in an Australian public mental health system to provide feasibility data that may assist future researchers in designing and implementing larger effectiveness trials. The first aim was to gain preliminary evidence of feasibility by investigating recruitment and attrition rates of BPD participants, as well as treatment adherence rates among public health clinicians involved in the 6-month DBT trial. The second aim was to gain preliminary estimate of effect sizes for three important outcomes of DBT for BPD (BPD symptoms, DBT skills-use, difficulties with emotion regulation) to assist future researchers with sample size estimates for larger trials.

While suicidality is a common outcome measure of BPD interventions, suicidality is only one of nine symptoms of BPD and is also present in disorders such as major depressive disorder (American Psychiatric Association, 2013). It is therefore not unique to BPD patients. Including a comprehensive measure of BPD symptoms ensures that the overall score captures not only suicidality, but also other clinically important symptoms of BPD. This enables for a more accurate representation of improvements in BPD symptoms, rather than focusing on specific isolated symptoms. The Borderline Symptom List 23 (BSL-23) has been commonly used to determine the severity of BPD symptoms, and effectively discriminates between BPD patients and healthy controls (Kleindienst, Jungkunz, & Bohus, 2020). The BSL 23-item was chosen as a measure of BPD symptoms in this study due to both its brevity, and inclusion of items which investigate all the DSM-5 criteria for BPD (rather than just suicidality and self-harm, which has often been the focus of previous research). Its inclusion also allowed us to compare the effectiveness of this 6-month metropolitan public mental health adult DBT programme with the effectiveness of the 6-month rural public mental health adolescent DBT programme (Lakeman et al., 2021).

Given that DBT aims to reduce symptoms of personality disorders by reducing emotional dysregulation (Linehan, 1993), difficulties with emotional regulation has become an important outcome measure. According to a systematic review by Harvey et al. (2019), the Difficulties with Emotion Regulation Scale (DERS) is one of the most common scales used to measure the effectiveness of DBT in improving emotion regulation. Therefore, in order to maximise the utility of this feasibility study for future trials, the DERS (Gratz & Roemer, 2004) was used in this study. Finally, while not the focus of the systematic review, Harvey et al. (2019) also noted that given the importance of mastery in skill development for enhancing emotion regulation abilities and self-efficacy, studies should incorporate data on mastery of DBT skill development. These recommendations are consistent with the findings of a recent review by Rudge, Feigenbaum, and Fonagy (2020) that difficulties with emotion regulation and DBT skills-use are among the most thoroughly investigated DBT specific outcome measures. As such, in addition to including the DERS we also included the DBT ways of coping checklist (DBT-WCCL) (Neacsiu, Rizvi, & Linehan, 2010) given that it is a common measure of DBT specific skills to improve emotion regulation.

This study, therefore, aimed to test the feasibility of a brief 6-month dose of DBT for BPD in a public health service by assessing (1) recruitment rates, (2) attrition rates, (3) treatment/research adherence rates, and (4) preliminary effect sizes in changes in BPD symptoms, difficulties in emotion regulation and DBT skill use to estimate the required sample sizes for future trials of effectiveness in public health services.

Method

Participants

The participants were adult outpatients with BPD participating in DBT programmes run through four public hospitals based across two South East Queensland Health districts. Metro North Hospitals included the Royal Brisbane and Women's Hospital and The Prince Charles Hospital. Metro South Hospitals included the Princess Alexandra Hospital and Logan Hospital. Ethical approval to collect data from both Metro North and Metro South Hospitals was obtained from the Prince Charles Hospital Human Research Ethics Committee and the University of Queensland (reference number: HREC/17/QPCH/172). As ethical approval was not approved for accessing diagnostic information of participants, details regarding comorbidity are not described.

Participants include those who consented to participate, completed 6 months of therapy, and consistently completed the required questionnaires (N = 24). Age ranged between 18 and 50 years (M = 27.58, SD = 9.91). The sample was predominantly female, unmarried, and almost half were unemployed. All participants had at least the equivalent of a high school education. The demographic characteristics of the sample are provided in Table 1.

Table 1 Demographics for the Sample of Completers

Note: RBWH = Royal Brisbane and Women's Hospital; TPCH = The Prince Charles Hospital; PAH = Princess Alexandra Hospital. N = 24.

Self-Report Measures

Borderline Symptom List 23-Items (BSL-23)

The BSL-23 (Bohus et al., 2009) was used to evaluate the subjective distress of BPD patients. Scored on a 5-point Likert Scale, each item ranges from 0 (not at all) to 4 (very strong), with the final score reflecting the average of the total. In the current sample, the internal reliability was excellent, α = 0.92. A higher score indicates a greater BPD symptom severity.

Difficulties in Emotion Regulation Scale (DERS)

The DERS (Gratz & Roemer, 2004) is a 36-item self-report measure designed to detect clinical levels of emotion dysregulation. It includes the subscales (1) non-acceptance of emotional responses (non-acceptance), (2) engaging in goal-directed behaviour when experiencing negative emotions (goals), (3) difficulties controlling impulses (impulse), (4) lack of emotional awareness (awareness), (5) clarity of emotional responses (clarity), and (6) limited access to emotion regulation strategies perceived to be effective (strategies). Items are measured on a Likert scale which ranges from a score of 1 (almost never) to 5 (almost always). Higher scores indicate greater difficulties in emotion regulation.

In the current sample, the DERS total score had a Cronbach's α of 0.92, and the internal reliability of the subscales were as follows: Non-acceptance α = 0.84; Goals α = 0.82; Impulse α = 0.79; Awareness α = 0.73; Clarity α = 0.68; and Strategies α = 0.76. It is of note that the clarity subscale is below the minimum effect size of 0.70. However, it has been shown that a sample of 30 is often required to achieve this minimum desired effect size (Bujang, Omar, & Baharum, 2018). Taking this into account, the Clarity subscale was not omitted, and the results of this subscale were interpreted with caution.

DBT Ways of Coping Checklist (DBT-WCCL)

The DBT-WCCL is a 59-item self-report measure assessed on a 4-point Likert scale ranging from 0 (never used) to 3 (regularly used). The full scale comprises two main subscales: DBT skills-use and dysfunctional coping. Only the DBT skills-use subscale is relevant for the current study and comprises 38 items, each assessing the frequency of skills-use over the past month. An example item includes, ‘Focused on the good things in my life’. The internal consistency in the current sample was excellent, α = 0.92. A higher score indicates greater levels of skills-use.

Recruitment and Design

Study participants were recruited via existing protocols for entry into one of the Queensland Health DBT programmes participating in the research. Referrals were emailed or faxed to the DBT coordinator of the relevant hospital by case managers, acute care teams, or doctors when the need for DBT was identified. Referrals were then discussed by the relevant DBT team during weekly consultation meetings. Exclusion criteria were assessed using the referral form filled out by the referring clinician, and included: intellectual functioning that would limit understanding of treatment (or a documented IQ < 80); a current involuntary treatment order; acute psychosis; dangerous behaviour (e.g. violence, aggression) that might endanger the well-being or safety of other group members; severe substance abuse; a medical condition that could interfere with engaging in or benefiting from the programme; or an assertion from the patient that they were unwilling to cease psychotherapy from another source.

Patients who met the programme's criteria were invited to attend an initial screen, where the Borderline Personality Disorder Severity Index IV (BPDSI-IV: Giesen-Bloo, Wachters, Schouten, & Arntz, 2010) was administered as a structured clinical interview by one of the DBT team who had received training in its administration, to confirm a BPD diagnosis. Acceptance into the programme required an overall BPDSI-IV score of at least 15 as well as impulsivity in at least two areas of functioning and/or non-suicidal self-injury (criteria four and five of BPD in the DSM-5). Other inclusion criteria included residing in the relevant hospital's catchment area and being eligible for public mental health services.

Patients who met criteria for the DBT programme were invited to sign the consent form to participate in the study by their individual DBT therapist. All participants were given a copy of the research participant information consent form and invited to sign the consent form. It was made clear that participation in the DBT programme was not contingent upon participating in the research, and the patient would be free to withdraw from the research component of the programme at any time.

Recruitment rate was assessed by analysing the number of participants who completed the BPDSI-IV, were accepted into DBT and signed the consent form to participate. The final attrition rate was calculated by the percentage of participants who did not complete the 6-month measures of DBT after having completed pre-treatment measures. To measure adherence, the skills group facilitators completed locally developed adherence checklists after every skills group. These measures were developed by the third co-author and included a checklist of each of the core components of required in each DBT skills group (mindfulness, homework review, checklist of planned skills, and provision of homework for the week).

DBT Programme

Treatment was the first 6 months of Marsha Linehan's 12-month comprehensive DBT programme, as outlined in the DBT Skills Training Manual (Marsha Linehan, 2015). The DBT programmes were run through the Outpatient Mental Health Service of each hospital, and so were conducted in a community setting. As per the standard DBT protocol, each participant was allocated an individual therapist and undertook a 4-week ‘pre-commitment’ phase of therapy prior to commencing skills group. The aim of this period was to establish a therapeutic alliance, orient the participant to the treatment, set personal goals and treatment targets, encourage use of a diary card, and introduce basic skills. This phase comprised weekly 1 h appointments with the participants individual DBT therapist. All therapists were also available to provide phone coaching between face-to-face sessions during this time. Pre-commitment is designed to increase motivation for treatment and readiness for change.

As per the comprehensive DBT programme model, each hospital's DBT programme consisted of a 1.5-h weekly therapist consultation group, 2.5 h of weekly skills training groups, weekly 1-h individual therapy, and access to 24-h telephone support between sessions with either individual therapists (during business hours) or with the geographically relevant acute care team (after hours). Psychiatrist reviews were required at 6-week intervals, or as required, for treatment monitoring and medication review. No structured therapy was provided during these appointments. Similar to previous studies of DBT (McMain et al., 2009), participants who missed four or more consecutive skills groups or individual therapy sessions were withdrawn from the programme.

Three of the four DBT skills modules (Distress Tolerance, Emotion Regulation, and Interpersonal Effectiveness) were run consecutively over 6 months, for 8 weeks, 10 weeks, and 7 weeks, respectively. Mindfulness skills and were integrated into the first 2 weeks of each of the other three modules, forming the fourth skills module. DBT groups were open to new patients at the beginning of each skills module, provided there was a place available. Participants started on the module available at the time. The modules were not completed in any specific order.

Therapists

The DBT programmes were run by a multidisciplinary team of psychiatric nurses, psychologists, social workers, psychiatrists, psychiatry registrars, and occupational therapists. All therapists had undergone a minimum of 4 days basic training, and some had received 10 days of intensive training from Linehan's training organisation, Behavioural Tech. No specialist DBT supervision was received, other than peer supervision during the weekly 1 h 30 min consultation meeting.

Statistical Analyses

Statistical analyses were conducted using version 27 of the statistical package for the social sciences (SPSS). Prior to analysis, raw data were screened for entry errors and patterns of missing data. Little's MCAR test indicated that the data were missing completely at random, with the number of missing values ranging from 0 to 2 per variable. Thus, the analyses proceeded with the original data. Subsequently, reverse scored items on the DERS were recoded.

Changes in symptoms of BPD, DBT skills-use and emotion regulation from baseline to 6 months of treatment were assessed using a series of paired samples t-tests (p < .05, two-tailed). Cohen's d effect size was calculated using Wiseheart's (2014) calculator, which accounts for the correlation between repeated measures based on Morris and DeShon (2002). The effect sizes were defined as small (d = .20–.49), medium (d = 0.50–0.79), and large (d ≥ 0.80) (Cohen, 1988). A statistical correction was not utilised, to protect against the chances of a Type II error. This is in line with cautions against correcting for multiple comparisons in exploratory psychological research where effects may be small or moderate (Cabin & Mitchell, 2000; Moran, 2003; O'Keefe, 2003). Sample size calculations were conducted using G*Power 3.1.9.7 for repeated measures ANOVA for within and between group differences using two groups and two measurement time points, and with alpha = .05, power = 0.95 and Cohen's f = Cohen's d/2 (Cohen, 1988). Sample size estimates were made for both assumptions of the control group showing no change across time, as well as showing half the effect of the intervention as done in other feasibility studies (Russell, Strodl, Connolly, & Kavanagh, 2021).

Results

Feasibility

Recruitment and attrition

Participants were recruited between June 2018 and June 2020. As the DBT programmes have been running between 8 and 15 years (depending on the hospital), awareness of their existence is widespread, and referrals were frequent. However, 17 (21.55%) of the participants who were recruited to the study dropped out prior to study commencement. The reason for drop-out was not recorded.

Figure 1 illustrates the flow of participants from initial identification to the completion of the programme. There was a higher-than-expected drop-out rate of 61.3% at the time of treatment enrolment. It is notable that the final proportion of the study was conducted during the COVID-19 pandemic. Only 3 of the 24 participants in this study completed the DBT programme in 2020, during the COVID-19 pandemic. It is also notable that while neither the Royal Brisbane Hospital nor the Princess Alexandra Hospital ceased Skills groups, the groups were conducted by Zoom between April and June 2020. There are anecdotal accounts of increased stress among both practitioners and patients throughout the COVID-19 pandemic, especially during periods when Brisbane citizens were required to adhere to lockdown procedures.

Figure 1. Flow diagram for the study.

Treatment Adherence

To calculate adherence, the elements completed within each group was divided by the number of standard DBT skills group elements. The groups had an overall adherence rate of 77%, with 81.3% adherence in the distress tolerance module, 69.2% in the interpersonal effectiveness module, and 80.6% adherence in the emotion regulation modules.

Effects of Treatment from Baseline to 6 Months

Change in BPD symptoms and future sample size estimation

As predicted, there was a statistically significant decrease in BPD symptoms from baseline (M = 2.80, SD = 0.65) to 6 months of treatment (M = 2.39, SD = 0.74), t ₍₂₃₎ = –2.71, p = .013, with a medium effect size, d = 0.55. Using this effect size, and the above-mentioned G*Power parameters, and assuming no change in the control group, the minimum required sample size needed for conducting an RCT against no treatment would be 30. If the control group had an improvement that was half the size of the experimental group, the required sample size would rise to 314. Assuming a similar attrition rate (i.e. drop-outs, asked to leave and measures lost) experienced in this study (i.e. 37%), then an intention-to-treat analysis would require a sample size of 48 and 499, respectively.

Change in difficulties in emotion regulation and future sample size estimation

As predicted, there was a significant decrease in the DERS from baseline (M = 3.77, SD = .45) to 6 months of treatment (M = 3.25, SD = 0.53), t ₍₂₃₎ = –4.60, p < .001, with a large effect size, d = –1.06. Using this effect size, and the above-mentioned G*Power parameters, and assuming no change in the control group, the minimum required sample size needed for conducting an RCT against no treatment would be 10. If the control group had an improvement that was half the size of the experimental group, the required sample size would rise to 86. Again, assuming a similar attrition rate experienced in this study (i.e. 37%), then an intention-to-treat analysis would require a sample size of 16 and 137, respectively.

In the case where researchers would like to base sample size estimation upon changes in the subscales of the DERS, these results are presented in Supplementary Appendix A.

Change in skills-use and future sample size estimation

As predicted, DBT skills-use significantly increased from baseline (M = 1.45, SD = .48) to 6 months of treatment (M = 1.94, SD = .32), t ₍₂₃₎ = 5.53, p < .001, with a large effect size, d = 1.13. Using this effect size, and the above-mentioned G*Power parameters, and assuming no change in the control group, the minimum required sample size needed for conducting an RCT against no treatment would be 10. If the control group had an improvement that was half the size of the experimental group, the required sample size would rise to 76. Again, assuming a similar attrition rate experienced in this study (i.e. 37%), then an intention-to-treat analysis would require a sample size of 16 and 121, respectively.

Discussion

While a solid evidence base has been accumulated for the full 12-month dosage of DBT for BPD, the implementation of this dosage is resource intensive and challenging within a public health setting. There is, therefore, a need to investigate first the feasibility, and second the effectiveness of briefer versions of DBT for BPD within public mental health settings, especially within the Australian context. While larger-scale research is warranted, feasibility studies are an important first step in identifying the challenges of implementation in real-world settings. This study aimed, therefore, to gain preliminary evidence of feasibility by investigating recruitment and attrition rates of BPD participants, and treatment adherence of public health clinicians across 6 months of a DBT trial. Second, preliminary effect sizes for three key outcomes of DBT for BPD (BPD symptoms, skills-use, and emotion regulation) were calculated to estimate needed sample sizes for future effectiveness trials. These findings may help guide the implementation of DBT services in Australian public mental health services.

Recruitment

The current study highlighted some challenges in conducting research within an outpatient mental health setting. While challenges with recruitment and attrition have also been noted another feasibility study implemented within the same public health service (Beames et al., 2020), the reasons for these challenges in this feasibility study appear to be different and so emphasise the importance of conducting feasibility studies for specific populations and interventions within public mental health services. In the case of this study, although the recruitment of participants was not difficult, due to the large number of referrals to the service, almost a quarter (21.5%), dropped out prior to the start of the DBT programme. It is difficult to compare our recruitment rate with other published studies as other similar feasibility studies have not measured recruitment rates. As such, this study adds to the literature by providing evidence of the significant challenges with recruitment to DBT programmes in public mental health services. One likely reason for this is the length of the waiting list. At the Royal Brisbane Women's Hospital, participants waited up to 18 months for a place on the programme, by which time many patients no longer wish to engage in DBT. This both demonstrates the need for DBT within the public health system, and the current lack of resources to meet this need.

Attrition

The level of attrition in this study was high, at 61.3%. The high proportion of dropout within the initial stages of the DBT programme meant that there were several participants for whom demographic data was not recorded. Consequently, the presence of missing data meant that it was difficult to accurately analyse the characteristics of those whom where more likely to drop out of the study. It is noted, however, that attrition is a commonly cited issue in studies of BPD, and previous DBT research in this population have attrition rates ranging between a low of 17% (Linehan et al., 2006) and 48% (Priebe et al., 2012). The severity of BPD symptoms in the current sample may provide some insight into the high attrition rate. DBT can be a challenging and confronting treatment which requires a high level of patient commitment. Individuals who are experiencing a high level of distress may be more likely to drop out, especially within the first 6 months where skills are learnt and are often not implemented effectively. Previous studies of BPD patients have found that severe psychopathology often leads to high levels of drop out (e.g. Barnicot et al., 2012). This may therefore provide one insight into the difficulties with retention within a public health service.

Previous research conducted in the UK has also noted that public health services often have high levels of staff turnover, lower levels of therapist training and less supervision than RCTs (Swales et al., 2012). These factors were prominent within the current DBT programmes. Therapists were also employed as case managers for Queensland Health and had a significant case load in addition to the patients being treated in the DBT programmes. It was often difficult to balance the additional work required to be a DBT therapist with case manager responsibilities. While no qualitative data was collected, anecdotal accounts suggest that therapists felt they were not provided an adequate amount of time to manage their case work and their DBT therapist roles, and there were frequent self-reported accounts of staff burn-out. The additional training the DBT therapists received in the treatment of BPD, often led to the allocation of BPD patients who were not involved in the DBT programme to a case manager who was also a DBT therapist. Due to the high-risk and complex psychiatric profile of many of these patients, this often led to additional constraints being placed upon the amount of time which could be dedicated to DBT-related activities.

The challenges of running the DBT programme amplified the difficulties associated with research in public mental health settings. For instance, the constraints on DBT therapist time led to difficulties administering and collecting self-report measures. The lack of administrative support meant many DBT therapists were individually responsible for tracking the progress of the patients within the research. However, this task was often not feasible given time constraints. The COVID-19 pandemic also made tracking patient progress throughout the programme difficult. This was due to many of the appointments being conducted via telehealth, such that there were limited opportunities to administer the self-report measures which have been analysed in this study.

It is also notable that 11 (17.7%) of the participants included in the drop-out analysis were still officially part of the DBT programme when the study was concluded. This is largely due to the complications associated with the COVID-19 pandemic, which both highlighted and exacerbated the challenges associated with running and researching DBT within the public system. Many DBT therapists were either re-deployed or ceased taking part in the DBT programmes when the pandemic was announced. The variability in technological resources between sites meant that only one service was equipped to run the DBT groups via Telehealth during the initial stages of quarantine. As a result, the DBT programme at one site was temporarily discontinued, and many participants were provided with treatment as usual (case management and psychiatric reviews) during the initial stages of the pandemic. The increased burden on staff, re-allocation of resources towards managing COVID-19, and lack of existing DBT-trained clinicians led to the decision to conclude the study prior to completion. These issues highlight the need to establish viable action plans which may be used to manage national or local emergencies and ensure patients continue to receive evidence-based care.

Treatment Adherence

The groups had an overall adherence rate of 77%, with 81.3% adherence in the distress tolerance module, 69.2% in the interpersonal effectiveness module, and 80.6% adherence in the emotion regulation modules on the locally developed adherence measure. Direct comparisons with previous research are difficult due to measurement inconsistencies. For instance, recent research in a British routine care setting used the DBT Adherence Coding Scale (DBT ACS; Linehan & Korslund, 2003) to measure individual therapist adherence, finding that only 44–48% of sessions were adherent (Harned, Schmidt, Korslund, & Gallop, 2021). Comparatively, previous research in the Australian public health system using measures similar to the current study found adherence varied between 75 and 90% (Pasieczny & Connor, 2011). Overall, therefore, the current findings are largely in line with expected levels of adherence and are considered high considering the number of therapists, variable levels of training, and different sites.

It must be noted, however, that the effectiveness of DBT may be impacted by the level of therapist training, and by extension, adherence to the standard DBT protocol (Pasieczny & Connor, 2011). Patients who receive treatment from therapists who undergo 10 days of training compared with those who experience basic training have been shown to demonstrate greater reductions in suicide attempts and non-suicidal self-injury (Pasieczny & Connor, 2011). Due to challenges with resourcing, many of the therapists involved in the present study received basic 4-day training, which may have lowered adherence to the DBT protocol and impacted the effectiveness of the intervention. While the current study did not report upon level of training due to frequent staff changes, it would be beneficial to track both staff retention levels and staff training levels in future research.

Change in BPD Symptoms

BPD symptoms significantly decreased following 6 months of DBT. The finding was similar to previous research evaluating 12-month DBT programmes conducted within the public mental health systems of Canada (Robinson et al., 2018) and Ireland (Flynn et al., 2017). The patients in these investigations (Flynn et al., 2017; Robinson et al., 2018) improved on the BSL-23 with a large effect size after 6 months of DBT (Cohen's d = 0.86−0.88) compared with the medium effect size found in the current study (Cohen's d = 0.55). Given that the baseline levels on the BSL-23 were lower in these two studies than in our study, it is possible that the lower effect size was representative of the difficulties in implementing DBT with a sample with more severe symptoms. These studies also both had external sources of funding either through the private hospital system in Canada (Robinson et al., 2018) or via a suicide prevention NGO (Flynn et al., 2017). The resources may have led to greater levels of staff support for the programmes, and thus increased clinician time, adherence to the DBT protocol, therapist engagement with participants, and reduced burn-out compared with the current programme. The most similar study to compare is the 6-month DBT trial conducted by Lakeman et al. (2021) in a rural public mental health service within the same state. The participants in this study had a similar severity of symptoms at the start of therapy, using the BSL-23, but experienced a greater reduction in symptoms over the 6-month intervention (Cohen's d = 0.95). Research to date is unclear on the difference in effectiveness between DBT applied to adults compared with adolescents. It is possible adolescents would have a greater capacity to benefit from skills training, as the length of their difficulties in emotion regulation would not have been as prolonged. More research is required, however, to verify this point.

The projected sample size of 30 required for comparison against a treatment-as-usual group (i.e. no active treatment) is promising. Between 10–12% of outpatient psychiatric patients and 20–22% of inpatient psychiatric patients have a diagnosis of BPD, meaning that recruitment would likely not be difficult (Ellison, Rosenstein, Morgan, & Zimmerman, 2018). Given the administrative demands of an RCT, however, it is likely that lack of time and resources described in this study would form a barrier to further research within this area.

Change in Emotion Regulation

The patient's perceptions of their ability to regulate emotions, significantly improved following 6 months of DBT. This is in line with the findings of Barnicot et al. (2016), where participants with BPD significantly decreased difficulties in emotion regulation on the DERS at 6 months. Studies within other clinical populations have also found emotion regulation improved significantly. Neacsiu, Bohus, and Linehan (2014) examined a cohort of 33 patients with depression and/or anxiety, finding emotion regulation improved with a large effect size (Cohen's d = 1.86). Similarly, an evaluation of 171 patients with alcohol use disorder found that scores on the DERS improved with a large effect size (Cohen's d = 1.17; Cavicchioli et al., 2020) over 3 months of DBT. Based upon the current findings, and the strong effects found in previous investigations, it is likely that future larger-scale studies will not require many participants to examine the effectiveness of the DBT programmes more thoroughly. An estimate of only 10 participants is required to compare a control group to DBT based upon the results of the current investigation.

Improvement in Skills-Use

DBT skills significantly increased over 6 months of treatment. The average increase in DBT skills-use over this period was 34% which is similar to the 36% increase in skills-use found by Robinson et al. (2018) over 6 months. Furthermore, 12 months of a comprehensive DBT programme (Neacsiu et al., 2010) showed an average increase of 32% in the skills-use, which is similar to the findings from the present study. Thus, the effectiveness of the DBT skills-use within the current DBT programme is comparable to previous research conducted within both controlled and public health settings. It is notable that the large effect size found within this study led to the estimation that only 10 participants would be required for a future RCT comparing DBT to treatment-as-usual. Thus, it is likely future research would be possible if the resources were provided to support the administrative component of the trial.

Limitations and Future Directions

There are several limitations in the present study, which must be acknowledged. First, participants were not screened for comorbid disorders. Comorbidities such as depression, anxiety, and post-traumatic stress disorder are common in BPD (Tomko, Trull, Wood, & Sher, 2014) and have the potential to influence the rate at which patients improve (Fitzpatrick, Bailey, & Rizvi, 2020; Hittman, 2019). Future research would benefit from examining the effect of comorbid disorders on the results by controlling for their effects using self-report measures. This may allow the effectiveness of the intervention to be examined for patients with different comorbidities, and so determine whether the intervention may need to be tailored for specific disorders.

The small sample size of completers was also a limitation of this feasibility study. This was predominantly due to the high drop-out rate. A larger sample would have allowed for more comprehensive statistical analyses to be performed, such as controlling for potential covariates such as demographics and baseline BPD symptom severity (Seow, Page, & Hooke, 2020). Furthermore, as the participants included those who persisted through the first 6 months of therapy, it is possible they were more engaged with the programme and consequently, more likely to practice and benefit from skills. As such, the results may not be representative of the sample initially recruited to the study. As the characteristics of those who dropped-out were not analysed, it is difficult to conceptualise the individual characteristics which may have contributed to the high attrition rate. This was largely due to the presence of missing data among those who did not complete the study (especially those who left within the initial stages of the programme) and was reflective of the challenges maintaining follow-up and data entry procedures throughout the trial within a public health setting with multiple staff. Finally, in an attempt to be helpful to future researchers, we calculated effect sizes and estimated sample sizes required for appropriately powered larger intervention trials. However, it is important to highlight that this practice is debated in the literature with some commentators cautioning against the limitations of such approaches and instead advocating for considerations of minimum important differences (Sim, 2019).

The challenges associated with implementing the current study highlight the need for resource allocation to the DBT programme. Due to lack of research funding for research assistance, the researchers were unable to follow up with clinicians to assess their caseloads on a regular basis. Clinicians would describe burn-out and high caseloads, often stating these were linked to the high staff turnover rates, staff absences and lack of clinician time for the research or DBT in team meetings. These meetings highlighted one of the major difficulties in the DBT programmes and in DBT research — the scarcity of clinician time. It would be beneficial to ensure DBT therapists’ caseloads are reduced to accommodate the time they spend on the DBT programme. The introduction of funding for an administrative assistant would also be likely to improve the feasibility of the programmes. There is a significant amount of administrative work associated with running DBT, including ensuring participants have materials (such as skills manuals, handouts, and associated resources), as well follow-up and input of questionnaire data. Having one person dedicated to these tasks would ensure therapists time is spent on improving patient care and reduce the level of burn-out due to difficulties managing clinical and administrative duties. A further suggestion would be to provide comprehensive training to therapists who wish to be part of the DBT team. This would not only ensure that the intervention is in line with the evidence-based treatment manual, but also increase the knowledge and effectiveness of public health service members. They may then be better equipped to take on BPD cases while part of the DBT programme and within other roles within the service.

Conclusion

BPD is a complex and chronic mental disorder which has proved challenging to treat. This study suggested that the effectiveness of DBT compared with treatment-as-usual over a 6-month period may be established with reasonably low levels of participants. Future research in public health services, however, will need to develop strategies to mitigate high drop-out rates be feasible. Future investigations within public health services are likely to require additional funding and administrative support to ensure success.