Who benefits from individual placement and support? A meta-analysis

Abstract

Aims

Individual placement and support (IPS) is an evidence-based service model to support people with mental disorders in obtaining and sustaining competitive employment. IPS is increasingly offered to a broad variety of service users. In this meta-analysis we analysed the relative effectiveness of IPS for different subgroups of service users both based on the diagnosis and defined by a range of clinical, functional and personal characteristics.

Methods

We included randomised controlled trials that evaluated IPS for service users diagnosed with any mental disorder. We examined effect sizes for the between-group differences at follow-up for three outcome measures (employment rate, job duration and wages), controlling for methodological confounders (type of control group, follow-up duration and geographic region). Using sensitivity analyses of subgroup differences, we analysed moderating effects of the following diagnostic, clinical, functional and personal characteristics: severe mental illness (SMI), common mental disorders (CMD), schizophrenia spectrum disorders, mood disorders, duration of illness, the severity of symptoms, level of functioning, age, comorbid alcohol and substance use, education level and employment history.

Results

IPS is effective in improving employment outcomes compared to the control group in all subgroups, regardless of any methodological confounder. However, IPS was relatively more effective for service users with SMIs, schizophrenia spectrum disorders and a low symptom severity. Although IPS was still effective for people with CMD and with major depressive disorder, it was relatively less effective for these subgroups. IPS was equally effective after both a short and a long follow-up period. However, we found small, but clinically not meaningful, differences in effectiveness of IPS between active and passive control groups. Finally, IPS was relatively less effective in European studies compared to non-European studies, which could be explained by a potential benefits trap in high welfare countries.

Conclusions

IPS is effective for all different subgroups, regardless of diagnostic, clinical, functional and personal characteristics. However, there might be a risk of false-positive subgroup outcomes and results should be handled with caution. Future research should focus on whether, and if so, how the IPS model should be adapted to better meet the vocational needs of people with CMD and higher symptom severity.

Introduction

Employment is key to improve community functioning and mental health in people with mental illnesses (Drake and Wallach, 2020). Work fosters a sense of pride and self-esteem, offers financial independence, provides coping strategies for psychiatric symptoms and ultimately facilitates the process of recovery (Dunn et al., 2008). However, depending on the diagnosis, only between 14 and 33% of the working-age adults (18–65 years old) with mental illnesses are employed, which is substantially lower than the general population (Marwaha et al., 2007; Kozma et al., 2010; Hakulinen et al., 2020). Therefore, ongoing support in obtaining and sustaining competitive employment is needed for people with mental illnesses to create a strong and inclusive labour market. Individual placement and support (IPS) is the most effective rehabilitation programme to help people with mental illnesses into competitive employment (Modini et al., 2016; Metcalfe et al., 2018a).

IPS was originally developed to support people with severe mental illness (SMI) in achieving competitive employment. IPS is based on eight basic principles (Becker and Drake, 2003; See Box 1). The overall effectiveness of IPS is well-established for people with SMI (Modini et al., 2016; Metcalfe et al., 2018a). Because of its success, IPS is also increasingly offered to people with other diagnoses, such as common mental disorders (CMD), affective disorders, post-traumatic stress disorders (PTSD) and substance use disorders (SUD) (Bond et al., 2019). Results indicated the beneficial effects of IPS for people with PTSD and SUD (Bond et al., 2019). However, mixed indications of the effectiveness of IPS for people with CMD (including affective disorders) were found (Hellström et al., 2021; Probyn et al., 2021). The reasons for the diminished effectiveness of IPS for CMD are unclear, because a consistent definition of CMD is lacking. Diagnostic criteria for labelling CMD differ between studies, but most studies define CMD to include affective and/or anxiety disorders, of varying duration of illness (Vollebergh et al., 2001; Steel et al., 2014; De Vries et al., 2016).

Box 1. The eight basic principles of IPS (Becker and Drake, 2003)

Employment outcomes in IPS programmes also vary between service users with different clinical, functional and personal characteristics, such as symptom severity, substance use, involuntary hospitalisation, social functioning, work experience, education level, duration of illness, age and age of onset of psychiatric disorder (Catty et al., 2008; Marwaha et al., 2009; Campbell et al., 2010; Luciano et al., 2014; Fyhn et al., 2020; Christensen et al., 2021). However, these moderating effects have been inconsistent across studies, resulting in ambiguity about the effectiveness of IPS in different subgroups. As IPS has been increasingly expanded to different populations, it is timely to investigate how well the effectiveness of IPS generalises to new target groups.

Therefore, in this meta-analysis we analysed the relative effectiveness of IPS for different subgroups of service users as reported in randomised controlled trials of IPS. We assessed the relative effectiveness of IPS by examining study-level outcomes for subgroups of studies with different diagnostic, clinical, functional and personal characteristics using sensitivity analyses of subgroup differences (Borenstein and Higgins, 2013). This is the first meta-analysis that specifically focused on the relative effectiveness of IPS in different target groups with a focus on both target groups with and without SMI. This gives some unique insights into the relative effectiveness of IPS, and valuable addition to the recent contributions about this topic in comparable reviews (i.e. Bond et al., 2019; Probyn et al., 2021). The meta-analysis addressed the following research questions:

1. 1. How does the effectiveness of IPS differ between subgroups of service users with distinct clinical, functional and personal characteristics?

2. 2. What is the relative effectiveness of IPS for specific diagnostic subgroups of service users with CMD, SMI, schizophrenia spectrum disorders and mood disorders?

Materials and methods

Our meta-analysis followed the latest PRISMA guidelines (Page et al., 2021). Our protocol was preregistered in PROSPERO (CRD42020220080).

Search strategy

We identified records through searches in PubMed, PsycInfo and Cochrane of peer-reviewed journals until July 2019. The search was based on terms related to specific primary diagnoses (e.g., schizophrenia, mood disorder, anxiety disorder, but also CMD and SMI), IPS and other vocational rehabilitation programmes and competitive employment (see online Supplementary materials 1). We found additional references through reference lists of identified studies and systematic reviews.

Study selection process

The included studies meet the following criteria:

Participant population

We included studies that investigated people who were diagnosed with any mental disorder, as determined by DSM-III to DSM-5 (American Psychiatric Association, 1997, 2000, 2013) or ICD 10–11 criteria (World Health Organization, 2016, 2019). Participants without mental disorders or at risk of developing mental health problems were excluded from the meta-analysis.

Study design

We included all randomised controlled trials that evaluated the effectiveness of IPS compared to at least one control condition in the meta-analysis.

Intervention

We included studies investigating a treatment arm comprised of IPS as a stand-alone intervention, not augmented with another active intervention, such as cognitive remediation or social skills training, confirmed by an IPS fidelity assessment, receiving at least ‘fair’ fidelity. For studies investigating the effectiveness of both IPS and IPS augmented with another intervention, we only included the IPS-only arm in the analyses and excluded the IPS augmented with another intervention- arm from the analysis.

Comparison

The control group could be any other vocational service or a passive control group (i.e., service as usual or waiting list).

Outcomes

The study reported competitive employment outcomes.

Two authors (LdW & CC) independently executed study selection, including both title and abstract screening and full-text screening. Disagreements of the full-text selection process were resolved by consensus.

Data extraction

We extracted study details, participant characteristics, treatment variable, outcomes and study design data from all studies in this meta-analysis. Author LdW executed data extraction and discussed and resolved uncertainties with CC. Details about the data-extraction are presented in online Supplementary materials 2.

Data synthesis

Assessment of outcomes

The included studies reported a variety of competitive employment outcomes. Therefore, we focused on three outcome measures of competitive employment that were reported by at least ten studies, the minimum number to provide outcomes with sufficient statistical power in meta-analyses (Jackson and Turner, 2017; Borenstein et al., 2021): (1) Competitive employment rate (i.e., the proportion of participants competitively employed for at least one day during the study period); (2) Job duration (i.e., days, weeks or months competitive employed during the study period); (3) Wages (i.e., total earnings from competitive employment during the study period).

Assessment of study design and region

The included studies also differed in study design, which might affect outcomes: studies compared IPS with a variety of control groups and outcomes were analysed over different follow-up periods. Furthermore, previous research also indicated regional differences (i.e., European versus non-European studies) in the effectiveness of IPS (Drake et al., 2019). Therefore, we analysed the study outcomes within specific subgroups based on these three confounding factors as follows: (1) type of control group: studies with an active control group encompassing treatment as usual combined with any other vocational services versus studies with a passive control group with treatment as usual and no primary focus on improvement of vocational functioning; (2) follow-up duration: an assessment period of 12 months or less versus more than 12 months; (3) Region: European studies versus non-European studies.

Assessment of moderators of outcomes

In order to answer our two research questions, we selected moderators of study outcomes from the included studies. The selection was based on the identification of relevant moderators analysed in previous studies and the availability of extractable raw data of these moderators in at least ten of our included studies (Borenstein et al., 2021).

Assessment of diagnostic subgroups . For this study, we assessed subgroups of SMI or CMD based on diagnosis, duration of illness and inclusion criteria of studies (see Table 1). These three criteria were partly based on previous literature (i.e., Steel et al., 2014; De Vries et al., 2016). However, due to the lack of a consistent definition of CMD, we pragmatically translated these criteria based on the availability of data in the included studies. If studies met none of the three criteria we labelled them as ‘unclear’ and did not include these studies in the analysis. We were also able to include SSD and major depressive disorder as separate moderators. We divided these moderators into subgroups of studies in which the majority (i.e., >50%) of the study sample was diagnosed with the specific diagnosis and subgroups of studies in which the minority was diagnosed with the specific disorder (see Table 1).

Table 1. Operationalisations of moderators

^a BPRS, Brief Psychiatric Rating Scale; CAPS-IV, Administered PTSD Scale for DSM-IV; DSM-IV or −5, Diagnostic and Statistical Manual 4 or 5; DTS, Davidson Trauma Scale; GAF, Global Assessment of Functioning; GAS, Global Assessment Scale; HADS, Hamilton Anxiety and Depression Scale; HDRS, Hamilton Depression Rating Scale; ICD-10, International Classification of Diseases – 10; MADRS, Montgomery-Åsberg Depression Self Rating Scale; MHI-5, Mental Health Inventory – 5; MINI, Mini-International Neuropsychiatric Interview; PANSS, Positive and Negative Symptom Scale; PCL-5, PTSD checklist for DSM-5; SANS, Scale for the Assessment of Negative Symptoms; SAS-II, Simpson Angus Scale – II; SCAN, Structured Clinical Assessment in Neuropsychiatry; SCID, The Structured Clinical Interview for DSM-5; SOFAS, Social and Occupational Functioning Assessment Scale; SF-12, Short Form Health Survey 12; UPSA, UCSD performance-based skills assessment; WHO-DAS 2.0, World Health Organization Disability Assessment Scale 2.0.

Clinical, functional and personal characteristics . We identified eight other moderators of outcomes: duration of illness, the severity of symptoms, level of functioning, age, comorbid alcohol and substance use, education level and employment history. We assessed these moderators at baseline and operationalised those into subgroups. Subgroups were generally assessed based on the available data in the included studies in this meta-analysis, in order to achieve equally distributed subgroups. Criteria for the operationalisation into subgroups of each moderator are described in Table 1.

Risk of bias assessment

We assessed the risk of bias for each study through the Cochrane Collaboration risk of the bias assessment tool (Higgins and Green, 2008). Potential bias (i.e., high, low or unclear) is assessed as a judgment for individual elements from five domains (selection, performance, attrition, reporting and other bias). Author LdW rated all studies and CC independently rated the risk of bias of 50% of all studies. The inter-rater reliability (Cohen's kappa; McHugh, 2012) was substantial (κ = 0.61; Landis and Koch, 1977) and disagreements were resolved through consensus.

Statistical analysis

Meta-analytic procedure

Meta-analyses were conducted using RevMan 5.3 (The Nordic Cochrane Centre, 2014). We assessed the effectiveness of IPS by analysing differences between IPS and the control group over the study period by calculating the standardised mean difference (d) for continuous outcomes (i.e., job duration and wages) and the odds ratio (OR) for categorical outcomes (i.e., employment rate). For studies reporting multiple outcome assessments for related outcome measures, we pooled the effect sizes into an overall effect size. We used random-effects models, weighted by the method of inverse variance (Higgins and Green, 2008). The magnitude of effect sizes was assessed based on the criteria described by Chinn (2000). Statistical heterogeneity was assessed by calculating the I² statistic (Higgins and Thompson, 2002). We performed the overall meta-analysis within separate subgroups based on the type of control group, follow-up duration and geographical region. We controlled for the potential influence of these factors using an analysis of subgroup differences (Borenstein and Higgins, 2013). One study had both an active and passive control group (Mueser et al., 2004), and multiple studies both followed service users after ⩽ 12 months and >12 months of follow-up. For the overall meta-analysis we pooled the effect sizes for all control groups or follow-up assessments within the study into one overall effect size, but we analysed both effect sizes separately during the analysis of subgroup differences, controlling for methodological confounders.

Calculating moderating effects

We analysed moderating effects through a sensitivity analysis of subgroup differences (Borenstein and Higgins, 2013), in which we compared subgroup outcomes with high levels or presence of the moderator versus those with low levels or absence of the moderator (see Table 1). Furthermore, the positive or negative influence of specific subgroups on employment outcomes was assessed by investigating which subgroups' confidence intervals of treatment effect exceeded the upper (‘positive’ influence) or lower (‘negative’ influence) bound of the confidence interval of the overall effect size of treatment effect.

Outliers and publication bias

We addressed the potential influence of outliers (i.e., if the confidence interval [CI] of an individual study outcome exceeded the CI of the overall effect size) by comparing the overall effect size of the outcome, including the outliers, with the overall effect size when outliers are removed through an analysis of subgroup differences. Potential publication bias was detected by visual inspection of funnel plots.

Results

Study flow

Of the 1333 records retrieved through database search and reference tracking, 1170 records were excluded after the title and abstract screening. Of the remaining 163 reports, 115 reports were excluded after full-text selection (see Fig. 1 for reasons of exclusion). The remaining 48 reports reported the results of 32 studies.

Fig. 1. Flow chart selection studies conform Prisma guidelines.

Study characteristics

As shown in Table 2, the 32 studies included 3818 participants receiving IPS and 3847 participants receiving a control intervention. The mean age of the aggregated sample (n = 7665) was 38.9 years (study range: 20.4–51.0); 44.1% of the participants were female. A total of 3454 (45.1%) participants were diagnosed with a schizophrenia spectrum disorder (SSD), and 2587 (33.8%) had a main diagnosis of major depressive disorder (MDD). The remaining 1624 (21.2%) had other diagnoses, such as anxiety disorder, PTSD, SUD or personality disorders. Twenty studies met the criteria for SMI and five studies met the criteria for CMD.

Table 2. Characteristics of included studies

^a References of reports of included studies: A. Areberg and Bejerholm (2013); B. Bejerholm et al. (2015); C. Bejerholm et al. (2017); D. Bond et al. (2007); E. Bond et al. (2013); F. Bond et al. (2015); G. Burns et al. (2007); H. Burns and Cathy (2008); I. Kilian et al. (2012); J. Drake et al. (1996); K. Christensen et al. (2019); L. Clark et al. (1998); M. Davis et al. (2012); N. Davis et al. (2014); O. Davis et al. (2018a); P. Davis et al. (2018b); Q. Drake et al. (1999); R. Dixon et al. (2002); S. Drake et al. (2013); T. Metcalfe et al. (2018b); U. Erickson et al. (2021); V. Gold et al. (2006); W. Hellström et al. (2017); X. Howard et al. (2010); Y. Heslin et al. (2011); Z. Hoffmann et al. (2012); AA. Hoffmann et al. (2014); AB. Jäckel et al. (2017); AC. Killackey et al. (2008); AD. Killackey et al. (2019); AE. Latimer et al. (2006); AF. Lehman et al. (2002); AG. Lones et al. (2017); AH. Michon et al. (2014); AI. Mueser et al. (2001); AJ. Mueser et al. (2004); AK. Mueser et al. (2014); AL. Oshima et al. (2014); AM. Poremski et al. (2017); AN. Reme et al. (2019); AO. Tsang et al. (2009); AP. Tsang et al. (2011); AQ. Twamley et al. (2012); AR. Viering et al. (2015); AS. Waghorn et al. (2014); AT. Wong et al. (2008); AU. Zhang et al. (2017)

^b A, active control group; E, excellent fidelity; F, Fair fidelity; G, good fidelity; NR, Not Reported; P, Passive control group;.

^c IPS-15 item scale (Bond et al., 1997): item scale range: 15–75; Fidelity ratings: <55 = No IPS; 56–65 = Fair fidelity (F); >65 = Good fidelity (G); IPS-25 item scale (Bond, Peterson, Becker and Drake, 2012): item scale range: 25–125; Fidelity ratings: <74 = No IPS; 74–99 = Fair fidelity (F); 100–114 = Good fidelity (G); 115–125 = Exemplary fidelity (E).

Twenty-one studies compared IPS with an active control group and 12 studies compared IPS with a passive control group (including one study with both a passive and active control group). The overall study attrition rate (i.e., lost to follow-up) was 16.3% and only two studies reached a ‘high’ attrition rate exceeding 40%. Instruments for fidelity assessment differed between studies (see Table 2), but the majority of the studies (75.0%) achieved at least ‘good’ IPS programme fidelity.

Quality assessment

Quality assessment is reported in Fig. 2. Overall we found low levels of selection and attrition bias, but relatively higher levels of performance and detection bias. The majority of studies (81.3%) reported a low risk of selection bias (i.e., random sequence generation and allocation concealment). In the majority of studies (53.1%) the participants and personnel were not blinded or information about blinding was unclear (43.4%). However, given the nature of the intervention and the study design, it was generally not feasible to achieve proper blinding of participants, so a certain level of performance bias was inevitable. In nine studies (29.0%) the outcome assessors were not blinded, indicating a high risk of detection bias. This is a relatively large number of studies with a high risk of detection bias, compared with the other risk of bias domains. However, the outcomes we used in our meta-analysis (i.e., employment rate, job duration and wages) are objective outcome measures and not sensitive for the interpretation of the outcome assessor. Therefore, this might not have a large influence on the study outcomes. In most studies (67.7%) we found a low risk of attrition bias (i.e., incomplete outcome data). Five studies reported other sources of bias: three studies reported baseline differences between IPS and the control group that potentially influenced outcomes and one study indicated the potential influence of allegiance bias because specialists favoured one intervention over the other and one study had a low fidelity score during the first part of the study which may have negatively influenced study outcomes at the start of the study. There were no indications of selective outcome reporting in any of the 32 studies.

Fig. 2. Cochrane risk of bias assessment.

Overall meta-analysis

Thirty-one studies reported employment rate outcomes (see Table 3). A higher percentage of IPS participants (48.8%) than control group participants (28.3%) were employed during follow-up, showing small effect sizes (OR  =  2.62 [2.37–2.89], p < 0.01). Outcomes were moderately heterogeneous (I² = 74% [67–80%]; p < 0.01). The overall effect sizes of the employment rate were not influenced by the follow-up duration. However, we did find more favourable employment rate outcomes for IPS in non-European studies compared with European studies (χ² = 10.54; p < 0.01) and in studies that compared IPS with an active control group compared with a passive control group (χ² = 10.77; p < 0.01).

Table 3. Overall meta-analysis of outcomes

^a Some studies have used multiple follow-up assessments or have multiple treatment arms. Therefore, some studies are included in the analysis of both follow-up subgroups and one study compared IPS with both an active and passive control group. Therefore, the total amount of studies and sample sizes analysed in each comparison is sometimes lower than the sum of studies analysed in both follow-up subgroups.

^b Summary statistics for each of the three employment outcomes are assessed as follows: Employment rate: number and percentage of people in competitive employment at the follow-up assessment; Job duration: percentage of time within the study period that participants are employed; Wages: monthly salary in euros during the study period.

^c d > 0 and OR > 1 indicates outcomes are beneficial for IPS compared to the control group; d < 0 and OR < 1 indicates outcomes are beneficial for the control group compared to IPS.

^d Magnitude of effect (Chinn, 2000): Not clinically relevant [N]: d > −0.2 – <0.2; OR > 0.67 – <1.5; Small effect [S]: d ⩽ −0.20 and >−0.50 – ⩾0.20 and <0.50; OR ⩽ 0.67 and >0.29 – ⩾1.5 and <3.5; Medium effect [M]: d ⩽ −0.50 and >−0.80 – ⩾0.50 and <0.80; OR ⩽ 0.29 and >0.20 – ⩾3.5 and <5; Large effect [L]: d < −0.80 – >0.80; OR < 0.20 – >5.

Twenty-three studies reported job duration outcomes. Results indicated that IPS participants were longer employed than those in the control group during follow-up, showing small effect sizes (d = 0.41 [0.30–0.52], p < 0.01). Outcomes were moderately heterogeneous (I² = 77% [69–83%]; p < 0.01). The overall effect sizes of job duration were not influenced by the type of control group, follow-up duration or region.

Fifteen studies reported outcomes of wages. Results indicated that IPS participants earned more wages during the study period than those in the control group, though effect sizes were small (d = 0.31 [0.19–0.44], p < 0.01). Outcomes were moderately heterogeneous (I² = 65% [51–76%]; p < 0.01). The overall effect sizes of wages were not influenced by the type of control group, follow-up duration or region.

Moderating effects on overall outcomes

Sensitivity analysis outcomes were reported in Table 4 and Fig. 3. We excluded some moderators in the sensitivity analysis of job duration and wages, because these moderators were reported in less than ten studies.

Fig. 3. Overview effect sizes of outcomes for each moderator.

Table 4. Sensitivity analysis of moderating effects on the outcomes

^a d > 0 and OR > 1 indicates outcomes are beneficial for IPS compared to the control group; d < 0 and OR < 1 indicates outcomes are beneficial for the control group compared to IPS.

^b Underlined moderators were significant moderators of outcome.

^c Magnitude of effect: Not clinically relevant [N]: d > −0.2 – <0.2; OR > 0.67 – <1.5; Small effect [S]: d ⩽ −0.20 and >−0.50 – ⩾0.20 and <0.50; OR ⩽ 0.67 and >0.29 – ⩾1.5 and <3.5; Medium effect [M]: d ⩽ −0.50 and >−0.80 – ⩾0.50 and <0.80; OR ⩽ 0.29 and >0.20 – ⩾3.5 and <5; Large effect [L]: d < −0.80 – >0.80; OR < 0.20 – >5.

^d Summary statistics for each of the three employment outcomes are assessed as follows: Employment rate: number and percentage of people in competitive employment at the follow-up assessment; Job duration: percentage of time within the study period employed that participants are employed; Wages: monthly salary in euros during the study period.

We found significant favourable employment rate outcomes in the IPS group compared with the control group in all subgroups. However, IPS showed more favourable outcomes in studies targeting participants with SMI than studies targeting CMD (χ ² = 10.79; df = 1; p < 0.01). These differences between both subgroups were specifically explained by differences in employment rates in the control group (i.e., 38.3% in the CMD subgroup versus 23.9% in the SMI subgroup; χ ² = 28.84; df = 1; p < 0.01). We also found more favourable outcomes for IPS in subgroups with a majority diagnosed with SSD (χ ² = 18.24; df = 1; p < 0.01), as well as in subgroups with a minority diagnosed with MDD (χ ² = 5.36; df = 1; p < 0.05), and in subgroups with a lower baseline level of symptoms (χ ² = 20.48; df = 1; p < 0.01).

Figure 3 shows all subgroup outcomes. Subgroups with SMI, the majority diagnosed with SSD, low symptom severity, and low comorbid alcohol and substance use problems at baseline had a positive influence on the relative effectiveness of IPS.

None of the potential moderators included in the sensitivity analysis had significant effects on either job duration and wages.

As the type of control group and the region in which the study is executed significantly influenced employment rate outcomes, above-mentioned moderating effects might be explained by an overrepresentation of a specific moderator in one of the subgroups based on the region or type of control group. However, chi-square analyses did not find any indications of overrepresentation in any of these subgroups. We could therefore not explain any moderating effects by regional differences or type of control group.

Assessment of outliers and publication bias

We found two negative outliers and six positive outliers for employment rate, three negative outliers and three positive outliers for job duration and one negative and one positive outlier for wages. Removing these outliers did not positively or negatively influence the study outcomes.

The funnel plots are presented in online Supplementary materials 3. For all outcomes (employment rate, job duration and wages) we found no indications of publication bias.

Discussion

This meta-analysis investigated the relative effectiveness of IPS for different subgroups based on diagnostic, clinical, functional and personal characteristics. Overall, we found that IPS is effective in improving employment outcomes regardless of sample characteristics. However, we did find that IPS was relatively less effective in supporting service users into competitive employment in European studies and in studies comparing IPS to a passive control group. Furthermore, we found that IPS was relatively more effective for people with SMIs, compared with CMD. We also found more favourable outcomes of IPS in subgroups in which the majority was diagnosed with a schizophrenia spectrum disorder (SSD), in which the minority was diagnosed with major depressive disorder (MDD), subgroups with a low baseline symptom severity, and subgroups with a low baseline level of substance and alcohol use problems. These subgroup effects could not be explained by an overrepresentation of non-European studies or an active control group within any subgroup. Despite the fact that we found overall effectiveness of IPS for all subgroups, the issue remains that in many studies the majority of service users that received IPS remain unemployed. This highlights the need for continuous refinement of the IPS model.

The fact that IPS was less effective on employment rate outcomes in European studies is most probably explained by the relatively extensive welfare systems with a disability benefit structure in most European countries. The risk of losing steady income from disability benefits after finding competitive employment (i.e., the ‘benefits trap’) might discourage service users from seeking employment (Burns and Cathy, 2008; Metcalfe et al., 2018a).

We also found that IPS was relatively more effective on employment rate outcomes when compared with an active control group than when compared with a passive control group. We found a slightly larger employment rate in the IPS group (i.e., 50.1% vs 48.3%) but a slightly smaller employment rate in the control group (i.e., 26.7% vs. 29.4%) when IPS was compared with an active control group. However, differences in both IPS and control groups were negligible and therefore we could not give any clinical meaningful explanations for the differences between both types of the control groups.

Our findings that IPS is relatively more effective for people with SMI and SSD and relatively less effective for people with MDD and CMD are in line with previous research (Hellström et al., 2021). The main explanation for differences in the effectiveness of IPS between CMD and SMI subgroups is that employment rate outcomes in the control group were larger in the CMD subgroup, whereas the outcomes were equal in the IPS group. Previous research found even more favourable employment outcomes in the control group for people with mood disorders or less severe thought disorders (Campbell et al., 2010; Jonsdottir and Waghorn, 2015). This may indicate that people with CMD also benefit from other vocational rehabilitation interventions. Nevertheless, our meta-analysis indicates that IPS leads to more favourable employment outcomes for people with CMD compared to any control group, and these indications of effectiveness were also found in another recently published meta-analysis (Probyn et al., 2021).

Another possible explanation for the differences between CMD and SMI subgroups is the fact that IPS is originally developed for people with SMI who are generally supported by professionals working in integrated treatment teams, whereas service users with CMD are often supported in different healthcare settings. Previous research indicated that the level of organisational characteristics, such as the type of clinical practice, service intensity and quality of mental health treatment could be an important prerequisite for successful implementation (Lockett et al., 2018). This explanation is supported by two studies that conducted IPS for service users with CMD (Hellström et al., 2017; Poremski et al., 2017) in another healthcare setting for this group. These differences can also partially be explained by the fidelity scores. Only fifty per cent of all studies that evaluated the effectiveness of IPS for people with CMD reached a fair fidelity. In contrast, 89% of the studies that evaluated the effectiveness of IPS for people with SMI achieved good or excellent fidelity. Given the fact that better fidelity scores lead to better outcomes in IPS (Bond et al., 2012; Kim et al., 2015; Lockett et al., 2016; De Winter et al., 2020), this might be an important explanation for the differences in outcomes between SMI and CMD subgroups. In addition to fidelity, other important factors, such as the quality of healthcare services and the intensity of employment support might also be relevant topics for further investigation. Therefore, poorer outcomes for people with CMD compared to people SMI might partially be explained by specific challenges in the implementation of IPS in a different healthcare setting, which underlines our recommendation to adapt implementation for specific subgroups.

We also found more favourable indications of the effectiveness of IPS for people with lower symptom severity and lower comorbid substance and alcohol use problems at the start of IPS. This is in line with previous studies which also indicated that lower symptom severity increased the odds of being employed for people who received any type of vocational rehabilitation (Michon et al., 2005; Campbell et al., 2010; Nygren et al., 2013). This might be explained by the fact that a lower symptom severity frees up more time for and focus on the adequate job support, because less focus on symptom stabilisation and intensive treatment programmes is needed.

The positive influence of low symptom severity on the effectiveness of IPS may contradict the superior outcomes of IPS for people with SMI compared with CMD, as SMIs are generally associated with higher symptom severity. However, four out of the five studies (80%) that reported outcomes of IPS for people with CMD had a high symptom severity at baseline. Therefore, symptom severity and the severity of illness are not interrelated in this meta-analysis.

This meta-analysis had several limitations. First, all our findings were analysed on a study level and subgroups were based on aggregated scores or percentages of the whole study sample. This analysis provides an overarching overview of the influence of specific service users' characteristics on the effectiveness of IPS, but does not reflect on the specific variability of individual client level characteristics or outcomes. However, despite this limitation, this meta-analysis gives valuable insights toward better understanding of making effective adaptations in the implementation of IPS in real-world settings. Analysis of outcomes on a study level inevitably leads to heterogeneity of outcomes because the context and setting in which the studies are executed differ (Ioannidis, 2008). Furthermore, this meta-analysis only focused on the effectiveness of IPS as a stand-alone intervention within a mental health population, in order to achieve a relatively homogeneous sample of studies. As a consequence, we did not include a number of relevant studies that exclusively investigating IPS with an add-on intervention (e.g., McGurk et al., 2015; Tsang et al., 2016) or studies focused on populations with a high risk of developing mental disorders (e.g., Sveinsdottir et al., 2019). This also partially explained the lack of available study data to investigate the influence of other relevant moderators (such as cognitive functioning). Second, some of our sensitivity analyses, in which we investigated moderating effects, were based on a relatively low number of studies. This might have limited the generalisability of outcomes for subgroups based on a small number of studies. Another limitation is that 12 (37.5%) of our included studies are conducted more than 10 years ago. During that time IPS was executed in a societal setting, that was applicable at that time, with most probably other welfare policies or treatment practices than implemented nowadays. Another potential limitation is the fact that the broad variety of studies might influence the interpretation and representativeness of some moderating effects. Our included studies investigated target groups with different diagnoses and clinical characteristics and were therefore in some cases using different assessment instruments. This was specifically the case in the moderating effects of the severity of symptoms and level of functioning. We tried to solve this issue by using normative data based on representative target groups that matches with each included study. However, this inevitably leads to heterogeneity in the outcomes. Interpretation of the findings on the influence of symptom severity and level of functioning on employment should therefore be handled with caution. Finally, we have executed a relatively high number of sensitivity analyses based on a relatively low number of studies. This increases the chance of false-positive outcomes and alpha inflation (Wang et al., 2017). We should therefore consider the results of this meta-analysis as exploratory and the findings suggesting potentially valuable trends for improving IPS for different target groups.

Overall this meta-analysis has shown that IPS is implemented for a wide variety of service users: IPS is effective for different subgroups, regardless of distinct diagnostic, clinical, functional and personal characteristics. However, future research should focus on the implementation of IPS for people with CMD and higher symptom severity. It is important to investigate whether, and if so, how to make more effective adaptations in the implementation of IPS to better meet the vocational needs of these groups.