Evaluation of a Pathway team for homeless mental health in-patients

Abstract

Aims and method

The Pathway model is an enhanced care coordination model for homeless people in hospital. We aimed to evaluate the first attempt to apply it on psychiatric wards, which started in 2015 in South London. We developed a logic model which expressed how the Pathway approach might work. Two predictions from this model were tested, using propensity scores and regression to estimate the effect of the intervention among people who were eligible for it.

Results

The Pathway team theorised that their interventions would reduce length of stay, improve housing outcomes and optimise the use of primary care – and, more tentatively, reduce readmission and emergency presentations. We were able to estimate effects on length of stay (−20.3 days; 95% CI −32.5 to −8.1; P = 0.0012) and readmission (a non-significant reduction).

Clinical implications

The marked reduction in length of stay, explicable in terms of the logic model, constitutes preliminary support for the Pathway model in mental health services.

People experiencing homelessness – even if ‘hidden’ – have worse health and higher mortality than people in stable accommodation.¹ Their use of health services also differs: they make less use of scheduled and preventive care,² but attend accident and emergency departments five times as often, are admitted three times as often and stay three times longer when in hospital.³ Since 2009, the UK homeless healthcare charity Pathway and the Faculty for Homeless and Inclusion Health have promoted the development of teams that work to improve the health and housing outcomes of homeless people in acute hospitals.⁴ One such team developed to cover the acute hospitals that form part of the King's Health Partners (KHP) Academic Health Sciences Centre and serve the London boroughs of Lambeth and Southwark. In 2014, funding was made available to extend this service to the adult mental health wards for Lambeth and Southwark, which are situated at the Maudsley Hospital and at Lambeth Hospital. These hospitals are operated by South London and Maudsley NHS Foundation Trust (SLaM), also part of KHP. As this was the first attempt to apply the Pathway approach to mental health services, some funding was also provided for evaluation.

Method

Logic modelling

The KHP Pathway Homeless Team was treated as a programme: that is, a set of resources and activities deployed with particular aims. Rather than use a ‘black-box’ approach to programme evaluation, we used logic modelling.⁵ This approach turns the situated knowledge of those involved in a programme into a basic theory of programme operation, which is represented as a set of inputs, activities, outputs and outcomes, together with the links between these. Our logic model was developed iteratively through several group interviews with programme staff, with the developing logic model progressively serving as the focus for discussion.

Initial planning of quantitative analyses

Having developed the model, we then considered how to test whether and how the team was having the effects anticipated. The new team's capacity was small relative to the likely number of homeless patients, making it feasible to base an evaluation partly on a comparison between patients who received the Pathway intervention and patients who would have been eligible but did not. As regards outcome measures, we were able to analyse the estimated effects of the programme on length of stay and on readmission – both of which, looking ahead to the results of logic modelling, were potentially important outcomes. (Because of the possibility that patients might move outside the SLaM catchment area and this might not be captured, we analysed both readmission to psychiatric wards operated by SLaM and, separately, readmission to any psychiatric ward in England.)

Data-set construction

Both these analyses were based mainly on data taken from the Clinical Record Interactive Search (CRIS) system, which is an anonymised version of SLaM's electronic patient record database, containing a mixture of structured and unstructured data,^6,7 and which may be accessed using the SQL programming language. We extracted all those admissions (a) to adult acute psychiatric wards (those treating 18- to 65-year-olds), (b) that included a period on a ward serving Lambeth or Southwark and (c) that led to a discharge between 1 February 2015 (the day that the team started operating) and 31 March 2018. The complete data-set was created by supplementing each combination of patient identifier, admission date and discharge date with (a) a variable indicating whether the patient concerned had received the Pathway intervention during the admission; (b) the length of stay (after removing days on extended leave from the ward); (c) the date of the first readmission to a SLaM psychiatric bed within the study period; (d) the date of death (where applicable); (e) the date of the first recorded move to an address outside the four boroughs served (where applicable); and (f) basic clinical and demographic data, including data on service use in the preceding period. The only variable not taken from CRIS was the date of first admission to any English mental healthcare provider: these data were taken from Hospital Episode Statistics (HES) and were made available under a data sharing agreement between SLaM's Biomedical Research Centre and NHS Digital. All data were extracted on 7 June 2018. SLaM readmissions were included up to 25 May 2018; HES data were only available up to 31 March 2017, so the analysis of national readmissions (see below) is based on a subset.

Selection of observations for the analysis

The entire data-set comprised a set of hospital admissions, during each of which an individual patient either received the Pathway intervention or did not. So that we could use these data to estimate the effects of the Pathway intervention, we used selection as the main way of minimising the effect of differences between those who were included in the analysis and had received the intervention (the treated observations – ‘treatment' being used here in its statistical sense of receiving an intervention whose effect is of interest) and those who were included in the analysis and did not receive the intervention (the control observations).

The Pathway team had explicit eligibility criteria: admission to one of the relevant wards, having no address to which discharge could be arranged and having no care coordinator. The team would accept patients irrespective of right to statutory entitlements, nationality or local connection. However, it is possible that other, implicit factors also influenced referral and acceptance. Therefore, rather than simply selecting treated and control observations using the explicit criteria, we instead used mixed-effects logistic regression to estimate the probability of treatment (treatment here again being used in its statistical sense of receiving an intervention of interest). We then selected observations whose probability indicated that they had had a realistic chance of either receiving or not receiving the Pathway intervention (for details see below). This increases the overall similarity of case and control observations; but more specifically in the context of a regression analysis it means that any observed combination of covariate values is likely to occur among both case and control observations (the so-called common support condition).⁸ Satisfying the common support condition helps to avoid estimation bias.

We worked with the entire data-set, as described above. Because homelessness was not reliably recorded, we developed a measurement strategy based on SQL processing of free-text progress notes. First, we counted the number of occurrences of the terms ‘NFA’, ‘no fixed abode’, ‘homeless’, ‘eviction’ and ‘being evicted’ across all notes made during the admission. Second, we used a receiver operating characteristics curve with programme participation as the dependent variable to define the count to use as a cut-off to determine programme eligibility. The resulting categorical variable was entered into the mixed-effects regression, along with allocation of a care coordinator, and an interaction term between the two, as well as age, gender, diagnosis, whether or not the person was detained under the Mental Health Act during the admission, longest admission in the previous year and number of discharges in the previous 2 years. (We did not include ethnicity and marital status in this regression analysis or the analyses of length of stay and readmission because of the presence of missing data; however, they are shown in Tables 1 and 2). The probability of programme participation was estimated based on the fixed effects in the model and the modal estimate of the subject-level random effect. We selected the subset with 0.1 < P < 0.9, eliminating observations for which common support was likely to be lacking (see above).⁹ The outcome analyses below were based on this subset.

Table 1 Characteristics of all hospital admissions involving the Pathway team and all other admissions^a

HoNOS item 11, Health of the Nation Outcome Scales, item ‘Problems with living conditions’.

a. The sample comprises admissions that included a period on a general adult ward and/or psychiatric intensive care unit serving Lambeth or Southwark, and which ended in discharge between February 2015 and March 2018. There were 5876 admissions overall, comprising 321 patients treated by the King's Health Partners Pathway Homeless Team and 5555 who were not. The P-values in the table derive from two-tailed t-tests for continuous variables and from overall χ²-tests for categorical variables.

b. Informal legal status includes patients initially detained under section 136 or section 4 of the Mental Health Act, but who were not made liable to detention under another section.

c. ZZ99 is a dummy postcode used in healthcare information systems in the UK to indicate various forms of non-standard residential status, primarily homeless. The Clinical Record Interactive Search (CRIS) database identifies as indicating homelessness the presence in an address of a postcode including ZZ99 or in which the first line is recorded as ‘homeless’ or ‘no fixed abode’.

d. The homeless text words and phrases counted in progress notes recorded during the admission were ‘homeless’, ‘NFA’, ‘no fixed abode’, ‘eviction’ or ‘being evicted’.

Table 2 Comparison of treated observations with control observations^a

a. The sample comprised admissions that included a period on a general adult ward and/or psychiatric intensive care unit serving Lambeth or Southwark, which ended in discharge between February 2015 and March 2018, and which were also in the subset used for analysis, with referral probability in the range 0.1 < P < 0.9. The P-values in the table derive from two-tailed t-tests for continuous variables and from overall χ²-tests for categorical variables.

b. Informal legal status includes patients initially detained under section 136 or section 4 of the Mental Health Act, but who were not made liable to detention under another section.

c. ZZ99 is a dummy postcode used in healthcare information systems in the UK to indicate various forms of non-standard residential status, primarily homeless. The Clinical Record Interactive Search (CRIS) database identifies as indicating homelessness the presence in an address of a postcode including ZZ99 or in which the first line is recorded as ‘homeless’ or ‘no fixed abode’.

Length of stay

We constructed a linear regression of length of stay, using robust standard errors. Age, detained status, diagnosis and the length of the longest admission ending in the preceding year were included as covariates, because they had been found to have effects in a previous analysis in the same population;¹⁰ gender was included because it has been consistently found to be associated with length of stay in American studies.¹¹ A functional form for continuous variables was defined using fractional polynomial transformations¹² and an indicator variable was included for those who had had no admission in the preceding period.

Readmission to SLaM wards

For the analysis of local readmissions, periods out of hospital were censored at the point that address data indicated a move outside the area covered by SLaM. Deaths were also treated as censoring events. Diagnosis, legal status, the length of the index admission and the number of discharges in the preceding 2 years were included as covariates because they had been found to have effects in a previous analysis in the same population.¹³ Age and gender were included based on results of a systematic review.¹⁴ Reparameterisation of diagnosis into psychotic/personality disorder/other non-psychotic was performed to reduce the number of parameters entering the analyses to no more than the number of readmissions divided by ten, therefore reducing the risk of overfitting.^15,16 Multivariable fractional polynomial regression was used to select an appropriate functional form for age; length of stay seemed to be overinfluenced by a small number of observations, so was instead categorised. A full multivariable Cox regression model was used to estimate the effect of programme participation.

Readmission to any English psychiatric ward

For the analysis of HES readmissions, we further restricted the data-set to admissions leading to discharge up to 31 March 2017, as readmission data were not available after that date. Periods out of hospital were censored only on death, if this occurred. Because of the more limited number of readmissions and greater risk of overfitting, gender was not included in the full model, as it had previously been found to be unassociated with readmission in the same population,¹² and diagnosis was dichotomised as psychotic or non-psychotic. The analysis was otherwise identical to that of SLaM readmissions.

Ethics and consent

We treated our study as a service evaluation rather than as research, in that its primary goal was not to generate generalisable theoretical knowledge. The logic model was created by programme staff, some of whom subsequently contributed to the writing of this paper (Z.K., S.K., R.R.) and others of whom are acknowledged below. All CRIS data for the quantitative study are fully anonymised and may be used without consent. (CRIS was approved on this basis as a data-set for secondary analysis by the Oxfordshire Research Ethics Committee C (08/H0606/71).)

Results

Logic model

Four versions of the logic model were produced: during the 1st, 8th, 15th and 20th month of the project's operation. Fig. 1 shows the last of these, produced on the 19 October 2016, reflecting four cycles of discussion.

Fig. 1 The final logic model.

adm, admission; AE, accident and emergency department; CHaIN, Combined Homelessness and Information Link; EEA, European Economic Area; FTE, full-time equivalent; GP, general practitioner; LOS, length of stay; NRPF, no recourse to public funds; PJS, Patient Journey – SLaM's electronic patient record system; QOL, quality of life; S117, S136, sections 117 and 136 of the Mental Health Act 1983; Spine, the NHS Spine – a system providing a central record of NHS registration information.

The logic model makes use of the typical elements of inputs, activities, outputs and outcomes. It is a Z-design, with outputs of an initial stage (the results of an assessment for housing eligibility) entering into most of the subsequent activities of the team. Inputs are represented only by the staffing of the team (there were of course overhead costs and costs associated with the management of the team within the broader organisation). Activities may broadly be considered as the provision of expert support and liaison with other agencies such as general practitioners (GPs), community health services, social services, local authority housing departments, hostels, outreach teams and a wide range of community and voluntary sector services. However, the logic model contains several specific interventions. The model's outputs were housing, provision of alternative support, repatriation (for refugees, asylum seekers and other migrants) and reconnection, for example to primary care services.

Finally, the model – which was developed before any quantitative analysis – covers the hoped-for outcomes of the programme. These were considered more or less certain by respondents. Thus, solid arrows in Fig. 1 represent mechanisms that were thought to be more likely and dashed lines represent more tentative assertions. It was thought most likely that programme participation would reduce length of stay, would promote rehousing and housing stability and would promote the use of physical healthcare.

The quantitative analyses presented here therefore test only two small parts of this logic model, seeing whether the effects on length of stay and readmission are consistent with what was anticipated.

Descriptive data on programme participants and non-participants

In total, there were 5876 admissions involving time on general adult wards serving Lambeth or Southwark that ended between February 2015 and March 2018. Of these, 321 were referred to the team and 5555 were not. Table 1 shows the characteristics of all those hospital admissions during which the Pathway team had been involved and all those admissions in which the Pathway team was not involved, using t-tests for differences in means and χ²-tests for differences in the distribution of categorical variables. There were missing data for some variables, notably marital status and the Health of the Nation Outcome Scales (HoNOS) housing variable. Referred patients tended to be younger, were more likely to be male, less likely to come from any kind of Black ethnicity, more likely to have a non-psychotic diagnosis, less likely to be detained under the Mental Health Act and had spent less time in hospital. They had more evidence of housing difficulties based on address data, HoNOS scores and prevalence of text terms related to homelessness, and were more likely to have recorded residential mobility during the admission or the month after discharge. As expected from the referral criteria for the team, they were much less likely to have a recorded care coordinator during the admission. Over the study period, the median number of patients taken on by the team was nine per month (interquartile range 7–10). A linear regression of monthly counts with robust standard errors indicated no change in activity over time (P = 0.12).

Figure 2 shows the distribution of the estimated probability of treatment, comparing all those admissions in which the Pathway intervention was delivered (n = 321) and all those in which it was not (n = 5555). Of these, there were 280 admissions during which the Pathway intervention was delivered whose probability of treatment was in the range 0.1 < P < 0.9, and 269 admissions during which the intervention was not given and where the treatment probability was in the same range. These comprised the treated and control observations for the analyses of length of stay and readmission.  It should be noted that many control observations fall outside the range of interest: inclusion of these would likely have introduced bias.

Fig. 2 Estimated treatment probability comparing treated and control observations.

Table 2 is the analogue of Table 1, but compares only the treated and control observations, as defined above. Even after control selection, there were significant differences between treated patients and controls in the distribution of several variables: diagnosis, legal status, longest admission in the preceding year and three variables indicating homelessness or residential mobility. However, differences in age, gender and number of discharges in the preceding 2 years were not statistically significant.

Analysis of length of stay

Analyses of length of stay were based on the total sample of 549 observations. The estimate of the effect of programme participation without adjustment for other factors was −32.0 days (95% CI −45.9 to −18.1; P < 0.001). The full model included the log of the length of the longest previous admission and a linear effect of age, alongside dummy variables for gender, diagnosis, legal status on admission and whether there had been any day in hospital in the preceding year. The adjusted effect of programme participation was estimated as −20.3 days (95% CI −32.5 to −8.1; P = 0.0012). Of the covariates in the analysis, only the effects of diagnosis reached a conventional level of statistical significance (P < 0.0001).

Analysis of readmissions to SLaM wards

Again the analysis was based on the total 549 observations, which yielded 153 readmissions. The unadjusted hazard ratio for readmission was 0.69 (95% CI 0.49–0.97; P = 0.03). The adjusted hazard ratio was 0.82 (95% CI 0.58–1.17; P = 0.28), with only the effects of age (P = 0.0001) and number of previous admissions (P = 0.0002) being significant in this model.

Analysis of admissions to any English psychiatric ward

Because of limited national readmission data, this analysis was based on 200 of the treated observations and 199 of the control observations, a total sample of 399 observations, which yielded 74 readmissions. The unadjusted hazard ratio for readmission was 0.65 (95% CI 0.40–1.07; P = 0.09). The adjusted hazard ratio was 0.70 (95% CI 0.42–1.16; P = 0.16). Only diagnosis entered significantly into the adjusted model (P = 0.04).

Discussion

The evaluation findings presented here comprise a logic model for the KHP Pathway Homeless Team coupled with limited but promising evidence for programme outcomes that largely concur with that model. On the one hand, the assertion that the programme would reduce length of stay was entirely borne out by the quantitative data. On the other, the uncertainty that programme staff expressed regarding effects on readmission was shown to be justified. Although we do not include any element of economic evaluation at this stage, it would seem likely that the effects on length of stay alone would yield significant cost savings. The logic model expresses our best understanding of the mechanism of the effect on length of stay. Members of the team were able to delineate a series of more or less definite steps that they took in those scenarios that they typically encountered. Effects of the programme on length of stay would be expected to have arisen from the effect of such actions, whether this effect was the expedition of the provision of housing and therefore discharge or, alternatively, the expedition of discharge without rehousing, in those cases where the latter was an unattainable goal.

This broadly positive conclusion is in line with previous evidence supporting the value of specialist homeless healthcare teams in improving health and housing outcomes for patients in hospitals providing acute physical care.^17–19 In service evaluations of the Pathway Homeless Teams established in acute hospitals, there appeared to be an immediate reduction in average bed-days for homeless patients, although this average may then have increased as the case-load of the teams came to be dominated by patients with more complex problems.¹⁷

Strengths and limitations

The evaluation was limited by our ability to look at only two quantitative outcomes. Some other outcomes could, in principle, have been measured. We did not attempt to measure outputs and activities themselves. Qualitative interviews, perhaps focusing on specific cases and including homeless in-patients themselves, might have allowed a deeper interrogation of the theories that were put forward in the group interviews that led to the logic model. The evaluation does, however, have the merit of being based on a ‘theory-driven’ rather than ‘black-box’ approach:²⁰ The effects we found can be interpreted within a logic model that was created based on expert practitioner knowledge, rather than standing in isolation.

The estimation and interpretation of programme effects based on non-experimental data needs to be cautious, but has a sound theoretical basis.⁸ The central issue is appropriate control selection, and here we were assisted by knowing the team's eligibility criteria and by the variety of data available to us. Estimation of the probability of referral to the team allowed us to show that we had not included observations whose treatment probability was either too high or too low. We attempted to address any remaining risk of bias using regression, following established methods. Here, more than in the case of control selection, we were limited by available data: in particular, there was little information on case complexity, which could conceivably have differed between treated and control observations. However, if anything, patients with more complex problems would have preferentially been selected for the Pathway intervention, which would be expected to reduce its apparent effect in our analysis. Information bias in the analysis of readmission to SLaM wards was addressed with an analysis of national readmissions.

Implications and future research

The evidence presented here suggests quite strongly that the KHP Pathway Homeless Team had important positive effects on length of stay among programme participants. Other effects of the programme remain unexamined or partially examined. These findings certainly provide some justification for continued funding of the programme itself and could provide a justification for the piloting of similar approaches elsewhere, given the importance of the underlying health needs. However, it is also to be hoped that our findings will provide a stimulus to further attempts at evaluation. There is certainly a need to extend our findings to cover other outcomes. There is also a need to identify the more and less important components of the intervention, perhaps using a more sophisticated blend of qualitative and quantitative techniques. Finally, there is the issue of context. In many cases, interventions that work in one place do not work in the same way elsewhere, and further evaluation looking across multiple sites might help to uncover the context–mechanism–outcome configurations responsible for this,²¹ enabling modification of the intervention so that it may be made more reliably effective across different settings.