Mental health policy reforms and case complexity in CMHTs in England: replication study

Abstract

Aims and Method

Community mental health team (CMHT) services in many Western countries have been remodelled to focus on people with the most severe illnesses and complex problems. Complexity scores using the Matching Resource to Care (MARC2) measure from CMHT cases in 2004–2005 (n=1481) are compared with scores in 1997–1998 (n=3178) in the same locations, before the introduction of the National Service Framework, and before the impact of the creation of integrated mental health trusts in England.

Results

The 2004–2005 baseline complexity scores are all worse than those in 1997–1998.

Clinical Implications

If increased targeting brought about by the National Service Framework and other reforms has led to a greater proportion of people with complex problems in case-loads, what care services, if any, are now being received by people who were in receipt of CMHT services before the reforms?

Mental health policies in many Western countries have been remodelled to focus services on people with the most severe illnesses, greatest risk and most complex problems. However, these policies have had limited success in some places, such as the USA (Wang et al, 2002) and Australia (Rosen, 2006). In the UK, the New Labour Government introduced a focus on severe illness in the National Service Framework (Department of Health, 1999) but also modernised the care planning process (Kingdon, 1994; Department of Health, 1999a ), community mental health services through the introduction of new teams (assertive outreach, early intervention and crisis resolution) (Department of Health, 2000) and partnership mental health trusts, which permitted the employment of social workers and other social care staff from the local authority in National Health Service (NHS) trusts; it was argued that the changes would produce a more integrated service (Venables et al, 2006). All these changes were intended to produce better outcomes (Holloway, 2001).

Many of the new teams created as part of the NHS plan were constituted from staff in existing community mental health teams (CMHTs), employing the same professionals in a new structure. Nevertheless, the Mental Health Policy Implementation Guide (Department of Health, 2001) stated that CMHTs should continue to have:

‘an important, indeed integral, role to play in supporting service users and families in community settings. They should provide the core around which newer service elements are developed. The responsibilities of CMHTs may change over time with the advent of new services, however they will retain an important role’. (Department of Health, 2001: p.7)

This paper explores the consequences for case complexity in CMHT case-loads of 7 years of service changes.

Method

This study is a replication of one conducted in the same eight locations in the late 1990s. The data from 1997–1998 provide an assessment of the nature of provision by community-based staff in the period prior to the introduction of the National Service Framework for Mental Health, the Mental Health Policy Implementation Guide and other related policy measures.

The present paper compares case complexity using the Matching Resource to Care (MARC2) measure (Huxley et al, 2000a , b ) and care programme approach (CPA) status in 1997–1998 with 2004–2005. The hypotheses are that, compared with the original study, the replication study will show: (a) a higher proportion of cases having higher complexity scores (because of better targeting of resources or more consistent use of eligibility criteria); and (b) professional staff carry cases of equal complexity (because of integration of health and social workers in the same teams).

Original study

In the original 1997–1998 Study data were collected using the MARC2, a single page form that records the main characteristics associated with severe mental illness on one side, and the Global Assessment Scale (GAS; Endicott et al, 1976) and Health of the Nation Outcome Scale (HoNOS; Wing et al, 1998) on the other.

The MARC2 is a 20-item scale, each item is equally weighted and each item is scored 0 or 1, aggregated to a total score (higher scores indicate greater complexity in terms of user characteristics (for example having been compulsorily admitted to hospital previously, psychotic illness, substance misuse problems), risk factors (for example risk of aggression and suicidal risk) and social problems (for example homelessness, unemployment)) (Huxley et al, 2000b ). The reliability and validity of the instrument used (MARC2) has been described elsewhere (Huxley et al, 2000a ).

Community mental health professionals within teams (mainly nurses, social workers and occupational therapists) in eight locations (representing a range of type of local authority and deprivation) across England completed the MARC2 form on a random half of their case-load within a 1-week census period, over the course of 11 months ending in summer, 1998. All CMHT keyworkers who carried case-loads and who were not on leave were included.

In the original study, a total of 3178 clients were included in the analysis. The average case-load size was 30 for nurses and 27 for social workers.

Present study

The design difference between the present study and the original one was minimal; (a) an additional study site was included in 2004–2005, but these results have been excluded from this paper, and (b) in order to reduce the data collection burden, the proportion of case-loads selected for inclusion was set at 20% rather than 50% (randomly selected using random numbers by the research staff from case-load lists). All teams were integrated in that the social workers were working in NHS-trust-managed-CMHTs by the time of the replication study. Research staff were trained by the Royal College of Psychiatrists’ Research Unit HoNOS trainers, and they in turn trained all of the teams in the study in the use of HoNOS and the other instruments. Shared care cases were always assigned to a primary care coordinator who completed the research instruments. However, since no further checks were made on the accuracy of the staff ratings (although research staff remained available to be consulted over rating questions), there is a possibility of individual rating idiosyncracies; rating drift is less likely given the short (usually one-off) data collection period.

Statistical analysis

The MARC2 and GAS mean scores are normally distributed and so t-tests and ANOVAs are used to make comparisons between the baseline MARC2 scores in 1997–1998 and 2004–2005, professional group differences in both years, and GAS outcomes.

Results

There were six locations in the north west of England and two in the south east. At the time of the study few had actually introduced assertive outreach or early intervention teams, so the bulk of the sample was drawn from CMHTs. Sample size was 1481 and less than 2% of the cases came from assertive outreach teams (n=26 out of 1481). On the basis of known mean team sizes at the time of the study (Boardman & Parsonage, 2005) the response rate was at least 58% (n=413 out of 710), and could be higher if allowance is made for staff leave and vacancies. Sixty percent (n=884) of the sample of clients were categorised as having a psychotic illness compared with 63% in 1997–1998. Approximately half of the clients were female for both 1997–1998 (n=1629, 51.5%) and 2004–2005 (n=732; 49.4%). There were 68 cases (5%) on the highest CPA level in 2004–2005, compared with 17.8% in 1997–1998. The average case-load size was 25.

HoNOS and GAS

An independent sample t-test was used to compare the HoNOS, GAS and MARC2 between the two research studies (1997–1998 and 2004–2005).

Table 1 shows that baseline mean scores were all significantly different, and that complexity and clinical severity were worse in 2004–2005 than in 1997–1998.

Table 1. Results for Health of the Nation Outcome Scale, Global Assessment Scale and Matching Resource to Care measure baseline scores in 1997–1998 and 2004–2005

Outcome measures	n	Mean	t 1	d.f.1	P (2-tailed)1	s.d.	s.e. mean
HoNOS
1997-1998	2833	10.04	-8.41	2377.43	0.000	7.00	0.13
2004-2005	1321	12.13				7.67	0.21
Gas
1997-1998	3107	56.66	2.29	2578.61	0.022	18.21	0.33
2004-2005	1416	55.25				19.49	0.52
MARC2
1997-1998	3144	3.06	-44.59	1886.91	0.000	2.80	0.05
2004-2005	1327	8.48				4.03	0.11

MARC2 scores by care programme approach level

A further indicator of change is the proportion of people on enhanced level CPA. A difficulty in making this comparison is that CPA policy has changed from a three level categorisation in 1997–1998 to two levels, standard and enhanced. Standard CPA relates to individuals, who require support or intervention from one agency or discipline, who pose no danger to themselves or to others, and who will not be at high risk if they lose contact with services. Enhanced CPA relates to individuals with multiple needs, who need to be in contact with more than one professional group or agency (including criminal justice agencies). This group may have more than one clinical condition, or a condition that is accompanied by alcohol or drug misuse, so requiring more intensive help from a range of services. Their circumstances are the most complex.

The mean score of those on the highest level of CPA in 2004–2005 was double that in 1997–1998 (Table 2).

Table 2. MARC2 scores by care programme approach level for 1997–1998 and 2004–2005

	n	Mean	s.d.	s.e.	95% CI mean
1997-1998
CPA level 1	460	3.07	2.94	0.14	2.80-3.34
CPA level 2	2087	3.57	2.72	0.06	3.45-3.68
CPA level 3	554	4.53	3.56	0.15	4.23-4.82
Total	3101	3.67	2.95	0.05	3.56-3.77
2004-2005
Standard CPA	410	6.17	3.44	0.170	5.83-6.50
Enhanced CPA	893	9.57	3.81	0.128	9.32-9.82
Total	1303	8.50	4.02	0.111	8.28-8.71

In each sample there is a significant difference in MARC2 scores between the CPA levels (1997–1998 data, f=34.83, d.f.=2, 3098, P<0.001; 2004–2005 data, t=–15.99, d.f.=871.72, P<0.001; mean difference –3.40, CI=–3.82 to –2.98).

MARC2 scores by professional group

There was a statistically significant difference in MARC2 scores for the professional groups, with social workers having significantly higher mean scores in both studies (Table 3).

Table 3. Analysis of Variance of MARC2 scores for different professional groups

	Mean	s.d.	s.e.	F	P
1997-1998
CPN (n=2060)	3.41	2.80	0.06	81.82	<0.001
Social worker (n=947)	4.55	3.17	0.12
OT (n=154)	1.88	1.85	0.15
2004-2005
CPN (n=757)	8.20	4.07	0.15	5.76	<0.01
Social worker (n=381)	9.02	4.00	0.20
OT (n=68)	8.00	3.57	0.43

In 2004–2005, the mean MARC2 scores for the nurse and occupational therapists are closer to those of social workers than in the original study, but mean scores for social workers are still significantly higher. A higher MARC2 mean score (>9) is significantly related to both the enhanced CPA level, and to the substantial and critical needs categories of the Fair Access to Care Criteria (FACS). Parabiaghi et al (2005) have reported that a mean clinical improvement score on HoNOS following treatment was 12 and for remission 5. In the present sample the mean HoNOS score for social workers’ cases was 13 and for nurses 11 (t=–3.5, d.f.=1117, P<0.01). The GAS scores of social worker cases were also significantly worse (means 58 and 54; t=2.91, d.f.=1158, P<0.01).

The individual MARC2 items where the case-loads of nurses and social workers differ is of some interest. At both time points a quarter of the social services’ cases had been homeless at some time, compared with a tenth of the nurses’ cases. In 1997–1998 a fifth of the social workers’ cases had a concurrent substance misuse problem, which was a higher proportion than that of nurses’ cases (16.3%). By 2005, a fifth of the community psychiatric nurses’ cases have concurrent substance misuse problems compared with 16% of social workers’ cases.

As in the previous study (1997–1998) the proportion of cases on social workers’ case-loads in 2004–2005 with a previous compulsory admission (52%), suicidal risk (47%) and self-neglect (69%) are all significantly higher than the proportions on community psychiatric nurses’ case-loads (41%, 38% and 56% respectively) in 2004–2005. There were the same proportions of cases on standard (32%) and enhanced (67%) CPA held by social workers and community psychiatric nurses. One could argue that it is appropriate that social workers should be working with people with a greater prevalence of social difficulties, and that this reflects a relevant division of labour within the teams.

Discussion

The overall differences in mean scores in the replication study suggest that improved targeting has increased the proportion of complex cases in CMHT case-loads. Each of the three professional groups had experienced a similar increase in mean scores between the two studies, while the total case-load size in all these locations remained broadly similar at about 6000 cases. The mean MARC2 scores of social workers’cases remains significantly higher than the scores of nurses and occupational therapists. There could be a number of possible explanations for this finding: the nature of the MARC2 items, which contain several items relating to serious social problems, and if social workers accept cases with more social problems then there will be higher complexity scores in their case-loads; at the same time, it may be that the nurses in the teams are increasingly focused on evidence-based interventions with people facing less complex problems but ones that are amenable to modern effective interventions. This explanation is unlikely given that mean scores for nurses’ cases in 2004–2005 are much higher than in 1997–1998. Given the long treatment history of many of the patients it could be that those returning for episodes of care were given the same keyworker as before, and many of the cases seen in 1997–1998 were likely to still be in contact with services in 2004–2005. Since the 1997–1998 data showed that social workers carried more of these cases, they may still have done so in 2004–2005, hence increasing the mean MARC2 scores.

In conclusion, the current research has shown that the reforms introduced over the period between the two studies may have led to better targeting of services on people facing complex issues. The targeting of service towards those people facing the most complex circumstances and for whom the service is designed increases ‘vertical integration’. However, an increase in vertical integration may be achieved at the expense of less ‘horizontal integration’, so that while a greater number of people in the target group are receiving services, this may be at the expense of those with moderate needs who, in 2004–2005 compared with 1997–1998, were excluded from services. Given that most of these service users had nurses or occupational therapists as keyworkers in 1997-1998 the consequences of the change may have more implications for primary healthcare services than for social care services. Further research, however, will be required to assess how widely this finding might apply, and how a reduction in horizontal integration is being managed in the context of revised commissioning and organisational arrangements post-2005.

Declaration of interest

This study was funded by the Department of Health.