Implementing the Short-Term Assessment of Risk and Treatability (START) in a forensic mental health service

Abstract

Aims and Method

To evaluate the practical utility and face validity of a new risk assessment guideline, the Short-Term Assessment of Risk and Treatability (START), within a forensic mental health service. Staff attended training and subsequently used the START to assess and formulate risk. Staff then completed an evaluation questionnaire about their experiences of using the START.

Results

The study proved useful in evaluating the practical utility and face validity of the START. the START demonstrated both good practical utility and face validity. Areas of difficulty in completing the START were identified and areas for developing the START were highlighted.

Clinical Implications

The service subsequently implemented a revised version of the START into routine practice. the results of the pilot study suggest that the START can usefully assist in structuring risk judgements in practice.

The Short-Term Assessment of Risk and Treatability (START) is a brief clinical guide for the dynamic assessment of risks, strengths and treatability, which is in the latter stages of development by its authors (Webster et al, 2004). The START is designed as a structured professional judgement guideline intended to inform evaluation of multiple-risk domains relevant to everyday psychiatric clinical practice (Webster et al, 2006). The areas of risk considered are: risk to others, suicide, self-harm, self-neglect, substance misuse, unauthorised leave and victimisation. Although it may be widely applied, the START was developed with forensic mental health units and services in mind.

The process of completing the START involves consideration of an individual's current mental state, behaviour and functioning in relation to 20 specific items (Box 1). Assessors need to consider both strengths and risks for each of these, with judgements anchored to item descriptors provided in the manual. The rationale for the START has been emphasised by Webster et al (2006) who highlight the importance of dynamic variables and the value of considering strengths as well as vulnerabilities. They also stress the extent to which clinicians must be attentive to multiple risk domains, as empirical evidence indicates that risk domains may overlap. Preliminary evaluations in clinical populations have demonstrated that the START has the potential to be a valuable guideline to inform clinical practice (Webster et al, 2004, 2006; Nicholls et al, 2006). The aim of the pilot study reported here was solely to evaluate the practical utility and face validity of the START within a medium secure forensic mental health service. This was viewed as essential ahead of implementation of the START in clinical practice, as few if any studies have considered this previously in a UK setting.

Box 1. Twenty items of the Short-Term Assessment of Risk and Treatability (START)

1. • Social skills

2. • Relationships

3. • Occupational

4. • Recreational

5. • Self-care

6. • Mental state

7. • Emotional state

8. • Substance misuse

9. • Impulse control

10. • External triggers

11. • Material resources

12. • Attitudes

13. • Medication adherence

14. • Rule adherence

15. • Conduct

16. • Insight

17. • Plans

18. • Coping

19. • Social support

20. • Treatability

Method

The study was conducted in a medium secure unit in Manchester, UK. A cross-section of qualified nursing staff from the unit attended a training workshop in which they:

1. • considered the background to risk assessment

2. • discussed potential benefits of using an evidence-based guideline in risk management

3. • actively participated in small group exercises using the START

4. • discussed the aims of the pilot study and issues of evaluation.

Following the training workshop, participants completed the START for three or four people whom they were working with, together with the accompanying evaluation questionnaire (available on request from the first author). The evaluation questionnaire had previously been validated and approved by the START authors.

Results

Descriptive statistics

Twelve members of staff attended the training workshop. Eleven of those who participated in the workshop returned a total of 39 questionnaires based on three or four assessments of individuals with whom they were familiar. The staff were evenly distributed throughout the service, which meant that the START was completed for individuals at various stages throughout the care pathway.

The participants in the study had practised within the service for an average of 144 weeks, ranging from 2 weeks to 576 weeks. Of the respondents, 82% felt they knew the person that they rated ‘fairly well’ or ‘very well’. In only two cases (5.2%) did staff feel they only knew them a ‘little’ or ‘not at all well’. The majority of START assessments (82%) were conducted as part of a regular review. In 37 (94.9%) of the assessments, the information required to rate the START was felt to be readily available. In nearly 95% of cases, staff were ‘moderately’, ‘fairly’ or ‘very confident’ in rating the items and only in 8 (20.5%) of the cases were items on the START found to be difficult to rate. In 85% (n=33), the START was found to be ‘moderately’ or ‘very useful’ in clarifying thinking in relation to the individual's risk.

On average, the START took 25 min to complete, although this gradually reduced to 22 min when participants had a chance to complete two and then three assessments. Thirty-two (82.1%) assessments were completed in 30 min or less.

In relation to the Risk Formulation section, in nine (23.1%) cases staff experienced difficulties, whereas nearly three-quarters (74.4%) had no difficulties. In nearly a half (42.9%), the difficulties in completing the Risk Formulation section were attributed to the unclear nature of the examples provided in training. Nearly 85% (n=33) of staff had no difficulties completing the Signature Risk Signs section. The vast majority (n=32, 83%) found this section ‘moderately’ or ‘very useful’. Nearly a quarter of staff experienced difficulties completing the Specific Risk Estimates section and this was attributed to the risk being difficult to assess and the guidance being too vague. Nearly three-quarters (72%) of staff found the Specific Risk Estimates section ‘moderately’ or ‘very useful’ although, only 12.8% were very confident completing this section.

Of the staff who returned the questionnaires, over half (51.3%) stated that they had previously received training in the use of the Historical, Clinical, Risk Management-20 (HCR-20) assessment guideline (Webster et al, 1997) or similar structured professional guidelines. When asked how often they thought the START should be completed in clinical practice, the majority (n=29, 79.4%) felt the START would be best used between monthly and three times monthly. Most felt that the START should be completed periodically during reviews.

Review of comments /themes

Questionnaire forms provided opportunity for participants to identify specific issues and make suggestions about the START. Comments were collated and reviewed. What appeared to be the most salient themes are outlined here.

Knowing the individual

Several comments highlighted the importance of knowing the person well. This affected the rating of specific items (e.g. mental state, emotional state, insight, signature risks, risk estimates) and consequently had an impact on how confident participants felt about clinical judgements made using the START.

Uncertainty about time frame

Comments revealed some uncertainty about the time frame over which the START ratings should be rated, particularly when there appeared to be significant long-term risk issues that had been historically important, but which through current conditions and/or care plan were now being successfully managed. Some participants felt unsure about how to rate such items in the context of a tool that places a focus on management of risks in the short-term.

Concerns in the absence of overt behaviour

A similar theme emerged where raters felt that an underlying vulnerability or tendency (e.g. towards substance misuse) might still be present but where current circumstances restrict access to and opportunity for use. Raters were not confident about how to address these questions in the rating process. This was linked to uncertainty about the time frame over which risks and strengths are considered.

Concerns about subjectivity and interpretation

Some participants expressed concerns that ratings were being made on a ‘subjective’ basis or that the process was very much ‘a matter of interpretation’. These comments particularly related to the completion of the Specific Risk Estimates section, in which a judgement was sought on whether risks were low, medium or high. There was less guidance given on how to make these decisions in contrast to the 20 START items, for which the manual provides specific criteria that assist raters to reliably anchor their ratings.

The START as a helpful framework for organising information

Several participants felt that that the process of completing the START itself was helpful in terms of organising information. Aspects of this included: highlighting risks and strengths; clarifying risk areas; promoting a greater understanding of what factors might contribute to specific risks; and what factors might serve to reduce risk and be necessary to incorporate in a management plan. It was noted that a systematic process of looking at specific risk-related areas revealed any gaps in information, alerting raters to the need to obtain further details in order to be able to make sound judgements and formulate risk.

Discussion

The comments and themes outlined suggest that, whereas the START was seen as offering many positives, participants also experienced difficulties that may need to be addressed in future staff training. Some of these were related to conceptual issues (e.g. questions about time frame, current care plans or situations restricting access to risk relevant factors). Other themes highlighted issues that are central to the whole enterprise of carrying out a good risk assessment (e.g. having adequate knowledge of the individual) or putting into place a satisfactory management plan (e.g. collaboration, communicating risk clearly).

Training might also need to address the implicit notion that there is a ‘right’ way to complete a START assessment. There is a need to convey that the endeavour may be more about giving full consideration to all likely areas that may bear on an individual's risk, in order to better understand the process by which they may become imminently likely to act. Covering these conceptual points in more depth with staff during training might also address some of the identified concerns about confidence. A well-informed and specified formulation should logically inform any subsequent care plans, with level of intervention easily translatable into a judgement about level of risk.

Apart from concerns that may be addressed through training, there were indications that the rating form might be improved or modified in order to make the process of rating more straightforward. Examples of issues that a revised form may address include: how ‘don't knows’ may be handled more effectively; adding a section that allows relevant historical information to be recorded or noted; aiding the rater in distinguishing between current (acute) and longer-term risk; and clear definitions of ‘key’ and ‘critical’ items.

The START training workshop and subsequent survey of staff proved useful in evaluating the practical utility and face validity of the START, and the service subsequently implemented the START into routine practice. Overall, our evaluation suggested that it could usefully aid a process of making risk judgements. The START authors state that the coding of items, without the Risk Formulation section, should take on average 8 min for experienced users. The START proforma including the Risk Formulation section took an average 25 min to complete, although this reduced to 22 min once participants were more familiar with it. Participants here did not re-rate the same individual. Rating time would likely reduce significantly when reassessing.

Conclusion

The Signature Risk Signs section was found to be useful and most participants were satisfied with the Risk Formulation section. Some problems were identified in relation to the Specific Risk Estimates section. A number of participants were not confident about completing this section, which involves rating risks low, medium or high. There was also some concern that the existing format did not permit documentation of previous history of risk behaviour, which would make it difficult to complete the Risk Formulation. We therefore suggest that there may be value in including a brief History section, retaining the Risk Formulation section, to simplify the Risk Estimates section and to assist users in distinguishing between long-term and acute risk. Future research and training should more clearly address some of the fundamental conceptual issues surrounding use of the START regarding risk communication, multidisciplinary team working, static v. dynamic risk factors, interrater reliability and predictive validity.

Declaration of interest

None.