Judgements about deprivation of liberty made by various professionals: comparison study

Abstract

Aims and method

A group of lawyers, psychiatrists, best interest assessors and independent mental capacity advocates were asked to make binary judgements about whether real-life situations in 12 vignettes amounted to deprivation of liberty. Kappa coefficients were calculated to describe the level of agreement within each professional group and for the total group of professionals.

Results

There was total agreement between all professionals about deprivation of liberty in only 1 of the 12 cases. The overall level of agreement for judgements made by all professionals was ‘slight’ (κ=0.16, P < 0.01).

Clinical implications

There are practical difficulties involved in making reliable deprivation of liberty judgements within the Deprivation of Liberty Safeguards (DoLS) legislation. A clear interpretation of deprivation of liberty is necessary to facilitate professionals' decision-making in this area.

The Deprivation of Liberty Safeguards (DoLS) ¹ were introduced in April 2009 to amend the Mental Capacity Act 2005 in England and Wales. The safeguards provide a legal framework for people with a mental disorder, over the age of 18, who lack capacity to make decisions about care or treatment in hospitals or care homes, when circumstances amounting to deprivation of liberty are required to provide such care. Leading up to and following introduction of the safeguards, there has been concern surrounding their complexity, how they should be implemented and how they work in parallel with current mental health legislation. ^2-5 A particular concern has been that as deprivation of liberty is not clearly defined in the DoLS Code of Practice, ¹ the DoLS are not user-friendly and are open to wide interpretation. ⁶

Aims

The purpose of this study was to gauge the level of agreement between assessments of deprivation of liberty made by lawyers and those made by different groups of mental health professionals. The aims were to:

1. • compare judgements made about deprivation of liberty by different groups of mental health professionals and lawyers;

2. • describe the interrater reliability of deprivation of liberty judgements within each professional group and for the total of all judgements made;

3. • examine whether rates of identification of deprivation of liberty differ between different groups of professionals.

Method

Design

Twelve detailed clinical vignettes were selected from a case series of 28 real-life cases to represent a range of clinical situations of varying complexity. The vignettes were sent out to four groups: psychiatrists, independent mental capacity advocates (IMCAs), best interest assessors and lawyers with expertise in mental health law. Each professional was asked to make a binary judgement (yes/no) about whether the clinical situation described in the vignette amounted to deprivation of liberty. Professionals were asked to briefly outline the reasons leading to each judgement and response sheets containing this information were returned to the research group. The study was approved by the Joint South London and Maudsley and the Institute of Psychiatry NHS Research Ethics Committee.

Vignettes

Anonymised case vignettes outlining circumstances of admission, restrictive measures imposed during admission and the patient’s views and their attitude towards this were prepared for informally admitted psychiatric patients who lacked capacity to make decisions about admission for assessment or treatment. Patients were identified from the electronic notes system and discussion with responsible clinical teams. Those agreeing to take part were seen for a capacity assessment to confirm capacity status and suitability for inclusion. Patients gave informed consent or, where capacity to consent to take part in the study was clearly lacking, Mental Capacity Act requirements were followed and a nominated or personal consultee was involved.

Information included in the vignettes was collected from the medical notes using a schedule of stem questions devised for this study. The interview provided information about: situations described in the DoLS Code of Practice as possibly amounting to deprivation of liberty; the patient’s experience of the admission process; and the experience of being an in-patient during the current admission, including whether or not the person thought he or she should be in hospital. A member of the nursing team was asked about the patient’s compliance with admission and treatment, based on both what the patient had said and his or her behaviour as an in-patient, and this was also included in the vignette. Where possible, the views of relatives and personal consultees with regard to the admission were sought and recorded in the case vignette. The vignettes had an average length of 782 words (ranging from 330 to 1018).

Patients were seen between February 2009 and August 2009 for a single 30-minute interview on a range of wards within South London and Maudsley NHS Foundation Trust. Twelve cases were chosen from the original series of 28 vignettes, described elsewhere, ⁷ to represent a range of clinical situations and complexity.

Professionals

Professionals were approached in writing with an invitation to take part in the study. The legal panel comprised six eminent barristers and solicitors with expertise in mental health law; this included involvement in cases that have provided guidance for identifying deprivation of liberty to date, most notably the ‘Bournewood’ ⁸ and ‘JE’ ⁹ cases. The six consultant psychiatrists involved were members of different Royal College of Psychiatrists’ faculties who had expressed an interest in taking part to their faculty chairs and were subsequently invited to do so by R.C. The six IMCAs were invited to take part via email and R.C.’s attendance at a London and South East Regional Independent Mental Capacity Advocates Network meeting. The five best interest assessors were identified through a list of registered assessors in the area in which the study was held. In response to feedback from other groups about the time taken to rate the 12 vignettes, which had in some cases led to incomplete data, 8 of the 12 vignettes were selected to be sent to the best interest assessors at a slightly later date.

Analysis

Data analysis was performed using STATA/IC 10.1 for Windows. Descriptive statistics were used to compare judgements made about deprivation of liberty by different groups of mental health professionals and lawyers. Kappa coefficient values were calculated to describe the correlation and level of interrater reliability for each professional group and for the total group of individuals making deprivation of liberty judgements. Random intercept logistic regression analysis was performed to examine whether there was a difference between the propensities of the four groups of professionals to identify deprivation of liberty, taking account of clustering by an individual rater.

Results

Comparison of deprivation of liberty judgements by different professional groups

The percentage proportion of professionals in each group who judged that the individual in the vignette had been deprived of his or her liberty (rather than having their liberty lawfully restricted) is summarised in Table 1 for the 12 cases. Full data from six lawyers, six psychiatrists, six IMCAs and five best interest assessors are presented for eight cases and a further four cases are included where incomplete data were available from three of the four groups (lawyers, psychiatrists and IMCAs). Total agreement between all four groups about the presence of deprivation of liberty was seen in only one case (case vignette 2). Agreement about the absence of deprivation of liberty was seen between three of the four professional groups also in one case (case vignette 9). The remaining ten cases attracted a range of different judgements about whether deprivation of liberty had occurred.

Table 1 Proportion of each professional group judging deprivation of liberty to have occurred and the level of agreement (kappa) between professionals

Case vignette	Lawyers	Psychiatrists	IMCAs	BIAs a	Total
	%
1	67	83	100	60	78
2	100	100	100	100	100
3	67	33	50	80	50
4	83	50	83	60	70
5	67	33	67	20	48
6	67	83	67	60	70
7	83	50	100	60	74
8	83	50	67 b (n = 3)		67
9	50	0	0	0	14
10	100 b (n = 1)	50	50 b (n = 4)		69
11	60 b (n = 1)	50	33 b (n = 3)		48
12	60 b (n = 1)	33	50 b (n = 4)		48
Kappa	–0.07 (P = 0.81)	0.12 (P = 0.05)	0.24 (P = 0.01)	0.11 (P = 0.15)	0.16 (P < 0.01)

BIAs, best interest assessors; IMCAs, independent mental capacity advocates.

a. Best interest assessors were asked to check only eight case vignettes (no. 1–7 and 9).

b. Incomplete data (n, number of missing assessments).

Interrater agreement of deprivation of liberty judgements

Table 1 shows the kappa values for the experts as a whole and by professional group. There was a ‘slight’ level of agreement ¹⁰ (κ = 0.16) for binary deprivation of liberty judgements made by all professionals who rated the 12 case vignettes. The IMCAs had the highest level of agreement with κ = 0.24, which is nonetheless only defined as ‘fair’. The lawyers had the lowest level of agreement with a kappa value suggesting their consistency of agreement was no better than chance. The agreement for best interest assessors and psychiatrists was only ‘slight’.

Comparison of rates of identification of deprivation of liberty between groups

We used logistic regression analyses to find out whether there was a difference between the propensities of different professionals to make deprivation of liberty rather than restriction of liberty judgements. The odds ratios from these analyses are shown in Table 2 and indicate that for the vignettes rated in this study, lawyers were significantly more likely to judge that deprivation of liberty had occurred than both best interest assessors and psychiatrists. It was also seen that IMCAs were more likely than psychiatrists to judge deprivation of liberty, although this finding was of borderline statistical significance.

Table 2 Comparison of the propensities of different professionals to make deprivation (rather than restriction) of liberty judgements ^a

	Odds ratio (95% CI)
	BIA	IMCA	Lawyer	Psychiatrist
BIA		2.07 (0.79–5.37), P = 0.14	2.76 (1.07–7.10), P = 0.036	0.92 (0.38–2.26), P = 0.86
IMCA			1.33 (0.56–3.15), P = 0.51	0.45 (0.20–1.00), P = 0.05
Lawyer				0.33 (0.15–0.73), P = 0.006
Psychiatrist

a. The rows are the reference groups; thus the odds of an independent mental capacity advocate (IMCA) judging the vignette to indicate deprivation of liberty are 2.07 times higher than a best interests assessor (BIA) making such a judgement.

Factors influencing deprivation of liberty decision-making processes

There was full agreement between all professionals that deprivation of liberty had occurred for only one of the 12 cases described (case vignette 2).

Case vignette 2

X is a 91-year-old man with advanced dementia seen in a National Health Service (NHS) continuing care ward where he has lived for 2 years. This followed a move from a care home where he was placed with a close friend (who also had dementia) but where he did not settle and where staff had difficulty managing his challenging behaviour. He was transferred to the current unit following an informal admission to an old age psychiatry ward. Although he had initially agreed to admission to hospital ‘for some tests’, X had been initially very agitated on the ward checking doors and threatening to break windows for hours at a time. His unsettled presentation returned after moving to the continuing care unit: X repeatedly asked to leave, made attempts to do so and made threats to kill himself if he was not allowed home. His separation from his close friend, still living in the care home, was thought to contribute towards this behaviour. Following one unsuccessful visit to see his friend no further meetings had been arranged. At the time of the research interview X lacked capacity to make decisions about treatment or place of residence. He continued to ask to go home on a regular basis but had made no recent moves to leave. The unit was locked at all times but X appeared to enjoy fortnightly trips out with staff. He almost always accepted his medication prescribed for depression and dementia. During the interview X was preoccupied with going home, said that living in the unit had not been his decision and described himself as a ‘prisoner’. He made threats to attempt suicide if he was not allowed home. The manager felt that he objected to his current living situation. His son did not object to the care X received.

In addition to X’s mental incapacity, the factors listed by professionals as important in influencing their decisions that there was deprivation rather than restriction of liberty included: lack of ‘freedom to leave’; the view that there was ‘complete and effective control’ over X’s life by staff, who did not take X to see his friend; the consistency, degree and intensity of X’s objections; and the long duration (2 years) for which X had been detained.

Case vignette 9

Y is an 80-year-old man with advanced dementia of mixed vascular and Alzheimer’s types who had lived in NHS continuing care accommodation for the past 4 years. He was seen for the research interview with his son but was unable to understand or answer the questions asked. According to the son, agitated behaviour, tearfulness and resistance to personal care had been features of his father’s dementia before moving to this continuing care unit 2 years ago. Y lacked capacity to make decisions about his accommodation and care when he was seen for the research interview. The door to the unit was kept locked at all times. Y wandered around the unit but did not repeatedly stand by doors or try to leave. He occasionally resisted help with personal care but responded to reassurance and distraction techniques. He usually accepted his antidepressant medication. Y was taken on an outing from the unit approximately once a month and the family were also encouraged to take him. He was not on any increased levels of nursing observations and had not been restrained. The medical team did not feel he was objecting to his care. His son felt that his father was very well looked after and did not think he behaved in a way that suggested he objected to living in the unit.

Regarding this vignette, psychiatrists, IMCAs and best interest assessors were all in agreement that there had been no deprivation of liberty. The reasons given for this decision included: the acceptable use of restrictions proportionate to the risks; provision of Y’s care in the least restrictive setting possible; a care plan that included open access for visits and regular (monthly) outings; the fact that Y had not demonstrated a desire to leave the unit; and the family’s belief that he did not object to his situation. The lawyers were not in full agreement about this and three members of the group (50%) felt that, despite monthly outings, Y was not free to leave the unit to live elsewhere and was therefore deprived of his liberty.

Case vignette 3

Z is a 40-year-old man who was informally admitted to a neuropsychiatry ward for assessment of cognitive impairment and rehabilitation needs following a brain injury. He presented with significant short- and long-term memory impairment, disorientation on the ward and paranoid and persecutory beliefs that staff were trying to poison him. At the time of the research interview Z lacked capacity to make decisions about admission for treatment on the locked ward from which he was allowed only escorted leave. A decision had been taken by the clinical team that he would require supported accommodation rather than returning home. Z said it was not his decision to come into hospital, although he added that ‘it wasn’t forced, it was arranged’, and he believed he was detained under the Mental Health Act. The nursing staff reported that he had not questioned the need to be in hospital and had not tried to leave over the course of his 4-month admission. His treatment included antipsychotic medication. He was concerned that ward staff had confused him with someone else and were giving him the wrong treatment. He also said that his privacy has been invaded on the ward and reported thefts from his room. He said his freedom was limited ‘by being here, full-stop’ and that he was not happy about the admission or his medication. He did not think he should be in hospital and had ‘no idea’ why he was. His family believed he needed to be in hospital and did not object to the care or treatment he was receiving.

For the majority of case vignettes (ten of twelve) there was little agreement about whether deprivation of liberty had occurred. Assessing case vignette 3, half of the professionals felt there was deprivation of liberty and the spread of judgements between groups was wide (range 33-80%). In addition to the patient’s lack of mental capacity, factors listed as important in determining that Z had been deprived of his liberty included: he had only escorted leave; he was not free to leave the ward to go elsewhere; the duration of his detention (4 months); placement decisions had been made on his behalf; he believed he was detained under the Mental Health Act; and he was not happy in his current situation. Factors listed in support of the situation representing a lawful restriction (rather than deprivation) of Z’s liberty included: the measures were reasonable to ensure his safety; he had not tried to leave and was not actively objecting to being in hospital; he had adhered to treatment; and he did not present as distressed on the ward.

Discussion

A previous study of the prevalence of likely deprivation of liberty used two different approaches to identifying patients who were deprived of their liberty; one method identified deprivation of liberty in 84% of informal incapacitous psychiatric in-patients, whereas the other in only 11%. ¹¹ Another study brought together a panel of eminent mental health lawyers to explore their thinking on the legislation in lieu of awaited future case law. ⁷ The lawyers experienced difficulty in interpreting many aspects of the DoLS and the DoLS Code of Practice. When identifying deprivation of liberty the lawyers attributed key importance to factors from previous case law not covered in the Code of Practice guidance. This suggested a possible lack of clarity and consistency over the distinction between deprivation and restriction of liberty in the law.

The first report on annual data for the DoLS from 2009-2010 showed that of the 7160 cases where deprivation of liberty was considered, in 3300 this was authorised. ¹² These activity levels were about a third of the level estimated before the introduction and reasons for low use of the DoLS are unclear. It may be that fewer people than previously thought receive care or treatment that involves deprivation of liberty. However, there has been great variation between the number and rate of applications in different areas (with five primary care trusts processing no such applications) and the report suggested that this may be due to local practices and the way supervisory bodies have implemented the legislation. A Department of Health briefing ¹³ in May 2010 reported that a number of best interest assessments (125 in the first 9 months of the DoLS operating) had not supported deprivation of liberty authorisations because the best interest assessors had taken a view that a less restrictive option was or could be available. Another explanation might be that deprivation of liberty is not being recognised or, where it is recognised, there may be a reluctance to use new complex legislation, relying, for example, on existing Mental Health Act legislation instead.

Main findings

This study aimed to investigate the level of agreement between different professionals making deprivation of liberty assessments. We found that the overall level of agreement between all professionals who rated the case vignettes was slight. This suggests that, in contrast to mental capacity which can be reliably assessed, ¹⁴ current approaches used to identify deprivation of liberty do not lead to reproducible and reliable outcomes.

Agreement was found to be fair for IMCAs but was slight for psychiatrists and best interest assessors. Two of the six sets of IMCA ratings were generated by IMCA teams as group exercises, where the team came to a consensus judgement after discussing the case, as they argued that this best reflected actual clinical practice within their particular teams. It is therefore possible that the increased reliability of IMCAs’ ratings compared with those of other professionals might reflect different approaches to assessments developed through team decision-making processes. According to the Code of Practice, deprivation of liberty ‘is ultimately a legal question, and only the courts can determine the law’ (p. 16). ¹ One might, therefore, have hoped for a high level of agreement between lawyers assessing deprivation of liberty. However, the interrater reliability of judgements made by the highly experienced lawyers in our study was no better than chance and we suggest that this highlights both the complexity of the DoLS and the difficulties arising from the uncertain definition of deprivation of liberty.

There are two main issues relating to the poor reliability of judgements. First, because the legal concept of deprivation of liberty and therefore how it should be assessed is unclear, it has not been possible to train professionals to interpret deprivation of liberty in a particular way and professional biases and idiosyncrasies of interpretation are likely to be introduced into judgements. If the concept could be consistently interpreted and narrowly defined, the level of reliability would presumably be much better. Second, the fact that this study looks at the reliability of deprivation of liberty judgements within the context of DoLS (i.e. for incapacitated patients where use of the Mental Health Act has not been considered clearly appropriate) means it is likely to be selecting for ‘grey’ cases, where reliability is hard to achieve. In other words, regardless of the poorly defined ‘deprivation of liberty’ concept, one might expect better agreement if the reliability study was not restricted to deprivation of liberty judgements within DoLS.

We also found that for the cases described in this study, lawyers were more likely to identify the presence of deprivation of liberty than either psychiatrists or best interest assessors. This is likely to be explained by the finding of our previous study that lawyers attributed key importance to whether the patient was ‘free to leave’. ⁷

For the majority of the case vignettes there was no clear agreement between professionals about whether deprivation of liberty had occurred. This reinforces the concern that in the absence of legal clarity about how deprivation of liberty should be distinguished from restriction of liberty, potential deprivation of liberty assessors are unable to make reliable decisions about this. In addition, a number of the reasons listed by professionals as important in influencing their judgements were, arguably, idiosyncratic and ultimately reflected clinical rather than legal judgements.

Strengths and limitations

Clinicians are now expected to identify incapacitated patients who are deprived of liberty. We maintain that our study is of high clinical relevance to mental health professionals who are faced with this difficult task. The study used real-life detailed vignettes to examine lawyers’ and mental health professionals’ decision-making processes around DoLS. To the best of our knowledge it is the first study that has examined interrater reliability of deprivation of liberty judgements. The vignettes used described a range of clinical presentations of incapacitated patients in different in-patient settings (old age, neuropsychiatry and general adult in-patient units, and NHS continuing care units) who were potentially being deprived of their liberty. We believe that this method recreated as closely as possible the clinical decisions that face psychiatrists, IMCAs and best interest assessors in the clinical setting and makes the findings of our study generalisable to clinical practice. The data described in this study relate only to whether deprivation of liberty was present and not to which detention regime (Mental Health Act or DoLS) would be most appropriate where this was the case.

An obvious limitation of this study is its small size which limits the statistical power that can be attached to our findings. However, the number of case vignettes distributed to raters was deliberately limited to 12 (8 in the case of best interest assessors) to enable them to convey real-life clinical context and to maximise the number of fully completed responses. Although data collection was incomplete for 4 of the 12 vignettes, this was corrected for in the statistical analyses. The assessors were not asked to comment on whether the vignettes contained sufficient information to allow decisions about deprivation of liberty to be made. It is therefore possible that the vignettes themselves might have contributed to a poor level of agreement between professionals by failing to include valuable information pertinent to the cases. However, the vignettes were detailed and made use of a combination of staff and patient record information as well as interviews with patients, staff and near relatives. Informal feedback from assessors in this study and also from a previous study using the same vignettes ⁷ did not suggest that there were concerns about inadequate detail or information for this purpose. The professionals who took part in the study were self-selecting and, particularly for the lawyers and psychiatrists, were chosen because of a special interest in the field. If bias were thus introduced, we would have expected this to be in the direction of greater consensus as these were the individuals best versed in the law. We believe it is unlikely that an unselected population of practitioners would have had better interrater reliability.

Implications

The findings of our study raise serious concerns about the complexity of the DoLS legislation and particularly the practical difficulties involved in making reliable deprivation of liberty judgements within this legislation. Arising from this is a concern that deprivation of liberty authorisations may be arbitrarily applied resulting in unequal levels of care and safeguarding for potentially vulnerable, incapacitated individuals, similar in all relevant respects. To address this, we suggest that clinicians require further legal guidance on what constitutes deprivation of liberty and how this should be assessed. It is an open question whether training can significantly improve the reliability of assessments. Much will depend on whether a clear interpretation of deprivation of liberty can be constructed for practical use. This may come about as future case law evolves. On a more positive note, we suggest that clinicians should feel reassured by our study that they are not alone in finding the legislation complicated and confusing: the findings suggest that all the professionals in this study, including some with high levels of expertise in the field, found themselves in the same position.