Randomised controlled trials relevant to aggressive and violent people, 1955–2000: a survey

Abstract

Background

Randomised trials remain the gold standard for evaluating health interventions. This applies to the criminal justice system as well as to health.

Aims

To identify and survey randomised trials relevant to forensic mental health services.

Method

We searched 29 electronic bibliographic databases and acquired randomised trials involving sex offenders, arsonists or people clearly and actively aggressive, or abusive of children or spouse. Two researchers reliably extracted data.

Results

Of 409 studies found, we were able to acquire 300 for further inspection. They all involved particularly violent people (total n=28 669), mostly adult men; the mean study size was 197 (median 52, mode 60, range 1–1200). In these 300 randomised trials over 700 interventions were evaluated and short-term outcomes were recorded on 345 different scales.

Conclusions

Wider collaboration, rationalising treatments and simplifying outcomes could further strengthen the tradition of trialling in forensic psychiatry. Systematic reviews of these studies are overdue.

The management of aggression and of potentially aggressive people forms a large part of the workload of forensic mental health services (Taylor & Gunn, 1999). This work is a priority at the highest political levels and society is becoming increasingly intolerant of aggression perpetrated by those with mental health difficulties. In the UK the government has acted to introduce new legislation (Department of Health, 2001). In this context of increasing public concern it is imperative that public policy is informed by the entirety of high quality research rather than by a proportion.

Although often imperfect (Chalmers et al, 1983; Thornley & Adams, 1998), randomised controlled trials remain the gold standard for the evaluation of mental health interventions (World Health Organization Scientific Group on Treatment of Psychiatric Disorders, 1991). This applies equally to research into the criminal justice system (Farrington & Petrosino, 2001). There are strong arguments for collecting and disseminating a regularly updated register of all randomised trials relevant to this area of work (Davies & Boruch, 2001). In mainstream healthcare the need of both providers and those receiving interventions to have ready access to all relevant high-quality research has been recognised, and the Cochrane Collaboration provides a structure by which this is undertaken. More recently, those working in education, social welfare and the criminal justice system have formed the Campbell Collaboration to address the needs of – among others – forensic mental health services (Farrington & Petrosino, 2001). However, forensic mental health straddles many professions and this fragmentation makes it difficult for healthcare professionals, criminal justice system workers, consumers, researchers and policy-makers to access relevant information. Anticipating this, Petrosino compiled a database of social, psychological, educational and criminological randomised and possibly randomised studies (Petrosino et al, 2000). Our work benefits from, supersedes and expands Petrosino's initiative. We created and surveyed a register of randomised controlled trials relevant to the management of violent and aggressive people.

METHOD

We searched 29 accessible electronic bibliographic databases (see Table 1) thought to be of relevance to the area. None of the relevant databases that we knew of was inaccessible. Published strategies for identifying randomised control trials were adapted as necessary. Participant-specific searches were then constructed (further details available from the author upon request). These broad electronic searches identified approximately 22 000 unique reports. One author (S.C.) inspected each electronic report and discarded irrelevant material; she then noted the participant group. Another author (C.A.) selected and recoded a random 10% sample. A total of 2184 reports of possibly randomised studies relevant to aggressive or potentially aggressive people remained.

Table 1 Databases searched in this study

Database	Dates covered		Number of records in total	Order of search	Proportion of selected high-priority studies (%)
	Start date	End date
AMED (Allied and Complementary Medicine Database)	1983	1998 (Dec)	29	8	0
ASSIA (Applied Social Sciences Index and Abstracts)	1987	1998 (Jun)	69	14	1
Biological Abstracts on CD	1985	1992	654	22	1
Biological Abstracts	1993	1999 (Sep)	1247	5	<1
Brainwave	U/K	2000 (Mar)	6	23	0
British Nursing Index/RCN (Royal College of Nursing Journals Database)	1988	1999 (Sep)	6	10	0
Cambridge Scientific Abstracts	1982	2000 (Jan)	4	21	0
CINAHL	1982	1999 (Oct)	1136	6	<1
Cochrane Library		1999 (Apr)	1771	15	11
Cochrane Schizophrenia Group's database of conference abstracts	1971	1999 (Dec)	109	16	3
Criminal Justice Abstracts	1999	1999 (Sep)	20	17	3
Current Controlled Trials Database		2000 (May)	3	24	0
Dissertations Abstracts	1861	1999 (Dec)	305	18	11
EMBASE	1980	1999 (Oct)	6057	4	7
GPO (Government Printing Office)	1976	1999 (Sep)	0	25	0
Health CD	1994	1999 (Dec)	212	11	0
IBSS (International Bibliography of the Social Sciences)	1951	2000 (Jan)	149	19	0
Index to Scientific and Technical Proceedings	1990	2000 (Mar)	1	26	0
International Pharmaceutical Abstracts	1970	1999 (Dec)	4	27	0
Medline	1966	1999 (Dec)	6475	2	19
National Research Register		2000 (May)	3	28	0
NCCAN (National Center on Child Abuse and Neglect)		1999 (Dec)	160	13	1
NCJRS (National Criminal Justice Reference Service)	1970	1999 (Dec)	141	12	1
PAIS (Public Affairs Information Service)	1972	1999 (Oct)	5	9	0
PASCAL	1984	2000 (Jan)	49	20	1
Petrosino bibliography1	1950	1993	122	29	1
PsycLIT	1887	1999 (Sep)	1943	3	13
Sociological Abstracts	1963	1999 (Sep)	242	7	1
SPECTR (Social, Psychological, Educational and Criminological Trials Register)	Compiled 1998		1053	1	12
ERIC (Education Resources Information Center)	1966	1998
Criminal Justice Abstracts	1968	1998
Sociological Abstracts	1974	1996
Serendipity2			18	N/A	4
All databases
Total (approximate)			22 000		88
Total relevant to management of aggressive people			2184
Total trials relevant to highly aggressive people or aggressive people with psychosis			409

A priori, we defined a subgroup of these studies as being of higher priority to forensic mental health services. These involved people who were clearly and actively aggressive, people abusive of children or spouse, sex offenders and arsonists, irrespective of age and whether they had underlying disorders. Studies of people at risk of becoming aggressive, for example juvenile offenders with no record of a specified aggressive act, were not included in this higher-priority group. Full copies of these high-priority studies were obtained and, using a data extraction sheet, S.C. recorded information on participants’ diagnoses, problematic behaviour, stage in criminal justice system, interventions and outcomes; C.A. checked the reliability of the coding by recoding a 10% random sample again. Methodological quality was scored according to the Jadad scale (Jadad et al, 1996). This rates the quality of reporting of randomisation (0–2), the quality of reporting of masking (0–2) and the quality of reporting of withdrawals (0–1). Low scores indicate poor reporting of methods and are linked with estimates of effect substantially greater than when a study is rated as good on the Jadad scale (Moher et al, 1998). This overestimate of effect from studies in which methodology is poorly reported is in keeping with other studies using different parameters to measure study quality (Juni et al, 2001). Data were stored in ProCite (Adept Scientific, Letchworth, UK) and then exported to Epi Info version 6.04d (Centers for Disease Control, Atlanta, Georgia, USA) for analysis.

RESULTS

None of the 29 databases we searched stood out as a definitive source of forensic studies (Table 1). We identified 2184 electronic reports of trials of aggressive and potentially aggressive people. These were included in 481 different journals, books or dissertations (all dissertations counted as one source). Many of the reports identified but not included in our detailed survey will nevertheless be of interest to the forensic mental health services; these lower-priority studies focused on possibly or potentially aggressive or violent people and involved groups such as juvenile offenders or prisoners for whom the level of aggression or violence was not explicit (Table 2).

Table 2 Frequencies of type of participant and problem in 2184 selected reports, categorised by priority designation

Participant/problem	n	%
Higher-priority studies
Aggressive juveniles	137	5.6
Aggressive/conduct disorder	135	5.5
Aggressive/psychotic disorder	131	5.4
Child abuse	69	2.8
Aggressive/learning disability	58	2.4
Aggressive/dementia	53	2.2
Aggressive adults	47	1.9
Sex offenders	47	1.9
Spouse abuse	46	1.9
Aggressive/personality disorder	37	1.5
Aggressive/mental illness (not psychosis)	33	1.4
Aggressive/substance misuse	15	0.6
Aggressive/autism	9	0.4
Aggressive/brain injury	9	0.4
Arsonists	3	0.1
Aggressive/epilepsy	2	0.1
Aggressive/Huntington's chorea	1	0.0
Aggressive/Tourette's syndrome	1	0.0
Lower-priority studies
Possibly or potentially aggressive adults	418	19.0
Possibly or potentially aggressive juvenile delinquents	299	13.6
Possibly or potentially aggressive adult offenders	229	10.4
Possibly or potentially aggressive/mental illness	223	10.2
Possibly or potentially aggressive juveniles	223	10.2
Possibly or potentially aggressive/conduct disorder	91	4.1
Possibly or potentially aggressive/substance misuse	48	2.1
Possibly or potentially aggressive/personality disorder	21	0.9
Possibly or potentially aggressive/learning disability	19	0.8
Possibly or potentially aggressive/dementia	16	0.7
Possibly or potentially aggressive/autism	11	0.5
Possibly or potentially aggressive/Huntington's chorea	2	0.1
Possibly or potentially aggressive/brain injury	1	0.0

Because of time constraints and despite our best efforts, we were only able to acquire and survey 300 of the 409 studies that we had identified as being of higher priority. There was an approximately 30% false-positive rate, so we estimate that about 70 studies remain outstanding. These proved inaccessible even through the British Library and direct approaches to the relevant people or institutions.

The reliability of most coding was good, with 90–100% agreement for type of publication, country of origin, year of publication, language, participants’ gender, age and previous offences, intervention, number finishing trial, duration of trial, description of randomisation, description of masking and description of withdrawal. Agreement was between 50% and 90% for number randomised, problematic behaviour and diagnosis. Outcomes were not rated reliably (10% full agreement), probably because data were difficult to identify and involved many variables. Each rater found additional outcomes. The proportion of papers for which raters agreed on most (70%) outcomes was 95%, but the numbers of scales listed below is likely to be an underestimate.

Detailed survey of high-priority reports

The final column of Table 1 shows the proportion of unique high-priority studies identified in each database as it was searched in turn. For example, after SPECTR (Social, Psychological, Educational and Criminological Trials Register) was searched, a Medline search still found 19% of the 300 studies. After 14 other databases had been searched the Cochrane Library still found 11% of the total, and Dissertation Abstracts, despite being 18th to be searched, also found 11% of the total. Most of the 300 reports we were able to acquire were fully published papers in academic journals (105 different journals), but no core set of journals deserves a reputation for having a special interest in this area, and 20% of reports were found only in dissertations or conference proceedings.

Three-quarters (76%) of randomised controlled trials relevant to the management of very aggressive people originate from the USA. Of the remaining studies, 7% were from the UK, 4% from Europe and 12% from rest of the world (1% not specified). From 1995 there has been a steady increase in the number of relevant studies (1 per month 1991–2000).

A total of 28 669 people had been randomised within the 300 trials (mean sample size 197, median 52, mode 60, range 1–1200), and 280 studies clearly reported both the numbers starting and finishing the trial: the average attrition rate was 19% (95% CI 15–27%). The great majority of reports involved men; only 15 trials (5%) solely randomised women. Most studies dealt with aggression in adulthood, although one-third focused on adolescents.

It was often difficult to ascertain diagnoses from reports, and when they were specified, often several were described in a single report. Specified diagnoses were categorised and frequencies tallied: psychotic disorders were the most commonly reported (178; 59%), followed by personality disorder (85; 18%), affective disorder (34; 11%), substance misuse (31; 10%), sexual disorders (30%; 10%), behaviour disorders (30; 10%), neurotic problems (26; 9%), problems of organic origin (21; 7%), learning disability (17; 6%) and dementia (7; 2%). Whether or not a diagnosis was specified, reports often listed the problematic behaviours of participants (Table 3). Almost a quarter of reports (n=68) specified that participants had been previously convicted.

Table 3 Top ten problematic behaviours stipulated in the trials

Specific problem	Number of different reports
Aggression
Specific
Assault	37
Destruction of property	18
Hostility	18
Murder	19
Non-sexual child abuse	17
Sexual child abuse (high-risk groups)	11
Exhibitionism	18
Paedophilia	22
Rape	36
Unspecified	11
Spouse abuse	22
Threatens to harm others	15
Unspecified	153
Behaviour
Specific
Agitation	31
Disruptiveness	11
Impulsivity	20

Multiple interventions per study were common and in 300 randomised trials over 700 interventions were evaluated, including 315 different drug treatments, 21 different packages of care, 328 named talking therapies and over 90 management techniques. It seems likely that many of these therapies are similar, making these figures an overestimate. This, however, cannot be said with certainty, as so many of the variations were specified to be discrete.

Commonly recorded outcome measures in the 300 reports were violence or aggressive behaviour (195; 65%), mental state (121; 40%) adverse effects (94; 31%), global impression (67; 22%), recidivism, arrest or time to arrest (56; 19%) and social function (58; 19%). Cognitive function, attitude or understanding (33; 11%), selfesteem (22; 7%), satisfaction with treatment by participant (25; 8%) and family function (19; 6%) were also measured. Only 13 papers (4%) reported service outcomes – admission, discharge, parole or release – and few (11; 4%) specified economic outcomes. We also recorded the specific tools used to measure outcome; in total, 345 different scales were used in the 300 high-priority trials. Most trials measured outcomes at 6 months or less: 38 (13%) up to a week; 68 (23%) between 1 week and 6 weeks; 97 (32%) between 6 weeks and 6 months. The proportion of trials (73; 24%) that were longer than 6 months was significantly larger than that seen in other surveys of evaluative studies in psychiatry (Thornley & Adams, 1998) and 7 (2%) lasted longer than 5 years.

Overall, the quality of reporting was poor (median and mode Jadad score 2). Almost three-quarters of the reports (n=220) had a Jadad score of 2 or less, and only four reports (1%) were ‘excellent’ (Jadad score of 5). These findings are similar to those of previous surveys of psychiatric trials (Thornley & Adams, 1998).

DISCUSSION

Despite the considerable limitations of even the best electronic search (Adams et al, 1994) and the inaccessibility of 25% of the high-priority sample, this survey suggests that there may be hundreds and even thousands of randomised studies directly relevant to the forensic mental health services. These trials are published in a broad range of journals, and many do not seem ever to be published except as the dissertation of a doctoral student or a presentation at a conference. Although one relevant study from the high-priority group is published per month, it is impossible to predict where that report will appear. These multiple sources are indexed in many databases. Enormous effort went into identification of these studies, and almost every database searched yielded reports of previously undiscovered trials. This underlines the need for registration of trials at inception and for a central repository of such trials (Dickersin, 1988; Hetherington et al, 1989; Stern & Simes, 1997).

The 300 studies surveyed in detail are likely to be a biased sample. Reports in English are easier to find than similar work in other languages (Nieminen & Isohanni, 1999). Work with statistically significant results tends to be more accessible than trials with equivocal findings (Egger et al, 1997). It seems unlikely, however, that a significant body of higher-quality, larger studies has gone unnoticed. Reliability of coding of the variables used in this report is high, so results should reflect the subpopulation of studies surveyed.

The overall quality of reporting was mediocre. This is also the case in other branches of psychiatry (Thornley & Adams, 1998) and medicine (Gotzsche, 1989; Vanderkerckhove et al, 1993; Fahey et al, 1995; Schulz et al, 1995a ; Cheng et al, 2000). This poor quality of reporting is likely to be associated with exaggerated estimates of effect (Schulz et al, 1995b ). It is hoped that with CONSORT (Moher et al, 2001), the quality of trial reporting should improve.

People in the trials prioritised for this study commonly had psychosis or personality disorder and exhibited extremely aggressive behaviour. The range of interventions that have been trialled is bewildering, but few studies focus on similar interventions for similar participants. Pioneers have undertaken these important and often ground-breaking studies, but there is little evidence of collaboration between individuals or institutions to rationalise interventions and increase the power of their evaluative studies. Most studies are grossly underpowered for clinically relevant outcomes. Without widespread collaboration this is likely to remain the case.

One in three schizophrenia trials contain a new outcome rating scale (Thornley & Adams, 1998). More than a third of these scales are not validated and produce biased estimates of effect (Marshall et al, 2000). The 300 high-priority studies in this survey contain 1.2 new scales per report. The proportion not validated is likely to be high. Considering the limited clinical usefulness of much scale-derived data, this seems a remarkable waste of resources in a sub-specialty in which concrete and relevant outcomes may be more plentiful than in general psychiatry.

All trials identified by the project were made available within the Cochrane Controlled Trials Register and also offered to the Campbell Collaboration to build on their SPECTR database of trials. It is hoped that this database will allow people in a range of disciplines to have ready access to trial-based information relevant to offenders and potential offenders, and to learn from past practice in order to inform future work.

This broad overview suggests that wider collaboration, rationalising treatments and simplifying outcomes could further strengthen the tradition of trialling in forensic psychiatry. Systematic reviews of these studies are overdue.

Clinical Implications and Limitations

CLINICAL IMPLICATIONS

1. ▪ Often randomised trials involving participants and interventions of interest to the forensic services do exist.

2. ▪ These studies have been difficult to find but are now available within the Cochrane Controlled Trials Register and have been offered to the Campbell Collaboration to add to their SPECTR (Social, Psychological, Educational and Criminological Trials Register) database.

3. ▪ Collaborative work is needed to evaluate practices common in forensic mental health services.

LIMITATIONS

1. ▪ The sample of studies included are the most accessible of those identified.

2. ▪ Additional studies are likely to exist in different databases or journals, or as unpublished manuscripts.

3. ▪ In the period between undertaking this research and publication of the present report many other relevant studies may have been performed.