NICE guidance in schizophrenia: how generalisable are drug trials?

Abstract

Aims and Method

To test the National Institute for Clinical Excellence (NICE) assertion that characteristics of participants in the majority of clinical drug trials in schizophrenia do not reflect clinical practice. In particular they were concerned about the relative exclusion of women, older adults and patients with comorbidity. The baseline characteristics of a sample of 600 patients with schizophrenia recruited to be as representative as possible of UK community practice were compared with those from one of the largest international drug trials of an atypical antipsychotic.

Results

Although comparisons could only be made on a limited range of characteristics the two samples were broadly comparable.

Clinical Implications

Current drug trials from pharmaceutical companies may have more relevance to clinical practice than their stated exclusion criteria may indicate.

The National Institute for Clinical Excellence (NICE) was established to advise on evidence-based practice in the UK. In its guidance for the use of the newer (atypical) antipsychotics in the treatment of schizophrenia (NICE, 2002) it echoes the concerns of prominent trialists (Thornley & Adams, 1998; Geddes, 2002) about the excess of very short-term studies which had been conducted mainly for registration purposes. It considers that the ‘… conclusions that can be drawn from the majority of studies are limited because of the lack of long-term follow up, high attrition rates…’ and also comments that ‘The generalisability of individual study results was limited by the exclusion of elderly people as well as individuals with treatment resistant schizophrenia, predominantly negative symptoms, learning disabilities, co-morbid depression and substance abuse disorder’. Most of the 147 trials overviewed in the NICE guidance were short term with high attrition rates, but there have been some longer-term studies of atypical antipsychotics in schizophrenia (Csernansky et al, 2002) and these tend to be more widely reported. However, it is unclear to what extent they suffer from the same limitations, and to what extent they reflect, and are generalisable to, routine practice. We have explored this by taking a sample of patients with schizophrenia collected specifically to reflect routine clinical practice in UK secondary care services and compared their characteristics with those of patients recruited to one of the largest controlled trials of atypical antipsychotics (Tollefson et al, 1997). We have specifically compared them on those characteristics about which NICE has expressed concern.

Method

As part of a naturalistic cohort study of routine treatment of schizophrenia (Schizophrenia Care and Assessment Programme, UK-SCAP) we have collected and followed up a convenience sample of 600 patients. Inclusion criteria were age 18 or older, a clinical diagnosis of schizophrenia, schizophreniform or schizoaffective disorder, adequate English to understand the nature of the study and give informed consent. Exclusion criteria were involvement in clinical drug trials in the preceding 30 days and follow-up considered unlikely by the clinical team. This latter condition was added to avoid including the up to one-fifth of inner city patients (Harrison-Read et al, 2002) who are transient and for whom follow-up data were highly unlikely to be obtained. The diagnosis of schizophrenia was confirmed by the local investigator by clinical assessment and examination of case notes against DSM-IV criteria (American Psychiatric Association, 1994). Patients were recruited from six centres in the UK which were chosen to give a representative spread across the regions and across an urban/rural spectrum (Belfast, Tooting, Liverpool, Tolworth, Bristol and Dumfries). Patients were recruited from team community case-loads, with 30% sampled overall (±10% per site) with an admission within the preceding year to ensure a spread of severity.

The trial used for comparison is one of the largest published, long-term drug trials in schizophrenia (Tollefson et al, 1997). This trial compared olanzapine with haloperidol across 17 countries. This was a typical ‘ regulatory’drug trial which aimed to provide evidence for registration, although with a substantially larger sample than most. Inclusion criteria were age 18 years and over and diagnoses of schizophrenia, schizophreniform disorder and schizoaffective disorder. Participants had to have clinically significant psychotic symptoms and demonstrate less than an optimal response to current treatment or have recently experienced an adverse event. Females of child-bearing age were required to be using a medically accepted means of contraception. Exclusion criteria included instability and any significant comorbid medical or psychiatric disorder or exposure to a range of listed medications. Of its 1996 participants, 139 were recruited in the UK.

Results

Of 751 patients approached for UK-SCAP, 602 entered the study over a 9-month recruitment period in 1999–2000. There were no significant differences between included and excluded individuals.

Table 1 outlines the baseline characteristics of the two samples. Contrary to expectations there was no marked excess of males in the drug trial consequent on excluding women at risk of pregnancy, but there was a modest excess of males in both samples (65%). The mean age was 5 years lower in the regulatory trial but the age range was equally extensive, with 3.0% aged 65 years and over compared with 5.4% in the UK-SCAP sample. The proportion from minority ethnic groups and those living independently were also similar, although there was considerable variation between sites in the UK-SCAP sample for minority ethnic groups and those in supported accommodation. The UK-SCAP sample had lower rates of employment in the open job market than the international sample (6 v. 13%), although this is echoed in the UK sample from the international study with a rate of 8%. Schooling and education levels are difficult to compare internationally but only 39% of the UK-SCAP sample had proceeded beyond secondary school compared with 56% of the international sample (only 43% for the UK subsample in this study).

Table 1. Baseline characteristics of SCAP v. drug trial sample

	UK—SCAP		Tollefson et al (1997)
	(n=602)		(n=1996)
Age, years: mean (s.d.)	43.1 (13.4)		38.6 (11.4)
Range	19-83		18-86
Male gender, n (%)	393	(65)	1296	(65)
White ethnicity, n (%)	511	(85)	1601	(80)
Single/divorced/widowed, n (%)	535	(89)	461	(86)
Open employment, n (%)	36	(6)	68	(13)
Accommodation, n (%)
Living alone/independent	184	(31)	191	(36)
Living with family	227	(38)	205	(38)
Age at onset, years: mean (s.d.)	23.0	(8.4)	23.9	(7.5)
PANSS positive score: mean (s.d.)	15.9	(6.1)	21.3	(6.1)
PANSS negative score: mean (s.d.)	18.2	(6.4)	24.2	(6.9)
PANSS total score: mean (s.d.)	70.2	(19.2)	90.8	(19.4)
MADRS score: mean (s.d.)	14.4	(9.8)	16.6	(8.8)

The overall symptom level was higher in the drug trial sample reflecting the positive and negative symptom scale (PANSS; von Knorring & Lindstrom, 1995) threshold scores routinely required for entry into such studies; however the mean level of negative symptoms was also higher. The levels of comorbid depression, as measured by the Montgomery-Åsberg Depression Rating Scale (MADRS; Montgomery & Åsberg, 1979), were also higher in the drug trial (16.6 compared with 14.4) (Tolleffson et al, 1998): 16+ on the MADRS is generally accepted as indicative of moderate depression and 53% of those in the drug trial compared with 39% in the UK-SCAP sample crossed this threshold.

Discussion

The belief of NICE that regulatory drug trials select highly atypical and unrepresentative patients was not borne out by our results. The age range and ethnic and gender mixes in the drug trial and cohort samples were surprisingly similar. There was no evidence either of patients with depression or those with predominantly negative symptoms being underrepresented in the drug trial. We are unable to comment on the proportions of those with comorbid substance misuse or learning difficulties. Although the UK-SCAP cohort was only a convenience sample, it was recruited by local random selection, with stratification to ensure a representative spread, and judged by the experienced clinicians in the six sites to reflect their practice.

We have only compared our clinical sample with one regulatory drug trial (and perhaps a rather atypical one because of its size and good follow-up). However, it is the results of such high profile drug trials that are likely to be read; its inclusion/exclusion criteria are fairly standard and are likely to skew the sample in the manner commented on by NICE (2002), Geddes (2002) and Thornley & Adams (1998). Overall our results give little support to the belief that the clinical and demographic characteristics of patients treated in routine UK practice differ markedly from those reported in this large-scale drug trial. The number of points of comparison presented here is relatively limited, however, and we have not shown that conclusions from published drug trials can be extrapolated without careful consideration of clinical practice. Our comparison raises the possibility that concerns about the extreme atypicality of samples included in drug trials may be exaggerated. A more detailed investigation is indicated of whether the inclusion criteria for drug trials really do result in a clinically unrepresentative sample.

Appendix

The UK-SCAP group comprises:

1. • Belfast: Dr S. Cooper, Dr M. Doherty, Rosalind McCaul

2. • Tooting: Dr R. Chaplin, Dr Yvette Dennis, Hannah Winfield

3. • Dumfries: Professor R. McCreadie, Heather Stevens, Jamie Henderson

4. • Liverpool: Dr S. O’Brien, Gillian Young, Toni Ellerker

5. • Tolworth: Dr I. A. Obuaya, Ritu Kalsi, Georgina Sutch

6. • Bristol: Professor G. Harrison, Dr A. Sipos, Matthew Spence

7. • Central: Professor T. Burns, Dr Debbie Stephenson, Jan McKendrick, Beth Barber

The writing group comprises:

1. • Professor T. Burns, Professor of Social Psychiatry, University of Oxford

2. • Dr R. Chaplin, Consultant Psychiatrist, SW London and St George's NHS Trust

3. • Dr S. Cooper, Senior Lecturer, Queen's University Belfast

4. • Professor G. Harrison, Norah Cooke Hurle Professor of Mental Health, University of Bristol

5. • Professor R. McCreadie, Director of Clinical Research, Crichton Royal Hospital, Dumfries

6. • Dr S. O’Brien, Consultant Psychiatrist, Mersey Care NHS Trust, Liverpool

7. • Dr L. A. Obuaya, Consultant Psychiatrist, SE London and St George's NHS Trust

8. • Dr Debbie Stephenson, Medical Adviser, Medical Department, Eli Lilly, UK

9. • Jan McKendrick, Health Outcomes Manager, Health Outcomes Department, Eli Lilly, UK

10. • Beth Barber, Health Outcomes Manager, Global Health Outcomes, Eli Lilly, Indianapolis

Declaration of interest

The UK-SCAP study was funded by Eli Lilly and three members of the UK-SCAP group are employees of Eli Lilly.