Trainee psychiatrists' assessment outcomes

Abstract

Aims and Method

The Record of In-Training Assessment grades of all registrars in the psychiatric specialties in the UK were analysed for each of three successive years, specialty by specialty, and compared with the average outcome for registrars in all disciplines over the same period.

Results

Over 98% of psychiatry registrars were assessed as satisfactory, 1.2% needed targeted training and 0.7% were graded as unsatisfactory, requiring repeat training. Targeted training and repeat training led to a satisfactory outcome in the majority of cases.

Clinical Implications

The process of assessment needs to become more objective, and should do so as the Royal Colleges develop better measures of clinical competence.

One of the most important aspects of the introduction of the unified specialist registrar grade in 1996-7 was the requirement for a formal annual documented assessment (Department of Health, 1998). The Record of In-Training Assessment (RITA) was developed, based on reports from educational supervisors about the clinical competence of the trainees, trainees’ logbooks in some specialties, and generic skills according to the criteria set out in Good Medical Practice (General Medical Council, 2001). The assessments were conducted by a panel representing the relevant deanery, the deanery specialty training committee and the Royal College concerned. The RITAs are graded as C if satisfactory (or G in the final year of training), D if recommended for targeted training and E if recommended for intensified supervision and repeat training. RITA F is a record of out-of-programme experience such as research, for which up to 12 months’ credit is usually given by most specialties.

The process of assessment took some time to develop and is not without criticism, especially concerning the lack of objective measurements of competency (Wragg et al, 2003). The principles of assessment were nevertheless rapidly accepted by the specialist registrars themselves, and they and their seniors recognise the value of the process of formative appraisal and summative assessment, which will soon be required of all members of the profession to inform their revalidation as doctors.

Method

Postgraduate deans in the UK have audited the results of the RITAs for all specialist registrars in their deaneries over the past 4 years. The analysis includes registrars on type 1 programmes leading to a Certificate of Completion of Specialist Training (CCST), whether holders of national training or visiting training numbers, and those on short-term fixed (type 2) appointments, whether as locums for training (LAT) or on a fixed-term training appointment (FTTA). The records reflect an average 87% ascertainment each year of those eligible for assessment, excluding recent joiners in the year and leavers assessed as satisfactory (G) the previous year. The shortfall of records, after allowing for sickness absence and maternity leave (1%), was mainly among registrars on short-term appointments (LATs and FTTAs), who left without adequate documentation.

Results

The outcomes of the assessments in the psychiatry specialties for each of the years 2000, 2001 and 2002 are shown in Table 1 and are compared with the average for registrars in all specialties over the same 3-year period. The results are consistent over each of the years. Overall 95% of psychiatry specialist registrars were graded as satisfactory (C and G); 1.2% needed targeted training (D) and 0.7% (E) needed to retrain for a defined period. An average of 2.7% of psychiatric specialist registrars were out of programme and given RITA grade F, reflecting the smaller proportion of psychiatrists in training who undertake research compared with the national average of 6.7%. Almost all RITA grade F reports are satisfactory, and overall the outcomes of C, F and G grades combined are probably not very different across all specialties if allowance is made for the different sizes of the specialty pools and the different durations of training – i.e. the ratio of the RITA grades G to C will be higher in shorter programmes. The grades of all psychiatry registrars assessed in the 12 months to 30 September 2002 are shown by specialty in Table 2 and compared with the national total for all specialist registrars in the same year. Numbers are too small in some specialties to permit valid comparison of the outcomes in any one specialty with another.

Table 1. Grades awarded in Record of In-Training Assessment of specialist registrars

		Grade (%)
	SpRs assessed n	C	D	E	F	G
Psychiatry
Yearly
2000	1001	73.6	1.3	1.0	4.1	20.0
2001	900	76.3	1.0	0.4	2.3	19.9
2002	998	75.8	1.2	0.6	1.8	20.6
Average 2000-2002	966	75.2	1.2	0.7	2.7	20.2
All specialties
Average 2000-2002	12 772	76.9	1.8	1.5	6.7	13.1

Table 2. Comparison of Record of In-Training Assessment grades in different psychiatric specialties

		Grade C		Grade D		Grade E		Grade F		Grade G
	SpRs assessed	n	%	n	%	n	%	n	%	n	%
Psychiatry
Child psychiatry	197	155	78.7	3	1.5	3	1.5	3	1.5	33	16.8
Forensic psychiatry	70	45	64.3	1	1.4	2	2.9	2	2.9	20	28.6
General psychiatry	438	344	78.5	4	0.9	1	0.2	7	1.6	82	18.7
Learning disability	66	51	77.3	1	1.5	0	0.0	0	0.0	14	21.2
Old age psychiatry	186	132	71.0	2	1.1	0	0.0	5	2.7	47	25.3
Psychotherapy	41	29	70.7	1	2.4	0	0.0	1	2.4	10	24.4
Total	998	756	75.8	12	1.2	6	0.6	18	1.8	206	20.6
All specialties	13 035	9990	76.6	233	1.8	217	1.7	790	6.1	1805	13.8

Reasons given for assignment to grade D included poor communication skills, poor interpersonal skills, lack of competencies in particular areas, weaknesses in management or organisational skills and weakness in research, as well as poor record-keeping and lack of documentation. Reasons given for a grade E were broadly similar but usually multiple and more severe, and judged to necessitate retraining. In some other non-psychiatric specialties, failure to pass College examinations was a barrier to progression of training. Nationally across all specialties, three-quarters of those graded D and more than half of those graded E had a subsequent satisfactory outcome at their next assessment up to a year later. However, about a quarter of those graded E in all specialties subsequently resigned or were withdrawn from the programme (Tunbridge et al, 2002).

Discussion

There is concern that a proportion of trainees, particularly those in short-term appointments (notably locums in post for 3-12 months), escape the assessment process. No one should leave an appointment without a properly documented assessment. A RITA grade D should not be perceived as damaging because it does not prevent progression of training; rather, it should be seen as constructive, and arguably used more often to focus training on areas needing attention. The outcomes at subsequent assessments are satisfactory in 75% of cases. A RITA grade E, on the other hand, does indicate serious concern about the trainee’s progress such that repeat training is needed, which inevitably postpones the achievement of the CCST. Trainees have the right of appeal against an E grade to the postgraduate dean, and a new panel of external as well as internal assessors is usually convened when necessary. The original assessment is usually upheld, but in some cases it has been modified or overturned by the appeal panel. It is encouraging that after further repeat training over half of those given a grade E nationally across all specialties achieved a satisfactory outcome. Those who at subsequent assessment again received an E grade did not usually complete their training programme and either withdrew voluntarily or at the direction of the deanery concerned. Trainees whose contract of employment is terminated have the subsequent right to go to an employment tribunal. However, postgraduate deans do their best to help trainees find a more suitable career direction.

The medical Royal Colleges have developed curricula for their specialties and are also developing measures by which the defined competencies required at different stages of training can be recorded. These will greatly help the assessment process to become more objective, but will require more time and effort on the part of trainees and trainers. The records of assessment continue to develop, and should inform the process of revalidation of doctors and reassure the public about the quality of doctors in training.