2 - Workplace-based assessment methods: literature overview

Summary

This chapter provides a short introduction and background to some of the WPBA methods: the long case, Assessment of Clinical Expertise (ACE), multi-source feedback (MSF), mini-Clinical Evaluation Exercise (mini-CEX), Direct Observation of Procedural Skills (DOPS), case-based discussion, and Journal Club Presentation.

For each assessment method, what the approach practically involves is first defined, before considering the key messages and research evidence from the literature.

The long case

Across most medical specialties, the traditional long case has historically occupied a central and critical role in the evaluation of clinical skills (Weiss, 2002). In the long case, trainees are given 30–60 min of unobserved time to interview and examine an unstandardised patient, before presenting and discussing the case with one or more examiners. This assessment can take up to an hour.

For examination purposes, the underlying belief is that within a single long case, active and usually unstructured questioning by an experienced examiner can determine a trainee's competency. The key assessment strength of this approach is that trainees are required to formulate differential diagnosis and management plans for real patients in an actual clinical setting. However, the method has been criticised for the poor reliability of its assessments, and the lack of direct examiner observation of the trainee/patient encounter (reducing the validity of assessments). Consequently, a new instrument for undertaking long-case assessments with psychiatric trainees has been developed – the ACE.

Reliability of the long case

Concerns have repeatedly been voiced about the reliability of information generated through the traditional long case. This is because it is usually based upon a single patient encounter and unstructured examiner questioning. This causes three problems (Norcini, 2001, 2002).

Inter-case reliability. The long case is typically based on one in-depth patient encounter. However, trainees’ performances will vary across cases, reflecting their strengths and weaknesses across different patient problems, and the different responses of patients to them. Good inter-case reliability requires that a larger number, and broader sample, of different cases are included.

Interrater reliability. The long case is typically based upon the scores of no more than two examiners. Research shows that examiners differ in their ratings when assessing the same event. Good interrater reliability requires that multiple examiners are used.