Changes to the MRCPsych examinations

The MRCPsych Examinations were analysed by a professional educationalist, Dr Helen Mulholland, in 1998 and a working party, chaired by the Dean, was set up to examine what changes would be desirable to increase the reliability and validity of the Examination, and to ensure it is in keeping with the principles of ‘adult learning’. In June 1999 the working party agreed that an option appraisal should be made of the alternatives proposed, and that this should be subject to a wide ranging consultation process with all relevant parties. The final recommendations were considered and agreed by the Court of Electors in December 1999.

Part I Examination

At present the Part I MRCPsych Examination consists of a multiple choice questionnaire (MCQ) examination and a clinical examination. It is taken after at least 12 months' experience in psychiatry.

MCQ examination

The main issues to be addressed when considering the current format of the MCQ paper were that the ‘stem’ technique often results in non-discriminatory questions, and the format inhibits the testing of competence in diagnosis and aetiology. The peer-referenced marking procedure penalises average candidates in a good cohort. The following modifications have, therefore, been agreed :

1. (a) With effect from Autumn 2001, the format of all MCQ questions will be modified into Individual Statements, reducing the total number of questions from 250 to 200.

2. (b) At the end of the Autumn 2000 MCQ Examination, candidates will be asked to complete a pilot examination consisting of 10-15 Extending Matching Items (EMI) which will not contribute to their examination result. An evaluation will then take place of the performance of EMIs against the standard MCQs with a view of phasing in EMIs as appropriate.

3. (c) The present peer-referencing procedure of marking will be replaced by fairer criterion referencing of MCQ and EMI scores.

4. (d) Questions where candidates' responses correlate poorly with their overall performance will be removed and the paper will be re-marked without these questions.

Clinical examination

The Working Party agreed that it was essential to continue to include a long case in the MRCPsych Examination despite its inherent variability. It was concluded that this assessment should be included in the second part of the examination, and that the Part I clinical examination be replaced by an Observed Structured Clinical Examination (OSCE). The OSCE format is suitable for assessing a range of essential core skills that a psychiatrist should possess but is not so effective in the assessment of more complex abilities which must be tested in the Part II Examination.

OSCEs have the advantage of being able to test clinical competence using a number of different scenarios in a relatively short period of time in a wellstandardised format. A minimum of 12 stations will be used comprising clinical scenarios including written vignettes, simulated patients and video material. It should be possible to examine 400 candidates in three or four centres using between 50 to 60 examiners.

As the introduction of OSCEs is a major undertaking, a new OSCE Working Party has been set up to oversee this process. Several pilot OSCE examinations will be required and they will not be introduced into the Part I Examination until Spring 2003.

Part II Examination

In view of the increasing number of candidates taking the Examination and the difficulties in finding sufficient clinical centres and examiners, it has been agreed that from Autumn 2001 screening criteria for the written papers in the Part II Examination will be introduced to determine eligibility to progress to the clinical examination.

MCQ papers

In order to address similar concerns to those raised in relation to the Part I - MCQ Paper, it has been agreed that the Basic Sciences and Clinical Topics Papers will be amalgamated to produce one MCQ paper comprising of 200 individual statements, with an emphasis on clinical topics questions.

If EMIs are shown to be successful in Part I, it is proposed that EMIs are also phased into the Part II MCQ Paper in Spring 2003.

As in Part I, the present peer-referencing procedure of marking will be replaced by criterion referencing of MCQ and EMI scores, and all questions with negative biserial correlations with overall score will be removed and the paper will be re-marked without these questions.

Essay paper

The key aim of the essay paper is to test candidates' ability to marshal evidence, synthesise and interpret the facts to present a coherent and logical argument. In order to give a greater opportunity to achieve this, it has been agreed that from Autumn 2001 candidates will be asked to write a single essay in 90 minutes with a choice of five topics requiring the integration of knowledge from the subspecialities with themes from general psychiatry.

A new Essay Panel will be established to mark scripts using a standardised marking scheme.

Critical review paper

This part of the Examination is relatively new and it is too soon to make a clear appraisal of any changes which may be required. However, it will be kept under regular review. It has also been agreed that the peer-referenced marking of this paper will be replaced by criterion referencing of scores.

Individual Patient Assessment (IPA)

There are similar problems with the Individual Patient Assessment (IPA) as in the Part I Clinical Examination. In addition, there is currently an excessive emphasis on history-taking and insufficient scope to test clinical reasoning and decision-making. However, it was agreed that it is essential to retain a single case presentation in this part of the examination. In order to address the problems identified, the following changes to the IPA examination have been agreed.

The time of examination of the candidate by the examiners will be increased from 30 to 40 minutes with effect from Spring 2003. In the interview with the examiners :

1. (a) there will be less time spent on the delivery of the history and greater stress placed on differential diagnosis and management ;

2. (b) there will be an exploration of aetiological factors in more depth and discussion of psychodynamic formulation.

Patient Management Problems (PMP)

At present the vignettes presented to candidates in this part of the Examination are not sufficiently structured, and much of the material presented could be examinable by written paper.

It has been agreed, therefore, that the present PMP Examination will be replaced by a Structured Oral Examination. Standardised vignettes, which will include suggested probes for examiners, will be developed by a new Structured Oral Examination Panel. These will test diagnostic skills, the clinical application of knowledge, basic science and clinical reasoning. Changes to the PMP component will take effect from Spring 2003.

It is hoped that, through careful planning and development of the new examination, candidates and tutors will not be adversely affected, and a more robust instrument will be established for the assessment of candidates' core knowledge and the ability to apply it in the assessment, management and treatment of patients with psychiatric illnesses.

Provisional dates for implementation of changes to the MRCPsych examination

February 2000

MRCPsych course organisers and College tutors informed of approved proposed changes.

September 2000

Proposed new examination material to be included in MRCPsych courses.

October 2000

Pilot EMI questions at end of Part I Written Paper. (These will not count towards final marks at this stage.)

October 2001

First Individual Statements in Part I and Part II Examination. Introduction of new Essay Examination. Introduction of screening criteria for written papers to determine eligibility for entry into the clinical part of the examination.

June 2002

OSCE : Pilot I.

October 2002

OSCE : Pilot II.

Spring 2003

EMIs to be phased into both parts of the examination. OSCEs to replace the Part I Clinical Examination. Changes to IPA and PMP examinations to be implemented.