Evidence-based journal clubs and the Critical Review Paper

The Critical Review Paper (CRP) has been part of the MRCPsych Part II examination for only two sittings and is still fairly uncharted territory. As two people who have separately sat the two papers, we examine whether evidence-based journal clubs (EBJCs) are useful preparation for the exams. We make an argument for changing the format of journal clubs to make them more relevant to the exam, as well as to promote the development of evidence-based practice.

There has recently been a change in the process of clinical decision-making across most areas of clinical practice. Newly formed bodies such as the National Institute for Clinical Excellence (NICE) will be evaluating a wide ranging evidence base as part of their role in improving clinical governance (Gilbody, 1999). The College, in keeping with this trend, introduced the Critical Review Paper to develop critical appraisal skills and evidence-based practice (Royal College of Psychiatrists, 2000). The exam tests skills necessary for reasoned evaluation of research and requires knowledge of study design, methodology and basic statistics. Candidates need to analyse methodological and mathematical data rapidly to answer a clinical question.

Candidates therefore have a very different task to that faced when presenting in traditional journal clubs. We suggest an ideal way for trainees to meet the needs of this part of the exam is to set up EBJCs open to everyone and able to fit neatly into routine academic programmes. We base this opinion on our own experiences and the observation that trainees who have taken part in such clubs are more confident about this part of the exam.

Setting up EBJCs may sound like a daunting task, but most can be quite easily integrated into conventional academic programmes. Initially it can be hard for everyone involved: some consultants even admit the difficulty of confidently teaching their juniors skills that they themselves are not familiar with. Some programmes may be more flexible and willing to allow change of format than others, and the impetus may have to come from the trainees. Many academic programmes currently have a more traditional didactic style of journal club, where on member of a team (usually the senior house officer) presents a recent paper to an audience of all grades. In such a setting there can be a tendency for the more junior doctors to feel intimidated about asking questions or making comments. They may not even be able to attend regularly, owing to clinical commitments. This style of journal club has been more popular with a broad range of clinicians (Warner & King, 1997).

There have been many formats and techniques suggested for the EBJC (Gilbody, 1996). One is to divide the audience into three groups, each of which looks at separate parts of a paper that has been introduced by the presenter, who sets questions relevant to the type of study (Sackett et al, 1999). This seems to us to be the best way to learn evidence-based medical practice (EBM) and also stimulate discussion involving all the audience. The basic ideas and format for suitable questions can be found in a range of textbooks (Crombie, 1996; Sackett et al, 1999). Other books provide worked examples of papers with suitable questions, and some clubs have found it useful to start with these (Dixon et al, 1997; Brown & Wilkinson, 1998). This type of interactive group approach to the EBJC is well suited to adult learning and also meets the demands of the CRP. This format also provides the correct forum for the trainees to practise as a group what they will be expected to do individually in the exam.

Some trainees may be intimidated by the idea of organising their seniors into groups and effectively getting them to do the work in EBJC. The statistics involved may also cause many to shy away. Unfortunately, EBJCs can tend to drift back into the traditional format because many feel they are in unknown territory and this leads to a lack of enthusiasm. To stop this happening we suggest that the EBJC initially be supervised by someone with some knowledge of critical appraisal (Geddes, 1998). We have found it invaluable to discuss the paper to be presented at the club's meeting and the style of the journal club with a senior who has an understanding of the subject, and ideally will also be attending the club. If there are no such people locally, senior clinicians might consider attending one of the many EBM training courses available.

In any EBJC, there can be a tendency for all trainees to present randomised controlled trials. This is understandable, as they are both the ‘gold standard’ of EBM and are usually first to be taught and used as an example of the subject. College guidelines do indicate, however, that the CRP is likely to become increasingly sophisticated, with more complex research methods being appraised (Royal College of Psychiatrists, 2000), including cohort studies, case control studies and meta-analysis. Ideally, the EBJC should cover all types of study design within each 6 month block. This obviously requires organisation and adequate supervision of the club.

There are other practical dilemmas in running an EBJC, such as whether the papers should be distributed to the audience in advance. Clearly this can save time during the club itself, with less time spent reading and more in discussion. However, this relies on everyone both receiving and actually reading the paper in time and requires forward planning. Another point is that the task in the exam itself is to read, digest and answer questions on a paper in just 90 minutes. Perhaps there is therefore an advantage in looking at papers ‘cold’ in the EBJC? A compromise would be to give the paper to a select few (ideally those preparing for the exam) and discuss it with them beforehand.

Once a culture has formed around this style of journal club it should be relatively easy to maintain. As time goes on more trainees will come up through the ranks having taken the CRP and will, it is to be hoped, be knowledgeable about EBM. Problems such as those discussed in this article should not deter trainees. Prospective candidates in particular should use the journal club as an opportunity to critically evaluate a paper for their own benefit and then assist others in their understanding. It is vital that trainees ask for their clubs to be run in an evidence-based way and that they be allowed time free from clinical work to attend them.

In day-to-day psychiatry one must often rely on experiences and skills outside those provided by research evidence, and the importance of evidence-based psychiatry must be carefully evaluated. Its weight in the exam, through the new CRP, must at some stage itself be carefully appraised in an evidence-based way. We have not attempted to address these issues. However, irrespective of the relative importance of EBM in clinical practice, it is now an integral part of the MRCPsych Part II examination and so must be faced by trainees, with guidance from their trainers. In fact, it would be advisable for candidates to postpone their deliberations on the importance of evidence base v. anecdote: only after having prepared for the exam can one truly appreciate the usefulness of an evidence base in clinical psychiatry. When the CRP is viewed in this way the candidate may recognise a gratifying reason to develop these particular skills.

The CRP is here to stay and it is important that trainees in the next few years are not placed at a disadvantage because of lack of adequate practice in critical appraisal skills. We suggest that the EBJC is the best place to learn these skills and encourage trainees and supervisors of academic programmes to make the most of this opportunity, both to help the trainees pass the exam and to encourage a wider understanding of evidence-based medicine.