In recent years, the climate of postgraduate neurosurgical training has undergone dramatic changes. Opportunities for direct neurosurgical instruction have been affected by duty-hour restrictions, increased emphasis on patient safety, rapid adoption of sophisticated neurosurgical techniques and resource constraints propagating the need to improve the efficiency of neurosurgical care.Reference Jagannathan, Vates and Pouratian 1 - Reference Haji, Dubrowski, Drake and de Ribaupierre 3 As a result, residents must acquire requisite knowledge and skills in less time and in more complex environments than ever before. Many choose to engage in at least one, if not multiple, additional years of fellowship training to gain the necessary technical skill to be ‘experts’ in their chosen subspecialty.Reference Desbiens, Elleker and Goldsand 2 , Reference Long 4 In turn, this raises important questions regarding what competencies fall within the purview of ‘general’ versus ‘subspecialty’ neurosurgical practice.Reference Toyota 5
This changing climate has not gone unnoticed by neurosurgical educators in Canada. In 2010, the Royal College of Physicians and Surgeons of Canada’s (RCPSC’s) Specialty Committee for Neurosurgery revised the specialty training requirements and objectives of training. Among the changes were an increase in ‘on-service’ training time from 36 to 42 months and the specification of procedural skills that graduating residents are expected to be proficient in performing, versus those they should be able to describe in detail but not necessarily perform. 6 Recently, as part of the RCPSC competence by design initiative 7 and the broader competency-based medical education (CBME) movement,Reference Long 4 , Reference Frank, Snell and Cate 8 neurosurgical educators have also been tasked with developing milestonesReference Green, Aagaard and Caverzagie 9 , Reference Peterson, Nabors and Frishman 10 to detail the abilities expected of neurosurgical trainees ‘across the continuum of their careers, from the start of residency training and throughout professional practice’. 11
Despite these broad changes to postgraduate neurosurgical education, there is relatively little literature to guide curricular reform. For instance, it may be helpful for neurosurgical educators to understand trainee and faculty perspectives on graduates’ readiness for independent practice as they begin to delineate the level of competence (i.e. the level of ability or performanceReference Frank, Snell and Cate 8 ) expected of neurosurgical trainees through CBME milestones. This report outlines the findings of a national survey of Canadian neurosurgical program directors (PDs) and graduate residents from 2011 conducted by the authors, with the support of the RCPSC Specialty Committee for Neurosurgery. The survey examined participants’ perspectives on whether these graduates had achieved a level of proficiency expected of an independently practicing neurosurgeon among domains contained in the 2010 objectives for training as a marker of their perceived readiness for independent practice across various domains of neurosurgical practice.
Methods
This cohort study used survey methodology and was conducted between October 2011 and August 2012.
Participants and Data Collection
Ethical approval was obtained from the University of Western Ontario Health Sciences Research Ethics Board. Administrators and PDs from each of the 14 Canadian neurosurgery training programs were contacted to generate a list of graduates in June 2011, resulting in 25 graduates across 11 programs. Each of these graduates completed training before the institution of the 2010 Objectives for Training. Because contact information was available for 22 graduates only, these individuals and the corresponding 11 PDs were invited to participate in the study. PDs were asked to complete one survey for each graduate of their program. Participants were emailed a letter of information and a unique, anonymized survey link. Informed consent was assumed from those who completed the questionnaire. Biweekly reminders were sent to nonresponders for the duration of the study.
Survey Design
Two structured, self-administered online questionnaires were developed for graduates and PDs, respectively (Appendix 1). In both surveys, participants provided basic demographic information, then rated whether the graduate in question possessed the knowledge and skills expected of an independently practicing neurosurgeon on items pertaining to the 2010 RCPSC Objectives for Training in neurosurgery. 6 These items were separated into four domains: (1) nonprocedural knowledge and skills (8 items); (2) the ability to independently perform general neurosurgical procedures (43 items); (3) knowledge regarding the indications, complications, nature and purpose of subspecialized neurosurgical procedures (26 items); and (4) nonmedical expert CanMEDS competencies (6 items). To reduce the length of the survey, a single survey item asked participants their opinion regarding graduates abilities on each of the nonmedical expert CanMEDS competencies (e.g. communicator, collaborator). Responses were scored on a five-point Likert scale ranging from one (strongly disagree) to five (strongly agree). Finally, open-ended questions were used to solicit participants’ opinions on factors that may have prevented the graduate from achieving the expected level of competence on any of the items.
Data Analysis
Statistical analyses were completed using SPSS, version 21 (IBM Corp, Armonk, NY). Demographic characteristics were summarized using means for continuous variables and proportions for dichotomous variables. Likert responses were treated as ordinal data and summarized using median and range of responses. However, the Mann-Whitney test was used to determine if differences between graduate and PD responses were statistically significant with the Bonferroni correction to adjust for multiple comparisons. In addition, the proportions of respondents agreeing (three, four or five on the Likert scale) and disagreeing (one or two on the Likert scale) on each item were calculated. Finally, responses to open-ended questions were summarized based on predominant themes identified by the authors.
Results
Response Rate and Demographics
Of 47 questionnaires distributed, 30 responses were received (overall response rate, 64%). Thirteen graduates completed the survey (59%), two graduates declined to participate and the remaining seven did not respond. Seven PDs (64%) completed 17 surveys (68% of PD surveys), with no responses from the remaining four PDs. Most graduate nonresponders were from Ontario; however, PD nonresponders did not fit a specific pattern (each was from programs varying in size and geographic region).
Participant demographics are summarized in Table 1. Most resident respondents were graduates of Canadian medical schools and nearly half completed a graduate degree or infolded clinical fellowship during residency. PDs had an average of 3.4 (range, 0-6) years’ experience leading the training program, with a mean of 5.3 graduates completing residency during this time. The number of responses by program is depicted in Figure 1, demonstrating the distribution of small, medium and large programs in the sample.
Figure 1 Distribution of graduate and program director responses by neurosurgical program.
Table 1 Participant and program demographic characteristics
CMG=Canadian Medical Graduate; IMG=International Medical Graduate; PD=program director.
* 3 programs did not have any graduates in 2011 and thus were not included in the study.
Perceptions Regarding Graduates’ Knowledge and Skills
No significant differences between the mean ranks of PD and graduate responses were observed (p>0.05 in all cases after correcting for multiple comparisons). Both groups perceived that the vast majority of graduates from this cohort met all objectives related to nonprocedural knowledge and skills and CanMEDS competencies to the level expected of an independently practicing neurosurgeon (Table 2).
Table 2 Perspectives on nonprocedural knowledge and CanMEDS competencies
CNS=central nervous system; PD=program director.
* Median and range of responses on a 5 point Likert scale, from 1=strongly disagree to 5=strongly agree.
† Indicates number of respondents who agree that the graduate in question achieved competence on the objective at the level of an independently practicing neurosurgeon (i.e. who provided a rating of 3, 4 or 5 on the Likert scale, indicating neutral, agree or strongly agree).
Almost all participants perceived that graduates in this cohort could perform the expected cranial and spinal procedures independently (Table 3). Similarly, with respect to oncological, vascular, pediatric, functional and peripheral nerve procedures, fewer than 15% of graduates were perceived to not be able to perform a selection of these procedures independently. Interestingly, with respect to craniotomy for repair of cerebral aneurysms, PDs indicated that 4/17 graduates (24%) would not be able to perform this procedure independently.
Table 3 Perspectives on ability to independently perform core neurosurgical procedures
CSF=cerebrospinal fluid; ICP=intracranial pressure; OC=Occipito-cervical; PD=program director.
* Median and range of responses on a 5 point Likert scale, from 1=strongly disagree to 5=strongly agree.
† Indicates number of respondents who agree that the graduate in question achieved competence on the objective at the level of an independently practicing neurosurgeon (i.e. who provided a rating of 3, 4 or 5 on the Likert scale, indicating neutral, agree, or strongly agree).
Although the majority of graduates were perceived to possess the level of knowledge expected of an independently practicing surgeon for subspecialty procedural skills listed (Table 4), between one and six graduates were not felt to have achieved this level. This was particularly the case for complex vascular procedures (e.g. extracranial/intracranial bypass), specialized techniques (e.g. stereotactic and functional procedures, stereotactic radiosurgery, endovascular embolization/stenting), uncommon spinal procedures (e.g. transoral spinal decompression) and complex peripheral nerve cases (e.g. surgical management of brachial plexus lesions, complex nerve tumours and sympathectomy).
Table 4 Perspectives on knowledge of ‘subspecialty’ procedures
CTS=cubital tunnel syndrome; EC/IC=extracranial/intracranial; PD=program director.
* Median and range of responses on a 5-point Likert scale, from 1=strongly disagree to 5=strongly agree.
† Indicates number of respondents who agree that the graduate in question achieved competence on the objective at the level of an independently practicing neurosurgeon (i.e. who provided a rating of 3, 4 or 5 on the Likert scale, indicating neutral, agree or strongly agree).
PD and graduate responses to open-ended questions were concordant and thus were analyzed as a single group. The most commonly cited reasons for a graduate not being able to independently perform or having limited knowledge of a given procedural skill were: (1) lack of exposure during training related to an inadequate caseload (e.g. percutaneous treatment of trigeminal neuralgia, endoscopy, complex vascular and peripheral nerve surgery), (2) lack of a formalized program for the subspecialty performing the procedure (e.g. functional neurosurgery) at the graduates’ training centre or (3) a reluctance to involve residents in specific complex cases (e.g. endovascular or complex endonasal procedures). Some graduates indicated that they were receiving additional exposure to some of these techniques (e.g. endoscopy) during fellowship training. Several participants also indicated that although there was sufficient exposure to cases during residency, because some graduates would not be performing certain procedures (e.g. cranial-cervical instrumentation, craniotomy for cerebral aneurysm clipping, pediatric neurosurgical procedures) routinely in their practice, they would not feel comfortable performing these independently. These respondents felt such procedures should be performed by subspecialized surgeons with fellowship training.
Discussion
First introduced by Halstead in 1889,Reference Long 4 a fundamental assumption of the traditional model of postgraduate surgical education is that the sheer volume of cases a trainee is exposed to will ensure mastery of expected competencies.Reference Reznick and MacRae 12 However, given the time and resource constraints pervasive in today’s surgical training climate, current residents are likely not exposed to the same volume and breadth of clinical experiences as residents from even a decade ago. Thus, it is not surprising that this model is being phased out in favour of CBME in an effort to foster outcome-driven curricula and assessments that build over time and focus on observable knowledge and skills. 7 , Reference Frank, Snell and Cate 8 In turn, the operationalization of CBME through specialty-specific milestonesReference Green, Aagaard and Caverzagie 9 , 11 and entrustable professional activitiesReference Peterson, Nabors and Frishman 10 , Reference ten Cate and Scheele 13 may provide faculty and trainees with a transparent pathway to the acquisition of competencies required for independent neurosurgical practice.Reference Frank, Snell and Cate 8 , 11 Because these milestones, entrustable professional activities and other curricular changes will be developed through expert consensus among neurosurgical educators, data on practice patterns, societal needs and competence of graduates in the various domains of neurosurgical practice may help advance these efforts.
The current study adds a small piece to this body of literature, providing new insight on the perceptions of Canadian neurosurgical graduates’ abilities related to the current objectives of training. However, before interpreting the result, several important limitations should be considered. First, because these data pertain to a single graduating class, the results may not be generalizable to future cohorts or programs not represented in the study. Similarly, the potential for nonresponder bias must also be considered given the response rates of 59% and 68%. Second, because of the small sample size, we are not able to comment on the impact of demographic characteristics (e.g. whether graduates completed medical school in Canada or abroad, the number of fellows in the training program) on the study findings. Third, because the questionnaire was not validated before use, participants may have misinterpreted some questions. In particular, participants may have interpreted items related to knowledge of subspecialty procedures as asking about graduates’ ability to perform these techniques. Although we set this question apart on a separate page with distinct instructions from the preceding section on performing core procedural skills, we cannot be certain that the question was answered with our intent in mind. Finally, the self-report measures used in the study introduce the potential for recall bias and ‘halo’ effects. Because we do not have objective data from in-training reports, RCPSC certification examinations or practice performance data to corroborate the survey results, the study findings should be interpreted with due caution.
The language of the survey questions themselves must also be considered in interpreting the results. Participants were explicitly asked whether graduates were proficient at the level of an ‘independently practicing neurosurgeon’. This differs from the expectations outlined in the RCPSC Objectives of Training, which state that trainees must ‘demonstrate proficiency’ in performing the procedural skills listed in Table 2 and for those listed in Table 3, be able to “describe the…procedural skills, along with… [their] indications…, …nature and purpose…, and…potential complications”. 6 Some graduates may have reached ‘proficiency’ sufficient to meet a given objective (e.g. craniotomy for aneurysm repair), but not feel comfortable as a junior consultant to perform the surgery without a senior colleague. That multiple respondents indicated they believed some procedures should only be performed by subspecialists supports this notion. Thus, these results should not be interpreted to reflect graduate or PD opinions about whether training objectives were met per se, but rather whether graduates had reached an independent level of practice in these domains. This threshold for proficiency is likely too high for some of the objectives and not reflective of the expectations of neurosurgical graduates and may account for the lower ratings observed on some survey items. However, this distinction does raise an important issue: there is a need to clarify exactly what it means for a graduating neurosurgical resident to be ‘proficient’ within each domain described in the objectives. Such a clarification is beyond the scope of this article, but underscores the importance of developing training milestonesReference Green, Aagaard and Caverzagie 9 and transparent definitions of the minimum competence standard expected of neurosurgical graduates for each training objective.
Notwithstanding these limitations, we believe the results of this study may be useful to inform neurosurgical curriculum design and assessment. For instance, that nearly half of graduates pursued a graduate degree or infolded fellowship during training may reflect a desire among residents to develop subspecialized academic and clinical skills not mandated by the existing training curriculum. This finding also demonstrates the freedom of trainees in Canadian neurosurgical programs to pursue individual educational objectives. The results also paint a positive picture for the calibre of neurosurgical training in Canada, given that graduates surveyed were perceived to be proficient to the level of an independently practicing neurosurgeon in the vast majority of objectives. This is particularly true for nonprocedural skills, CanMEDs competencies and performance of core neurosurgical procedures, perhaps with the exception of ‘craniotomy for cerebral aneurysm clipping’, ‘ventricular endoscopy for tumour biopsy or excision’ and ‘percutaneous techniques for trigeminal neuralgia’. That the former two procedures were rated lower is of interest but not entirely surprising given the growing role of endovascular therapy and the relatively recent addition of ventricular endoscopy to the neurosurgical armamentarium. The perception that residents have inadequate exposure to feel comfortable performing these procedures independently and that these procedures should be performed by subspecialists underscores the importance of monitoring the adequacy of exposure and proficiency of graduating residents in these domains to inform curricular decisions both for individual programs and at the national level.
Although the majority of graduates were perceived to have a level of knowledge expected of an independently practicing neurosurgeon on the majority of subspecialized techniques, for some procedures (e.g. complex vascular, stereotactic, functional, peripheral nerve, radiosurgery, complex spine), a proportion of graduates may not receive sufficient exposure to these subspecialty cases to achieve this level of proficiency. This trend is once again not surprising given that fellowship-trained subspecialists almost exclusively perform these procedures and exposure to these cases depends on whether such services are offered at the centre in question; for some of these services (e.g. radiosurgery), patients may be sent to another centre when these interventions are indicated. In addition, at certain centers, some of these procedures (e.g. complex peripheral nerve surgery, endovascular interventions, some stereotactic and functional procedures) are not managed by the neurosurgical service, but rather by colleagues in other surgical specialties. However, because these are not procedures that graduates are expected to be able to perform, and because all Canadian medical residents must successfully complete the RCPSC certification examination (which thoroughly examines knowledge and decision-making in these domains), ratings on these objectives are arguably less critical for decision-making.
These results provide some insight into the perceived level of competence of graduating neurosurgical residents across the existing objectives of training. These data may be useful in future decision-making regarding competency milestones and training objectives as well as to articulate which of these fall within ‘core’ neurosurgical training and which should be considered subspecialty-level skills. Because of the aforementioned limitations of the data, it would be helpful to follow the trends observed in this study among future graduate cohorts to determine if the findings are isolated or if they represent the experience of Canadian neurosurgical trainees in general. In addition, because the graduates in this study completed training before the institution of the 2010 curricular reforms, it would be of interest to compare these results with trainees graduating after 2016 (who will complete 42 months ‘on-service’ during residency) or those who are trained under the CBME paradigm to determine if observed trends change with these curricular reforms. Additional data on graduates’ level of proficiency, practice patterns and changing societal needs would also be helpful as objective evidence to corroborate these results. Thus, we would advocate for ongoing monitoring of this nature that PDs and the RCPSC specialty committee may draw upon in the future to inform curriculum decisions.
Conclusions
We have conducted a national survey of graduating neurosurgical residents and PDs, to determine graduates’ preparedness for independent neurosurgical practice based on the RCPSC Objectives for Training. The results of the current study suggest that, in the majority of cases, PDs and graduates feel that this graduating cohort achieved a level of proficiency expected of an independently practicing neurosurgeon on the current list of objectives, despite the fact that these residents completed training before these objectives were incorporated into Canadian neurosurgical training curricula. Among the select number of cases in which this proficiency criterion was not met, inadequate exposure to certain procedures during training or a perception that such cases should be performed only by fellowship-trained surgeons may have been a contributing factor. The trends identified in this study should be monitored on an ongoing basis to provide much needed data to guide curricular decisions in Canadian neurosurgical training.
Acknowledgements
The authors sincerely thank the members of the Royal College Specialty Committee for Neurosurgery for their input and constructive feedback on this study.
Disclosures
FAH does not have anything to disclose.
Supplementary material
To view supplementary material for this article, please visit http://dx.doi.org/S0317167114001036
