Certification of clinical and translational researchers: An ill-conceived notion

Abstract

Abstract

The National Board of Medical Examiners (NBME) has instituted a new multiple choice examination in order to “certify” clinical and translational investigators. As experienced research educators, we argue that this certification process is unnecessary, values knowledge over competency, may be counter-productive, and is unlikely to achieve any worthwhile outcome. We lay out these arguments in the hope of stimulating a robust discussion among leaders, faculty, and learners engaged in clinical research education and training.

The National Board of Medical Examiners (NBME), with little to no input from experts in the field, has begun offering certification for clinical investigators. As leaders of programs that train clinical researchers, we believe this is unnecessary, ill-conceived, and potentially will threaten the pipeline of future clinical/translational science (CTS) investigators.

First, what is the need that this certification is intended to address? In an email solicitation of potential exam item writers, NBME described “an immediate ethical need for this assessment,” citing the organization’s mission, “to protect the health of the public through state of the art assessment of health professionals” (Personal communication from Kathleen Short of NBME, February 6, 2015). There are several reasons why this need is neither immediate nor ethically mandated. “Clinical research professionals” are not “health professionals,” in any way that is analogous to the largest group that NBME certifies: physicians (through its US Medical Licensure Examination). One can qualify to sit for this new certification exam by being “currently enrolled in, or a graduate of a Master’s or Doctoral degree program in medicine, science, at an academic institution meeting NBME criteria” [1]. An individual with an M.Sc. in Biochemistry or a Ph.D. in Molecular Genetics does not meet any definition of a “health professional.” Although an objective “competency” assessment is required in order to license physicians for the public’s safety and benefit, we can find no evidence that there is an analogous need to “license” researchers, including no evidence of any current threat to the public’s health from inadequately certified clinical research professionals. Furthermore, NBME has not proposed how their examination will mitigate their perceived threat. We believe that no matter whether the researcher has an M.D. or a Ph.D., certification, with its attendant time and cost expenditures, is not needed.

Second, the proposed approach to the certification warrants serious scrutiny. In an era emphasizing competency-based education and assessment, the proposed multiple-choice exam represents a step back to a bygone era. Competencies, or “specific learned abilities that the practitioner has adopted as a consequence of his or her education” [2], are assessed by determining the learner’s attainment of observable skills, rather than by assessments focused solely on knowledge. Undergraduate and graduate medical educators seek to translate this important theoretical construct into practice and strive to assess our students’ and trainees’ abilities to practice clinical medicine, despite the challenges of doing this reliably and objectively. In the case of CTS investigators, core competencies have been carefully developed through the NIH’s Clinical and Translational Science Award program [3]. Moreover, generally accepted and objective metrics already exist to assess a researcher’s competency, largely through the investigator’s publication of manuscripts, receipt of research grants, successful completion of formal training programs, employment, promotion, and retention in research. These may be imperfect metrics [4], but they are better assessments of research competency than multiple-choice examinations.

Third, this exam could create real problems. Given the NBME’s imprimatur in the medical realm, there is a real danger that funding organizations (perhaps even including NIH) will evolve to require that investigators receive such certification as a condition of funding. Such an approach would be counter-productive for several reasons:

• ∙ It would incentivize the established research educational programs to “teach to the test,” thereby emphasizing knowledge over skills, and arcana over substance, with no clear evidence of a salutary effect. As medical educators have long struggled to balance educational goals against the hegemony of the US Medical Licensure Examination, the CTS education community should view the history of medical education as a cautionary tale.

• ∙ It fails to recognize that most CTS education is offered within accredited educational institutions (eg, universities), generally offering advanced degrees at the conclusion of this training (eg, Master’s and PhD degrees). Accredited educational and degree programs already include extensive assessments of knowledge and skills, which, although generally not standardized across institutions, provide much more robust assessments than a single multiple-choice exam. Scores on certification exams will become yet another purely quantitative metric that is poorly predictive of accomplishment and future success.

• ∙ Providing this certification as a marker of competency may spur the development of for-profit CTS education programs that will tend to draw the least promising students and provide the minimum instruction necessary to pass the NBME certifying exam.

• ∙ Such an exam will place an additional financial burden on clinical investigators, many of whom are already making financial sacrifices in the form of longer training periods and lower salaries than their clinician peers. The evolution of test prep companies akin to Kaplan that are focused on high score attainment will likely occur. The current imperative to enhance our CTS workforce [5] may be substantially derailed by the addition of this increased financial and regulatory burden.

• ∙ Although perhaps a more subtle problem, we are equally concerned that this approach, as reflected by the exam content and sample questions available at the NBME Web site, overemphasizes the regulatory considerations in the performance of clinical trials. Although many clinical researchers conduct investigator-initiated, hypothesis-driven research, the conflation of “clinical research” with “clinical trials” has made it difficult for academic leaders to recognize that clinical research is just as “scholarly” an enterprise as basic/laboratory research. A certification examination that devotes only 14%–16% of its content to “Study Design and Statistics” (Table 1), with the remaining 85% largely devoted to regulatory and procedural content, will only exacerbate this challenge.

Table 1 Examination content outlines [1]

Finally, even if there were evidence of a current threat to public health, this exam would not mitigate it. If NBME’s assertion of harm is based on investigators conducting and/or reporting research in an unethical or inappropriate manner (as inferred from the nature of 85% of the questions) they are ignoring that there has already been substantial attention to these issues, including NIH requirements for Responsible Conduct of Research training, Institutional Review Board regulations and oversight, and required demonstration of research ethics knowledge through existing programs such as Collaborative Institutional Training Initiative certification. While one could meaningfully ask to what extent these existing programs are efficacious, there is no reason to presume that the NBME examination will do any better. Indeed, we suggest readers look at the first “sample question” in the exam brochure [1] (see text below). The answer they consider “correct” (B; coercion) is actually incorrect (the vignette meets no reasonable definition of “coercion”), supporting our concerns about both the focus and the quality of this exam.

A 48-year-old woman with stage III breast cancer is referred to an oncologist participating in a Phase 2 clinical trial testing a novel chemotherapy agent. Based on the review of the patient’s medical history and current status, it is determined that she would likely meet entry criteria for the clinical research trial. During the initial interview the investigator tells the patient, “If my wife had a similar type of breast cancer, I would enroll her in the trial.” After hearing this information, the patient decides to enroll in the clinical trial. Which of the following best describes this investigator’s actions?

(A) Clinical equipoise(B) Coercion(C) Enrollment of a vulnerable population(D) Therapeutic misconceptionAnswer: B

The comments above are a critique of the NBME’s “Investigator and Scientist Certification Examination,” which is directed at the investigators we train. However, they also are offering a “Monitor, Associate, and Coordinator Certification Examination.” We leave it to others to comment on this pathway, but we would like to emphasize that two highly regarded and commonly pursued certification pathways already exist for these research professionals: one offered by the Association of Clinical Research Professionals and the other offered by the Society of Clinical Research Associates.

In sum, we are concerned that this effort by the NBME attempts to solve a problem that does not exist and will undermine the already perilous prospects for promoting a robust pipeline of CTS investigators. NBME has instituted this certification without the input of the CTS community; we would like to encourage a thoughtful and robust conversation among leaders and faculty of training programs before this program is accepted. This conversation should include input from clinical investigators and students as to whether this proposed additional credential is warranted.