INTRODUCTION
In the past decade, there has been a substantial increase in the numberReference Cadogan, Thoma, Chan and Lin 1 and utilizationReference Purdy, Thoma, Bednarczyk, Migneault and Sherbino 2 - Reference Thurtle, Banks, Cox, Pain and Furyk 3 of online educational resources by health care practitioners, particularly in emergency medicine (EM). These resources include subscription-based content by formal educational organizations such as medical schools and journals, medical websites produced by individuals or groups of educators, and content posted on general open-access platforms such as Facebook, YouTube, and Wikipedia. These resources are meant to complement traditional learning from textbooks, primary literature, and on-shift learning.
The subset of these resources that are openly accessible online has been dubbed “free open access medical education” (FOAM).Reference Nickson and Cadogan 4 The origins of this movement stem beyond the inception of the term FOAM in 2012,Reference Nickson and Cadogan 4 as evidenced by the prevalence of EM and critical care blogs that emerged before the time in which the term was coined.Reference Cadogan, Thoma, Chan and Lin 1 This paper focuses on these freely accessible online resources and in particular, the websites, blogs, podcasts, videos, and social media outlets that make up this online community. While subscription-based or pay-walled resources have similar educational goals, they have been excluded from this term as they are not universally accessible or open to contributions.
Previously, Thoma and colleagues described strategies for learners to collate and aggregate online resources using a number of digital tools.Reference Thoma, Joshi, Trueger, Chan and Lin 5 Since then, there has been a growing trend in the FOAM movement to enhance the quality of material using checklists,Reference Paterson, Colmers, Lin, Thoma and Chan 6 rating systems,Reference Chan, Grock, Paddock, Kulasegaram, Yarris and Lin 7 - Reference Chan, Thoma and Krishnan 8 and peer reviews.Reference Thoma, Chan, Desouza and Lin 9 , Reference Sidalak, Purdy, Luckett-Gatopoulos, Thoma and Chan 10 Rapid development in this area has changed the landscape for users attempting to identify the most useful resources, appraise content critically, and contribute to content. This article outlines four strategies to help users find reliable FOAM resources, critically evaluate these resources, and participate in the FOAM movement.
Strategy one: cultivate digital mentors
Traditionally, learners have cultivated mentors from their local medical school and residency program, their elective rotations, and possibly conferences and other networking opportunities. With the large growth in online education within our specialty of EM, there is vast potential for digital mentorship in the FOAM community. Digital mentors are online accounts that learners can follow that serve several mentorship roles. First, they help steer learners toward new resources and other community members to follow. Second, by observing and interacting with these mentors, learners can benefit from role modelling, such as how to contribute to the online community, access FOAM material sustainably, and integrate this learning into practise.
Mentors may range from local to international in scope, from single author accounts to FOAM sites with teams of editors, and from general EM topics to specialty niches in EM. Each learner can personalize their choices for digital mentors to meet their learning goals. To start, learners can seek out websites and social media accounts representing their local department, residency program, or regional and national medical bodies. Next, learners can follow notable blogs, podcasts, and social media accounts for individual medical educators whom they consider to be mentors. Lastly, learners can identify FOAM resources of interest, whether in general or a specialty area of EM, and follow the social media accounts of these sites and their editors.
While the online EM community crosses multiple platforms and websites, attempting to use a variety of these can be overwhelming. Twitter is arguably the current “virtual town hall” of the FOAM community where novices and mentors interact and share ideas, making it an ideal place to start seeking out digital mentors. A recent network analysis showed how certain participants on Twitter could help to connect users to new ideas or people.Reference Riddell, Brown, Kovic and Jauregui 12 Such analyses may hold value for those who are seeking mentors. By observing and interacting with these accounts on social media platforms such as Twitter, learners benefit from the mentorship roles mentioned above.
Strategy two: browse the most popular FOAM websites
In virtual communities of practice, there is substantial sharing of information. As such, finding the highest yielding content can be difficult. General Web searches (e.g., Google, Bing, or Yahoo) are optimized for the lay population, but not for practising emergency physicians and trainees. While using custom search engines (e.g., FOAMsearch.net, also known as GoogleFOAM),Reference Raine, Thoma, Chan and Lin 13 RSS feeds and Twitter hashtags may help to filter resources for learners as mentioned in previous publicationsReference Thoma, Joshi, Trueger, Chan and Lin 5 ; it can still be daunting to decide what to read.
As emergency physicians are well networked,Reference Kovic, Lulic and Brumini 14 their followership of certain online resources may suggest usefulness. Novices can start by looking at the most popular content among the entire community of FOAM as ranked by the Social Media Index (SMi) that can be found at the following link: https://www.aliem.com/social-media-index/.Reference Thoma, Sanders and Lin 11 The SMi is built upon the premise that FOAM content is directed at the specific community of practising emergency physicians. Broad followership lends some credibility as the websites included on the SMi are of minimal interest to the general public but of significant interest to this community, and followership correlates with other measures of impact and value to the specialty (such as journal impact factor and pageviews of a website).Reference Thoma, Sanders and Lin 11 The SMi is somewhat analogous to an impact factor for journals and h-index for scholars and is the only published tool for such a measure to date that was developed specifically for online resources. This comparative index is derived from three indicators: Alexa rank of the website, Twitter followers of the most prominent editor, and Facebook likes.Reference Thoma, Sanders and Lin 11 The SMi is published on the ALiEM website and updated quarterly to capture the most popular and current FOAM sources (Figure 1).Reference Thoma, Sanders and Lin 11
Figure 1 Social Media Index ranking for November, 2016
The SMi is controversial within the FOAM community. Criticisms of the SMi include: 1) it can be gamed through the purchase of Facebook and Twitter followers; 2) the Alexa Score, which makes up a significant portion of the ratings, is proprietary, US-centric, and opaque; and 3) the constituent elements of SMi measure popularity, not quality per se.Reference Thoma 15 These criticisms are fairly similar to those of impact metrics for medical journals. However, SMi is one of the few blog metrics that has evidence demonstrating that it is correlated with other measures of impact if applied to medical journals.Reference Thoma, Sanders and Lin 11 While the relationship of the SMi with quality is being studied, the SMi is the most highly cited metric in the field and can at the very least direct learners to impactful, prominent, and generally respected resources within the EM community.
Strategy three: use existing critical appraisal tools developed for FOAM
Since the start of the evidence-based movement in the late twentieth century, critical appraisal of scientific literature has become better integrated into medical school and residency curricula. 16 Scientific articles may be appraised using tools such as the JAMA Users’ Guide to the Medical Literature Reference Guyatt, Drummond, Meade and Cook 17 ; however, secondary resources such as lectures and textbooks are seldom held to such standards. Blogs and podcasts follow in the tradition of secondary educational resources but are met with disproportionate skepticism, likely because of their novel and disruptive nature.Reference Thoma, Chan, Benitez and Lin 18 In addition, by the nature of being open access, the wide availability of the internet, and with the ease of online publication, there are inevitably many FOAM articles and resources that are purely opinion based and lacking in quality evidence. Therefore, as the reach of these resources continues to expand rapidly, new tools and additional training are needed to assist with their critical evaluation, as gestalt ratings have proven unreliable.Reference Krishnan, Thoma, Trueger, Lin and Chan 19 , Reference Thoma, Sebok-Syer and Krishnan 20
Tools including the DISCERN scoreReference Rees, Ford and Sheard 21 and Health On the Net (HON) Code of ConductReference Boyer, Selby and Appel 22 have been developed to appraise online resources for patient use. Along similar lines, a number of recent studies have developed and examined quality indicators for online medical education resources. Three resulting critical appraisal systems have been published in the literature to date. The first is the Quality Checklists for Blogs and Podcasts.Reference Paterson, Colmers, Lin, Thoma and Chan 6 The checklists are qualitative sets of criteria that learners can apply to appraise critically specific online content for their personal learning. These checklists were derived from recently published studies on blog and podcast quality indicators (a systematic review,Reference Paterson, Thoma, Milne, Lin and Chan 23 an expert Delphi with podcasters and bloggers,Reference Thoma, Chan and Paterson 24 and a Delphi with expert educatorsReference Lin, Thoma and Trueger 25 ).
The second is the METRIQ score.Reference Chan, Thoma and Krishnan 8 A recent derivation study proposed two shortened scores named METRIQ5 and METRIQ8 that are more efficient for individuals seeking to evaluate online educational resources.Reference Chan, Thoma and Krishnan 8 Lastly, the ALiEM Approved Instructional Resources (AIR) score,Reference Lin, Joshi and Grock 26 has also been shown to be reliable and consistent with expert educator opinions of quality, if used by a calibrated team.Reference Chan, Grock, Paddock, Kulasegaram, Yarris and Lin 7 , Reference Chan, Thoma and Krishnan 8 The AIR Score was developed by the ALiEM group of educators to help select high-quality FOAM resources for their self-study curriculum, the ALiEM AIR Series. The AIR Score was derived through consensus by medical educators involved in the curation and production of FOAM. It has been determined to have reasonable reliability with nine raters and to correlate with expert educator gestalt. It has not yet been validated for use among non-educators or learners.Reference Chan, Grock, Paddock, Kulasegaram, Yarris and Lin 7 , Reference Lin, Joshi and Grock 26 Ongoing studies are presently looking to assess the validity evidence supporting the METRIQ scores and to begin to optimize these quality assessment tools for widespread adoption.
The ubiquity of online education resources in EM educationReference Cadogan, Thoma, Chan and Lin 1 - Reference Thurtle, Banks, Cox, Pain and Furyk 3 makes it important for teachers to consider the role of critical appraisal skills that are required for this unique genre of information. Table 1 summarizes some tools of importance for teachers who wish to address the issue of critical appraisal with their learners. It may be worthwhile to use structured appraisal tools such as METRIQ or ALiEM AIR scoresReference Chan, Thoma and Krishnan 8 , Reference Lin, Joshi and Grock 26 and Quality Checklists for Blogs and PodcastsReference Paterson, Colmers, Lin, Thoma and Chan 6 during residency training. While the evidence regarding these scores is mounting, faculty-guided discussions around how best to appraise and apply knowledge from these secondary resources can help junior learners develop skills at critically considering online resources. In the future, these evaluation instruments may guide producers of FOAM materials to improve their resources and allow institutions to have a more objective measure for the quality of FOAM content.
Table 1 A selection of critical appraisal tools developed for online educational resources
AIR=Approved Instructional Resources; HON=Health On the Net.
Strategy four: participate by contributing and producing
Beyond navigating and evaluating online educational resources, learners can further engage in the FOAM community by participating in discussion, criticism, peer review, and creation of open educational products.
As an alternative to creating a new website or podcast to host content, new developers should consider contributing new content to existing resources (Table 2). Not only will this save content developers the time and effort of creating a new platform, but also many of these sites incorporate a peer-review process to provide feedback and coaching for new contributors. Finally, these resources will offer a considerably larger readership for newly contributed material than most new sites.
Table 2 A selection of EM FOAM outlets that have open submission processes
EM=emergency medicine; FOAM=free open access medical education.
Other practical and valuable ways of contributing to FOAM resources include participating in the peer review of content, reporting errors in blogs or podcasts to authors, commenting and engaging in a debate, and participating in discussions by joining in Tweet chatsReference Choo, Ranney and Chan 27 and online journal clubsReference Roberts, Perera and Lawrentschuk 28 (e.g., #JGMEScholarReference Lin and Sherbino 29 and #ALiEMJCReference Lin, Joshi, Hayes and Chan 30 ).
Next steps
The online education landscape is rapidly evolving. Some academic institutions are starting to support the production of open educational resources that have lead to the continued success of several prominent FOAM resources such as Core EM, which is supported by New York University,Reference Swaminathan and Core 31 and Emergency Medicine Cases, which is supported by the University of Toronto Division of Emergency Medicine and the Schwartz/Reisman Emergency Medicine Institute.Reference Chartier and Helman 32 New definitions of scholarship are now created by international consensus, highlighting key changes on how online educational resources might be reconceptualized as scholarly work by academic institutions.Reference Sherbino, Arora and Van Melle 33 Additionally, some institutions have begun rewarding the development of FOAM products as recognized acts of scholarship that add value toward promotion.Reference Cabrera 34
We encourage both educators and learners to use the strategies in this article to integrate the use of FOAM into their teaching and continuing medical education (CME). Innovative teaching strategies using FOAM, such as the “flipped classroom,” are increasingly described in medical education literature.Reference Sherbino, Chan and Schiff 35 , Reference Prober and Khan 36 The appearance and popularity of FOAM-related projects such as online journal clubsReference Roberts, Perera and Lawrentschuk 28 - Reference Lin, Joshi, Hayes and Chan 30 suggest both interest and advantage in these novel self-learning formats.
Notably absent from the menu of strategies presented within this article is guidance on how to find and evaluate educational videos. While many resource producers embed videos from VimeoTM or YouTubeTM onto their websites, many video-based resources are uploaded directly to video-hosting websites. Research in this area is sorely needed, as contrary to blog posts and podcasts, there are currently no guidelines to assist in the production or critical appraisal of these resources, nor are there strategies to find high-quality channels.
To foster FOAM integration into formal medical education, it will be important to develop a more rigorous approach for content production and evaluation techniques. While we describe advancements in the assessment of production quality, further development of more robust measures of educational quality and tools for learner evaluation is warranted. Those motivated to pioneer in this niche may benefit from incorporation of existing educational techniques such as identifying a target audience and subject matter, developing a formal curriculum (e.g., using Kern’s six-step framework),Reference Kern, Thomas and Hughes 37 and incorporating measures for evaluating learners beyond attitudinal data (e.g., using Kirkpatrick levels of evaluation).Reference Yardley and Dornan 38
CONCLUSIONS
It is an exciting time to be both an EM practitioner and a citizen of the online community. We have presented four strategies for navigating and evaluating online resources for medical learners at all levels. Because of the ease of publication in the online environment, vigilance around these resources should be maintained, and we emphasize that FOAM resources only complement and do not replace traditional learning from textbooks and reading of primary literature. However, both educators and learners should be familiar with the critical appraisal processes and thoughtful educational design of FOAM to become informed consumers and producers of online educational resources—a medium of medical education that is only expanding in its impact on the practise of EM.
Competing interests: NST reports receiving stipends for his roles as a Social Media Editor for Annals of Emergency Medicine and Emergency Physicians Monthly. BT is the founder and editor of CanadiEM.org, as well as an editor at ALiEM.com and EMSimCases.com. TC is an associate editor at ALiEM.com and a senior editor at CanadiEM.org, writes and assists with FeminEM.org and Emergency Medicine Cases, and has received teaching honorariums for her duties within the ALiEM Faculty Incubator.
