Success and failure in establishing national physician databases: a comparison between Canada and Israel

Abstract

Physician databases constitute an essential component of health workforce planning. However, while some countries have established functioning national physician databases, others have failed to do so. We compared the healthcare systems of two technologically and economically developed countries, Canada and Israel, which represent cases of respective success and failure in establishing physician databases. A comparative analysis was conducted using a historical-institutionalist approach to examine contemporary health policy outcomes. White papers, studies on healthcare human resources, and reports by professional committees were examined to explore the aims, interests, positions, and actions of stakeholders. In Canada, state–medical profession cooperation, deep-rooted in a longstanding regulatory bargain between the two parties, has facilitated the creation and management of physician databases, albeit limited and in need of urgent improvement, on national and jurisdictional levels. The lack of such regulatory arrangement coupled with enduring conflicted relations between stakeholders due to particular historical developments have hindered the development of an Israeli equivalent database so far. Finally, health policy outcomes may be explained against the backdrop of broader political, governance, and organisational contexts. How medical organisations respond to governmental healthcare initiatives is heavily influenced by their institutional position vis-à-vis the state, shaped by historical processes and regulatory arrangements.

1. Introduction

Workforce planning is one of the most important and challenging tasks of any healthcare system. Accordingly, many countries maintain a periodically updated national database containing essential information on health professionals, including physicians (Ono et al., 2013; Horowitz et al., 2017; Consumers, Health, Agriculture and Food Executive Agency et al., 2021). Such databases can provide comprehensive, up-to-date, reliable, and useful information concerning the overall number of licensed and practicing physicians and their geographical and sectorial segmentation, including age, years of experience, specification of medical specialties, and so forth. Additionally, by detecting gaps in medical workforce supply and distribution, which may compromise access to health services, these databases can support healthcare human resources policies, such as role substitution and delegation of medical tasks to non-physicians (Martínez-González et al., 2015; Butler et al., 2020).

The maintenance of up-to-date and comprehensive physician data infrastructure requires taking regulatory measures such as mandatory periodic re-registration of practicing physicians in a national registry administered by state agency or a professional organisation (Borow et al., 2013; OECD, 2021). In many countries, periodic re-registration is linked to a requirement to demonstrate professional competence, fitness to practice or continuing medical education (CME) credentials, sometimes as a condition of renewal of license to practice medicine or medical specialisation certificate. Although the level of institutionalisation of such mechanisms varies, there seems to be a current global trend from a voluntary to a more state-sanctioned mechanism of such policies often referred to as re-licensing, re-certification, or re-validation (Solé et al., 2014; Sandy, 2017).

Through these mechanisms, the state tries to manage its medical workforce both in terms of quantity and quality, assuming that such requirements imposed on physicians lead to better medical care. Such measures underscore the often-coercive regulatory nature of physician registries and databases rendering them not only technical projects for bureaucratic or statistical purposes but also as political arenas of potential cooperation or conflict between the state and the medical profession as will be further demonstrated.

Despite the importance of physician databases for health workforce planning, studies on such databases are scarce. This paucity of scholarship is particularly notable compared to the broad interest in healthcare workforce planning, especially regarding shortages and maldistributions of physicians both geographically and professionally worldwide (Kovacs et al., 2019; Consumers, Health, Agriculture and Food Executive Agency et al., 2021). The current literature mainly consists of case studies focusing on technical, methodological, and organisational challenges in establishing and operating a physician database in different countries. Among them are inaccurate personal information, organisational challenges in establishing and maintaining physician registries, over- and underestimation of medical workforce, and discrepancies between different data sources (Konrad et al., 2000; Lovkyte et al., 2003; Knight-Madden and Gray, 2008; Langins, 2013; Aeenparast et al., 2014; Levi and Borow, 2018). Very little attention has been paid to the political and regulatory aspects of physician databases, which require cooperation, coordination, and mutual trust between relevant actors, namely state authorities and physician organisations. These relationships are dependent on the nature of the regulatory arrangements, changing dynamics between stakeholders and often contradictory interests.

In this study, we offer a comparative analysis of two case studies – those of Canada and Israel – representing success and failure, respectively, in establishing a periodically updated physician database. Why Canada has successfully established such a database while Israel has not will be examined by focusing on the development of state–medical profession relations in both countries through the lens of historical institutionalism.

In this study, we offer a novel perspective on the formation of physician databases in three aspects: First, at the conceptual level, while building on previous works, our point of departure is in the framing of the physician database as a political space – instead of an organisational or technical one – where stakeholders' dynamics, interests, and power relations are at the centre of analysis. Second, at the methodological level, this is presumably the first comparative study conducted on physician databases. While previous works usually offered a description of a chain of events, challenges, barriers, and possible solutions within the framework of isolated test cases, we adopted a comparative methodology to allow variation that may help detect a causal mechanism. Finally, at the theoretical level, our explanation to the research question relies on the historical institutionalism approach. To the best of our knowledge, this is the first time that this approach has been applied in the context of healthcare human resource databases in the purpose of promoting better understanding of health policy outcomes by considering the pivotal role of historical contexts in explaining current contingencies.

2. Management of medical workforce data as a joint state–medical profession project

Many countries have some division of labour between state authorities and the medical profession regarding the management of medical workforce physician data. Medical associations often constitute a significant source of information on the occupational and demographic status of physicians (e.g., Czech Republic, Germany, and France). In some countries, medical associations maintain the primary national physician databases (e.g., Austria and Finland) (Matrix Insight, 2012; Kovacs et al., 2019; OECD, 2021). This role may relate to the fact that in many countries, physicians are statutorily required to be members of physician organisations to practice medicine (Kovacs et al., 2019; Ahpra, 2020; New York State Education Department – Office of the Professions, 2021; OECD, 2021). Organised medicine also plays a significant part in the regulation of re-registration and re-licensing procedures in other countries as well (Borow et al., 2013).

There are countries where medical associations have been entrusted with carrying out periodic national surveys that serve as key sources of information regarding physician employment (e.g., Canada and USA). Moreover, medical associations are often the entities that report to the Organisation for Economic Co-operation and Development (OECD) on the number of active physicians in their countries (Matrix Insight, 2012; Kovacs et al., 2019; OECD, 2021).

It is here worth distinguishing between regulatory bodies and medical associations. The former usually act as state-agents or public law corporations delegated with tasks such as licensing and obligatory registration through state legislation, whereas the latter mainly engage in the promotion of physicians' economic and professional interests and their membership is usually voluntary (Morioka, 2012; Kovacs et al., 2014; Bautista and Lopez-Valcarcel, 2019). However, this differentiation is not always clear (Kovacs et al., 2014; The European Union of General Practitioners/Family Physicians, 2016).

Whatever the arrangement, the management of medical workforce data is a joint collaborative project of the state and the medical profession. However, not all countries have been equally successful in establishing a national physician database. Canada has several physician databases varying in their scale and purpose, with Scott's Medical Database (SMDB)¹ constituting the main national source of information on the supply, distribution, and migration of physicians across the country. Run by a private company, data are collected mainly from provincial physician registries administered by provincial medical colleges (Leddin et al., 2018; CIHI, 2022). In Israel, the Ministry of Health (MoH) maintains a database of licensed medical practitioners, though it is not specific to those who practice medicine due to lack of physician re-registration mechanism. There is also no system for periodically updating physicians' employment and demographic data. Yet, the need in Israel for a registry to help with medical workforce planning has been expressed many times (State Comptroller, 2003, 2009; The Israel National Institute for Health Policy Research, 2008; Levy, 2010).

In Israel, the MoH and the Israeli Medical Association (IMA) experience great difficulties achieving comparable cooperation to that which exists in Canada between state authorities and medical colleges. This lack of cooperation hinders the establishment of an Israeli up-to-date physician database such as the Canadian SMDB.

3. Comparability of the Canadian and Israeli cases

Several works comparing Canada and Israel have addressed various domains of health policy, including patient safety, disaster preparedness, and quality of primary care (Ilan and Donchin, 2012; Shapira et al., 2016; Glazier, 2018). Others focus on policy lessons both countries can learn from each other regarding pandemic response, health workforce challenges, and cost control (Barer et al., 1989; Rosen, 2018; Marchildon, 2021). These comparisons include similarities between the two healthcare systems, such as a common commitment to universal health coverage and fairly high population health status, as well as structural, organisational, and cultural differences.

The present study follows that path. Both Canada and Israel are economically and technologically developed countries capable of creating the necessary infrastructure for such a database. Second, both countries are committed to providing publicly funded universal health services. Third, both healthcare systems are characterised by a low number of physicians per capita compared to the OECD average, and unequal distribution of physicians nationwide. In addition, both heavily rely on physicians who studied medicine abroad (MoH, 2022; Wang et al., 2023). This reliance on foreign-trained physicians creates additional challenges of monitoring and planning of the medical workforce. Overall, both countries share a strong impetus to collect and maintain updated physician data in support of an optimal allocation of medical workforce (OECD, 2022). These shared commonalities of both countries and their healthcare systems allow for the investigation of institutional and political contexts while holding broader economic and technological factors relatively constant.

Of course, no two cases are identical. While most physicians in Canada work as independent practitioners within a single payer system, the Israeli healthcare system appears more fragmented with most physicians working as independent contractors or employees in one of the four health insurance funds, the government, or the private medical market (Waitzberg and Rosen, 2020; Marchildon and Allin, 2021). The fragmented structure of the Israeli system might make it more difficult to collect and monitor physician data compared to Canada's more centralised/single-payer (at the provincial/territorial level) structure. On the other hand, Canada is a federation of ten provinces and three territories whereas Israel is a unitary country, which may suggest that it is easier for Israel to develop a national database due to its centralised political system compared to Canada's federative one. Despite these offsetting institutional factors, Canada succeeded in this task, both at the provincial and national levels, not Israel. This may imply another factor at play, namely the nature of the interactions between the various players, especially between physicians and governments, regarding the management of medical workforce and specifically the establishment and maintenance of physician data.

4. Methods

To explain each country's different outcome, it is necessary to explore the institutional position of the medical profession vis-a-vis the state and the historical processes that shaped their current dynamics. A historical institutionalism approach was applied to examine the existence or absence of a physician database within the broader development of their respective state–medical profession relationships. This allows us to understand contemporary occurrences as an outcome of long-term processes shaped by various political and social institutions. Historical institutionalism identifies important milestones or decision junctures to trace how a specific historical trajectory has led to a particular outcome (Skocpol, 1995; Thelen, 1999; Kickert and Van Der Meer, 2011).

Historical institutionalism and similar approaches are thus useful tools for analysing health system reforms, sometimes from a comparative perspective (Wilsford, 1994; Tuohy, 1999; Zehavi, 2012; Powell, 2016). Innovative scholarship in the field has also argued for more nuanced conceptions of the dynamics of policy change and makes key distinctions in regard to the scale and pace of changes (Tuohy, 2018).

Our data were collected from a variety of secondary sources, including academic works and ‘grey literature’, including a wide range of materials produced by organisations outside of academic publishing. Sources were collected to allow diversity of perspectives of the relevant stakeholders as well as viewpoints of professional parties, including official government publications, medical organisation documents, professional committee reports, working groups, the Israeli State Comptroller reports, and news media, obtained using search engines and databases such as PubMed, Google, Google-scholar, and ProQuest. Additional sources included provincial colleges of physicians and surgeons websites, The Royal College of Physicians and Surgeons of Canada (RCPSC), the Canadian Institute for Health Information (CIHI), Scott's directories, the Canadian Medical Association (CMA), the IMA, and the Israeli MoH. Main search words used: medical associations, medical profession, physician registry, physician databases, regulation, state, and workforce planning (both in Hebrew and English).

Literature gaps regarding the context and evolution of the formation of Canadian physician databases were detected. In order to overcome these gaps and obtain the necessary information, four consultations were conducted with Canadian key informants who were involved with the formation and/or maintenance of physician databases at the provincial level or national level.

5. Results

5.1 Canada

In Canada, though provinces and territories are primarily responsible for planning, financing, and administering health systems for their residents, there are multiple physician databases compiled at the national level. For example, the Canadian Post-M.D. Education Registry (CAPER), maintained by the Association of Faculties of Medicine of Canada, tracks Canadian postgraduate training statistics, including first year trainees and estimated practice entry cohort. CIHI's National Physician Database (NPDB) contains socio-demographic, payment, and service-utilisation data on physicians, generated at the individual or specialty level. The CMA Masterfile is populated by regular inputs from the RCPSC and College of Family Physicians of Canada (CFPC), as well as from individual members themselves. The non-members data are more difficult to maintain since the CMA does not get regular updates on them (Leddin et al., 2018).

Historically, data on the physician workforce in Canada were owned and maintained as a directory by private companies, first Secombe House Publishing and then Southam Directories Inc. which bought out its predecessor. The Southam medical directory was the only source of information on physician supply across the country, and included contact information (phone number, address), specialty, etc., and was available in libraries and for purchase, allowing patients to be able to find a physician, physicians to identify other physicians to refer their patients to, and companies to market their products to physicians, for example. With the Medical Care Act of 1966 which established a national framework for federal cost sharing of universal coverage of medical (physician) services in the provinces and territories, the federal government had an interest in compiling data on physician supply and payments, and they contracted Southam to maintain the Canadian Medical Database. At this time, the federal government was financing half of all hospital and medical (physician) care costs in the provinces and territories, and over time this has decreased to closer to 20 per cent. Starting in 1971, the federal government would recieve a snapshot of the physician supply data and maintained the files on a mainframe as the Southam Medical Database (SMDB).²

A 1991 report by the National Task Force on Health Information (the Wilk Report) recommended creating a national health information coordinating council and an independent institute for health information, later known as CIHI. The recommendation aimed to bring together under one roof vital health information that was scattered among different entities that operated almost unrelated to each other (National Task Force on Health Information, 1991). Since its inception in 1994, the CIHI inherited the existing relationship between Health Canada and Southam Directories, thereby acquiring an annual copy of the database from Southam Directories in a raw data format and performs data edit and verification before publishing it. In the late 2000s, Scott's Directories – a private business information company that maintains and sells directories across multiple industries – acquired Southam Medical Database. Currently, SMDB is used by federal, provincial, and territorial governments, healthcare stakeholders, researchers, and private organisations for longitudinal data on the supply, distribution, and migration of physicians. The database includes historical physician data files beginning from 1968, collecting its data from several organisations, primarily Canadian Medical Association (that, among other things, routinely surveyed physician), jurisdictional (provincial/territorial) physician registrars, RCPSC, CFPC, and from physicians who contact them directly (CIHI, 2010, 2022; Leddin et al., 2018).

Periodic comparisons are performed with data from selected provincial medical associations. Because physicians can be uniquely identified in this database, changes in physician practice category, as well as physician movement between provinces and territories, can be tracked over time, unlike with individual provincial/territorial physician registries. The SMDB includes, but is not limited to, data on physician year of birth, gender, country of MD graduation, residence postal codes, sub-specialty, and rural/urban distribution. There is no permanent linkage made between the SMDB and CIHI's NPDB database (CIHI, 2010, 2022; Leddin et al., 2018). CIHI reports provide extensive data on supply, distribution, and migration of physicians. Together with individual provincial registries and CMA data, these data constitute vital sources for healthcare human resource planning endeavours nationwide (Government of Alberta, 2008; Health Forum, 2019; Bourcier et al., 2022).

Jurisdictional licensing authorities are key data sources for Scott's Directories and, in turn, CIHI's SMDB. These data are used to identify new physicians and update historical physician information such as change of address and medical specialty. All postgraduate residents and all practicing physicians must hold an educational or practice license from the provincial/territorial medical regulatory authority in which they study or practice (Leddin et al., 2018; CIHI, 2022). Jurisdictional licensing authorities in Canada are the provincial/territorial colleges of physicians and surgeons. All practicing physicians and surgeons must be registered and licensed with their provincial college (RCPSC, n.d.). In most Canadian jurisdictions, in order to obtain a license, a CFPC certification is required for family physicians and from the RCPSC for all other specialists (Government of Canada, n.d.; Reichert and Associates Program Evaluation and Research, 2022). All practicing doctors must be members of their respective college. Doctors seeking licensure, apply to the college, which ensures they have met all credentialing requirements (CPSO, n.d.). For example, the College of Physicians and Surgeons of Ontario issues certificates of registration to physicians, allowing them to practice medicine in Ontario. All practicing doctors in the province must be members of the college. In British Columbia, a certificate of professional conduct is a professional document confirming a registrant's full registration history, credentials, and standing with the provincial college (CPSBC, n.d.). In both provinces, registered physicians are required to submit an annual certification renewal application. In addition, in both provinces all physicians are required to participate in a Continuing Professional Development (CPD) programme that meets requirements set by the RCPSC or CFPC (CPSBC, n.d.; CPSO, n.d.).

The SMDB is useful for physician resource planning because it was designed to capture almost all physicians in Canada and, due to the longevity of the data series, provides a historical perspective of physician supply and migration. Additionally, SMDB data reflect physicians engaged in both clinical and non-clinical activities such as research, teaching, and administration (CIHI, 2010). However, there are some important limitations to the SMDB. Traditionally, Scott's Directories relied primarily on the colleges for its physician data, yet this arrangement came to an end in the mid-2010s. It is possible that in the internet era, many physicians do not perceive cooperation with Scott's Directories as beneficial as it used to be, since publicity and promotion are now carried out mainly through digital channels rather than books of registries. As a result, provincial regulatory bodies no longer automatically share their data with Scott's but require an opt-in model where physicians had to provide consent for their data to be shared and included in the directory. In addition, Scott's Directories has started using publicly available data through college websites. This has compromised the data's richness and integrity.

Physician databases at the provincial level are where most health workforce planning takes place (since provinces are primarily responsible for planning and administering their health systems). For example, since 1992, the Ontario Physician Reporting Centre (OPRC) has maintained several databases funded by the province's ministry of health. The purpose of these databases is to support physician and human resources planning in Ontario through data sharing agreements between several stakeholders, among them the CPSO and the Ontario Medical Association (OMA). Reports are derived substantially from physician registry data (Ministry of Health, 2018; Ontario Physician Reporting Centre, n.d.). Using OPRC reports and surveys together with data from the SMDB, the MoH, OMA, and CPSO jointly strive to attain a robust physician dataset that supports provincial planning efforts as well as funding negotiations between the OMA and the ministry.

Despite Canada's efforts to build physician data infrastructure, scholars and health professionals point to critical workforce data gaps and call for improved data collection and analysis (Aggarwal et al., 2021; Canadian Health Workforce Network, 2022). Poor health workforce planning in Canada is attributed in large part to inadequate health workforce data which also have economic costs due to lack of efficient distribution of human resources (Bourgeault et al., 2019). In a 2022 petition to the government of Canada, professional associations from across the country called on the federal government to address critical data and information gaps and help coordinate the collection and analysis of enhanced workforce data in support of decision-making by provinces, territories, regions, and training programmes (Canadian Health Workforce Network, 2022). Scholars also point to the wide dispersion of physician data across many databases with no single national data source that can be used to describe the work of family physicians (Aggarwal et al., 2021). Indeed, improving health workforce data is a current policy priority. A parliamentary committee has recently reported on national workforce crisis and included a section about the need for better data as existing health workforce datasets were found insufficient as they may be limited in their scope and lacking necessary information for planning (House of Commons, 2023). Recent federal funding included the aim to ‘support the creation of a Centre of Excellence on health workforce data and underpin efforts to use data to improve health care’ (Prime Minister of Canada, 2023), and then in 2024, Health Workforce Canada was established, funded by Health Canada to work with CIHI and other partners to address health workforce data gaps and support workforce planning.

5.2 Israel

Surveys designed to estimate the number of practicing physicians among the Jewish community were conducted periodically by the IMA even before the establishment of the state of Israel in 1948 (Gelber, 1990). Since the 1950s, periodic information on the number of practicing physicians after the establishment of Israel was generally obtained from the workforce surveys of the Central Bureau of Statistics (CBS), while registry of the number of physicians licensed to practice was kept by the MoH (Ministry of Health, 2021).

In 1990, a State Committee of Inquiry for the performance and effectiveness of the healthcare system (‘Netanyahu Committee’) recommended, among the other things, establishing a comprehensive database that would be regularly updated using a periodic registration system for all professionals in the healthcare system, under the authority of the MoH (State Comptroller, 2003; Levy, 2010). Similar recommendations were given by another public committee in 2002, and a year later by the State Comptroller (Committee for the examination of public healthcare and the status of physicians, 2002; State Comptroller, 2003).

During the annual ‘Dead Sea Conference’ in 2008, which focused on the country's healthcare human resources, expert working groups supported re-registering all health professionals in order to create a periodically updated database of human resources for the purpose of healthcare planning, to be implemented by the MoH (The Israel National Institute for Health Policy Research, 2008). However, IMA representatives opposed the establishment of such a registry under the auspices of the MoH, arguing that it should be maintained by the CBS. The main reason for this position was the IMA's concern that re-registration would serve as the first step towards a further stage that the MoH was planning – the imposition of a re-licensing requirement, tied to an obligation to undergo CME (The Israel National Institute for Health Policy Research, 2008). It should be noted that in contrast to the process in other countries, in Israel physician participation in CME activities is completely voluntary. There is no binding arrangement concerning the matter, and the accreditation system for participation in educational activities is regulated by the IMA (Israeli Medical Association, n.d.-a).

Following the 2008 conference, the MoH announced its intention to advance the issue of periodic registration and licensing for physicians. According to the MoH, physicians would be required to register with the Ministry every few years as a condition for the renewal of their medical license. Several potential registration mechanisms were suggested, from administrative registration alone to a model based on physicians being required to take CME courses and exams to have their licenses renewed. Another possibility was to ensure physicians satisfy quality assurance requirements, and that they report on medical malpractice lawsuits and complaints brought by patients against them. The IMA responded, stating that it opposed any step that included compulsion, which could infringe upon physicians' professional autonomy (Yasour Beit-Or, 2008).

The State Comptroller subsequently reexamined the issue of Israel's medical workforce planning, and in a report published in 2009, found that a main reason for the issue not yet having been solved was the IMA's fundamental opposition to physician re-registration, and in particular, to such a process being managed by the MoH (State Comptroller, 2009). The need for a periodically updated national physician and other health professionals registry resurfaced in 2010 and 2014 as a recommendation of two MoH expert committees, and again in 2020 as part of the National Outline Plan for Health Institutions (The Committee for Medical and Nursing Workforce Planning in Israel, 2010; The Advisory Committee for Strengthening the Healthcare System in Israel, 2014; Planning Administration, 2020).

Despite the issue being discussed for many years, workforce planning in Israel is conducted without a periodically updated database of practicing physicians. Thus, the MoH has been impelled to rely on relatively crude estimations based on national workforce surveys carried out by the CBS. It should be noted, however, that in recent years the MoH's ability to estimate the number of practicing physicians has significantly improved. This is reflected in several ministry reports on the country's medical workforce, based on cross-referenced data from various sources, chiefly the MoH's database of licensed physicians, the Tax Authority database, and the Population Registry. These reports also contain IMA administrative and demographic data on medical residents, thus indicating some improvement in data transfer between the two organisations (Levi and Borow, 2018).

Nevertheless, MoH efforts have not resolved the issue of physician workforce ‘stocktaking’. In a sense they made it worse by creating conflicting sources regarding the number of physicians in Israel. Currently, the answer to the seemingly simple question – how many practicing physicians are there in Israel? – depends on the source one chooses to rely on. Due to differences in definitions and methods of data collection, the number of active physicians according to the CBS differs from the MoH cross-referencing number, which is also the figure reported to the OECD. These differences are quite significant. According to CBS data, between 2018 and 2020, the annual average number of active physicians in Israel was 35,500. During the same years, the figure indicated on Israel's reports to the OECD was about 29,600 (about 83 per cent of the CBS figure). Such considerable gaps between the two main data sources make it difficult to plan the country's medical workforce (Levi and Davidovitch, 2022).

An expert committee on healthcare human resource planning recently recommended establishing under the MoH a national medical residents database to provide updated data on all medical residents for the purpose of national human resource planning. Following this recommendation, the MoH has initiated legislation that requires medical residents to report to the MoH data such as postgraduate training start date, place of residency, name of medical specialty, and voluntarily answer questions on the nature of their residency programme (Ministry of Health, 2023). The MoH refused the IMA's suggestion to provide these data on behalf of the residents through its scientific council, which regulates medical residency programmes. The MoH has also declared that its next step will be to establish a national human resource advisory body to advise the ministry on healthcare human resource planning. In response, the IMA has questioned the initiative's legal basis (Israeli Medical Association, n.d.-b).

In conclusion, both Canadian and Israeli authorities have demonstrated a desire to establish and maintain a periodically updated database of practicing physicians. While in Canada, the SMDB and other databases have been operating for decades, though quality and comprehensiveness has deteriorated over time, an Israeli equivalent has yet to be established. The main differences between Canada and Israel on the issue of physician databases are summarised in Table 1.

Table 1. Summary of findings

The findings highlight the pivotal role of physician organisations in both countries in facilitating or hindering the establishment of such databases. While in Canada the medical profession appears to be committed to the maintenance of physician databases through its willingness to cooperate with state agencies, though nationally, this commitment has waned, Israel is characterised by conflicted relations between the state and the medical profession that prevents the formation of such a database. Therefore, the question of why physician organisations choose to behave in one way or another, that is, to support or oppose the existence of a physician database, will be at the centre of our discussion. An historical-institutionalist analysis of the state–medical profession relationship will be applied to examine the historical trajectories that have led to contemporary policy outcomes in both countries.

6. Discussion

The main obstacle preventing the implementation of a periodically updating database for physicians in Israel is a continuing dispute between the MoH and the IMA regarding the ownership and control of such a database and the nature of the re-registration mechanism. This dispute appears to involve conflicting interests on each side: the state aims to expand its control over the medical workforce to serve public health needs, while the medical profession sees this as a direct attack on physicians' professional autonomy and fear it will lead to mandatory re-registration linked to compulsory CPD/CME. This contrasts with the willingness of the Canadian provincial colleges and medical associations to provide up-to-date information on their members to a national database based on a system of re-registration connected to mandatory CPD.

In many countries, governments delegate regulatory powers to medical associations, which thus serve as state proxies for professional regulation (Borow et al., 2013; Schmitt et al., 2023). In exchange, physicians enjoy professional autonomy. This arrangement works for the benefit of both parties and has generated a longstanding tradition of delegation of authority by the state to the profession as far as viewing the medical profession as an entity that was created by the state itself, therefore essentially indistinguishable from the state apparatus (Freeman, 2000). (This does not mean, however, that there are no conflicts between the state and the profession, simply that they share norms of regulatory cooperation deeply rooted in historical arrangements.)

In the UK, following the creation of the General Medical Council (GMC) by the 1858 Medical Act, the system of physician governance has been based on the principle of state-sanctioned self-regulation. This arrangement, which has been profoundly challenged by reforms in the last decade, has served both the medical profession and the state in the framework of a regulatory bargain: the physicians gain professional autonomy (albeit not complete) in exchange for their obligation to act in the public interest (Adams, 2016; Levi et al., 2018). Canada also appears to have moved in this direction, where professionals are empowered by the state to govern their own affairs, especially entry to practice and the conduct of professional practice under what has been named the ‘Anglo-American model’, which grants health professionals a measure of autonomy (Adams, 2020). These arrangements have deep roots. In 1839, the College of Physicians and Surgeons of Upper Canada was created through provincial legislation. The legislation was blocked by Queen Victoria; however, following further amendments to the act following Confederation in 1867, the College of Physicians and Surgeons of Ontario was formed in 1869. The College of Physicians and Surgeons of Lower Canada was established in Quebec through legislation in the late 1840s, replacing the medical boards that were initially created jointly by state and medical leaders similar to Ontario, in a move towards a model of state-sanctioned self-regulation as the colleges were governed by physicians elected by their peers (Adams, 2020).

We do not wish to idealise state–medical profession relationship in Canada by overstating the cooperation and collaboration between the two parties. In Canada and elsewhere, self-regulation is often questioned and criticised by the public, medical dominance is challenged by non-medical health professionals, and professional independence is gradually eroded both by state intervention and market forces (Light, 1995; Adams and Wannamaker, 2022). As a general rule, medical associations strive to promote the economic interests of their members which do not necessarily coincide with the public interest, and Canadian medical associations are no exception, for example, advocating for private healthcare nationwide (Sibbald, 2005; Lee et al., 2021). Therefore, it is useful to understand state–profession dynamics as countervailing forces, constantly struggling to find a point of balance (Light, 1995). In the Canadian context, it seems that this balance relies on the historical regulatory bargain between the state and the medical profession as a fundamental principal that defines their relations. The Canadian healthcare system, which is based on the single payer model at the provincial and territorial level, created a bilateral monopoly between the state and the medical profession (Tuohy, 2012). This monopoly allows considerable influence of organised medicine in the development and implementation of health policies (Tuohy, 2021). Maintaining a register is among these responsibilities, placed at the hands of the colleges. For example, the Ontario Medical Register was already in place as early as 1871 (Price, 1977). Today, each of Canada's ten provinces and three territories independently license general practitioners and specialists (Government of Alberta, 2011). These registries are used for provincial healthcare human resource planning at the provincial level, and, until recently, were the principal data sources for the SMDB, thus supporting national healthcare human resource planning efforts. By doing so these registries demonstrate an important – yet rarely discussed – aspect of the Canadian regulatory bargain between the state and the medical profession. This is consistent with Tuohy's account of Canada's medical profession–state accommodation that emerged in the 1970s as the guiding institutional logic of health policy making of the then-new single-payer system (Tuohy, 1999). This accommodation depended on data sharing across regulatory colleges (maintaining their registries of all licensed physicians) and the ministry of health as the single-payer (population-wide physician billing data), with the representatives of physicians in fee negotiations (provincial medical associations) and thus facilitated physician data collection for health workforce planning efforts provincially/territorially. However, the longstanding commitment and aligned interests among federal government, medical profession (as represented by the CMA), private companies (Scott's), and these same provincial/territorial colleges that enabled the development and maintenance of a physician database nationally have recently become fractured.

The historical process that took place in Israel, however, is quite different. First, the Israeli healthcare system began its development decades before the country's establishment and therefore, medical organisations preceded national government institutions as opposed to having developed alongside them. The pre-state healthcare was created from scratch by non-governmental organisations, with almost no involvement on the part of the British rule (Neederland, 1983). The State of Israel was established in May 1948, and the MoH came into being along with it. As argued by several scholars, framed by a socialist ethos which idealised the ‘working pioneer’ as the backbone of the Zionist movement, medical professionals were perceived as part of the bourgeoisie. They were therefore excluded from the state's political and administrative systems (Canyon et al., 2006; Levi et al., 2018). In light of these historical, political, and social circumstances, the Israeli government denied the IMA's request to serve as a chamber of physicians, which would allow it to hold a considerable degree of regulatory authority in a way that resembles some of its European counterparts and medical self-regulation bodies of the Anglo-American Model. At the time, the IMA wanted all practicing physicians to be members of the association by law, and power over granting and revoking (if needed) medical licenses. However, the government rejected the IMA's request (Yishay, 1990).

As Levi et al. noticed (2018), the decision to deny the IMA the status of a ‘Chamber of Physicians’ marked a historic moment that determined the future nature of the relationship between the state and the medical profession. In view of Tuohy's insight that national systems are ‘products of the eras of their birth’ (Tuohy, 1999, viii), this may be viewed as a critical moment in time that has fundamentally shaped the Israeli health system in a way that is completely different from its Canadian counterpart. The state's mistrust of physicians has been repeatedly reflected in statements made about the IMA by public officials over the years. On both left and right sides of the political spectrum, the IMA was viewed as a self-serving interest group, and a belief commonly held by members of parliament was that the IMA's participation in policy matters should be minimised (Yishay, 1990). The state-centric perceptions that prevailed among politicians in Israel's early years effectively dismissed the medical profession from decision-making circles and promoted the concentration of powers within the hands of government institutions. Ever since, the IMA has had relatively minor statutory responsibilities and serves no role delegating authority with respect to physician licensing and registration. Postgraduate training is the only prominent jurisdiction of the IMA through its Scientific Council. Furthermore, the state tried more than once to transfer this responsibility to the MoH – a step that would have left the IMA with almost no formal standing (Levi et al., 2018).

The IMA is an association tasked mainly with advocacy, lobbying, negotiating, and deliberating with employers. This lack of delegated authority does indeed limit its power, but at the same time it reduces its dependence on the state and liberates the organisation from almost any commitment to cooperate. Moreover, since medical specialty associations are an integral part of the IMA with no formal standing of their own, the IMA enjoys a wide span of control over the profession to execute its policies and agendas to the state's dismay.

Table 2 summarises several key features of physician organisations in Canada and Israel. The table shows considerable differences in the standing of physician organisations in both countries: while Canadian provincial colleges can be viewed as an integral part of the state apparatus within the framework of a regulatory bargain between the state and the profession, the IMA seems to play the role of an outsider interest group as a similar arrangement has not been achieved in Israel.

Table 2. The standing of physician organisations in Canada and Israel

Not only did the state relegate the IMA to the position of an external interest group – one that hardly constitutes a quasi-governmental body – but it has also reinforced the IMA's dependence on its members since physicians are not obligated to be IMA members to practice medicine. Therefore, from the standpoint of the IMA, pushing back against unpopular governmental reforms is a central mode of action to satisfy its members and attract physicians to its ranks. Indeed, there exists a vast chasm between the positions taken by the state and the medical profession regarding various policy issues such as quality measurement, scope of practice, and the financing of health services with very little room for deliberation and cooperation (Rozin and Davidovitch, 2009; Levi et al., 2018).

In addition, unlike the Canadian SMDB, the idea of creating a partnership between the state, the profession, and the market in establishing and maintaining a physician database has not been realised in Israel. While Public–Private Partnerships (PPPs) are a common practice in the Anglo-Saxon world (Krumm, 2016), a similar arrangement has not been realised or even considered in the Israeli case although PPPs are increasingly appearing in the country (Razin et al., 2022).

So far, no understandings that would allow the establishment of a periodically updated physician database have been reached in Israel. A recent OECD report (2023b) pointed to Israel's lack of data-driven long-term medical workforce planning as a major cause of its struggle to address its physician shortage and maldistribution. However, it seems that Canada is not doing any better, as recent works point to wide geographic disparities and long wait time for health services, short hospital staffing and shortage of family physicians across the country (Gillespie, 2023; Li et al., 2023; Moir et al., 2023), thus reinforcing the recent urgent calls presented earlier to enhance the Canadian health workforce infrastructure which suffers from profound gaps and discrepancies due to its fragmented nature and patched structure. Moreover, while persistent health workforce imbalances and health outcome disparities characterise both countries, OECD health statistics indicate that not only does the Israeli healthcare system outperforms its Canadian counterpart in leading health indicators such as life expectancy for both sexes, infant mortality, and age-standardised avoidable mortality, but it manages to do so with significantly lower spending per capita (OECD, 2022, 2023a). Although a comparison of the performance of both healthcare systems is beyond the scope of this paper, these results may suggest that the Canadian state–medical profession regulatory arrangements have not resulted so far in superior workforce planning or better health outcomes in comparison to Israel. Considering other factors in play, it is possible that what the Israeli healthcare system lacks in data infrastructure and long-term planning, it compensates to a certain degree with remarkable adjustability and flexibility as was demonstrated in its COVID-19 vaccination rollout and swift response to the 7th October 2023 attack (Rosen et al., 2021; Taub Center Researchers, 2023). By the same token, it seems that Canada's traditional ‘Anglo-American model’ of cooperation as a facilitator of a monolithic database is offset by a fragmented structure of the Canadian healthcare system.

This point also reveals one of the limitations of this paper: while the two healthcare systems share prominent commonalities that allow a viable comparative analysis as mentioned earlier, salient dissimilarities in their political structure make it somewhat difficult to isolate the impact of state–medical profession relationship on the formation of physician database given the distinct features of their political systems and health services especially along the lines of federative versus unitary structure. Another limitation refers to the theoretical framework of this work. Focusing on enduring patterns of behaviour resulting from historical circumstances, institutionalist approaches in social sciences tend to explain ‘stability’ much more comfortably than they do ‘change’ (Farrell, 2018). However, changes do occur, sometimes in rather unexpected directions as evident in both cases. In Canada, there are difficulties in the development and maintenance of a federal data infrastructure due to recent decreased willingness of physicians to share their personal data with Scott's directories. As witnessed in the Israeli case, the MoH has initiated lately more active measures vis-a-vis the IMA in its effort to collect physician data and establish a central planning body. This may mark a significant change of policy which goes against the ‘direction of history’ as framed by the historical-institutionalist logic. It remains to be seen whether the MoH succeeds in its efforts. In this case, it is possible that change may come about as a result of external shocks such as the impact of the COVID-19 pandemic or ‘Iron Swords’ war on the healthcare system that may reconfigure the institutional logic that prevailed thus far. Similarly, an exogenous factor, that is technological change (digital channels) that rendered books of registries redundant in the eyes of physicians, has weakened their inclination to cooperate and share data. From a theoretical perspective, this might involve the transformation of institutions from explanatory variables to dependent or intervening variables thus losing their explanatory power to a large extent, as scholars point out (Aspinwall and Schneider, 2000; Capoccia, 2016).

7. Conclusion

Contemporary occurrences may be explained against the backdrop of broader political, governance, and organisational contexts. Our study underscores the importance of historical circumstances in shaping present health policy outcomes. At the same time, it also reveals to some extent the limitations of the historical-institutional approach in explaining change by pointing to the recent difficulties in the development and maintenance of a national Canadian database as opposed to Israel's increasing efforts to produce an updated and comprehensive database, although from an institutionalist perspective, the directions of these changes are consistent with the governance structure of both countries, that is federalism and unitary system respectively.

Finally, how medical organisations respond to governmental healthcare initiatives is heavily influenced by their institutional position vis-à-vis the state, which in turn depends on historical processes that shaped existing regulatory arrangements. Therefore, state–medical profession dynamics may help explain the success or failure of policies such as the formation of periodically updated physician databases.