Evidence of effectiveness is increasingly requested by payers to support coverage decisions, and also for medical devices (Reference Migliore, Ratti, Cerbo and Jefferson22). Health Technology Assessment (HTA) agencies are repeatedly confronted with a lack of high-level clinical evidence when assessing the value of innovative high-risk devices when they enter the market in Europe. At the same time, a randomized controlled trial (RCT) is often initiated as a requirement for obtaining market access in the United States. These RCTs are sometimes partly conducted in Europe with CE (Conformité Européenne) marked products (Reference Fann, Chronos and Rowe9;Reference Kaplan, Baim and Smith19). Similar observations were made in three rapid assessments by the Belgian Health Care Knowledge Centre (KCE), covering percutaneous aortic valves, endobronchial valves and interspinous devices (Reference Stordeur, Gerkens and Roberfroid26;Reference Van Brabandt and Neyt30;Reference Van Brabandt and Neyt31). Payers in Europe are placed in a difficult situation. Postponing the reimbursement decision until the RCT results are available is often met with mixed feelings by industry and clinicians. For the patient, the current system in Europe can mean earlier access to a potentially lifesaving device, but at the risk of insufficiently documented efficacy and safety. Furthermore, the cost of the device may be fully charged to the patient. This growing tension between early market introduction and the reimbursement of innovative high-risk devices in Europe led us to take a closer look at the premarket clinical evaluation of such devices.
Both in Europe and the United States, medical devices are categorized into different classes based on the risk they pose to patients, with class III devices representing the highest risk category. We define innovative high-risk devices in Europe as class III devices and implantable devices, that differ more than slightly from existing devices in the same indication with regard to design, expected effectiveness, or both. In addition, the use of a high-risk device (new or existing) in a new indication has also to be considered an innovation. In the United States, innovative high-risk devices (class III) typically undergo a premarket approval (PMA) process.
Contrary to what health professionals and users of high-risk medical devices may assume, the premarket clinical evaluation of high-risk medical devices in Europe is not comparable to the evaluation of pharmaceuticals in terms of safety and efficacy. Patient safety aspects of the European approach for high-risk devices have not been widely discussed in peer reviewed journals. During the finalization of our study, a policy report by the European Society of Cardiology was published (Reference Fraser, Daubert and Van de Werf13), as well as a series of articles in the British Medical Journal (Reference Cohen and Billingsley4;Reference Fraser, Krucoff, Brindis, Komajda and Smith12;Reference Freemantle14;Reference Thompson, Heneghan, Billingsley and Cohen28;Reference Wilmshurst32) that call for more transparency and a higher quality clinical evaluation of medical devices in Europe during both pre- and postmarketing.
The aim of this study was to review and discuss the premarket clinical evaluation of innovative high-risk medical devices in Europe and to compare it with evaluations in the United States. We also compare the evaluation of devices with the evaluation of medicines, where appropriate. In addition, we discuss issues regarding patient safety and the transparency of information. This report does not cover the preclinical evaluation or the manufacturing practices of devices. Furthermore, our conclusions should not be extrapolated to the vast number of lower risk devices outside the scope of this study. The full KCE report (number 158) on which this manuscript is based is available at www.kce.fgov.be.
METHODS
We consulted the peer reviewed literature using a Medline search. The yield of this search was very poor, at least for descriptions and critical reviews of the European system when compared with the United States.
We consulted experts of member organizations of EUnetHTA (a European network of HTA agencies), and we checked the grey literature and European and U.S. regulatory documents. Furthermore, representatives from Competent Authorities, the U.S. Food and Drug Administration (FDA), Ethics Committees, and the Medical Device Industry were consulted.
Three major Notified Bodies were requested by Belgian Competent Authorities to complete a simple questionnaire on the number of patients exposed to the innovative device pre-CE mark and on the type of trial design (randomized or not). This information (without identifying the device) was requested for cardiovascular implantable devices with or without an active substance and for other implantable devices with an active substance.
The list of study protocols reviewed by Belgian Ethics Committees in 2010 was searched for trials with high-risk medical devices by Professor Marc Bogaert, Belgian Advisory Committee on Bioethics. In addition, the Ethics Committees were requested to provide the regulatory status of the device under study.
Clinical Development Phases
The clinical development of medical interventions and technologies for a given indication is characterized by different phases. Exploratory studies, often generating a hypothesis, are followed by confirmatory studies to test the hypothesis (17). This phased approach is now the standard for the premarket clinical development of pharmaceuticals. Exploratory trials of high-risk devices are more complex compared with drug trials: it is not just the device itself that is studied but the complete system. This includes the most optimal procedures for handling the device and ways of shortening the learning curve. In addition, exploratory device trials often lead to minor modifications of the new device, a phenomenon that makes it even more difficult to determine when an innovation is ready for confirmatory studies.
Randomized control trials (RCTs) are the highest standard for documenting clinical safety and demonstrating any incremental efficacy, also for devices. For example, in the absence of well-controlled trials, it may be impossible to judge whether a 30-day mortality rate of 5 or 10 percent after the implantation of a percutaneous aortic valve should be considered high or low (Reference Van Brabandt and Neyt30). High quality RCTs can also be conducted with devices, even if these trials involve no or only the partial blinding of patients, physicians, or outcome assessors (Reference Freemantle14). Experience at the FDA has shown that valid and realistic study designs are possible for the controlled evaluation of the efficacy and safety of medical devices. The FDA considers RCTs to be critical for statistical validity, and they are ethical if designed according to current U.S. Good Clinical Practice. Randomized controlled trials with sham (or placebo) surgery as a comparator for proving device efficacy may sometimes be considered unethical. It may be more appropriate to document efficacy versus the reference treatment (watchful waiting, another device, drug treatment, conventional surgery, etc.). To not expose too many patients unnecessarily to a high-risk intervention, the conduct of an RCT should not be delayed. This was recently illustrated for stents used to treat intracranial arterial stenosis, where the stents had been approved 6 years previously based on a humanitarian device exemption and on the registry data of only forty-five patients. After thousands of patients received the device, an RCT was stopped early because the 30-day rate of stroke or deaths was significantly higher than after medical treatment (Reference Chimowitz, Lynn and Derdeyn3).
Different Systems of Premarket Clinical Evaluation in Europe and the United States
The premarket regulations of the FDA and Europe remain very different despite the creation of a Global Harmonization Task Force (GHTF) for device regulations in 1992 (www.ghtf.org) (Reference Jefferys18).
The European system is based on the demonstration of safety and performance, whereas the U.S. system requires the premarket demonstration of safety and efficacy/effectiveness (Table 1). This leads to entirely different clinical trials. For the demonstration of device performance, an RCT is neither necessary nor appropriate, whereas it is essential for the demonstration of clinical safety and efficacy in a controlled way. The level of study evidence required in Europe is also much less specified compared with the FDA requirements for a PMA application. The net result is an earlier market introduction in Europe compared with the United States.
Table 1. The sequence of clinical development and timing of the market introduction of novel invasive high-risk devices in Europe and the US
Demonstration of Device Performance in Europe
In Europe, novel high-risk medical devices do not undergo a premarket review by Member States or European Authorities. Instead, they have to go through a system of CE (Conformité Européenne) marking, which is mainly based on the 1993 Medical Device Directive, which was amended in 2007.
Conformité Européenne marking for high-risk novel medical devices is applied for by the manufacturer and the conformity assessment is checked by a Notified Body. Notified Bodies are for-profit organizations certified by a Member State Competent Authority. The latter vary between Member States. In some cases, national medicines agencies are endowed with the overview of medical devices; in other cases, specific agencies have been set up and, finally, some Member States do not have any specific pathway for the approval of medical devices, relying purely on Notified Bodies.
This premarket evaluation of a device by a Notified Body is performed only once for the entire European market. It includes the assessment and verification of the clinical evaluation, defined as the assessment and analysis of clinical data pertaining to a medical device to verify the clinical safety and performance of the device when used as intended by the manufacturer. It is very important to note that an evaluation of clinical efficacy is not part of the premarketing evaluation in Europe.
Member States can invoke a safeguard clause if evidence of a major public health concern is identified. In the case of devices containing a human blood derivative, the Notified Body can only decide positively if the European Medicines Agency's scientific opinion is favorable (Reference Jefferys18). However, for drug-device combination products, the Notified Body can overrule the opinion of the Medicines Agency.
There are approximately eighty Notified Bodies currently designated for a total of thirty-one countries in the European Economic Area (EEA). Some Notified Bodies only have two or three staff (Reference Cohen and Billingsley4). Notified Bodies are not on a level playing field in terms of their criteria for approval; companies can work with different Notified Bodies for different devices and Competent Authorities are aware that companies could select a Notified Body that is likely to be less stringent in the assessment of a particular device (Reference Cohen and Billingsley4;Reference Tinkler29). The Notified Body Operating Group (http://www.nbog.eu/) is working on this issue and the ongoing revision of the device directives is expected to provide a solution (5).
Directive 2007/47/EC, effective from March 2010, states that, in the case of implantable devices and class III devices, clinical investigations shall be performed unless reliance on existing clinical data is duly justified (27). Since May 2011, all clinical investigations (protocol title and primary objective only) are entered into a nonpublic database (Eudamed) by the Competent Authority where the trial is notified first (8); however, the trial authorization procedure differs significantly by Member State. Some Competent Authorities actively review the documentation submitted, including the trial design. Other Member States, however, have opted for the passive permission route (no review).
The December 2010 MEDDEV 2.7/4 guidelines (7) do not provide any specific requirements regarding the depth and extent of premarketing clinical evaluations of high-risk devices. The RCT design is not specifically endorsed.
Premarket Clinical Trials in Europe Are not Made Public
Based on grey literature (Reference Tinkler29) and informal contacts within Competent Authorities, it seems that the number of patients evaluated in premarket trials in Europe varies by the indication but is typically less than 100 patients. These studies are often referred to as single-arm “feasibility” trials or “performance” trials. Only for drug-eluting stents is the minimum number of patients exposed in the premarket phase more or less in the public domain: based on the potential safety issues that were identified, the number of patients studied increased over time from 50 to 300 (no RCT required).
Different initiatives to obtain more information on premarket clinical data did not succeed. Belgian Competent Authorities received no answer from three large Notified Bodies asked to provide the numbers of patients exposed and the trial design used for the novel high-risk devices they had approved. In the UK, researchers failed to obtain premarket clinical data from the manufacturers and six Notified Bodies for 192 recalled devices (Reference Heneghan, Thompson, Billingsley and Cohen15;Reference Thompson, Heneghan, Billingsley and Cohen28). Only four companies (2 percent) provided any clinical data. In the Netherlands, Competent Authorities encountered difficulties in obtaining technical documentation of class III devices from industry (coronary stents, total hip implants, and silver-containing wound dressings) (Reference Roszek, de Bruijn, Pot and van Drongelen23). It is of interest to note that even Competent Authorities have to rely on a Google internet search to identify the class III devices being marketed as there was (and still is) no comprehensive list or database. Major shortcomings were identified in the documentation received, in particular concerning the clinical evaluation of the device (Reference Roszek, de Bruijn, Pot and van Drongelen23). Confidentiality seems to overrule transparency in Europe much more than in the United States (Reference Cohen and Billingsley4).
Based on a large subset of protocols that were reviewed in 2010, representatives of Belgian Ethics Committees reported that nearly all trials with high-risk devices in Belgium are conducted after the CE mark has been obtained, and that the identified pre-CE mark trials were not randomized.
Premarket Clinical Review in the United States
In the United States, premarket device review procedures were introduced in 1976. The evaluations are performed by the FDA, mainly by its Center for Devices and Radiological Health (CDRH). For many devices entering the U.S. market the “premarket notification” route, the 510(k), applies. Innovative high-risk medical devices, however, are, in principle, subject to a more stringent procedure called the “premarket approval” (PMA) procedure, which typically requires the conduct of a randomized controlled trial (Reference Hines, Lurie, Yu and Wolfe16).
Under the PMA process each manufacturer must independently demonstrate “reasonable assurance of the safety and effectiveness” of its device for its intended use. To start research trials in the United States to be included in a PMA, the sponsor first needs to obtain an Investigational Device Exemption (IDE) approval from the FDA (Reference Saviola24). For diseases or conditions affecting small patient populations (under 4,000 patients in the United States), the FDA, since 1996, has allowed Humanitarian Device Exemption (HDE) applications, which are similar in both form and content to a PMA application, but are exempt from the effectiveness requirements of a PMA.
Nevertheless, PMA applications have been criticized as they often contain only a single study or they lack adequate strength or coverage of specific study populations (Reference Dhruva, Bero and Redberg6;Reference Hines, Lurie, Yu and Wolfe16;Reference Kramer, Mallis, Zuckerman, Zimmerman and Maisel20). Recent high-profile cases involving potentially dangerous defects in widely used cardiovascular devices have increased concerns about the adequacy of premarket trials and postmarket surveillance for establishing the safety of these devices. Furthermore, the 510(k) process has also been the subject of debate. A 2011 report by the U.S. Institute of Medicine (www.iom.edu) called for a new regulatory framework using a risk-based approach to replace the 510(k) process (Reference Challoner and Vodra2).
Other Effects of the Difference in Systems
Another poorly documented effect that might be linked to the difference in regulatory systems is that only one in five of the 8,500 medical device companies in Europe (probably fewer if one excludes the companies belonging to a U.S. group) has approached the U.S. market (Reference Audry and Ghislain1). In addition, more devices of a particular type are often marketed in Europe compared with the United States; for example, 28 drug-eluting stents are CE marked whereas only 5 have obtained FDA approval. Unfortunately, final negative decisions on PMA dossiers are not made public on the FDA Web site. Therefore, the possibility that devices continue to be marketed in Europe even though they failed to demonstrate efficacy and safety in the RCT required in the context of a PMA in the United States cannot be excluded. This is illustrated by the negative FDA panel vote for an endobronchial valve system in 2008 (10). Several similar examples are given in a recent study by Cohen and Billingsley (Reference Cohen and Billingsley4).
Premarket Demonstration of Clinical Efficacy Is Needed in Europe
The discrepancy between the European and U.S. situations highlights the need for a significant change in legislation on this side of the Atlantic. The FDA believes that RCTs are critical for statistical validity. In a recent report of the World Health Organization, regulatory agencies were called upon to ensure that, whenever possible, high quality randomized trials are completed before granting marketing approval of high-risk medical devices (33).
Despite the increase in clinical trial activity induced by the EU Directive 2007/47/EC, the remaining variations in the stringency of clinical reviews both at the level of Notified Bodies and Competent Authorities do not guarantee patient safety in a uniform way for EU citizens. Therefore, for innovative high-risk devices, the future EU Device Directive should go beyond requiring safety and “device performance” data only to also requiring premarket data that demonstrate “clinical safety and efficacy”. It should also be noted that for devices where only slight modifications were made to an existing device, a shortened development cycle may or may not be appropriate. This needs to be evaluated case by case.
Members of Ethics Committees and Competent Authorities are often confronted by a lack of clinical research expertise at small device companies. In addition, Notified Bodies may also lack the expertise to interpret clinical data and clinical safety signals (Reference Thompson, Heneghan, Billingsley and Cohen28). In this regard, the European Society of Cardiology has recommended that independent expert physicians should be involved in reviewing the decisions of new class III devices (Reference Fraser, Daubert and Van de Werf13).
Initiatives to develop international guidance documents for premarket clinical trials by the type of high-risk device need to be stimulated (Reference Fraser, Krucoff, Brindis, Komajda and Smith12). Such guidance documents have been proven to be of value for the more standardized, high quality clinical development of medicinal products. Not only professional societies (Reference Fraser, Daubert and Van de Werf13) but, ideally, HTA agencies should also be involved in this activity, for example by proposing study endpoints, which will be needed for cost-effectiveness evaluations. Such documents should also further the standardization of evaluations across Notified Bodies and Competent Authorities. Industry could benefit if a positive reimbursement can be better planned and if potential liability risks of entering the market too early can be reduced.
Instead of trying to streamline a very fragmented system of Notified Bodies and Competent Authorities, a more straightforward way of achieving the goals discussed above could be to centralize expertise at the European level, as was done for advanced therapy medicinal products, and as suggested by the European Society of Cardiology (Reference Fraser, Daubert and Van de Werf13). The transfer of evaluation and approval to a centralized body may, however, be more urgent for innovative high-risk devices than for other lower risk devices.
Payers in Europe should consider the co-financing of innovative high-risk devices used in premarket clinical trials, provided that these trials are well-controlled and have clinically relevant endpoints. This would reduce the financial hurdle for industry, which primarily comprises small and medium enterprises. This system already exists in the United States, where the Centers for Medicare and Medicaid Services (CMS) reimburse the cost of medical devices studied under the IDE if they meet certain criteria. The level of reimbursement is up to the cost of a currently marketed, similar product. As for medicines, specific regulations are needed for devices aimed at very small target populations.
The Need for More Transparency in Europe
In the United States, since 2007, full transparency is assured by the obligatory registration of trial protocols and study results, both for drugs and devices (11). The EU Clinical Trials Register now also provides protocol-related information to the public, but only for trials with medicines. The trial results for new medicines are made public in European Public Assessment Reports (EPARs). This is in sharp contrast with the approval system of devices in Europe (Reference Thompson, Heneghan, Billingsley and Cohen28) (Table 2).
Table 2. Transparency of trials and their results for devices and pharmaceuticals in Europe and the US
* This is in conflict with the Declaration of Helsinki, which is to be respected according to Medical Device Directives.
There is a clear need for a public trial registry (Reference Sim25) and a publicly available summary describing the basis for granting a CE mark (Reference Cohen and Billingsley4). Directive 2007/47/EC specifies that all parties involved in the application of directives are bound to observe confidentiality with regard to all information obtained in carrying out their tasks (27). The same Directive, however, also requires that clinical trials respect the Declaration of Helsinki. This suggests entering the trial in a publicly available trial registry before the first patient is entered. Negative and positive results should be published or otherwise made publicly available. This requirement of the Directive is clearly not respected in most cases.
The regulations in Europe allow for an early market introduction of high-risk medical devices at a time when the device cannot yet be considered to have an established indication in routine clinical use, suggesting that the use could be called experimental. Often, the confirmatory clinical development phase is still to be initiated or is ongoing.
European regulations of research ethics (e.g., the Declaration of Helsinki, The European Convention on Human Rights and Biomedicine, The Clinical Trials Directive) are mainly focused on clinical trials; however, there seems to be no reason to grant less protection to patients who are treated with a high-risk medical device in the experimental phase outside the context of a trial.
The new premarket procedure should result in an approved indication for the device and publicly available product documentation including the full results of all trials. This transparency is required to allow physicians to practice evidence-based medicine, patients to make an informed decision and HTA agencies to produce the correct assessment. It may, however, be appropriate to also study whether or not the rules that protect innovation need to be improved in view of this transparency requirement.
Transparency is also required in cases where the inventor of the device is also the principal investigator in a clinical trial of the device, a situation that is not unusual (Reference Wilmshurst32).
Measures to be Taken at the Member State Level
When awaiting a reworked Medical Device Directive, patient risk should be minimized at the Member State level by improving transparency with regard to the available clinical data and by limiting the market introduction of novel high-risk devices with minimal clinical data to centers with the necessary expertise. Preferably, this should be done under an appropriate research protocol (RCT if possible). This requires the commitment of the Competent Authorities, the marketing company, the physicians, and the hospitals.
The impact of HTA is limited as this occurs during the postmarket phase. Despite HTA reports concluding that evidence is insufficient for justifying reimbursement, marketing of the device continues and patients may be charged for its cost. Therefore, the ethical issues associated with the early market introduction of innovative high-risk devices should be studied further by the commissions and organizations that provide ethical guidance to physicians and hospitals.
With regard to patient safety of medical devices, the major control point in Europe is still postmarketing rather than premarketing surveillance, as is the case for medicines (Reference Jefferys18). Without active analysis, postmarket surveillance often remains an empty shell and a false reassurance of patient safety. It is hampered by an ongoing struggle with both under-reporting of the numerator (the number of adverse events) and a lack of data about the denominator (the total number of exposures) (Reference Lenzer21). These numbers are often lacking for devices (Reference Thompson, Heneghan, Billingsley and Cohen28). In this context, it has been reported that many manufacturers fail to fully fulfill their legal responsibility to collect product data once their device is on the market (Reference Cohen and Billingsley4). Therefore, the industry statement that no information is available to suggest that patient safety in Europe has been compromised needs to be interpreted with caution (Reference Cohen and Billingsley4).
CONCLUSIONS
HTA agencies in Europe are often confronted with a lack of high-quality clinical evidence when evaluating novel high-risk devices. We showed that this is linked to the European regulatory system that allows early market introduction based on limited clinical data. We recommend that Europe should require high-quality randomized trials demonstrating clinical efficacy and safety before granting the marketing approval of innovative high-risk medical devices. To allow physicians to practice evidence-based medicine, patients to make an informed decision and HTA agencies to produce the correct assessment, a major improvement in the transparency of information is also needed for such devices in Europe.
CONTACT INFORMATION
Frank Hulstaert, MD, MSc, Senior Expert, Belgian Health Care Knowledge Centre (KCE), Brussels, Belgium; Mattias Neyt, MSc, PhD, Health Economist, KCE; Imgard Vinck, MSc, Lawyer, KCE; Sabine Stordeur, RN, MA, PhD, Senior Expert in Clinical and Health Services Research, KCE; Mirjana Huić, MD, MSc, Assistant Director and Head of Department for Development, Research and Health Technology Assessment, Agency for Quality and Accreditation in Health Care and Social Welfare, Zagreb, Croatia; Stefan Sauerland, MD, MPH, Head of Department Non-Drug Interventions, Institute for Quality and Efficiency in Healthcare (IQWiG), Cologne, Germany; Marja R. Kuijpers, MSc, PharmD, Advisor, Dutch Health Care Insurance Board (CVZ), The Netherlands Payam Abrishami, MD, MA, Health Services Researcher, CVZ; Hindrik Vondeling, PhD, Associate Professor in Health Technology Assessment (HTA), Centre for Applied Health Services Research and Technology Assessment (CAST), University of Southern Denmark, Odense, Denmark; Bruno Flamion, MD, PhD, Professor in Physiology and Pharmacology, University of Namur, Belgium; Silvio Garattini, MD, Professor of Chemotherapy and Pharmacology, Director, Mario Negri Institute for Pharmacological Research, Milano, Italy; Mira Pavlovic, MD, Deputy Director, Health Technology Division, Direction de l'Évaluation Médicale, Économique et de Santé Publique (DEMESP), Haute Autorité de Santé (HAS), Paris, France; Hans Van Brabandt, MD, Cardiologist, KCE.
CONFLICTS OF INTEREST
Stefan Sauerland has received honoraria from Ethicon Endo-Surgery (Europe). The other authors report they have no potential conflicts of interest.
Open access
