Coverage with Evidence Development: An examination of conceptual and policy issues

Abstract

The application of conditionality to coverage decisions for healthcare technologies is increasing. Coverage with Evidence Development (CED) represents a specific approach to coverage for promising technologies for which the evidence remains uncertain. CED demands that additional evidence is generated to address the sources of uncertainty and secure ongoing coverage. This study explores the conceptual and policy issues relating to CED and discusses issues involved in operationalizing CED in practice, including presenting criteria for which technologies may be most suitable for CED. This study is intended to further the debate on the use of CED as well as highlight areas that warrant further research.

With continued pressures on healthcare budgets, innovative technologies are subject to increasing scrutiny before healthcare decision makers agree to provide coverage for their use. Health services throughout the world have adopted a range of approaches to evaluate new technologies before their widespread adoption. Whereas the majority of activities have focused on pharmaceuticals and medical devices, other healthcare interventions, including diagnostics, imaging technologies, and surgical procedures, are increasingly being evaluated before widespread adoption.

Health technology assessment (HTA) is one technique that has been widely adopted to help to manage the introduction and appropriate use of new technologies (1). HTA involves the medical, social, ethical, and economic implications of the development, diffusion and use of a health technology. HTA has been positioned as a “bridge between scientific evidence and the needs of policymakers” (2). HTA emerged partly in response to the uncontrolled diffusion of health technologies that occurred in the past (8) and is now frequently used to inform decisions on the coverage and reimbursement of new technologies.

The point in the life-cycle at which a technology should be assessed remains a contentious issue (11;16). From the perspective of a decision maker, assessment of technologies close to the time of their regulatory approval and/or launch allows for a timely decision to be made regarding their coverage and availability. However, the level of evidence available on a technology at the time of launch is inevitably limited and, to a large extent, determined by the regulatory requirements that apply to the technology. Any assessment of a technology at this point in its life-cycle will inevitably involve a degree of uncertainty around the clinical and economic data.

The main source of data for clinical effectiveness will be regulatory studies. In the case of pharmaceuticals, these will be phase III randomized controlled trials (RCTs), but, for devices and diagnostics, data from RCTs may not be available. For pharmaceuticals, the end points of the trials will be measures of efficacy, often surrogates for final clinical outcomes (e.g., reduction in blood pressure or cholesterol levels, rather than reduction in cardiovascular disease events or patient survival). Even where the surrogate is accepted as a good predictor of final outcomes, there may be additional uncertainty over the duration of the effect of treatment beyond the period observed in the trial. Many trials now include measurement of patient-reported outcomes, such as quality of life and utilities (using multi-attribute utility indices). This is a major step forward in assessing the value to the patient of any clinical benefits (and disbenefits from adverse events), but it still does not overcome the problems related to the short duration and controlled environment of many trials.

As with the patient health outcomes, healthcare resource use can be monitored in regulatory trials, providing a direct comparison between the new treatment and the control. The same issues arise over the relevance of the comparator arm and the duration of observation. There is also the issue of protocol-driven costs that may not occur in routine practice.

Although uncertainty around the cost and effectiveness data is important, it is not the only source of material uncertainty. Coverage decisions usually have to be made at a time when data on all the relevant variables and comparisons are not available from high-quality studies. Many qualitative judgments and assumptions can come into play in structuring a model to address the relevant questions. Formal analytical techniques can be used to measure the “value of information” (3;4), but ultimately a judgment must be made as to whether the benefit of more evidence is greater than the cost of delaying the decision until it is available.

Making decisions under uncertainty for technologies early in their life cycle may result in the coverage of technologies that are subsequently shown to be clinically or cost-ineffective. Conversely, coverage may be restricted for technologies that later prove to be clinically and cost-effective. In both cases, there are opportunity costs in terms of healthcare expenditure and/or health benefits associated with these inappropriate decisions.

Table 1. Potential Advantages and Disadvantages Associated with CED

CED, Coverage with Evidence Development.

Coverage with Evidence Development (CED) is one of several policy options that have been posited to overcome the problems associated with making coverage decisions under uncertainty. The application of conditionality to coverage decisions has been widely discussed (9;14). Conditionality allows a technology to be made available under specific conditions, usually for a defined period, after which the benefits of the technology are reviewed. Where conditionality is dependent upon the generation of further evidence through formal studies to support the value of a technology, this approach has been labeled as CED (17). The application of CED to date has tended to consider promising but unproven technologies in indications characterized by limited alternative treatment options (e.g., treatments for advanced colorectal cancer), suggesting that decision makers are more cognizant of uncertainty in areas of high unmet need. CED ensures that patient access to promising new technologies is not prevented but is managed in a coordinated way, while also generating additional evidence to reduce any uncertainty about the value of the technology. CED differs from traditional postmarketing evidence generation in that the objective of the additional evidence generation is to reduce uncertainty around a specific aspect of the evidence base and, thus, help to inform further decisions about ongoing coverage, often at predetermined points in the future. The role of the decision maker in determining the nature of the research is also expected to be greater than in traditional postmarketing studies.

An example of the use of CED is that of the Centers for Medicare and Medicaid Services (CMS) in the United States. CMS issued guidance in 2005 (5), with further revisions in 2006 (13), to describe when CED should be applied and how it should operate. The guidance was issued following the organic development of CED within CMS. CMS has now made some promising technologies with an equivocal evidence base available on the grounds that the technologies are only used in clinical trials or as part of a registry to help provide further evidence on their effectiveness. The introduction of CED offered a formal option for CMS to make promising technologies available that would otherwise fail to meet their criteria of “reasonable and necessary” and thus, be ineligible for coverage within their system. In Australia, the Medical Services Advisory Committee (MSAC), which determines the coverage of medical devices, has the ability to “provide interim funding to enable data collection, within an agreed research framework, in order to establish the evidence base” (12). Other countries to have piloted the use of CED include Canada and England, where the process has been driven by the findings of HTA reviews of technologies, and the Netherlands, where CED is being increasingly applied to pharmaceuticals for use in hospital settings. Similarly, the Catalan Agency for Health Technology Assessment in Spain has made recommendations on the funding of further research to support coverage decisions (6).

The following sections of this study discuss the key conceptual and policy issues associated with CED, attempt to further develop thinking on the practical applicability of CED, and identify further issues to be addressed. The study reflects discussion between and correspondence with those attending a meeting of the Health Technology Assessment International (HTAi) Policy Forum in February 2007, although not all those individuals, nor the organizations they come from, necessarily agree with all the opinions expressed in this study, for which the authors take full responsibility. (Names of those participating in the meeting of the Health Technology Assessment International Policy Forum in February 2007.)

POSITIONING OF CED

This section addresses the following two questions: (i) What factors should be considered in deciding whether to develop a CED system to help overcome uncertainty in coverage decisions? (ii) If a CED system is developed, what criteria should be applied to identify technologies suitable for CED?

Considerations in Developing CED as an Option

CED attempts to balance the wishes of patients, manufacturers, and healthcare decision makers, all of whom want to make promising technologies available to patients while also ensuring the efficient use of scarce healthcare resources. This strategy has similarities with ongoing debate about the regulation of new medicines and how the approval of new medicines can be expedited without detriment to patient safety. The pressure to expedite both regulatory and coverage decisions is driven at least in part by more informed patients demanding access to promising technologies.

Whereas CED appears to offer a potentially attractive solution to this issue, it is a complex system to implement, with impacts on patients, healthcare decision makers, and technology manufacturers and needs to be carefully considered before adoption. Table 1 summarizes some of the advantages and disadvantages of CED for each of the major stakeholder groups.

For healthcare decision makers, CED introduces an option that allows the product to be made available in a controlled manner while also allowing the decision maker to define what evidence is required to support further use of the technology. Potential disadvantages for decision makers are the extra demands of agreeing the study design and monitoring and reviewing the data collected, and the challenge of withdrawing coverage if this is the conclusion reached. (There is a general consensus that it is politically more difficult for decision makers to withdraw coverage—even if formally temporary—than to refuse coverage in the first place, in situations where the case against coverage is of similar weight [15]). Decision makers—or closely related bodies—may also be expected to contribute to the costs of collecting the data in CED, as has been the case in Spain (7). A further important consideration is whether the existence of a CED option will reduce the incentives for manufacturers to undertake appropriate research both before and after the licensing of a product.

For manufacturers, the introduction of a CED system allows promising technologies to be made available that might otherwise be rejected. However, the burden of proof often rests with the manufacturer, and adoption of a CED system may therefore increase costs for manufacturers. Moreover, there are concerns that the existence of a CED option may make decision makers more likely to demand further evidence for technologies on which they would otherwise make decisions. Depending on how the CED process operates, it may also lead to the manufacturer being forced to reconsider the pricing of the technology, although this may not always be feasible.

Healthcare providers may welcome the introduction of CED if it allows for earlier access to promising technologies, thus increasing treatment options available for their patients. However, they may also exercise some degree of caution about the use of technologies that are not fully evaluated or recommended by guidance. Given the increasing trends for litigation, some providers may remain reluctant to use a technology that remains under evaluation, particularly if the uncertainty relates to the clinical aspects of the technology.

The main benefit of CED to patients is access to technologies that have apparent benefits but remain unproven by conventional evidentiary standards. There are many examples of patients lobbying for access to developmental or unproven technologies in recent times, most notably in cancer and HIV indications. CED can allow patients access to such technologies and to participate in further research to generate evidence that will inform their own and other patients’ subsequent treatment. The main risk for patients is that they will be exposed to a technology which subsequently proves to be ineffective. In the case of pharmaceuticals, in non–life-threatening diseases, this finding may not have a significant impact on long-term outcomes, other than causing the patient to miss-out on optimal treatment for a period of time. In the case of medical devices, this may result in the need for re-operation or removal of a device if it fails to deliver expected health benefits (e.g., in the case of hip prostheses).

The degree of uncertainty that affects a technology at the time of launch is linked, at least in part, to the regulatory framework that applies to the technology. The requirements for approval of new pharmaceuticals are relatively stringent, specifying the need for well-designed, robust, randomized controlled trials on safety and efficacy end points. Medical devices can be approved on a more limited evidence base that shows that the product is capable of meeting a stated need and many are approved for use with no evidence from randomized controlled trials (depending on the class of product). The regulation of medical diagnostics requires evidence on their sensitivity and specificity to predefined clinical markers but takes no account of the impact of the diagnostic on treatment decisions or health outcomes. Surgical procedures meanwhile, remain largely unregulated with the exception of professional peer-review and clinical audit.

As the level of evidence required for approval for each of these classes of technology becomes less robust, the likelihood of uncertainty occurring increases. It could be argued that surgical procedures, diagnostics, and medical devices are more appropriate candidates for CED, although other factors may also influence whether CED is appropriate (for example, the budgetary impact of a technology). However, most systems that have developed a CED option seem to consider each technology on an individual basis to determine whether CED is a feasible solution to overcoming uncertainty, rather than restricting CED to certain classes of technology. Suggested criteria for identifying suitable technologies for CED are developed below.

Identifying Technologies Suitable for CED

Whereas all coverage decisions involve some degree of uncertainty, many can be resolved without resorting to CED. Even where CED exists as a policy option, it can be costly and complex and decision makers should give careful consideration to other ways of proceeding in the face uncertainty before choosing to go down that route.

Where there is uncertainty over the cost-effectiveness of the technology, it may be possible to bring the upper estimate of cost-effectiveness below an acceptable threshold by reducing the price of the technology. Although this can potentially lead to a rapid resolution, it requires a degree of flexibility from the manufacturer and assumes that margins on technology development are sufficient to allow prices to be readily reduced. Equally, it may be possible for the decision maker to exercise some flexibility on their threshold for cost-effectiveness to accommodate the uncertainty. Previous studies have discussed the appropriateness of accepting higher cost-effectiveness ratios for specific drugs (e.g., orphan drugs) or conditions (10).

Where uncertainty relates to the clinical effectiveness of a technology, further analysis of the available data with some degree of modeling or extrapolation may help to reduce, or at least quantify, the degree of uncertainty. Value of information analysis (VOI) may then show that additional evidence generation is not worthwhile.

Increased contact between manufacturers and decision makers during the development of a product may also help to overcome uncertainty by agreeing on what evidence is required to support coverage and defining the acceptable degree of uncertainty. Such discussions would follow the model increasingly being adopted by regulators but would require decision makers to be adequately resourced to engage in a timely and informed dialogue. There might also be scope for decision makers and regulators to work together more closely with one another in agreeing expectations with manufacturers.

When, then, should CED be used? As a general rule, it is suggested that CED is best suited to the following circumstances: where there are reasonable grounds for believing that a technology will offer significant benefits but there is uncertainty around the clinical or cost-effectiveness of the technology that can be overcome through evidence that can be generated in an appropriate time frame, and is the main source of equivocality in a coverage decision.

Figure 1 attempts to illustrate which technologies are suitable for CED. The diagram is a schematic representation of the selection of technologies suitable for CED. The horizontal axis represents the likely extent of net positive or negative performance in relation to current alternative treatment options (that is the origin represents a point where the technology is equivalent to current treatment), and the vertical axis indicates the degree of uncertainty.

Figure 1. Identifying technologies suitable for Coverage with Evidence Development (CED). Illustrative representation of when to consider CED.

Technologies that fall under the shaded area (beneath 0A) are those that are deemed suitable for coverage. That is, the balance between the expected net benefit and the degree of uncertainty is acceptable to the healthcare decision maker.

Those technologies that fall in the shaded area (between 0A and 0B), that is those with a potential net benefit but some degree of uncertainty, may be deemed suitable for CED. These should be reviewed for the presence of “material uncertainty,” in the evidence base. That is uncertainty to such a degree that it prevents a clear-cut decision, but whose resolution will enable a definite decision to be made. For example, the point estimate of the cost-effectiveness of a technology may be within an accepted threshold (e.g., £20,000 per quality-adjusted life-year in the United Kingdom), but the variability in the cost and effectiveness data means that there is a high probability that the technology may not be cost-effective. Collection of further data may reduce that variability and increase confidence in the validity of the point estimate as a basis for the decision.

Technologies that fall above 0B are less likely to be considered suitable for CED on the grounds that the expected net benefit is relatively small and the uncertainty relatively high. However, there may be cases where technologies in this sector may be considered suitable for CED on other grounds (for example, the limited availability of alternative treatment options).

The diagram is intended to be a simple schematic and does not take into account the intricacies of CED in practice. For example, there may be cases where a technology that has a small but negative effect may be considered suitable for CED if the degree of uncertainty means that the effect could actually be positive. It is also worth considering whether healthcare decision makers have a tolerance for uncertainty, even under CED. In which case, there may be technologies that, despite having an expected net benefit, are simply subject to too much uncertainty to be suitable for CED and may require the manufacturer to generate additional data before discussing a CED arrangement. This would take into account the potential perverse incentives for manufacturers to disinvest in research before launch on the grounds that a CED approach to coverage will always be available. These are issues that warrant further consideration.

A checklist of particular questions that need to be considered before entering into a CED process for a technology is given below.

• • Does the technology offer a potentially significant advance on current treatments (e.g., in severe conditions with a high unmet need) but with material uncertainty as a result of equivocal evidence?

• • Will collection of additional data reduce the uncertainty?

• • Will collection of that data within a CED process, that is, in a routine care situation, increase the relevance of the data to the coverage decision?

• • Can CED provide information to help reduce uncertainty within an appropriate period (for example, the patent life of the technology or before the technology is likely to be superseded)?

• • Will granting of conditional reimbursement prevent the use of optimal study designs in the additional data collection, for example, by reducing incentives for patients to participate in trials?

• • Is the benefit of an improved decision sufficient to justify the costs of a CED process?

PRACTICAL ISSUES

Agreement on Data Requirements and Study Design

Once the variables on which new data are needed have been identified, agreement must be reached on the evidentiary standards to which data should be collected to reduce the level of uncertainty around the decision to acceptable levels. From the cost-effectiveness analysis, it will be possible to identify the range within which the value of a variable must fall to give reasonable confidence that an appropriate level of clinical benefit and/or cost-effectiveness will be achieved. The new data collection should be designed to show more robustly whether the variable falls within the required range. All stakeholders should agree in advance to the design of the study, and to accept its results.

Time Horizon

For each application, the CED process should have a target date by which the revised decision will be made. The completion time for any new data collection will be determined with this in mind. In view of the pace of technical change in health care, a CED process lasting more than 3 years risks becoming of limited relevance in the face of changing clinical practice. This risk may be greater for surgical procedures and devices, where continual incremental improvements to the technology take place. Difficulties can arise, for example, if patient recruitment rates in studies are lower than expected. If CED becomes indefinite, without the benefit of new evidence, it is no different from coverage with inadequate evidence. On the other hand, if CED is stopped because the new data are not available in a timely manner, then patients and manufacturers may believe they have legitimate objections. Decision makers must be sure that collection of relevant new data is feasible within a relatively short period before embarking on a CED process.

Finance and Management of Studies

The funding of any further data collection is a potentially contentious issue. Whereas the general expectation is that manufacturers and sponsors of technologies will finance the extra data collection in CED, there are examples of government funding (e.g., MSAC in Australia, Catalan Agency for Health Technology Assessment [CAHTA] in Spain). For surgical procedures, there may be no manufacturer involved, so public sector or health system support may be needed. However, many procedures now use specific equipment that requires the involvement of a manufacturer.

The overall management of studies will generally be best undertaken by, or on behalf of, the decision-making body. A steering group representing all stakeholders may be helpful in maintaining support for the process.

Data Collection and Analysis

This should be undertaken by a body independent of the decision maker and the manufacturer, such as a contract research organization or research institute. If the new data are from an extension of an ongoing clinical trial, then the existing arrangements can be extended. Subject to the normal rules regarding confidentiality of patient data, and issues of commercial confidentiality, the final data set as well as the analysis should be available to all stakeholders on request.

Decision with Further Evidence

The new data should be incorporated into an analysis that the decision maker can use to underpin the Decision with Further Evidence (DFE). How this is done will vary depending on the decision-making system in each country, for example, with regard to the degree of public consultation on the new analysis. Any appeal of the DFE should follow the standard process of that system. It should be acknowledged that, even after additional evidence generation, there will continue to be some degree of uncertainty and it may be that the level of evidence remains suboptimal. In that situation, the decision maker will nonetheless need to be prepared to decide whether to agree coverage (with or without conditions) or withhold coverage. A decision to embark on a further round of CED would only be reached in exceptional circumstances, for example, where there had been significant changes in the decision context from when the time scale and other conditions of the original CED were decided.

OPERATIONALISING A DECISION WITH FURTHER EVIDENCE (DFE)

Following consideration of the additional evidence generated, the outcomes of the DFE may range from no coverage, through restricted coverage, to unconditional coverage of a technology. If the evidence generated is sufficient to overcome all the pre-defined aspects of uncertainty, then it may be appropriate for the technology to be made freely available for use (within the licensed indication). Conversely, if the additional evidence fails to reduce the uncertainty then there may be a rationale for suggesting that use of the technology should stop. However, these outcomes should be regarded as the two ends of a spectrum, and the more likely outcome is that some form of restricted access is indicated, (e.g., use in patient sub-groups). The issues discussed above, relating to the adequacy of the outcomes and the limits on evidence generation are critical to ensuring a resolution can be reached.

The means of ensuring appropriate use of a technology after a DFE will vary depending on the remit of the body involved in establishing the CED process. Where the CED process is managed by a body with direct control of coverage, then disinvestment in a technology is relatively simple. The removal of coverage is usually a sufficient incentive mechanism to contain any further use, at least on a widespread basis. Similarly, removing restrictions on coverage or increasing coverage levels should also allow for expanded uptake.

Where CED is managed by a body that does not directly control healthcare budgets (e.g., National Institute of Clinical Excellence [NICE] in England and Wales), then changing investment may be more challenging. As noted earlier, the removal of health technologies that subsequently prove to provide inadequate benefit compared to existing technologies is more difficult than restricting access to and diffusion of technologies at their launch. Similarly, there may be reluctance to increase use of a technology where patients and clinicians have become accustomed to restrictions based on earlier assumptions of limited effectiveness.

Whatever the source of uncertainty, it will affect confidence in the cost-effectiveness of the technology. As stated above, where technologies prove to be less effective than originally thought, it may be appropriate to reduce the price of the technology to maintain cost-effectiveness. Conversely, where the technology proves to be more effective than initially thought, the manufacturer may also want the freedom to increase price to such a level that the price reflects the value of the benefits (e.g., the cost-effectiveness moves to the limit of any accepted threshold). These processes are, of course, dependent on the agreement of the pricing authorities and manufacturers and may be more applicable to some systems than in others.

It is clear from the above discussion that implementation of a DFE may be unpopular with some stakeholders in some circumstances. The potential implications of embarking on the CED process need to be defined at the outset and made clear to all parties involved.

Discussion

The previous sections have addressed the main considerations in the design and operation of a decision-making system using CED. It can be seen that the details of implementation are dependent on the specific context in which reimbursement decisions are made in individual healthcare systems. Discussion at the HTAi Policy Forum meeting covered several issues relating to this, for which there is insufficient space in this study for full discussion. The most important of these issues are links with the regulatory approval system and patient involvement in the CED process.

If decisions regarding coverage of technologies are to be made close to launch, the main source of data to inform such decisions will be from the regulatory process. These processes vary in their evidentiary standards for different types of technology, but generally focus on clinical benefits and patient safety issues. Other data relevant to coverage decisions, such as patient quality of life outcomes and healthcare resource use can be collected in trials designed to meet the needs of regulators, and this is increasingly done in the pharmaceutical sector. Use of such data in coverage decisions introduces uncertainty over the relation between behavior (of both clinicians and patients) observed in a controlled clinical trial environment and behavior in a routine care situation. If the uncertainty over the cost-effectiveness of a technology, at the time of launch, results from concerns over what will happen in routine practice, then CED offers a way of generating further evidence on these variables without delaying access to treatments with clinical benefits demonstrated in trials. If the uncertainty over the cost-effectiveness results from the lack of strong clinical evidence, then there may be more of an argument for delaying coverage until new clinical studies are carried out. This would be especially the case if more randomized studies were needed, as partial coverage through CED may reduce incentives for patients to take part in such studies. It can be argued that, if the clinical evidence is inadequate, then a product will not be licensed, especially a pharmaceutical. However, the comparators and outcomes used in regulatory studies may not match those required for coverage decisions; what is adequate for licensing may not be sufficient for a reliable coverage decision. Any coverage decision maker contemplating the use of a CED process should fully explore the potential benefits of closer collaboration with regulators in generating the most relevant, reliable, and timely data.

Patients are involved in the use and evaluation of technologies in multiple ways. They are the beneficiaries of successful new treatments; they bear the risk of adverse events (and in some countries the cost of using products); they are the participants in trials to test new products; and they may be part of the coverage decision-making process through membership of committees such as the NICE Appraisal Committee and the Scottish Medicines Consortium. How patients become involved in coverage decision making currently varies between countries, but the whole issue is worthy of further study. Data from patients is a vital part of the analysis to support decisions. For example, the outcome measures used in cost-effectiveness studies increasingly reflect patient and societal preferences. There is no consensus on how far the views of patients should directly influence coverage decisions, especially in CED. For example, patients do not want ineffective treatments, but they might be willing to accept a higher level of risk and uncertainty around the benefits of treatment than clinicians or health system decision makers.

In attempting to cover the whole range of issues pertinent to CED, the discussion at the HTAi Policy Forum inevitably raised more questions than answers. This study is an attempt to capture the flavor of those discussions and to draw attention to the most important factors to be considered before using CED. In so doing, it is hoped that this approach contributes to identifying those specific circumstances in which CED might provide a better way forward than current procedures in securing the most benefit from existing and emerging health technologies.

CONTACT INFORMATION

John Hutton, BPhil (jh602@york.ac.uk), Professor, Paul Trueman, MA (pt507@york.ac.uk), Director, York Health Economics Consortium, University of York, York YO10 5NH, UK

Chris Henshall, PhD (ch523@york.ac.uk), Pro-Vice Chancellor, University of York, York YO10 5DD, UK