To the Editor
Dear Dr. Mäkelä,
As strong advocates and adopters of early cost-effectiveness modeling for test evaluation, we were excited to see the recent article “The Early Bird Catches the Worm: Early Cost-Effectiveness Analysis of New Medical Tests,” in which Buisman and colleagues present a succinct and generalizable framework for conducting this type of analysis. The authors have done an excellent job in identifying and summarizing the key components of this methodology, as well as highlighting fundamental differences between early analyses versus the traditional late-stage approach. As the number of medical tests coming to market continues to grow, there is an ever growing need to develop rigorous methodology in this area, and we commend the authors for this timely work.
Early economic evaluations are an extremely useful tool that can be used in a range of different contexts to inform future investment decisions. Whilst the authors clearly highlight the key elements involved in early test evaluations, they say little about the contexts in which we believe these types of evaluations provide most benefit. We would, therefore, like to share our experiences in this area to provide further justification and depth to this framework.
Step 1 in the early economic evaluations framework is to “narrow down the scope of analysis by defining the test's application, target population, outcome measures, and investigating current test strategies and test strategies if the new test were available”; what we would call “care pathway analysis.” This is a key priority in early evaluations, as the optimal positioning(s) of a test in the clinical pathway is usually unclear in the early stages of development. Often there are several patient groups that could benefit from the test and multiple potential roles that the test could be used in, for example, the test could be useful in both a diagnostic and monitoring role. Identifying early on where and for whom the test provides the strongest potential value is crucial to focus subsequent research efforts and maximize the likelihood of downstream adoption.
In our experience, this is the key component of the framework that is generally well accepted but rarely conducted with sufficient rigor. Many companies reach the late stages of test development, having invested substantial money in getting a test to market, only to have that test fail as a result of insufficient research into the optimal placement of the test on the care pathway. Similarly government research bodies may invest in late-stage research on a new technology, only to discover that the test could have performed better if positioned elsewhere on the pathway, or on a specific patient subgroup that the study was not powered to capture. Relatively minimal investment into an early economic evaluation exploring the optimal placement of a technology at the beginning of the research pipeline can help to mitigate the risk of such late-stage failures.
Step 2 of the framework is to conduct an inventory of available evidence and data on the current test strategy. We wholly agree that this is an important step in the process, and would emphasize the growing potential utility of routinely collected data to model the care pathway. Relying on the literature and guidelines to define clinical pathways is unlikely to capture the messy reality of clinical practice, and it is well documented that diagnostic pathways differ notably across different regions. The increasing availability of routinely collected electronic data affords new opportunities to compile individual linked records databases, to provide “real world” clinical pathway maps and better inform the structure and outputs of economic models.
Steps 4 and 5 of the framework concern the early cost-effectiveness analysis and developing recommendations regarding further test development. Here, the authors briefly touch on the use of sensitivity analysis and value of information analysis to help inform future research decisions. In our experience, these analyses are a crucial step in helping to (a) inform the stop/go decision, (b) identify the direction of future research, and (c) determine the optimal design of that research. Developing the economic model early on forces you to define all of the parameters that require evidence up front; key uncertainties in these parameters can then be explored using sensitivity and value of information analysis, to identify which parameters are primary drivers of uncertainty in the cost-effectiveness argument and thus determine what type of further research is required. As the authors highlight, this method can be used iteratively as evidence is generated and added to the model to inform future research priorities. Even when there is no data to inform a parameter, an uninformative prior distribution can be set to explore that parameter's potential impact: we do not believe, therefore, that such analyses are an optional add-on to early cost-effectiveness analyses, but are rather a key component of any early evaluation.
In summary, early cost-effectiveness modeling is an extremely valuable contribution to test evaluation and one which, we believe, can structure, inform, and streamline the evidence generation process. The authors of this study have clearly highlighted the core components of this methodology and we hope that it is used more in the future.
Note: All authors of this letter are supported by the National Institute for Health Research (NIHR) Diagnostic Evidence Co-operative Leeds. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.
