Objectives: The English National Horizon Scanning Centre routinely scans thirty-five sources to identify new and emerging health technologies. The aim of the study was to develop and apply evaluation criteria and scores to assess the value of sources, and to identify a cutoff score below which sources would be recommended for removal from routine horizon scanning. Criteria to evaluate each source scanned could result in a more efficient approach in the selection process.

Methods: Evaluation criteria were developed following a review of the literature and discussions with horizon analysts. Proposed criteria were piloted on a random selection of six sources, and then applied to all thirty-five sources. The criteria were assessed using the Analytic Hierarchy Process.

Results: Eight criteria were identified as being most relevant for assessing the value of scanning sources. The three most important (primary) criteria were coverage (approximate percentage of relevant information), quality (reliable, accurate, objective), and efficiency (estimated time to identify one potentially significant health technology or other relevant information). Seven sources fell beneath the cutoff score and were recommended for removal from routine scanning.

Conclusions: The criteria were considered useful in the assessment of current sources, and have the potential to be used to assess new ones. These criteria may be useful for other horizon scanning centers to pilot and validate.

Objectives: The aim of this study was to analyze the EuroScan Database and to describe and compare the characteristics of the included technologies and participating agencies.

Methods: Data of interest were exported from the EuroScan Database to Excel and to SPSS. A descriptive analysis depending on the agency, type of technology, stage of diffusion, and technology purpose was conducted. A frequency distribution analysis of the diffusion stage for different technology types and assigned purposes was made with the EpiCalc 2000 statistical calculator. A p value of less than .05 was considered to be statistically significant.

Results: Four agencies introduced the great majority of the technologies (81 percent), with drugs representing the 46.26 percent of the total, followed by devices (21.21 percent). The purpose of 24.45 percent of the identified technologies was not specified, and 34.58 percent of them were identified at the investigational or phase III stage. The frequency distribution of diffusion stage at identification was found to be similar for devices and diagnostics (p = .543), whereas drugs were identified earlier than devices (p <.001). Some agencies were found to focus their work on drugs, whereas others focused mainly on devices. Interagency differences were also observed with regard to the stage of diffusion at which technologies were identified.

Conclusions: This is the first analysis of one of the most important databases on new and emerging health technologies. Our study suggests that more active strategies should be designed to provide an earlier identification, mainly in the case of devices.

Objectives: Uncertainty is pervasive in decision making on new health technologies; therefore, some countries have put systems in place to support decision makers with timely information. An important, but as yet undocumented, determinant of the potential value for decision making of these so-called horizon scanning systems (HSSs) is how the most significant health technologies are selected.

Methods: All thirteen member organizations of EuroScan, a collaborative network for HSSs, were surveyed and interviewed on how they prioritize technologies for assessment.

Results: The majority of HSSs directly serves a customer. Some customers actively request early assessments of new health technologies, thereby diminishing the need for priority setting for the HSSs. All systems express a concern to miss an important technology and/or to select an unimportant technology. Almost all HSSs use explicit selection criteria, but these criteria hardly ever are operationalized. The number of criteria used varies, but costs and health benefit of the technology are always taken into account. The process of reaching a final decision is implicit, undocumented in all but one system, and is based on agreement by consensus.

Conclusions: The process of making the final decision on which technologies to assess can be improved by applying existing criteria more consistently and transparently. Current practice does not safeguard against missing an important technology. This finding is probably most important to act upon for systems with customers that do not actively request assessment of specific technologies.