Objectives: This study was designed to assess the sensitivity of three Ovid MEDLINE search filters developed to identify studies reporting health state utility values (HSUVs), to improve the performance of the best performing filter, and to validate resulting search filters.

Methods: Three quasi-gold standard sets (QGS1, QGS2, QGS3) of relevant studies were harvested from reviews of studies reporting HSUVs. The performance of three initial filters was assessed by measuring their relative recall of studies in QGS1. The best performing filter was then developed further using QGS2. This resulted in three final search filters (FSF1, FSF2, and FSF3), which were validated using QGS3.

Results: FSF1 (sensitivity maximizing) retrieved 132/139 records (sensitivity: 95 percent) in the QGS3 validation set. FSF1 had a number needed to read (NNR) of 842. FSF2 (balancing sensitivity and precision) retrieved 128/139 records (sensitivity: 92 percent) with a NNR of 502. FSF3 (precision maximizing) retrieved 123/139 records (sensitivity: 88 percent) with a NNR of 383.

Conclusions: We have developed and validated a search filter (FSF1) to identify studies reporting HSUVs with high sensitivity (95 percent) and two other search filters (FSF2 and FSF3) with reasonably high sensitivity (92 percent and 88 percent) but greater precision, resulting in a lower NNR. These seem to be the first validated filters available for HSUVs. The availability of filters with a range of sensitivity and precision options enables researchers to choose the filter which is most appropriate to the resources available for their specific research.

Objectives: Economic evaluation (EE) is an accepted element of decision making and priority setting in healthcare. As the number of published EEs grows, so does the number of systematic reviews (SRs) of EEs. Although search methodology makes an important contribution to SR quality, search methods in reviews of EEs have not been evaluated in detail. We investigated the resources used to identify studies in recent, published SRs of EEs, and assessed whether the resources reflected recommendations.

Methods: We searched MEDLINE for SRs of EEs published since January 2013 and extracted the following from eligible reviews: databases searched, health technology assessment (HTA) sources searched, supplementary search techniques used. Results were compared against the minimum search resources recommended by National Institute for Health and Care Excellence (NICE) (MEDLINE, Embase, NHS EED, EconLit) for economic evidence for single technology appraisals, and resource types suggested in the summary of current best evidence from SuRe Info (economic databases, general databases, HTA databases, HTA agency Web pages, gray literature).

Results: Sixty-five SRs met the inclusion criteria; data were extracted from forty-two. Five reviews (12 percent) met or exceeded the NICE recommended resources. Nine reviews (21 percent) searched at least four of the five types of resource recommended by SuRe Info. Five reviews (12 percent) searched all five. Twenty-three reviews (55 percent) did not meet the NICE recommendations or four of five of the SuRe Info recommended resource types. Search reporting was frequently unclear or incorrect.

Conclusions: Searches conducted for the majority of recently published SRs of EEs do not meet two published approaches.

Objectives: The objective of this study was to explore the degree to which databases other than MEDLINE contribute studies relevant for inclusion in rapid health technology assessments (HTA).

Methods: We determined the extent to which the clinical, economic, and social studies included in twenty-one full and four rapid HTAs published by three Canadian HTA agencies from 2007 to 2012 were indexed in MEDLINE. Other electronic databases, including EMBASE, were then searched, in sequence, to assess whether or not they indexed studies not found in MEDLINE. Assessment topics ranged from purely clinical (e.g., drug-eluting stents) to those with broader social implications (e.g., spousal violence).

Results: MEDLINE contributed the majority of studies in all but two HTA reports, indexing a mean of 89.6 percent of clinical studies across all HTAs, and 88.3 percent of all clinical, economic, and social studies in twenty-four of twenty-five HTAs. While EMBASE contributed unique studies to twenty-two of twenty-five HTAs, three rapid HTAs did not include any EMBASE studies. In some instances, PsycINFO and CINAHL contributed as many, if not more, non-MEDLINE studies than EMBASE.

Conclusions: Our findings highlight the importance of assessing the topic-specific relative value of including EMBASE, or more specialized databases, in HTA search protocols. Although MEDLINE continues to be a key resource for HTAs, the time and resource limitations inherent in the production of rapid HTAs require that researchers carefully consider the value and limitations of other information sources to identify relevant studies.

Objectives and Methods: Health technology assessment (HTA) often requires the identification and review of economic evaluations and models. This study surveys the available specific and general resources to search to identify economic evaluations. It also provides information on efficient searching of those resources and comments on the current evidence-base.

Results: Published checklists recommend searching for economic evaluations in specific information resources which collect economic evaluations such as NHS EED and HEED, followed by top-up searches of large biomedical bibliographic databases (such as MEDLINE and EMBASE). Other resources such as the HTA and DARE databases can yield reports of economic evaluations. Searches within NHS EED and HEED can be made more efficient by using database-specific search options. Searches within large biomedical databases such as MEDLINE and EMBASE require the use of economic search terms called search filters. Search filters are highly sensitive, retrieving most economic evaluations, but suffer from low precision returning many irrelevant records which need to be assessed.

Conclusions: It is relatively easy to identify rapidly a high proportion of economic evaluations but more research is required to improve the efficiency of this process. There are key high yield resources to search but more evidence is required on their overlap and unique contribution to searches. The value of other resources, particularly those providing access to gray literature, should be explored. Research into efficient retrieval requires clear definitions of economic evaluations to allow comparison across studies.

Objectives: The aim of this study was to analyze the information needs of Spanish health technology assessment (HTA) agencies and units to facilitate access to the resources they require to substantiate their reports.

Methods: A questionnaire was designed and distributed among HTA bodies to ascertain the actual situation of subscriptions to information resources and what information specialists from these bodies considered would be the ideal subscription situation. Their information needs were then studied, and the resources that best met these needs were put forward. Following this definition, a subscriptions policy was adopted with suppliers and publishers.

Results: The survey showed that HTA bodies share a minimum of core subscriptions that includes open sources (MEDLINE, DARE) and sources that the government subscribes to for the health community (ISI Web of Science, Cochrane Library Plus). There was no common approach to determining which databases to subscribe to (UpToDate, EMBASE, Ovid EBMR, CINAHL, or ECRI). After identifying the information needs, a list of resources was proposed that would best cover these needs and, of these, subscription to the following was proposed: Scopus, Ovid EBMR, Clinical Evidence, DynaMed, ECRI, and Hayes.

Conclusions: There are differences in the way that HTA agencies and units access the different resources of biomedical information. Combined subscription to several resources for documentation services was suggested as a way of resolving these differences.

Objectives: The aim of this study is to review briefly different methods for determining the optimal retrieval of studies for inclusion in a health technology assessment (HTA) report.

Methods: This study reviews the methodology literature related to specific methods for evaluating yield from literature searching strategies and for deciding whether to continue or desist in the searching process.

Results: Eight different methods were identified. These include using the Capture–recapture technique; obtaining Feedback from the commissioner of the HTA report; seeking the Disconfirming case; undertaking comparison against a known Gold standard; evaluating retrieval of Known items; recognizing the Law of diminishing returns, specifying a priori Stopping rules, and identifying a point of Theoretical saturation.

Conclusions: While this study identified a variety of possible methods, there has been very little formal evaluation of the specific strengths and weaknesses of the different techniques. The author proposes an evaluation agenda drawing on an examination of existing data together with exploration of the specific impact of missing relevant studies.

Objectives: Health technology assessment (HTA) agencies assessing the cost-effectiveness of healthcare technologies seek evidence from economic evaluations. As well as searching economic evaluation databases, researchers often search MEDLINE and EMBASE, using search filters whose current performance is unclear. We assessed the performance of search filters in identifying economic evaluations from MEDLINE and EMBASE.

Methods: A gold standard of economic evaluations was compiled from National Health Service Economic Evaluation Database (NHS EED) records for 2000, 2003, and 2006. Corresponding records were retrieved in MEDLINE and EMBASE. Search filters were identified from the InterTASC Information Specialists’ SubGroup Web site and from Canadian Agency for Drugs and Technologies in Health (CADTH) Information Services. The sensitivity and precision of search filters in retrieving gold standard records from MEDLINE and EMBASE were tested.

Results: A total of 2,070 full economic evaluations were identified from NHS EED. Of these, 1,955 records were available in Ovid MEDLINE and 1,873 were available in Ovid EMBASE. Thirteen MEDLINE and eight EMBASE filters were identified. NHS Quality Improvement Scotland (full and brief filters), the NHS EED and Royle and Waugh filters achieved over 0.99 sensitivity in MEDLINE. NHS Quality Improvement Scotland, CADTH, Royle and Waugh, and NHS EED filters achieved greater than 0.99 sensitivity in EMBASE. Filters demonstrated low precision.

Conclusions: This research provided new performance data on search filters to identify economic evaluations in MEDLINE and EMBASE. It demonstrated that highly sensitive economic evaluation filters are available, but that precision is low, yielding perhaps 5 relevant records per 100 records scanned.

Objectives: The aim of this study was to demonstrate the best way of identifying all relevant published health economic evaluation studies, which have increased in number rapidly in the past few decades. Nevertheless, health technology assessment projects are often faced with a scarcity of relevant studies.

Methods: Six bibliographic databases were searched using various individually adapted strategies. The particular example involves the cost-effectiveness of diagnosing gastroesophageal reflux disease. Inclusion and exclusion criteria were formulated.

Results: After irrelevant studies and duplicates had been excluded, sixty-eight abstracts were reviewed. We chose forty-one of them as relevant for full-text review, which identified fourteen papers as having met the inclusion criteria. Most of the relevant studies were identified by searching the National Health Service Economic Evaluation Database (NHS EED) and PubMed databases.

Conclusions: A search in NHS EED, by means of the Cochrane Library or the Center for Reviews and Dissimination, along with a supplementary search in PubMed, is generally an appropriate, cost-effective strategy. However, because “cost-effectiveness” is not consistently indexed with Medical Subject Heading terms in PubMed, all economic search terms need to be used to fully identify the relevant references.

Objectives: The aim of this study was to evaluate the success of search strategies in retrieving key documents for a technology assessment report (TAR) on a social care topic.

Methods: This study measured the differential yield of relevant studies from various information sources and evaluated strategies in different databases, with particular reference to capturing randomized controlled trials (RCTs) as a study design.

Results: A combination of four major databases would have found all thirty-two key references. One database alone would have found 78 percent, with another two each locating 59 percent. Sixteen percent of the trials were unique references. In non–health care databases, more sensitive search strategies would have resulted in a higher yield of relevant studies, in part due to inconsistent indexing and in part to attempts to restrict searches to RCTs. Although additional terms could be used to increase the sensitivity of the original strategies, this raises the question of trading off time against exhaustiveness, given the greater number of irrelevant references likely to be retrieved.

Conclusions: A successful search for evidence on this social care topic would be possible using a combination of MEDLINE, EMBASE, the Cochrane Library and PsycINFO, supplemented by only limited use of supplementary databases. In areas such as social care where evidence-based research is not yet well established, attempts to replicate searches based on study design do not seem to be advisable, although this may be an area for future research.

Biomedical databases are an important source of information for health technology assessment. However, there is considerable variation in the costs of accessing commercial databases. We sought to measure the quality, amount of overlap, and costs of information retrieved from two of the main database sources — MEDLINE and EMBASE. Librarians at two health technology assessment agencies ran a total of eight literature searches on various medical technologies, using both databases. All search results were independently reviewed by two researchers. The researchers were asked to identify relevant references and to rank each of these according to a level of evidence scale. The results were tabulated to show the number of references identified by each database, the number of relevant references ranked by level of evidence, and the number of these references that were unique to one or the other database. The cost of retrieving references from each source was also calculated. Each database contained relevant references not available in the other. Because of the longer time lag for indexing in MEDLINE, many of the references that originally appeared to be unique to EMBASE were subsequently available in MEDLINE as well. Since our study was conducted, MEDLINE has been made available worldwide, free of charge, via the Internet. Hence, the cost difference between the databases is now even greater. However, notwithstanding the costs, it appears that literature searches that rely on only one or the other database will inevitably miss pertinent information.