Objectives: The English language is generally perceived to be the universal language of science. However, the exclusive reliance on English-language studies may not represent all of the evidence. Excluding languages other than English (LOE) may introduce a language bias and lead to erroneous conclusions.

Study Design and Setting: We conducted a comprehensive literature search using bibliographic databases and grey literature sources. Studies were eligible for inclusion if they measured the effect of excluding randomized controlled trials (RCTs) reported in LOE from systematic review-based meta-analyses (SR/MA) for one or more outcomes.

Results: None of the included studies found major differences between summary treatment effects in English-language restricted meta-analyses and LOE-inclusive meta-analyses. Findings differed about the methodological and reporting quality of trials reported in LOE. The precision of pooled estimates improved with the inclusion of LOE trials.

Conclusions: Overall, we found no evidence of a systematic bias from the use of language restrictions in systematic review-based meta-analyses in conventional medicine. Further research is needed to determine the impact of language restriction on systematic reviews in particular fields of medicine.

Objectives: To analyze sources searched in Cochrane reviews, to determine the proportion of trials included in reviews that are indexed in major databases, and to compare the quality of these trials with those from other sources.

Methods: All new systematic reviews in the Cochrane Library, Issue1 2001, that were restricted to randomized controlled trials (RCTs) or quasi-RCTs were selected. The sources searched in the reviews were recorded, and the trials included were checked to see whether they were indexed in four major databases. Trials not indexed were checked to determine how they could be identified. The quality of trials found in major databases was compared with those found from other sources.

Results: The range in the number of databases searched per review ranged between one and twenty-seven. The proportion of the trials in the four databases were Cochrane Controlled Trials Register=78.5%, MEDLINE=68.8%, Embase=65.0%, and Science/Social Sciences Citation Index=60.7%. Searching another twenty-six databases after Cochrane Controlled Trials Register (CCTR), MEDLINE, and Embase only found 2.4% additional trials. There was no significant difference between trials found in the CCTR, MEDLINE, and Embase compared with other trials, with respect to adequate allocation concealment or sample size.

Conclusions: There was a large variation between reviews in the exhaustiveness of the literature searches. CCTR was the single best source of RCTs. Additional database searching retrieved only a small percentage of extra trials. Contacting authors and manufacturers to find unpublished trials appeared to be a more effective method of obtaining the additional better quality trials.

Objectives: The aim of the Consensus on Health Economic Criteria (CHEC) project is to develop a criteria list for assessment of the methodological quality of economic evaluations in systematic reviews. The criteria list resulting from this CHEC project should be regarded as a minimum standard.

Methods: The criteria list has been developed using a Delphi method. Three Delphi rounds were needed to reach consensus. Twenty-three international experts participated in the Delphi panel.

Results: The Delphi panel achieved consensus over a generic core set of items for the quality assessment of economic evaluations. Each item of the CHEC-list was formulated as a question that can be answered by yes or no. To standardize the interpretation of the list and facilitate its use, the project team also provided an operationalization of the criteria list items.

Conclusions: There was consensus among a group of international experts regarding a core set of items that can be used to assess the quality of economic evaluations in systematic reviews. Using this checklist will make future systematic reviews of economic evaluations more transparent, informative, and comparable. Consequently, researchers and policy-makers might use these systematic reviews more easily. The CHEC-list can be downloaded freely from http://www.beoz.unimaas.nl/chec/.

Utilities and quality-adjusted life years (QALYs) are reviewed, with particular focus on their use in technology assessment. This article provides a broad overview and perspective on these two techniques and their interrelationship, with reference to other sources for details of implementation. The historical development, assumptions, strengths/weaknesses, and applications of each are summarized.

Utilities are specifically designed for individual decision-making under uncertainty, but, with additional assumptions, utilities can be aggregated across individuals to provide a group utility function. QALYs are designed to aggregate in a single summary measure the total health improvement for a group of individuals, capturing improvements from impacts on both quantity of life and quality of life– with quality of life broadly defined. Utilities can be used as the quality-adjustment weights for QALYs; they are particularly appropriate for that purpose, and this combination provides a powerful and highly useful variation on cost-effectiveness analysis known as cost-utility analysis.

Objectives: In an interrupted time series (ITS) design, data are collected at multiple instances over time before and after an intervention to detect whether the intervention has an effect significantly greater than the underlying secular trend. We critically reviewed the methodological quality of ITS designs using studies included in two systematic reviews (a review of mass media interventions and a review of guideline dissemination and implementation strategies).

Methods: Quality criteria were developed, and data were abstracted from each study. If the primary study analyzed the ITS design inappropriately, we reanalyzed the results by using time series regression.

Results: Twenty mass media studies and thirty-eight guideline studies were included. A total of 66% of ITS studies did not rule out the threat that another event could have occurred at the point of intervention. Thirty-three studies were reanalyzed, of which eight had significant preintervention trends. All of the studies were considered “effective” in the original report, but approximately half of the reanalyzed studies showed no statistically significant differences.

Conclusions: We demonstrated that ITS designs are often analyzed inappropriately, underpowered, and poorly reported in implementation research. We have illustrated a framework for appraising ITS designs, and more widespread adoption of this framework would strengthen reviews that use ITS designs.

The standard way to assess medical technologies is to conduct a randomized clinical trial. Patients are randomly assigned to groups receiving alternative treatments, and outcomes are monitored over a long period of time. For example, some victims of left main coronary artery disease may undergo coronary artery bypass surgery, and others may receive medical treatment with nitroglycerine and beta blockers. Comparison of five-year mortality and morbidity in the two groups helps to determine the relative appropriateness of the two procedures. In addition, information about quality of life and cost can also be collected and compared.

Advances in science and technology are generally the product of multiple investigations. This article discusses statistical methods for combining empirical results from a series of different experiments or clinical investigations. We delineate the steps an assessor might take in combining data from different studies and provide references for topics not discussed in detail. The article reviews some of the most commonly used statistical techniques for combining results in the medical and social sciences.

The expression meta-analysis refers to the practice of using statistical methods to combine the outcomes of a series of different experiments or investigations. The terminology arises because a meta-analysis uses a study as the unit of observation, and, thus, the data point for each study is itself a statistical summary based on an analysis of primary data.

Assessing the quality of randomized controlled trials is a relatively new and important development. Three approaches have been developed: component, checklist, and scale assessment. Component approaches evaluate selected aspects of trials, such as masking. Checklists and scales involve lists of items thought to be integral to study quality. Scales, unlike the other methods, provide a summary numeric score of quality, which can be formally incorporated into a systematic review. Most scales to date have not been developed with sufficient rigor, however. Empirical evidence indicates that differences in scale development can lead to important differences in quality assessment. Several methods for including quality scores in systematic reviews have been proposed, but since little empirical evidence supports any given method, results must be interpreted cautiously. Future efforts may be best focused on gathering more empirical evidence to identify trial characteristics directly related to bias in the estimates of intervention effects and on improving the way in which trials are reported.

Health technology assessment (HTA) is a dynamic, rapidly evolving process, embracing different types of assessments that inform real-world decisions about the value (i.e., benefits, risks, and costs) of new and existing technologies. Historically, most HTA agencies have focused on producing high quality assessment reports that can be used by a range of decision makers. However, increasingly organizations are undertaking or commissioning HTAs to inform a particular resource allocation decision, such as listing a drug on a national or local formulary, defining the range of coverage under insurance plans, or issuing mandatory guidance on the use of health technologies in a particular healthcare system. A set of fifteen principles that can be used in assessing existing or establishing new HTA activities is proposed, providing examples from existing HTA programs. The principal focus is on those HTA activities that are linked to, or include, a particular resource allocation decision. In these HTAs, the consideration of both costs and benefits, in an economic evaluation, is critical. It is also important to consider the link between the HTA and the decision that will follow. The principles are organized into four sections: (i) “Structure” of HTA programs; (ii) “Methods” of HTA; (iii) “Processes for Conduct” of HTA; and (iv) “Use of HTAs in Decision Making.”

Objectives: The objective of physician continuing medical education (CME) is to help them keep abreast of advances in patient care, to accept new more-beneficial care, and discontinue use of existing lower-benefit diagnostic and therapeutic interventions. The goal of this review was to examine effectiveness of current CME tools and techniques in changing physician clinical practices and improving patient health outcomes.

Methods: Results of published systematic reviews were examined to determine the spectrum from most- to least-effective CME techniques. We searched multiple databases, from 1 January 1984 to 30 October 2004, for English-language, peer-reviewed meta-analyses and other systematic reviews of CME programs that alter physician behavior and/or patient outcomes.

Results: Twenty-six reviews met inclusion criteria, that is, were either formal meta-analyses or other systematic reviews. Interactive techniques (audit/feedback, academic detailing/outreach, and reminders) are the most effective at simultaneously changing physician care and patient outcomes. Clinical practice guidelines and opinion leaders are less effective. Didactic presentations and distributing printed information only have little or no beneficial effect in changing physician practice.

Conclusions: Even though the most-effective CME techniques have been proven, use of least-effective ones predominates. Such use of ineffective CME likely reduces patient care quality and raises costs for all, the worst of both worlds.

Objectives: Telemedicine applications could potentially solve many of the challenges faced by the healthcare sectors in Europe. However, a framework for assessment of these technologies is need by decision makers to assist them in choosing the most efficient and cost-effective technologies. Therefore in 2009 the European Commission initiated the development of a framework for assessing telemedicine applications, based on the users’ need for information for decision making. This article presents the Model for ASsessment of Telemedicine applications (MAST) developed in this study.

Methods: MAST was developed through workshops with users and stakeholders of telemedicine.

Results: Based on the workshops and using the EUnetHTA Core HTA Model as a starting point a three-element model was developed, including: (i) preceding considerations, (ii) multidisciplinary assessment, and (iii) transferability assessment. In the multidisciplinary assessment, the outcomes of telemedicine applications comprise seven domains, based on the domains in the EUnetHTA model.

Conclusions: MAST provides a structure for future assessment of telemedicine applications. MAST will be tested during 2010–13 in twenty studies of telemedicine applications in nine European countries in the EC project Renewing Health.

Economic evaluations of health interventions pose a particular challenge for reporting. There is also a need to consolidate and update existing guidelines and promote their use in a user friendly manner. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement is an attempt to consolidate and update previous health economic evaluation guidelines efforts into one current, useful reporting guidance. The primary audiences for the CHEERS statement are researchers reporting economic evaluations and the editors and peer reviewers assessing them for publication.

The need for new reporting guidance was identified by a survey of medical editors. A list of possible items based on a systematic review was created. A two round, modified Delphi panel consisting of representatives from academia, clinical practice, industry, government, and the editorial community was conducted. Out of 44 candidate items, 24 items and accompanying recommendations were developed. The recommendations are contained in a user friendly, 24 item checklist. A copy of the statement, accompanying checklist, and this report can be found on the ISPOR Health Economic Evaluations Publication Guidelines Task Force website (www.ispor.org/TaskForces/EconomicPubGuidelines.asp).

We hope CHEERS will lead to better reporting, and ultimately, better health decisions. To facilitate dissemination and uptake, the CHEERS statement is being co-published across 10 health economics and medical journals. We encourage other journals and groups, to endorse CHEERS. The author team plans to review the checklist for an update in five years.

Objectives: The aim of this study was to develop the GRIPP (Guidance for Reporting Involvement of Patients and Public) checklist to enhance the quality of PPI reporting.

Methods: Thematic analysis was used to synthesize key issues relating to patient and public involvement (PPI) identified in the PIRICOM and PAPIRIS systematic reviews. These issues informed the development of the GRIPP checklist.

Results: The key issues identified included limited conceptualization of PPI, poor quality of methods reporting, unclear content validity of studies, poor reporting of context and process, enormous variability in the way impact is reported, little formal evaluation of the quality of involvement, limited focus on negative impacts, and little robust measurement of impact. The GRIPP checklist addresses these key issues.

Conclusion: The reporting of patient and public involvement in health research needs significant enhancement. The GRIPP checklist represents the first international attempt to enhance the quality of PPI reporting. Better reporting will strengthen the PPI evidence-base and so enable more effective evaluation of what PPI works, for whom, in what circumstances and why.

There is growing demand for translations of health status questionnaires for use in multinational drug therapy studies and for population comparisons of health statistics. The International Quality of Life Assessment (IQOLA) Project is conducting a three-stage research program to determine the feasibility of translating the SF-36 Health Survey, widely used in English-speaking countries, into other languages. In stage 1, the conceptual equivalence and acceptability of translated questionnaires are evaluated and improved using qualitative and quantitative methods. In stage 2, assumptions underlying the construction and scoring of questionnaire scales are tested empirically. In stage 3, the equivalence of the interpretation of questionnaire scores across countries is tested using methods that closely approximate their intended use, and empirical results are compared. Data analyses from Sweden and the United Kingdom, as well as other research cited, support the feasibility of cross-cultural health measurement using the SF-36.

This paper offers a theory to explain the diffusion of new medical technologies into local practice. Based on several hundred interviews with community hospital physicians, it anchors technology decisions in the norms and relationships of local practice. Physician descriptions of their use of different types of assessment information provide insight into the way in which local consenses on appropriate practice are formed, guide behavior, and change. To understand new technology adoption, it is necessary to (a) differentiate “formed” (complete) and “dynamic” (still developing) technologies, and (b) appreciate the extent to which medical practice is locally organized. Concepts from organizational literature, then, become useful in explaining the penetration of these medical communities and the circumstances under which a new modality takes hold in them. Within the framework presented, previously puzzling findings regarding variations in local practice and the poor relationship between practice behavior and the published literature become understandable.

Objectives: The goal of meeting the desire of older adults to remain independent in their home setting while controlling healthcare costs has led to the conceptualization of “smart homes.” A smart home is a residence equipped with technology that enhances safety of residents and monitors their health conditions. The study aim is to assess older adults’ perceptions of specific smart home technologies (i.e., a bed sensor, gait monitor, stove sensor, motion sensor, and video sensor).

Methods: The study setting is TigerPlace, a retirement community designed according to the Aging in Place model. Focus group sessions with fourteen residents were conducted to assess perceived advantages and concerns associated with specific applications, and preferences for recipients of sensor-generated information pertaining to residents’ activity levels, sleep patterns and potential emergencies. Sessions were audio-taped; tapes were transcribed, and a content analysis was performed.

Results: A total of fourteen older adults over the age of 65 participated in three focus group sessions Most applications were perceived as useful, and participants would agree to their installation in their own home. Preference for specific sensors related to sensors’ appearance and residents’ own level of frailty and perceived need. Specific concerns about privacy were raised.

Conclusions: The findings indicate an overall positive attitude toward sensor technologies for nonobtrusive monitoring. Researchers and practitioners are called upon to address ethical and technical challenges in this emerging domain.

Objective: Severe to profound hearing impairment affects one-half to three-quarters of a million Americans. To function in a hearing society, hearing-impaired persons require specialized educational, social services, and other resources. The primary purpose of this study is to provide a comprehensive, national, and recent estimate of the economic burden of hearing impairment.

Methods: We constructed a cohort-survival model to estimate the lifetime costs of hearing impairment. Data for the model were derived principally from the analyses of secondary data sources, including the National Health Interview Survey Hearing Loss and Disability Supplements (1990–91 and 1994–95), the Department of Education's National Longitudinal Transition Study (1987), and Gallaudet University's Annual Survey of Deaf and Hard of Hearing Youth (1997–98). These analyses were supplemented by a review of the literature and consultation with a four-member expert panel. Monte Carlo analysis was used for sensitivity testing.

Results: Severe to profound hearing loss is expected to cost society $297,000 over the lifetime of an individual. Most of these losses (67%) are due to reduced work productivity, although the use of special education resources among children contributes an additional 21%. Life time costs for those with prelingual onset exceed $1 million.

Conclusions: Results indicate that an additional $4.6 billion will be spent over the lifetime of persons who acquired their impairment in 1998. The particularly high costs associated with prelingual onset of severe to profound hearing impairment suggest interventions aimed at children, such as early identification and/or aggressive medical intervention, may have a substantial payback.

A health care program may influence both costs and health effects. We developed the Health and Labor Questionnaire (HLQ), which consists of four modules, to collect data on absence from work, reduced productivity, unpaid labor production, and labor-related problems. We applied the HLQ in several studies, and the results are encouraging.

Controlled clinical trials are recognized as the best source of data on the efficacy of health care interventions and technologies. Because economic evaluation is dependent on the quality of the underlying medical evidence, clinical trials have increasingly been viewed as a natural vehicle for economic analysis. However, the closer integration of economic and clinical research raises many methodological issues. This paper discusses these issues in trial design, collection of resource use data, collection of outcome data, and interpretation and extrapolation of results. Some guidelines are suggested for economic analysts wishing to undertake evaluations alongside clinical trials.