Healthcare decision makers are increasingly demanding that health technology assessment (HTA) is patient focused, and considers data about patients' perspectives on and experiences with health technologies in their everyday lives. Related data are typically generated through qualitative research, and in HTA the typical approach is to synthesize primary qualitative research through the conduct of qualitative evidence synthesis (QES). Abbreviated HTA timelines often do not allow for the full 6–12 months it may take to complete a QES, which has prompted the Canadian Agency for Drugs and Technologies in Health (CADTH) to explore the concept of “rapid qualitative evidence synthesis” (rQES). In this paper, we describe our experiences conducting three rQES at CADTH, and reflect on challenges faced, successes, and lessons learned. Given limited methodological guidance to guide this work, our aim is to provide insight for researchers who may contemplate rQES. We suggest several lessons, including strategies to iteratively develop research questions and search for eligible studies, use search of filters and limits, and use of a single reviewer experienced in qualitative research throughout the review process. We acknowledge that there is room for debate, though believe rQES is a laudable goal and that it is possible to produce a quality, relevant, and useful product, even under restricted timelines. That said, it is vital to recognize what is lost in the name of rapidity. We intend our paper to advance the necessary debate about when rQES may be appropriate, and not, and enable productive discussions around methodological development.

Objectives: Synthesizing qualitative research is an important means of ensuring the needs, preferences, and experiences of patients are taken into account by service providers and policy makers, but the range of methods available can appear confusing. This study presents the methods for synthesizing qualitative research most used in health research to-date and, specifically those with a potential role in health technology assessment.

Methods: To identify reviews conducted using the eight main methods for synthesizing qualitative studies, nine electronic databases were searched using key terms including meta-ethnography and synthesis. A summary table groups the identified reviews by their use of the eight methods, highlighting the methods used most generally and specifically in relation to health technology assessment topics.

Results: Although there is debate about how best to identify and quality appraise qualitative research for synthesis, 107 reviews were identified using one of the eight main methods. Four methods (meta-ethnography, meta-study, meta-summary, and thematic synthesis) have been most widely used and have a role within health technology assessment. Meta-ethnography is the leading method for synthesizing qualitative health research. Thematic synthesis is also useful for integrating qualitative and quantitative findings. Four other methods (critical interpretive synthesis, grounded theory synthesis, meta-interpretation, and cross-case analysis) have been under-used in health research and their potential in health technology assessments is currently under-developed.

Conclusions: Synthesizing individual qualitative studies has becoming increasingly common in recent years. Although this is still an emerging research discipline such an approach is one means of promoting the patient-centeredness of health technology assessments.

Objectives: The purpose of this review was to examine whether studies from the medical literature focusing on efficiency of diagnostic facilities reported economic evaluation methods appropriately, following guidelines for conducting and reporting economic evaluations.

Methods: A Medline search was conducted, and studies that concerned a diagnostic technology and fulfilled the Drummond criteria were selected for methodological review. The reliability of selection and methodological review based on the abstracts was determined by scoring a random sample of both abstracts and full articles. lnterrater reliability was determined by scoring a random sample of abstracts by both authors. Kappa values were calculated. Nine methodological aspects were reviewed: study design, the type of economic evaluation, the comparison made, the study's perspective, the cost-effectiveness ratio used, the definition of cost-effective, the types of costs analyzed, the cost calculation method, and the use of sensitivity analysis.

Results: Two hundred fifty studies published between 1992 and 1997 were found regarding efficiency of diagnosticfacilities; 134 studiesfulfilled the Drummond criteriaand were selected for methodological review. Kappavalues showed reliability of selection and methodological review and interrater reliability. The existing literatue on the economic evaluation of diagnostic facilities does not adhere well to guidelines for economic evaluation. In 95%, no perspective was mentioned, in 50% of the cases no ratio was given, in 82% the cost calculation method was not mentioned, and in 66% no sensitivity analysis was reported.

Conclusions: Our review suggests that to improve the quality of reporting economic evaluations, editorial boards could issue and enforce guidelines for standard reporting of such studies.

Objectives: The aim of this study was to examine the availability and consistency of economic evidence for the detection, diagnosis, treatment, and follow-up of colorectal cancer.

Methods: A systematic review of UK economic evaluations of colorectal cancer interventions was undertaken. Searches were undertaken across ten electronic databases. Studies were critically appraised through reference to a conceptual model of UK colorectal cancer services.

Results: Forty-seven studies met the inclusion criteria. There is a substantial economic evidence base surrounding population-level colorectal screening, surgical procedures, and cytotoxic therapies for the adjuvant and palliative treatment of colorectal cancer. There is limited evidence concerning the diagnosis of suspected colorectal cancer, curative treatments for metastatic disease and follow-up regimens for nonmetastatic disease. No studies were identified relating to the economics of radiotherapy, surveillance of increased-risk groups, end-of-life care, or the management of hereditary colorectal cancer. Where evidence is available, studies are subject to important differences concerning treatment options, decision criteria, and incongruent assumptions concerning the disease and its management.

Conclusions: Across many aspects of the colorectal cancer service, current practice appears to have emerged without the consideration or support of economic evidence. There is a need to develop a common understanding how colorectal cancer models should be structured and implemented.

Objectives: The aim of this study was to examine (i) the consistency of reporting research findings presented in conference abstracts and presentations and subsequent full publications, (ii) the ability to judge methodological quality of trials from conference abstracts and presentations, and (iii) the effect of inclusion or exclusion of data from these sources on the pooled effect estimates in a meta-analysis.

Methods: This report is a case study of a selected health technology assessment review (TAR) of a rapidly evolving technology that had identified and included a meta-analysis of trial data from conference abstracts and presentations.

Results: The overall quality of reporting in abstracts and presentations was poor, especially in abstracts. There was incomplete or inconsistent reporting of data in the abstract/presentations. Most often inconsistencies were between conference slide presentations and data reported in published full-text articles. Sensitivity analyses indicated that using data only from published papers would not have altered the direction of any of the results when compared with those using published and abstract data. However, the statistical significance of three of ten results would have changed. If conference abstracts and presentations were excluded from the early analysis, the direction of effect and statistical significance would have changed in one result. The overall conclusions of the original analysis would not have been altered.

Conclusions: There are inconsistencies in data presented as conference abstracts/presentations and those reported in subsequent published reports. These inconsistencies could impact the final assessment results. Data discrepancies identified across sources included in TARs should be highlighted and their impact assessed and discussed. Sensitivity analyses should be carried out with and without abstract/presentation data included in the analysis. Incomplete reporting in conference abstracts and presentations limits the ability of reviewers to assess confidently the methodological quality of trials.

Objectives: Current policy and practice regarding identification of and extent of use of data from conference abstracts in health technology assessment reviews (TARs) are examined.

Methods: The methods used were (i) survey of TAR groups to identify general policy and experience related to use of abstract data, and (ii) audit of TARs commissioned by the National Institute for Health and Clinical Excellence (NICE) and published between January 2000 and October 2004.

Results: Five of seven TAR groups reported a general policy that included searching for and including studies available as conference abstracts and presentations. A total of sixty-three published HTA reports for NICE were identified. Of these reports, thirty-eight identified at least one randomized controlled trial available as an abstract/presentation. Twenty-six (68 percent) of these thirty-eight TARs included studies available as abstracts.

Conclusions: There are variations in policy and practice across TAR groups regarding the searching for and inclusion of studies available as conference abstracts. There is a need for clarity and transparency for review teams regarding how abstract data are managed. If conference abstracts are to be included, reviewers need to allocate additional time for searching and managing data from these sources. Review teams should also be encouraged to state explicitly their search strategies for identifying conference abstracts, their methods for assessing these abstracts for inclusion and, where appropriate, how the data were used and their effect on the results.

Evidence-based nutrition is essential to move forward in the science of community nutrition. The present study is a review of the epidemiological evidence of dairy products and health. There is an inverse association between the intake of dairy products and hypertension, stroke and colorectal cancer. There is no evidence of an association between the consumption of dairy products and breast cancer. There is some evidence linking high-fat dairy products and an incremental risk of prostate cancer and weak evidence of the protective capacity of dairy products on bone health. More prospective studies should be developed in order to establish better evidence of the relationship between dairy products and health. Due to the importance of dairy products in public health nutrition, quantitative recommendations should be established in the light of the scientific evidence.

Objectives: This study aimed to investigate whether evidence-based clinical practice in health care improves outcomes for patients, personnel, and/or organizations.

Methods: A systematic review of studies was conducted with various quantitative and qualitative methods up to the Spring of 2002. Protocols were used in quality assessment. Data synthesis is descriptive in a narrative form.

Results: Of 305 assessed articles, eight studies were included. The outcomes in the included studies were related to the experiences of patients and personnel and to organization concerning changed patient care and resource utilization. Because the included studies are heterogeneous in design, focus of research area, and scientific quality, the scientific foundation for the findings is weak. There is some support that evidence-based clinical practice guidelines, when put to use, improve outcomes (i) for patients—less likelihood of showing worsening of skin condition and disruption of skin condition improves more rapidly for infants; (ii) for personnel—support in daily work situation; and (iii) for organizations—decreased admission rates and length of stay, less resource utilization and reduced costs.

Conclusions: There is a need for further research as the findings are based on a rather limited number of studies. There is a tendency toward support for the idea that outcomes improve for patients, personnel, or organizations if clinical practice in health care is evidence-based, that is, if evidence-based clinical practice guidelines are used, although these findings could be specific to the settings and context of the studies reported in this systematic review.

A systematic review and meta-analysis of randomized trials of nonpharmaceutical and nonhormonal interventions to reduce postnatal depression was carried out to summarize the effectiveness of interventions grouped in terms of the nature and timing of the intervention and whether the trial population was universal, selective, or indicated.

Objective: To evaluate the sensitivity and precision of various extended search methods in identifying randomized controlled trials (RCTs) for systematic reviews.

Method: Prospective analysis of extended search methods (specialized databases or trial registries, reference lists, hand-searching, personal communication, and Internet) used in two systematic reviews of RCTs. The gold standard was the total number of RCTs identified by major databases (MEDLINE, EMBASE, etc.) and extended search strategies combined. Sensitivity was the proportion of all known RCTs identified by any extended search method. Precision reflected the proportion of all items uncovered by any extended search method that actually were RCTs.

Results: The extended search identified 94 additional RCTs for the systematic reviews beyond those identified with the major databases. Specialized databases and trial registries had the highest sensitivity and precision for the lipid-lowering project (13.6% and 52.7%, respectively; p < .05) followed by scanning of reference lists (7.2% sensitivity and 41.9% precision; p < .05). Hand-searching was more effective than personal communication and Internet searching (1.7% sensitivity and 12.2% precision; p < .05). The acupuncture project had slightly different results, with the specialized databases and trial registries tied with the review of reference lists for highest sensitivity (14.2%). The precision followed the same trend as the lipid-lowering project (17.6% specialized databases; 8.3% reference lists; p < .05). A post-hoc analysis showed that 75 of the 94 RCTs were indexed in the major databases but missed by the major database search.

Conclusions: Extended searching identified additional RCTs for the systematic reviews beyond those found in major databases. Specialized databases and trial registries were most effective. An important number of RCTs were missed by the major database search. Timing and accuracy of indexing may explain this finding. The definitive measure, whether there is an association between the method used to uncover RCTs, the quality of the items uncovered and their impact on systematic review results, is yet to be determined.

Objectives: To review the scientific evidence on the efficacy, effectiveness, and safety of intra-articular injections of Hylan G-F 20 for the treatment of knee osteoarthritis.

Methods: Systematic review of experimental and observational studies performed in humans up to December 1999. Qualitative and quantitative (meta-analytic) techniques were used for data synthesis.

Results: A single course of intra-articular Hylan G{-}F 20 provides a statistically significant and clinically relevant short-term decrease of the painful symptomatology of knee osteoarthritis and improves joint function. It also seems to delay the need for knee replacement, if results observed in noncontrolled studies are confirmed. Hylan G-F 20 has a comparable efficacy to that of oral NSAID, and a smaller risk of gastrointestinal adverse effects. It seems to be well tolerated and safe, but the short follow-up in most studies limits any extrapolation of the effectiveness and safety over the longer term. There is also scarce evidence on the effect of multiple courses of Hylan G-F 20, and the scientific rigor of both experimental and nonexperimental studies reviewed is somewhat limited.

Conclusions: Whereas there is good quality scientific evidence showing that Hylan G-F 20 is a safe and well-tolerated therapy providing a short-term decrease of the pain symptoms while improving joint function, the delay of the need for knee replacement as well as the durability of the effect over the longer term have only been demonstrated in noncontrolled clinical series. The available evidence is not sufficient to reach firm conclusions on the effect of multiple courses of intra-articular injections of Hylan G-F 20 on health outcomes.

Objectives: The objectives of this paper were: a) to determine what can be learned from conclusions of systematic reviews about the evidence base of medicine; and b) to determine whether two readers draw similar conclusions from the same review, and whether these match the authors' conclusions.

Methods: Three methodologists (two per review) rated 160 Cochrane systematic reviews (issue 1, 1998) using pre-established conclusion categories. Disagreements were resolved by discussion to arrive at a consensual score for each review. Reviews' authors were asked to use the same categories to designate the intended conclusion. Interrater agreements were calculated.

Results: Interrater agreement between two readers was 0.68 and 0.72, and between readers and authors, 0.32. The largest categories assigned by methodologists were “positive effect” (22.5%), “insufficient evidence” (21.3%), and “evidence of no effect” (20.0%). The largest categories assigned by authors were “insufficient evidence” (32.4%), “possibly positive” (28.6%), and “positive effect” (26.7%).

Conclusions: The number of reviews indicating that the modern biomedical interventions show either no effect or insufficient evidence is surprisingly high. Intterrater disagreements suggest a surprising degree of subjective interpretation involved in systematic reviews. Where patterns of disagreement emerged between authors and readers, authors tended to be more optimistic in their conclusions than the readers. Policy implications are discussed.

Objectives: The aim was to determine which generations of the evolving technology of magnetic resonance angiography (MRA) are currently of clinical relevance in two clinical applications. Our purpose was to plan a systematic review that would be valuable both to purchasers driven by cost-effectiveness and to practicing clinicians.

Methods: Information was gathered from a search of major bibliographic databases, from a short questionnaire sent to 500 U.K. vascular radiologists and vascular surgeons, and from local clinical experts. We asked which of the MRA techniques were currently used and, assuming availability, what would be their technique of choice.

Results: There were 206 published articles that satisfied preliminary inclusion criteria: 69 discussed 2D time of flight (TOF); 47, 3D TOF; and 38, contrast-enhanced techniques. There were 162 questionnaires returned (60 radiologists, 102 surgeons). Of the total respondents, 77/162 (48%) used MRA in the assessment of carotid artery stenosis; 47/77 (61%) used 2D TOF; 32/77 (42%), 3D TOF; and 26/77 (34%), contrast-enhanced techniques. Thirty-five of 162 (22%) respondents used MRA in the assessment of peripheral vascular disease (PVD); 15/35 (43%) used 2D TOF, 4/35 (11%) used 3D TOF, and 22/35 (63%) used contrast-enhanced techniques. For those wishing to use MRA, contrast-enhanced techniques were the method of choice.

Conclusions: The TOF methods that represent earlier generations of the technology remain clinically relevant, and will therefore be included in our systematic review. To ensure complete and relevant coverage in reviews of other evolving technologies, it would be advisable to obtain data for guidance in a similar way.

Quality of life assessment as part of clinical practice in head and neck oncology began over 40 years ago. Early studies were narrative and cross-sectional; these were followed, at first, by simple quantitative measures of various parameters and later by longitudinal studies of greatercomplexity. More recently quality of life has been employed in a randomized clinical trial of head and neck cancer.

Quality of life has evolved to become a standard means of assessing clinical outcomes, and an accepted end point measurement in clinical trials, to be considered alongside survivorship and side effects/complications.

This study reviews 57 papers dealing with the issue of hearing preservation in vestibular schwannoma surgery published in otolaryngologic and neurosurgical literature between theyears 1977 and 1994. The authors', in this review, have made an attempt to verify whether the claims of hearing preservation are real, whether there is a price to be paid in terms of morbidity and whether there are univocal criteria for reporting results. The review shows that there is a wide disarray in reporting hearing results and the claims of hearing preservation are often unreal and misleading. On retabulating the results of a few series according to the minimal prerequisites for normal hearing (PTA ≤3O dB and SDS ≥70 per cent) and according to other various commonly reported criteria, it became evident that rates of hearing preservation differeda lot depending upon criteria. While any measurable hearing could be preserved in many cases, only a few had normal hearing preserved.