OP96 Assessing Impact Of UK Health Technology Assessment Programme Trials

Abstract

Introduction

Citation analysis is a standard tool for measuring the impact and influence of scientific work. One purpose behind controlled trials is to answer clinical and policy questions and to contribute directly or indirectly (contributing to systematic review and meta-analyses) to the production of practice guidance. The citation of trials within systematic reviews and policy or guidance documents therefore represents an authentic and meaningful measure of impact.

Methods

All 136 randomized controlled trials published by the United Kingdom (UK) Health Technology Assessment (HTA) programme in a 10-year period (2006-2015) were identified. Web of Science citation index was used to collect citation data relating to each trial. Altmetrics were used to identify additional policy and guidance documents. Citation data were collected and tabulated, and descriptive statistics produced. Additional data were collected for principal ‘spin-off’ publications.

Results

Eighty-eight percent of trials were cited by at least one Cochrane or non-Cochrane systematic review or meta-analysis; 37 percent by at least one Cochrane review (90 Cochrane reviews in total); 85 percent by at least one non-Cochrane systematic review or meta-analysis (365 in total). Forty-four percent of trials were cited by at least one unique piece of published policy or guidance. Mean number of review citations per published trial: 25.30; mean number of systematic reviews/meta-analyses per trial: 3.34; mean number of guidance documents per trial: 0.85. Trial investigators published the primary clinical outcome data in 27 additional peer-reviewed journal articles, generating citations in a further 66 unique reviews and 22 unique guidance documents.

Conclusions

Based on the payback model, this sample of 136 UK HTA trials represent meaningful impact: 88 percent of trials were cited in systematic reviews and 44 percent in guidance documents. Chronological data indicate that there might be a sizeable time-lag between publication and impact, especially for policy documents and Cochrane reviews.