OP22 Using Threshold Analysis To Guide Searches For Additional Sources Of Evidence

Abstract

Introduction

Threshold analysis is a novel statistical approach which can be used to investigate which direct comparisons in a network meta-analysis (NMA) have estimated relative effects that may not be robust to changes in the evidence, either due to possible bias, sampling variation, or relevance.

Methods

In a health technology assessment of the clinical effectiveness of ablative and non-invasive therapies for patients with early hepatocellular carcinoma (HCC), we conducted a threshold analysis to identify treatment comparisons that would be sensitive to changes in the randomized controlled trial (RCT) evidence used in the NMAs, potentially leading to a change in the recommended treatment. The results of the threshold analysis were used to guide a targeted systematic review of high-quality, non-randomized, prospective comparative studies that could strengthen the evidence for those comparisons identified as sensitive to change.

Results

We conducted NMAs of RCT evidence for four outcomes: overall survival (16 RCTs), progression-free survival (6 RCTs), overall recurrence (7 RCTs), and local recurrence (10 RCTs). The results of the NMAs displayed a high level of uncertainty, attributable to the sparse nature of the network, characterised by interventions being mainly compared in small trials. A targeted systematic review was conducted on relevant interventions that were identified as being sensitive to changes in evidence by the threshold analysis. The studies identified in this review were incorporated into a second NMA to support the RCT evidence.

Conclusions

Threshold analysis has been typically used as a tool to assess how robust comparisons in an NMA are to additional sources of evidence, but it can also be used to guide the search for additional non-randomized evidence when the available RCT evidence is sparse.