OP141 Expert Knowledge Elicitation in Health Technology Assessment: Our Experience Using the Sheffield Elicitation Framework

Abstract

Introduction

Expert judgement has an important role in health technology assessment (HTA), including as a source of evidence to inform economic modeling when published data are lacking. Quantitative information may be elicited from experts to inform model inputs and associated uncertainty using one of many expert elicitation methodologies. Here, the feasibility and potential benefits of one expert elicitation method, the Sheffield Elicitation Framework (SHELF), to the HTA process is examined.

Methods

The SHELF method seeks to express the knowledge of multiple experts in the form of a subjective probability distribution. Eliciting a subjective probability distribution allows the uncertainty of experts to be included in probabilistic sensitivity analysis, which is becoming an increasingly prominent feature of HTAs. The individual knowledge of participating experts is combined through behavioral aggregation, where experts participate in a discussion before being asked to provide judgments from the perspective of a rational impartial observer. The whole process is led by a facilitator who ensures all participants contribute and confirm that the final distribution is a product of consensus, not compromise.

Results

We recently conducted two SHELF elicitations as part of an ongoing project aiming to streamline the assessment of positron emission tomography (PET) in Australia. These elicitations provided insight into the usefulness of SHELF within the HTA setting. Given the constraints imposed by the COVID-19 pandemic, the elicitation sessions were conducted online rather than in the ideal face-to-face manner. In collaboration with one of the developers, we successfully adapted the method by making use of video conferencing technology to provide an online environment that mimicked the face-to-face setup as much as possible.

Conclusions

SHELF provides a rigorous and scientific method by which to elicit the knowledge of multiple experts in the form of a probability distribution. However, the method is resource intensive and may be best reserved for when data on key drivers are lacking.