Public health (PH) interventions are crucial for ensuring sustainable healthcare infrastructures. Nevertheless, they represent a neglected area in HTA due to various methodological issues and their complex design that goes beyond clinical setting. Our study provides an environmental scan of HTA initiatives related to the assessment of PH technologies on a global level.

The Initiative for Public Health Outcomes Research and Measurement (INPHORM) interest group has conducted a survey among European and international societies, health bodies and networks during September 2018. The questionnaire evaluates what kind of PH technologies and/or interventions have been evaluated in the last five years, or are planned for the future.

Our preliminary findings from November 2018 indicate a total of 94 initiated and 44 completed surveys. Among the completed ones, the majority of respondents came from European countries (36%), followed by North (30%) and South America (16%) countries. Sixty-eight percent of institutions reported engagement in any aspect of HTA in the area of PH (N = 30). Medical aspects of the PH technology are considered by 83 percent of the institutions, followed by organizational impact (67%), economic evaluation (60%) and societal consequences (60%). An average of four PH technologies has been evaluated by the responding institutions in the last five years. In reference to methodological aspects, 90 percent of institutions used a classical HTA approach for evaluating PH interventions, while 40 percent used budget impact analyses. Among the barriers for reaching a decision, conflicting stakeholder priorities, lack of data and clear methodological frameworks were most commonly cited.

Data analysis is currently on-going and final results will be presented during the Cologne meeting. This study will allow to raise awareness about the importance of PH interventions in HTA, identify existing gaps and propose future methodological developments.

Medical devices (MDs) differ regarding their risk class (I to III), operational area (in-, outpatient), intended use (diagnostic, monitoring, intervention), and with regard to available clinical evidence. Therefore, the market access processes as well as the approach of gaining reimbursement differ significantly. From a variety of potential approaches the underlying analysis illustrates five MD-specific processes.

Based on a systematic search of publicly available regulations the main pathways of potential reimbursement for MDs were evaluated.

MDs to be used in the in-patient setting can be divided into three categories: an innovative MD (a) is exceeding a current reimbursement framework (German Operations and Procedures Key (OPS) / diagnosis related groups (DRG)), (b) falls within an existing reimbursement rate, or (c) the MD is based on a known mode of action (MoA) for which already adequate reimbursement exists. Due to less empirical data from MDs for a) and b), a health technology assessment (HTA) is required before inclusion in a DRG, whereas a MD with known concept (c) will be grouped into existing price structures. Initiators of these processes are hospitals through a so-called NUB application. MDs entering the outpatient sector are covered by another reimbursement catalogue (EBM/GOÄ) and have to pass an assessment by the G-BA (rapid HTA) if based on new MoA (d). Such an assessment can only be initiated by respective umbrella organizations of service providers (e.g. KBV (National Association of Statutory Health Insurance Physicians)). MDs not being positively recommended by the G-BA are not reimbursable. For MDs (e) with known MoA no HTA is required.

For a successful market launch including sufficient reimbursement not only the market potential, but also the specific regulatory pathways have to be considered carefully. New and innovative MDs in the outpatient sector may have a longer application process to gain a positive reimbursement decision than MDs used in inpatient setting.

Telerehabilitation shows promise in many fields though strong evidence of benefit has been limited. We reviewed progress in the use of telerehabilitation for persons with chronic obstructive pulmonary disease (COPD). A challenge in caring for persons with this condition is the ability to achieve high levels of patient participation and compliance with rehabilitation processes.

Relevant publications were identified through literature searches from November 2009 to May 2018. We selected those that described studies of telerehabilitation in the management of COPD and reported clinical or administrative outcomes. Study quality was assessed using an approach that considers both study performance and study design. Judgments were made on whether the telerehabilitation application had been successful, if reported outcomes were clinically significant, and if further data were needed to establish the application as suitable for routine use.

Twenty-five publications, on 26 studies, were selected. Twelve were of high or good quality. In 11 studies the telemedicine application was successful. Nine studies had unsuccessful applications, and for six studies success was unclear. Further data before routine use would be required or desirable for all successful applications. In many studies there were difficulties associated with availability of skilled mentors, motivational support for patients and access to reliable remote monitoring and communication technology.

Various types of telerehabilitation are potentially helpful in the management of COPD. Availability and access to these technologies should improve. However, in management of this clinically challenging condition their use must be linked to suitable training and education of patients with COPD and timely support for them from healthcare professionals.

Cellphones can be used to support treatment and disseminate health information. Literature has shown an unmet need for information for people living with human immunodeficiency virus (PLHIV) and others affected by the epidemic. The World Health Organization (WHO) emphasizes the incorporation of cellphones as a tool to support HIV adherence and information dissemination. We sought to assess rates of utilization of health information provided through the Call for Life Uganda (CFLU) platform among HIV-positive individuals.

CFLU uses the Mobile Technology for Community Health (MoTeCH) software Call for Life™ developed by Janssen and adapted to the Uganda setting in collaboration with Infectious Diseases Institute (IDI). It offers daily pill reminder calls/sms, health info tips; symptom reporting and clinic appointment reminders. CFLU was used in a randomized control trial (RCT) undertaken to improve outcomes in HIV patients providing information categorized into Antiretroviral therapy (ART) and adherence, positive living, general health, pregnancy, breast-feeding, and sexuality. We used data from the RCT between August 2016 to June 2018 to generate frequency distributions and gender differences regarding utilization of health information.

From a total of 300 respondents receiving the CFLU intervention, a majority were: females (70%), aged 16 to 35 years (62%), married (74.7%), had attained secondary and higher education (57.3%); and employed (67.7%). Overall, 255/300 (85%) utilized at least one of the health-tips categories. Participants utilized mostly general health information 211/300 (70%); followed ARTs and adherence 173/300 (57.7%); pregnancy and breast-feeding 137/300 (45.7%), sexuality 113/300 (37.7%), and positive living 98/300 (32.7%). Gender differences were noted regarding ARTs and adherence utilization with higher percentage of females to males (61% vs 50%) and for sexuality, a higher percentage of males to females (41.6% vs 33.3%, p < 0.05).

The findings indicate that when availed with platforms for health-related information, PLHIV populations will utilize them mostly for adherence. We recommend increased incorporation of such technologies to disseminate information in this key population.

ABC-VEN analysis is an easy method of clinical and economic analysis on the costs of drug coverage and an important tool for monitoring and ensuring the rational use of medicines. However, this methodology is difficult to apply in assessing the viability of medical equipment procurement (MEP) in hospital. Using a combined model of ABC analysis and Multiple criteria decision analysis (MCDA) may be more appropriate to apply to MEP.

We created five standardized criteria, which present the main results of assessment of the viability of MEP for implementing new health technologies (HTs). These criteria address the following: 1) Novelty/innovation; 2) Comparative clinical effectiveness and safety; 3) Relevance (demand); 4) Economic effectiveness; and 5) Payback period. Based on these criteria we determine the threshold values of priority for MEP: 1) High priority; 2) Medium priority; 3) Low priority.

Using the ABC model and five standardized criteria, we analyzed all proposals from the Hospital units for implementing new HTs connected with MEP for 2018. In total, proposals contained 11 items of ME, among them three items were in group A (27%), two items were in group B (18%), and six items were in group C (55%). All items were high priority for procurement with the exception of one item from group B with medium priority. Items with low priority were not revealed which can be considered as a direct indicator of the operational effectiveness of Hospital-based HTA Unit. Excluding ME with a medium priority from the procurement plan would reduce Hospital costs by 13.5 percent.

Combined ABC and MCDA analysis in the process of assessment the viability of MEP can give the opportunity to make comparative assessment of different types of ME based on standardized criteria; determine the priority for procurement of new ME; and avoid the influence of subjective factors of the managerial decision-making process in hospital.

Antibiotics impregnated calcium sulfate (AI-CaSO4) is an innovative practice to ensure local diffusion of antibiotics especially in the treatment of prosthesis or medical implants infections. A recent introduction of AI-CaSO4 at CHU de Québec-Université Laval (CHU de Québec) was followed by a rapid increase in use and costs. A hospital-based health technology assessment (HTA) was then requested to assess the clinical relevance of AI-CaSO4 in surgical site infection (SSI) management.

A systematic review of the effectiveness and adverse effects of AI-CaSO4 was performed in indexed databases and grey literature. The local context analysis included different methodologies: 1) interviews with pharmacists, surgeons and operating room managers, 2) data extraction from electronic patient records (EPR), 3) procurement database on CaSO4, and 4) interdisciplinary working group including orthopedic and vascular surgeons, pharmacists, infectiologists, and hospital managers.

Available evidence suggest that AI-CaSO4 could contribute in the treatment of osteomyelitis whereas no conclusion can be drawn for other medical indications in both treatment and prevention of SSI. A review of 113 surgical procedures showed that AI-CaSO4 was rapidly adopted after only one year and used for various medical indications in neuromodulation, orthopedic and vascular surgery. Osteomyelitis treatment accounted for less than 3% of cases. AI-CaSO4 was mainly used in prevention of SSI (65%) and surgical revisions (74%). Furthermore, local safety issues were raised by a lack of standardization for the preparation and under recording of antibiotics use with AI-CaSO4.

The current state of knowledge does not support the widespread use AI-CaSO4 at CHU de Québec. This study highlights the importance of adapting HTA approach to the local context to influence decision-making especially in the context of innovating practice in order to insure the relevance, safety and sustainability of care.

Ultra-orphan therapies (prevalence: <1:50,000) can have trouble meeting Health Technology Assessment (HTA) clinical- and cost-effectiveness criteria, set by HTA bodies to inform reimbursement decision-making, due to low patient numbers limiting the supporting clinical evidence generated and high per-patient prices. Since 2013, National Institute of Health and Care Excellence (NICE) appraise Highly Specialised Technologies (HST) (“for use in the provision of services for rare and very rare conditions”) using a distinct appraisal framework. This research compares NICE HST appraisal outcomes with corresponding guidance by other HTA bodies.

All NICE HST technology guidance was screened (1 January 2013–6 November 2018) alongside corresponding guidance by Gemeinsamer Bundesausschuss (G-BA), Haute Autorité de Santé (HAS), Scottish Medicines Consortium (SMC), and National Centre for Pharmacoeconomics (NCPE).

NICE have published eight HST guidances all with positive recommendations after a median of 21 months (range: 7–38) after European Marketing Authorization (MA). An additional eight HST have guidance in-development despite having European MA for a median of 12 months (range: 2–46) with 5/8 having draft guidance issued, all being “not recommended”. Of the 18 HSTs with NICE guidance published/in-development, 29 percent (2/7), and 33 percent (2/6) have been assessed with positive outcomes (definition: “recommended”/”accepted”/“conditional”/”restricted”) by SMC, and NCPE, respectively vs. 100 percent (9/9) by G-BA (definition: any additional benefit), and 50 percent (5/10) by HAS (definition: ASMR I-III). Median delays between European MA and positive appraisal outcomes were seven (G-BA), nine (HAS), 12 (NCPE), and 19.5 months (SMC).

Although all NICE HST final guidances to date have been positive, few technologies have completed this process after substantial delays from MA. Other cost/QALY HTA bodies (i.e. excluding the G-BA and HAS clinical-assessment HTAs) have shown low appraisal and recommendation rates for these technologies; therefore, ultra-orphan technologies may require a distinct appraisal process/framework but the HST may not (yet) represent best-practice.

The National Institute for Health and Care Excellence (NICE) in England has a separate appraisal process for drugs for very rare conditions, i.e. Highly Specialised Therapies (HST). In April 2017, the HST process has been changed to incorporate a quantitative approach: automatically fund treatments with incremental cost-effectiveness ratio (ICERs) up to GBP 100,000 (EUR 113,008 based on the 2018 average GBP / EUR exchange rate) per quality-adjusted life year (QALY). For treatments with an ICER above GBP 100,000 per QALY, NICE will consider treatments that offer a substantial magnitude of improvement, with additional QALY weighting. We investigated the impact of this more quantitative approach on the likelihood of a HST receiving a positive recommendation.

All HST appraisals and draft guidance documents were reviewed (up to November 2018) and data were extracted on ICERs, incremental QALY gain, budget impact, and recommendations. The extracted data from each HST were assessed based on the interim HST guidance.

Eighteen products have been or are currently going through the NICE HST process. Of these, 8/18 (44%) have received a positive recommendation, while 5/18 (28%) have received a draft negative guidance, and for 5/18 (28%) products, no recommendations have been published. For the products with a positive outcome, 5/8 (63%) had incremental QALY gain of at least 10, qualifying these products for additional QALY weighting. For the products that received a draft negative recommendation, the negative decision was related to the cost-effectiveness estimates being higher than GBP 100,000 per QALY (5/5 reported) in all cases, while none of these products were eligible to receive a ‘QALY modifier’.

It has become more difficult for HSTs to get recommended by NICE under the new guidance, which requires cost-effectiveness analyses, whereas previously there was no official ICER threshold. The additional weighting of QALYs may be insufficient to meet an ICER threshold of GBP 100,000 per QALY for many products.

Written evidence is submitted to the National Institute for Heath and Care Excellence (NICE) by patient organisations for all ultra-orphan evaluations. To enhance the capability of patient involvement at NICE and to further develop understanding of how patient generated evidence and input in ultra-orphan conditions can support the Health Technology Assessment (HTA) agencies beyond 2020, the Public Involvement Programme systematically reviews the impact the evidence has on committee decision making.

This study captured data from September 2017 to August 2018 for seven ultra-orphan evaluations. A paper questionnaire was given to each committee member to complete for each evaluation and entered in to an online system for analysis. Findings were used to inform the committee views which were highlighted in feedback letters to the patient groups. The questions included: how much impact and what sort of impact the patients had; both qualitative and quantitative data; and, a specific question on clarification of quality of life data

We obtained 83 responses showing the submissions: had a moderately high or high impact; gave the committee particular insight into quality of life data not provided elsewhere; provided new evidence; interpret the data from other sources; and, demonstrated consistency with other sources

Patient evidence is particularly useful for ultra-orphan conditions where other forms of evidence are limited. Patients can provide a unique insight into the burden of disease, the patient population, any updates of treatments and the impact on patient and carers. They provide real life data to the committee including information that standard Quality Adjusted Life Years measures do not. Evidence varied by condition with impact themes highlighting the effects on patient and carers including fear, stress and anxiety. The examples are recorded, updated annually and will be shared with national patient groups and offered internationally through the HTAi Interest Group on Patient and Citizen Involvement.

The Scottish Medicines Consortium (SMC) advises NHS Scotland on the clinical and cost-effectiveness of new medicines. Since 2014, evidence from patients and carers on end-of-life and orphan medicines has been gathered during Patient and Clinician Engagement (PACE) meetings. The output is a consensus statement which describes the added value of a new medicine from the perspective of the patient/carer and clinician. This study investigates the importance of factors identified through PACE to committee members and how these are used in their decision-making.

Survey methodology was used to gain an understanding of the factors from the PACE statement that are most likely to influence members (n = 26) in decision-making. The survey instrument was informed by a literature review and observation of PACE and SMC meetings. Likert scale questions were used to determine the relative importance of factors in the PACE statement, including information relating to eight prominent ‘quality of life’ themes (family/carer impact, health benefits, tolerability, psychological benefit, hope, normal life, treatment choice and convenience), that were identified by an earlier thematic analysis of these statements.

Analysis of survey responses will use mainly descriptive techniques to generate percentages and ranges. Correlation analysis will be considered to investigate relationships between members’ demographics, type of medicine (end-of-life, orphan) and the importance of different factors in the PACE statement. Preliminary results indicate that key quality of life themes highly valued by patients/carers are also important to committee members in their decision making. Challenges in assimilating qualitative patient-based evidence from PACE alongside quantitative clinical and economic data were highlighted.

Findings from this survey will provide valuable insight into how PACE evidence is used by SMC decision makers alongside traditional clinical and economic evidence and will help shape future improvements to the PACE methodology.

Some systematic review topics are context-sensitive and informed by evidence about a specific geographic setting. Smart information retrieval methods are required to identify such evidence in an efficient manner. This presentation will discuss how validated geographic search filters enable smart literature searching for context-sensitive reviews using the National Institute for Health and Care Excellence (NICE) United Kingdom (UK) filters for MEDLINE and Embase (OVID) as examples. The NICE UK filters were developed in 2016. The filters demonstrated high recall and high precision, however, further research was required to confirm these results.

In 2018, the filters’ recall of references from 100 UK-based multidisciplinary reviews was calculated. Reproducible search strategies were identified from twenty-six of the 100 reviews in MEDLINE and from nine reviews in Embase. From this, the precision and number-needed-to-read (NNR) were calculated.

The MEDLINE filter achieved 96 percent recall (1401 out of 1454 UK references), 2.1 percent precision and a NNR of forty-seven. The Embase filter achieved 97 percent recall (1520 out of 1560 UK references), 0.7 percent precision and a NNR of 146. Compared to not using a filter, the MEDLINE and Embase filters reduced the number of search results by an average of 87 percent and 80 percent, respectively.

The filters retrieve the majority of evidence for UK topics while reducing search result volumes and so enable smart literature searching for context-sensitive topics. Large literature search result volumes can increase development time-frames for systematic reviews. Using the filters can therefore save time for reviews with a UK focus. There are currently two other validated geographic search filters for Africa and Spain. It is hoped that the NICE UK filters’ successful retrieval performance will encourage the development of validated search filters for more geographic regions.

A key discussion point during HTAi's 2018 Meeting was how Health Technology Assessment (HTA) practitioners might borrow ideas from other industries or academic areas. Organisational learning (OL) is the study of how individual knowledge is shared within an organisation to become institutional/group knowledge. There are several models of OL, all focusing on how tacit knowledge (abstract, personalised, hard to define, action-based) is converted to explicit knowledge (definable, concrete, fixed, information-based). Effective knowledge sharing is crucial to leveraging individual knowledge to drive innovation, efficiency and effectiveness. Information retrieval is a knowledge-intensive field, with many processes requiring both tacit and explicit knowledge. Ideas from OL demonstrate ways to improve practice by increasing knowledge sharing.

Nonaka & Takeuchi's (1994) SECI model describes the cyclical process by which knowledge is shared. The model includes 4 stages: socialisation (tacit-to-tacit), externalisation (tacit-to-explicit), combination (explicit-to-explicit) and internalisation (explicit-to-tacit). Each stage describes how knowledge sharing takes place and highlights ways to ameliorate these processes. Information retrieval involves many elements that require or benefit from knowledge sharing and both tacit and explicit knowledge is required.

In the SECI model the Socialisation stage is characterised by face-to-face learning. Peer reviewing of search strategies, open dialogue and team working are ways of facilitating this stage. The Externalisation stage is crucial to OL. This can be seen as the practice-into-research stage; the results of successful experimentation, for example with search filters. The Combination stage is the easiest to understand. Communities of practice and inter-organisational networks can widen knowledge sharing and help refine or increase detail of best practice. The Internalisation stage is the hardest to conceptualise or measure. The extent to which guidelines become adopted in individual practice is one way to gauge Internalisation.

Information retrieval practitioners could benefit from thinking about ways to improve knowledge sharing. Models of OL can be instructive in this regard.

It is a challenge to stay up-to-date with the latest developments in information retrieval for health technology assessment (HTA). Summarized Research in Information Retrieval for HTA (SuRe Info) is a well-established open-access website with a selection of up-to-date key papers presented in summarized overviews. SuRe Info is maintained by the HTAi Interest Sub-Group on Information Resources; its main target group are information specialists. SuRe Info is updated twice a year by experienced information specialists. Publications on information retrieval methods are identified by running topic-specific search strategies in relevant databases. Publications fulfilling the SuRe Info inclusion criteria receive a structured abstract together with a brief critical appraisal prepared by one SuRe Info information specialist and checked by another. The key messages from the appraisals are summarized in topic-specific chapters.

The structure and work of SuRe Info is presented on the basis of the chapter “Value of using different search approaches”.

The chapter “Value of using different search approaches” was last updated in September 2018. It examines various alternative search approaches in bibliographic databases beyond the conventional Boolean search. According to this chapter, the most well-known approach, offering the most evidence, is citation searching (direct or indirect citation relationships). In contrast, little research has so far been conducted on other approaches such as full-text searches, automated retrieval methods or hand searching.

SuRe Info is an important resource for information specialists to keep up-to-date with the literature on information retrieval. In particular when information specialists work alone and not within a larger team, it is necessary to rely on collaboration projects such as SuRe Info.

When developing a health technology that requires clinical studies, developers institute working relations with clinical investigators. In certain diseases areas, patients' representatives create their own advisory boards, which proved their utility in the early 90s, in particular for the development of products to treat HIV infection. Inspired by this model, where patients with a same disease join and meet with relevant developers and discuss all aspects of the research, the European Organisation for Rare Diseases (EURORDIS) proposes a new programme of such Community Advisory Boards for Rare Diseases (CAB).

For this programme, EURORDIS invites developers to sign a Charter of principles when engaging with patients, and provides guidelines on CABs, together with a mentoring programme for patients’ networks that are less experienced with the development and the evaluation of health technologies. CABs are driven by patients who set their agenda, who sign a Memorandum of Understanding with each developer, and who organise the sessions. Sessions typically last for two to four days during which different meetings with different developers can take place, or trainings. All meetings can take place under confidentiality arrangements, and minutes are written to keep track and to follow-up with all points discussed. Participants and agendas are made public

As of 2018, four CABs exist and operate (for tuberous sclerosis complex, for scleroderma, for cystic fibrosis, for Duchenne muscular dystrophy) and 18 others are in discussion with many due to start in 2019. Topics discussed cover the target population, the study feasibility, the endpoints including patient reported outcomes, the comparator choice and/or the acceptance of a placebo controlled trial, the quality of life, the practical aspects of the trials, and the identification of previously unknown or unmet patient needs/preferences. For products which are more advanced in their life-cycle, discussions can also cover compassionate use, pricing policy, relative efficacy etc.

This represents a well-structured programme for the engagement of patients, where collective thinking and exchange between different patients ensure high quality dialogue with developers and can inform HTA also.

AQuAS is gradually incorporating patient involvement in health technology assessment (HTA). We present two experiences conducted during 2018 and the different methods and approaches used. The aim is to reflect on learnings from those experiences to improve ways for increasing patient involvement with HTA at AQuAS.

We conducted two experiences using different quantitative and qualitative techniques (mix-methods approach). The first, a focus group discussion regarding the use of 3D-technologies for maxillofacial reconstruction with a selection of hospital patients that received maxillofacial reconstruction, which included the use of a quality-of-life retrospective self-assessment tool. The second, a sequence of email correspondence regarding cataract surgery technology (femtosecond laser-assisted cataract surgery (FLACS), in the context of an EUnetHTA Joint Assessment) with a patients’ association representative, to learn their opinion regarding the use of laser technology.

Main learnings were: (1) patients and associations have different levels of knowledge and expertise to consider when planning an HTA; (2) sharing experience and knowledge among peers (i.e. focus group) proved to have a positive impact on patients and worked as a resource for some to improve knowledge on their condition; (3) critiques were received regarding a specific Patient Reported Experience template for HTA patient involvement (iv) quality of life retrospective self-assessment tool provided unexpected positive and negative results.

Recent patient involvement in HTA at AQuAS proved to be useful to complement HTA reports. We believe that patients’ experiences and opinions can help decision-making regarding the use, disinvestment or incorporation of health technologies, contribute to the improvement of tools and survey accuracy and improve doctor-patient communication. Their involvement might be beneficial for them to gain more knowledge, share experiences, reflect on their health situation and improve communication with the professionals that treat them. Future patient-involvement in HTA is needed, e.g. considering the social perspective in economic evaluations or including patients in recommendation consensus panels.

The Patient Experience, Expectations and Knowledge (PEEK) protocol was developed so that a holistic, comprehensive, independent, proactive and systematic approach could be taken to inform decisions made in the context of health technology assessment and other parts of the health sector. Each PEEK study is made publicly available which over time will result in a global repository of patient experience data.

The PEEK protocol is a single protocol that can be implemented across disease areas and includes a quantitative and qualitative component. The quantitative component is based on a series of validated tools that provide baseline health and demographic data for the study population. The qualitative component is the result of two years of protocol testing to develop a structured interview that solicits comprehensive and holistic patient experience data, and provides participants with the opportunity to provide advice on their future expectations.

PEEK studies in breast cancer, bladder cancer, lung cancer, spinal muscular atrophy, atopic dermatitis, chronic kidney disease, chronic heart failure and mitochondrial disease have been completed in the Australian context (www.cc-dr.org/peek). Holistic patient experience themes are presented commencing with symptoms and diagnosis experience, through to communication, information, treatments experienced and quality of life. Information is also available in relation to participant's expectations of future treatment, care, information and communication. The result is a freely available repository of patient experience data that anyone in the sector can access to complement clinical and economic evidence.

The process of providing patient feedback and real-world evidence in the context of health technology assessment is often ad-hoc. The lack of consistency means that it has been difficult to assess the impact of patient engagement and feedback in the context of health technology assessment. The PEEK protocol and program is an example of a systematic, independent and holistic approach to patient experience and real-world evidence data collection that provides the sector with an opportunity to proactively engage the community in decisions that are made about treatment, care and support.

Patient-reported outcomes (PRO) data are important in understanding patients’ experience of disease and treatment; however, PRO data are not universally collected or consistently included as part of a Health Technology Assessment (HTA) submission. Additionally, the HTA bodies’ response to PRO data vary, making the impact unclear. To understand the impact of PRO data on reimbursement decisions for Acute Myeloid Leukemia (AML) indications, an in-depth analysis of HTA bodies’ appraisals of AML and analogous indications was conducted.

This analysis was conducted using IQVIA's HTA Accelerator, which contains HTA appraisals from ≥100 HTA agencies in thirty-nine countries. Included in the analysis were single-technology assessments (original submissions, resubmissions, extensions of original indications, and renewals); relevant regulatory approvals and pivotal trials were also analyzed.

Of the 185 AML appraisals from sixteen HTA bodies, 66 (36%) included PRO data. Within these, thirteen different PRO instruments were identified, none of which have been validated in patients with AML. For seven of twenty in-scope products, PRO evidence positively impacted ≥1 of the HTA decisions. Although the same HTA bodies (i.e., Scottish Medicines Consortium, pan-Canadian Oncology Drug Review, and the National Institute of Health and Care Excellence) generally accepted the PRO evidence, others were critical of the evidence (i.e., Haute Autorité de Santé and the Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen). The most common concerns raised by the HTA bodies regarding the PRO evidence included trial design and low patient response rate.

Of the products that included PRO evidence in their HTA submissions, 35% received positive feedback from ≥1 HTA body on their submitted PRO evidence. Attention to PRO data collection is key to demonstrate the value of AML products to HTA bodies. Without these data, a clear gap in the understanding of patients’ experience is evident.

Hospital-based health technology assessment (HB-HTA) needs to consider all relevant data to help decision-making, including patients’ preferences. In this study, we comprehensively describe the process of identification, refinement and selection of attributes and levels for a discrete choice experiment (DCE).

A mixed-methods design was used to identify attributes and levels explaining low back pain (LBP) patients’ choice for a non-surgical treatment. This design combined a systematic literature review with a patients’ focus group, one-on-one interactions with experts and patients, and discussions with stakeholder committee members. Following the patient's focus group, ranking exercises were conducted. A consensus about the attributes and levels was researched during discussions with committee members.

The literature review yielded 40 attributes to consider in patients’ treatment choice. During the focus group, one additional attribute emerged. The ranking exercises allowed selecting eight attributes for the DCE. These eight attributes and their levels were discussed and validated by the committee members who helped reframe two levels in one of the attributes and delete one attribute. The final seven attributes were: treatment modality, pain reduction, onset of treatment efficacy, duration of efficacy, difficulty in daily living activities, sleep problem, and knowledge about their body and pain.

This study is one of the few to comprehensively describe the selection process of attributes and levels for a DCE. This may help ensure transparency and judge the quality of the decision-making process. In the context of a HB-HTA unit, this strengthens the legitimacy to perform a DCE to better inform decision-makers in a patient-centered care approach.

Controversy regarding the efficacy of transcatheter mitral valve repair with a clip (TMVRc) in reducing the mitral regurgitation is related to the lack of solid scientific evidence. Worldwide, refusal or conditional acceptance for implementation of TMVRc, reflect ongoing uncertainty. We sought to apply a systematic multicriteria framework to ensure a fair and reasonable decision regarding the use of TMVRc in Quebec.

The framework included the following domains: context, quality of evidence concerning safety, efficacy and effectiveness, unmet patient needs, expected volume of patients, and impact on the health system including costs. Each domain within the framework was examined by a review of the literature and through consultations with a scientific advisory committee, a TMVRc clinical expert committee, TMVRc clinical teams, industry representatives and the Institut national d'excellence en santé et en services sociaux (INESSS) clinical excellence committee.

The literature review indicated that uncertainty about the efficacy and effectiveness of TMVRc persists, particularly in the real world context, and this view was supported by scientific experts. The TMVRc clinical teams provided insight into the burden of mitral insufficiency on patients and the health system and their belief in the promise of TMVRc. They also highlighted the challenges of patient selection and organizational issues related to the introduction of TMVRc within their institutions. The advisory committee stressed the need for further evaluation prior to wide diffusion.

Using a multicriteria framework facilitated a more standardized and transparent approach to our literature review and consultations as well as to the development of the proposed recommendations. This was especially important in the context of an evaluation of a promising new approach to treat mitral valve disease with many important uncertainties. This multi-criteria approach will facilitate a more standardized process for deliberation on how new health technologies should be implemented into the Quebec health system.