The decision-making (DM) process in public administration is the subject of research from different perspectives and disciplines. Evidence-based policies, such as health technology assessment (HTA), are not the only support on which public policies are designed. During the COVID-19 pandemic WHO, national and subnational institutions developed HTA reports to guide DM. Despite this, inadequate variability was observed in the health technologies recommended and reimbursed by different provincial Health Ministries in a federally organized developing country like Argentina. The processes and results of DM on health technologies for COVID-19 in Health Ministries of Argentina were inquired.

A retrospective research design was developed, with triangulation of quantitative and qualitative methods. We retrieved information for the years 2020–2021 through document review of official webpages, surveys, and interviews with decision-makers of the 25 Argentinian Ministries of Health. We analyzed the recommendations and reimbursement policies of seven health technologies.

In contradiction with WHO’s policies, ivermectine, inhaled ibuprofen, convalescent plasma and equine serum were widely recommended by most of Argentina’s health ministries outside a clinical trial context, with risks for patients and a huge opportunity cost.

Despite an important HTA institutional capacity, the impact of HTA organizations and their technical reports was limited. Health Ministries with institutionalized HTA units had more adherence to WHO recommendations, but the influence of different technical and political criteria was identified. Power relations within and outside the administration, the pharmaceutical industry and academics, the media, social pressure, the judicial and legislative powers, and the political context strongly influenced DM.

The sandbox approach, developed in the financial technologies sector, creates an environment to collaboratively develop and test innovative new products, methods and regulatory approaches, separated from business as usual. It has been used in health care to encourage innovation in response to emerging challenges, but, until recently, has not been used in health technology assessment (HTA). This article summarizes our learnings from using the sandbox approach to address three challenges facing HTA organizations and to identify implications for the use of this approach in HTA.

We identified three challenging contemporary HTA-related topics to explore in a sandbox environment, away from the pressures and interests of “live” assessments. We convened a pool of 120 stakeholders and experts to participate in various sandbox activities and ultimately co-develop solutions to help HTA organizations respond to the identified challenges.

Important general learnings about the potential benefits and implementation of a sandbox approach in HTA were identified. Consequently, we developed recommendations to guide its use, including how to implement an HTA sandbox in an effective way and the types of challenges for which it may be best suited.

For many HTA organizations, it is difficult to carefully consider emerging challenges and innovate their processes due to risks associated with decision errors and resource limitations. The sandbox approach could reduce these barriers. The potential benefits of addressing HTA challenges in a collaborative “safe space” are considerable.

The costs of low back pain (LBP) are complex and difficult to estimate. This study aims to adapt the Cost for Patients Questionnaire (CoPaQ) for use in LBP populations.

In a cross-sectional qualitative study, we conducted cognitive interviews to assess the CoPaQ’s suitability for addressing costs related to LBP. Three groups of participants were included (n = 5 each): (i) persons with a history of LBP or primary caregiver, (ii) researchers with expertise in LBP, and (iii) primary care providers specialized in treating LBP. The interpretation, analysis, and summary of results used Knafl et al.’s qualitative content analysis method.

Persons with a history of LBP (n = 5), had a median age of 60 years (Interquartile Range (IQR): 26–71.5), and varying durations of LBP, the median duration of LBP 7 years (IQR: 4–32.5). Researchers (n = 5) had a median age of 33 years (IQR: 29–45). Primary care providers (n = 5) had a median age of 40 years (IQR: 37.5–65), and a background in chiropractic care (n = 3) and physiotherapy (n = 2). Content analysis of the interviews revealed sources of error with five pre-determined themes (clarity/comprehension, relevance, inadequate response definition, reference point, perspective modifiers) and one developed theme (organization). We modified the questionnaire for LBP populations based on the feedback.

Our study evaluated the content validity of a questionnaire that assesses the direct and indirect costs associated with LBP. Future studies should pilot this questionnaire with persons of varying LBP severity and compare it with cost diaries.

We draw from the Health Technology Assessment (HTA) literature to propose how hospitals and local health networks can prepare the key components of early economic evaluations to support the development and management of health service interventions.

Using the case example of a proposed intervention for older people in the Emergency Department (ED), a conceptual logic model of a new health service intervention is articulated to inform the structuring and population of a decision-analytic model using observed data on the existing care comparator and structured elicitation exercise of initial stakeholder expectations of intervention effects.

The elicited patient pathway probabilities and lengths of stay quantities profile which of the existing types of patients are expected to avoid the ED and how this impacts the lengths of stay across the system. The exercise also quantifies the stakeholders’ uncertainty and disagreement, with qualitative insights into why. The elicitation exercise participants draw upon the rationale for how the intervention is expected to affect a change within the local context, as captured within the logic model, together with the descriptive analyses of the characteristics and utilization of their target population. Feedback indicates the methods are acceptably robust yet pragmatic enough for healthcare delivery settings.

As proposed in this paper, HTA methods can be used to capture how key stakeholders initially expect a service intervention to affect a change within their local context. The example results can be used in a decision-analytic model to guide the development and management of an intervention.

The Health Technology Assessment International (HTAi) 2023 Annual Meeting included a novel “fishbowl” session intended to 1) probe the role of HTA in the emergence of generative pretrained transformer (GPT) large language models (LLMs) into health care and 2) demonstrate the semistructured, interactive fishbowl process applied to an emerging “hot topic” by diverse international participants.

The fishbowl process is a format for conducting medium-to-large group discussions. Participants are separated into an inner group and an outer group on the periphery. The inner group responds to a set of questions, whereas the outer group listens actively. During the session, participants voluntarily enter and leave the inner group. The questions for this fishbowl were: What are current and potential future applications of GPT LLMs in health care? How can HTA assess intended and unintended impacts of GPT LLM applications in health care? How might GPT be used to improve HTA methodology?

Participants offered approximately sixty responses across the three questions. Among the prominent themes were: improving operational efficiency, terminology and language, training and education, evidence synthesis, detecting and minimizing biases, stakeholder engagement, and recognizing and accounting for ethical, legal, and social implications.

The interactive fishbowl format enabled the sharing of real-time input on how GPT LLMs and related disruptive technologies will influence what technologies will be assessed, how they will be assessed, and how they might be used to improve HTA. It offers novel perspectives from the HTA community and aligns with certain aspects of ongoing HTA and evidence framework development.

A conceptual framework, called Innovation of Health Technology Assessment Methods (IHTAM), has been developed to facilitate the understanding of how to innovate methods of health technology assessment (HTA). However, the framework applicability has not been evaluated in practice. Hence, we aimed to explore framework applicability in three cases of method innovation that are part of the HTx project and to develop a roadmap to improve framework applicability.

The IHTAM framework was applied to three cases of innovating HTA methods. We collected feedback from case study leaders and consortium members after a training session, an approximately 1-year follow-up of periodic case study meetings, and a general assembly meeting where innovation progresses of the three cases were reported through surveys and interviews. Feedback was then summarized using an open-coding technique.

According to feedback, the framework provided a structured way of deliberation and helped to improve collaboration among HTA stakeholders. However, framework applicability could be improved if it was complemented by a roadmap with a loop structure to provide tailored guidance for different cases, and with items to elaborate actions to be taken by stakeholders. Accordingly, a 48-item roadmap was developed.

The IHTAM framework was generally applicable to the three case studies. A roadmap, with loop structure and actionable items, could complement the framework, and may provide HTA stakeholders with tailored guidance on developing new methods. To further examine the framework applicability, we recommend stakeholders to apply the IHTAM framework and its roadmap in future practice.

Evaluating the impact of patient involvement in health technology assessments (HTA) may help improve practices and avoid ineffective activities. Evaluation, however, continues to be infrequent, inconsistent, and often only relates to process quantity or quality. The Patient and Citizen Involvement in HTA Interest Group (PCIG) within Health Technology Assessment International set out to contextualize this impact to support evaluation.

Given the lack of established methodology to measure impact, the team performed a qualitative analysis of first-hand accounts about perceived changes in HTA due to involvement of patient stakeholders. A questionnaire was developed, piloted, and rolled out to collect personal perspectives from stakeholders with relevant experience. The stories were analyzed in the aggregate to identify themes in the data.

From January 2019 to September 2021, twenty-four responses were collected through PCIG’s network. Responses (including one joint industry-HTA body submission) came from patient representatives (12), HTA bodies (11), and industry representatives (2) from North America (5), South America (3), Europe (13), and Asia Pacific (3). Based on themes commonly reported, a three-domain framework for evaluating impact is proposed: impact on basis of HTA result or recommendation, impact on HTA body, and impact on patient participants. The framework includes components under each domain to support reporting.

Using the Three-Domain Impact Framework may be useful in identifying, evaluating, and communicating the value of patient involvement in HTA. Enhancing and increasing reporting practices may improve transparency and facilitate process improvements for meaningful integration of patient stakeholders into HTA appraisals across jurisdictions.