The burden of cancer is steadily increasing in Australia, with almost 50,000 deaths from cancer in 2019, compared to 44,171 in 2014 (1). Chemotherapy has traditionally been used to treat most cancers (Reference Schirrmacher2), but the last 15 years has witnessed a paradigm shift to more targeted therapies (Reference Pottier, Fresnais and Gilon3) and novel agents that activate the immune system (Reference Zhang and Chen4) to fight cancer. These new agents show great promise in terms of health benefits, with increased survival and delay of disease progression. Regulators such as the European Medicine Agency and the US Food and Drug Administration have created rapid assessment and approval processes (Reference Martinalbo, Bowen and Camarero5), which has resulted in sponsors looking for every opportunity to accelerate the time to market authorization. Acceleration of assessment is rarely the case when it comes to reimbursement. In Australia, it is estimated that only 23.8 percent of the reimbursement submissions for checkpoint inhibitors receive first-time approval (Reference Kim, Liew and Goodall6). Moreover, checkpoint inhibitors required an average of 2.23 submissions before getting approved, compared to 1.70 submission for pharmaceuticals in general (Reference Lybrand and Wonder7).
All medicines, including cancer drugs, undergo a comprehensive health technology assessment (HTA) process before being approved for reimbursement. The process starts with a sponsor (usually a pharmaceutical company) submitting an application for the listing of the drug in question. The submission is assessed by the Australian Pharmaceutical Benefits Advisory Committee (PBAC), which makes recommendations to the Department of Health and the Minister of Health. The PBAC comprises independent clinicians, other healthcare professionals, health economists, consumer representatives, and an industry-nominated member. Submission guidelines issued by the PBAC mandate HTAs, which often include cost-effectiveness analysis (CEA) and health economic modeling for the assessment of pharmaceuticals including oncology drugs (8). The evolution of oncology-specific issues is well illustrated by the PBAC submission guidelines. In its 2002 guidelines, (9) time-to-event data, which are the primary way of reporting cancer survival data, were only mentioned four times, compared to twenty-three times in the 2006 guidelines (8). In the 2016 update of the guidelines, time-to-event data were mentioned fifty-four times (10).
Anecdotally, the pharmaceutical industry and the government occasionally have adversarial interactions. The pharmaceutical industry claims that PBAC guidelines updates increase in complexity, leading to more uncertainty (Reference Chollet, Lindsay and Gonzalo11) and it has been suggested that early dialogue may be beneficial (Reference Backhouse, Wonder and Hornby12). Interaction between the government and pharmaceutical companies is possible through presubmission meetings, (13) but they are most commonly behind closed doors and are commercial in confidence.
It is possible that misaligned incentives between manufacturer (private sector) and funder (public sector) may lead to differing views regarding the current issues facing HTA. For example, manufacturers want early access at the maximum price; therefore, they are prepared to use early evidence (14,15), whereas the funder, who needs to manage multiple competing requests for funding, must consider the opportunity cost of making the wrong decision. Consequently, manufacturers and funders have difference in opinions with respect to attributes of risk and uncertainty. This leads to delays in access to medicines for patients as multiple submissions are needed to resolve the differences (16).
A recent review of HTA in oncology (Reference Kim, Goodall and Liew17) found the following major themes to be important: analysis of clinical evidence, quality of life (QoL), modeling, financial implications, and how to deal with uncertainty. Uncertainty is a perpetual problem and stalls decision making; sponsors and payers often fail to agree on a suitable price that reflects the level of uncertainty. This is reflected in the low first-time approval rate of immune checkpoint inhibitors in Australia.
The aim of this study was to find ways of bridging the gap in opinions concerning HTA in reimbursement submission between manufacturers and payers to avoid access delays for patients of vital medicines such as oncology drugs. This will be done by investigating differences and similarities of opinion with respect to HTA in oncology among key stakeholders from the private sector (pharmaceutical industry and specialist consultants), as well as public sector stakeholders (academia and government) in Australia.
Methods
A quantitative survey was conducted of stakeholders involved in CEA of oncology agents, representing academia, government, and industry. An invitation was emailed to potential participants identified in the memberships of local professional societies. Initial invitees were encouraged to forward the survey to other relevant colleagues. Respondents were kept anonymous as required by the relevant Human Research Ethics Committee.
The survey was developed using the Qualtrics Survey platform and was completed between October 2020 and March 2021. Questions based on a review of the most common topics arising in the public summary documents describing PBAC deliberations for oncology products from 2006 to 2019 were developed and discussed with a group of experienced health economists. The survey comprised twenty-two questions arranged in nine sections (Supplementary Material): background demographics, general statements on cost-effectiveness of oncology treatments, clinical effectiveness claim, extrapolations, QoL, costs and health resource utilization, agreements, decision making, and capability and capacity. The survey took respondents approximately 10 minutes to complete. Most responses were categorized in a Likert scale (1 = strongly disagree, 2 = somewhat disagree, 3 = neither disagree nor agree, 4 = somewhat agree, 5 = strongly agree, 6 = do not know). For the analyses, the categories “strongly agree” and “somewhat agree” were collapsed into one group and also the categories “strongly disagree’ and ‘somewhat disagree”. This was done to ensure that there were an adequate number (> Reference Martinalbo, Bowen and Camarero5) of observations available in each category for the chi-square tests. The category “neither disagree nor agree” is henceforth referred to as “neither” within the text. Some of the questions were ranking exercises, in which the respondents were asked to rank items from best to worst.
Responses to each question were summarized using descriptive statistics. Test for difference between responses from private and public sector participants was performed using chi-square statistics as the outcome variable consisted of three categories. A standard 5 percent significance level was used.
All analyses were performed using SAS and Excel on a MS Windows platform.
The study was approved by the Monash University Human Research Ethics Committee (project ID: 25627).
Results
Characteristics of the Respondents
There were ninety-seven respondents in total, with thirty-seven from the public sector (academia and government) and sixty from the private sector (pharmaceutical companies and consultancies).
Mean (standard deviation, SD) years of working experience in health economic oncology modeling was 14.1 (7.6) years, and there was good representation of leadership/managerial and nonmanagerial positions. The majority of respondents in both groups had a postgraduate degree (master 48.4 percent and doctoral 34.7 percent) and the top three primary qualifications were in economics (28.4 percent), pharmacy (20.0 percent), and science (16.8 percent).
Data Sources
The participants were asked to rank six different clinical data sources (not randomized clinical trials). Preferred data sources by order were single arm clinical trials, local registries, propensity adjusted data, and historical published data (Figure 1). Chart reviews and drugs claims data were the least preferred. No difference between public sector and private sector respondents was observed.
Figure 1. Ranking of data sources.
The Clinical Effectiveness Claim
Use of surrogate endpoints was perceived to be a major source of uncertainty with respect to clinical effectiveness claims by both sectors (all respondents: agree = 57.9 percent, p = .1164); however, there was no consensus that superiority should be demonstrated in a head-to-head trial (disagree: public = 36.1 percent, private = 11.9 percent, p-value = .0120) and that statistically superiority was not considered sufficient for clinical effectiveness claims (agree: public = 8.3 percent, private = 25.4 percent, p-value = .0350). This was consistent with the fact that 61.1 percent of all respondents disagreed with the statement that the minimal clinically important difference (MCID) was not relevant versus neither = 24.2 percent and agree = 14.7 percent (p-value = .0704 for difference between sector) (Table 1).
Table 1. Summary of Response
Extrapolation of Survival Data
Private sector respondents did not see extrapolation of survival data as a “black box” (disagree: private = 52.5 percent, public = 24.4 percent, p-value = .0433). Regarding the question on whether graphical checks and Akaike information criterion/Bayesian information criterion were sufficient justification for choice of extrapolation both groups predominantly responded disagree (all respondents: agree = 17.9 percent, p = .3748). Both groups were indifferent to extrapolation methods like splines (all respondents: agree = 12.6 percent, p = .9085). The majority of private respondents agreed that the observed Kaplan–Meier curve should be used for part of the modeling (agree: public = 44.4 percent vs. private = 78.0 percent, p = .0023).
It is common in Australia for the PBAC to request extrapolated survival curves that converge over time (10), but a third of the respondents from the private sector disagreed that the survival curves should be forced to converge over time, compared to only 5.5 percent of public sector respondents (p = .0007). Similarly, a large proportion of private sector respondents agreed that it was reasonable to assume cure (agree = 40.7 percent), which was significantly different (p = .0002) when compared to public sector respondents (agree = 5.6 percent). Having a time horizon of no longer than 10 years was another point on which private (disagree = 66.1 percent, neither = 23.7 percent, agree = 10.2 percent) and public responses (disagree = 25.0 percent, neither = 61.1 percent, agree = 13.9 percent) were not aligned on (p = .0003).
External validation was thought to be inadequate by both groups (all respondents: agree = 52.6 percent, p = .0528). While access to individual patient data (IPD) was seen as helpful by a majority in groups (agree: public = 64.9 percent, private = 69.5 percent) more private sector respondents disagreed (disagree: private = 15.3 percent vs. public = 0.0 percent, p = .0185). Finally, both groups disagreed (all respondents: agree = 26.3 percent, p = .4742) that there was enough guidance in the PBAC guidelines.
Quality of Life
A generic utility instrument such as EQ-5D was preferred (85 percent of respondents ranked it either one or two), whereas mapping of a nonutility instrument to a utility instrument was the least preferred method (92 percent of respondents ranked it either three or four) (Figure 2).
Figure 2. Ranking of quality-of-life instruments.
Both private and public respondents disagreed that validation of QoL instruments was not important (all respondents: agree = 29.5 percent, p = .6444). Only a minority of respondents agreed that using proxies instead of patients to generate QoL valuations were appropriate (all respondents: agree = 17.9 percent, p = .1185).
Overall, the respondents preferred utilities to be measured directly in the clinical trial (preferred = 86.3 percent) and the least preferred method was to source QoL information from the literature (least preferred = 57.5 percent) (Figure 3). There was no difference between private and public respondents (p > .05).
Figure 3. Ranking of quality-of-life data sources.
Costs and Health Resource Utilization
Regarding the question on whether duration of treatment was a major source of uncertainty only few of public sector respondents disagreed compared to private sector respondents (disagree: public = 2.8 percent, private = 22.0 percent, p = .0040). A similar trend was observed for other costs and health resources: postprogressive treatments should always be accounted for (disagree: public = 0.0 percent, private =20.3 percent, p = .0126), adverse events (AEs) are a major source of uncertainty (disagree: public = 5.6 percent, private = 16.9 percent, p < .0001). No difference was observed in terms of treatment beyond progression as a major concern (all respondents: agree = 47.4 percent, p = .2598), palliative/best supportive care costs (all respondents: agree = 60.0 percent, p = .2188) and whether different discount rates should be used for costs and outcomes (all respondents: disagree = 47.4 percent, p = .6588).
Risk Share Agreements
Both public and private sector respondents agreed that outcomes-based risk share agreements (RSAs) are difficult to implement (all respondents: agree = 73.7 percent, p = .1852). On the other hand, there was no consensus that financial agreements are necessary to ensure cost-effectiveness (disagree: private = 42.2 percent vs. public = 13.9 percent, p = .0056). Likewise, there was a difference in the response to the statement that managed entry schemes are good tools when clinical data are limited with the majority of private respondents agreeing with the statement (agree: private = 67.8 percent vs. public = 36.1 percent, p = .0019). Relative few public sector respondents agreed that RSAs are not balanced and that there is too much risk for either the payer or the sponsor compared to private sector respondents (agree: private =44.1 percent vs. public = 16.7 percent, p = .0123).
The majority of respondents both public sector and private respondents agreed that rebates are a good tool to reach an agreement between payer and the sponsor (agree: private = 81.3 percent, public = 55.5 percent, p = .0159) and that review of cost-effectiveness is useful to assess value for money (agree: private = 71.2 percent, public = 66.7 percent, p = .0369).
Decision Making
Private and public sector respondents agreed that the ICER threshold is higher for cancer compared to other therapeutic areas (all respondents: agree = 49.5 percent, p = .1875), and that HTA and CEA are useful tools for assessing reimbursement of oncology therapies (all respondents: agree = 69.5 percent, p = .0990). They also both disagreed that the evidence requirements are higher for oncology when compared to other therapeutic areas (all respondents: disagree = 61.0 percent, p = .8623).
Very few public sector respondents thought that opinions of patient advocates are important for informing cost-effectiveness (agree: private = 30.5 percent vs. public = 8.3 percent, p = .0239) and that alternative funding methods are more appropriate than current practice (agree: private = 33.9 percent vs. public = 5.6 percent, p = .0061). The majority of private sector respondents agreed that the reimbursement process is too long (agree: private = 69.5 percent vs. public = 19.4 percent, p < .0001) and that more public transparency is needed surrounding reimbursement decisions of oncology therapies (agree: private = 58.3 percent vs. public = 27.8 percent, p = .0058).
Finally, very few private sector respondents agreed that CEA of oncology therapies is often black boxes (agree: private = 6.8 percent vs. public = 16.7 percent, p = .0154).
Capacity and Capabilities
More private sector respondents compared to public sector respondents disagreed that they were comfortable with their technical level of CEA (disagree: private = 18.6 percent vs. public = 5.5 percent, p = .0305). The majority from both groups concurred that their organization has plenty of capacity to deal with methodological issues (all respondents: agree = 64.2, p = .2394). However, a majority also felt that continued CEA training opportunities are scarce (all respondents: agree = 56.8 percent, p = .1610).
The sources for methodological advances seemed to come from a variety of sources such as on the job learning, the peer reviewed literature, seminars, discussion with peers, and workshops by professional bodies. The least popular way was guideline updates (see Figure 4).
Figure 4. Sources of methodological advances.
Discussion
To our knowledge, this is the first study surveying both industry and payers with respect to opinions on cost-effectiveness and HTA at a national level in Australia.
Main Findings of the Study
Private sector and public sector respondents had similar views on clinical effectiveness claims as they agreed that clinical superiority needs to be demonstrated in a head-to-head clinical trial and that superiority claims made on surrogate endpoints lead to major uncertainty. However, they were divided when it came to extrapolation of survival data. Costs and health resource utilization were also a contentious issue between private sector and public sector respondents. With respect to RSAs and decision making, respondents generally agreed that rebates are a useful tool, outcomes-based agreements are difficult to implement, managed entry schemes are required when data are limited, review of CEA is useful to assess value for money, evidence requirements are not higher for oncology than other areas, and the ICER threshold is higher in cancer compared to other therapeutic areas. In general, HTA training takes place through on the job training, keeping abreast of the peer reviewed literature and through discussions with peers. Guideline updates were a least favored opportunity for continued training in CEA.
The differences in responses were not surprising as the two sectors have different goals. For example, private sector respondents downplay additional costs which makes a drug less cost-effective option whereas public respondents err on the side of caution in the absence of robust evidence. Unfortunately, the differences are likely the cause delay in access to medicines due to multiple resubmissions and extended pricing negotiations.
Both private sector and public sector respondents were largely aligned with the PBAC guidelines. For example, both disagreed that MCID is not relevant for superiority claims. Interestingly, both groups agreed that the observed Kaplan–Meier curves should be used for part of the modeling, which is specific to the PBAC guidelines, for example, the UK National Institute for Health and Care Excellence (NICE) guidelines do not require this (18).
Comparison to Previous Literature
Previously reported studies have primarily focused on the response of payers in a broader perspective than what is presented here.
Stephens et al. (Reference Stephens, Handke and Doshi19) performed an international survey of methods used in HTA in 2012. Thirty respondents were recruited from HTA bodies worldwide, with three respondents from Australia. Common attributes among the HTA bodies were evaluation of cost-effectiveness, safety, and QoL. Issues that the HTA bodies faced were lack of evidence and data availability for emerging technologies. The authors concluded that future efforts should expand the respondent sample to include more emerging markets and update the results of this survey to specifically address additional aspects of research methods in HTA.
Moloney et al. (Reference Moloney, Mohr and Hawe20) conducted a survey of payers in the USA and Europe on comparative effectiveness research. A total of fourteen respondents provided feedback on study design, methods/analytics, data capacity, and burden of evidence generation and accountable care. The responses were very broad and did not include any detail on specific issues with respect to CEA. The authors concluded that more engagement with payers, manufacturers, and regulatory agencies was needed to discuss key methodological tradeoffs. It should be noted that this research was sponsored by the pharmaceutical industry and so conclusions might have been biased.
Another survey by Ciani et al. (Reference Ciani, Wilcher and Blankart21) surveyed thirty-six HTA agencies across twenty non-European countries with respect to HTA and medical devices. The survey compared processes and methods between medical device agencies and nonmedical device agencies. The conclusion was that scientific methods for HTA of devices need further development to adapt to new methods for medical devices. No feedback was sought from the private sector.
Strengths and Limitations
As described above, previous surveys have primarily included payers and been small. The present study was based on ninety-seven respondents from across industry, academia, and government. One of the main strengths is that it comprises of in-depth questions with respect to specific methodology issues in HTA of oncology drugs. Furthermore, the respondents were in general very experienced, with more than an average of 14 years of experience. Hence, the findings are likely to be a reliable representation of opinions of health economists familiar with this subject in Australia.
A main limitation of the present study was the generalization of questions. Issues differ between drugs, tumor types, and lines of therapy, which makes it difficult to make general statements with respect to CEAs in oncology. Respondents were asked to fill out the survey with an average case in mind. It was possible to assess the response rate for the survey as the pool of potential respondents was not known. Specific affiliation (e.g., pharmaceutical company, university or government department) was not collected due to risk of identification of the respondents. This meant that generalizability could not be assessed. Another limitation was that the question of using proxies as source for health-related QoL evidence was possibly confounded by combining clinicians with the general population.
Implications for Research
Based on this survey, the ideal seems to be: head-to-head clinical trial versus a superiority claim on a final outcome measured directly in the clinical trial. However, this is unfortunately not always the case and therefore there are still many areas that need further research in terms of HTA and oncology. For example, there are still issues with respect to surrogate to final outcomes which is recognized by the majority of the respondents in this survey. Previous recommendations surrounding transparency and quantification of exploration of uncertainty have been published both in general (Reference Elston and Taylor22,Reference Ciani, Grigore and Blommestein23) and for several tumors types like breast cancer (Reference Gyawali, Hey and Kesselheim24) and colorectal cancer (Reference Ciani, Buyse and Garside25). While extensive guidance is available from different HTA organizations (Reference Latimer26,27), the issue of extrapolation is still perceived by public sector respondents as black boxes. Validation is seen as inadequate and further research is needed.
Opinions differed markedly with respect to costs and health resource utilization between the two sectors and therefore offer an opportunity for further research. These include issues with respect to the duration of treatment, inclusion of postprogressive treatment in the CEA, and costs and health resources in connection with AEs.
Implications for Improving Capability/Capacity
The PBAC guidelines have been updated three times the past 20 years, which explains why the majority of the respondents ranked guidelines updates as the least preferred source for methodological advances. The UK NICE is supported by a Decision Support Unit which publishes reports on methodological advances on an ongoing basis (28). There are examples of this approach in Australia. For example, Cancer Australia supports a health economic support program in which an academic institution is commissioned to support the clinical trials groups with health economic training and input into their trial protocols (29). The PBAC and its stakeholders could benefit from something similar.
Both groups of respondents agreed that continued training opportunities in cost-effectiveness analyses are scarce. Moreover, they ranked courses and conferences low in terms of sources of methodological advances. As such, universities and health economic societies should be encouraged to cover this gap by setting up short courses and host conferences in this field.
Implications for Policy
Outcomes-based RSAs are seen as difficult to execute and therefore specific set of guidelines and policies on this particular topic could be helpful. Public transparency was seen as important by the respondents. There are pros and cons of improved transparency. More transparency could give the public more clarity as to why reimbursement decisions are made and in particular offer guidance when a drug is rejected. On the other hand, confidentiality mitigates the risk of reference pricing for pharmaceutical companies and enables them to offer lower prices (Reference Iunes, Uribe and Torres30).
It appears that both private and public stakeholders are in favor of inclusion of IPD as part of the reimbursement submission. Requirements for submission of IPD are a novel idea in the regulatory space as the FDA has required this for some time as part of new drug applications (31). There are of course some considerations if such a policy was to be implemented since it may contain sensitive personal data. These include ethics approval by appropriate bodies, development of processes to ensure that access to the data is limited to relevant staff, and that everything be kept commercial-in-confidence. Alternatively, pharmaceutical companies could calculate both AIC/BIC values and QoL estimates with country specific weights as part of the CSR. This would of course mean that there would be more planning from the company side, as all analyses would have to be prespecified in the statistical analysis plan before the unblinding. However, this would give regulators and payers more confidence in the estimates.
A convergence of understanding between the public and private sector is desirable as it would result in faster access to medications for patients. A formal presubmission process could improve this. Currently, there is an opportunity for a manufacturer to meet with the Department of Health before a submission. Any advice given at these meetings is nonbinding and is therefore of limited value in terms of reaching agreement on opinions. For medical devices and medical procedures, the first step in the reimbursement process is to establish the framework for the economic evaluation (Reference Kim, Byrnes and Goodall32). This is done by clearly articulating the population, intervention, comparator, outcomes and confirming the relevant clinical algorithms. A similar process for pharmaceuticals would be beneficial. This would ensure that the manufacturer and the payer have agreed on key issues up front thereby avoiding multiple submissions as well as minimize delays to access for patients.
Conclusions
The study presented here is the largest survey of both private sector and public health economic practitioners in Australia. Not surprisingly the private sector respondents favor methods that reduce the ICER when compared to the public sector respondents. While the field of health economics is mature, this study has identified a number of challenges for HTA in oncology that warrant further research.
Supplementary Materials
To view supplementary material for this article, please visit http://doi.org/10.1017/S0266462322000368.
Conflicts of Interest
H.K. and S.G. declare no conflict of interest. D.L. has participated in advisory boards and/or received grants from Abbvie, Astellas, AstraZeneca, Bristol-Myers Squibb, CSL-Behring, Edwards Lifesciences, Novartis, Pfizer, Sanofi, and Shire.
Open access
