Introduction
Health technology assessment (HTA) is used in many countries to inform decisions about reimbursement and access to interventions based on multidisciplinary processes that use explicit methods to determine the value of a health technology at different points in its life cycle (1). Patients and citizens are key stakeholders affected by HTA and resulting decisions. Hence, their perspectives should be fully understood in the HTA appraisal process (Reference Facey, Ploug Hansen and Single2). Even though evidence relating to clinical effectiveness, cost-effectiveness, and affordability are important elements in HTA, patient-based evidence and patient or citizen input can reflect the experiences, needs, and attitudes with the disease or the health technology as reported by patients, their families, carers, legal representatives, patient organizations (POs), or patient and citizen advocacy groups (PCAGs) (Reference Facey, Ploug Hansen and Single2).
HTA agencies across the globe have been developing models and methods for patient and citizen involvement (PCI) in access, coverage, and policy decision-making processes relating to health technologies in both public and private health sectors (Reference Facey, Ploug Hansen and Single2;Reference Facey3). Such PCI models were spearheaded by HTA agencies from some high-income countries, and examples are described in Facey et al. (Reference Facey, Ploug Hansen and Single2) or in Abelson et al. (Reference Abelson, Wagner, DeJean, Boesveld, Gauvin and Bean4).
Patient and citizen (social) engagement practices are increasingly encouraged by global health policy actors in healthcare-system design, research, and governance at all levels. For example, the World Health Organization released a new handbook on social participation for universal health coverage (in 2020 (5)). The Brazilian CONITEC partnered with several stakeholders to improve its model and methods for social engagement by implementing both social appropriation and knowledge translation in HTA, coverage, and policy decision making (Reference Sousa, Figueiró, Silva, Silva, Silva and Pereira6). The underlying motivation for such PCI initiatives is to allow all social actors with an interest in the management of their individual and collective health (systems), the right to have a voice via institutionalized processes at some or all levels of their national health systems (Reference Sousa, Figueiró, Bicudo and Pileggi7). Moreover, PCI comprises participatory processes that can lead to better health experiences or patient health outcomes, as outlined by the WHO (5) and the UK's government-funded INVOLVE initiative (Reference Hickey, Richards and Sheehy8), which was established in 1996 to support active public involvement in the National Health Service (9).
Additionally, other initiatives sprout along with the breadth of healthcare innovation and delivery, with examples from the life sciences industry (Reference Cook, Cave and Holtorf10), the medical device industry (11), regulatory agencies (12;13), or the grassroots-driven design and conduct of research projects of relevance to patients or their families, especially for rare and familial diseases such as multiple endocrine neoplasia type two (Reference Sousa, Gallian and Maciel14). These patients or their families participate in study participant selection, data analysis, and interpretation of the data collected by them and their family members, and health care and research partners. Hence, researchers, practitioners, and the public join efforts and share responsibilities to develop, implement, monitor, evaluate, and redevelop scientific approaches and technological innovation (Reference Prainsack, Schicktanz and Werner-Felmayer15).
Decision fairness, equity, and patient relevance are concepts that should be true not only for high-income countries (HICs) that are rich in resources and have advanced health(care) systems, but also for low- and middle-income countries (LMICs) (Reference Sousa, Gallian and Maciel14). Many LMICs struggle in finding equitable and sustainable solutions to expand their health(care) systems toward universal health coverage (UHC). A resolution (WHA67.23) from the 67th World Health Assembly underlined the importance of HTA in support of UHC (16). Identifying and understanding pathways for citizens and patients in LMICs, and the varied ways by which they can collaborate with HTA stakeholders for maximizing their health despite scarce resources might help more LMICs in advancing PCI and in creating more sustainable healthcare-decision pathways.
However, PCI in LMICs must be aligned with the countries’ sociocultural, ethical, regulatory, economic, and political perspectives to support attainment of their healthcare priorities, which often differ from those in HICs. To identify how PCI can be realized in the LMIC context, a collaborative initiative of the Developing Countries and Patient and Citizen Involvement Interest Groups (DCIG and PCIG) from the Health Technology Assessment international (HTAi) society set out to explore whether and how LMICs are involving patients and citizens or co-producing patient and citizen (social) engagement in formal and informal HTA processes or in healthcare decision making where HTA is lacking or is at its infancy.
In this paper, we describe the first pilot analyses performed in four diverse countries: Brazil, Indonesia, South Africa, and Nigeria. The long-term objective of applying the questionnaire across LMICs is to uncover aspects of PCI in LMICs related to context and HTA practice, share experiences, and identify best practices on how patients and citizens can make meaningful contributions to the local decision- and policy-making processes in LMIC healthcare environments.
Methods
A questionnaire with fifteen questions was developed by the multi-stakeholder and international project team. The questionnaire (see Supplementary File 1) is structured into three sections: [A] Healthcare system covering the type of healthcare system and healthcare priorities; [B] PCI or advocacy related to any form of influence patients or citizens could take on healthcare decisions; and [C] Healthcare decision making related to its elements including HTA, pricing, or coverage and the degree of patients' influence on each of these elements. The questionnaire used in this pilot study is a refined version evolved from a previous format, which had been developed and tested within the project team and was revised to the current form based on their feedback and difficulties encountered.
To pilot-test the questionnaire format, it was rolled out to four diverse LMICs: South Africa, Brazil, Indonesia, and Nigeria. The selection criteria for these countries were geographical diversity, economic diversity, and willingness of project team members to take the in-country lead. Each questionnaire was completed by one or two project team members from the respective country, who took the lead but also consulted further with patient advocacy groups (PAGs), academics, or the country's Ministry of Health (MoH) for answering some of the questions (full results per country in Supplementary File 2). Different approaches were used in the four countries to confirm the quality of the collected data. In Brazil and Nigeria, the questionnaire was reviewed by the HTA agency and MoH experts, respectively; in Indonesia, the information was confirmed with academic health policy experts; in South Africa, the responses were corroborated with published documents and extensively referenced.
The findings were compiled into one table to allow for comparison and identification of potential similarities and differences.
Each country lead was asked to share their experiences of completing the questionnaire items by responding to five questions: (i) How easy was it to fill the table? (ii) How much time did it require approximately? (iii) How many other people had to be consulted and for what? (iv) What was specifically difficult? (v) Any suggestions for improvement?
The responses were reviewed question by question to inform a potential further revision of the questionnaire before the full rollout.
Results
The pilot results are summarized in Tables 1–3 for each of the three questionnaire sections.
Table 1. Healthcare (HC) system related survey responses
Table 2. Patient and citizen involvement related survey responses
Table 3. Patient and public involvement related survey responses
Healthcare System
Table 1 shows the responses to the question relating to the healthcare system in investigated countries. In all four countries, healthcare funding is provided by a mix of public and private funds as well as out-of-pocket costs paid by patients. The mix of how much is contributed by each of the funding organizations (government, private health care, and insurance) differs between the countries. For example, 80 percent of the population can access publicly funded health care in Indonesia, 12 percent pay out of pocket, and 8 percent are covered through private health insurance. In Nigeria, however, most patients pay out of pocket and only employees of bigger companies or of the government are covered through publicly funded healthcare services; only 1.5 percent of the population are covered through private health insurance. Brazil, on the other hand, offers health coverage via its Unified Health System, which is poor in resources, whereas the more affluent population subscribes to private healthcare insurance schemes.
Maternal and child health (Brazil, South Africa, and Nigeria), infectious diseases such as HIV/AIDS (South Africa), tuberculosis (South Africa), malaria (Nigeria), or general diseases (Nigeria) were also reported as health priorities. In addition, mental health was mentioned in Indonesia and other items such as environmental or work health in Brazil.
UHC was reported by all four countries as healthcare policy priority—with differing stages of implementation—as well as items such as quality improvement (Brazil), improvement in health outcomes in specific diseases (Brazil and Nigeria), re-engineering of primary health care (South Africa), or improvement in financial management (South Africa).
Patient and Citizen Involvement
The responses to the question relating to PCI in HTA decision making in the investigated countries are displayed in Table 2. In Brazil, patients and citizens have some influence through the tripartite National Health Council (NHC) (representing patients, healthcare partners, POs, PCAGs, nongovernmental organizations (NGOs), healthcare professionals, and the health industry sector). In South Africa, patients are formally represented in some policy initiatives but not in the decision-making processes relating to specific health areas or interventions. Otherwise, the influence of POs or PCAGs is, in all four countries, mostly exerted through advocacy and awareness activities or through the legal pathways to get access to therapies (litigation).
Concerning other stakeholders, who may influence healthcare decision making in the country, the respondents mentioned religious communities, advocates, or politicians (Brazil and Indonesia), consumer organizations (Indonesia), civil society groupings (South Africa) (international), and NGOs (Nigeria).
Within the realm of POs, different levels of patient advocacy and activism were reported. In Nigeria, patient engagement is still exceptional, starting with HIV/AIDS and a few organizations engaging for diabetes, malaria, or vaccination. More patient advocacy is observed in Indonesia with POs engaging for oncology (breast cancer and thalassemia), diabetes, stroke, cardiovascular disease, hemodialysis, psoriasis, tuberculosis, or autism. In South Africa, patient advocacy relates to specific diseases (HIV/AIDS, cancer, tuberculosis, NCDs, and mental health) or to the healthcare policy level (access to treatment; use TRIPS in relation to IP; NHI bill, Presidential Health Compact (17)). In Brazil, POs are also actively engaging at the disease and the policy level within the National Congress.
The responses relating to the most likely decision area for the involvement of patients or the public varied from “all” (Brazil), through healthcare policy decisions (South Africa) and reimbursement (South Africa and Nigeria), to influence through advocacy or indirect influence through other stakeholders (Indonesia).
Healthcare Decision Making
A high variability can also be seen for the aspects of healthcare decision making in the four different countries, which are summarized in Table 3. Although HTA is defined in Brazil with published processes and responsibilities, it is either not transparent (Indonesia), only partially applied (South Africa), or nonexisting (Nigeria). Where HTA is done, it is implemented either for only pharmaceuticals (i.e., Essential Medicines List and high-cost medicines at the national MoH level in South Africa, by private health insurance companies for reimbursement decisions in South Africa, or for new and high-cost medicines in Indonesia) or for interventions and/or programs in health care (Brazil).
Several challenges for HTA or evidence-based decision making have been listed by Brazil, Indonesia, and South Africa, collectively, and these challenges are: difficulty in adoption or realization of the HTA recommendations into the healthcare operational levels, a lack of transparency of frameworks, a lack of human resource capacity, capability, and expertise, division between state and healthcare delivery structure (public–private components), and corruption or other routes of impacting decisions beyond HTA.
In terms of where patients most likely could influence technology-related decisions, listing was mentioned in Nigeria, South Africa, and Indonesia, and coverage decisions were mentioned in Brazil, Indonesia, and South Africa. In addition, in Brazil, a potential role for patients was seen in negotiating prices with pharmaceutical companies to increase the alignment of stakeholders and consequently, reduce the level of litigation relating to access to medicines in the country.
Questionnaire Experiences
-
Easiness of implementation: Two respondents experienced the questionnaire as fairly easy to complete (Indonesia and Brazil) and the other two thought that it was neither easy nor difficult (Nigeria and South Africa).
-
Time required: It took the respondents between 20 min and 2 h to complete the questionnaire (average 45 min). However, to collect the additional information from other experts (Indonesia, Brazil, and Nigeria) or published material (South Africa) took additional time. Particularly, if it was sought from official sources such as the MoH, it took about 1 month to get the response (Brazil and Nigeria).
-
Additional expertise required: In Indonesia, two additional people were contacted for more information (one from MoH and one from a PAG). In Brazil, four technical officers from MoH gave input or reviewed the responses. In South Africa, five different stakeholders were contacted to clarify and update the respondent's knowledge. In Nigeria, one member of the MoH, one community pharmacist, one hospital expert, and one PCAG were asked to provide information on government and private sector approaches to health and patient involvement in decision making.
-
Difficulties: Three respondents felt that it was specifically difficult to seek input from public healthcare organizations due to bureaucratic hurdles (e.g., formal application by an authority requested) and a lack of transparency in this subject. In addition, it was challenging to condense the information on the complex healthcare system into the format of the questionnaire.
A few suggestions for improving the questionnaire were proposed by the respondents. These included, to offer the questionnaire in an online version suitable for collaborative approaches (i.e., several experts or stakeholders can work together to complete the questionnaire and build a consensus on the responses or content), more guidance should be given, for example, which information exactly was required in each of the tables (descriptive and using examples), and a formal support letter from the HTAi might facilitate access to the required information from the MoH. In addition, it was suggested to adopt an additional question relating to the existence, extent, and decision-making process of an “Essential Medicines List” for characterizing the healthcare-system level.
Discussion
Under the postulate that health technologies should ultimately benefit the health of the patient, consideration of the patient or citizen perspective including their needs and experiences with the disease and current or new interventions within their healthcare context should be a formal part of assessing the value of such health technologies alongside those from health professionals who care, implement, and manage the adequate use of those health technologies (Reference Facey, Ploug Hansen and Single2). Although in many advanced and resource-rich healthcare systems, formal value frameworks providing for PCI have been established, less is known about this in LMICs.
This pilot questionnaire was applied to four diverse LMICs and was aimed at understanding how patients are involved in HTA or healthcare decision making in each of these countries.
For the next phase of the project, an improved template will be used by the project team to describe PCI in additional LMICs, whereby the initial focus may be those countries with known incidence of PCI in HTA such as Thailand and other South American LMICs. Identification and selection of the next group of target countries and experts with local knowledge will rely on the HTAi network and the extended networks of the project team members. If possible, the countries will be clustered by certain criteria (e.g., by contextual situations of the healthcare system, epidemiology profile, or the extent of HTA use) to identify patterns. However, in this paper, we have chosen countries, which allowed us to pilot-test the questionnaire's applicability across diverse settings. The downside of this approach is, that, at this stage, the case observations offer only a limited base for drawing general conclusions. Expanded use of the questionnaire will allow for more in-depth analysis. In addition to delivering information on the respective status of PCI in diverse countries, dissemination of the finding of the analyses through publications or conferences will help increase awareness and transparency across stakeholders as well as raise the focus on the routes for PCI in HTA globally.
The responses to this pilot analysis indicate a mixed picture on PCI in the diverse healthcare systems and decision-making processes in the four countries, reflecting the diverse sociocultural, regulatory, political, and economic contexts as those from Brazil (Latin America), Indonesia (Asia-Pacific), Nigeria, and South Africa (Africa).
The first part of the analytical questionnaire tested in this study focused on healthcare-system elements. Healthcare systems in the four pilot countries are diverse in terms of quality, coverage, and efficiency. Although in all of these LMICs, government's priority is to provide coverage for the whole population, the funding mechanism (a mixture of public and out-of-pocket healthcare financing) and the path for improving the quality and achieving efficiency differ. These differences may be partly due to the distinct historical contexts prevailing in these LMICs and due to both social and cultural ways through which their populations engage with health care and technologies locally (Reference Barello, Graffigna, Vegni and Bosio18). There are similarities in terms of priority health areas targeted by the governments such as NCDs, maternal and child health, and infectious diseases. In addition to improvement in health outcomes in specific disease areas, these LMICs are keen to improve efficiency and quality of the health services through the establishment of different mechanisms—more effective healthcare planning and management, community health services, early health education, primary healthcare re-engineering, sound financial structure, and policies. This provides an opportunity for knowledge sharing, not only in disease-specific areas, even more for the mechanisms on improving outcomes or achieving coverage for the population. Engaging with patients and communities could serve as a strong multiplier in implementing these strategies for achieving UHC (Reference Odugleh-Kolev and Parrish-Sprowl19) and for establishing formal assessment approaches for health technologies. This expectation is corroborated through other examples with a proven impact of community engagement on the effectiveness of policy implementation (Reference Makaula, Funsanani, Mamba, Musaya and Bloch20–22).
Except for Brazil, HTA has not yet been formally included to facilitate the decision-making process in the other three countries. South Africa and Indonesia are at different stages of introducing HTA, whereas in Nigeria, it is nonexistent. All countries face challenges, as detailed in Table 3. Although the varying patient characteristics as well as implementation issues should be considered, current HTA methodologies and decision making only partly respond to these requirements and patient experiences or preferences are only rarely considered as has also been observed by Oortwijn et al. (Reference Oortwijn, Determann, Schiffers, Tan and van der Tuin23) when they compared the comprehensiveness of HTA practices around the globe.
To inform healthcare and policy decisions, HTA should take into account the perspectives of the stakeholders with interest in the decision to ensure contextual and democratic validity, legitimacy, and fairness of the recommendations (Reference Oortwijn, Determann, Schiffers, Tan and van der Tuin23;Reference Facey, Bedlington, Berglas, Bertelsen, Single and Thomas24). To assess the intended, unintended, or indirect effects, feasibility of implementation, as well as barriers and facilitators, engagement with all stakeholders concerned is necessary, including patients and citizens (Reference Oortwijn, Determann, Schiffers, Tan and van der Tuin23). The broader rollout of the questionnaire presented and tested in this study may help identify different pathways to PCI in LMICs as well as barriers and facilitating factors. Making the examples and key success factors visible may help LMICs to learn from one another. From the four LMICs in our study, only Brazil has started to establish formal ways for involving patients in the HTA process, with further improvement required (Reference Sousa, Figueiró, Silva, Silva, Silva and Pereira6). In general, patient engagement currently is observed only in disease-specific areas and is dependent on the prominence of the different PCAGs or POs. Even though Brazil has a formal structure for involvement through the National Health Council, PCAGs, and other advocacy groups such as religious parties play an important role in all levels of the healthcare system and try to enforce their (perceived) rights (e.g., access to a specific therapy). In all countries analyzed, the involvement of religious communities (Indonesia) and consumer associations and communities (Indonesia and South Africa) or the participation of civil society has emerged corresponding to the cultural, religious, and societal context of patients and the public in these countries. Such pathways could contribute patients’ and citizens’ perspectives to HTA and healthcare decision making.
Various examples of inclusion of patient perspectives in HTA have been reported from HICs. CADTH, in Canada, has a formal well-structured system for incorporating patient input on health outcomes and issues in both the assessment and appraisal phases (Reference Abelson, Wagner, DeJean, Boesveld, Gauvin and Bean4;Reference Berglas, Jutai, MacKean and Weeks25). In Germany, patients/patient representatives participate through a consultation process by IQWIG during the analysis phase, as well as in the generation of reports and dossier assessment and guidance for the “involvement of people affected” (https://www.iqwig.de/en/getting-involved/contributing-the-perspective-of-patients-and-other-affected-persons.3070.html). In our study, the Brazilian response also describes a consultation process as an opportunity for patient input. The National Institute for Health and Care Excellence (NICE) in England has a policy for patient and carer involvement in its decision making and provides direct support, resources and training to patients and POs who participate in the NICE's work (https://www.nice.org.uk/about/nice-communities/nice-and-the-public/public-involvement/public-involvement-programme/patient-public-involvement-policy). Many other countries such as Sweden, Taiwan, Australia, Denmark, or Italy have foreseen the involvement of patients or the consideration of patient-based evidence for certain aspects of their HTA processes (Reference Facey, Ploug Hansen and Single2).
Such participatory initiatives are also being designed for the Brazilian context, as reported in our study, and observed elsewhere (Reference Sousa, Figueiró, Bicudo and Pileggi7), whereas the other three countries fall short in this aspect. Although the POs demand to be heard in all four LMICs under study, other pathways for contributing with the patient experiences and knowledge to HTA and decision making are used, mostly through advocacy rather than formal involvement pathways. Mostly, PCAGs engage for increasing awareness of the disease burden, advocate for the adoption of certain technologies, or use legal pathways to compel arrangement for the coverage of certain treatments. More formal and active stakeholder involvement may increase their acceptance and agreement with HTA recommendations and the final decisions.
Limitations
This was the first time the questionnaire was rolled out to the first four LMICs under the leadership of project team members. For a broader rollout, it will be important to ensure that the questionnaire is self-explanatory and easy to use in order to receive useful and relevant content. All questionnaire leaders thought that the format was acceptable and easy to use. However, a broader rollout may require more guidance and definition.
In addition, as we have analyzed only four diverse countries, no general conclusions are possible with such a limited data set. A broader rollout of the revised and improved questionnaire will be necessary to derive more representative information.
The questionnaire helps describe complex issues related to the healthcare system and the degree of PCI in healthcare decision making with direct implication on PCI in HTA. In addition, different stakeholders may have different experiences and viewpoints, especially relating to PCI. It is unlikely that one individual has all expertise to fill the questionnaire alone and to describe the whole spectrum of viewpoints. Therefore, for further rollout of the questionnaire, we will encourage collaborative approaches in assembling the information and recommend web-based collaboration to do so.
Conclusions
All respondents commented that a strong support structure from policy and decision makers is required, which involves raising awareness and advocating for patient and citizen participation. In addition, good practices should be followed such as those defined by the PCIG (https://htai.org/interest-groups/pcig/values-and-standards/) or other relevant organizations. The establishment of a functional framework for PCI in HTA and thus the decision-making process can be accelerated by networking, learning from, and collaborating with ongoing international PCI initiatives and by adapting the tools and material for local use and application. As mentioned in the Introduction, it can be expected that an understanding of contextual factors, for instance, population size, wealth level, disease burden, progress toward UHC, or other relevant indicators need to be considered in LMICs. Further rollout of the questionnaire across a broader range of LMICs will help better define these requirements. Rolling out the questionnaire to as many countries as possible should help categorize and cluster them and compare their PCI strategies with other LMICs as well as those from HICs. This provides an opportunity for actors, who work across both HICs and LMICs to understand the differences and to select and tailor solutions that match the LMIC perspective.
Comparing possible processes of PCI, a broader understanding of stakeholder relationships, as well as the feasibility of applying standards and methodologies across countries should help support LMICs in achieving more systematic routes for participatory healthcare decision making. It could also inspire more sustainable and equitable developments within the realm of PCI in HTA and decision making for both LMICs and HICs.
Open access
Introduction
Health technology assessment (HTA) is used in many countries to inform decisions about reimbursement and access to interventions based on multidisciplinary processes that use explicit methods to determine the value of a health technology at different points in its life cycle (1). Patients and citizens are key stakeholders affected by HTA and resulting decisions. Hence, their perspectives should be fully understood in the HTA appraisal process (Reference Facey, Ploug Hansen and Single2). Even though evidence relating to clinical effectiveness, cost-effectiveness, and affordability are important elements in HTA, patient-based evidence and patient or citizen input can reflect the experiences, needs, and attitudes with the disease or the health technology as reported by patients, their families, carers, legal representatives, patient organizations (POs), or patient and citizen advocacy groups (PCAGs) (Reference Facey, Ploug Hansen and Single2).
HTA agencies across the globe have been developing models and methods for patient and citizen involvement (PCI) in access, coverage, and policy decision-making processes relating to health technologies in both public and private health sectors (Reference Facey, Ploug Hansen and Single2;Reference Facey3). Such PCI models were spearheaded by HTA agencies from some high-income countries, and examples are described in Facey et al. (Reference Facey, Ploug Hansen and Single2) or in Abelson et al. (Reference Abelson, Wagner, DeJean, Boesveld, Gauvin and Bean4).
Patient and citizen (social) engagement practices are increasingly encouraged by global health policy actors in healthcare-system design, research, and governance at all levels. For example, the World Health Organization released a new handbook on social participation for universal health coverage (in 2020 (5)). The Brazilian CONITEC partnered with several stakeholders to improve its model and methods for social engagement by implementing both social appropriation and knowledge translation in HTA, coverage, and policy decision making (Reference Sousa, Figueiró, Silva, Silva, Silva and Pereira6). The underlying motivation for such PCI initiatives is to allow all social actors with an interest in the management of their individual and collective health (systems), the right to have a voice via institutionalized processes at some or all levels of their national health systems (Reference Sousa, Figueiró, Bicudo and Pileggi7). Moreover, PCI comprises participatory processes that can lead to better health experiences or patient health outcomes, as outlined by the WHO (5) and the UK's government-funded INVOLVE initiative (Reference Hickey, Richards and Sheehy8), which was established in 1996 to support active public involvement in the National Health Service (9).
Additionally, other initiatives sprout along with the breadth of healthcare innovation and delivery, with examples from the life sciences industry (Reference Cook, Cave and Holtorf10), the medical device industry (11), regulatory agencies (12;13), or the grassroots-driven design and conduct of research projects of relevance to patients or their families, especially for rare and familial diseases such as multiple endocrine neoplasia type two (Reference Sousa, Gallian and Maciel14). These patients or their families participate in study participant selection, data analysis, and interpretation of the data collected by them and their family members, and health care and research partners. Hence, researchers, practitioners, and the public join efforts and share responsibilities to develop, implement, monitor, evaluate, and redevelop scientific approaches and technological innovation (Reference Prainsack, Schicktanz and Werner-Felmayer15).
Decision fairness, equity, and patient relevance are concepts that should be true not only for high-income countries (HICs) that are rich in resources and have advanced health(care) systems, but also for low- and middle-income countries (LMICs) (Reference Sousa, Gallian and Maciel14). Many LMICs struggle in finding equitable and sustainable solutions to expand their health(care) systems toward universal health coverage (UHC). A resolution (WHA67.23) from the 67th World Health Assembly underlined the importance of HTA in support of UHC (16). Identifying and understanding pathways for citizens and patients in LMICs, and the varied ways by which they can collaborate with HTA stakeholders for maximizing their health despite scarce resources might help more LMICs in advancing PCI and in creating more sustainable healthcare-decision pathways.
However, PCI in LMICs must be aligned with the countries’ sociocultural, ethical, regulatory, economic, and political perspectives to support attainment of their healthcare priorities, which often differ from those in HICs. To identify how PCI can be realized in the LMIC context, a collaborative initiative of the Developing Countries and Patient and Citizen Involvement Interest Groups (DCIG and PCIG) from the Health Technology Assessment international (HTAi) society set out to explore whether and how LMICs are involving patients and citizens or co-producing patient and citizen (social) engagement in formal and informal HTA processes or in healthcare decision making where HTA is lacking or is at its infancy.
In this paper, we describe the first pilot analyses performed in four diverse countries: Brazil, Indonesia, South Africa, and Nigeria. The long-term objective of applying the questionnaire across LMICs is to uncover aspects of PCI in LMICs related to context and HTA practice, share experiences, and identify best practices on how patients and citizens can make meaningful contributions to the local decision- and policy-making processes in LMIC healthcare environments.
Methods
A questionnaire with fifteen questions was developed by the multi-stakeholder and international project team. The questionnaire (see Supplementary File 1) is structured into three sections: [A] Healthcare system covering the type of healthcare system and healthcare priorities; [B] PCI or advocacy related to any form of influence patients or citizens could take on healthcare decisions; and [C] Healthcare decision making related to its elements including HTA, pricing, or coverage and the degree of patients' influence on each of these elements. The questionnaire used in this pilot study is a refined version evolved from a previous format, which had been developed and tested within the project team and was revised to the current form based on their feedback and difficulties encountered.
To pilot-test the questionnaire format, it was rolled out to four diverse LMICs: South Africa, Brazil, Indonesia, and Nigeria. The selection criteria for these countries were geographical diversity, economic diversity, and willingness of project team members to take the in-country lead. Each questionnaire was completed by one or two project team members from the respective country, who took the lead but also consulted further with patient advocacy groups (PAGs), academics, or the country's Ministry of Health (MoH) for answering some of the questions (full results per country in Supplementary File 2). Different approaches were used in the four countries to confirm the quality of the collected data. In Brazil and Nigeria, the questionnaire was reviewed by the HTA agency and MoH experts, respectively; in Indonesia, the information was confirmed with academic health policy experts; in South Africa, the responses were corroborated with published documents and extensively referenced.
The findings were compiled into one table to allow for comparison and identification of potential similarities and differences.
Each country lead was asked to share their experiences of completing the questionnaire items by responding to five questions: (i) How easy was it to fill the table? (ii) How much time did it require approximately? (iii) How many other people had to be consulted and for what? (iv) What was specifically difficult? (v) Any suggestions for improvement?
The responses were reviewed question by question to inform a potential further revision of the questionnaire before the full rollout.
Results
The pilot results are summarized in Tables 1–3 for each of the three questionnaire sections.
Table 1. Healthcare (HC) system related survey responses
Note: AIDS, Acquired Immunodeficiency Syndrome; HIV, Human Immunodeficiency Virus; HMO, Health Management Organization; HTA, Health Technology Assessment; NCD, Non-Communicable Disease; UHC, Universal Health Coverage.
Table 2. Patient and citizen involvement related survey responses
Note: EDL, Essential Drug List, HC, Healthcare; IP, Intellectual Property; NHC, National Health Council; NCD, Non-Communicable Disease; NGO, Non-Governmental Organizations; NHI, National Health Insurance; PAG, Patient Advocacy Groups; PMB, Prescribed Minimum Benefits; PO, Patient Organizations; PCI, Patient and Citizen Involvement; TRIPS, Agreement on Trade-Related Aspects of Intellectual Property Rights; by World Trade Organization, Pts, Patients; UN, United Nations.
a NGO's providing support to hospitals or communities (training, medicines, etc.).
b Other Countries initiatives to support HC in Nigeria.
Table 3. Patient and public involvement related survey responses
Note: EDL, Essential Drug List, HC, Healthcare; MoH, Ministry of Health; NHC, National Health Council; PCI, Patient and Citizen Involvement.
Healthcare System
Table 1 shows the responses to the question relating to the healthcare system in investigated countries. In all four countries, healthcare funding is provided by a mix of public and private funds as well as out-of-pocket costs paid by patients. The mix of how much is contributed by each of the funding organizations (government, private health care, and insurance) differs between the countries. For example, 80 percent of the population can access publicly funded health care in Indonesia, 12 percent pay out of pocket, and 8 percent are covered through private health insurance. In Nigeria, however, most patients pay out of pocket and only employees of bigger companies or of the government are covered through publicly funded healthcare services; only 1.5 percent of the population are covered through private health insurance. Brazil, on the other hand, offers health coverage via its Unified Health System, which is poor in resources, whereas the more affluent population subscribes to private healthcare insurance schemes.
Maternal and child health (Brazil, South Africa, and Nigeria), infectious diseases such as HIV/AIDS (South Africa), tuberculosis (South Africa), malaria (Nigeria), or general diseases (Nigeria) were also reported as health priorities. In addition, mental health was mentioned in Indonesia and other items such as environmental or work health in Brazil.
UHC was reported by all four countries as healthcare policy priority—with differing stages of implementation—as well as items such as quality improvement (Brazil), improvement in health outcomes in specific diseases (Brazil and Nigeria), re-engineering of primary health care (South Africa), or improvement in financial management (South Africa).
Patient and Citizen Involvement
The responses to the question relating to PCI in HTA decision making in the investigated countries are displayed in Table 2. In Brazil, patients and citizens have some influence through the tripartite National Health Council (NHC) (representing patients, healthcare partners, POs, PCAGs, nongovernmental organizations (NGOs), healthcare professionals, and the health industry sector). In South Africa, patients are formally represented in some policy initiatives but not in the decision-making processes relating to specific health areas or interventions. Otherwise, the influence of POs or PCAGs is, in all four countries, mostly exerted through advocacy and awareness activities or through the legal pathways to get access to therapies (litigation).
Concerning other stakeholders, who may influence healthcare decision making in the country, the respondents mentioned religious communities, advocates, or politicians (Brazil and Indonesia), consumer organizations (Indonesia), civil society groupings (South Africa) (international), and NGOs (Nigeria).
Within the realm of POs, different levels of patient advocacy and activism were reported. In Nigeria, patient engagement is still exceptional, starting with HIV/AIDS and a few organizations engaging for diabetes, malaria, or vaccination. More patient advocacy is observed in Indonesia with POs engaging for oncology (breast cancer and thalassemia), diabetes, stroke, cardiovascular disease, hemodialysis, psoriasis, tuberculosis, or autism. In South Africa, patient advocacy relates to specific diseases (HIV/AIDS, cancer, tuberculosis, NCDs, and mental health) or to the healthcare policy level (access to treatment; use TRIPS in relation to IP; NHI bill, Presidential Health Compact (17)). In Brazil, POs are also actively engaging at the disease and the policy level within the National Congress.
The responses relating to the most likely decision area for the involvement of patients or the public varied from “all” (Brazil), through healthcare policy decisions (South Africa) and reimbursement (South Africa and Nigeria), to influence through advocacy or indirect influence through other stakeholders (Indonesia).
Healthcare Decision Making
A high variability can also be seen for the aspects of healthcare decision making in the four different countries, which are summarized in Table 3. Although HTA is defined in Brazil with published processes and responsibilities, it is either not transparent (Indonesia), only partially applied (South Africa), or nonexisting (Nigeria). Where HTA is done, it is implemented either for only pharmaceuticals (i.e., Essential Medicines List and high-cost medicines at the national MoH level in South Africa, by private health insurance companies for reimbursement decisions in South Africa, or for new and high-cost medicines in Indonesia) or for interventions and/or programs in health care (Brazil).
Several challenges for HTA or evidence-based decision making have been listed by Brazil, Indonesia, and South Africa, collectively, and these challenges are: difficulty in adoption or realization of the HTA recommendations into the healthcare operational levels, a lack of transparency of frameworks, a lack of human resource capacity, capability, and expertise, division between state and healthcare delivery structure (public–private components), and corruption or other routes of impacting decisions beyond HTA.
In terms of where patients most likely could influence technology-related decisions, listing was mentioned in Nigeria, South Africa, and Indonesia, and coverage decisions were mentioned in Brazil, Indonesia, and South Africa. In addition, in Brazil, a potential role for patients was seen in negotiating prices with pharmaceutical companies to increase the alignment of stakeholders and consequently, reduce the level of litigation relating to access to medicines in the country.
Questionnaire Experiences
Easiness of implementation: Two respondents experienced the questionnaire as fairly easy to complete (Indonesia and Brazil) and the other two thought that it was neither easy nor difficult (Nigeria and South Africa).
Time required: It took the respondents between 20 min and 2 h to complete the questionnaire (average 45 min). However, to collect the additional information from other experts (Indonesia, Brazil, and Nigeria) or published material (South Africa) took additional time. Particularly, if it was sought from official sources such as the MoH, it took about 1 month to get the response (Brazil and Nigeria).
Additional expertise required: In Indonesia, two additional people were contacted for more information (one from MoH and one from a PAG). In Brazil, four technical officers from MoH gave input or reviewed the responses. In South Africa, five different stakeholders were contacted to clarify and update the respondent's knowledge. In Nigeria, one member of the MoH, one community pharmacist, one hospital expert, and one PCAG were asked to provide information on government and private sector approaches to health and patient involvement in decision making.
Difficulties: Three respondents felt that it was specifically difficult to seek input from public healthcare organizations due to bureaucratic hurdles (e.g., formal application by an authority requested) and a lack of transparency in this subject. In addition, it was challenging to condense the information on the complex healthcare system into the format of the questionnaire.
A few suggestions for improving the questionnaire were proposed by the respondents. These included, to offer the questionnaire in an online version suitable for collaborative approaches (i.e., several experts or stakeholders can work together to complete the questionnaire and build a consensus on the responses or content), more guidance should be given, for example, which information exactly was required in each of the tables (descriptive and using examples), and a formal support letter from the HTAi might facilitate access to the required information from the MoH. In addition, it was suggested to adopt an additional question relating to the existence, extent, and decision-making process of an “Essential Medicines List” for characterizing the healthcare-system level.
Discussion
Under the postulate that health technologies should ultimately benefit the health of the patient, consideration of the patient or citizen perspective including their needs and experiences with the disease and current or new interventions within their healthcare context should be a formal part of assessing the value of such health technologies alongside those from health professionals who care, implement, and manage the adequate use of those health technologies (Reference Facey, Ploug Hansen and Single2). Although in many advanced and resource-rich healthcare systems, formal value frameworks providing for PCI have been established, less is known about this in LMICs.
This pilot questionnaire was applied to four diverse LMICs and was aimed at understanding how patients are involved in HTA or healthcare decision making in each of these countries.
For the next phase of the project, an improved template will be used by the project team to describe PCI in additional LMICs, whereby the initial focus may be those countries with known incidence of PCI in HTA such as Thailand and other South American LMICs. Identification and selection of the next group of target countries and experts with local knowledge will rely on the HTAi network and the extended networks of the project team members. If possible, the countries will be clustered by certain criteria (e.g., by contextual situations of the healthcare system, epidemiology profile, or the extent of HTA use) to identify patterns. However, in this paper, we have chosen countries, which allowed us to pilot-test the questionnaire's applicability across diverse settings. The downside of this approach is, that, at this stage, the case observations offer only a limited base for drawing general conclusions. Expanded use of the questionnaire will allow for more in-depth analysis. In addition to delivering information on the respective status of PCI in diverse countries, dissemination of the finding of the analyses through publications or conferences will help increase awareness and transparency across stakeholders as well as raise the focus on the routes for PCI in HTA globally.
The responses to this pilot analysis indicate a mixed picture on PCI in the diverse healthcare systems and decision-making processes in the four countries, reflecting the diverse sociocultural, regulatory, political, and economic contexts as those from Brazil (Latin America), Indonesia (Asia-Pacific), Nigeria, and South Africa (Africa).
The first part of the analytical questionnaire tested in this study focused on healthcare-system elements. Healthcare systems in the four pilot countries are diverse in terms of quality, coverage, and efficiency. Although in all of these LMICs, government's priority is to provide coverage for the whole population, the funding mechanism (a mixture of public and out-of-pocket healthcare financing) and the path for improving the quality and achieving efficiency differ. These differences may be partly due to the distinct historical contexts prevailing in these LMICs and due to both social and cultural ways through which their populations engage with health care and technologies locally (Reference Barello, Graffigna, Vegni and Bosio18). There are similarities in terms of priority health areas targeted by the governments such as NCDs, maternal and child health, and infectious diseases. In addition to improvement in health outcomes in specific disease areas, these LMICs are keen to improve efficiency and quality of the health services through the establishment of different mechanisms—more effective healthcare planning and management, community health services, early health education, primary healthcare re-engineering, sound financial structure, and policies. This provides an opportunity for knowledge sharing, not only in disease-specific areas, even more for the mechanisms on improving outcomes or achieving coverage for the population. Engaging with patients and communities could serve as a strong multiplier in implementing these strategies for achieving UHC (Reference Odugleh-Kolev and Parrish-Sprowl19) and for establishing formal assessment approaches for health technologies. This expectation is corroborated through other examples with a proven impact of community engagement on the effectiveness of policy implementation (Reference Makaula, Funsanani, Mamba, Musaya and Bloch20–22).
Except for Brazil, HTA has not yet been formally included to facilitate the decision-making process in the other three countries. South Africa and Indonesia are at different stages of introducing HTA, whereas in Nigeria, it is nonexistent. All countries face challenges, as detailed in Table 3. Although the varying patient characteristics as well as implementation issues should be considered, current HTA methodologies and decision making only partly respond to these requirements and patient experiences or preferences are only rarely considered as has also been observed by Oortwijn et al. (Reference Oortwijn, Determann, Schiffers, Tan and van der Tuin23) when they compared the comprehensiveness of HTA practices around the globe.
To inform healthcare and policy decisions, HTA should take into account the perspectives of the stakeholders with interest in the decision to ensure contextual and democratic validity, legitimacy, and fairness of the recommendations (Reference Oortwijn, Determann, Schiffers, Tan and van der Tuin23;Reference Facey, Bedlington, Berglas, Bertelsen, Single and Thomas24). To assess the intended, unintended, or indirect effects, feasibility of implementation, as well as barriers and facilitators, engagement with all stakeholders concerned is necessary, including patients and citizens (Reference Oortwijn, Determann, Schiffers, Tan and van der Tuin23). The broader rollout of the questionnaire presented and tested in this study may help identify different pathways to PCI in LMICs as well as barriers and facilitating factors. Making the examples and key success factors visible may help LMICs to learn from one another. From the four LMICs in our study, only Brazil has started to establish formal ways for involving patients in the HTA process, with further improvement required (Reference Sousa, Figueiró, Silva, Silva, Silva and Pereira6). In general, patient engagement currently is observed only in disease-specific areas and is dependent on the prominence of the different PCAGs or POs. Even though Brazil has a formal structure for involvement through the National Health Council, PCAGs, and other advocacy groups such as religious parties play an important role in all levels of the healthcare system and try to enforce their (perceived) rights (e.g., access to a specific therapy). In all countries analyzed, the involvement of religious communities (Indonesia) and consumer associations and communities (Indonesia and South Africa) or the participation of civil society has emerged corresponding to the cultural, religious, and societal context of patients and the public in these countries. Such pathways could contribute patients’ and citizens’ perspectives to HTA and healthcare decision making.
Various examples of inclusion of patient perspectives in HTA have been reported from HICs. CADTH, in Canada, has a formal well-structured system for incorporating patient input on health outcomes and issues in both the assessment and appraisal phases (Reference Abelson, Wagner, DeJean, Boesveld, Gauvin and Bean4;Reference Berglas, Jutai, MacKean and Weeks25). In Germany, patients/patient representatives participate through a consultation process by IQWIG during the analysis phase, as well as in the generation of reports and dossier assessment and guidance for the “involvement of people affected” (https://www.iqwig.de/en/getting-involved/contributing-the-perspective-of-patients-and-other-affected-persons.3070.html). In our study, the Brazilian response also describes a consultation process as an opportunity for patient input. The National Institute for Health and Care Excellence (NICE) in England has a policy for patient and carer involvement in its decision making and provides direct support, resources and training to patients and POs who participate in the NICE's work (https://www.nice.org.uk/about/nice-communities/nice-and-the-public/public-involvement/public-involvement-programme/patient-public-involvement-policy). Many other countries such as Sweden, Taiwan, Australia, Denmark, or Italy have foreseen the involvement of patients or the consideration of patient-based evidence for certain aspects of their HTA processes (Reference Facey, Ploug Hansen and Single2).
Such participatory initiatives are also being designed for the Brazilian context, as reported in our study, and observed elsewhere (Reference Sousa, Figueiró, Bicudo and Pileggi7), whereas the other three countries fall short in this aspect. Although the POs demand to be heard in all four LMICs under study, other pathways for contributing with the patient experiences and knowledge to HTA and decision making are used, mostly through advocacy rather than formal involvement pathways. Mostly, PCAGs engage for increasing awareness of the disease burden, advocate for the adoption of certain technologies, or use legal pathways to compel arrangement for the coverage of certain treatments. More formal and active stakeholder involvement may increase their acceptance and agreement with HTA recommendations and the final decisions.
Limitations
This was the first time the questionnaire was rolled out to the first four LMICs under the leadership of project team members. For a broader rollout, it will be important to ensure that the questionnaire is self-explanatory and easy to use in order to receive useful and relevant content. All questionnaire leaders thought that the format was acceptable and easy to use. However, a broader rollout may require more guidance and definition.
In addition, as we have analyzed only four diverse countries, no general conclusions are possible with such a limited data set. A broader rollout of the revised and improved questionnaire will be necessary to derive more representative information.
The questionnaire helps describe complex issues related to the healthcare system and the degree of PCI in healthcare decision making with direct implication on PCI in HTA. In addition, different stakeholders may have different experiences and viewpoints, especially relating to PCI. It is unlikely that one individual has all expertise to fill the questionnaire alone and to describe the whole spectrum of viewpoints. Therefore, for further rollout of the questionnaire, we will encourage collaborative approaches in assembling the information and recommend web-based collaboration to do so.
Conclusions
All respondents commented that a strong support structure from policy and decision makers is required, which involves raising awareness and advocating for patient and citizen participation. In addition, good practices should be followed such as those defined by the PCIG (https://htai.org/interest-groups/pcig/values-and-standards/) or other relevant organizations. The establishment of a functional framework for PCI in HTA and thus the decision-making process can be accelerated by networking, learning from, and collaborating with ongoing international PCI initiatives and by adapting the tools and material for local use and application. As mentioned in the Introduction, it can be expected that an understanding of contextual factors, for instance, population size, wealth level, disease burden, progress toward UHC, or other relevant indicators need to be considered in LMICs. Further rollout of the questionnaire across a broader range of LMICs will help better define these requirements. Rolling out the questionnaire to as many countries as possible should help categorize and cluster them and compare their PCI strategies with other LMICs as well as those from HICs. This provides an opportunity for actors, who work across both HICs and LMICs to understand the differences and to select and tailor solutions that match the LMIC perspective.
Comparing possible processes of PCI, a broader understanding of stakeholder relationships, as well as the feasibility of applying standards and methodologies across countries should help support LMICs in achieving more systematic routes for participatory healthcare decision making. It could also inspire more sustainable and equitable developments within the realm of PCI in HTA and decision making for both LMICs and HICs.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462320002263.
Acknowledgments
The work for this manuscript and the underlying research resulted from the project on patient and citizen involvement in low- and middle-income countries in the frame of the Patient and Citizen Involvement Interest Group of the HTAi. LP is a director of PNME Consulting (Pty) Ltd T/A Prime Consulting. SAS was a consultant of the Ministry of Health of Brazil (Secretariat of Science, Technology and Strategic Inputs, Department of Management of Health Technology and Innovation Incorporation). APH is employed by Health Outcomes Strategies GmbH and is a member of the HTAi PCIG. We would like to thank Maria J. Vicente-Edo (Spain), Rosiane Simeon PhD (c), MPH (Canada, Haiti), John V Amanya (Uganda), and the peer reviewers for critically reviewing and helping to improve the manuscript.
Funding
This research received no specific funding from any agency, commercial, or not-for-profit sectors.