CHD represents a broad range of conditions, spanning from low to high complexity. Surgical and catheter interventions, required from the early neonatal period through to adulthood, contribute to high resource utilization and cost in the care of CHD patients across the life span. Reference Burstein, Shamszad and Dai1–5 Improved medical management and procedural outcomes have allowed most children with CHD to survive into adulthood. Reference Oster, Lee, Honein, Riehle-Colarusso, Shin and Correa6–Reference Mandalenakis, Rosengren, Skoglund, Lappas, Eriksson and Dellborg9 However, living with chronic disease affects all facets of daily life, including physical ability, success in education and employment, financial stability, mental health, and quality of life. While acute medical outcomes and complications have become routinely measured and effectively benchmarked through collaboratives such as those represented in Cardiac Networks United, longitudinal outcomes of greatest interest to patients and families have not been systematically measured. Reference Laussen10–Reference Marino, Lipkin and Newburger13
The International Consortium of Health Outcomes Measurement is a non-profit organization driving patient-centred outcomes measures within conditions to create better healthcare value. The International Consortium of Health Outcomes Measurement was formed to increase value in the highest burden medical conditions worldwide by creating multi-stakeholder informed sets of feasibly measured outcomes for those conditions. Since inception, there are dozens of sets across medicine with utilization in six continents and a particular focus on outcomes that are measurable with low- or middle-income countries. 14 All sets are free of use and publicly accessible.
Following initiation of a general paediatric set of outcomes, the CHD set was one of the first paediatric working groups to form, including cardiologists, surgeons, researchers, social workers, nurses, patients, and patient representatives. Reference Hummel, Whittaker and Sillett15 This group of experts created a robust yet feasible set of outcomes that was externally validated and disseminated within the field and represents the most comprehensive collection of patient-reported outcomes in the CHD population. 14–Reference Benning, Das-Gupta, Sousa Fialho, Wissig, Tapela and Gaunt16 The set combines case mix variables and patient-reported outcomes, the latter of which include the quality of and satisfaction with life, arrhythmia burden, school and work productivity, financial impact, depression, anxiety, and others.
Conditions targeted by the International Consortium of Health Outcomes Measurement sets are those within integrated practice units, in which a multidisciplinary team follows patients longitudinally throughout the lifespan of their condition. Reference Porter and Lee17 The majority of patients with CHD receive care within integrated practice units, in the form of “Heart Centres,” which represent an opportunity for full cycle-of-care outcomes measurement given the breadth of care received within the same system, from diagnosis through treatment and during times of stability. Reference Porter and Lee17–Reference Chowdhury, Johnson and Baker-Smith20
Implementation of the CHD standard set has been challenging. In this work, we describe our pilot implementation efforts in a generally well population of children and adults with diverse CHD undergoing pulmonary valve replacement. The aim was to assess contextual barriers and facilitators of implementation in a cohort of patients with outcomes that have potential for modification, and outcomes related to equitable care, in alignment with institutional goals.
Methods
In a large, quaternary, freestanding children’s hospital, which routinely cares for both children and adults with CHD, patients scheduled for surgical or catheterization pulmonary valve replacement between February 2022 and December 2022 were screened for eligibility. Inclusion criteria were (1) patient aged ≥8 years at the time of pulmonary valve replacement and (2) patient/family reading fluency in English at 3rd grade level or higher. Patients were known to be pending catheter-based pulmonary valve replacement by an existing process, with project personnel (research assistant or cardiology fellow) included on a weekly email thread notifying valve vendors of upcoming cases for the following week. The few cases of surgical repair were excluded based on low numbers. The project was approved by the Boston Children’s Hospital Institutional Review Board.
The International Consortium of Health Outcomes Measurement set was developed across all age ranges of CHD, a heterogenous population, with specific outcomes and tools respective of age. In this pilot of teenagers and adults with a subset of CHD, the International Consortium of Health Outcomes Measurement subcategories chosen included arrhythmic complaints, financial burden, school and work absence, work productivity, anxiety, depression, quality of life, and perceived health status. The International Consortium of Health Outcomes Measurement CHD standard set recommends specific tools for some subcategories such as the General Anxiety Disorder-7, Patient Health Questionnaire-9 for depression, PROMIS Global Health Scale, Pediatric Cardiac Quality of Life, Satisfaction with Life Scale, and the Work Productivity and Activity Impairment Questionnaire. Scoring for these specific tools was used in the pilot. For questions within the International Consortium of Health Outcomes Measurement set that do not have validated scoring systems (either due to lack of available validated tools or because of high burden or cost of use for the tools), the total allowable score was derived from even distribution of the tools. These individual questions, replicated from the International Consortium of Health Outcomes Measurement standard set, related to arrhythmia, financial burden, and work and school absence are shown in Table 1.
Table 1. Survey questions adopted from ICHOM CHD standard set
ICHOM = International Consortium of Health Outcomes Measurement.
* Number of patients recorded in each category represent survey responses received on the initial survey.
The set was populated in a REDCap survey. Reference Harris, Taylor, Thielke, Payne, Gonzalez and Conde21
Patient approach occurred in three phases sequentially over time. All patients received an email via REDCap 1 week prior to pulmonary valve replacement. This was followed with the first cohort who received an in-person introduction to the International Consortium of Health Outcomes Measurement during the pre-procedure clinic appointment after which a patient-reported outcome survey link was sent to a mobile phone, the second cohort who received no additional communication, and a third cohort who received a phone call or text message to complete the patient-reported outcome survey within 24 hours of the email being sent if the survey was not completed. Selection of which cohort was strictly sequential in time. The pre-procedural visit is several hours long with interspersed testing, allowing ample time for survey completion between tests. Patients who completed the initial pre-procedure survey were sent the same survey 3 months following their procedure via email from REDCap. Patients were sent a follow-up email reminder 24 hours after the first notification if the survey remained incomplete. If no response was received 1 week later, personnel would call patients/families from a hospital-secure phone number encouraging completion. If still no response was received, personnel would send a secure text message from the hospital via the Doximity application. Personnel time included creating the REDCap survey, inputting selected patients into a REDCap database for automated email transmission, in-person introduction to the International Consortium of Health Outcomes Measurement, and phone call or text messaging in follow-up to the email.
Observational data analysis was completed comparing survey responses via means, medians, interquartile ranges, and simple percentages. Implementation analysis was conducted via a hybrid type 2 design, to determine the effectiveness of data collection while attempting to determine the feasibility of the implementation strategy employed. Reference Curran, Bauer, Mittman, Pyne and Stetler22,Reference Landes, McBain and Curran23 Our institutional review board approved this study; the consent was not obtained as data were collected as part of a quality improvement project.
Results
Of the 74 patients undergoing pulmonary valve replacement during the study period and meeting inclusion criteria, 32 (43%) completed initial surveys following the approach by the different methodologies: email followed by in-person explanation (n = 16/25 patients undergoing pulmonary valve replacement approach, with response rate 100%), email only (n = 13/25 patients undergoing pulmonary valve replacement approach, with response rate 54%), and email followed by text or call (n = 17/24 patients undergoing pulmonary valve replacement approach, with response rate 64%). Cumulative response and capture rates by the different implementation strategies are as follows: of the 32 patients who completed the initial survey, 9 (28%) completed the 3-month follow-up survey. Patient demographics are shown in Table 2.
Table 2. Patient demographics and CHD
IQR = interquartile range.
* Data above representative of all patients (27 of 32) who could be identified in REDCap for demographic data collection.
Mean age at the time of pulmonary valve replacement was 30 years (range 8–67 years), with a majority of patients in their late teens and early adulthood. The majority of patients were White, non-Hispanic (n = 18, 67%), and 41% had private insurance (n = 11). Underlying CHD was predominantly tetralogy of Fallot, with a smaller number of other right-sided obstructive lesions and only two patients with right ventricular outflow reconstruction for management of underlying left-sided obstructive lesion.
Pre-procedurally, 34% of patients reported having symptomatic arrhythmia. The change in arrhythmia burden for the nine patients who completed the post-procedure survey is as follows: five patients who had previously reported symptoms, the symptoms resolved for four patients following their pulmonary valve replacement, and one patient whose symptoms persisted. Of patients who had not reported pre-procedural arrhythmia burden in the preceding 30 days, only one developed symptoms following pulmonary valve replacement, and the majority remained asymptomatic. Health-related impact on school and work functioning is shown in Figure 1.
Figure 1. Productivity burden: health-related impact on school absenteeism (a), work absenteeism and hours (b and c), and work and daily activity productivity (d), prior to pulmonary valve replacement.
For children enrolled in school, 43% missed at least 6 days of school in a given year due to their health, consistent with the World Health Organization definition for significant school absenteeism. More than half of adult patients experienced work absenteeism in the preceding 7 days related to their health, and the majority worked less than 40 hours in the past 7 days. Financial concerns were reported by 34% of patients and families related to the CHD diagnosis.
Prior to pulmonary valve replacement, half of the patients indicated being extremely satisfied with their life (n = 16, 50%, Fig. 2), and none reported being dissatisfied or extremely dissatisfied. Similarly, a majority of patients scored in the “good” and “very good” categories (n = 15, 52%, and n = 11, 38%, respectively) on health-related quality of life scale. No patient indicated scores in “fair,” “poor,” or “very poor” range. Pre-procedural anxiety and depression data are shown in Figure 2.
Figure 2. Satisfaction and mental health: pre-procedural PRO data; (a) satisfaction with life, (b) health-related quality of life, (c) anxiety severity, (d) depression severity. PRO = patient-reported outcome.
Anxiety symptoms were reported in 62.5% (n = 18), with most being minimal or mild; 6% of patients had moderate or severe anxiety symptoms. Of all respondents, 84% (n = 27) of patients reported depression symptoms, though they were all minimal or mild based on the Patient Health Questionnaire-9 scoring. Of the nine patients who completed post-pulmonary valve replacement surveys, 75% (n = 6) had improvement in both anxiety and depression symptoms, from mild to minimal or minimal to no symptoms.
Discussion
Foundational outcomes of CHD palliation and treatment, such as survival and major complications, have been importantly improved over the past decades. However, patient-reported outcome measures as in the International Consortium of Health Outcomes Measurement CHD set are increasingly what impact patients on a daily basis. The International Consortium of Health Outcomes Measurement CHD standard set is the first known, comprehensive, patient-vetted patient-reported outcome package intended for regular clinical care across the life span of patients with CHD. This project represents a first step in the process of implementing the International Consortium of Health Outcomes Measurement CHD collection of outcomes in clinical practice. This peri-procedural pilot was successful in the following: (1) presenting previously unknown data within longitudinal care of patients with CHD, (2) identifying workflow needs and challenges for future success in the implementation of outcomes measurement, and (3) demonstrating the importance of patient/family patient-reported outcomes in the longitudinal care of patients with CHD.
In applying the International Consortium of Health Outcomes Measurement CHD standard set to patients pending pulmonary valve replacement, we have learned novel information about the patient experience of living with CHD. Our patients have high satisfaction with life. They face financial burdens related to their disease and challenges in school and work attendance and productivity. Some face anxiety and/or depression. Many experience arrhythmia symptoms, which can change with procedures. Some facets of the International Consortium of Health Outcomes Measurement CHD standard set, such as quality of life, anxiety, and depression, have been studied in isolation in patients with CHD. Reference Wernovsky and Licht12,Reference Marino, Lipkin and Newburger13,Reference Reiner, Oberhoffer and Ewert24,Reference Gonzalez, Kimbro and Cutitta25 The comprehensive International Consortium of Health Outcomes Measurement set allows a holistic understanding of our patients. Specific variables were incorporated into the CHD set at patient recommendation, including longitudinal arrhythmia burden, school and work absenteeism, and financial stressors.
The pulmonary valve replacement population was chosen as a pilot cohort in part because of the use of outcomes metrics in comparative effectiveness research, with the idea that pre- and post-assessment of patient-reported outcomes may help distinguish benefits between surgical and catheter-based pulmonary valve replacement. However, within the institution and across the field, momentum and patient preference had already shifted to catheter-based intervention prior to roll-out of the study. In the preceding year 55% of patients underwent surgical versus 45% catheter-based intervention; in the pilot year, 94% of patients underwent catheter-based intervention. This highlights the importance for full cycle-of-care patient-reported outcome measurement in an ever-evolving field such as CHD to compare the benefits and drawbacks from relatively new interventions. The use of patient-reported outcomes, and specifically the International Consortium of Health Outcomes Measurement standard sets in other conditions, has driven intervention decisions, particularly in the case of prostate and cataract surgery. Reference Queirós, Redondo and França26,Reference Evans, Millar and Moore27
While the pilot demonstrates success in collecting patient-reported outcome data on the patient level, a major finding of the project was identifying barriers to system-wide implementation. A primary barrier appreciated is the workforce efforts required to approach individual patients. The process outlined, requiring a research assistant or ancillary team member to approach patients, is not sustainable for system-wide patient-reported outcome data collection. The International Consortium of Health Outcomes Measurement sets of outcomes are intentionally compiled to not include outcomes metrics that require a proctor and can be completed in totality by the individual patient. Given this, REDCap was utilized as a mechanism to provide direct email links to the study survey to allow completion. While this was logistically simple, survey methodologies suggest text messaging triggers as an acceptable source of data return, if not done in person with the care team. Reference Guo, Kopec, Cibere, Li and Goldsmith28
As attention nationally has begun to shift to the importance of patient-generated data, information technology organizations have arisen to offload this burden; information technology expertise, as these data grow, offers opportunity for machine learning and artificial intelligence with predictive outcomes modelling. It may benefit systems or teams seeking broad and robust patient-reported outcome data to utilize information technology expertise in this way, either from within the institution or via outside products. Specific opportunities to operationalize data collection separate from individual research projects (with research coordinators and funding) may come from lowering the threshold for patients to provide short, simple, open-ended feedback. For example, time-directed text-based questions of symptoms prior to procedure or appointment dates, rather than exclusively fully validated surveys, may provide clinically relevant data. Investment from stakeholders who stand to benefit from improved patient communication (institutions, health systems, payers) will likely be necessary to fully operationalize this process outside of research.
A second barrier was deciding on the appropriateness of retrospective versus real-time discussion of results with patients. Operationalizing patient-reported outcomes such as depression scales poses risk of a response suggesting the need for acute intervention for the patient. If there is not a “red flag” mechanism or real-time evaluation of the survey, teams could miss time-sensitive concerns from patients. This risk is mitigated by following outlined techniques described by the National Institute of Health. Reference Pearson, Stanley, King and Fischer29–31 Within the pilot process described—and within the International Consortium of Health Outcomes Measurement set as a whole—validated tools were used that have been evaluated to not include “red flag” questions, including the Patient Health Questionnaire-9 and General Anxiety Disorder-7 for depression and anxiety.
A final barrier implicit to survey-based research and clinical operations utilizing feedback is response rates. While the strategies piloted elicited response rates exceeding those of routine patient feedback scales, the response rate to follow-up remained limited at only 28%. Critical to garnering feedback from patients to improve their care, and that of the system, is incentivizing patients to provide feedback. While some percentage of patients will share feedback for the good of the system, to yield higher results requires a forward feedback loop that uses data patients provide to improve their own care. Leverage data systems and integrated teams so that individual patient providers can use patient feedback in the care of those patients can incentivize patients further than simply using the data for the gradual improvement of the system. It is critical for patient-reported outcomes (and more broad patient feedback) to be usable to providers to allow integration into the clinical realm.
These lessons and findings through the implementation phase of this pilot will be crucial to address for teams and systems integrating patient-reported outcome measurement in their care processes.
Limitations
While barriers identified were a key outcome of this project, there were limitations to the study. The cohort size, while expected to be small in a pilot study, limited statistical analyses from comparing different groups of patients, including those of different race/ethnicities. This study was also conducted during shifting environments due to the coronavirus disease 2019 pandemic, influencing the availability of in-person study team members and patients. The International Consortium of Health Outcomes Measurement CHD set is intended for use during annual evaluations rather than peri-procedurally. The use of this set during annual cardiology care may indeed enhance data capture and allow for further engagement of the clinical team in real-time discussion of data, as well as longitudinal tracking of patient responses.
Conclusions
This study was successful in its pilot nature to implement patient-reported outcomes within the CHD population effectively to generate new data. While the feasibility of these outcomes was assessed in this study in the relevant population, next steps require vertical implementation across the cycle of care, horizontal implementation across other sub-populations of CHD, as well as comparisons and benchmarking across centres caring for CHD patient populations. Beyond implementations and measuring outcomes, care strategies of our CHD populations need to evolve to ease patient burden in domains not routinely addressed.
Acknowledgements
The authors acknowledge the support of the International Consortium of Health Outcomes Measurement on the initial development of the CHD set of outcomes, as well as the Boston Children’s Hospital patients and families.
Financial support
None.
Competing interests
None.
