Evaluating adoption and reach in a pragmatic randomized trial of community paramedicine for intermediate acuity patient care

Abstract

Introduction:

Pragmatic trials aim to speed translation to practice by integrating study procedures in routine care settings. This study evaluated implementation outcomes related to clinician and patient recruitment and participation in a trial of community paramedicine (CP) and presents successes and challenges of maintaining pragmatic study features.

Methods:

Adults in the pre-hospital setting, emergency department (ED), or hospital being considered for referral to the ED/hospital or continued hospitalization for intermediate-level care were randomized 1:1 to CP care or usual care. Referral and enrollment data were tracked administratively, and patient characteristics were abstracted from the electronic health record (EHR). Enrolled patients completed baseline surveys, and a subset of intervention patients were interviewed. All CPs and a sample of clinicians and administrators were invited to complete a survey and interview.

Results:

Between January 2022 and February 2023, 240 enrolled patients (42% rural) completed surveys, and 22 completed an interview; 63 staff completed surveys and 20 completed an interview. Ninety-three clinicians in 27 departments made at least one referral. Factors related to referrals included program awareness and understanding the CP practice scope. Most patients were enrolled in the hospital, but characteristics were similar to the primary care population and included older and medically complex patients. Challenges to achieving representativeness included limited EHR infrastructure, constraints related to patient consenting, and clinician concerns about patient randomization disrupting preferred care.

Conclusion:

Future pragmatic trials in busy clinical settings may benefit from regulatory policies and EHR capabilities that allow for real-world study conduct and representative participation. Trial registration: NCT05232799.

Introduction

Pragmatic trials are designed for and conducted in real-world settings to generate relevant and generalizable findings and to speed translation to routine practice settings [1–3]. Operational and research procedures embedded in clinical infrastructure and workflows are meant to generate practically useful, real-world evidence [4,5]. They may also support research equity by reducing barriers to participation, thereby increasing diversity in trials [6–8].

Unlike explanatory trials that limit participant eligibility to a narrower set of patients than would normally be served by the examined intervention, pragmatic trials enroll patients that are similar to those in usual care settings [2].

However, the pragmatic approach also has important challenges, including infrastructure requirements to support point-of-care trial enrollment and concerns among clinicians about potential burden on clinical workflows or limitations on care autonomy [9,10].

The field of implementation science is similarly focused on reducing the time it takes to get evidence-based practices (EBPs) into routine clinical settings [11,12]. Implementation science researchers use a variety of study designs, including pragmatic trials, to implement and evaluate EBPs in routine care settings. Exploration of implementation factors during a trial of intervention effectiveness (i.e., a hybrid type 1 study design) can provide insights into whether the intervention and study procedures can be implemented as intended and inform strategies to support implementation success [13–15]. This includes whether and how to recruit and enroll clinicians and patients in a pragmatic and equitable way. This manuscript reports the successes and challenges of recruiting and enrolling clinicians and patients in a pragmatic randomized trial of a community paramedicine (CP) program to reduce acute care utilization [16].

Methods and materials

Setting and intervention

This study took place at Mayo Clinic Rochester and Mayo Clinic Health System (MCHS) sites as part of the Care Anywhere with Community Paramedics (CACP) trial (NCT05232799) [16]. Mayo Clinic is an integrated healthcare delivery system with an academic and clinical hub in Rochester, Minnesota. MCHS is a regional network of community clinics and hospitals serving 54 communities across Southern Minnesota and Western Wisconsin.

Often, patients are hospitalized for intermediate acuity conditions or for conditions that require limited interventions because the necessary level of care is not available in the home. CPs are emergency medical professionals with specialized training in delivering preventive care, chronic disease management, and intermediate-level care (e.g., wound care and intravenous medication administration) in community and home settings [17,18]. While CP programs have been implemented for chronic disease management in the home setting, the CACP trial assessed effectiveness of a CP program for patients who required an intermediate level of care that would otherwise be provided in the emergency department (ED) or hospital [16]. Eligible patients were adults aged ≥ 18 years being treated in the pre-hospital setting (e.g., outpatient clinic, home), ED, or hospital, and who were being considered for referral to the ED/hospital or for continued hospitalization because they required an intermediate level of care (e.g., intravenous diuretic, fluid electrolyte, or antiemetic administration in hemodynamically stable patients; wound care and wound vac management requiring skilled services; daily or twice daily monitoring of laboratory and vital sign parameters while receiving treatment for an acute condition) not otherwise available to them in the home setting but which could be managed by CPs. The trial was open to patients within the CP service area at the time of the trial (approximately 40 miles surrounding Rochester, Minnesota, or in the catchment area of MCHS in Northwest Wisconsin [NWWI]).

Pragmatic study design

The CACP study was a pragmatic 1:1 randomized controlled trial of the effectiveness of CP care in the home setting (versus usual care). Patient outcomes were related to intervention components and their hypothesized mechanisms of change, including access to home-based care (e.g., medication management, wound care), self-management education, and care coordination, which are expected to improve access to care, knowledge, quality of life, and satisfaction. The primary outcome was days alive outside the hospital or ED within 30 days following randomization. Secondary outcomes included 30-day return to the ED, 30-day return to the hospital, 30-day mortality, patient satisfaction, and patient-reported quality of life and treatment burden. Implementation outcomes were guided by the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) implementation evaluation framework [19–22]. Figure 1 presents the intervention components (i.e., CP services), as well as patient and implementation outcomes. This report focuses on adoption and reach outcomes.

Figure 1. Pragmatic study features, intervention components, and outcomes.

Pragmatic trial features related to intervention delivery and outcomes, as well as to participants, are also displayed in Figure 1. Intervention delivery and implementation were planned to be as pragmatic in design as possible, using guidance from the PRECIS-2 tool, which was developed to help trialists consider the features of their trial on an explanatory-pragmatic continuum from 1 (very explanatory) to 5 (very pragmatic) in nine domains [2]. This report focuses on domains related to eligibility and recruitment, as these may be most relevant to trials where enrollment happens at the point of care. They are also most relevant to two known barriers to trial conduct that happen at the point of care: lack of pragmatic trial infrastructure and perceived disruptions to clinical workflow. Average scores for all nine PRECIS-2 domains for the CACP trial are shown in Supplementary File 1, and brief rationale for each score is included in Supplementary File 2. Scoring was completed independently by nine members of the study team and then averaged. Scoring took place after presentation of the PRECIS-2 toolkit and a discussion during which team members were given an opportunity to ask questions about each domain’s definition.

Patient recruitment and consent were planned to be flexibly conducted at the point of care to encourage broad representation. Clinical settings were diverse and included several primary care (internal medicine and family medicine), medical and surgical specialty care, and acute care settings, including locations with infection prevention controls.

Patient inclusion and exclusion criteria were set to allow broad representation of all adult patients likely to receive CP services in the future, including rural dwelling and low-income patients, patients with psychosocial challenges, patients with English language or cognitive limitations, and those with infection control precautions. Eligibility criteria were identical to those for patients in usual care with two exceptions: 1) inability of the patient or their legally authorized representative to provide informed consent; and 2) prior enrollment in the trial. Patients experiencing homelessness were not offered enrollment into the trial because their usual source of medical care in Rochester, Minnesota, is a community paramedic-run clinic [23]. The program did not charge insurance for CP care, so insurance coverage was not considered in eligibility criteria.

Potentially eligible patients were referred by their healthcare team at the point of care and reviewed by the CP coordinator to ensure appropriateness for CP care with a focus on patient and CP safety as care would be delivered outside the clinical setting and without medical monitoring or support beyond that available from the CP during the CP visit. After review, eligible patients were approached by a member of the research team, who reviewed the program and completed informed consent before randomization, ensuring eligible patients were given an opportunity to ask questions and decide if they would like to participate. As shown in Table 1, consent could be obtained in person, by telephone, or by video conference, depending on patient and study team member location and based on requirements of a greater-than-minimal-risk study, as enforced by the Institutional Review Board (IRB). The randomization decision was immediately communicated to the patient, the referring team, and the CP service, so they could adjust their clinical care plans accordingly. The study was approved as greater than minimal risk with the requirement of written informed consent by the Mayo Clinic IRB (IRB# 21-010816). This study was conducted according to our published protocol [16] and is reported using the Standards for Reporting Implementation Studies (StaRI) Statement) [24].

Table 1. Examples of consent procedures based on population and setting

Data collection and analysis

Program metrics and patient demographics

This report focuses on two implementation outcomes relevant to recruitment and enrollment in pragmatic trials (reach and adoption), where reach was defined as the number, proportion, and representativeness of the patients willing to participate in CACP, and adoption was defined as the representativeness of settings and clinicians willing to initiate a referral to CACP. Dates and settings of referrals, referral decision (and reason for decline), and mode of informed consent were tracked in an enrollment administrative dataset. Baseline patient demographics were abstracted from the electronic health record (EHR). Data were summarized descriptively and compared to those of the empaneled population of the affiliated community internal medicine and family medicine practices (seen in the last 3.5 years) to assess representativeness of enrolled patients. All patients paneled at a Rochester or NWWI location on the first date of enrollment of the trial were included in analyses.

Surveys

All enrolled patients were recruited to complete paper surveys at baseline. At the end of the enrollment period, all CPs (n = 8) and a sample of clinicians (n = 94) and administrators (n = 9) were invited by email to complete an electronic survey. Clinicians (physicians, advanced practice professionals, nurse case managers, and social workers) were invited to complete a survey if they placed an order for CP care through the CACP program, provided primary care, or were the assigned inpatient clinician to at least two patients who were enrolled in the program. All clinicians invited to complete an interview were also invited to complete a survey. Surveys were analyzed using descriptive statistics.

Interviews

At the end of their study participation, all CPs and a sample of clinicians, administrators, and patients in the CACP intervention arm were also invited to complete an individual semi-structured interview by telephone or video conferencing software. Interviews were audio-recorded and transcribed for analysis. Transcripts were analyzed using methods of content analysis and a coding framework guided by constructs from RE-AIM.

Results

Between January 2022 and February 2023, 240 patients were enrolled and completed baseline surveys, and 22 patients in the intervention group completed an individual interview. Sixty-three staff (clinicians [53% response], CPs [88% response], and others [67% response]) completed an end-of-study survey, and 20 completed an interview (6 CPs, 12 clinicians, 2 administrators).

Reach

A total of 323 patients were assessed for eligibility, and 83 were excluded (including 48 excluded by staff for not meeting inclusion criteria or because of program capacity limits, 10 because of a status change, and 25 because the patient declined). Reasons that patients declined enrollment included feeling like they had enough care assistance in the home already, not wanting people in their home, feeling like there were already too many appointments, and general disinterest in the program. Most enrolled patients were referred from the Rochester service area and from the hospital setting, as shown in Table 2. Among those enrolled, 173 (72.1%) consented face-to-face using paper consent documents, 39 (16.3%) consented using electronic/phone procedures, and 28 (11.7%) consented using paper/phone procedures with a paper consent document provided to them by a local agent.

Table 2. Enrolled patients by setting and region

The broad inclusion criteria resulted in a study population that was diverse on several characteristics. Although most patients were enrolled in Rochester settings, enrolled patients were 42% rural dwelling. Intervention reach demonstrated representativeness of patient demographics in the service area, as shown in Table 3, including in terms of rural residency. The study population was older than the empaneled primary care population and had significantly higher mean comorbidities. They also had more social risk factors.

Table 3. Characteristics of enrolled patients and primary care empaneled patient population

* Rural residence was classified using Rural-Urban Commuting Area codes (codes 4-10) from the US Department of Agriculture Economic Research Unit. ** Domains from the Epic Social Determinants of Health (SDOH) questionnaires as collected through the Epic EHR for all empaneled patients. Only reported for patients with completed questionnaires.

Factors that were important in their decision to enroll in the trial – captured on baseline surveys – are shown in Table 4 and include distance to clinic or hospital and burden of care that may fall on family or friends. Distance to the clinic or hospital was the only factor that was significantly different between rural and urban patients. In interviews, patients described considering the convenience of care at home and caregiver availability, as noted by this participant: “I lived alone, so I did not have someone else to be able to do [wound care]. I was unwilling and unable to go to a skilled facility because the nearest ones that could do both physical therapy and my dressing changes would have been a step-down unit, which would have been in [distant communities]. So, all of my resources, all of my family, all of my friends would have been here in Rochester and not out there. And I also have a significant other who is [elderly], and I had had him in an assisted living here in Rochester; and things were going poorly there, so I needed to be able to intervene where I could, long distance as it might be, but it’s still here in Rochester.”

Table 4. Factors important in decision-making

* Survey question: “In thinking about your preference, what factors are important to you in deciding between hospital care and in-home care with visits by a community paramedic?”

In interviews, clinicians described referring patients with diverse clinical characteristics, as well as using the program when there were no other options, as reported by this clinician: “I try to focus on the patients where the community paramedics can meet their needs based on the inclusion criteria that they have developed. But I will say that, oftentimes, they are the patients that aren’t being accepted to or can’t get the care elsewhere. So, community paramedics become kind of the last step to see if they could help them, which then kind of creates some complexities for the community paramedic team.” While several clinicians in interviews noted lack of clarity about enrollment criteria, they described medically complex patients as likely ideal candidates, for example, “I am not quite sure about the age inclusion/exclusion criteria. I wasn’t sure if it was just for older adults. My impression was that they served anyone in the community who needed it. So, I feel like older adults, particularly, probably would use it more just because they’ve got higher health complexity. But ideally, anyone who would qualify for post-acute care or some sort of having someone lay eyes on them, even just once or twice to make sure that all their social needs are addressed.”

Adoption

Adoption was concentrated among clinicians in the hospital setting, but a total of 93 clinicians (including case managers and nurse practitioners) in 27 departments had at least one referral. In addition to larger numbers of referrals from hospital internal medicine, there were also several referrals from primary care and cardiology, as well as referrals from emergency medicine, hematology/oncology, orthopedics, palliative medicine, and gastroenterology. Most clinicians made one to two patient referrals during the trial. In surveys, 95% of clinicians said they would recommend the program to other clinicians. There were challenges to fully assessing adoption, though, related to limitations in data tracking. Referrals were not routinely captured in the EHR, and the broad implementation of the program made assessment of the proportion and representativeness of settings and clinicians infeasible.

Among providers who did not refer patients to the program (i.e., non-adopters), the main reason reported on the post-program survey was that their patients did not meet all eligibility criteria (40%). In interviews, clinicians further described lack of clarity around enrollment criteria and around the CP scope of practice as a barrier, as noted by this clinician, who also raised a concern about patient randomization: “I feel like, initially, it was well-advertised as an option. But when it came down to the nitty gritty, like what they can or cannot actually do or that it was actually a pilot where they could get randomized into no-visits was not. There were some details that were missing that, from our perspective, are kind of important to know. Like I said, if your backup plan was community paramedics, and they were randomized into no-visits…And that wasn’t – in my opinion, that wasn’t clearly defined in the beginning that this was actually more like a study and that the patient would have to consent beyond us saying, ‘Oh, are you okay if we make the referral to the study?’” Another clinician described difficulties in identifying appropriate patients: “There are certain patients that definitely will be a good fit for the program and will benefit most from the program. So, it’s just really, as a clinician, maybe trying to be aware of the program, that it’s there, and then being able to identify the patients that most likely will benefit from the program.” Other clinicians similarly reported that they were aware of the program, but there were challenges to remembering that it was available at the point of care when they were developing a care plan for an appropriate patient.

Some clinicians also noted lack of understanding about how to refer to the program, for example, “I don’t even know how to make a referral. Like the one patient I did have presented to the ER with his blood sugars out of control, and the paramedic program was perfect. It was exactly what he needed.” Still, clinicians reported high levels of satisfaction with the CACP referral process (very satisfied: 61.7%; satisfied: 29.8%) and communication with the CP leadership team (very satisfied: 85.1%; satisfied: 8.5%). Similarly, high levels of satisfaction were noted among providers and administrators with the range of provided services (very satisfied: 64.9%; satisfied: 22.8%) and geographic areas served (very satisfied: 47.2%; satisfied: 22.8%).

Most CPs felt patients were well prepared for the CACP program (60%), patients were appropriate for this care delivery model (84%), and the process of accepting new patients into the program went smoothly (83%). However, CPs also noted concerns about clinician awareness of the referral process and patient eligibility/CP scope of practice. In surveys, CPs indicated the information needed to care for their patients (agree: 50%) and corresponding orders (agree: 33%) was not adequately documented in the EHR. Additionally, CPs reported the support from referring providers was low (very low: 17%; somewhat low: 50%).

In interviews, CPs talked about the value of word-of-mouth in promoting the program and the benefit of building awareness through repeat referrals, both of which were described by this CP: “So, when those referrals would come in, most of the time, it was care coordinators, case managers and stuff that knew about us through word of mouth from the road shows. And then, they kind of said, ‘Well, this patient doesn’t fit into any of these molds. Let’s try the community paramedics’…But I think once the referrals were placed, we were starting to get those repeat providers, where hospital internal medicine, we work with them…and the hospitalists and stuff, you start to see the names that are like, ‘They’ve had my patient before,’ or we see repeat patients that keep coming back where they see that it was beneficial. They see that the patient did well after we saw them because there’s some that are – they just needed fluids, right? So, ‘Oh, we just needed to give them some fluids. The paramedics can do that.’” The study team also gave presentations to clinical and administrative teams in the months leading up to the start of trial enrollment, and they continued to answer questions about the trial and promote the service in the months after enrollment commenced.

CPs described challenges with ED referrals, which may have impacted the number of ED referrals, for example, “So, the CACP trial wasn’t working a lot with the ED patients, and that’s not really a fault of the CACP trial, specifically. It actually has more to do with the people that do our ED referrals – are doing their own trials, so they were not referring through the CACP trial, unfortunately. So, that would have been a population I would have liked to have seen more of. But of course, we still see them; we just don’t see them through the trial, so that’s not a number captured well through the trial, unfortunately…I’d also like to capture more of the outpatient population. We saw a lot of inpatient referrals, hospitals referring to us for various things. But I had a lot of people – paramedics, specifically, asking, ‘I picked up this patient at the [outpatient] building…How do I get them enrolled in the trial?’ And I was like, ‘Well, they have to go through a provider’…You have a patient who went to an appointment with the primary care provider at 2:00; but by 3:00, they’re gone, right, and how do you capture that? How do you get them the help that they need?”

Discussion

Pragmatic trials that enroll representative populations in diverse settings may more quickly translate effective interventions to practice than tightly controlled trials. Tools like PRECIS-2 help study teams plan pragmatic trial features, but there may be challenges to executing the design as planned. Post hoc exploration of implementation outcomes after a pragmatic randomized trial offers an opportunity to assess how planned pragmatic trial features facilitated or hindered representative trial enrollment and broad clinical uptake. This study highlights our team’s experience deploying a pragmatic trial with point-of-care screening and referral in busy care settings, including hospitals and EDs, in small metropolitan and rural communities. As assessed by PRECIS-2, eligibility and recruitment features of the trial were among the most pragmatic. Most notably, patient eligibility criteria closely matched the usual care environment, and the trial was able to enroll patients with a range of health conditions and care needs. Using the empaneled population as a comparator, we found that trial participants were at least as diverse and medically complex as empaneled patients. Referred patients in this trial had significantly more social risks that the empaneled population. CPs have expertise in the social determinants of health that may further support patients with intermediate healthcare needs, but future larger-scale studies may be needed to investigate how those services are conveyed to patients and clinicians during recruitment.

While rural enrollment was also representative of the empaneled population, referrals from the ED – which was expected to be an important referral base – were low. Post hoc assessment by the study team suggests that most ED-referred patients were directed into another clinical program introduced after the CACP trial was designed, which leveraged hospitalists working in the ED. Low enrollment in the ED may also be related to clinician perceptions that research procedures will delay care decisions – an issue potentially most relevant in time-sensitive and busy ED settings. Clinician perceptions of research procedure disruptions may have also contributed to higher complexity CACP referrals if ED clinicians utilized the program most when other options were not available. Assessment of patients referred to the CACP program from the ED suggests that they did, in fact, require a higher level of care than patients seen by CPs as part of the competing hospitalist-led program.

There were also barriers to in-person consent because the trial was deemed greater than minimal risk by the IRB because it evaluated a new care delivery model that took high-risk patients out of a higher level of care and into the home. As a result, the trial required individual patient written rather than oral informed consent, with downstream implications on the time required to obtain consent and either need for in-person study coordinator access or ability to provide the patient with paper or electronic consent documents for a remote consent process. This hindered the recruitment of patients unable to complete the written consent procedures due to lack of access to study coordinators (which disproportionately affected patients recruited from NWWI, a rural practice setting located far from the academic hub and robust research infrastructure), lack of access to internet and technology (which disproportionately affected older, lower income, and rural residents), or required interpreter services (which disproportionately affected racial and ethnic minoritized individuals, particularly from low-prevalence backgrounds for whom in-person interpreters are not available). These populations have been traditionally excluded from trial participation. In this study, problem-solving to ensure reasonable options for point-of-care consent, including in rural settings without on-site study coordinators, allowed our team to prioritize diversity reflective of the eligible patient population. While the pragmatic design of our trial purposefully reduced barriers to participation, it could not eliminate them entirely.

It will be important to re-examine the requirements for informed consent and adapt them to meet the needs of diverse clinical settings and patient populations [25]. For example, allowing consent via interactive phone Short Message/Messaging Service (SMS) texts would allow for remote electronic consent even without access to internet or technology. Eliminating the requirement for interpreters and legally authorized representatives to be physically in the same room as the patient being consented would reduce barriers to participation by patients with limited English proficiency and with cognitive impairment, respectively. This study used a pool of study coordinators that were available for extended hours, especially in early evenings and occasional weekends, to accommodate patient dismissal hours, but other organizations may not have resources such as this.

This study also highlights the need for early and ongoing engagement with clinicians when they are responsible for screening and referral. In the four months leading up to the start of study enrollment, the team gave at least 17 formal presentations for clinical and administrative groups that included opportunities for questions and discussion [26]. However, clinicians reported remaining questions about patient eligibility and enrollment procedures. Clinicians also raised concerns about patient randomization and the potential that their patients would not get enrolled into CP care, which could disrupt care planning and force standard of care during hospital diversion periods (i.e., when hospital capacity is very low, and patients are diverted to other care options as appropriate). The study team continued to reach out to practice areas to provide information and answer questions, but proponents may need multipronged implementation strategies to increase and sustain enrollment at the point of care. Given the reported success of repeated referrals from some clinicians, future studies might consider implementation strategies, such as identification of clinical champions, in addition to educational presentations aimed at increasing clinician awareness of the program. Limiting eligibility to a smaller number of settings or clinical scenarios may have also concentrated outreach efforts and bolstered enrollment, but doing so would have limited program reach.

Finally, this study highlights challenges due to the lack of EHR capabilities to support point-of-care patient consent and randomization. Healthcare organizations that aim to conduct pragmatic trials must have EHRs that support these aspects of trials. There were also associated challenges to ascertaining patient reach and clinician adoption due to the inability to track study referrals directly in the EHR, which is a critical aspect of assessing equitable implementation. EHR infrastructure is foundational for learning health system approaches that support ongoing evaluation and implementation of clinical practices as part of the translational pipeline [27,28], and the use of real-world data to examine equitable inclusion in clinical trials is critical for reducing disparities in trial participation [6]. To be feasible, though, research incentives and infrastructure investment must also be directed toward these types of trials [29].

Conclusion

Our team successfully operationalized trial procedures within busy care settings, including hospital and ED settings, and constructs from implementation science helped guide assessment of how broadly the program was implemented. Many of our implementation challenges stemmed from human subjects’ research requirements, involvement of clinicians in enrollment at the point of care, and the desire to keep inclusion criteria broad to allow for equitable reach. There were also challenges to assessing adoption and reach using the EHR. Future pragmatic trials would benefit from IRB expertise in pragmatic trials, advances in EHR capabilities for enrollment and capture of referral data, and EHR systems to help clinicians identify best-candidate patients for referral, ensuring equitable program reach.