OnTrackNY’s learning healthcare system

Abstract

Worldwide, early intervention services for young people with recent-onset psychosis have been associated with improvements in outcomes, including reductions in hospitalization, symptoms, and improvements in treatment engagement and work/school participation. States have received federal mental health block grant funding to implement team-based, multi-element, evidence-based early intervention services, now called coordinated specialty care (CSC) in the USA. New York State’s CSC program, OnTrackNY, has grown into a 23-site, statewide network, serving over 1800 individuals since its 2013 inception. A state-supported intermediary organization, OnTrackCentral, has overseen the growth of OnTrackNY. OnTrackNY has been committed to quality improvement since its inception. In 2019, OnTrackNY was awarded a regional hub within the National Institute of Mental Health-sponsored Early Psychosis Intervention Network (EPINET). The participation in the national EPINET initiative reframes and expands OnTrackNY’s quality improvement activities. The national EPINET initiative aims to develop a learning healthcare system (LHS); OnTrackNY’s participation will facilitate the development of infrastructure, including a systematic approach to facilitating stakeholder input and enhancing the data and informatics infrastructure to promote quality improvement. Additionally, this infrastructure will support practice-based research to improve care. The investment of the EPINET network to build regional and national LHSs will accelerate innovations to improve quality of care.

Introduction

Schizophrenia is a serious mental illness which has a prevalence of about 1% in the USA [1,2] with peak age of onset occurring around the ages of 15–25 [3]. Symptoms may include positive symptoms (such as delusions, hallucinations, and illusions), negative symptoms (such as apathy, lack of emotion, and affect), cognitive deficits [2,4,5], and high rates of disability; societal costs are estimated at over $155 billion in 2013 [4].

However, growing evidence over the past two decades has shown that early intervention after onset of psychosis has the potential to reduce the poor prognosis of this disorder, at least in the short run [2]. Notably, early intervention services in countries including Denmark, the UK, Italy, Australia, and Hong Kong have demonstrated improvements in client outcomes [5]. In the USA, the Recovery After an Initial Schizophrenia Episode (RAISE) studies (including the RAISE Early Treatment Program and RAISE Implementation and Evaluation Study) demonstrated improvements in quality of life, psychopathology and work/school participation [6], and occupational and social functioning [7]. Worldwide, early intervention services have been shown to be associated with reduced risk of treatment discontinuation, psychiatric hospitalization and symptom severity, and increased involvement in school or work participation [8]. The online supplement includes a bibliography of additional studies.

In 2014, the US Congress allocated a 5% set aside (approximately $25 million) in 2014 for the Substance Abuse and Mental Health Services Administration (SAMHSA)’s Community Mental Health Block Grant to states to support the implementation of early intervention services [3]. This was later raised to 10% (or approximately $50 million) in 2016. Thus, by 2019, nearly all states have implemented early intervention services for people with recent-onset psychosis, with approximately 285 clinics operating in 49 states [9].

The Mental Health Block Grant legislation called for SAMHSA and the National Institute of Mental Health (NIMH) to work together to define the set of evidence-based practices that are covered under these services. Thus, a 2014 NIMH white paper established the concept of coordinated specialty care (CSC) [3]. While the services provided vary across CSC programs, services typically include a set of evidence-based practices [3], which may include psychotherapy, medication management, substance use treatment, peer support, supported employment and education services, and family education and support. Services are provided using a coordinated team-based approach and are generally provided for a defined period of time (e.g. 2 years), after which clients are expected to transition to community-based care.

The Development of OnTrackNY

As an outgrowth of New York State’s participation in the RAISE Implementation and Evaluation Study, the state developed a statewide CSC program, which began enrolling participants in October 2013. Since its inception, OnTrackNY has grown into a 23-site network, under the leadership of Lisa Dixon, MD, MPH. The New York State Office of Mental Health (OMH) currently subsidizes OnTrackNY services, with the goal of developing a sustainable financing model.

OnTrackNY is a nationally recognized model for providing CSC for adolescents and young adults with the recent onset of a psychotic disorder. As a CSC program, OnTrackNY provides access to coordinated, team-based services, including outreach, psychotherapy and support, pharmacological treatment, recovery support (including treatment of co-occurring substance use), suicide prevention, supported employment and education services, family support and education, and peer support. Individuals are eligible if they are New York State residents aged 16–30 with non-affective psychosis, with onset of illness at least a week, but less than 2 years, prior to program entry. The program aims to provide treatment for an average of 2 years (depending on client needs). Thanks to its sponsorship and financing by OMH, OnTrackNY is able to serve clients regardless of ability to pay.

The 23 OnTrackNY teams are located in licensed outpatient clinics at community agencies, state-operated facilities, and community and academic hospitals in urban, suburban, and rural areas throughout New York. Agencies are required to maintain key elements of the treatment model in order to maintain OMH funding.

Since its inception in October 2013, through January 2020, the network has served 1812 individuals, with 438 new enrollees in fiscal year 2019.

The Role of the Center for Practice Innovations

The placement of OnTrackNY within the Center for Practice Innovations (CPI) is an essential conduit of the team’s continuous quality improvement efforts. CPI is funded by the New York State OMH and is charged with identifying and disseminating evidence-based practices, to promote recovery for people with serious mental illnesses throughout the state. As an intermediary and purveyor organization [10,11], CPI promotes the implementation of evidence-based practices with fidelity and positive outcomes and seeks to build the capacity of providers and systems to implement and sustain best practice models. CPI thus serves to support this large network of OnTrackNY sites, operated by independent agencies and programs, within the structure defined by the State OMH. The CPI infrastructure thus provides the foundation for ongoing program evaluation, continuous quality improvement, and training in evidence-based practices.

As new OnTrackNY teams are created and funded, they are provided with intensive training in the CSC model. The initial training includes a multi-day in-person introductory training of the principles and practices of the CSC model, followed by remote consultation, which also includes training in data collection forms, procedures, and the data collection platform (the Acquire Electronic Data Capture system). Role-specific trainings are provided for team members, based on their specialty. Over the first 2 years of each program’s operation, intensive technical assistance is provided, including frequent individual, collaborative, role and team-based phone calls, monthly care consultation calls, webinars, and use of data and fidelity reports to highlight strengths and identify areas for improvement. Ongoing, self-paced education is also offered through CPI’s online Learning Management System. After the initial 2-year start-up period, ongoing technical assistance can be reduced as needed. While the agency retains ultimate responsibility for treatment, OnTrackCentral serves as an intermediary organization and provides ongoing training and support in the implementation of the OnTrackNY model.

OnTrackNY Treatment Model

When a program is ready to begin operations, it receives referrals from a variety of sources, including inpatient units, outpatient providers, schools, and self and family referrals generated through outreach efforts. Fig. 1 illustrates the OnTrackNY conceptual model of the continuum of care. Within OnTrackNY, the client has access to an array of services delivered by a multidisciplinary team, as previously described. These services are designed to work together to support the targeted outcomes which typically involve individual goal attainment and are personalized to individual needs and preferences. Processes and outcomes are monitored during treatment. Assessment of post-discharge outcomes is a long-term goal of OnTrackNY; currently, outcomes can be assessed using Medicaid claims data, and in specific subpopulations as part of research studies.

Fig. 1. Conceptual model of OnTrackNY treatment model.

Data Collection Process

A robust set of data is collected for the purpose of quality improvement, to assess participant trajectories and improvements over time (Fig. 2). Within routine care, each client’s primary clinician completes client assessment forms at enrollment, every 90 days during treatment, and at discharge. This involves gathering relevant information from multiple sources including client and family, clinical records, outside providers, and other team members. At discharge, clinicians report on reasons for discharge and the client’s progress toward identified goals. Additionally, program-level data are collected every 3 months to monitor team functioning (e.g., staffing, services offered, and frequency of team meetings).

Fig. 2. Data elements in OnTrackNY.

OnTrackNY has developed a multistage process for data collection. Site clinicians enter data, having received training from OnTrackCentral. Data are entered into a data management system, Acquire Electronic Data Capture, developed by the Innovative Clinical Research Solutions group at the Nathan Kline Institute (NKI), that enables sites to enter data securely (requiring minimal infrastructure – only an internet connection – for end users). The data are securely transmitted, meeting all Federal Information Security Management Act [12], Health Insurance Portability and Accountability Act, and 21 Code of Federal Regulations Part 11 regulatory requirements. The system also has built-in mechanisms to flag common data entry errors, such as dates occurring out of range, and prompts the user to correct such errors. Once the data are entered, NKI assembles the data and transmits it monthly to the New York State Office of Performance Measurement and Evaluation (PME) within OMH for aggregation and cleaning. During the data cleaning process, OnTrackCentral staff contact sites as needed to resolve missing data or other data inaccuracies, resulting in very low rates of missing data.

Once files are clean, PME sends the data set to OnTrackCentral for program evaluation. OnTrackCentral delivers aggregated monthly data reports to sites, which allows tracking of key variables, such as client demographic characteristics, care processes (e.g., days from referral to evaluation and inclusion of families in care), and key outcomes (e.g., hospitalization, work/school participation, and symptom ratings) and allows for comparisons of these variables over time. Additionally, PME also develops de-identified data files for research purposes.

Program Evolutions to Date

The OnTrackNY program has been committed to continuous quality improvement and thus has evolved since its inception. For example, OnTrackNY has continually modified its program manuals, including the overall team manual and primary clinician’s manual, as well as manuals focused on specific aspects of care, including outreach and recruitment, medication management, family support, and supported employment and education. Manuals were also created to address cognitive health and the provision of culturally competent care. An online module was created to address care for lesbian, gay, bisexual, and transgender individuals. These manuals are publicly available on the OnTrackNY website.

Additionally, the peer specialist role was added to the OnTrackNY teams in 2016, and the peer support manual and other tools were developed to assist in the implementation of this new role. The OnTrackCentral Technical Assistance/Training team includes a lead recovery and peer support role, and nationally prominent peer leaders (i.e., Pat Deegan) have been integrally involved in program development and coaching. The OnTrackNY team has also developed outreach and engagement strategies on social media. The dissemination of these materials enables OnTrackNY sites, as well as CSC programs in other states, to adapt these tools to their programs.

The OnTrackNY program has supported a number of innovative research and demonstration projects that pilot, test, and refine innovations to CSC. This has included both secondary analyses of the existing OnTrackNY program data to examine trends (such as factors influencing work/school outcomes and the identification of predictive factors to identify trajectories in functioning), as well as testing new program components. Current research studies are examining a wide variety of treatment innovations, including the addition of interventions to address cognitive health, identification of ways to reduce duration of untreated psychosis, substance use reduction, and interventions to improve physical health.

Next Steps: Participation in the National Early Psychosis Intervention Network Consortium

In 2019, OnTrackNY was awarded funding to support a regional hub within the NIMH-sponsored Early Psychosis Intervention Network (EPINET). The national EPINET initiative aims to develop a national learning healthcare system (LHS), capitalizing on the breadth of resources available in this national network, to promote a system in which ongoing learning can lead to improvements in care. The national EPINET initiative currently supports five regional networks of clinical sites providing CSC care (“hubs”) and a national coordinating center; this network includes 59 CSC programs across nine states and is expected to serve at least 5000 participants over the 5-year project period [9]. The participating programs represent a myriad of delivery systems, including hospital-based, community-based, urban, suburban, and rural. The national EPINET initiative aims to use the diversity of the system to promote the development of evidence-based quality improvement initiatives and practice-based research (i.e., research that is aimed at improving treatment and can be implemented into CSC clinics to improve quality of care).

OnTrackNY’s LHS Model

OnTrackNY’s participation in the EPINET network provides a way to conceptualize the OnTrackNY quality improvement activities, by providing the resources to further invest in infrastructure to move into a systematic, dynamic LHS.

The design of our LHS is based on the principles of the Institute of Medicine (IOM)’s model for a continuously LHS [13], which brings together multiple stakeholders to review data, utilize technology, and identify strategies designed to improve quality, introduce innovations, and increase efficiency. We use the IOM’s circular LHS model, operationalizing the components following the six phases of LHS as described by the Group Health Cooperative [14]:

• Scanning and surveillance: a healthcare system that is willing to learn and identify gaps in quality and efficiency and identify solutions;

• Design: design of solutions includes key stakeholders to ensure that solutions meet their needs;

• Implement: piloting or testing solutions;

• Evaluate: evaluate pilot results with feedback from all key stakeholders;

• Adjust: make program changes based on evaluation feedback;

• Disseminate: dissemination of knowledge tools and products in a timely manner through evidence-based communications.

While the IOM’s model provides a general aspirational roadmap for a continuous improvement approach, significant knowledge gaps limit how best to translate knowledge into evidence-based communications, campaigns, guidelines and other tools, products, and interventions. We thus draw on the Knowledge to Action (KTA) framework (Fig. 3) because it has direct relevance to the goals of an LHS: discovery and translation of research to practice, and it provides practical guidance at each step in the translation process. The KTA framework includes (i) knowledge creation which is surrounded by (ii) the action cycle (Fig. 3 [15,16]). The KTA planning tool directs knowledge creation and action cycles through questions for stakeholders to consider during each LHS phase. Through an iterative process, knowledge creation may be derived from stakeholders, data, and mixed methods practice-based research [16] during the LHS scanning and surveillance phase (knowledge creation). Using the KTA planning tool, we conduct knowledge inquiry [17]. Through an iterative and dynamic process, action phases may be carried out sequentially or simultaneously; knowledge phases may impact the action phases. The action cycle outlines a process, representing the activities needed for knowledge to be applied in practice; knowledge is adapted to the local context, and barriers and facilitators to its use are explicitly assessed. Involvement of stakeholders, and tailoring knowledge to those who will use it, is crucial. Thus, the LHS framework provides a guide and structure that will allow the program to improve services systematically and efficiently. This strategy connects the Group Health Cooperative LHS phases to principles of implementation science including adaptation and assessment of barriers (design phase), selection, tailoring and implementation of intervention (implement, adjust phases), evaluation (evaluate phase), and sustainment (disseminate phase). Additional references for community-based participatory research are available in the online supplement.

Fig. 3. OnTrackNY LHS model: adapted from Group Health Cooperative (GHC) LHS model and Knowledge to Action (KtA) framework.

Operationalizing the LHS in OnTrackNY

In order for OnTrackNY to optimize its ability to function as an LHS, our analysis pointed to the need to strengthen infrastructure for systematic stakeholder involvement (including clients, families, clinicians, payers, administrators, and government leaders) and enhance data collection through every LHS phase. These LHS components do not follow in a stepwise sequence but, instead, operate in parallel and interact to facilitate and enhance quality improvement processes. For example, stakeholders may help identify problems or areas for improvement (see Fig. 3), implement identified changes, and then collaborate with the research/implementation team(s) (or central leadership) to determine the extent to which changes in processes produce the expected results or whether other challenges have arisen. In this way, leadership and stakeholders can work together to effect continuous quality improvement. Data and informatics infrastructure will be enhanced to further improve the efficiency and effectiveness of data collection and utilization, including the provision of client-level data reports to clients and clinicians, to aid in treatment planning. Ultimately, stakeholders and data will work in a cyclical, interconnected manner throughout the process, delivering real-time, actionable information to improve clinical care. This infrastructure will also, in turn, support the development of practice-based research to create further innovations to improve care. For example, an initial research project will focus on the development of a suicide prevention protocol adapted to the needs of this population.

Conclusion

Through its participation as an EPINET regional hub, the OnTrackNY LHS will continue to innovate [18–27], while emphasizing and enhancing two critical foundational components – proactively engaging stakeholders to optimize understanding of key problems and their solutions at every LHS phase and developing data systems with enhanced standardized data collection, informatics, and analytics. By including multilevel stakeholder input with shared leadership, we will continue to build the ongoing communication channels required for a highly effective LHS. The voices of stakeholders will drive change. OnTrackNY’s participation in the EPINET initiative has helped to conceptualize its quality improvement activities and will help to develop activities that can improve quality of care and can be implemented into clinical practices to improve outcomes for individuals with recent-onset psychosis.