Introduction
Over the past two decades, early detection and early intervention in psychosis have become essential goals of psychiatry (Birchwood, Todd, & Jackson, Reference Birchwood, Todd and Jackson1998; McGlashan & Johannessen, Reference McGlashan and Johannessen1996; McGorry, Killackey, & Yung, Reference McGorry, Killackey and Yung2008). Only a small proportion of cases undergo the transition to psychosis, and the process often takes place over very long periods of time; these circumstances are among the main reasons why the focus of high-risk studies has shifted from the traditional or genetic high-risk model to the clinical high-risk (CHR) model. The CHR concept has emerged to describe cases that are likely to progress to psychosis soon (Yung et al., Reference Yung, McGorry, McFarlane, Jackson, Patton and Rakkar1996). A number of prospective cohort programs have been introduced for help-seeking people who have not yet developed psychosis (Cannon et al., Reference Cannon, Cadenhead, Cornblatt, Woods, Addington, Walker and Heinssen2008; Nelson et al., Reference Nelson, Yuen, Wood, Lin, Spiliotacopoulos, Bruxner and Yung2013; Riecher-Rossler et al., Reference Riecher-Rossler, Gschwandtner, Aston, Borgwardt, Drewe, Fuhr and Stieglitz2007; Ruhrmann et al., Reference Ruhrmann, Schultze-Lutter, Salokangas, Heinimaa, Linszen, Dingemans and Klosterkotter2010); additionally, various terms, such as at-risk mental state and ultra-high-risk, in addition to CHR for psychosis, and basic symptoms have been established to describe this population (Fusar-Poli et al., Reference Fusar-Poli, Borgwardt, Bechdolf, Addington, Riecher-Rossler, Schultze-Lutter and Yung2013; Schultze-Lutter, Schimmelmann, Ruhrmann, & Michel, Reference Schultze-Lutter, Schimmelmann, Ruhrmann and Michel2013; Yung, Fusar-Poli, & Nelson, Reference Yung, Fusar-Poli and Nelson2012). However, the incidence rate of psychosis in a CHR individual decreases over time (Yung et al., Reference Yung, Yuen, Berger, Francey, Hung, Nelson and McGorry2007). Only one-fourth of CHR patients develop psychosis within 3 years (Fusar-Poli et al., Reference Fusar-Poli, Bonoldi, Yung, Borgwardt, Kempton, Valmaggia and McGuire2012, Reference Fusar-Poli, Cappucciati, Rutigliano, Schultze-Lutter, Bonoldi, Borgwardt and McGuire2015), and there are even studies that report an incidence rate of <15% (Addington et al., Reference Addington, Epstein, Liu, French, Boydell and Zipursky2011b; Katsura et al., Reference Katsura, Ohmuro, Obara, Kikuchi, Ito, Miyakoshi and Matsumoto2014; Koike et al., Reference Koike, Takano, Iwashiro, Satomura, Suga, Nagai and Kasai2013; Pruessner et al., Reference Pruessner, Faridi, Shah, Rabinovitch, Iyer, Abadi and Malla2017), although this risk state is not pluripotential but is specific to psychosis (Webb et al., Reference Webb, Addington, Perkins, Bearden, Cadenhead, Cannon and Woods2015; Woods et al., Reference Woods, Powers, Taylor, Davidson, Johannesen, Addington and McGlashan2018). Moreover, one-third of CHR patients remit from the risk state (Simon et al., Reference Simon, Borgwardt, Riecher-Rossler, Velthorst, de Haan and Fusar-Poli2013), and they showed no cognitive impairment or have good functional outcomes (Glenthoj, Kristensen, Wenneberg, Hjorthoj, & Nordentoft, Reference Glenthoj, Kristensen, Wenneberg, Hjorthoj and Nordentoft2020; Lee et al., Reference Lee, Shin, Shin, Kim, Jang, Kang and Kwon2014b). On the other hand, other non-converters also have attenuated psychotic symptoms or low levels of functioning even if they do not develop psychosis (Addington et al., Reference Addington, Cornblatt, Cadenhead, Cannon, McGlashan, Perkins and Heinssen2011a; Lee et al., Reference Lee, Kim, Correll, Byun, Kim, Jang and Kwon2014a; Lin et al., Reference Lin, Wood, Nelson, Beavan, McGorry and Yung2015). Thus, the CHR state is a heterogeneous clinical syndrome, only a small percentage is converted to psychosis and also develops other conditions than psychosis, and for this reason, the dilution of the pretest risk of psychosis due to intensive, predominantly general-population-oriented outreach campaigns and a high rate of self-referrals has been discussed (Fusar-Poli, Schultze-Lutter, & Addington, Reference Fusar-Poli, Schultze-Lutter and Addington2016b; Fusar-Poli et al., Reference Fusar-Poli, Schultze-Lutter, Cappucciati, Rutigliano, Bonoldi, Stahl and McGuire2016c; Mitter, Nah, Bong, Lee, & Chong, Reference Mitter, Nah, Bong, Lee and Chong2014). Therefore, the ability to identify true-positive patients who will later develop psychosis can immensely broaden our understanding of the pathophysiology of the long-term course of schizophrenia, and it will deepen the phenomenological, biological, and causal understanding of schizophrenia (Bentall, Jackson, & Pilgrim, Reference Bentall, Jackson and Pilgrim1988; Fusar-Poli & Schultze-Lutter, Reference Fusar-Poli and Schultze-Lutter2016; Guloksuz & van Os, Reference Guloksuz and van Os2018).
Predictive medicine is a discipline that entails predicting the probability of a disease's incidence or prognosticating its course, thus reducing the uncertainty in clinical decision making (Fusar-Poli, Hijazi, Stahl, & Steyerberg, Reference Fusar-Poli, Hijazi, Stahl and Steyerberg2018; Steyerberg, Reference Steyerberg2008; Wasson, Sox, Neff, & Goldman, Reference Wasson, Sox, Neff and Goldman1985). In contrast to classical statistics, predictive modeling has high clinical utility in that it not only indicates the average characteristics of the patient's group but also provides rich information about the onset or trajectory of illness at the individual level (Braitman & Davidoff, Reference Braitman and Davidoff1996; Hahn, Nierenberg, & Whitfield-Gabrieli, Reference Hahn, Nierenberg and Whitfield-Gabrieli2017; Lee, Bang, & Kim, Reference Lee, Bang and Kim2016). Furthermore, with the very high levels of clinical heterogeneity arising from phenotype-based diagnosis, the application of a clinical prediction model to real-world situations would facilitate better decision making in psychiatry. However, clinical impressions are insufficient for predicting psychosis outcome in CHR cases (Nelson & Yung, Reference Nelson and Yung2010); more rigorous and quantitative prediction models are needed. Thus, risk stratification has been applied in CHR studies (Fusar-Poli et al., Reference Fusar-Poli, Cappucciati, Borgwardt, Woods, Addington, Nelson and McGuire2016a, Reference Fusar-Poli, Rutigliano, Stahl, Schmidt, Ramella-Cravaro, Hitesh and McGuire2016b; Nieman et al., Reference Nieman, Ruhrmann, Dragt, Soen, van Tricht, Koelman and de Haan2014; Ruhrmann et al., Reference Ruhrmann, Schultze-Lutter, Salokangas, Heinimaa, Linszen, Dingemans and Klosterkotter2010; Schmidt et al., Reference Schmidt, Cappucciati, Radua, Rutigliano, Rocchetti, Dell'Osso and Fusar-Poli2017), this is similar to the model applied in heart failure and stroke (Goldman et al., Reference Goldman, Cook, Johnson, Brand, Rouan and Lee1996; Janes, Pepe, & Gu, Reference Janes, Pepe and Gu2008; Lip, Nieuwlaat, Pisters, Lane, & Crijns, Reference Lip, Nieuwlaat, Pisters, Lane and Crijns2010). Moreover, a number of prediction studies have been conducted (Addington et al., Reference Addington, Farris, Stowkowy, Santesteban-Echarri, Metzak and Kalathil2019; Oliver et al., Reference Oliver, Reilly, Baccaredda Boy, Petros, Davies, Borgwardt and Fusar-Poli2020; Studerus, Ramyead, & Riecher-Rossler, Reference Studerus, Ramyead and Riecher-Rossler2017), and the least absolute shrinkage and selection operator (LASSO) model, which incorporates machine learning techniques, has been introduced into such studies (Addington et al., Reference Addington, Liu, Perkins, Carrion, Keefe and Woods2017; Ciarleglio et al., Reference Ciarleglio, Brucato, Masucci, Altschuler, Colibazzi, Corcoran and Girgis2019) to overcome the problem of overfitting (Koutsouleris, Upthegrove, & Wood, Reference Koutsouleris, Upthegrove and Wood2019; Nelson, Yung, & McGorry, Reference Nelson, Yung and McGorry2019; Tibshirani, Reference Tibshirani1996; Tibshirani, Reference Tibshirani1997). More recently, several web-based personalized risk calculators have been developed (e.g. riskcalc.org/napls, psychosis-risk.net, link.konsta.com.pl/psychosis) (Cannon et al., Reference Cannon, Yu, Addington, Bearden, Cadenhead, Cornblatt and Kattan2016; Fusar-Poli et al., Reference Fusar-Poli, Rutigliano, Stahl, Davies, Bonoldi, Reilly and McGuire2017; Kotlicka-Antczak et al., Reference Kotlicka-Antczak, Karbownik, Stawiski, Pawelczyk, Zurner, Pawelczyk and Fusar-Poli2019), and their scope of use is being expanded through internal and external validation (Carrion et al., Reference Carrion, Cornblatt, Burton, Tso, Auther, Adelsheim and McFarlane2016; Fusar-Poli et al., Reference Fusar-Poli, Werbeloff, Rutigliano, Oliver, Davies, Stahl and Osborn2019b; Osborne & Mittal, Reference Osborne and Mittal2019; Zhang et al., Reference Zhang, Xu, Tang, Li, Tang, Cui and Group2019). These individual-level risk measures can be applied in real-world clinical practice to quantify the risks that patients may face and to develop appropriate treatment strategies. However, several additional considerations must be addressed to increase the predictive accuracy of personalized risk prediction models. First, to increase the accuracy of prediction, a modeling process based on longer-term follow-up is required to reduce false negatives that have not yet been developed due to insufficient observation period. Studies with a follow-up period of <3 years tend to have an incidence of less than one-quarter (Addington et al., Reference Addington, Epstein, Liu, French, Boydell and Zipursky2011b; Katsura et al., Reference Katsura, Ohmuro, Obara, Kikuchi, Ito, Miyakoshi and Matsumoto2014; Koike et al., Reference Koike, Takano, Iwashiro, Satomura, Suga, Nagai and Kasai2013; Morrison et al., Reference Morrison, French, Parker, Roberts, Stevens, Bentall and Lewis2007; Pruessner et al., Reference Pruessner, Faridi, Shah, Rabinovitch, Iyer, Abadi and Malla2017; van der Gaag et al., Reference van der Gaag, Nieman, Rietdijk, Dragt, Ising, Klaassen and Linszen2012), whereas those of more than 3 years tend to have an incidence of more than one-quarter (Nelson et al., Reference Nelson, Yuen, Wood, Lin, Spiliotacopoulos, Bruxner and Yung2013; Schultze-Lutter, Klosterkotter, & Ruhrmann, Reference Schultze-Lutter, Klosterkotter and Ruhrmann2014; Spitz et al., Reference Spitz, Studerus, Koranyi, Rapp, Ramyead, Ittig and Riecher-Rossler2017; Ziermans et al., Reference Ziermans, de Wit, Schothorst, Sprong, van Engeland, Kahn and Durston2014), with some exceptions (An et al., Reference An, Kang, Park, Kim, Lee and Lee2010; Armando et al., Reference Armando, Pontillo, De Crescenzo, Mazzone, Monducci, Lo Cascio and Schultze-Lutter2015), although meta-analytic results indicated that the incidence of psychosis in high-risk groups peaks within the first 2 years after entry (Kempton, Bonoldi, Valmaggia, McGuire, & Fusar-Poli, Reference Kempton, Bonoldi, Valmaggia, McGuire and -Poli2015). A sufficient period of observation will reveal the results of false negatives who have not yet developed and enable more accurate modeling. Second, CHR samples have different risks for psychosis, depending on their referral sources. For example, samples such as those recruited from the community via the Prodromal Questionnaire–brief form and other self-report measures, as well as patients who visit a primary clinic with non-prodromal symptoms and are suspected of having attenuated psychotic symptoms, would have different pretest risk levels than help-seeking individuals visiting a secondary or specialized clinic for CHR patients. This suggests that the diversification of referral sources and the rise of intensive outreach activity are major contributors to the increased variance of pretest risk enrichment in CHR samples and that the risk of developing psychosis is becoming more heterogeneous. Given this combination of factors, samples recruited by a fixed strategy at a single institution may allow greatly improved models to be developed if external validation can be conducted.
The primary goal of this study was to develop a model for predicting the 10-year risk of psychotic transition in patients who visited the Seoul Youth Clinic (SYC) at Seoul National University Hospital. A second goal was to perform internal validation of the resulting prediction model and stratify the CHR sample by risk level.
Methods
Participants
Two hundred and twenty-two help-seeking CHR individuals (aged 15–35 at baseline) were enrolled in the prospective, naturalistic cohort program for prodromal psychosis at the SYC between November 2004 and November 2019 (Kim et al., Reference Kim, Jang, Park, Shim, Hwang, Kim and Kwon2012; Kwon, Byun, Lee, & An, Reference Kwon, Byun, Lee and An2012; Lee et al., Reference Lee, Kim, Correll, Byun, Kim, Jang and Kwon2014a). All participants were recruited mainly from a psychiatric outpatient clinic in the Seoul National University Hospital and made initial contact with the SYC via the website (http://www.youthclinic.org). All subjects were diagnosed with the Korean version of the Structural Interview for Prodromal Syndrome (SIPS) and belonged to at least one of CHR groups: attenuated positive symptoms (APS), brief intermittent psychotic symptoms (BIPS), and genetic risk with deterioration (GRD) (Jung et al., Reference Jung, Jang, Kang, Choi, Shin, Kim and Kwon2010). The exclusion criteria for all subjects were as follows: any current or lifetime Axis I psychotic disorder or substance dependence other than cigarette smoking, past or present neurological disease or traumatic brain injury with loss of consciousness, a significant medical condition that could manifest as a psychiatric condition, past or current use of antipsychotic medications to manage attenuated psychotic symptoms, and/or an intelligence quotient (IQ) below 70. Clinical and cognitive function assessment was conducted by psychiatrists and clinical psychologists at 6-month intervals for the first 2 years and at 1-year intervals afterward for up to 10 years (online Supplementary Table S1). All subjects received case management and supportive psychotherapy monthly from psychiatrists. If the development of psychosis was suspected during these treatment sessions, the subjects also received an additional assessment session within a week, even during the regular evaluation period. The transition to psychosis was determined if the subject met the Presence Of Psychotic Syndrome (POPS) criteria of SIPS (Jung et al., Reference Jung, Jang, Kang, Choi, Shin, Kim and Kwon2010). Of the overall sample, 14 cases were excluded due to withdrawal; the final sample consisted of 208 CHR individuals. Written informed consent forms were obtained from all subjects, and if they were under the age of 18 years, guardian consent was also obtained. The present study was approved by the Institutional Review Board of the Seoul National University Hospital.
Clinical and cognitive assessments
Depressive and anxious symptoms were evaluated using the Hamilton Depression Rating Scale (HAM-D) (Yi et al., Reference Yi, Bae, Ahn, Park, Noh, Shin and Kim2005) and the Hamilton Anxiety Rating Scale (HAM-A) (Hamilton, Reference Hamilton1959), respectively. The Global Assessment of Functioning (GAF) was administered to assess the current levels of overall symptoms and functioning (Yi, Chung, Lee, & Lee, Reference Yi, Chung, Lee and Lee2003), and the percentage drop over the past year was calculated. Social functioning was also assessed using the Korean version of the Social Functioning Scale (SFS), which consists of seven domains of social behavior: social engagement/withdrawal, interpersonal behavior, independence – performance, independence – competence, recreation, prosocial activities, and employment/occupation (Kim & Lee, Reference Kim and Lee2009).
Each subject's IQ was estimated using the Korean version of the Wechsler Adult Intelligence Scale-III (WAIS) (Yeom, Park, Oh, & Lee, Reference Yeom, Park, Oh and Lee1992). We implemented four subsets of WAIS consisting of the Vocabulary, Arithmetic, Block Design, and Picture Arrangement, and assessment was performed by a trained researcher who majored in psychology with a master's degree or higher, or by a skilled clinical psychologist. The following neuropsychological tests were administered to assess cognitive function in the CHR population: Digit span, a subset of the WAIS, was used to measure attention/working memory. Processing speed was assessed with the Trail Making Test Part A (TMT-A) (Reitan, Reference Reitan1958). Measures of divided attention were assessed with the Trail Making Test Part B (TMT-B) (Reitan, Reference Reitan1958) and set-shifting from perseverative errors in the Wisconsin Card Sorting Test (WCST) (Chelune & Baer, Reference Chelune and Baer1986). Verbal fluency was evaluated with the Korean version of the verbal fluency task for semantic fluency (Kim et al., Reference Kim, Lee, Oh, Hong, Lee, Son and Kim2013) and the Controlled Oral Word Association Test (COWAT) for phonemic fluency (Kang, Chin, Na, Lee, & Park, Reference Kang, Chin, Na, Lee and Park2000). Verbal memory was assessed with the Korean version of the California Verbal Learning Test (K-CVLT), for which we examined the sum of the immediate and delayed scores (Kim & Kang, Reference Kim and Kang1999). The sum of the immediate and delayed scores on the Rey–Osterrieth Complex Figure Test (ROCF) was used to evaluate visual memory (Shin, Park, Park, Seol, & Kwon, Reference Shin, Park, Park, Seol and Kwon2006).
Statistical analysis
All analyses were conducted in Stata version 16 (Stata Corp.) and R version 3.6.0. (Comprehensive R Archive Network). Comparisons of baseline demographic and clinical characteristics were performed with χ2 tests and independent t tests. The cumulative incidence rates of transition to psychosis during the follow-up period were estimated with the Kaplan–Meier analysis. In developing the predictive model for psychosis, the LASSO-penalized Cox regression was used to developing the predictive model with subjects who had different follow-up periods (Simon, Friedman, Hastie, & Tibshirani, Reference Simon, Friedman, Hastie and Tibshirani2011). Predictors included a total of 56 candidate variables, including base demographic, clinical, and cognitive variables. Missing data were handled using the multiple imputation method with k = 10. We selected the LASSO model that resulted in minimal prediction error using 10-fold cross-validation. We then conducted a bootstrap test with 1000 iterations to estimate the 95% confidence interval of the predictive performance. For internal validation, a predictive individual prognostic index (PI) was generated, and k-means clustering analysis was performed to stratify the risk of transition to psychosis using the LASSO model with the elbow method to determine the optimal k. Kaplan–Meier analysis was performed to estimate the incidence of psychosis in each cluster. Then, the log-rank test was used to determine the different survival functions.
Results
The total SYC sample consisted of 208 CHR participants. The follow-up time ranged from a minimum of 30 days to a maximum of 12 years (online Supplementary Table S1). The mean follow-up duration was 3.5 years (s.d. 2.6 years). Thirty-eight participants developed a psychotic disorder during the follow-up period. Table 1 shows a Kaplan–Meier estimate of the survival function for the time to transition to psychotic disorders. The total cumulative incidence rate of transition was 32.6% (95% CI 21.8–46.9). Table 2 presents the baseline demographic and clinical characteristics of the participants. There was no significant difference between the groups in demographic status. Converters had higher positive symptom scale scores on the Scale of Prodromal Symptoms (SOPS), a percentage of changes in GAF scores in 1 year, and lower estimated current IQ, CVLT, and Strange Stories task scores. The social engagement/withdrawal and prosocial domains of the SFS showed trend-level differences between groups.
Table 1. Kaplan–Meier estimates of transition rates over 10 years
Table 2. Demographic and clinical characteristics of the participants
BIPS, Brief Intermittent Psychotic Syndrome; APS, Attenuated Psychosis Symptoms Syndrome; GRD, Genetic Risk and Deterioration Syndrome; SOPS, Scale of Prodromal Symptoms; HAM-D, Hamilton Depression Rating Scale; HAM-A, Hamilton Anxiety Rating Scale; GAF, Global Assessment of Functioning; SFS, Social Functioning Scale; CVLT, California Verbal Learning Test; ROCF, Rey–Osterrieth Complex Figure Test; TMT, Trail Making Test; COWAT, Controlled Oral Word Association Test; WCST, Wisconsin Card Sorting Test.
a Scores of 1–3 indicate high status and scores of 4–5 indicate low status.
In the SYC sample, 7.4% of the data overall were missing. The ROCF scores had the highest amount of missing data, with 24.5% of values missing. Category verbal fluency and COWAT scores had 21.2% of values missing. The LASSO Cox prediction model identified six baseline variables, including a percentage of changes in GAF scores in 1 year, IQ, CVLT, Strange Stories task, and two domains of the SFS (Table 3). The prediction model had a cross-validated Harrell's C-index of 0.78 (95% CI 0.777–0.783). The optimal number of clusters for the stratification of risk using the elbow method was 3. Figure 1 shows a Kaplan–Meier estimate of the survival function for the time to transition to psychotic disorders for each cluster. The cut-off values for each cluster were −1.86 for cluster 1 and −2.33 for cluster 3. The 10-year survival rate for each cluster was 15.9% for cluster 1 (n = 36), and 72.1% for cluster 2 (n = 109), 89.6% for cluster 3 (n = 99), and the clusters showed a significant difference in survival rate (cluster 1 v. 2: χ2 = 37.06, p < 0.001, cluster 2 v. 3: χ2 = 9.1, p = 0.003). Patients in cluster 1 showed an approximately 44% risk of developing psychosis at 3 years, whereas none of the patients in cluster 3 developed psychosis before 3 years (Table 4; online Supplementary Table S2).
Fig. 1. Kaplan–Meier survival estimates for the three clusters. Cluster 1 is a high-risk subgroup with 84.1% incidence (n = 36), cluster 2 is a medium-risk subgroup with 27.9% incidence (n = 109), and cluster 3 is a low-risk subgroup with 10.4% incidence within 10-year follow-up.
Table 3. The baseline variables identified by the LASSO Cox model that significantly predicted transition to psychosis
a California Verbal Learning Test.
b Global Assessment of Functioning.
c Social Functioning Scale.
Table 4. Kaplan–Meier estimates of transition rates in three clusters
Discussion
This study aimed to develop and internally validate a model for predicting the incidence of psychosis in CHR individuals to provide useful assistance in clinical practice. We developed a model that includes social functioning, social cognition, functional decline, verbal memory, and IQ; this model demonstrated fair predictive ability. Using this model, we divided the high-risk groups into three clusters, all of which showed significant differences in the incidence of psychosis. To the best of our knowledge, this is the first study using modern machine learning techniques to model a wide range of variables covering demographic, clinical, and cognitive functions in long-term cohorts spanning more than 10 years.
In this study, approximately 32% of subjects developed to psychosis during up to 10 years of follow-up. This transition rate is somewhat higher incidence compared to the results of a meta-analysis with an incidence rate of 20% (Fusar-Poli et al., Reference Fusar-Poli, Cappucciati, Borgwardt, Woods, Addington, Nelson and McGuire2016a). Although half of the total incidence occurs within the first 8 months of the 2-year follow-up period (Fusar-Poli et al., Reference Fusar-Poli, Cappucciati, Borgwardt, Woods, Addington, Nelson and McGuire2016a), in a long-term follow-up result of Nelson et al, 17% of cases are converted during the follow-up period after 3 years, and 5% are converted after 5 years (Nelson et al., Reference Nelson, Yuen, Wood, Lin, Spiliotacopoulos, Bruxner and Yung2013). This is in line with the result of our cohort, which showed a conversion rate of 21% after 3 years and 10% after 5 years. In our predictive model with a long-term follow-up duration, the estimated predictive ability had a C-index of 0.78, which is comparable to the results of other previous studies with similar designs (Addington et al., Reference Addington, Liu, Perkins, Carrion, Keefe and Woods2017; Ciarleglio et al., Reference Ciarleglio, Brucato, Masucci, Altschuler, Colibazzi, Corcoran and Girgis2019), but never meaningfully higher. Addington et al. tracked 172 subjects for 2 years and reported that 29 subjects were converted, while Ciarleglio et al. followed 199 subjects for 2 years and 64 transitions (Addington et al., Reference Addington, Liu, Perkins, Carrion, Keefe and Woods2017; Ciarleglio et al., Reference Ciarleglio, Brucato, Masucci, Altschuler, Colibazzi, Corcoran and Girgis2019). Given this, it can be assumed that long-term follow-up periods need not be mandatory to increase the predictive power of the model. However, since we have not yet performed external validation of this model, it will be necessary to examine whether the predictive accuracy of our model can be replicated in an independent sample with long-term follow-up duration. In regard to predictive models using the LASSO method, Addington et al. externally validated their model (Addington et al., Reference Addington, Liu, Perkins, Carrion, Keefe and Woods2017; Cornblatt et al., Reference Cornblatt, Carrion, Auther, McLaughlin, Olsen, John and Correll2015). Of course, predictive models developed using classical statistical techniques have been externally validated and have undergone refinements several times (Fusar-Poli et al., Reference Fusar-Poli, Davies, Rutigliano, Stahl, Bonoldi and McGuire2019a, Reference Fusar-Poli, Werbeloff, Rutigliano, Oliver, Davies, Stahl and Osborn2019b; Osborne & Mittal, Reference Osborne and Mittal2019; Zhang et al., Reference Zhang, Li, Tang, Niznikiewicz, Shenton, Keshavan and Wang2018), but it will be necessary to continuously refine the model through the optimization of coefficients and variable selection using the LASSO technique in the same sample. More recently, an advanced dynamic prognostic model that combines demographic and clinical variables to forecast the development of psychosis was introduced (Studerus, Beck, Fusar-Poli, & Riecher-Rossler, Reference Studerus, Beck, Fusar-Poli and Riecher-Rossler2020). This model can keep up with changes in symptoms over time, allowing the model to be updated at various time points as the patient is followed. In addition to the prediction models for CHR, an alternative perspective model has also been developed and validated for those who have not yet been diagnosed with CHR (Fusar-Poli et al., Reference Fusar-Poli, Rutigliano, Stahl, Davies, Bonoldi, Reilly and McGuire2017, Reference Fusar-Poli, Werbeloff, Rutigliano, Oliver, Davies, Stahl and Osborn2019b). This model can be applied to a large number of populations in the community. Therefore, in the future, it will be necessary to develop appropriate models that reflect the characteristics of the sample at each stage, such as patients who show attenuated symptoms but have not yet been diagnosed with CHR, those who have just been diagnosed with CHR, and those who are being followed up.
Moreover, based on the results of the clustering analysis, our predictive model classified the CHR into three subgroups characterized by different levels of risks. Risk stratification has already been implemented several times (Addington et al., Reference Addington, Liu, Perkins, Carrion, Keefe and Woods2017; Michel, Ruhrmann, Schimmelmann, Klosterkotter, & Schultze-Lutter, Reference Michel, Ruhrmann, Schimmelmann, Klosterkotter and Schultze-Lutter2014; Ruhrmann et al., Reference Ruhrmann, Schultze-Lutter, Salokangas, Heinimaa, Linszen, Dingemans and Klosterkotter2010). Ruhrmann et al. first classified the samples into four PIs (Ruhrmann et al., Reference Ruhrmann, Schultze-Lutter, Salokangas, Heinimaa, Linszen, Dingemans and Klosterkotter2010). At 18 months, the incidence of class I was 3.5%, and that of class IV was 85.1%. They also developed a new predictive model that was divided into four classes using diagnostic criteria and cognitive function (Michel et al., Reference Michel, Ruhrmann, Schimmelmann, Klosterkotter and Schultze-Lutter2014). In this study, the cumulative hazard rate of class I was 0, and that of class IV was 1.29. However, these studies were not cross-validated; thus, it is difficult to ignore the risk of overfitting problems. Recently, Addington et al. introduced a cross-validated predictive model (Addington et al., Reference Addington, Liu, Perkins, Carrion, Keefe and Woods2017). They divided CHR into three risk groups, but there was no data on the incidence of each group. In contrast, in our model, we found that the three clusters that were stratified using the LASSO model had significantly different degrees of risk. Interestingly, in cluster 1, approximately half of the cases transitioned to psychosis within 3 years of follow-up, whereas in cluster 3, no transition occurred within that time period. Moreover, unlike simply distinguishing between the converters and the non-converters, there were a distinct demographic, symptom, and cognitive differences in each subgroup (online Supplementary Table S3). Our result will help facilitate a personalized therapeutic approach to different degrees of risks among high-risk individuals and will enrich future recruitment efforts, such as targeting only CHR patients with moderate to high risk.
Consistent with previous studies, our model included general functioning, social cognition, social functioning, and verbal memory as significant predictors (Cannon et al., Reference Cannon, Yu, Addington, Bearden, Cadenhead, Cornblatt and Kattan2016; Cornblatt et al., Reference Cornblatt, Carrion, Auther, McLaughlin, Olsen, John and Correll2015; Malda et al., Reference Malda, Boonstra, Barf, de Jong, Aleman, Addington and Pijnenborg2019; Zhang et al., Reference Zhang, Xu, Tang, Li, Tang, Cui and Group2019). However, demographic and prodromal symptom variables were not significant in our analyses and were not included in our model. A recent meta-analysis revealed that the demographic variables of subjects are significant predictors of transition to psychosis. In this study, each of the predictors was stratified into groups ranging from convincing evidence (class I) to weak evidence (class IV) (Oliver et al., Reference Oliver, Reilly, Baccaredda Boy, Petros, Davies, Borgwardt and Fusar-Poli2020). However, out of 26 putative risk/protective factors, there were no class I predictors, while only global functioning and attenuated positive psychotic symptoms are in line with previous findings that the dynamic functional change, rather than the static functional status at a single timepoint, better predicts prognosis (Tarbox et al., Reference Tarbox, Addington, Cadenhead, Cannon, Cornblatt, Perkins and Woods2013; Velthorst et al., Reference Velthorst, Nelson, Wiltink, de Haan, Wood, Lin and Yung2013; Zhang et al., Reference Zhang, Xu, Tang, Li, Tang, Cui and Group2019). Similarly, we observed no significance for the SOPS score. Unlike our model, some other predictive models include both cognitive function and clinical symptoms (Addington et al., Reference Addington, Liu, Perkins, Carrion, Keefe and Woods2017; Cornblatt et al., Reference Cornblatt, Carrion, Auther, McLaughlin, Olsen, John and Correll2015). However, these studies used the rescaled SOPS scores for their models and not the raw scores, which may have resulted in the discrepancy of the results. Further investigation into whether the cognitive function is more influential and essential than clinical symptom scores for predicting psychosis is needed.
The present study has several limitations. First, we did not utilize the variables that were acquired during the patient follow-up, including pharmacotherapy, cognitive-behavioral therapy, and compliance, in our model. Our primary goal was to predict the risk of psychosis using only baseline information. This is reasonable, as at the time of the patients' first evaluation, no interventions had been administered. Although we used 10-year follow-up data, we assumed that this long duration might compensate for the delay in the transition to psychosis related to the use of medication, a dynamic model that reflects both the baseline and longitudinal change may be a better way to solve this problem in the future (Studerus et al., Reference Studerus, Beck, Fusar-Poli and Riecher-Rossler2020; Yuen et al., Reference Yuen, Mackinnon, Hartmann, Amminger, Markulev, Lavoie and Nelson2018). Second, external validation of the model was not performed in our current study. For external validation, the variables used in the model should be identical. However, the actual tasks used to assess specific cognitive domains may differ from study to study. To overcome this problem, a model needs to be developed that utilizes only the variables that overlap across studies. Third, this study was conducted as a data-driven study. Given the number of variables being injected, the sample size may not be large enough. This is a frequently mentioned issue in clinical prediction. Even with the application of data science technique to the current study, it is still probable that the model developed could be biased against the sample's characteristics. It is still necessary to be conducted with the selection of variables based on hypotheses and the development of a model. Last, it has not been explored what life events have affected the transition to psychosis. Further research on this will be needed, especially since the appraisal of life experiences is an important axis of the bio-physico-social model of schizophrenia (Garety et al., Reference Garety, Waller, Emsley, Jolley, Kuipers, Bebbington and Freeman2015).
In summary, we developed a predictive model with 10-year follow-up data of CHR patients. One-third of the CHR patients developed psychosis over a sufficient follow-up period. Our model showed that verbal ability, social cognition, social functioning, functional decline, and general intelligence are important predictors of the transition to psychosis. CHR sample was divided into three clusters according to the degree of risk. We believe that our model could facilitate a personalized therapeutic approach to different risks in high-risk individuals.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0033291720004675
Acknowledgment
This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT and Future Planning (grant no. 2019R1A2B5B03100844; 2017M3C7A1029610). We declare that there is no potential conflict of interest in this study.
Open access
