Structure of psychopathology in adolescents and its association with high-risk personality traits

Abstract

The present study examined high-risk personality traits and associations with psychopathology across multiple levels of a hierarchical-dimensional model of psychopathology in a large adolescent, general population sample. Confirmatory factor analyses were run using data from two randomized controlled trials of Australian adolescents (N = 8,654, mean age = 13.01 years, 52% female). A higher-order model – comprised of general psychopathology, fear, distress, alcohol use/harms, and conduct/inattention dimensions – was selected based on model fit, reliability, and replicability. Indirect-effects models were estimated to examine the unique associations between high-risk personality traits (anxiety sensitivity, negative thinking, impulsivity, and sensation seeking) and general and specific dimensions and symptoms of psychopathology. All personality traits were positively associated with general psychopathology. After accounting for general psychopathology, anxiety sensitivity was positively associated with fear; negative thinking was positively associated with distress; impulsivity was positively associated with conduct/inattention; and sensation seeking was positively associated with alcohol use/harms and conduct/inattention, and negatively associated with fear. Several significant associations between personality traits and individual symptoms remained after accounting for general and specific psychopathology. These findings contribute to our understanding of the underlying structure of psychopathology among adolescents and have implications for the development of personality-based prevention and early intervention programs.

Introduction

Personality is a well-established risk factor for psychopathology, with evidence for links with a variety of mental and substance use disorders (Kotov et al., 2010; Tackett, 2006; Watson et al., 2005, 2019; Widiger et al., 2019; den Akker et al., 2013). However, there are high rates of comorbidity among disorders, making it difficult to identify reliable links between personality traits and mental disorders. Recent advances in the study of the underlying structure of psychopathology supports a data-driven, hierarchical-dimensional model of psychopathology which accounts for comorbidity among disorders and enables the study of relations with external variables at various levels of specificity. Yet, only a small number of studies have examined the associations between personality traits and psychopathology within this framework among adolescents to date (Lynch et al., 2021). Further, past research has primarily focused on associations between normal-range trait domains (e.g., “the Big 5” or five factor model traits) and distinct disorders (Sellbom et al., 2020). Examining established high-risk personality traits (e.g., lower-order facets of neuroticism or sub-dimensions of disinhibition) may be informative in terms of refining our understanding of the underlying structure of psychopathology and for advancing knowledge of personality-related risk for psychopathology. Focusing on these associations in a hierarchical-dimensional model of psychopathology, for example, may be particularly useful for clarifying the role personality may play in the development of general and specific forms of psychopathology, from individual symptoms up to broad transdiagnostic dimensions.

Hierarchical-dimensional models of psychopathology

Psychopathology has historically been conceptualized in terms of discrete diagnostic categories. However, categorical approaches to conceptualizing psychopathology tend to have poor reliability and low specificity, as evidenced by the high rates of comorbidity between disorders and heterogeneity within disorders (Kotov et al., 2017; Ofrat & Krueger, 2012). In response to these issues, there has been a renaissance of empirical studies examining the underlying structure of psychopathology. This work has generated a wealth of evidence for conceptualizing psychopathology in a hierarchical-dimensional framework, such as the Hierarchical Taxonomy of Psychopathology (HiTOP; Kotov et al., 2017, 2021). At the apex of hierarchical-dimensional models sits a general factor of psychopathology, which captures shared variance among mental and substance use disorders. Beneath the general factor sit more specific factors that reflect shared variance among closely related disorders, such as internalizing and externalizing dimensions. Internalizing captures comorbidity among, for example, phobias, eating, obsessive-compulsive, and mood and anxiety-related disorders, whereas externalizing reflects shared variance among, for example, substance use, conduct, antisocial and impulse related disorders. There is also evidence that these dimensions may be partitioned into even narrower dimensions (Krueger et al., 2021; Watson et al., 2022). For example, internalizing includes sub-dimensions of fear and distress, and externalizing includes sub-dimensions of substance use and antisocial behavior.

Personality and psychopathology

Previous research has consistently shown that there are strong associations between certain personality traits and certain forms of psychopathology (Brandes & Tackett, 2019; Haltigan et al., 2018; Kotov et al., 2010; Widiger et al., 2019). For example, neuroticism has been established as an important risk factor for internalizing and general psychopathology dimensions (Brandes et al., 2019; Castellanos-Ryan et al., 2016; Etkin et al., 2021; Kotov et al., 2010). Similarly, antagonism and impulsivity traits are both associated with externalizing and substance misuse problems (Castellanos-Ryan et al., 2016; Etkin et al., 2022; Kotov et al., 2010; Lynam & Miller, 2019). However, currently very little is known about associations at the subfactor (e.g., fear and distress) and symptom levels of hierarchical-dimensional models of psychopathology (Brandes & Tackett, 2019; Kotov et al., 2010). Similarly, although personality can also be conceptualized hierarchically, most research thus far has focused on broad personality traits, rather than the underlying facets or aspects of these traits (Brandes & Tackett, 2019; Tackett, 2006; Watts et al., 2019). Amid renewed calls for research on the integration of and differentiation between personality and psychopathology (Hopwood et al., 2022; Wright & Hopwood, 2022), exploration of associations between narrower components of personality and subfactor and symptom levels of psychopathology could help clarify the structure of lower levels of a hierarchical model of psychopathology or point to shared or distinguishable elements of personality and psychopathology.

The four-factor model of vulnerability integrates and distills previous research linking neuroticism as well as inhibited and disinhibited personality traits to substance misuse and comorbid psychopathology via distinct cognitive and motivational pathways (Castellanos-Ryan et al., 2016; Castellanos-Ryan & Conrod, 2012). Although this model was initially conceptualized as a model of personality-based risk for substance use, there is considerable evidence that the traits are also associated with higher levels of and increased risk for other forms of psychopathology (Carragher et al., 2016; Castellanos-Ryan et al., 2016). In contrast to comprehensive models of personality, such as the Big Five, the four-factor model of vulnerability is comprised of four particularly compelling personality-based risk factors for substance use problems, and psychopathology more broadly (Castellanos-Ryan & Conrod, 2012). Inhibited/neurotic traits of negative thinking (tendency to experience hopelessness and low positive affect) and anxiety sensitivity (fear of anxiety-related physical sensations relating to beliefs that such sensation could lead to harmful consequences) are associated with mood and anxiety-related problems, as well as increased substance use problems (to manage or relieve symptoms of anxiety/depression). Disinhibition is partitioned into two sub-domains: impulsivity, which broadly reflects a failure to inhibit behaviors likely to result in negative consequences, and sensation seeking, which reflects a willingness to take risks for the sake of novel experiences. Individuals high in impulsivity have difficulties with emotion and behavioral regulation, tend to experience more conduct related problems and are at increased risk for substance misuse through enhancement, coping and conformity motives. Whereas individuals high in sensation seeking are more likely to develop substance use problems due to a heightened susceptibility to the rewarding properties of alcohol and other substances. Sensation seeking appears to be more directly related to substance misuse problems than other externalizing related problems (Castellanos-Ryan & Conrod, 2011).

Prior research on the four-factor model of vulnerability and hierarchical-dimensional model of psychopathology have revealed theoretically aligned patterns of association with transdiagnostic dimensions (though there are some exceptions). For example, negative thinking and anxiety sensitivity appear to be prospectively and concurrently associated with greater internalizing and general psychopathology (Carragher et al., 2016; Castellanos-Ryan et al., 2016), and either unrelated or inversely related to externalizing (although one study reported a positive association between negative thinking and externalizing, but internalizing symptoms were not included in the model (e.g., Castellanos-Ryan & Conrod, 2011). Similarly, impulsivity and sensation seeking appear to be more closely related to externalizing related dimensions, with impulsivity more closely related to conduct/general externalizing and sensation seeking more closely aligned with substance misuse and related harms. One study also reported a negative association between sensation seeking and negative thinking (Carragher et al., 2016). Exploration of unique associations with sub-dimensions of internalizing and externalizing, or indeed individual symptoms, may help clarify some of the inconsistent findings from previous studies. To our knowledge, no studies have examined associations between these high-risk personality traits and lower levels of a hierarchical-dimensional model of psychopathology.

Methodological considerations

Despite strong empirical support for hierarchical-dimensional models of psychopathology, there are some outstanding conceptual and methodological issues. Critically, there is currently no clear consensus on which statistical model is most appropriate for studying the structure of psychopathology. Correlated factors, bifactor and higher-order models appear most frequently in the literature (Forbes, Greene, et al., 2021; Lahey et al., 2021). These models are closely related yet offer different interpretations of the underlying structure of psychopathology. For example, a higher-order model’s general factor reflects the shared variance among the lower-order specific factors, whereas a bifactor model’s general factor directly reflects shared variance among all indicators. Further, in correlated factors and higher-order models, the specific factors reflect shared variance among a set of observed variables, whereas in a bifactor model the specific factors are uncorrelated and reflect variance unique to the factor (after the shared variance among indicators is accounted for by the general factor). When these models are directly compared using traditional goodness-of-fit statistics, the bifactor model typically outperforms the others (e.g., Greene et al., 2019). However, there are increasing concerns about relying on goodness-of-fit statistics to adjudicate between models, as bifactor models tend to overfit data which can result in inflated goodness-of-fit statistics and consequently lead to the premature dismissal of other plausible structures (Bonifay et al., 2017). As such, there have been calls to consider additional metrics for model reliability and replicability when studying the underlying structure of psychopathology (Forbes, Greene, et al., 2021; Rodriguez et al., 2016b). Although very few studies have reported on these additional metrics to date, two previous studies have found a higher-order model to outperform a bifactor model of psychopathology in young people (Lees et al., 2020; Sunderland et al., 2020).

Another important methodological issue that requires further attention is the unit of measurement used for observed variables. Much of the past research on hierarchical-dimensional models has been based on diagnostic level indicators (Forbes, Sunderland, et al., 2021). This may inadvertently constrain models to the framework of the prevailing diagnostic taxonomies. Symptom-level approaches are theorized to be better able to capture the underlying structure of psychopathology because they are not bound by the constraints of existing diagnostic categories. Further, symptom-level approaches may be more sensitive to detecting emerging forms of psychopathology (e.g., cases in which symptoms are present, but the individual does not meet full diagnostic criteria; Forbes, Sunderland, et al., 2021). Given that many mental disorders first emerge during adolescence (Costello et al., 2011; Kessler et al., 2011), it is likely that symptom-level analyses may be more appropriate for studying psychopathology in adolescents. Another advantage of symptom-level analysis is that it enables the identification of important symptoms with unique links to risk or vulnerability factors. These symptoms may highlight potential etiological mechanisms and therefore could be salient intervention targets. In summary, symptom-level analyses are important both for advancing our understanding of the underlying causes of mental and substance use disorders and ultimately, for facilitating the identification of better intervention targets.

The present study

The aim of the present research was to conduct a more thorough exploration of the structure of psychopathology and associations with high-risk personality traits among adolescents than previously available. We aimed to examine a variety of hierarchical-dimensional structures of psychopathology using a symptom-level approach and evaluate the models using more rigorous methods of model evaluation and selection. Specifically, we assessed four alternative models of adolescent psychopathology: bifactor, higher-order, four-correlated factors and a one-factor unidimensional model. As we planned to evaluate the structural validity through additional metrics beyond traditional fit indices which not been commonly examined in previous research, we did not have any specific expectations about which model would perform the best.

Extending previous research, we also aimed to examine the direct and indirect effects of high-risk personality traits on psychopathology across three hierarchical levels: general psychopathology, specific factors, and symptoms. To our knowledge, this is the first study to examine symptom-level associations with high-risk personality traits among adolescents. As such, we did not have any specific expectations about associations between the high-risk personality traits and individual symptoms. We did, however, expect that all high-risk personality traits would be positively associated with general psychopathology; impulsivity and sensation seeking would be positively associated with externalizing related specific dimensions; and anxiety sensitivity and negative thinking would be associated with internalizing related specific dimensions.

Methods

Participants

The sample was derived from two large cluster randomized controlled trials investigating the effectiveness of eHealth prevention programs in Australia – the Climate and Preventure (CAP) and Climate Schools Combined (CSC) studies (Newton et al., 2012; Teesson et al., 2014). The present study examined baseline data from these cohorts. The CAP cohort comprises 2,268 students with a mean age of 12.96 years (SD = 0.46) recruited through 27 schools in 2012. Within the CAP cohort, 972 were female (42.89%) and 1,941 were born in Australia (86.84%). The CSC cohort comprises 6,386 students with a mean age of 13.03 years (SD = 0.61) from 71 schools in 2014. Within the CSC cohort, 3,502 were female (54.83%) and 5,147 were born in Australia (84.17%). The combined sample contained 8,654 students, 53 of which were missing on all variables and were excluded from analyses, resulting in a final sample size of 8,601 (mean age 13.01 years, SD = 0.57) from 98 schools, of which 4,474 were female (51.71%) and 7,088 were born in Australia (84.88%).

Measures

Psychopathology

Item-level responses from measures of psychopathology used in both CAP and CSC baseline assessments were used in the present study. Due to the low prevalence of substance use and psychopathology in this general population sample, all items were recoded into binary indicators to reduce the number of sparse cells and improve the stability of the models and overall precision of the estimates. Cut points were determined based on inspection of the distribution of responses (further details provided below). The measures used to assess psychopathology are described below and a summary of the items, proportions and counts is provided in Supplementary Table S1.

Strengths and Difficulties Questionnaire (SDQ). The SDQ is a brief, 25-item questionnaire that measures emotional and behavioral difficulties over the past 6 months and is comprised of four subscales: conduct problems, emotional symptoms, hyperactivity, peer problems and prosocial behavior (Goodman, 2001). Items were selected to load onto the fear, distress and conduct dimensions as informed by previous analyses (Carragher et al., 2016; Goodman et al., 2010). Reverse-scored items were removed due to poor performance and previously documented problems (Van De Looij-Jansen et al., 2011). Items from the SDQ were recoded into binary indicators with levels representing “not true” or “true” (i.e., “somewhat true” or “certainly true”).

Kessler Psychological Distress Scale (K6). The K6 is a 6-item screening tool for psychological distress due to symptoms of depression and anxiety over the past 4 weeks (Kessler et al., 2002, 2003), and has been found to be a valid and reliable measure of psychological distress among adolescents (Ferro, 2019; Mewton et al., 2016). Two items were loaded onto the fear dimension, and the remaining four items loaded onto the distress dimension. Items were recoded as “none of the time” or “any time” (i.e., “a little of the time,” “some of the time,” “most of the time,” “all of the time”).

Rutgers Alcohol Problem Index. The Rutgers Alcohol Problem Index measures alcohol-related consequences experienced over the past 6 months and has been validated amongst high-school aged people as a measure of alcohol-related problems (Neal et al., 2006; White & Labouvie, 1989). A shortened 8-item version that had previously demonstrated adequate validity and reliability for assessing alcohol-related problems among young people was administered to the CAP cohort (Topper et al., 2011). As such, only these items have been used in the present study. Items were recoded as “did not experience in the last 6 months” and “experienced at least one time in the last 6 months.”

Patterns of Alcohol Use. Patterns of alcohol use over the past 6 months were assessed using three items adapted from the School Health and Alcohol Harm Reduction Project’s “Patterns of Alcohol Use” index (McBride et al., 2004). Specifically, there were three items measuring frequency of alcohol use in the past 6 months, quantity of alcohol consumed in the past 6 months and frequency of drinking above low risk levels in the past 6 months. Items were recoded into “none or less than monthly” and “once a month or more.”

High-risk personality traits

Substance Use Personality Risk Profile Scale (SURPS) is a 23-item measure of personality risk for substance misuse, comprised of four distinct subscales: hopelessness/negative thinking, anxiety sensitivity, sensation seeking and impulsivity (Woicik et al., 2009). The SURPS asks participants to indicate the extent to which they agree with each item on a 4-point scale (1 = strongly disagree, 2 = disagree, 3 = agree, 4 = strongly agree). Total scores were calculated for each subscale and used in subsequent analyses. The SURPS has demonstrated good validity and reliability as a measure of personality-related risk for substance use and co-occurring psychopathology among young people across multiple cohorts (Castellanos-Ryan et al., 2013; Newton et al., 2016; Woicik et al., 2009).

Analytic plan

Analyses in the present study were conducted in the following broad steps: 1) model estimation, 2) model evaluation via traditional goodness-of-fit and contemporary model reliability and replicability indices, 3) measurement invariance testing, and 4) finally the associations between personality traits and psychopathology dimensions were assessed via regression and indirect-effects models. Further details of each step are provided below.

First, we estimated four alternative structural models of psychopathology using confirmatory factor analysis: 1) a one-factor model with all items loading on a single latent variable representing general psychopathology; 2) a four-correlated factors model with four latent variables representing fear, distress, alcohol use/harms and conduct/inattention; 3) a bifactor model with all indicators loading onto a single latent variable representing general psychopathology as well as four orthogonal (i.e., uncorrelated) latent variables representing fear, distress, alcohol use/harms and conduct/inattention, and 4) a higher-order model comprising four lower-order factors representing fear, distress, alcohol use/harms and conduct/inattention and a higher-order general psychopathology latent variable that accounts for the correlations among the lower-order factors.

The structural models were based on prior symptom-level studies among adolescents, which have consistently found evidence for a general psychopathology factor, and at least two specific or correlated factors representing externalizing and internalizing symptoms (Afzali et al., 2017; Carragher et al., 2016; Haltigan et al., 2018; Levin-Aspenson et al., 2019). Notably, one study found that internalizing bifurcated into fear and distress sub-dimensions (Levin-Aspenson et al., 2019), and another study found evidence for separate attention and externalizing factors (Haltigan et al., 2018).

All models accounted for school-level clustering and were estimated in Mplus version 8.4 using robust weighted least squares and robust maximum likelihood estimation methods to generate a range of fit statistics.

Second, the structural validity of each model was evaluated with goodness-of-fit and latent variable reliability indices (Forbes, Greene, et al., 2021; Rodriguez et al., 2016a). Incremental fit indices, including root mean square error of approximation (RMSEA) comparative fit index (CFI, values >0.95) and Tucker–Lewis index (TLI) were used to assess model fit, where RMSEA values <0.6, and CFI and TLI values >0.95 indicate close fit (Brown, 2014). Models were also compared using the information criteria, including the Akaike information criterion, Bayesian information criterion, and the sample size-adjusted Bayesian information criterion, for which lower values indicate superior fit (Raftery, 1995).

Given the tendency for goodness-of-fit statistics to be biased towards selecting bifactor models, additional reliability and replicability indices were calculated (Forbes, Greene, et al., 2021; Rodriguez et al., 2016a). Specifically, the H coefficient, which gives an indication of the construct replicability (H, ideally >0.8), omega (ideally >0.75), which represents the proportion of variance accounted for by a single latent variable, omega hierarchical (OmegaH, ideally >0.8), indicates the proportion of variance accounted for by the general factor, and omega hierarchal subscale (OmegaHS, ideally >0.75) which represents the variance accounted for by a specific factor after removing variance accounted for by the general factor. Additionally, we calculated the explained common variance which provides an indication of the importance of the general factor relative to the specific factors (Explained Common Variance, ideally >0.7), and the explained common variance of specific factors which gives an indication of the uniqueness of a specific factor (ECV_S = Explained Common Variance of specific factors, ideally >0.7). Unidimensionality was examined by calculating the percent of uncontaminated correlations (Percent of Uncontaminated Correlations, values >0.7 indicate unidimensionality) and absolute relative parameter bias (Absolute Relative Parameter Bias; 10–15% is acceptable). Percent of Uncontaminated Correlations indicates the proportion of unique correlations among indicators (i.e., parameter estimates) that can be explained by a general factor alone, thus high Percent of Uncontaminated Correlations indicates that the parameter estimates are relatively unbiased by multidimensionality and supports a unidimensional structure (Reise et al., 2013). Absolute Relative Parameter Bias compares the absolute difference between parameter estimates between a unidimensional model and a bifactor (or other multidimensional) model. For the higher-order model, these indices were calculated following a Schmid–Leiman transformation (SLT), which orthogonalizes the latent variables. Following a SLT, the lower-order factors in a higher-order model are like the specific factors in a bifactor model. Whereas the latent variables from a correlated factors model are like the lower-order factors prior to SLT and are useful for understanding their reliability as standalone constructs. Models found to have acceptable structural validity according to the goodness-of-fit, reliability and replicability indices progressed to the next step.

Third, to examine the robustness of the models selected in the previous step and ensure that it was appropriate to combine data from both samples, measurement invariance was tested across the CAP and CSC groups within a multigroup confirmatory factor analysis framework. Specifically, we tested invariance in the following sequence as recommended by Brown (2014): 0) test the model separately in each group 1) test invariance of the overall factor structure simultaneously (i.e., configural invariance); 2) test the invariance of the factor loadings (i.e., metric or weak factorial invariance); 3) test the invariance of item intercepts/thresholds (i.e., scalar or strong factorial invariance); and 4) test the invariance of item residual variances (i.e., residual or strict invariance). For higher-order models, we assessed invariance using the procedure described by Rudnev et al. (2018), which assess invariance of the first-order factor alone, and the invariance of the first and higher-order factors simultaneously at each level of invariance.

As the chi-square difference test is too sensitive to be informative in the context of large sample sizes, invariance was evaluated by comparing changes in CFI and RMSEA (Brown, 2014; Chen, 2007; Kline, 2015). Factor structures with changes in CFI less than 0.01 and RMSEA less than 0.015 (from the previous model in the sequence) were considered to demonstrate invariance across groups. Structures demonstrating adequate invariance progressed to the next step. If there was evidence for non-invariance, which would suggest that factor structures or that the interpretation of the latent variables differed across the groups, then alternative tests of measurement invariance would be considered, such as partial measurement invariance, and we conducted additional post hoc analyses to determine whether there are cohort-specific associations between the personality traits and psychopathology dimensions.

Finally, associations with the personality traits were added to the model(s) found to have adequate structural validity. Regression analyses were conducted to obtain total effect estimates for the association between each trait and general psychopathology. Indirect-effects models were estimated to obtain total, direct, and indirect effect sizes at the specific factor and symptom levels. This approach enabled us to test associations between personality traits and psychopathology across three levels of the structural model, and to disentangle unique associations from those that are accounted for by broader dimensions of psychopathology (Conway et al., 2021).

Availability of data and analysis code

The Mplus output files for these analyses are publicly available and can be accessed online (https://osf.io/cq2rz/). Data may be shared with other researchers upon reasonable request.

Results

Structure of adolescent psychopathology

Goodness-of-fit indices are presented in Table 1 and standardized factor loadings for each of the latent variable models using weighted least square mean and variance adjusted are shown in Table 2. All models, except the one-factor model, were found to have acceptable fit according to traditional fit indices (i.e., CFI and TLI > 0.95). Based on the information criteria (Akaike information criterion, Bayesian information criterion, sample size-adjusted Bayesian information criterion), the bifactor model was the best fitting model, followed by the correlated factors and higher-order models. However, standardized factor loadings in the bifactor model were generally weak for the specific factors and some loadings were negative. In particular, standardized factor loadings on the fear, distress and conduct/inattention specific factors were weak, and the alcohol use/harms related indicators mostly loaded poorly onto the general factor. Further, a Heywood case (i.e., negative residual variance) was detected in the bifactor model on the fear factor (item “restless or fidgety”). Thus, the bifactor model was not considered for further analysis. Standardized factor loadings in the one-factor, four-correlated factors and higher-order models were all positive and reasonably strong (>0.4). In the higher-order model, standardized factor loadings indicate that the general psychopathology factor was more reflective of fear (b = 0.948) and distress (b = 0.876) dimensions followed by conduct/inattention (b = 0.744) and alcohol use/harms (b = 0.388) dimensions.

Table 1. Fit indices for different structural models of adolescent psychopathology (n = 8,589)

Note. χ ² = Chi-square statistic; df = degree of freedom; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; CI = confidence interval; AIC = Akaike’s Information Criterion; BIC = Bayesian Information Criterion; SSABIC = sample size-adjusted BIC; WLSMV = weighted least square mean and variance adjusted. The bifactor could not be estimated using the default integration methods for MLR in Mplus. In order to compare the models, the MLR models were then estimated using the INTEGRATIONS = montecarlo(5000) command in Mplus.

Table 2. Standardized factor loadings on general and specific (fear, distress, alcohol use/harms, conduct/inattention) factors using WLSMV estimator and inter-factor correlations

Note. SD = items from Strengths and Difficulties Questionnaire; AH = Alcohol Harms, items from Rutgers Alcohol Problem Index (RAPI); K6 = Kessler 6 Plus scale (K6+); AUC = Alcohol use, AUDIT-C items; WLSMV = weighted least square mean and variance adjusted. Factor loadings and correlations with a p value ≤0.05 are shown in bold.

Model reliability indices are shown in Table 3. Overall, the general psychopathology factor showed good internal reliability (omega range 0.96–0.97) and construct reliability (H range 0.93–0.97) across the one-factor and higher-order models. The specific factors (fear, distress, alcohol use/harms and conduct/inattention), also showed good internal reliability (omegaS range 0.72–0.98) across the four-correlated factors and higher-order models. Construct reliability (H range 0.73–0.98) in the four-correlated factors model was good. However, in the higher-order model, only the alcohol use/harms specific factor had adequate reliability (i.e., H > 0.7).

Table 3. Reliability indices alternative models of adolescent psychopathology

Note. Results in bold indicate acceptable reliability. Indices for Higher-Order model cannot be calculated, indices presented are based on Schmid–Leiman transformed (SLT) model. ECV = Explained Common Variance, ARPB = Absolute Relative Parameter Bias, ECV_S = Explained Common Variance of specific factors, H = measure of construct replicability, Omega = internal reliability of general factor/s, OmegaS = internal reliability of specific factor/s, OmegaH = Omega Hierarchical, OmegaHS = Omega Hierarchical subscale, PUC = Percent of Uncontaminated Correlations, SLT = Schmid–Leiman transformation.

Omega hierarchical subscale (OmegaHS) indices were low for fear, distress, and conduct/inattention factors, indicating that the majority of variance in these factors may be attributable to the general factor. However, OmegaHS was high for the alcohol use/harms factor in the higher-order model, suggesting that the variance in this specific factor may not be attributable to the general factor.

Overall, the general factor appears to have good reliability across the one-factor and higher-order models and the specific factors appear to have poor reliability, except for the alcohol use/harms factor, across the bifactor and higher-order models. Although the correlated factors model demonstrated better fit and reliability, the lower-order factors of a higher-order model are comparable to the correlated factors model (i.e., the correlated factors model is similar to the lower-order level of the higher-order model). Thus, support for the reliability of the factors in the correlated factors model also suggests there is evidence for the lower-order factors of a higher-order model. Furthermore, the reliability indices for the lower-order factors are based on residualized factors (i.e., the Schmid–Leiman transformation is applied before the indices are calculated). As such, the lower-order factors following the SLT are very similar to the specific factors in a bifactor model whereas the correlated factors model gives a closer approximation to the lower-order factors (prior to the SLT) as standalone constructs. An advantage of the higher-order model is that it allows inclusion of both the narrower constructs and a general psychopathology factor. We therefore selected the higher-order model on the basis that there was evidence for the general factor having good reliability, along with the evidence for the reliability of the correlated/lower-order factors. However, model fit indices suggest that perhaps the higher-order general factor may not be required to account for the associations between factors over and above use of correlations. Therefore, additional external validity was assessed for the correlated factors model (See Supplementary Table S7). A diagram showing the higher-order model and standardized factor loadings is shown in Figure 1.

Figure 1. Higher-order structural model of adolescent psychopathology with standardized parameter estimates. Note. All estimates statistically significant (p ≤ 0.05). Standardized factor loadings for indicators of psychopathology are presented in Table 2.

Following inspection of factor loadings, model fit and reliability indices, additional models were examined including bifactor and higher-order models comprised of general internalizing and general externalizing factors (rather than a single general psychopathology factor). However, these models were found to have inadequate structural validity (see supplementary materials for further details).

The reliability of the higher-order model was further corroborated by measurement invariance tests, as shown in Table 4. The baseline model fit the data well in both the CAP and CSC cohorts. However, a high correlation (0.987) between two items (AH1, “Acted bad” and AH3, “Neglected responsibilities”) was found in the CAP cohort. One of the items was removed (AH1, “Acted bad”) from the model in subsequent analyses and this higher-order model demonstrated invariance across the CAP and CSC cohorts.

Table 4. Results of measurement invariances tests of a higher-order model of psychopathology

Note. * p < 0.001. χ² Δ computed using Mplus DIFFTEST function; CFI Δ = difference in CFI from previous model; RMSEA Δ = difference in RMSEA from previous model. Initial baseline model in the CAP cohort revealed a correlation between AH1 & AH3 of 0.987. The AH1 item was removed from subsequent analyses, and the above table shows the results of measurement invariance tests with AH1 removed.

High-risk personality risk traits and associations with psychopathology dimensions and symptoms

The standardized total and direct effects are available in the supplementary material (Tables S3 to S6). Figure 2 shows the standardized direct effect estimates and 99% confidence intervals for each of the personality traits and general and specific factors of psychopathology. Anxiety sensitivity, negative thinking, impulsivity, and sensation seeking all had significant, positive total effects with general psychopathology. Differential patterns of association emerged with specific factors of psychopathology (Figure 2) and in symptom-level analyses (Figure 3).

Figure 2. Effect sizes and standard errors for standardized direct effect of each personality trait on first-order psychopathology factors (fear, distress, conduct/inattention, and alcohol use/harms) and total effect on general psychopathology. Note. Significant effects (p < 0.001) shown in black, non-significant (p > 0.001) effect shown in gray. Vertical gray solid lines show 0.0 effect size.

Figure 3. Effect sizes and standard errors for standardized direct effects of each personality profile on symptoms of psychopathology. Note. Significant effects (p < 0.001) shown in black, non-significant (p > 0.001) effect shown in gray. Black dotted lines mark boundaries between first-order factors, vertical gray solid lines show 0.0 effect size.

Anxiety Sensitivity: Anxiety sensitivity had a significant positive direct effect with fear and distress, with only 58 and 65% of the variance accounted for by for general psychopathology (Table S3). There was also a significant negative direct effect of anxiety sensitivity with alcohol use/harms, which represents a change in direction from the total effect (i.e., the direct effect reversed in sign compared to the total effect). This may be due to a suppressor effect, and it is likely that the association between alcohol use/harms and anxiety sensitivity was accounted for by general psychopathology (Watson et al., 2013). The direct effect with conduct/inattention was not statistically significant, indicating general psychopathology also accounted for the association between anxiety sensitivity and conduct/inattention symptoms. Overall, these results indicate that the adolescents with greater levels of anxiety sensitivity had significantly higher fear and distress levels (but not alcohol use/harms or conduct/inattention) than adolescents with lower levels of anxiety sensitivity.

Symptom-level indirect-effects models revealed that 24 of 28 associations between symptoms and personality risk traits were accounted by higher-order factors (i.e., general psychopathology, and the specific dimension that the symptom is loaded on). For the remaining direct effects, between 53 and 79% of the variance a large proportion of variance accounted for by the higher-order factors. There were significant direct effects for anxiety sensitivity and “nervous in new situations” and “many fears” with small direct effects (b = 0.139 and 0.185, p < 0.001, respectively), over and above levels of fear and general psychopathology. Similarly, there was a significant and small direct effect for anxiety sensitivity with “worries,” over and above levels of distress and general psychopathology (b = 0.165, p < 0.001). Finally, the direct effect between “easily distracted” and anxiety sensitivity had a small effect size (b = 0.048, p < 0.001), over and above conduct/inattention and general psychopathology. All other symptom-level associations were either non-significant or the association was fully accounted for by the higher-order factors.

Negative thinking: Negative thinking, had a positive direct effect with distress, with only 50% of variance accounted for by general psychopathology (b = 0.268, p < 0.001; Table S4). There were negative direct effects of negative thinking with fear, conduct/inattention, and alcohol use/harms specific factors (b = 0.076, 0.067, 0.180, respectively, ps < .001), representing a reversal of their total effects, indicating that the association was accounted for by general psychopathology. Overall, this indicates that adolescents with greater levels of negative thinking had significantly higher distress levels (but not fear, alcohol use/harms or conduct/inattention) than adolescents with lower levels of negative thinking.

At the symptom level, the associations with 21 of 28 symptoms were accounted for by higher-order factors. Of the remaining symptoms, the effects were small and between 75 and 88% of the variance was accounted for by the higher-order factors. Notably, there were significant direct effects for negative thinking and “unhappy,” “depressed,” and “worthless” (b = 0.095, 0.063 and 0.063, ps < 0.001, respectively), over and above distress and general psychopathology. The remaining symptom-level associations were either non-significant or the association was fully accounted for by the higher-order factors.

Impulsivity: Impulsivity had a small, positive direct effect with conduct/inattention, with only 53% of variance accounted for by general psychopathology (b = 0.270, p < 0.001). There were negative direct effects with alcohol use/harms, distress, and fear, which represented a reversal of their total effects, indicating that the association was accounted for by general psychopathology (b = −0.088, −0.059, −0.150, p < 0.001). Overall, this indicates that adolescents with greater levels of impulsivity had significantly higher conduct/inattention levels (but not fear, distress, or alcohol use/harms) than adolescents with lower levels of impulsivity.

At the symptom level, the associations between impulsivity and 22 of 28 symptoms were accounted by the higher-order factors. The remaining direct effects were small, with a large proportion of variance accounted for by the higher-order factors (72 to 83%). Notably, there were significant, positive direct effects with the symptoms “fight a lot,” “easily distracted” and “lies or cheats,” over and above the conduct/inattention and general psychopathology factors (b = 0.101, 0.078, 0.105, p < 0.001, respectively). Direct effects with alcohol use/harms were either fully accounted for by general psychopathology or were not significant, suggesting that the effects of impulsivity with alcohol use/harms were accounted for by the higher-order factors.

Sensation seeking: Sensation seeking had small, positive direct effects with alcohol use/harms and conduct/inattention factors with only 10 and 13% of variance accounted for by general psychopathology, respectively (b = 0.182, 0.141, ps < 0.001, respectively, see Table S6). There were also small negative direct effects with distress and fear (b = −0.117, −0.170, ps < 0.001, respectively), and the association was mostly accounted for by general psychopathology (100 and 93%, respectively). Overall, this indicates that adolescents with greater levels of sensation seeking had significantly higher alcohol use/harms and conduct/inattention levels, and significantly lower levels of fear than adolescents with lower levels of impulsivity.

At the symptom level, the associations between sensation seeking and 26 of 28 indicators of psychopathology were accounted for by the higher-order factors. There was a small negative direct effect with “many fears” (b = −0.170, p < 0.001), and a small positive direct effect with “fight a lot” (b = 0.066, p < 0.001).

Discussion

The current study extends prior work on the underlying structure of psychopathology by using a symptom-level approach and more rigorous methods of assessing the structural validity, reliability, and replicability of different statistical models. Our results align with previous research on the structure of psychopathology in adolescents and extend this work by illuminating important patterns of association with four high-risk personality traits that have implications for the development of targeted prevention and early intervention programs and our understanding of the underlying structure of psychopathology.

High-risk personality traits and psychopathology

Overall, the results indicate that personality measures could be used to identify adolescents at risk of developing general psychopathology, as well as certain specific forms of psychopathology. Findings showed that all four personality traits were associated with general and specific dimensions of psychopathology in theoretically expected ways and consistent with previous research. Consistent with the four-factor model of vulnerability, our results broadly indicated that inhibited traits (i.e., negative thinking and anxiety sensitivity) were more closely related to internalizing forms of psychopathology (i.e., fear and distress), and disinhibited traits (i.e., impulsivity and sensation seeking) were associated with externalizing forms of psychopathology (i.e., alcohol use/harms and conduct/inattention (Castellanos-Ryan & Conrod, 2012). These findings also align with prior research with young people indicating that neuroticism is positively associated with fear, distress and broad externalizing dimensions (Watts et al., 2019), and that different facets of neuroticism are differentially related to internalizing and externalizing dimensions (Brandes et al., 2019).

Given that associations between each of the four personality traits and substance use are well-established, we anticipated that there would be positive associations with the alcohol use/harm factor. However, we found that only sensation seeking was positively associated with alcohol use/harms above and beyond general psychopathology in our sample. Because most research has typically focused on a broad externalizing factor and sensation seeking (Carragher et al., 2016; Castellanos-Ryan et al., 2016), our results are consistent with only one other study which reported on a bifactor model comprised of a general externalizing factor and a substance use and conduct specific factor. Castellanos-Ryan and Conrod (2011) found that sensation seeking was uniquely linked with substance use and that impulsivity was related to the specific conduct factor (along with general externalizing). This suggests that impulsivity may be related to broader externalizing (i.e., the overlap between substance use and conduct/antisocial problems) and may also have unique links to conduct/behavioral problems, whereas sensation seeking may be more specifically related to substance misuse.

Consistent with previous research, we found that anxiety sensitivity was related to the internalizing dimensions of fear and distress (Carragher et al., 2016; Castellanos-Ryan et al., 2016). However, we also found that anxiety sensitivity was related to lower alcohol use/harms after accounting for general psychopathology. Although this finding was unexpected, literature on the association between anxiety and alcohol use provides important context for interpreting our results. A recent systematic review revealed inconsistent findings for the association between alcohol use and anxiety (Dyer et al., 2019), and general population research has shown that anxiety may not increase alcohol use until after age 14 (Birrell et al., 2015). This is consistent with longitudinal research that has found dynamic associations among anxiety symptoms and alcohol use in early adolescence (Pardee et al., 2014). Specifically, young adolescents with higher initial levels of anxiety demonstrated more rapid increases in alcohol use, compared to peers with low or declining anxiety symptoms. In contrast, there was evidence to suggest that social anxiety specifically had protective effects in early adolescence before later increasing risk for substance misuse.

Within the present study, our finding that greater anxiety sensitivity was related to lower alcohol use/harms may mean that anxiety sensitivity does not have a meaningful unique association with alcohol use/harms, and that the association is better explained by general psychopathology. Alternatively, anxiety sensitivity may protect against alcohol-related harms in early adolescence or delay the onset/escalation of alcohol use until later in adolescence. For example, as the alcohol use/harms factor in the present study is more heavily defined by alcohol-related harms, it is possible that anxiety sensitivity is protective against experiencing alcohol-related harms within this age group. Indeed, at the symptom level we found that anxiety sensitivity, impulsivity and (to a lesser extent) sensation seeking generally had negative direct effects with alcohol use items (i.e., frequency, binge, and quantity), but positive direct effects with most of the alcohol harm items. Furthermore, given that in the present sample the prevalence of alcohol use and related harms was relatively low (which is expected given the mean age was 13 years), it is possible that anxiety sensitivity may delay the onset of alcohol use or slow the escalation of alcohol consumption until later in adolescence. For example, it is possible that once individuals with high anxiety sensitivity have experienced the stress dampening effects of alcohol, their association with alcohol may change such that anxiety sensitivity leads to greater alcohol use (Stapinski et al., 2015). Ultimately, longitudinal research is needed to further unpack the association between anxiety sensitivity, alcohol use, and psychopathology more broadly within the hierarchical-dimensional model of psychopathology.

As expected, negative thinking was associated with greater general psychopathology. Negative thinking was also directly related to distress, whereas associations with fear, alcohol use/harms and conduct/inattention were accounted for by general psychopathology. This suggests that negative thinking may be a broader risk factor for psychopathology and that interventions targeting negative thinking may result in reductions in a wide range of psychiatric symptoms. This is consistent with other research linking related traits, such as neuroticism, emotion regulation and dysregulation, to general psychopathology (Brandes et al., 2019; Haltigan et al., 2018; Santens et al., 2020). Indeed, it has even been suggested that general psychopathology reflects emotional/behavioral dysregulation broadly, and maps closely with trait neuroticism. Examination of the intersection between neurotic/inhibited traits and psychopathology over the adolescent period would be a valuable avenue for future research.

In the present study, the general psychopathology factor was more heavily defined by fear and distress dimensions, which complicates the conclusions that could be drawn from the associations between symptom dimensions and the personality traits. However, this is consistent with other studies of general psychopathology among adolescents which have shown that general psychopathology is typically defined by either thought disorder or internalizing dimensions, depending on the symptom domains included in the model (Gomez et al., 2019; Watts et al., 2020). Current knowledge of the onset and temporal sequencing of internalizing and externalizing problems during adolescence suggest that it would be reasonable for a general psychopathology factor to be more reflective of internalizing problems in early adolescence, as seen in the present study, compared to later adolescence (Birrell et al., 2015; Slade et al., 2015; Solmi et al., 2021).

Clinical and classification implications

Our findings have important implications for research on the early detection and prevention of mental and substance use disorders. Adolescents characterized by a fear of anxiety-related sensations (anxiety sensitivity); a sense of hopelessness or low positive affect (negative thinking); difficulties regulating behavioral responses (impulsivity); and/or a desire for novel experiences (sensation seeking) may be at greater risk for developing a wide range of psychiatric problems. Individuals with higher levels of fear or distress may benefit most from receiving interventions targeting anxiety sensitivity and negative thinking; and adolescents with greater levels of alcohol misuse/harms or conduct/inattention problems may benefit from interventions targeting impulsivity and sensation seeking. Indeed, this assumption is corroborated by prior research demonstrating the effectiveness of a personality-targeted prevention program reducing substance use and co-occurring emotional problems by addressing these specific personality traits (Lammers et al., 2017; Newton et al., 2020; O’Leary-Barrett et al., 2013). However, further research is needed to determine whether these effects hold when examining substance use and mental health outcomes with a hierarchical-dimensional framework.

From a classification perspective, our findings support the utility of conceptualizing psychopathology in a hierarchical-dimensional framework and align with prior research on the structure of psychopathology among adolescents. We found evidence for a higher-order model of psychopathology comprised of a general psychopathology dimension, and four specific dimensions: fear, distress, alcohol use/harms and conduct/inattention. While most previous research has selected a bifactor model of psychopathology, when considering model reliability and replicability along with traditional fit indices we found that a higher-order model fit the data best. Although this differs from past research, it is consistent with other more recent studies on hierarchical-dimensional models of psychopathology that have considered additional metrics of model reliability and replicability, underscoring the importance of assessing these indices in future research (Lees et al., 2020; Sunderland et al., 2020). Further, the four specific factors are consistent with prior research indicating that internalizing may be comprised of fear and distress specific sub-dimensions and externalizing may be comprised of substance misuse and conduct/behavioral sub-dimensions factors (Blanco et al., 2015; Levin-Aspenson et al., 2019; Platt et al., 2017; Slade & Watson, 2006). Further studies are needed, particularly longitudinal research, to confirm the validity and reliability of this underlying structure.

Limitations and future directions

There are some limitations that should be considered when interpreting our findings. Importantly, the present study is cross-sectional and cannot determine causality between personality and psychopathology and the generalizability of our findings are limited by the use of a non-representative community sample of Australian adolescents. Self-reported alcohol use/harms and conduct/inattention problems can be affected by self-report biases among, for example, children and young adults with attention-deficit/hyperactivity disorder and young adults following treatment (Hoza et al., 2012; Nirenberg et al., 2013; Sodano et al., 2021). Although the self-reported psychopathology outcomes did not have corroborating information, such as parent or teacher reports, data were collected using structured and validated instruments. Within this context, self-report methods have been shown to be a valid and reliable approach to measuring substance use and mental health outcomes in adolescents (Smith et al., 1995; Smith, 2007; van der Ende et al., 2020). In addition, although our study incorporated a wide variety of mental health symptoms, there are some notable forms of psychopathology that were not included. We were unable include psychosis-related symptoms, for example, as these were only assessed in one the cohorts, and other common youth-onset disorders such as obsessive-compulsive disorder and eating pathology were not assessed. It is also worth noting that six of the seven items in the negative thinking subscale were worded positively (e.g., “I am happy,” “I am very enthusiastic about my future”) and then reverse-scored. As such, this subscale may be more reflective of low positive thinking, rather than a direct measure of negative thinking. This is akin to evidence that negative and positive affect are independent dimensions, rather than opposite poles of a single dimension (Curran et al., 2014; Jovanović & Gavrilov-Jerković, 2016; Watson & Tellegen, 1985). Ultimately, as this study was a secondary analysis of data from two randomized controlled trial cohorts our data were limited to what was available. Additional evidence using more extensive and robust measures would be of value. Furthermore, longitudinal studies with greater coverage of psychiatric disorders may provide more comprehensive insight into the underlying structure of psychopathology and personality-based causal pathways.

Another potential limitation of this study concerns some of the observed differential patterns of association at the symptom level (i.e., symptom-level negative direct effects, but positive total/higher-order effects), which could reflect potential measurement error or model misspecification. For example, the “restless & fidgety” item from the K6 and the observed negative association with anxiety sensitivity and positive association with sensation seeking (positive direct effect) suggests this item could reasonably serve as an indicator of internalizing/distress or externalizing/conduct/inattention. As such, an individual’s interpretation of the question may influence whether “restless & fidgety” is an indicator of distress or conduct/inattention. Similarly, the unique positive association between anxiety sensitivity and “worries” (and negative association with negative thinking), may suggest that “worries” would be a more appropriate indicator for fear rather than distress. It is also possible that the symptom-level effects may reflect nuances in our sample. Thus, further longitudinal research is needed to confirm the reliability of these effects and clarify the placement of these potentially cross-cutting symptoms.

Conclusions

Although there is extensive evidence linking personality with psychopathology, much of the research has failed to take into account the empirical structure of psychopathology. Findings from the present study describe the complex links between four high-risk personality traits and their associations with a hierarchical-dimensional framework of psychopathology in a large sample of early adolescents. The results support the four-factor model of vulnerability as a useful tool for identifying adolescents at risk of experiencing psychopathology and provide useful information for the development and optimization of prevention and early intervention programs. Consistent with prior research, the present study indicates that a tendency toward low positive affectivity (negative thinking), a fear of anxiety-related sensations (anxiety sensitivity); difficulties regulating behavioral responses (impulsivity) and/or a desire for novel experiences (sensation seeking) may be associated with a greater risk for developing mental health problems. Although further longitudinal research is needed to better understand the complex interactions between personality and psychopathology, the present study highlights the importance of symptom-level analyses in delineating personality-related risk for psychopathology and the role personality may play in the development of individual symptoms through to broad dimensions of psychopathology. More broadly, the findings contribute to the ongoing the debate surrounding the structure and classification of adolescent psychopathology.