Introduction
Our ability to predict future suicidal behavior is limited, with single risk factors having minimal predictive power (Franklin et al., Reference Franklin, Ribeiro, Fox, Bentley, Kleiman, Huang and Nock2017). In contrast, a recent report from our study, the Emergency Department Study of Teens at Risk for Suicide (ED-STARS), a prospective, multi-center study evaluating the prediction of suicidal risk in adolescent attendees to pediatric emergency departments (EDs), found that past-week suicidal ideation, lifetime severity of ideation, lifetime history of suicidal behavior, and lower levels of school connectedness together predicted a suicide attempt within 3 months with relatively high accuracy [Area Under the Curve (AUC) = 0.86] (King et al., Reference King, Grupp-Phelan, Brent, Dean, Webb, Spirito and Casper2019).
While these findings are promising, a major limitation to our ability to accurately assess and predict future suicidal behavior is the exclusive reliance on patient self-report (Franklin et al., Reference Franklin, Ribeiro, Fox, Bentley, Kleiman, Huang and Nock2017). Patients may be motivated to deny suicidal ideation or past behavior because of stigma or to avoid hospitalization. One approach that complements self-report is the Death Implicit Association Test (IAT) (Nock et al., Reference Nock, Park, Finn, Deliberto, Dour and Banaji2010; Nock & Banaji, Reference Nock and Banaji2007a). This version of the IAT presents word stimuli, related to ‘death’ (e.g. dead, suicide) and ‘life’ (e.g. alive, living) as well as words related to ‘me’ (e.g. I, my, mine) and ‘not me’ (e.g. they, them). Respondents are asked to classify ‘life’ and ‘me’ related stimuli together in one set of trials, and ‘death’ and ‘me’ related stimuli in another set of trials, with response times for each trial measured in milliseconds. The Self-Harm-IAT (SH-IAT) presents words or images related to self-harm rather than suicide or death, following parallel procedures to the Death-IAT.
Prior research has demonstrated that suicidal people tend to respond more quickly than non-suicidal people when ‘death’ and ‘me’ are paired together, and more slowly when ‘life’ and ‘me’ are paired (Cohen's d from 0.32 to 0.67) (Glenn et al., Reference Glenn, Werntz, Slama, Steinman, Teachman and Nock2017b; Millner, Coppersmith, Teachman, & Nock, Reference Millner, Coppersmith, Teachman and Nock2018). At least eight studies of the IAT have focused on the prediction of self-harm and suicidal risk in adolescents (Cha et al., Reference Cha, Augenstein, Frost, Gallagher, D'Angelo and Nock2016; Dickstein et al., Reference Dickstein, Puzia, Cushman, Weissman, Wegbreit, Kim and Spirito2015; Glenn et al., Reference Glenn, Kleiman, Coppersmith, Santee, Esposito, Cha and Auerbach2017a; Glenn, Kleiman, Cha, Nock, & Prinstein, Reference Glenn, Kleiman, Cha, Nock and Prinstein2016; Glenn, Millner, Esposito, Porter, & Nock, Reference Glenn, Millner, Esposito, Porter and Nock2019; Millner et al., Reference Millner, Augenstein, Visser, Gallagher, Vergara, D'Angelo and Nock2019; Nock & Banaji, Reference Nock and Banaji2007a, Reference Nock and Banajib). Participants in these studies were drawn from a variety of settings including community samples (Nock & Banaji, Reference Nock and Banaji2007a, Reference Nock and Banajib), inpatient units (Millner et al., Reference Millner, Augenstein, Visser, Gallagher, Vergara, D'Angelo and Nock2019), schools (Glenn et al., Reference Glenn, Kleiman, Cha, Nock and Prinstein2016), residential treatment facilities (Glenn et al., Reference Glenn, Kleiman, Coppersmith, Santee, Esposito, Cha and Auerbach2017a), and outpatient clinics (Dickstein et al., Reference Dickstein, Puzia, Cushman, Weissman, Wegbreit, Kim and Spirito2015; Nock & Banaji, Reference Nock and Banaji2007a). Overall, an implicit association with death or suicide has been related to the frequency, severity, and duration of current ideation and predicted future suicidal ideation and/or an attempt in the subsequent 3 to 6 months, even after controlling for clinical risk factors (Barnes et al., Reference Barnes, Bahraini, Forster, Stearns-Yoder, Hostetter, Smith and Nock2017; Ellis, Rufino, & Green, Reference Ellis, Rufino and Green2016; Glenn et al., Reference Glenn, Kleiman, Coppersmith, Santee, Esposito, Cha and Auerbach2017a, Reference Glenn, Werntz, Slama, Steinman, Teachman and Nockb, Reference Glenn, Millner, Esposito, Porter and Nock2019; Nock & Banaji, Reference Nock and Banaji2007a; Randall, Rowe, Dong, Nock, & Colman, Reference Randall, Rowe, Dong, Nock and Colman2013; Tello, Harika-Germaneau, Serra, Jaafari, & Chatard, Reference Tello, Harika-Germaneau, Serra, Jaafari and Chatard2020). The IAT may be more sensitive to behavior that is more recent (Glenn et al., Reference Glenn, Werntz, Slama, Steinman, Teachman and Nock2017b), and may show greater predictive power if administered after a negative mood induction (Cha et al., Reference Cha, O'Connor, Kirtley, Cleare, Wetherall, Eschle and Nock2018).
In three prospective studies in adolescents, different versions of the IAT were associated with either current or future suicidal ideation or attempt, with some qualifications. The SH-IAT discriminated among adolescent inpatients who had a history of a suicide attempt v. non-suicidal adolescent inpatients (Millner et al., Reference Millner, Augenstein, Visser, Gallagher, Vergara, D'Angelo and Nock2019). The Death-IAT predicted ideation at discharge in adolescents in residential treatment whose stay in the facility was at least 2 weeks in duration (Glenn et al., Reference Glenn, Kleiman, Coppersmith, Santee, Esposito, Cha and Auerbach2017a). In adolescents either in outpatient treatment or who had a mental health diagnosis, the Death-IAT predicted ideation and attempts over a 6-month follow-up (Glenn et al., Reference Glenn, Millner, Esposito, Porter and Nock2019). However, the Death-IAT's prediction of ideation was attenuated after controlling for baseline ideation, and the prediction of attempts was strongest in those with a previous history of suicidal behavior (Glenn et al., Reference Glenn, Millner, Esposito, Porter and Nock2019). While different versions of the IAT have been studied in a variety of settings, the Death-IAT (referred to below simply as the IAT) has never been studied in pediatric ED settings, where clinicians frequently are called upon to make rapid decisions about suicidal risk.
In this sample drawn from the ED-STAR study, we examine the ability of the IAT to discriminate among suicidal and non-suicidal patients at baseline, and to predict suicide attempts in the subsequent 3 months follow-up, in the whole sample, as well as in subsamples stratified by gender and the presence or absence of suicidal ideation. We then compare the accuracy of a parsimonious set of self-report variables to the IAT with respect to the prediction of suicide attempts and examine the extent to which the IAT augments the ability of self-report to predict a future suicide attempt.
Methods
ED-STARS
ED-STARS is a multi-site pediatric ED-based study that aims to improve the ability of ED-based clinicians to identify and triage suicidal adolescents. Adolescents were identified through 13 pediatric EDs that were part of the Pediatric Emergency Care Applied Research Network (PECARN) and were recruited between June 2015 and July 2016. Informed assent and consent from adolescents and their parents were obtained in accordance with the University of Michigan Institutional Review Board (the ED-STARS Clinical Coordinating Center), as well as the IRB (Institutional Review Board) of each site. Recruitment time blocks were randomly selected within available staffing hours at each site (see Supplementary Methods, S1).
Adolescents were eligible if they were aged 12–17 years, accompanied by a parent or legal guardian, were English-speaking, and had not been in the study previously. They were excluded if compromised due to medical acuity or cognitive impairment. Of the 10 664 adolescents approached, 6641 consented (62.3%) and 6448 completed a baseline assessment (60.5%). Of those, 2897 were enrolled for follow-up, oversampling for those at higher suicidal risk (King et al., Reference King, Grupp-Phelan, Brent, Dean, Webb, Spirito and Casper2019). Oversampling was done purposely, in order to limit the number of follow-ups, while retaining a sample with enough participants in each risk category, and especially, enough high-risk participants to have a sufficient number of suicide attempts on follow-up.
High risk was defined by suicidal or homicidal ideation with plan or intent, history of a suicide attempt, or non-suicidal self-injury five or more times in the past year. Moderate risk was defined as having suicidal or homicidal ideation without plan or intent, or at least two risk factors for suicide attempt (e.g. depression, aggression) (King et al., Reference King, Grupp-Phelan, Brent, Dean, Webb, Spirito and Casper2019). Of the 2897 adolescents selected for follow-up, 1063 (36.7%) were at moderate risk, and 1372 (47.4%) were at high risk. Of these 2897 adolescents, 2443 also completed an IAT at the time of enrollment, 2310 adolescents had valid IAT data, and 1679 of these 2310 adolescents had both valid IAT data and were followed up at 3 months, for a retention rate of 72.7%.
Assessment (see online Supplementary Table S1)
A 92-item questionnaire for adolescents (with 27 possible follow-up items) was constructed to cover a broad range of potential risk and protective factors for suicidal behavior, including hopelessness, depression, anxiety, agitation, impulsivity, aggression, adaptive functioning, history of maltreatment or assault, social and school connection, family connection and conflict, and sexual and gender minority status (King et al., Reference King, Grupp-Phelan, Brent, Dean, Webb, Spirito and Casper2019).
Suicidal ideation and behavior
Suicidal ideation at baseline within the past week was determined by a positive response to the third question of the 4-item Ask Suicide Screening Questionnaire (ASQ) (Horowitz et al., Reference Horowitz, Bridge, Teach, Ballard, Klima, Rosenstein and Pao2012): ‘In the past week, have you been having thoughts about killing yourself?’ This item (ASQ-3) has three response options: yes, no, and no response, with the latter being associated with a higher risk for suicidal behavior than a response of ‘no’ (Hengehold, Boyd, Liddy-Hicks, Bridge, & Grupp-Phelan, Reference Hengehold, Boyd, Liddy-Hicks, Bridge and Grupp-Phelan2019). Suicidal ideation was also assessed with the self-report version of the Columbia Suicide Severity Rating Scale (C-SSRS), which asks about suicidal ideation over the past month, lifetime ideation, and lifetime attempts, and from item 9 of the Patient Health Questionnaire (PHQ-9), which asks about suicidal ideation or thoughts of self-harm in the past two weeks; these items have been shown to predict suicide attempts (Conway, Erlangsen, Teasdale, Jakobsen, & Larsen, Reference Conway, Erlangsen, Teasdale, Jakobsen and Larsen2017; Gipson, Agarwala, Opperman, Horwitz, & King, Reference Gipson, Agarwala, Opperman, Horwitz and King2015; King et al., Reference King, Grupp-Phelan, Brent, Dean, Webb, Spirito and Casper2019; Posner et al., Reference Posner, Brown, Stanley, Brent, Yershova, Oquendo and Mann2011; Richardson et al., Reference Richardson, McCauley, Grossman, McCarty, Richards, Russo and Katon2010; Rossom et al., Reference Rossom, Coleman, Ahmedani, Beck, Johnson, Oliver and Simon2017). The C-SSRS was also used to determine if the youth experienced suicidality between the baseline ED visit and the 3-month follow-up. The primary outcome was a suicide attempt, with secondary outcomes being any suicidal behavior (including aborted and interrupted attempts) and suicidal ideation with a method, plan, or intent.
Death implicit association test (IAT)
A total of 1769 adolescents were administered an IAT and had a 3-month follow-up. Based on standard IAT conventions (Greenwald, Nosek, & Banaji, Reference Greenwald, Nosek and Banaji2003), 90 (5.1%) participants were removed because they made responses faster than 300 ms on >10% trials and/or because they made >30% of errors throughout the IAT, resulting in 1679 adolescents. For each adolescent, an IAT D-algorithm score was calculated (Greenwald et al., Reference Greenwald, Nosek and Banaji2003; for details, see Supplementary Methods S2).
Statistical analyses
To account for the oversampling of higher-risk groups for follow-up, a weight equal to the inverse of the sampling probability of each of the three risk groups was applied in analyses (King et al., Reference King, Grupp-Phelan, Brent, Dean, Webb, Spirito and Casper2019). The characteristics of those who were randomized to follow-up but who did not have a valid IAT and/or were not followed up were compared to those who both had a valid IAT and were followed up using standard univariate statistics. The accuracy of prediction of the IAT was assessed using the AUC with respect to receiver operating characteristics (ROC), along with 95% confidence intervals (CIs). An ROC curve is a graphical representation of the tradeoff between sensitivity and specificity across all possible cutoffs of a predictive measure or model. The test of significance of an ROC curve is the comparison of the results with pure chance, i.e. an AUC of 0.5 (DeLong, DeLong, & Clarke-Pearson, Reference DeLong, DeLong and Clarke-Pearson1988).
We examined the performance of the IAT stratified by sex and by suicidal ideation, with respect to prediction of suicide attempts and suicidal ideation. We defined suicidal ideation as suicidal ideation in the past week on the ASQ-3. Since a person might be negative on one measure of ideation, and not on another, suicidal ideation was alternatively defined as a positive response to either the ASQ-3, item 9 of the PHQ-9, or on the C-SSRS, meaning suicidal ideation within the past week, 2 weeks, or month, respectively. Ideation, regardless of measure, was treated as a dichotomous variable. Because the predictive ability of the IAT appeared greater in those without v. those with current suicidal ideation, we estimated and tested for an interaction between current ideation and the IAT in the prediction of future attempts using the product of IAT and SI (IAT*ASQ3). For multivariable analyses using logistic regression, the ASQ-3 item (suicidal item in the last week) was employed.
Univariable associations between baseline demographic and clinical risk factors and SAs at 3 months were determined, and predictors with significant associations (p < 0.1) were candidates for inclusion in multivariable logistic regression models (Hosmer, Lemeshow, & Sturdivant, Reference Hosmer, Lemeshow and Sturdivant2013) (see online Supplementary Table S2). In stage one, demographics and variables pertaining to suicidal thoughts, suicidal behaviors, and NSSI were added to the model in a stepwise fashion; the model with the lowest Akaike Information Criterion (AIC) was carried forward. The remaining candidates, including all other clinical risk factors examined were considered using forward stepwise selection. In the final stage, variables were dropped using backward selection (p > 0.05), such that all variables were statistically significant in the final model. For checks on collinearity, correlation matrices of prediction variables were examined, and the variance inflation factors (VIF) were calculated; no VIF was greater than 1.6, indicating no significant collinearity (see online Supplementary Tables S3a and S3b). As secondary analyses, we examined the predictive power of the IAT with respect to a broader definition of suicidal behavior (attempts, aborted attempts, interrupted attempts), and suicidal ideation with a method or plan and/or intent. We also conducted 10-fold cross-validation of the full population models, both univariable and multivariable. For each fold, we calculated the area under the precision-recall curve (AUPRC), the proportion of 3-month attempts for comparison with the AUPRC, and the corresponding AUCs, with results consistent with those presented herein (Saito & Rehmsmeier, Reference Saito and Rehmsmeier2015; for methods and results see Supplementary materials S3 and online Supplementary Table S4a-c, respectively). Analyses were performed using SAS Version 9.4 (SAS Institute, Cary, NC).
Results
Characteristics of the sample (see Table 1)
Participating adolescents were mostly mid-adolescents [mean age 15.1 years, standard deviation (s.d.) = 1.6 years], in high school (65.9%), female (64.1%), with more than half of maternal and paternal figures reporting more than a high school education (70.8% and 51.9%, respectively), and less than half (42.9%) reported receiving public assistance; 55.9% were White, 23.0% were Black, and 22% were Latinx. Of the 1679 adolescents in our sample, 343 (20.4%) responded ‘yes’ to ASQ-3 regarding baseline ideation, 123 (7.3%) marked no response, and 1211 (72.3%) responded ‘no.’ Of those who answered ‘no’ to ideation on the ASQ-3, 14.4% reported ideation on item 9 of the PHQ-9; on the C-SSRS, 40.8% reported lifetime suicidal ideation, 19.2% reported a previous attempt, and 42.4% reported any of the above suicidal indicators. Of 1679 in this sample, 503 (30.0%) reported a previous suicide attempt.
Table 1. Comparison of those selected for randomization but who did not have a valid Implicit Association Test (IAT) and/or were not followed up, to those with both a valid IAT and follow-up
Population: Subjects who completed baseline and were assigned to follow-up and to complete the IAT (Death/Suicide Implicit Association Test).
a Wilcoxon Rank Sum test.
b First and third quartile.
c Chi-squared test.
d Fisher's exact test.
e Columbia Suicide Severity Rating Scale.
Comparison of those included to those who did not have an IAT and/or follow-up
The 1679 youth who had a valid IAT and were followed up at 3 months, when compared to the remaining 1100 participants who were randomized for follow-up but either did not follow-up or did not have a valid IAT. Those who were retained were more likely to be White, non-Latinx, to have parental figures with higher education levels, were less likely to receive public assistance, and had greater lifetime severity of suicidal ideation on the C-SSRS (all p's < 0.005; see Table 1).
Discriminative validity of the IAT
At baseline, the IAT was able to differentiate between current suicidal ideators and non-ideators (AUC = 0.58, 95% CI 0.54–0.61) and between those with and without a history of a suicide attempt with modest accuracy (AUC = 0.55, 95% CI 0.52.–0.58). When the definition of suicidal ideation was expanded to include a positive response on either the ASQ-3, C-SSRS, or PHQ-9, the IAT continued to identify those with suicidal ideation at a rate above chance (AUC = 0.55, 95% CI 0.53–0.58). For those with suicidal ideation, the IAT did not differentiate between those with and without a history of a suicide attempt (p = 0.97). When ‘suicidal’ was defined as a past suicide attempt or suicidal ideation within the past week, 2 weeks, or month (on either the ASQ-3, PHQ-9, or the C-SSRS), then the IAT continued to show modest discrimination between suicidal and non-suicidal groups with an AUC = 0.55, (95% CI 0.53–0.58).
Prediction of suicidal outcomes by the IAT
Upon 3-month follow-up, 85/1679 youth had made a suicide attempt (5.1%). The IAT predicted a future attempt at 3 months with an accuracy of AUC = 0.59 (95% CI 0.52–0.65). The IAT was a stronger predictor of first-time suicide attempts (AUC = 0.69, 95% CI 0.54–0.82). The IAT's ability to predict ideation with a method, plan, or intent, or a broader set of suicide behaviors (attempt, aborted, interrupted attempts) was modest (AUC's 0.54, 95% CI 0.49–0.60 and 0.55, 95% CI 0.50–0.60, respectively).
Logistic regression of predictor variables
Logistic regression was used to identify a parsimonious set of variables from those that showed some association with an attempt by 3 months on univariate analysis: past-week ideation, greater lifetime severity of ideation, lower school connectedness, and the total number of negative life events, which together accurately predicted an attempt at 3 months (AUC = 0.87, 95% CI 0.84–0.90). The addition of the IAT to this regression did not add appreciably to the AUC (AUC = 0.87, 95% CI 0.84–0.90) (see Table 2). Among participants with no previous history of a suicide attempt, the IAT predicted future attempts with an AUC = 0.69 (95% CI 0.54–0.83). Only one covariate, hopelessness, was identified, but after controlling for it, the relationship between the IAT and 3-month suicide attempt was no longer statistically significant (p < 0.12) (see online Supplementary Table S5).
Table 2. Multivariable prediction of suicide attempts, include the IAT in full population
a Death/Suicide Implicit Association Task.
b Ask Suicide Screen Questionnaire, item 3.
c Columbia Suicide Severity Rating Scale.
Performance of the IAT stratified by gender and by the presence of suicidal ideation
Table 3 shows the performance of the IAT in strata defined by gender and ideation, with AUCs ranging from 0.52 to 0.67. Surprisingly, the IAT appeared to perform best in participants who reported no suicidal ideation on the ASQ-3 at baseline (AUC = 0.67, 95% CI 0.55–0.79). In fact, there was a statistically significant interaction between the past week ideation score and the IAT with respect to prediction of suicide attempts, with there being a non-significant relationship between the IAT and suicide attempts when ideation was present (OR = 1.06, 95% CI 0.36–3.09) or when there was no response (OR = 1.03, 95% CI 0.06–19.03), whereas there was a statistically significant relationship between the IAT and future suicide attempts if no ideation was reported in the past week (OR = 9.8, 95% CI 1.8–52.1). The predictive validity of the IAT was re-tested with a broader definition of suicidality (i.e. positive on past month, past 2 weeks, or past week), with similar results to those obtained with past week ideation alone (see Table 3; for D-values for the IAT across stratifications, see online Supplementary Table S6a-e).
Table 3. Performance of the Implicit Association Test (IAT) in predicting suicide attempts in the full sample, and sample stratified by sex and by suicidal ideation
a Death Implicit Association Test.
b Area Under Curve.
c Confidence Interval.
d Ask Suicide Screen Questionnaire, Item 3.
e Columbia Suicide Severity Rating Scale.
f Patient Health Questionnaire-9, Item 9.
Given the stronger performance of the IAT in those who were non-suicidal within the past week, we used logistic regression to identify a set of variables that predicted suicide attempt at 3 months, in order to determine the added value of the IAT in predicting attempts in this subsample. The self-report variables that predicted an attempt were: previous attempt, history of multiple suicide attempts, past suicidal behavior, negative life events, and lower social connectedness, which together were strongly predictive of an attempt within 3 months (AUC = 0.90, 95% CI 0.85–0.95). The addition of the IAT to the regression did not appreciably increase predictive accuracy (AUC = 0.91, 95% CI 0.86–0.96), although its contribution just escaped statistical significance (p = 0.051) (see Table 4).
Table 4. Multivariable prediction of suicide attempt at 3 months including the death/suicide implicit association test for the non-suicidal (Negative on the Ask Suicide-Screening Questionnaire: ASQ-3) subgroup
a Death Implicit Association Test.
b Columbia Suicide Severity Rating Scale.
Discussion
In this prospective study of adolescents screened for suicidal risk in pediatric EDs, we found that the Death IAT was a statistically significant, albeit modest predictor of suicide attempts in the subsequent 3 months. Self-report of ideation and previous suicidal behavior along with specific risk and protective factors for suicidal behavior, such as school or social connectedness, outperformed the IAT with respect to the prediction of the suicide attempt. One unexpected finding was that there was a significant interaction between reported ideation in the previous week and performance on the IAT. The IAT was a stronger predictor of an eventual attempt when the adolescent did not report ideation at baseline, but even in that sub-group, the IAT did not add significantly to the predictive accuracy of selected self-report measures. The IAT was also a stronger predictor of suicide attempts when analyses were restricted to first-time suicide attempts.
Researchers and clinicians have become discouraged with our current limited ability to predict suicidal behavior (Franklin et al., Reference Franklin, Ribeiro, Fox, Bentley, Kleiman, Huang and Nock2017). However, as previously reported (King et al., Reference King, Grupp-Phelan, Brent, Dean, Webb, Spirito and Casper2019), a handful of self-report items can accurately predict suicide attempts in adolescent suicide attendees to a pediatric ED within the next 3 months. In part, our approach follows recommendations distilled from recent meta-analyses, which includes looking at more than one risk factor at a time, focusing on suicide-specific (e.g. suicidal ideation, lifetime severity of ideation), rather than diagnostic risk factors (e.g. depression), and examining risk and protective factors that are consistent with empirically supported theories of suicide (e.g. school connection, total negative life events) (Glenn et al., Reference Glenn, Kleiman, Cha, Deming, Franklin and Nock2018).
While the IAT had modest success in predicting suicide attempts in this sample, items from the self-report battery, such as past suicidal behavior, or connectedness, in aggregate, had much higher predictive accuracy. This contrasts with some previous studies in adults showing that the IAT was a strong predictor of a suicide attempt, even after controlling for some of the most common risk factors for suicidal behavior (Barnes et al., Reference Barnes, Bahraini, Forster, Stearns-Yoder, Hostetter, Smith and Nock2017; Nock et al., Reference Nock, Park, Finn, Deliberto, Dour and Banaji2010; Tello et al., Reference Tello, Harika-Germaneau, Serra, Jaafari and Chatard2020). The findings in ED-STARS may differ compared to previous studies because of the greater breadth and depth of the ED-STARS assessment of risk and protective factors, which, in this study, led to a more accurate prediction of suicide attempt relative to the IAT.
The modest performance of the IAT in the prediction of suicide attempts in this study could be because the IAT may perform better in adults than in youth (Barnes et al., Reference Barnes, Bahraini, Forster, Stearns-Yoder, Hostetter, Smith and Nock2017; Nock et al., Reference Nock, Park, Finn, Deliberto, Dour and Banaji2010). However, a more recent study of German adult psychiatric inpatients found that the Death-IAT was not predictive of future attempts, nor was there a significant suicidal ideation by IAT interaction with respect to the prediction of suicide attempts (Rath et al., Reference Rath, Teismann, Schmitz, Glaesmer, Hallensleben, Paashaus and Forkmann2021). Also, since the prediction of future attempts in one study was enhanced when it was preceded by a negative mood induction, it is possible that variation in the current mood at the time of administration of the IAT could account for some of the inconsistencies in the literature (Cha et al., Reference Cha, O'Connor, Kirtley, Cleare, Wetherall, Eschle and Nock2018). One study reported strong performance of the IAT in predicting suicide attempts among adolescents with a history of previous attempts (Glenn et al., Reference Glenn, Millner, Esposito, Porter and Nock2019). In contrast, we found that the IAT predicted future attempts more accurately in those without a history of previous suicide attempts. Nevertheless, the results of this paper raise questions about the clinical utility of the IAT for the assessment and prediction of suicidal behavior in adolescents.
This study's strengths include a large, diverse sample that is likely representative of patients who present to pediatric EDs, a prospective evaluation of predictors of future suicide attempts using a broad array of risk and protective factors, and a standard assessment of suicide attempts. This is one of the largest prospective studies of the IAT in adolescents, and the only one, to our knowledge, that compares its performance to a broad range of self-report risk and protective factors. Our findings were convergent with those obtained using 10-fold cross-validation. While a future suicide attempt was our primary outcome, we also report on the relationship of the IAT to a broader range of suicidal outcomes, namely, suicidal ideation with method, plan or intent, or an aborted or interrupted suicide attempt, with results convergent with our primary analyses.
Consistent with our previous communication, we found that a handful of self-report items were highly accurate in predicting future suicide attempts in adolescent attendees to a pediatric ED (King et al., Reference King, Grupp-Phelan, Brent, Dean, Webb, Spirito and Casper2019). These self-report questions were more accurate in predicting future suicide attempts than was the IAT, which did not add significantly to the predictive accuracy of these items. Since the IAT had the strongest predictive power when patients did not report suicidal ideation or had not made a previous suicide attempt, the IAT may have a role in screening patients for whom there may be some behavioral risk factors for suicidality, but who do not report current suicidal ideation and/or past suicidal behavior. However, these results, consistent with some other recent studies (Rath et al., Reference Rath, Teismann, Schmitz, Glaesmer, Hallensleben, Paashaus and Forkmann2021), suggest that the role of the IAT for clinical assessment and prediction of suicidal behavior may be limited.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0033291721001215.
Acknowledgements
The authors thank the following individuals from the PECARN: Joel Fein from the Violence Prevention Initiative and Division of Emergency Medicine at the University of Pennsylvania, Marlene Melzer-Lange from the Department of Emergency Medicine at the Medical College of Wisconsin, Daniel Cohen from the Department of Emergency Medicine at Nationwide Children's Hospital, Robert Hickey from the Department of Pediatrics at the University of Pittsburgh, Margaret Rea from the Department of Psychiatry and Human Behavior at Brown University, Allison Keller from the Department of Pediatric Emergency Medicine at the University of Utah, Rakesh Mistry from the Departments of Pediatrics and Emergency Medicine at University of Colorado School of Medicine, and Dale Woolridge from the Department of Emergency Medicine and Pediatrics at the University of Arizona. The authors also thank several NIMH-affiliated individuals for their research collaboration and study facilitation: Lisa Colpe, member of ED-STARS Steering Committee; Galia Siegel, Clinical Trials Program Coordinator; Joel Sherrill, Division of Services and Intervention Research; and members of the NIMH DSMB. The authors appreciate the study and data management assistance of Marie Kay, Michelle Robinson, and Casey Evans from the PECARN Data Coordinating Center at the University of Utah; Esther Ullman, Lisa Carn, and Maureen O'Brien from Survey Research Operations at the University of Michigan; and Rebecca Lindsay and Taylor McGuire from the Department of Psychiatry at the University of Michigan. The authors also thank the PECARN research coordinators at study sites, the telephone follow-up interview team at the University of Michigan, and study research assistants. We appreciate the excellent editorial and administrative support for preparing this manuscript from Afton Kirk-Johnson.
Author contributions
Concept and design: Brent, Grupp-Phelan, Nock, King
Acquisition, analysis, or interpretation of data: Brent, Grupp-Phelan, O'Shea, Mahabee-Gittens, Rogers, Duffy, Shenoi, Chernick, Casper, Nock, King
Drafting of the manuscript: Brent, O'Shea, Shenoi, Chernick
Editing of the manuscript: Grupp-Phelan
Critical revision of the manuscript for important intellectual content: Brent, Grupp-Phelan, O'Shea, Patel, Mahabee-Gittens, Rogers, Duffy, Shenoi, Casper, Nock, King
Statistical analysis: Casper, Webb, Nock
Obtained funding: Brent, Grupp-Phelan, O'Shea, Nock, King
Administrative, technical, or material support: Brent, O'Shea, Patel, Mahabee-Gittens, King
Supervision: Shenoi, King
Declaration of interest
D.A.B. has received research support from NIMH, AFSP, the Once Upon a Time Foundation, and the Beckwith Foundation, received royalties from Guilford Press, from the electronic self-rated version of the C-SSRS from eRT, Inc., and from performing duties as an UptoDate Psychiatry Section Editor, receives consulting fees from Healthwise, and receives Honoraria from the Klingenstein Third Generation Foundation for scientific board membership and grant review. M.K.N. has received research support from NIMH, DoD, the US Air Force, and several private foundations; he received royalties for academic textbook publishing from Macmillan and Pearson; and he is an unpaid scientific advisor for TalkLife and Empatica. C.A.K. has received research support from NIMH, the CDC, and AFSP, and received royalties from Guilford Press. J.G.-P. has received research support from NIMH and HRSA. B.A.O'S., S.J.P., E.M.M.-G., A.R., S.J.D., R.P.S., L.S.C., T.C.C., and M.W.W. have no potential conflicts of interest to report.
Financial support
This study was supported by a grant from the National Institute of Mental Health (NIMH), ‘Emergency Department Screen for Teens at Risk for Suicide’ (ED-STARS, U01 MH104311). It was also supported in part by the Health Resources and Services Administration (HRSA), Maternal and Child Health Bureau (MCHB), and Emergency Medical Services for Children (EMSC) Network Development Demonstration Program under cooperative agreements U03MC00008, U03MC00001, U03MC00003, U03MC00006, U03MC00007, U03MC22684, U03MC28845, H3MC26201, and U03MC22685. Dr Mahabee-Gittens receives research support from the National Institutes of Health: NIH Grant Number R01ES027815 and R01HD083354. Dr O'Shea receives support from the EU Horizon 2020 Marie Curie Global Fellowship (794913).
Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis and interpretation of the data; preparation, review, or approval of the manuscript; and the decision to submit the manuscript for publication.
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
