Social-emotional development includes the ability to express and manage positive and negative emotions, develop interpersonal relationships, as well as explore the environment to learn about one's surroundings (Pontoppidan, Niss, Pejtersen, Julian, & Vaever, Reference Pontoppidan, Niss, Pejtersen, Julian and Væver2017). Previous research has suggested that atypicalities in social-emotional development appear in community samples of children by 2 years of age and predict later mental health (Alink et al., Reference Alink, Mesman, Van Zeijl, Stolk, Juffer, Koot and Van IJzendoorn2006; Briggs-Gowan & Carter, Reference Briggs-Gowan and Carter2008; Briggs-Gowan, Carter, Bosson-Heenan, Guyer, & Horwitz, Reference Briggs-Gowan, Carter, Bosson-Heenan, Guyer and Horwitz2006; van Zeijl et al., Reference van Zeijl, Mesman, Stolk, Alink, Van IJzendoorn, Bakermans-Kranenburg and Koot2006; Wakschlag & Danis, Reference Wakschlag, Danis and Zeanah2009). For example, social-emotional atypicalities in 12- to 36-month-olds, as indexed by the Brief Infant-Toddler Social-Emotional Assessment (BITSEA), predicted children who later met diagnostic criteria for mental health disorders at elementary school entry (Briggs-Gowan & Carter, Reference Briggs-Gowan and Carter2008).
Social-emotional development can be difficult to measure in children under the age of two. Infancy and toddlerhood are marked by rapid changes in all areas of development, including social-emotional aspects, but it is difficult to separate typical (e.g., tantrums, separation anxiety) from atypical social-emotional development, as these behaviors may fall along a continuum, from typical to atypical, depending on contextual factors (Shields, Cicchetti, & Ryan, Reference Shields, Cicchetti and Ryan1994; Lasch, Wolff, & Elison, Reference Lasch, Wolff and Elison2019). Nevertheless, there is an increasing awareness of the need for monitoring social-emotional development in young children, as early problems have been associated with decreased social competence in preschool-aged children (Schmidt, Demulder, & Denham, Reference Schmidt, Demulder and Denham2010), poor academic performance in school-aged children (Campbell, Spieker, Burchinal, & Poe, Reference Campbell, Spieker, Burchinal and Poe2006), and later mental health challenges (Campbell, Shaw, & Gilliom, Reference Campbell, Shaw and Gilliom2000; Campbell et al., Reference Campbell, Spieker, Burchinal and Poe2006; Fox, Reference Fox2004; Shaw, Keenan, & Vondra, Reference Shaw, Keenan and Vondra1994). Early interventions for preschool-aged children for social-emotional problems show efficacy in improving later socialization and academic performance (Gross et al., Reference Gross, Fogg, Webster-Stratton, Garvey, Julion and Grady2003; Kelleher, Campo, & Gardner, Reference Kelleher, Campo and Gardner2006; Reid, Webster-Stratton, & Baydar, Reference Reid, Webster-Stratton and Baydar2004; Thomas & Zimmer-Gembeck, Reference Thomas and Zimmer-Gembeck2007).
The BITSEA (Briggs-Gowan et al., Reference Briggs-Gowan, Carter, Bosson-Heenan, Guyer and Horwitz2006) is a 42-item parent-completed questionnaire for children between the ages of 11 and 48 months used to rate social and emotional problem behavior and social competence in the general population. Previous studies using the BITSEA have shown its utility in predicting mental health challenges and parent and teacher ratings of psychopathology in children screened before age 4 years (Briggs-Gowan & Carter, Reference Briggs-Gowan and Carter2008; Briggs-Gowan et al., Reference Briggs-Gowan, Carter, McCarthy, Augustyn, Caronna and Clark2013; Giserman Kiss et al., Reference Giserman Kiss and Carter2017). The BITSEA developers have added scales specific to autism spectrum disorder (ASD) in an attempt to support its use as an early detection tool for ASD (Briggs-Gowan et al., Reference Briggs-Gowan, Carter, Bosson-Heenan, Guyer and Horwitz2006; Gardner et al., Reference Gardner, Murphy, Campbell, Tylavsky, Palmer and Graff2013; Giserman Kiss & Carter, Reference Giserman Kiss and Carter2017). They reported acceptable levels of sensitivity (>.70) for distinguishing children with ASD from children without ASD children under 4 years old (Giserman Kiss, Feldman, Sheldrick, & Carter, Reference Giserman Kiss, Feldman, Sheldrick and Carter2017; Kruizinga et al., Reference Kruizinga, Visser, van Batenburg-Eddes, Carter, Jansen and Raat2014). Although the hallmarks of ASD are impairments in social communication and restricted, repetitive behavior and interests (American Psychiatric Association, 2013), social-emotional difficulties have been recalled by parents of children with ASD (De Giacomo & Fombonne, Reference De Giacomo and Fombonne1998) and prospectively noted by parents of infant siblings of children who are later diagnosed with ASD (Sacrey et al., Reference Sacrey, Zwaigenbaum, Bryson, Brian, Smith, Roberts and Armstrong2015) by their first birthday.
Understanding the associations between early social-emotional development and ASD symptom presentation is important, as social-emotional atypicalities often co-occur with ASD. This literature cited above noted accuracy for identifying ASD in children using a social-emotional development questionnaire. This literature, however, only included children who are already diagnosed with ASD and compared them to a cohort of typically developing children. The utility of the BITSEA to identify social-emotional atypicalities in infant siblings of children diagnosed with ASD has yet to be explored. These younger siblings are at increasing risk of also being diagnosed with ASD as the number of older siblings with ASD increases (16% for simplex families to 36% for multiplex families; McDonald et al., Reference McDonald, Senturk, Scheffler, Brian, Carver, Charman and Klin2019). Furthermore, examining social-emotional difficulties in siblings of children with ASD is important because they are also at heightened risk for attention deficit hyperactivity disorder, conduct problems, anxiety disorders, and other neurodevelopmental and mental health conditions (Jokiranta-Olkoniemi et al., Reference Jokiranta-Olkoniemi, Cheslack-Postava, Sucksdorff, Suominen, Gyllenberg, Chudal and Sourander2016). In this study, we assessed whether parent endorsement of social-emotional behavioral problems and competencies, as captured by the BITSEA, differentiated between high-risk (HR) toddlers who later received an ASD diagnosis (HR-ASD), HR toddlers who did not receive an ASD diagnosis (HR-N), and low-risk (LR) controls. We also examined the association between social-emotional development and clinical presentation at 36 months. We predicted that parents of children diagnosed with ASD at age 3 years would report more social-emotional atypicalities at 18 months compared to parents of children who would not be diagnosed with ASD.
Methods
Participants
Infant siblings of children with ASD were recruited from families attending one of five multidisciplinary ASD diagnostic and treatment centers in Canada: the Glenrose Rehabilitation Hospital in Edmonton, the Hospital for Sick Children and Holland Bloorview Kids Rehabilitation Hospital in Toronto, the IWK Health Centre in Halifax, and the McMaster Children's Hospital in Hamilton. This study was approved by the research ethics boards at each institution, and all families gave written informed consent before enrollment into the study. This study was part of a larger multisite prospective longitudinal study exploring early development in infant siblings of children with ASD, including social-emotional development, language and cognitive ability, and symptom presentation in infant siblings who themselves also receive a diagnosis not receive a diagnosis of ASD (see Brian et al., Reference Brian, Bryson, Garon, Roberts, Smith, Szatmari and Zwaigenbaum2008 for more details).
For the HR group, diagnosis of ASD in the older sibling (i.e., proband) was confirmed through a review of diagnostic records or clinical assessment using Diagnostic and statistical manual of mental disorders (fourth edition, text revision) (DSM-IV-TR) criteria. The HR infant siblings and probands had no identifiable neurological conditions, genetic conditions, or severe sensory or motor impairments. LR controls were recruited from local communities on the basis that they had no first- or second-degree relatives with an ASD diagnosis. All infant participants were born at 36–42 weeks’ gestation and had a birth weight greater than 2500 g.
Children from the larger HR cohort were included in this study if (a) they had undergone a 3-year diagnostic assessment, and (b) their parents had completed an Infant-Toddler Social Emotional Assessment (ITSEA; from which the BITSEA items were drawn) at 18 months. Of the 465 HR children and 178 LR children with 3-year follow-up, 154 and 50, respectively, did not have a completed ITSEA and were excluded. The children without ITSEAs had completed their 18-month assessment prior to inclusion of the ITSEA in the study protocol. We adopted Kruizinga et al.'s (Reference Kruizinga, Visser, van Batenburg-Eddes, Carter, Jansen and Raat2014) exclusion criteria recommending that children who were missing more than three subscale items be excluded from analyses (n = 7; 2 HR-ASD, 5 HR-N). Overall, data from 311 HR infant siblings and 128 LR controls were included in the current analyses. Participant characteristics are displayed in Table 1.
Table 1. Participant characteristics
LR = low-risk toddlers (a); HR-N = high-risk toddlers who did not meet criteria for autism spectrum disorder (b); HR-ASD = high-risk toddlers diagnosed with autism spectrum disorder (c); SD = Standard Deviation; ABC = Adaptive Behavior Composite; ADI-R = Autism Diagnostic Interview—Revised; ADOS = Autism Diagnostic Observation Schedule; MSEL = Mullen Scales of Early Learning, Vineland = Vineland Adaptive Behavior Scales.
Measures
BITSEA (Briggs-Gowan, Carter, Irwin, Wachtel, & Cicchetti, Reference Briggs-Gowan, Carter, Irwin, Wachtel and Cicchetti2002) is a 42-item parent questionnaire designed to screen for social-emotional and behavioral problems, in addition to delays in social-emotional competence (Briggs-Gowan et al., Reference Briggs-Gowan, Carter, Bosson-Heenan, Guyer and Horwitz2006). The BITSEA is the brief version of the ITSEA (Briggs-Gowan & Carter, Reference Briggs-Gowan and Carter2000). Our sample of parents completed the ITSEA when their children were 18 months of age, and items that comprise the BITSEA were extracted (as per Kruizinga et al., Reference Kruizinga, Visser, van Batenburg-Eddes, Carter, Jansen and Raat2014). Each item is rated on a three-point Likert scale: 0 = not true/rarely, 1 = somewhat true/sometimes, and 2 = very true/often. As a Level 1 screen (Briggs-Gowan et al., Reference Briggs-Gowan, Carter, Irwin, Wachtel and Cicchetti2002), the BITSEA has two subscales, Social Competency (hereafter, Competency) and Problem Behaviors (hereafter, Problems). Subscales were calculated if fewer than three items were missing (Giserman Kiss & Carter, Reference Giserman Kiss and Carter2017). For the scales with two or fewer missing items, 3-year outcome group mean values for the missing item were placed in the empty cell, resulting in replacement of less than 1% of values for each group (0.09% of cells in the LR group and 0.37% of cells in the HR group).
To evaluate potential as a Level 2 screen (i.e., targeted to children at-risk on the basis of specific symptoms or positive family history; Zwaigenbaum et al., Reference Zwaigenbaum, Bauman, Fein, Pierce, Buie, Davis and Choueiri2015), we used the criteria of Gardner et al. (Reference Gardner, Murphy, Campbell, Tylavsky, Palmer and Graff2013) to calculate the ASD-specific subscales, ASD Competency (items 1, 10, 13, 15, 22, 25, 29, & 31), ASD Problems (items 9, 14, 21, 35, 36, 37, 38, 39, & 40), Total ASD Screening Score (ASD Problems—ASD Competency), and Red Flags (items 2, 14, 18, 24, 32, 34, 35, 36, 37, 38, 39, 40, 41, 42; Gardner et al., Reference Gardner, Murphy, Campbell, Tylavsky, Palmer and Graff2013). The items included in each subscale are displayed in Table 2. As can be seen, the items comprising ASD Social Competence (n = 8) and BITSEA Social Competency (n = 11) are very similar, whereas the items comprising BITSEA Problem Behaviors (n = 31) are more inclusive of a range of potentially problematic behaviors than the ASD Problem Behaviors (n = 9), which are more characteristic of ASD.
Table 2. Items included in each of the BITSEA and ASD subscales
The Autism Diagnostic Observation Schedule
(ADOS; Lord et al., Reference Lord, Rutter, Goode, Heemsbergen, Jordan, Mawhood and Schopler1989) is a semi-structured assessment with excellent interrater reliability; it uses standardized activities and ‘presses’ to elicit communication, social interaction, imaginative use of play materials, and repetitive behavior (Lord et al., Reference Lord, Rutter, Goode, Heemsbergen, Jordan, Mawhood and Schopler1989; Lord et al., Reference Lord, Risi, Lambrecht, Cook, Leventhal, DiLavore and Rutter2000). We used the updated ADOS algorithms of Gotham, Risi, Pickles, and Lord (Reference Gotham, Risi, Pickles and Lord2007), organized into two domains: Social Affect (including communication and social items) and Restricted Repetitive Behaviors. The original ADOS consists of four modules, each of which is appropriate for individuals of differing language levels. The current study used Modules 1 (minimal or no language), 2 (regular use of non-echoed 3-word phrases) and 3 (fluent language). Comparability across modules and language levels was optimized using the 36-month ADOS severity metric (Gotham, Pickles, & Lord, Reference Gotham, Pickles and Lord2009). The ADOS was administered at age 36 months by research-reliable raters.
The Autism Diagnostic Interview-Revised
(ADI-R; Lord, Rutter, & Le Couteur, Reference Lord, Rutter and Le Couteur1994) is an investigator-directed interview that elicits information regarding social development, verbal and non-verbal communication skills, and the presence of repetitive, stereotyped interests and behavior required to make an ICD-10 or DSM-IV-TR diagnosis of ASD. The questions are designed to distinguish qualitative impairments from developmental delays. The ADI-R discriminates well between ASD and other forms of developmental disability, and inter-rater reliability is excellent (Lord et al., Reference Lord, Rutter and Le Couteur1994). The ADI-R was administered to parents of 36-month-olds by research-reliable interviewers.
The Mullen Scales of Early Learning
(MSEL; Mullen, Reference Mullen1995) consists of five scales, four of which (Visual Reception, Receptive Language, Expressive Language, and Fine Motor) assess nonverbal, cognitive, and language ability, while the fifth scale measures gross motor development (from 0 to 29 months only). An Early Learning Composite is calculated based on scores from the first four scales for children aged 0–69 months. Inter-rater and test-retest reliability are excellent (Mullen, Reference Mullen1995). The MSEL was administered at 36 months of age.
The Vineland Adaptive Behavior Scales
(Vineland; Sparrow, Balla, Cicchetti, Harrison, & Doll, Reference Sparrow, Balla, Cicchetti, Harrison and Doll1984) is a semi-structured parent interview designed to assess adaptive behavior across four subdomains outlined by typical developmental milestones that are anchored to specific ages - Communication, Daily Living, Socialization, and Motor skills (with the last domain limited to children younger than 30 months). The scale has excellent reliability and concurrent validity and is sensitive to impairments experienced by children with ASD (Carter et al., Reference Carter, Volkmar, Sparrow, Wang, Lord, Dawson and Schopler1998; Volkmar, Carter, Sparrow, & Cicchetti, Reference Volkmar, Carter, Sparrow and Cicchetti1993). The Vineland or Vineland II was administered at 36 months of age.
Diagnostic Assessment
Each participant underwent an independent gold-standard diagnostic evaluation at age 3 years conducted by a clinician blind to previous study visits. Diagnoses were assigned using DSM-IV-TR criteria, based on the best judgement of the clinician (developmental pediatrician, child psychiatrist, or clinical psychologist, all with at least 10 years’ diagnostic experience), considering information from the ADOS, ADI-R, and concurrent developmental assessments using the MSEL & Vineland.
Statistical Analyses
A series of one-way ANOVAs was conducted to compare the BITSEA subscales (BITSEA Problems, BITSEA Competence, ASD Problems, ASD Competence, Total ASD Score, and Red flags) by 3-year diagnostic outcome group (HR-ASD, HR-N, LR). Post hoc comparisons were explored using Bonferroni corrections. We included comparisons between the LR groups and the two HR groups for two reasons. First, previous reports of social-emotional differences in children with ASD included a group of already diagnosed children and typically developing peers. In this respect, our HR-ASD group serves as the diagnosed group and the LR controls serves as an analogue of typically developing peers. Second, for the HR siblings who were not diagnosed with ASD, we wanted to clarify their place on a continuum of social-emotional behavior. That is, if they differ from HR siblings who were diagnosed with ASD (rated as showing more social competence and less problem behavior), are they also different from LR controls (rated as showing less social competence and more problem behavior) due to the heightened risk for other developmental / mental health conditions in siblings of children with ASD compared to typically developing siblings (Jokiranta-Olkoniemi et al., Reference Jokiranta-Olkoniemi, Cheslack-Postava, Sucksdorff, Suominen, Gyllenberg, Chudal and Sourander2016).
To examine the association between social-emotional development and clinical outcomes in the HR sample, Pearson correlations and receiver operator characteristics (ROC) were calculated. All six BITSEA subscales were correlated with ADOS severity scores, ADI-R Total algorithm scores, MSEL subscales and Early Learning Composite, and Vineland subscales and Adaptive Behavior Composite. To control for multiple comparisons, we used Benjamini and Hochberg (Reference Benjamini and Hochberg1995) corrections. In this method, the p-values are ordered smallest to largest. The alpha level for each test is then set at (k*a)/m, with k corresponding to the p-value's rank (lowest p = 1) and m corresponding to the number of comparisons, which in this case was 13 (each subscale was run separately). This method decreases the chance of false positives; comparisons stop once one of the t-tests is rejected (this method uses ‘q’ rather than ‘p’ to denote the critical alpha level). In addition, area under the curve (AUC) was calculated to assess 18-month BITSEA's sensitivity and specificity with respect to ASD diagnosis. To examine specifically the BITSEA's potential predictive properties within an HR context, ROC analyses were limited to the HR cohort (ASD + versus ASD -). The AUC was calculated to represent an accuracy index, with higher AUC values representing better sensitivity and specificity (AUC of 0.50 = chance relationship; AUC 0.70–0.90 = moderate, ≥ 0.90 high accuracy; Akobeng, Reference Akobeng2007). Youden indexes (defined as the maximum vertical distance between the ROC curve and the chance/diagonal line [Youden's Index (J) = sensitivity + specificity −1]) were calculated to determine optimal cut-scores for each subscale (Akobeng, Reference Akobeng2007). In this method, the highest J value represents the recommended cut score (Akobeng, Reference Akobeng2007). Estimates of screening accuracy were calculated, including: (1) sensitivity, the proportion of children with ASD who were correctly classified by the BITSEA; (2) specificity, the proportion of children not diagnosed with ASD correctly classified by the BITSEA; (3) positive predictive value (PPV), the proportion of children exceeding the BITSEA cut-point who were diagnosed with ASD; and (4) negative predictive value (NPV), the proportion of children who did not exceed the BITSEA cut-point who were not diagnosed with ASD (Fischer, Bachmann, & Jaeschke, Reference Fischer, Bachmann and Jaeschke2003). Statistical analyses were completed using SPSS version 25.
Results
Participant Characteristics
Three groups were identified for comparison based on the 36-month diagnostic assessments: (1) HR infant siblings who received a diagnosis of ASD (‘HR-ASD’; n = 84; 61 boys and 23 girls); (2) HR infant siblings who did not receive a diagnosis of ASD (‘HR-N’; n = 227; 115 boys and 112 girls); and (3) LR controls who did not receive a diagnosis of ASD (‘LR’; n = 128; 67 boys and 61 girls). A significant sex difference (χ2 = 12.62, p = 0.02) showed a higher boy-to-girl ratio in the HR-ASD group than in the LR and HR-N groups, who did not differ (ps < 0.01 and = .42, respectively). No differences were found relating to the child's age at completion of the BITSEA (F(2,438) = 0.96, p > 0.05).
As expected, there were both overall and pair-wise group differences between the HR-ASD, HR-N, and LR controls on the ADOS Social Affect (p’s < .001), Restricted Interests and Repetitive Behavior (ps < .001), and Overall Severity scores (ps < 0.001), ADI-R Total score (ps < 0.002), MSEL standard scores (p’s < 0.001), and Vineland standard scores (ps < 0.001), with the exception of the Daily Living Skills standard score, which did not differentiate between the HR-N and LR groups (p = .14). Overall, HR-ASD toddlers had higher ADOS and ADI-R scores and lower scores on the Vineland and MSEL relative to the HR-N and LR groups. Descriptive data on these 36-month measures are summarized in Table 2.
Completers versus Non-Completers
Independent-sample t-tests were conducted to determine whether clinical characteristics differed between the BITSEA completers and non-completers at 36 months. There was no significant difference between the two groups for ADOS severity scores at 36 months [SA (t(637) = −0.57, p = .57); RRB (t(364.19) = −1.93, p = .054); Total Severity (t(637) = −0.67, p = .50], nor the ADI-R Total score at 36 months (t(601) = −0.90, p = 0.37). Similarly, completers versus non-completers did not differ on MSEL Early Learning Composite (t(616) = −0.90, p = .37) nor the Vineland Adaptive Behavior Composite (t(632) = 0.82, p = .41).
BITSEA Subscales
An overall group difference was observed for each BITSEA subscale, including BITSEA Problems (F(2,436) = 17.62, p < .001), BITSEA Competence (F(2,436) = 43.60, p < .001), and Red Flags (F(2,436) = 25.53, p < .001). In addition, there was a group difference on the ASD-specific subscales, including ASD Problems (F(2,436) = 22.92, p < .001), ASD Competence (F(2,436) = 38.51, p < .001), and ASD Total Score (F(2,436) = 43.77, p < .001). Overall, the HR-ASD group had lower scores on two subscales (the Competencies) and higher scores on the remaining four subscales compared to the HR-N and LR groups (ps < .003), who did not differ, except on BITSEA Competence (p < .001). These results are summarized in Table 3.
Table 3. Descriptive statistics for the Brief Infant-Toddler Social Emotional Assessment (BITSEA) subscales
Note. LR = low-risk toddlers (a); HR-N = high-risk toddlers who did not meet criteria for autism spectrum disorder (b); HR-ASD = high-risk toddlers diagnosed with autism spectrum disorder (c)
Associations between Social-Emotional Development and Clinical Presentation
To examine associations between children's social-emotional problems and competencies at 18 months of age and their clinical characteristics at 36 months of age, correlations were computed between BITSEA subscales and ADOS severity scores (SA, RRB, and overall severity scores), ADI-R algorithm scores, Vineland subscale scores and Adaptive Behavior Composite, and the MSEL subscale scores and Early Learning Composite. Because the ADOS was not designed to detect meaningful clinical variation in typically developing children, we have only included the HR group in the correlational analyses. Pearson's correlations were run for each of the HR groups separately (HR-N and HR-ASD) and multiple comparisons were corrected using Benjamini and Hochberg (Reference Benjamini and Hochberg1995) corrections. The results are presented in Table 4.
Table 4. Relationships between Brief Infant-Toddler Social Emotional Assessment (BITSEA) subscales and clinical characteristics in the high-risk sample
Note: HR-N = high-risk toddlers who did not meet criteria for autism spectrum disorder; HR-ASD = high-risk toddlers diagnosed with autism spectrum disorder; ADI-R = Autism Diagnostic Interview—Revised; ADOS = Autism Diagnostic Observation Schedule; MSEL = Mullen Scales of Early Learning; ELC = Early Learning Composite; Vineland = Vineland Adaptive Behavior Scales; ABC = Adaptive Behavior Composite; p-values corrected for familywise error rate using Benjamini and Hochberg (Reference Benjamini and Hochberg1995) correction (*= q significant) and Range N is the range of lowest to highest number of participants in each comparison.
HR-N Group
There were no significant correlations between any BITSEA subscale and the ADOS severity scores or MSEL subscales or Early Learning Composite (ps >.01).
There were significant correlations between the BITSEA and other parent-report measures. All BITSEA subscales were associated with the ADI-R Total score (ps < .01). For the Vineland, (2) the Communication Scale was related to BITSEA Problems (ps = .002), (3) the Daily Living Scale was related to BITSEA Competence (p = .001), ASD Competence (p = .001), and ASD Total Score (p = .002), (3) the Motor Skills Scale was related to BITSEA Competence (p < .001), ASD Competence (p < .001), and ASD Total Score (p < .001), and (4) the Vineland Adaptive Behavior Composite (ABC) was related to BITSEA Problems (p = .003) and Red Flags (p = .005). The Socialization Scale was not related to any BITSEA subscale (ps > .01).
HR-ASD Group
There were no significant correlations between any BITSEA subscale and the ADOS severity scores (ps > .05). In contrast, there was a significant correlation between the ADI-R Total and the BITSEA Competence, Red Flags, ASD Competence, and ASD Total Score subscales (ps < .002).
For the MSEL, there was a significant correlation between the Fine Motor and the Receptive Language subscales with the BITSEA Problems subscale (p = .007 and p = .003, respectively). There were no significant relationships between the Expressive Language Scale, the Visual Reception Scale, or the Mullen Early Learning Composite and BITSEA subscales (p < .05).
Significant correlations were also found for the Vineland subscales: (1) the Daily Living Skills scale was related to BITSEA Competence (p = .004) and ASD Competence (p = .005), and (2) the Motor Skills scale was related to BITSEA Competence (p = .004) and ASD Competence (p = .007). There were no significant relationships between the Communication Scale, the Socialization Scale, or the Vineland ABC and BITSEA subscales (p < .05).
Receiver Operating Characteristic (ROC) Curve Analyses
Analyses were completed on the HR group (HR-ASD and HR-N) to determine the ability of social-emotional development at 18 months to distinguish between HR children who would and would not be diagnosed with ASD at age 3. The AUC analyses for each subscale were significant (i.e., differed from 0.5) and ranged from 0.64 (BITSEA Problem) to 0.73 (Total ASD Screening score).
Youden Index analysis [(sensitivity + specificity) - 1] was used to identify the optimal BITSEA subscale cut-scores at 18 months of age. The ASD-specific scales yielded relatively poor sensitivity (0.40–0.52) but high specificity (0.84–0.90) values. The non-ASD Competency and Problems scales had similar results; that is, relatively poor sensitivity (0.38–0.46) and high specificity (0.84–0.87). In contrast, the Red Flags scale had higher sensitivity than ASD and non-ASD scales (0.71) but relatively poor specificity (0.60). When comparing the optimal parameters, discrimination between HR children who would and would not be diagnosed with ASD was best achieved by the Total ASD screening score, as determined by the highest AUC value, Youden Index, and sensitivity value among the relevant BITSEA subscales. Table 5 displays the results of ROC analyses for each subscale.
Table 5. ROC characteristics of BITSEA subscales for the HR-ASD and HR-N groups only
Note. AUC = area under the curve; CI = confidence interval (95%); LHR+ = positive likelihood ratio; Sens = sensitivity; Spec = specificity; LHR- = negative likelihood ratio; OR = odds ratio; J = Youden's index; PPV = positive predictive value; NPV = negative predictive value. aShading indicates the optimal cutoff (via Youden's J). No statistics were generated for ASD Problems subscale with cut-off of 0 due to BITSEA questions being on 3-point Likert scale.
True Positives versus False Negatives
We were interested in determining clinical differences between the children with ASD who were correctly (true positive) versus incorrectly (false negative) classified by the Total ASD screening cut-off of 9 on the BITSEA. We used Mann-Whitney U tests to compare scores on the ADOS, ADI-R, MSEL, and Vineland for correctly versus incorrectly classified children. Overall, correctly classified children with ASD had higher scores on the Vineland Adaptive Behavior Composite (M ± SD = 89.39 ± 15.70 versus 76.40 ± 13.85; Z = −2.41, p = .016) and lower scores on the ADI-R Total algorithm score (M ± SD = 20.42 ± 10.69 versus 27.14 ± 9.70; Z = −2.87, p = .004).
There were no statistically significant differences between correctly and incorrectly classified children with ASD on the ADOS SA (Z = −1.3, p = .17), ADOS RRB (Z = −0.023, p = .98), ADOS Total Severity score (Z = −1.11, p = .26), nor the MSEL Early Learning Composite (Z = −0.068, p = .95). Similarly, no differences were observed between the groups on any of the MSEL (ps > .30) or Vineland subscales (ps > .06).
Discussion
The aim of this study was to examine, in infants at high risk, the associations between early social-emotional behavior and competencies and later clinical presentation with respect to ASD in a HR population. Primary caregivers completed the BITSEA at 18 months of age and all children underwent an ASD diagnostic assessment at age three. There were three main results: (1) social-emotional development, as indexed by BITSEA subscales, differentiated children with ASD from children without ASD at 18 months of age, with only one subscale (BITSEA Competence) differentiating between the HR-N and LR groups; (2) social-emotional development was associated with clinical presentation at age three, as indexed by significant associations between BITSEA subscales and ADI-R algorithm scores in both HR groups; and (3) screening cut-off thresholds for social-emotional development, as indexed by the BITSEA, did not meet recommended criteria for screening of ASD (Zwaigenbaum et al., Reference Zwaigenbaum, Bauman, Fein, Pierce, Buie, Davis and Choueiri2015). The results demonstrate that the BITSEA provides clinically meaningful information regarding social and emotional problems in HR siblings but is not an adequate stand-alone screen for ASD based on individual classification metrics.
Parents of toddlers later diagnosed with ASD endorsed more social-emotional problems compared to parents of toddlers without ASD at 18 months of age. Five of the six BITSEA subscales show good discrimination of ASD from non-ASD, with only one subscale, BITSEA Competence, differentiating between the non-diagnosed children (HR-N and LR). These results are similar to previous reports using the BITSEA as an index of social-emotional development in children at risk for ASD and children with confirmed diagnoses of ASD. Gardner et al. (Reference Gardner, Murphy, Campbell, Tylavsky, Palmer and Graff2013) had parents complete the BITSEA when their children were 12 and 24 months of age and compared children based on elevated scores on the Modified Checklist for Autism in Toddlers (M-CHAT). Parents of children in the elevated M-CHAT scores group endorsed significantly more social-emotional development atypicalities (as indexed by the same BITSEA subscales used in this study) at both time points. Higher endorsements of social-emotional atypicalities were also reported for children with ASD compared to typically developing controls (Boone, Brown, & Keim, Reference Boone, Brown and Keim2018; Giserman Kiss et al., Reference Giserman Kiss, Feldman, Sheldrick and Carter2017; Karabekiroglu, Briggs-Gowan, Carter, Rodopman-Arman, & Akbas, Reference Karabekiroglu, Briggs-Gowan, Carter, Rodopman-Arman and Akbas2010; Kruizinga et al., Reference Kruizinga, Visser, van Batenburg-Eddes, Carter, Jansen and Raat2014; Raza et al., Reference Raza, Sacrey, Zwaigenbaum, Bryson, Brian, Smith and Garon2019). Combined, these results suggest that the BITSEA can help parents identify social-emotional difficulties their children may be experiencing, which can inform clinical referrals and potentially, diagnoses.
Exploration of the associations between early social-emotional development and later clinical outcomes indicated a strong relationship between social-emotional problems and competencies and ADI-R algorithm scores in both HR groups. Similarly, Daily Living Skills and Motor subscales on the Vineland were associated with both the BITSEA and ASD Competence scales in both HR groups. These results corroborate the valuable contributions of parents with respect to their ability to identify early differences in children who later are classified as having behavioral problems, developmental delays, or ASD (Robins et al., Reference Robins, Fein, Barton and Green2001; Glascoe, Reference Glascoe2003; Sacrey et al., Reference Sacrey, Zwaigenbaum, Bryson, Brian, Smith, Roberts and Armstrong2015; Sacrey et al., Reference Sacrey, Bryson, Zwaigenbaum, Brian, Smith, Roberts and Garon2018). In contrast, no ADOS score was related to social-emotional development as measures by the BITSEA in either HR group. The lack of association between the BITSEA and ADOS may result from their different purposes (detecting social-emotional development versus ASD symptoms), the narrow focus of items included in the ADOS severity algorithms, and the nature of the assessment context (i.e., a one-time observation over roughly a one-hour period) compared to the broader array of behavior queried on the BITSEA and the larger sampling frame from which parents can pull. This may also have contributed to the small number of associations between the BITSEA and the MSEL. It has been reported that parents and clinicians do not often agree with respect to signs of ASD (Stadnick et al., Reference Stadnick, Chlebowski, Baker-Ericzen, Dyson, Garland and Brookman-Frazee2017), even when the two assessments (parent questionnaire and clinician observation) query the same ASD-related behavior (Sacrey et al., Reference Sacrey, Bryson, Zwaigenbaum, Brian, Smith, Roberts and Garon2018). The BITSEA, ADI-R, and Vineland rely on parent report and thus share method variance, which may account for the stronger relationship compared to the ADOS and MSEL.
The similar pattern of associations between the BITSEA subscales and the clinical assessments for both HR groups suggests that the questionnaire may not be suited to differentiating ASD from non-ASD in an HR sample. We would temper this statement, however, by acknowledging that HR siblings of children with ASD are at increased risk of being diagnosed with other neurodevelopmental conditions compared to siblings of typically developing children, such as anxiety disorders (Shivers, Deisenroth, & Taylor, Reference Shivers, Deisenroth and Taylor2013; Reference Shivers, Jackson and McGregor2019) and ADHD (Ghirardi et al., Reference Ghirardi, Brikell, Kuja-Halkola, Freitag, Franke, Asherson and Larsson2018; Miller et al., Reference Miller, Musser, Young, Olson, Steiner and Nigg2019). As such, the similar pattern of relationships between the BITSEA and the clinical assessments may partially be due to parents recognizing social-emotional differences in their children who may be diagnosed with conditions such as attention deficit hyperactivity disorder, rather than ASD. With this in mind, the BITSEA may be a useful method of collecting and tracking social-emotional behaviors and differences in siblings of children with ASD.
Our ROC analyses produced sensitivity and specificity scores that are below recommended levels for screening (Cicchetti, Volkmar, Klin, & Showalter, Reference Cicchetti, Volkmar, Klin and Showalter1995; Council on Children with Disabilities, 2006; Dumont-Mathieu & Fein, Reference Dumont-Mathieu and Fein2005; Volkmar et al., Reference Volkmar, Cicchetti, Dykens, Sparrow, Lechman and Cohen1988). This is in contrast to previous reports of screening accuracy of the BITSEA, with sensitivity estimates ranging from .70 to .93 and specificity estimates ranging from .68 to .89 (Boone et al., Reference Boone, Brown and Keim2018; Giserman Kiss et al., Reference Giserman Kiss, Feldman, Sheldrick and Carter2017; Karabekiroglu et al., Reference Karabekiroglu, Briggs-Gowan, Carter, Rodopman-Arman and Akbas2010; Kruizinga et al., Reference Kruizinga, Visser, van Batenburg-Eddes, Carter, Jansen and Raat2014). Within the current group of children who were diagnosed later with ASD, we compared the clinical characteristics of those who were identified versus not identified by the Total ASD Score cutoff (subscale with highest AUC, sensitivity, and specificity). Surprisingly, the screen-positive children did not differ from the screen-negative children on the three ADOS severity scores, the Vineland and MSEL subscales, nor on the MSEL Early Learning Composite. The screen-positive children only differed on the Vineland Adaptive Behavior Composite (higher scores than screen-negative children) and ADI-R Total algorithm scores (lower scores than screen-negative children), in line with the correlation findings. Although the BITSEA lacked discriminatory ability to classify ASD, the BITSEA Competence scale differentiated between the HR-ASD and HR-N groups, as well as the HR-N and LR groups, suggesting that this scale may be able to index behaviors related to inherited risk of ASD. A similar relationship was found by Lasch et al. (Reference Lasch, Wolff and Elison2019), who observed a strong association between the ITSEA competence subscale and reciprocal social behavior measured with the Video-Referenced Rating of Reciprocal Social Behavior (vrRSB; Marrus et al., Reference Marrus, Glowinski, Jacob, Klin, Jones, Drain and Sant2015), an index of heritable traits for ASD (Marrus et al., Reference Marrus, Grant, Harris-Olenak, Albright, Bolster and Haber2020).
These results highlight important considerations regarding social-emotional development in HR siblings. Although more social-emotional challenges are reported in HR siblings with ASD compared to non-diagnosed siblings, these differences are not specific to ASD. Furthermore, both the lack of differences on the ADOS and MSEL, as well as the counter-intuitive directions of differences on the ADI-R and Vineland (screen-positive children had better performance on the measures than screen-negative children) provide further evidence for the lack of discriminatory ability of the BITSEA. We would be remiss not to point out that the previous studies compared children diagnosed with ASD to typically developing peers. These screening accuracy estimates are likely inflated due to exclusion of children with ambiguous presentations, such as those of HR non-ASD children who often present with social-emotional and other developmental differences (for a recent review of clinical characteristics of non-diagnosed siblings, see Pisula & Ziegart-Sadowska, Reference Pisula and Ziegart-Sadowska2015). When compared to the findings of Gardner et al. (Reference Gardner, Murphy, Campbell, Tylavsky, Palmer and Graff2013), who employed an arguably more comparable sample (completed BITSEAs at age 1 and 2 years and screening accuracy computed using ASD risk classifications on the M-CHAT), our results are similar. Both samples (ours and those of Gardner et al.) had AUC within the .60 to .70 range and estimates of screening accuracy were below recommended values (Cicchetti et al., Reference Cicchetti, Volkmar, Klin and Showalter1995; Council on Children with Disabilities, 2006; Dumont-Mathieu & Fein, Reference Dumont-Mathieu and Fein2005; Volkmar et al., Reference Volkmar, Cicchetti, Dykens, Sparrow, Lechman and Cohen1988). Again, it is important to acknowledge that the siblings of children with ASD are at a heightened risk for many conditions (Ghirardi et al., Reference Ghirardi, Brikell, Kuja-Halkola, Freitag, Franke, Asherson and Larsson2018; Jokiranta-Olkoniemi et al., Reference Jokiranta-Olkoniemi, Cheslack-Postava, Sucksdorff, Suominen, Gyllenberg, Chudal and Sourander2016; Miller et al., Reference Miller, Musser, Young, Olson, Steiner and Nigg2019; Shivers et al., Reference Shivers, Deisenroth and Taylor2013; Reference Shivers, Jackson and McGregor2019) and the early manifestation of these may contribute to the BITSEA's inability to differentiate HR siblings with ASD from those who do not have ASD (but who may have other conditions not yet diagnosed).
There is evidence to support the use of the BITSEA with a high-risk population in a primary care setting even though our results do not support its use as a stand-alone screener for ASD. Asking caregivers to fill out a questionnaire about social-emotional development may reduce discomfort that can accompany questionnaires that query ASD specifically, especially for families who have not raised any concerns about ASD-related behavior in their younger children (Giserman Kiss & Carter, Reference Giserman Kiss and Carter2017; Zwaigenbaum et al., Reference Zwaigenbaum, Bauman, Fein, Pierce, Buie, Davis and Choueiri2015). Furthermore, querying social-emotional differences in HR siblings provides clinically meaningful information by age one; up to 30% of caregivers have retrospectively recalled social-emotional developmental challenges (De Giacomo & Fombonne, 1998) and parents of HR siblings have prospectively endorsed more such challenges in children who are later diagnosed with ASD (Sacrey et al., 2015) during this period.
This study provides an in-depth analysis of the association between social-emotional problems and competencies and clinical outcomes with respect to ASD in a sample of HR siblings. This study benefits overall from a prospective longitudinal design, large HR toddler cohort with and without ASD, and comprehensive dataset encapsulating many clinical measures over time. This study is not without limitations, however. First, families of HR toddlers in this study have at least one child already diagnosed with ASD. As such, a better awareness of early developmental and behavioral signs associated with ASD may have influenced parent reporting on the BITSEA. Second, ongoing feedback parents receive regarding their child's development may influence parent report on the BITSEA and other measures. Third, additional variability may have been introduced due to BITSEA items having been pulled from the longer ITSEA (that is, responses to items that are not included as part of the BITSEA may have influenced how parents responded to items that were included in the BITSEA). Finally, prospectively studying our sample only to 36 months of age may have limited our ability to appreciate differences in social-emotional development trajectories among HR siblings. Despite these limitations, parents of HR toddlers who were diagnosed with ASD at 36 months endorsed more social-emotional differences relative to parents whose children were not diagnosed with ASD. Because social-emotional challenges are associated in the long-term with poor academic performance (Campbell et al., Reference Campbell, Spieker, Burchinal and Poe2006) and neurodevelopmental and mental health disorders (Campbell et al., Reference Campbell, Shaw and Gilliom2000; Campbell et al., Reference Campbell, Spieker, Burchinal and Poe2006; Fox, Reference Fox2004; Shaw et al., Reference Shaw, Keenan and Vondra1994), collecting information on early social-emotional development is valuable for children at risk of any psychopathology, as early interventions have been shown to improve outcomes in preschool-aged children, with associated long-term consequences (Gross et al., Reference Gross, Fogg, Webster-Stratton, Garvey, Julion and Grady2003; Kelleher et al., Reference Kelleher, Campo and Gardner2006; Reid et al., Reference Reid, Webster-Stratton and Baydar2004; Thomas & Zimmer-Gembeck, Reference Thomas and Zimmer-Gembeck2007).
Our results highlight the potential benefit of collecting information on multiple areas of development for HR siblings. Although the ROC analyses do not support use of the BITSEA as a stand-alone screener for ASD, it can be used as an adjunct measure to capture clinically meaningful information that may not be derived from an in-person clinical assessment. It is important to consider social-emotional development in the overall assessment and potentially intervention planning for children at risk of or diagnosed with ASD.
Funding Source
This study was supported by Women & Children's Health Research Institute (WCHRI), Brain Canada, Kids Brain Health Network (formerly NeuroDevNet), Canadian Institutes of Health Research (CIHR), Azrieli Foundation, Autism Speaks Canada, and Stollery Children's Hospital Foundation Chair in Autism.
Open access
