INTRODUCTION
The Wechsler Intelligence Scale for Children–Fourth Edition (WISC–IV; Wechsler, Reference Wechsler2003) is a widely used test of children's psychometric intelligence. As part of the revision of its predecessor, the Wechsler Intelligence Scale for Children–Third Edition (WISC–III; Wechsler, Reference Wechsler1991), several changes were incorporated. The most substantial of these included the removal of the traditional Verbal and Performance IQ scores, as well as the elimination of various subtests and simultaneous addition of several new ones. Perhaps most relevant for the context of neuropsychological assessment, the Perceptual Organization index was restructured and renamed “Perceptual Reasoning,” with a noticeably reduced emphasis on speeded performance, while the Freedom from Distractibility index was modified and renamed “Working Memory,” with less of an emphasis on mathematical skills (for further details, see Yeates and Donders, Reference Yeates, Donders, Prifitera, Saklofske and Weiss2005).
When a substantially revised test is used in clinical practice, it is important that research be conducted to ensure that the instrument is meaningfully related to relevant characteristics of the population with which it is used (American Educational Research Association et al., 1999). The purpose of the current investigation was to determine the criterion validity of the WISC–IV when used with children with traumatic brain injury (TBI). This condition was selected because it is one of the most common acquired neurological conditions in children (Kraus, Reference Kraus, Broman and Michel1995). To date, there have been no published studies of the criterion validity of the WISC–IV in neurological samples.
Previous research with the WISC–III had demonstrated particular sensitivity of its Perceptual Organization and Processing Speed indexes to the severity of TBI (Donders, Reference Donders1997; Tremont et al., Reference Tremont, Mittenberg and Miller1999). However, the WISC–III placed a high premium on rapid and efficient responding in its Perceptual Organization subtests whereas two of the WISC–IV Perceptual Reasoning subtests (Picture Concepts and Matrix Reasoning) have no time limits, and the third one (Block Design) includes far fewer opportunities to earn ‘bonus’ points for shorter completion times than was the case on the WISC–III. In contrast, the Processing Speed index is defined on the WISC–IV by the same subtests (Coding and Symbol Search) as on the WISC–III, although an optional subtest (Cancellation) is also available. Thus, it is possible that the WISC–IV might not yield the same patterns of factor index scores after TBI than did the WISC–III.
Freedom from Distractibility is the other WISC–III factor index that has seen a fair amount of application in neuropsychological research. Prior studies with samples of children with attention-deficit/hyperactivity disorder yielded fairly disappointing results in terms of the discriminatory power of this index (Doyle et al., Reference Doyle, Biederman, Seidman, Weber and Faraone2000; Reinecke et al., Reference Reinecke, Beebe and Stein1999). However, the WISC–IV Working Memory index no longer includes Arithmetic as a core subtest but instead uses Letter-Number Sequencing, which is less confounded by computational ability. Thus, it is possible that WISC–IV Working Memory might be more sensitive to the effects of acquired cerebral dysfunction than WISC–III Freedom from Distractibility was.
We determined a priori that criterion validity of the WISC–IV in the assessment of children with TBI would be demonstrated by the following: (a) statistically significant differences on the factor index scores between a group of children with TBI and a demographically matched control group, and (b) statistically significant correlations between those same factor index scores and length of coma in the clinical group.
METHODS
Research Participants
Following institutional review board approval, two groups of participants were included in this investigation. All data included in this manuscript were obtained in compliance with the regulations of this review board. The 40 clinical patients were selected from a 4-year consecutive series of referrals to a Midwestern rehabilitation hospital, according to the inclusion criteria listed below. A control group (n = 40) was then obtained, with permission from the publisher, from the standardization sample of the WISCV–IV. These control participants were matched to the clinical patients on age, gender, and (when possible) ethnicity, and none of them had a neurological, psychiatric, or special education history. Demographic characteristics of both groups are presented in Table 1.
Table 1. Demographic characteristics and WISC–IV performances of clinical patients and standardization controls
Note
TBI = traumatic brain injury.
a Standardization data derived from the Wechsler Intelligence Scale for Children–Fourth Edition (WISC–IV). Copyright © 2003 by Harcourt Assessment, Inc. Used with permission. All rights reserved.
The following criteria were used to select the clinical participants: (1) diagnosis of TBI, defined as an acute, external force to the head with associated intracranial findings on neuroimaging; (2) age between 6 and 16 years; (3) absence of any prior neurological, psychiatric, or special education history; and (4) evaluation with the WISC–IV within 1 year after injury. During the time period that these data were collected, children with TBI associated with intracranial lesions were routinely referred for neuropsychological evaluation, and the WISC–IV was always administered during this process unless there were circumstances that would have compromised validity (e.g., non-English language background, severe orthopedic injury to the dominant hand). Only first evaluations, not repeat evaluations, were included. All these children had successfully completed either the Test of Memory Malingering (Tombaugh, Reference Tombaugh1996) or the Word Memory Test (Green, Reference Green2003) as a check for sufficient effort and motivation for the psychometric assessment. This investigation was limited to patients who had at least complicated mild or moderate TBI (as evidenced by the presence of intracranial lesions on neuroimaging), because uncomplicated mild TBI is typically not expected to be associated with significant lasting cognitive impairment in children who do not have compromised psychosocial backgrounds (Ponsford et al., Reference Ponsford, Willmott, Rothwell, Cameron, Ayton, Nelms, Curran and Ng1999; Satz et al., Reference Satz, Zaucha, McCleary, Light, Asarnow and Becker1997).
The children with TBI were seen for evaluation with the WISC–IV at a median of 82 days postinjury (range, 24–297 days); always on an outpatient basis, and only when they were medically stable and could recall meaningful information from day to day. The majority (n = 26; 65%) had sustained TBI as the result of a motor vehicle accident, with other causes including falls and recreational activities (n = 12; 30%) and other circumstances (n = 2; 5%). Initial Glasgow Coma Scale scores (when available) spanned the full range of 3–15 but all clinical participants had positive neuroimaging findings, including diffuse (e.g., edema, shear injury; n = 18; 45%) and/or focal (e.g., contusion, hemorrhage; n = 36; 90%) lesions. Median duration of coma, defined as the days until verbal commands were followed, was 1 day (range, 0–25 days). Seventeen children (43%) had coma < 24 hr (including 8 children with < 30 min of loss of consciousness), and 23 (57%) had coma ≥ 1 day.
Procedure
The WISC–IV was administered to clinical children as part of neuropsychological evaluations in the context of their rehabilitation. As part of these evaluations, only the 10 subtests that are needed to compute the four factor index scores (Verbal Comprehension, Perceptual Reasoning, Working Memory, and Processing Speed) and the single Full Scale IQ score were always administered. Standard scores (M = 100; SD = 15) for these five variables, and subtest scaled scores (M = 10; SD = 3), were used in the statistical analyses. Because of relatively small sample size, we maintained an alpha level of .05 for all analyses but also specified a priori a minimum acceptable effect size of 0.05 (η2 for group contrasts; r 2 for correlations).
RESULTS
The average WISC–IV factor index standard scores, as well as the associated Full Scale IQ scores, are presented in Table 1 for the clinical group and the demographically matched control group. The associated subtest patterns are presented in Figure 1 for illustrative purposes. A multivariate analysis of variance with groups (n = 2) as the independent variable and WISC-IV factor index scores (n = 4) as the dependent variables yielded a statistically significant main effect of groups [F(4,75) = 6.69; p < .0001]. However, post hoc analyses revealed that this was accounted for exclusively by the group contrast on Processing Speed [F(1,78) = 14.83; p < .0002; η2 = 0.16], whereas none of the group differences on any of the other factor index scores met even liberal standards for statistical significance [p > .10 and η2 < 0.05 for all three variables]. The performance of the TBI group on Processing Speed was approximately 2/3 SD lower than that of the control group. A trend for a moderation of this difference by the presence or absence of prolonged (≥ 1 day) coma, with a more pronounced group contrast for children with prolonged coma, fell short of the a priori specified statistical criteria [p > 0.09; η2 < 0.07].
Fig. 1. WISC–IV subtest profiles for traumatic brain injury group (TBI; N = 40) and control group (N = 40). BD, Block Design; SI, Similarities; DS, Digit Span; PC, Picture Concepts; CD, Coding; VO, Vocabulary; LN, Letter-Number Sequencing; MR, Matrix Reasoning; CO, Comprehension; SS, Symbol Search. Control group standardization data derived from the Wechsler Intelligence Scale for Children–Fourth Edition (WISC–IV). Copyright © 2003 by Harcourt Assessment, Inc. Used with permission. All rights reserved.
Approximately a quarter of the sample (n = 11; 28%) scored at or below the 10th percentile (standard score ≤ 80) on Processing Speed. To explore the degree to which performance on this variable could potentially be used to classify individual children as having versus not having TBI, we used logistic regression analysis. Processing Speed classified 67.50% (54/80) of the participants correctly, with a sensitivity of 72.50% (29/40) and a specificity of 62.50% (25/40). This yielded a likelihood ratio (sensitivity/[1 − specificity]) of 1.93. The associated false positive (34.10%) and false negative (30.56) rates were fairly high.
Spearman correlations between the various WISC–IV summary standard scores and length of coma in the clinical TBI group achieved statistical significance only for Processing Speed, with longer length of coma being associated with poorer performance on this variable; accounting for approximately 19% of the variance [r = −0.44; p < .01]. Correlations of the other factor index and IQ scores with length of coma ranged from −0.05 to −0.23; all being statistically nonsignificant [p > .10 for all variables]. None of the 5 WISC–IV summary standard scores showed a statistical degree of covariance with time since injury [p > .10 for all variables].
Because Processing Speed was the only index that met our a priori specified criteria for criterion validity, we investigated the findings from its component subtests in the clinical sample in greater detail. In addition to the two core subtest Coding and Symbol Search, data from the supplementary subtest Cancellation were also available for the vast majority (n = 38; 95%) of the clinical sample. Only Coding [r = −0.33; p < .04] and Symbol Search [r = −0.37; p < .02] but not Cancellation [r = −0.20; p > .10] met the standards for statistical significance and effect size. This finding was, however, not specific to Cancellation. None of the other 8 WISC–IV core subtests had a statistically significant correlation with length of coma [p > .10 for all variables].
DISCUSSION
The goal of this investigation was to determine the criterion validity of the WISC–IV in children with TBI. The results indicated that, of the four factor index scores, only Processing Speed met the a priori specified minimum criteria in terms of yielding a statistically significant mean difference with the demographically matched control group, as well as a statistically significant correlation with length of coma. However, interpretation of Processing Speed must be done relative to the other factor indexes, either by considering the base rates of index contrasts in the test manual (Wechsler, Reference Wechsler2003) or by using ipsative comparisons (Naglieri & Paolitto, Reference Naglieri and Paolitto2005).
The sensitivity of Processing Speed to injury severity after pediatric TBI is consistent with previous research with the WISC–III (Donders, Reference Donders1997; Tremont et al., Reference Tremont, Mittenberg and Miller1999). However, the associated specificity was fairly modest, resulting in a likelihood ratio that fell just short of the value of 2 or greater that has been recommended as a standard for clinical decision making (Grimes & Schulz, Reference Grimes and Schulz2005). It should also be realized that relative decrements on Processing Speed are not unique to TBI and have also been reported in some other clinical samples, ranging from depression to learning disability (Calhoun & Mayes, Reference Calhoun and Mayes2005). Thus, although this WISC–IV index is sensitive to TBI, it cannot be relied upon in isolation to rule in or rule out acquired cerebral/cognitive impairment. Furthermore, given the relatively modest negative correlation between the supplementary subtest Cancellation and length of coma, caution is necessary when substituting this subtest for either Coding or Symbol Search.
The lack of criterion validity of WISC–IV Perceptual Reasoning stands in contrast with previous findings with WISC–III Perceptual Organization after TBI (Donders, Reference Donders1997; Tremont et al., Reference Tremont, Mittenberg and Miller1999). The most likely explanation is the decrease in emphasis on speeded performance on WISC–IV Perceptual Reasoning subtests. Thus, practitioners should not expect to find factor index or subtest profiles on the WISC–IV after TBI that are similar to those previously encountered on the WISC–III.
The finding that WISC–IV Working Memory, and in particular the Letter-Number Sequencing subtest, did not appear sensitive to the effects of TBI is disappointing and contrasts with the findings of this subtest on the Wechsler Adult Intelligence Scale–Third Edition (WAIS–III; Wechsler, Reference Wechsler1997) in adults with TBI (Donders et al., Reference Donders, Tulsky and Zhu2001). The reason for this difference may be that this subtest simply does not measure the same latent construct in children with TBI as it does in adult patients with TBI. However, that possibility will need to be addressed in future research. Such research should also address the possibility that WISC–IV Perceptual Reasoning subtests may need to be interpreted along two distinct dimensions, including visual processing and fluid reasoning, as has been suggested by Keith et al. (Reference Keith, Goldenring-Fine, Taub, Reynolds and Kranzler2006).
Potential limitations of this investigation include the use of a referred convenience sample that was limited to children who had intracranial lesions on neuroimaging. Therefore, these findings cannot be generalized to children with uncomplicated mild TBI. We were also not able to perform morphometric lesion analyses in the subacute or long-term phase after injury, which is an opportunity for future research that may also consider novel approaches such as diffusion tensor imaging and associated tractography, or functional neuroimaging (Bigler, Reference Bigler, Hunter and Donders2007).
With these reservations in mind, we conclude that Processing Speed, when computed on the basis of its core subtests, has acceptable criterion validity in the evaluation of children with complicated mild to severe TBI. The degree to which Processing Speed provides incremental value to the diagnostic process will need to be evaluated in future research in a larger sample, with inclusion of additional tests of abilities that are not represented on the WISC–IV, such as delayed recall.
ACKNOWLEDGMENTS
This research was based in part on standardization data derived from the Wechsler Intelligence Scale for Children–Fourth Edition (WISC–IV). Copyright © 2003 by Harcourt Assessment, Inc. Used with permission. All rights reserved. The authors thank the publisher of the WISC–IV for allowing access to these standardization data. The authors had no financial interest in this context and they do not have any other potential conflicts of interest to disclose. There was no external financial support for this research.
