Childhood depression and anxiety disorders in Serbia: a psychometric study of four screening questionnaires

Dear Editor

Self-report questionnaires are efficient, less expensive, and can initially be deployed for screening purposes to determine whether interviews that are more detailed should be subsequently employed to diagnose depression and anxiety disorders in children and adolescents. Over the past two decades, different questionnaires for depression and anxiety disorders were developed, but none of them is available in Serbia. As a part of an ongoing project aiming at providing the Serbian child health care service with measures for anxiety and depression, this study is the first one set to analyze the psychometric properties of the Screen for Child Anxiety Related Emotional Disorders (SCARED) (Birmaher et al. 1999), Child Anxiety Sensitivity Index (CASI) (Silverman et al. 1991), Short Mood and Feeling Questionnaire (SMFQ) (Angold et al. 1995), and Children's Depression Inventory-Short form (CDI-S) (Kovacs, 2003) in a Serbian community sample. These questionnaires were selected based on their measurement characteristics, above all sound reliability, validity, sensitivity, and availability for children and adolescents aged 8–18 years (Brooks & Kutcher, 2001, 2003).

Methods

Subjects

The study was conducted among schoolchildren in two elementary and two secondary schools in the Belgrade district. School psychologists randomly contacted 300 pupils (aged 8–18 years, equal boys and girls), as well as their parents and informed them about the purpose of the study. Of all those contacted, only 236 (78.67% response rate) returned the written consent from their parents and agreed to participate. The children completed the questionnaires at schools in order to prevent a low responding rate.

Of 236 subjects, 109 (46.2%) were boys and 127 (53.8%) were girls, while 118 (50%) were children aged 8–12 years and 118 (50%) were adolescents aged 13–18 years. The mean age of the entire sample was 12.84 (2.16) years.

Questionnaires

SCARED is a self-report questionnaire assessing symptoms of childhood anxiety disorders (Birmaher et al. 1999). It contains 41 items assigned to panic/somatic disorder (PD, 13 items), generalized anxiety disorder (GAD, 8 items), separation anxiety disorder (SAD, 8 items), social phobia (SPH, 7 items), and school avoidance (SA or school anxiety symptoms, 4 items) scale. All items are scored on a 3-point scale (not true or hardly ever true 0, somewhat true or sometimes true 1, and very true or often true 2) and the sum of all items in a scale is the total score of the corresponding scale, while the sum of all items is the SCARED Total score (possible range 0–82).

CASI is a self-report questionnaire assessing the extent to which children and adolescents believe the experience of anxiety will result in negative consequences (Silverman et al. 1991). It is available as a 13-item and an 18-item form with items scored on a 3-point scale (none 1, some 2, and a lot 3). The CASI Total is calculated as the sum of all items (the CASI-13 score ranges from 13 to 39 and CASI-18 from 18 to 54). A hierarchical model for anxiety sensitivity with four factors labeled Disease Concerns, Unsteady Concerns, Mental Incapacitation Concerns, and Social Concerns was supported (Silverman et al. 2003).

The SMFQ is a self-report questionnaire for children and adolescents for mood assessment during the preceding 2 weeks (Angold et al. 1995; Messer et al. 1995). The SMFQ has 13 items developed from a 33-item questionnaire as a unidimensional questionnaire with items reflecting major depression symptoms. All items are scored on a 3-point scale (true 2, sometimes 1, and not true 0) and the sum of all items is the total score (possible range 0–26).

The CDI-S is a self-report questionnaire assessing the degree of depressive symptoms over the past 2 weeks. It is a 10-item form developed as a unidimensional questionnaire from the original questionnaire with 27 items (Kovacs, 2003). All items are scored on a 3-point scale (not true 0, sometimes 1, and very true 2) and the sum of all items gives the CDI-S Total score (possible range 0–20).

All questionnaires were translated and culturally adapted for the Serbian language following the same steps in order to develop versions equivalent to the original but culturally sensitive, too. Specifically, the questionnaires and permissions for their translation were obtained from the developers and copyright holders. The team for translation consisted of researchers familiar with psychological constructs and translators. Two members independently translated the measures from English to Serbian. From these two translated versions, a single form of every questionnaire was developed (Reconciliation I), which was then translated back into English by two independent English–Serbian translators. A single back translation was developed for every questionnaire (Reconciliation II), which was compared with the original (Harmonization) by the author and two members not previously included. The entire process resulted in the pre-final versions that were pretested in semi-structured interviews with a group of 10 children. Four separate semi-structured interviews were organized to explore comprehensibility, judgment, and response process for every item on the questionnaires separately, as well as clarity and appropriateness of the items (Cognitive Debriefing). The children were asked to ‘think aloud’ about items and how they formed answers, to try to explain the meaning of each item, and to give an opinion whether all items were clear and appropriate. Afterwards, an expert panel met to evaluate the content and face validity of the versions, the results of the pretesting, and the equivalence with the original (conceptual, item, semantic, operational, and measurement equivalence). In the Serbian versions, all items were felt to be comprehensive, precise, and relevant for emotions assessment, so they were unchanged and no items were added, replaced or omitted.

Statistical analysis

Descriptive statistics included the distribution of missing data, mean, standard deviation (s.d.), and floor and ceiling effects. Floor and ceiling effects of 15% or more were considered significant.

Internal consistency reliability was assessed using Cronbach's alpha coefficient, with coefficients of 0.7 or higher was considered to be acceptable.

Confirmative factor analysis (CFA) using Analysis of Moment Structures Version 7 (AMOS-7) was applied to analyze construct (factorial) validity. For the SCARED, it was a tested model including five correlated factors each representing the scales as originally suggested (Birmaher et al. 1999). A model including four first-order factors and one second-order factor was tested for CASI-13 and CASI-18 (Silverman et al. 2003). Finally, a single-factor model was tested for the SMFQ and CDI-S. The adequacy of a model fit was evaluated using the following statistics to assess the degree of fit between estimated and observed variance: χ²/df ratio (<3 good), the comparative fit index (CFI; >0.90 acceptable and >0.95 excellent), the goodness of fit index (GFI; >0.90 acceptable and >0.95 excellent), and root mean square error of approximation (RMSEA; <0.08 acceptable and <0.06 excellent).

Convergent validity was assessed analyzing correlations between the questionnaires. Partial correlations with age and gender as controlling variables were used, considering that in the study children and adolescents, as well as boys and girls, were analyzed together. It was assumed that convergent validity was demonstrated when correlations between questionnaires that measure the same construct were higher than correlations between questionnaires that measure different constructs. Correlation coefficients ranging from 0.1 to 0.3 were considered low, those ranging from 0.31 to 0.5 were considered moderate, and those over 0.5 were considered high (Cohen, 1988).

Results

The overall amount of missing data was 0.27%, while for all items in the questionnaires it ranged from 0.4 to 1.3%. The missing data were replaced using linear interpolation. In Table 1, the mean and s.d. values, and floor and ceiling effects of the questionnaires are given. A significant floor effect was found only for the SCARED–SAD (16.9%), SCARED–SA (45.3%), and CDI-S Total score (25.8%), but no ceiling effect.

Table 1. Descriptive statistics and Cronbach's coefficients of the SCARED, CASI-13, CASI-18, SMFQ, and CDI-S scores

PD, panic/somatic disorder; GAD, generalized anxiety disorder; SAD, separation anxiety disorder; SPH, social phobia; SA, school avoidance.

Internal consistency reliability coefficients below 0.7 were found only for the SCARED–SAD (0.68) and SCARED–SA scale (0.54) (Table 1).

The fit indices indicated acceptable to excellent fits of the data to the unidimensional structure (single-factor model) of the SMFQ and CDI-S and acceptable fits of the data to the four-factor model of the CASI-13 (Table 2). However, while the CFI and GFI fit indices indicated a poor fit of the data to the original five factors of the SCARED and four factors of the CASI-18, the other two indices, χ²/df ratio and RMSEA, indicated acceptable fit of the data.

Table 2. Confirmatory factor analysis of the SCARED, CASI-13, CASI-18, SMFQ, and CDI-S questionnaire (N = 236)

The correlations between the SCARED Total, CASI-13 and CASI-18 Total were high, ranging from 0.52 to 0.97, as well as between the SMFQ and CDI-S Total, 0.6 (Table 3). Low to moderate correlations between the questionnaires that measure different constructs were found. All SCARED scales correlated moderately with the CASI-13 and CASI-18 Total (range: 0.31–0.46). However, the SCARED–PD correlated moderately with the SMFQ Total (0.38), while the SCARED–GAD correlated highly with the SMFQ Total (0.54) and moderately with the CDI-S (0.46).

Table 3. Correlation coefficients of the SCARED, CASI-13, CASI-18, SMFQ, CDI-S Total scores (N = 236)

*Correlations between the SCARED Total and SCARED scales were not presented.

PD, panic/somatic disorder; GAD, generalized anxiety disorder; SAD, separation anxiety disorder; SPH, social phobia; SA, school avoidance.

Discussion

This study evaluated the general psychometric characteristics of four most frequently used questionnaires for depression and anxiety disorders screening in a sample of Serbian youth.

SCARED

The scales that measure SADs and school anxiety symptoms possess low internal consistency coefficients, 0.68 and 0.54, respectively, whereas the other scales and total possess high internal consistency coefficients. These findings agree with a meta-analysis data showing that the SCARED–SA internal consistency coefficients (α) in different studies ranged from 0.53 to 0.70 and from 0.68 to 0.75 for the SCARED–SAD, whereas internal consistency coefficients for the other scales and total were above 0.7 (Hale et al. 2011). However, the previous data from a meta-analysis demonstrated that four studies using CFA to analyze the factor structure supported the five-factor model, whereas only one supported a three-factor model with GAD, PD, and SP (Boyd et al. 2003; Hale et al. 2011). Recently, the five-factor model was also confirmed for the Brazilian translation (Isolan et al. 2011). The original SCARED with five correlated factors representing scales could not be confirmed in our sample, indicating that appropriate construct (factorial) validity for the original theoretical model was not possible to support. Possible explanations for not confirming the five-factor model for the Serbian version could be cultural differences in experiencing anxiety symptoms, differences in importance of particular items, and availability of more/less than five underlying factors.

CASI

Internal consistency coefficients for the CASI were high, 0.77 for a 13-item form and 0.85 for an 18-item form. The longer form possesses a higher α coefficient that could be an effect of more items included, besides greater homogeneity among them. In different studies, it was demonstrated that sound internal consistency coefficients (α) for the CASI, ranging from 0.76 to 0.89 (Reiss et al. 2008), which was confirmed for the Serbian translation, too. Further, the CASI was developed as a unidimensional measure of anxiety sensitivity, but factorial analysis studies demonstrated two-, three-, or four-factor models for both forms, with the strongest support found for a hierarchical model with a single second-order and four first-order factors – Disease Concerns, Unsteady Concerns, Mental Incapacitation Concerns, and Social Concerns (Silverman et al. 2003). Acceptable goodness-of-fit indices were found for the 13-item CASI Serbian version with four factors, which was also demonstrated for the German version (Adornetto et al. 2008), while for the CASI-18, the four-factor model could not be confirmed in this sample. Considering that this is a comparative study of four different questionnaires, we tested only the most superior model with four factors, without exploring other possibilities of two- and three-factor models (van Widenfelt et al. 2002; Silverman et al. 2003; Adornetto et al. 2008).

SMFQ

The internal consistency reliability of the SMFQ was 0.85 indicating high homogeneity of the items that measure depressive symptoms. The original SMFQ questionnaire had the internal consistency reliability of 0.85, the same as in this study, while it ranged from 0.81 to 0.89 in different samples (Angold et al. 1995; Messer et al. 1995; Cheng et al. 2009). Moreover, the goodness-of-fit indices of CFA indicated acceptable fits of the data to the unidimensional structure of the SMFQ, implying that the items measure indeed one underlying construct, depressive symptoms. These data agree with the previous studies of the SMFQ and its unidimensional structure (Messer et al. 1995; Sharp et al. 2006).

CDI-S

The internal consistency reliability of the CDI-S was also high, 0.74. However, the reported internal consistency reliability of the original version was 0.80 (Kovacs, 2003) and 0.79 for the Irish version (Meehan et al. 2008). Nevertheless, the goodness-of-fit indices of CFA indicated excellent fits of the data to the unidimensional structure CDI-S, implying that the items measure one underlying construct, depressive symptoms as the SMFQ. However, no data are available for the original CDI-S to compare the findings.

Finally, considering convergent validity, high correlations between the questionnaires that measure the same construct were found, for anxiety between the SCARED, CASI-13 and CASI-18 and for depression, between the SMFQ and CDI-S. On the other hand, low to moderate correlations between the questionnaires that measure different constructs were found. These findings confirm that all Serbian versions have appropriate convergent validity as the original ones. Convergent validity was also supported for all SCARED scales that correlated in a predicted fashion with the other questionnaires, except the SCARED–GAD that had higher correlations with the SMFQ/CDI-S. Finally, moderate correlations between the SCARED and SMFQ/CDI-S, observed in other studies (Su et al. 2008; Cheng et al. 2009; Isolan et al. 2011) and in a Serbian study as well (Damnjanovic et al. 2011), might imply on high comorbidity between anxiety disorders and depression reported in children.

There are some limitations of the study. First, all children and adolescents came from a non-clinical sample, and therefore the results cannot be extrapolated to clinical samples. Second, we have not used a clinical diagnostic interview that could further establish the validity of the questionnaires in our sample. Third, test–retest reliability and sensitivity and specificity were not evaluated.

In summary, the SCARED questionnaire possesses acceptable internal consistency, except its scale about separation and school anxiety, and sound convergent validity. However, appropriate construct validity for the original five-factor model was not possible to support and the SCARED needs to be revised before wider use in Serbia. Further, both CASI versions (13- and 18-item form) possess acceptable internal consistency, sound convergent validity, but only the 13-item version possesses appropriate construct (factorial) validity and the use of this form is encouraged. Finally, both questionnaires for depression screening, SMFQ and CDI-S, possess acceptable internal consistency, construct validity (unidimensional factorial structure), and sound convergent validity. Being aware of the limitations of the study, as well as the findings in our ongoing projects, the factorial structure of the SCARED and CASI is further evaluated including multiple groups CFA to examine factorial invariance across different groups (configural invariance, metric invariance, and scalar invariance), whereas all questionnaires are further evaluated in clinical samples, and their sensitivity and specificity are being determined.