Background
Refugees face a myriad of pre- and post-migration stressors that have an impact on their mental health and well-being.Reference Miller and Rasmussen1 Levels of disability and functioning in refugee populations are moderated by both environmental and biopsychosocial factors. Environmental factors, such as limited access to basic needs and harsh living conditions, can lead to reduced abilities to function.Reference Drescher, Kiselev, Akhtar, Acarturk, Bryant and Ilkkursun2 Psychosocial distress and poor physical health directly have an impact on disability and functioning.Reference Li, Liddell and Nickerson3 There is a well-known link between emotional distress and related impairments in functioning, meaning those with higher levels of distress are likely to experience more marked impairment and disability.Reference Schick, Zumwald, Knöpfli, Nickerson, Bryant and Schnyder4
The International Classification of Functioning, Disability, and Health (ICF) framework defines disability as an impairment in individual functions, activities and participation, leading to a decrement in the individual's ability to function.5 Disability can be considered a broad construct spanning several domains of daily life, making it challenging to measure. With the increased focus on the mental health of conflict-affected populations in emergency settings, programmatic initiatives must be able to effectively screen for disability to better inform clinical practice and public health policy. The World Health Organization (WHO) Disability Assessment Schedule (WHODAS) was developed to effectively measure disability across the six domains as outlined in the ICF.Reference Üstün, Chatterji, Kostanjsek, Rehm, Kennedy and Epping-Jordan6
The WHODAS 2.0 is a generic biopsychosocial measure that aims to capture levels of functioning and disability, and aspects of health-related quality of life, across the six domains of functioning: (a) cognition, (b) mobility, (c) self-care, (d) getting along, (e) life activities (household and work), and (f) participation.Reference Üstün, Kostanjsek, Chatterji and Rehm7 There are three versions of the WHODAS 2.0; 12-item, 12 + 24-item and 36-item versions, and it has been widely translated into over 30 languages. Stringent tests performed on the WHODAS have shown it can be used across cultures, genders and age groups, and across different diseases and health conditions, including both mental health and physical disabilities.Reference Federici, Bracalenti, Meloni and Luciano8
The psychometric properties of the WHODAS 2.0 36-item version have been widely documented through the initial development work of the measure,Reference Üstün, Chatterji, Kostanjsek, Rehm, Kennedy and Epping-Jordan6 as well as subsequent research, yet fewer validation studies have explored the psychometric properties and factor structure of the 12-item version.Reference Federici, Bracalenti, Meloni and Luciano8 During the initial development of the measure, the 12-item version was found to explain 81% of the variance of the 36-item scale. The 12-item version is often recommended for contexts that are characterised by time constraints and in turn could be considered an appropriate measure for screening purposes.Reference Üstün, Chatterji, Kostanjsek, Rehm, Kennedy and Epping-Jordan6,Reference Üstün, Kostanjsek, Chatterji and Rehm7 Because of its brevity and ease of use, the WHODAS 2.0 has been widely used as a screening tool in large-scale studies.Reference Saltychev, Katajapuu, Bärlund and Laimi9 The psychometric properties of the 12-item version have since been explored in both general populations and a variety of clinical settings.Reference Saltychev, Katajapuu, Bärlund and Laimi9 Exploratory factor analysis (EFA) of the 12-item measure have found varied results as to the number of factors with results varying between one and five structures. Predominantly though, results of EFA feature one- and three-factor structures across diverse populations.Reference Abedzadeh-Kalahroudi, Razi, Sehat and Asadi-Lari10–Reference Subramaniam, Abdin, Vaingankar, Sagayadevan, Shahwan and Picco19 Confirmatory factor structures of the 12-item version have been more seldom explored and often constrained to either a one-factor structure with the individual items falling onto one global disability factor or the original second-order structure of the 36-item version with six domains of disability loading onto one disability factor.Reference Saltychev, Bärlund, Mattie, McCormick, Paltamaa and Laimi15,Reference Sousa, Dewey, Acosta, Jotheeswaran, Castro-Costa and Ferri18–Reference Younus, Wang, Yu, Fang and Guo32
With the increase in global mental health research in humanitarian contexts, it is imperative that commonly used psychological measures are validated to provide evidence for future use.Reference Tol, Barbui, Galappatti, Silove, Betancourt and Souza33 Although the WHODAS was developed to allow for a standardised way to effectively measure disability cross-culturally,Reference Denu, Yassin, Bisetegn, Biks and Gelaye22,Reference Tay, Rees, Miah, Khan, Badrudduza and Morgan34 no studies have reported its applicability in a population of Syrian refugees so far. Additionally, the psychometric properties of the WHODAS have not been explored in populations detained in refugee camps, where specific stressors may have an impact on functioning and how it is measured, such as poor living conditions, limited mobility, unmet basic needs and restricted social access.Reference Rasmussen, Nguyen, Wilkinson, Vundla, Raghavan and Miller35,Reference Al-Rousan, Schwabkey, Jirmanus and Nelson36
Aims
The aim of this study was to examine the psychometric properties of the Arabic 12-item version of the WHODAS 2.0. More specifically, we aimed to conduct an EFA and confirmatory factor analysis (CFA), as well as exploring the reliability and internal consistency of the WHODAS 2.0 as a screening tool in large sample of Syrian refugees residing in a camp in Jordan.
Method
Setting and participants
There are approximately 650 000 Syrian refugees registered in Jordan, of which 125 000 (20%) reside in one of three official refugee camps.37 Azraq camp is the second largest in Jordan with a population of 36 657 as of June 2020.38 There are currently 8660 shelters in use across four residential villages, two of which were used to recruit participants for this study. Participants were recruited between August and December 2019, as part of a screening procedure for a larger randomised controlled trial (RCT).Reference Akhtar, Giardinelli, Bawaneh, Awwad, Naser and Whitney39 Potential participants were identified through door-to-door screening of consecutive caravans. One adult from each caravan was subsequently invited to participate in the screening procedure for the trial if they met the following criteria: (a) Syrian refugee, (b) ≥18 years old, and (c) had a child or dependent living in the household aged 10–16 years.
Screening was conducted by assessors who received 4 days training in research ethics, the assessment battery, data collection and general interviewing techniques. Assessments were conducted on a digital tablet to ensure that data could be reliably collected and uploaded. Assessors administered the questionnaires in an interviewer format.
The study has been approved locally by the Institutional Review Board at the King Hussein Cancer Centre in Amman, Jordan and the University of New South Wales Human Research Ethics Committee. Informed consent was solicited prior to participation in the study; participants completed a written consent form and those who were unable to do this provided witnessed oral consent, in line with recommendations from the WHO. The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2000.
Instruments
WHODAS 2.0 – 12-item version
The WHODAS 2.0 is a general measure of disability, encompassing six domains (cognition, mobility, self-care, getting along, life activities and participation), assessing difficulties people have experienced during the past 30 days. The 12-item interviewer-administered version is an abridged version of the full measure and has two items from each of the six domains. Items are scored on a 5-point Likert scale (0, none; 4, extreme or cannot do), with total scores calculated by the sum of the 12 items, resulting in a range of 0–48. Higher scores on the WHODAS 2.0 correspond to greater levels of disability. A cut-off score of 17 was adopted as an indicator of moderate impairment as this represents the 90th percentile of impairment based on WHO normative data.Reference Üstün, Kostanjsek, Chatterji and Rehm7 We adapted the WHODAS 2.0 Arabic version, which was last translated in 1999, to the updated 2010 version of the WHODAS 2.0 in accordance with gold-standard translation practices.Reference Bontempo40 Items were translated and back translated by accredited translators, with discrepancies rectified jointly by the research team and an independent bilingual individual with previous experience working with health-related questionnaires. The feasibility and comprehensibility of the adapted version was piloted among Syrian refugees and found to be acceptable (not reported). The measure was completed by participants with the assistance of Arabic-speaking assessors during an in-person screening.
Kessler Psychological Distress Scale (K10)
The K10 is a general measure of psychological distress.Reference Kessler, Andrews, Colpe, Hiripi, Mroczek and Normand41,Reference Easton, Safadi, Wang and Hasson42 The ten items measure symptoms of anxiety and depression experienced in the preceding 30 days. Responses are scored on a scale of 1 (none of the time) to 5 (all of the time) with total scores calculated as the sum of all items with a range of 10–50. Higher scores indicate greater levels of psychological distress. The K10 has been validated in Arabic-speaking populations.Reference Sulaiman-Hill and Thompson43
Statistical analysis
Reliability of the WHODAS 2.0 was assessed in the current sample by analysing the internal consistency with Cronbach's alpha coefficient. Following Nunnally's recommendations of reliability, a cut-off score of 0.7 was considered acceptable for research purposes.Reference Nunnally44 In addition, to assess whether the individual items were related to the same construct, the item-deleted Cronbach's alpha coefficients were calculated. Internal consistency was further explored through item-total correlations, correlations between individual items of the scale and the total score.
Convergent validity was assessed by considering WHODAS 2.0 responses in relation to the K10, a general measure of psychological distress; this approach was conducted because of demonstrated associations between common mental disorders and impaired functioning. To this end, convergent validity between the WHODAS 2.0 and the K10 was assessed.
An EFA was conducted to determine the construct validity of the questionnaire, and to assess the number of latent factors. Prior to conducting an EFA, the Kaiser-Meyer-Olkin test (KMO) and Bartlett's test of sphericity were computed to determine factorability and whether an EFA would be appropriate. To determine the number of latent factors of the measure, the scree plot was observed and the number of factors scoring above an Eigenvalue of 1 was used for the EFA. The varimax rotation was used to extract the factors. Items were considered loaded onto a factor using a conservative value of 0.4 for the correlations between items and components.Reference Osborne, Costello and Kellow45
The results of the EFA were used to inform the model structure of the CFA. As varimax rotation was used for the EFA, correlations between the individual factors were also modelled into the CFA. The final factor structure and results were reported as standardised correlation coefficients. The model fit for the observed data are reported with the root mean square error of approximation (RMSEA), the comparative fit index (CFI), and the Tucker–Lewis Index (TLI). Acceptable model fit was defined as RMSEA <0.08, CFI >0.90 and TLI >0.90; good model fit was defined as RMSEA <0.05, CFI >0.95 and TLI >0.95.Reference Bentler46 All analyses were performed using SPSS 26.0 and SPSS Amos 26.0.47
Results
There were 650 participants screened as part of the larger RCT. Of those who were screened, 446 (69%) were female, 596 (92%) were married, and the average age was 40.4 years (s.d. = 7.1). The majority of participants had not previously received any formal education (n = 146, 22%) or had been enrolled in a basic education certificate (n = 381, 59%) which less than 50% (n = 179) completed. Very few respondents reported having previously attended post-secondary education (n = 16, 2%). The average WHODAS 2.0 score was 20.5 (s.d. = 7.6), with a minimum and maximum score reported of 0 and 44, respectively. There were no missing data for any of the WHODAS 2.0 items.
The overall internal consistency of the measure was acceptable, with a Cronbach's α of 0.74 (Table 1). The ‘item-deleted’ analysis indicated that all items were interdependent and related to one another, with Cronbach's alpha scores ranging from 0.71 to 0.75. When exploring construct validity, the scores of the individual items were significant and positively correlated with the overall score of the WHODAS 2.0 and K10 scores. Correlations for items 10 (0.39) and 11 (0.36) were relatively weak (Pearson's R < 0.40) with the other ten items having moderate to strong correlations ranging from 0.40 to 0.62. Correlations were positive when exploring the convergence between the WHODAS 2.0 and K10, with correlation coefficients ranging from 0.18 to 0.35, and the overall scores having a correlation coefficient of 0.57 (P < 0.001).
Table 1 Internal consistency and convergent validity
Kessler 10-item Psychological Distress Scale.
a. All values significant at α = 0.05.
We next explored the factor structure of the WHODAS 2.0. The Bartlett's test of sphericity was significant (P < 0.001) coupled with a KMO score of 0.782 indicating factorability and appropriateness to proceed with an EFA. Three factors were extracted while maintaining an Eigenvalue >1, which explained 51% of overall variance of the measure. Factor loadings of the individual items were strong, ranging from 0.46 to 0.88 (Table 2). The first factor had six loadings (items 1, 2, 5, 6, 7, 12), all of which could be classified under the umbrella of ‘internal activities’ and explained 27% of variance. A second factor had four loadings (items 3, 4, 10, 11) explaining 13% of the variance with items relating to ‘external activities’. The last factor had the remaining two items relating to ‘self-care’ (items 8, 9) and accounted for an additional 11% of variance. The resulting component matrix and organization of individual items in the corresponding factor variables is logical from a content point of view.
Table 2 Rotated component matrixa
a. Extraction method: principal component analysis. Rotation method: Varimax with Kaiser Normalization – rotation converged in four iterations.
The three-factor structure of the WHODAS 2.0 according to the EFA was used to inform the subsequent CFA. As an oblique rotation was used for the EFA, between-factor correlations were built into the CFA model. The results of the CFA are presented in Fig. 1. To explore model fit, residual covariances were utilised to observe if any loading variables had significant correlations within a factor. Based on low values of modification indices, none were included. The final CFA model fit statistics were good with an RMSEA of 0.043 indicating a close-fit, a CFI of 0.954 and an acceptable TLI of 0.941.
Fig. 1 Factor structure of World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0) – 12-item Arabic version.
Discussion
Main findings and interpretation
These results are the first to explore the validity of using the WHODAS 2.0 as a screening tool to identify impairment in functioning in both a Syrian refugee context and a refugee camp. Using a representative sample, we found internal consistency to be acceptable for individual items and the total WHODAS 2.0 score, and that convergent validity was attained. The EFA presented evidence for a three-factor model, and a subsequent CFA confirmed good fit. The final three-factor model had variables relating to internal activities, external activities and self-care.
Internal reliability assessments suggested all 12-items of the WHODAS 2.0 measured the same construct and that no items in the abridged version should be excluded. It is important to note that the 12-item version was not developed to have separate constructs, rather it was proposed for use as a screener or in contexts marred by limited resources. The Cronbach's alpha score of 0.74 supports the use of the WHODAS 2.0 as a screening measure for disability among Syrian refugees and in a camp-based setting.
When examining correlations in relation to the K10, the WHODAS 2.0 total score had a moderate, yet significant correlation coefficient of 0.57, demonstrating convergence. Two items of the WHODAS 2.0 were found to be weakly correlated whereas the remaining ten were either moderate or strongly correlated. Functioning and psychological distress are distinct constructs, and although we would expect an overlap and in turn correlation between the two measures, convergent validity would have been better explored in relation to other measures of functioning. The moderate convergence might be explained by contextual factors unique to a camp environment. In a refugee camp there are a number of day-to-day factors, such as limitations to mobility and opportunities to socialise, that may directly have an impact on functioning separately from distress. For example, the ability to practice self-care is somewhat limited in camp environments where the focus tends to be primarily on attaining basic needs, rather than engaging in enjoyable activities to maintain well-being. These restrictions that are unique to camp-based settings may explain the moderate convergency between the constructs of functioning and distress.
The results of the EFA indicated strong support for a three-factor structure logically comprising ‘internal activities’, ‘external activities’ and ‘self-care’. In addition, using a conservative correlation cut-off score of 0.4 to determine factor loading, the items were categorised into a single latent factor without overlap. Three-factor structures of the measure have been previously reported in people with anxiety and stress disorders,Reference Axelsson, Lindsäter, Ljótsson, Andersson and Hedman-Lagerlöf12 motor disabilities,Reference Papadopoulou, Stasi, Bakalidou, Papageorgiou, Tsokani and Bratsi14 brain injuryReference Snell, Iverson, Panenka and Silverberg17 and in older adults in the general populationReference Gaskin, Lambert, Bowe and Orellana13 with a number of similarities in coupled items and factor loading.
The factor structure informed by the results of the EFA, displayed good model fit in the subsequent CFA. The current study only found two indicators measuring the ‘self-care’ factor. Although it is commonly cited that a minimum of three indicators are needed per factor, there are specific circumstances in which two indicators are sufficient.Reference Bollen48 Given the orthogonal nature of the CFA and that the covariances between the three factors are non-zero, the two-indicator loading of the self-care factor is acceptable. Additionally, given the conceptual logic of the final factors and their respective loading variables, the strong loading for each of the ‘self-care’ indicators (0.71–0.85), and the overall good model fit, having only two indicators for the self-care factor is not considered problematic.
An important consideration of EFA is that the factor structure and number of factors of a measure can vary between different populations, depending on variability in sample selection.Reference Comrey49 This has been specifically observed for the WHODAS 2.0 12-item version.Reference Gaskin, Lambert, Bowe and Orellana13 Numerous studies utilising the WHODAS 2.0 12-item version, have assumed identical factor structures to that of the 36-item version when conducting CFA. Varying reports of factor structures observed in published EFAs suggests assuming a similar factor structure to that of the 36-item may be problematic. Additionally, constraining the model to two-loading indicators per variable could result in violations of the CFA statistical assumptions.
Limitations
As this study was nested within a larger RCT, the ability to test for other often reported psychometric properties were limited. As a number of participants who would have screened negative for participation into the trial were not followed up we were unable to observe the test–retest reliability or the sensitivity to change of the measure. In addition, we were not able to observe the sensitivity or specificity of the tool in relation to the full measure or a gold standard. Finally, the study was conducted in a closed refugee camp so the generalizability of the results may be limited.
Implications
This is the first reported factor structure of the WHODAS 2.0 12-item Arabic version among Syrian refugees, and in a camp setting. Our findings are consistent with the original intention of the development of the WHODAS 2.0 12-item version, to be used as a screener; the internal reliability of the measures was acceptable, and a Cronbach's alpha above 0.7 indicated that it is additionally appropriate for use in a research setting. The exploratory and subsequent CFA provided evidence for a three-factor structure in this population. Further testing of the WHODAS 2.0 measure in refugee populations would be beneficial and would assist with the validation for further use of the tool.
Data availability
The data is initially available to the STRENGTHS Consortium to allow individual patient data meta-analyses of all PM+ trials conducted within this consortium. Following this, data will be freely available on request.
Acknowledgements
We would like to thank Ahmad Bawaneh, Manar Awwad, Hadeel Al-Hayek and the staff at International Medical Corps Jordan Office for their contribution to the project.
Author contributions
A.A., P.C., N.M., M.S. and R.B. are responsible for the study concept and design. A.A. and R.B. developed the methods and conducted the data analysis. A.A. wrote the first draft of the manuscript, and P.C., N.M., M.S. and R.B. contributed to writing and critically revising the final manuscript. All authors reviewed and approved the final manuscript.
Financial supports
This work was supported by a National Health and Medical Research Council-European Union Grant [1142605] and a European Union Horizon 2020 grant [733 337]. The study funders had no role in study design; collection, management, analysis and interpretation of data; writing of the report; or the decision to submit the report for publication.
Declaration of interest
None.
Open access
eLetters
No eLetters have been published for this article.