Hostname: page-component-cd9895bd7-gvvz8 Total loading time: 0 Render date: 2024-12-28T13:18:42.866Z Has data issue: false hasContentIssue false

The dynamic nature of refugee children's resilience: a cohort study of Syrian refugees in Lebanon

Published online by Cambridge University Press:  15 June 2022

C. M. Popham
Affiliation:
Department of Biological and Experimental Psychology, Queen Mary University of London, London, UK
F. S. McEwen
Affiliation:
Department of Biological and Experimental Psychology, Queen Mary University of London, London, UK
E. Karam
Affiliation:
Institute for Development, Research, Advocacy and Applied Care, Beirut, Lebanon Saint George Hospital University Medical Center, Beirut, Lebanon Faculty of Medicine, Balamand University, El-Koura, Lebanon
J. Fayyad
Affiliation:
Institute for Development, Research, Advocacy and Applied Care, Beirut, Lebanon Saint George Hospital University Medical Center, Beirut, Lebanon Faculty of Medicine, Balamand University, El-Koura, Lebanon
G. Karam
Affiliation:
Institute for Development, Research, Advocacy and Applied Care, Beirut, Lebanon Saint George Hospital University Medical Center, Beirut, Lebanon Faculty of Medicine, Balamand University, El-Koura, Lebanon
D. Saab
Affiliation:
Institute for Development, Research, Advocacy and Applied Care, Beirut, Lebanon
P. Moghames
Affiliation:
Medecins du Monde, Beirut, Lebanon
M. Pluess*
Affiliation:
Department of Biological and Experimental Psychology, Queen Mary University of London, London, UK
*
Author for correspondence: M. Pluess, E-mail: m.pluess@qmul.ac.uk
Rights & Permissions [Opens in a new window]

Abstract

Aims

Children's responses to war and displacement are varied; many struggle, while others appear resilient. However, research into these outcomes disproportionately focuses on cross-sectional data in high-income countries. We aimed to (1) investigate change in resilience across two timepoints in a highly vulnerable sample of Syrian refugee children in Lebanon, and (2) explore predictors of their mental health problems across time.

Methods

In total, 982 Syrian child–caregiver dyads living in refugee settlements in Lebanon completed questionnaires via interview at baseline and follow-up one year later. We categorised children into groups based on their risk for mental health problems across both timepoints (stable high risk/SHR, deteriorating, improving, stable low risk) according to locally validated cut-offs on measures of post-traumatic stress disorder (PTSD), depression and behavioural problems. Analyses of covariance identified how the groups differed on a range of individual and socio-environmental predictors, followed up by cross-lagged panel models (CLPMs) to investigate the directionality of the relationships between significantly related predictors and symptoms.

Results

The sample showed a meaningful amount of change in mental health symptoms from baseline to follow-up. Over half (56.3%) of children met SHR criteria and 10.3% deteriorated over time, but almost one-quarter (24.2%) showed meaningful improvement, and 9.2% were consistently at low risk for mental health problems at both timepoints. Several predictors differentiated the groups, particularly social measures. According to CLPMs, maternal acceptance (β = −0.07) predicted child mental health symptoms over time. Self-esteem (β = −0.08), maternal psychological control (β = 0.10), child maltreatment (β = 0.09) and caregiver depression (β = 0.08) predicted child symptoms and vice versa (βse = −0.11, βb = 0.07, βmpc = 0.08, βcm = 0.1, βcd = 0.11). Finally, child symptoms predicted loneliness (β = 0.12), bullying (β = 0.07), perceived social support (β = −0.12), parent–child conflict (β = 0.13), caregiver PTSD (β = 0.07), caregiver anxiety (β = 0.08) and the perceived refugee environment (β = −0.09).

Conclusions

Our results show risk and resilience are dynamic, and the family environment plays a key role in children's response to war and displacement. Conversely, children also have a significant impact on the family environment and caregiver's own mental health. Interventions to promote resilience in refugee children should therefore consider family-wide mechanisms.

Type
Original Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
Copyright © The Author(s), 2022. Published by Cambridge University Press

There are 5.6 million Syrian refugees worldwide, half of whom are children. Most have been exposed to a wide range of war experiences, displacement and post-displacement adversities. Many resettle in unstable contexts such as informal settlements (United Nations High Commissioner for Refugees (UNHCR), Reference United Nations High Commissioner for Refugees2020). Despite these challenges, children's mental health varies substantially; while many develop mental health problems including post-traumatic stress disorder (PTSD), depression, and behavioural problems (Kien et al., Reference Kien, Sommer, Faustmann, Gibson, Schneider, Krczal, Jank, Klerings, Szelag, Kerschner, Brattström and Gartlehner2019; Blackmore et al., Reference Blackmore, Gray, Boyle, Fazel, Ranasinha, Fitzgerald, Misso and Gibson-Helm2020; Henkelmann et al., Reference Henkelmann, de Best, Deckers, Jensen, Shahab, Elzinga and Molendijk2020), a notable proportion show no evidence of such difficulties (Müller et al., Reference Müller, Gossmann, Hartmann, Büter, Rosner and Unterhitzenberger2019; Scherer et al., Reference Scherer, Hameed, Acarturk, Deniz, Sheikhani, Volkan, Örücü, Pivato, Akıncı, Patterson and Polack2020). Given the extreme nature of the adversity refugee children face, we argue these children demonstrate manifested resilience, defined as better than expected development in the context of adversity (Masten, Reference Masten2016; Miller-Graff, Reference Miller-Graff2020).

Better understanding of refugee children's resilience could inform interventions for those struggling, but definitions of resilience vary (Cosco et al., Reference Cosco, Kaushal, Hardy, Richards, Kuh and Stafford2017). While some define resilience based on available resources, others focus on developmental outcomes of a putative process of resilience (i.e. manifested resilience; Miller-Graff Reference Miller-Graff2020). However, the process of adapting to adversity can take different trajectories (Popham et al., Reference Popham, McEwen, Pluess and Ungar2021). Children struggling at one timepoint may recover, while continuing accumulation of stressors may cause a child originally doing well to deteriorate (Müller et al., Reference Müller, Gossmann, Hartmann, Büter, Rosner and Unterhitzenberger2019). This could be particularly complex in populations exposed to ongoing adversity, such as refugees living in camps. Research thus far suggests that the mental health of conflict-affected children generally improves over time, but some children may not improve, and some may deteriorate (Müller et al., Reference Müller, Gossmann, Hartmann, Büter, Rosner and Unterhitzenberger2019; Hermosilla et al., Reference Hermosilla, Metzler, Savage and Ager2021).

Many individual and socio-environmental factors, such as coping strategies or social support, have been linked to refugee child mental health, but much of this research comes from high-income countries and cross-sectional data (Scharpf et al., Reference Scharpf, Kaltenbach, Nickerson and Hecker2021). Longitudinal research to date emphasises the importance of the family environment: caregiver mental health, parenting, and other aspects of family functioning are predictive of emotional and behavioural problems in refugee children (Panter-Brick et al., Reference Panter-Brick, Grimon and Eggerman2014; Sangalang et al., Reference Sangalang, Jager and Harachi2017; Bryant et al., Reference Bryant, Edwards, Creamer, O'Donnell, Forbes, Felmingham, Silove, Steel, Nickerson, McFarlane, Van Hooff and Hadzi-Pavlovic2018). However, although the focus is often on how socio-environmental factors impact the child, Syrian refugee mothers report how their children's mental health can also affect their own mental health and parenting (Rizkalla et al., Reference Rizkalla, Mallat, Arafa, Adi, Soudi and Segal2020). Further longitudinal research is needed to investigate such reciprocal relationships between children and their environment.

We aimed to further the research on child resilience following war and displacement, using two waves of data from Syrian refugee children included in the BIOPATH study (McEwen et al., Reference McEwen, Popham, Moghames, Smeeth, De Villiers, Saab, Karam, Fayyad, Karam and Pluess2022b). Specifically, we had three key aims: (1) identify the proportion of children at low risk for mental health problems in our sample and describe changes over time; (2) identify predictors of change in risk and resilience; (3) investigate the directionality of the relationships between identified predictors and mental health symptoms over time. We used low risk for clinical levels of PTSD, depression and externalising to approximate manifested resilience.

Methods

Study design

We addressed our aims using two waves of data from a large sample of Syrian refugee child–caregiver dyads from the BIOPATH cohort study (McEwen et al., Reference McEwen, Popham, Moghames, Smeeth, De Villiers, Saab, Karam, Fayyad, Karam and Pluess2022b). First, we created four groups based on change in risk for three common mental health problems in response to war and displacement (Kien et al., Reference Kien, Sommer, Faustmann, Gibson, Schneider, Krczal, Jank, Klerings, Szelag, Kerschner, Brattström and Gartlehner2019) from baseline to follow-up: (1) children with low symptoms on PTSD, depression and externalising behaviour problems at both waves (stable low risk/SLR), (2) children with low symptom scores on all three outcomes at baseline whose symptoms meaningfully worsened at follow-up (deteriorating), (3) children with high symptoms at baseline who showed meaningful improvement at follow-up (improving) and (4) children with continuously high symptom scores on any outcomes at both waves (stable high risk/SHR). We ran group comparisons to determine what factors characterised each of the four groups, and finally investigated the directionality of associations between children's mental health symptoms and the predictors identified in group comparisons using cross-lagged panel models (CLPMs). All procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation (McEwen et al., Reference McEwen, Popham, Moghames, Smeeth, De Villiers, Saab, Karam, Fayyad, Karam and Pluess2022b).

Setting and participants

Data were collected in the Beqaa region of Lebanon in 2017–2019. We used purposive cluster sampling, approaching small-to-medium-sized ITSs representing a range of vulnerabilities according to the UNHCR vulnerability index (McEwen et al., Reference McEwen, Popham, Moghames, Smeeth, De Villiers, Saab, Karam, Fayyad, Karam and Pluess2022b). Following agreement with community leaders, we approached all families present, and invited one child per eligible family (i.e. child aged 8–16 years, left Syria in the preceding 4 years, primary caregiver available) to participate. If more than one child in a family was eligible we invited the child whose birthday was closest to the recruitment date, to avoid selection bias. Informed consent and assent were given by each caregiver and child, respectively. Questionnaire data were collected by a team of interviewers in the settlements. Interviews took approximately 50–60 min. All measures were repeated one year later with approximately two-thirds of the original baseline sample. For a more detailed explanation of recruitment, see McEwen et al. (Reference McEwen, Popham, Moghames, Smeeth, De Villiers, Saab, Karam, Fayyad, Karam and Pluess2022b).

Variables

All participants were interviewed in their homes by trained (online Supplementary 1.1), local, native Arabic-speaking interviewers. Different interviewers conducted the child and caregiver interviews simultaneously. Some measures were exclusively child or caregiver reported, while others were reported by both (online Supplementary Table S1).

Mental health outcomes

The primary outcomes were self-reported PTSD (Child PTSD Symptom Scale/CPSS, Foa et al., Reference Foa, Johnson, Feeny and Treadwell2001), self-reported depression (Centre for Epidemiological Studies Depression Scale for Children/CES-DC, abridged, Faulstich et al., Reference Faulstich, Carey, Ruggiero, Enyart and Gresham1986), and parent-reported externalising behaviour problems, measured using the externalising subscale of the Strengths and Difficulties Questionnaire (SDQ, Goodman, Reference Goodman1997) and additional items related to conduct disorder and oppositional defiant disorder administered separately (McEwen et al., Reference McEwen, Popham, Moghames, Smeeth, De Villiers, Saab, Karam, Fayyad, Karam and Pluess2022b). Scales were chosen according to availability of Arabic versions and validity in similar populations. Following pilot testing with Syrian refugees in Lebanon, the CES-DC was abridged to ten items and minor changes to phrasing (including Arabic dialect) were made to the CES-DC and CPSS (McEwen et al., Reference McEwen, Moghames, Bosqui, Kyrillos, Chehade, Saad, Abdul Rahman, Popham, Saab, Karam, Karam and Pluess2020; online Supplementary 1.1). Cut-off scores on each outcome (12 out of 51 on the adjusted CPSS, 10 out of 30 on the adjusted CES-DC, and 12 out of 44 on the combined externalising scale total) were derived from structured clinical interviews (MINI-KID, Sheehan et al., Reference Sheehan, Sheehan, Shytle, Janavs, Bannon, Rogers, Milo, Stock and Wilkinson2010) and clinical judgement in a representative subsample (n = 119) of the cohort (McEwen et al., Reference McEwen, Moghames, Bosqui, Kyrillos, Chehade, Saad, Abdul Rahman, Popham, Saab, Karam, Karam and Pluess2020). Cut-offs had sensitivity of 81–85%, but specificity fell below 80%, meaning that some children flagged as at risk may not represent clinical cases. Children below cut-offs likely do not have clinical symptoms (negative predictive value of 79–91%). For more detailed information see online Supplementary 1.2. Finally, we measured wellbeing using the World Health Organisation – Five Wellbeing Index (Bech, Reference Bech2012; Topp et al., Reference Topp, Østergaard, Søndergaard and Bech2015).

Predictor variables

Individual and social factors: We investigated a variety of individual and social predictors that have been associated with children's mental health in previous research (online Supplementary Table S1). Individual-level predictors included optimism (Ey et al., Reference Ey, Hadley, Allen, Palmer, Klosky, Deptula, Thomas and Cohen2005), self-efficacy (Schwarzer and Jerusalem, Reference Schwarzer, Jerusalem, Weinman, Wright and Johnston1995), a single self-esteem item (Harris et al., Reference Harris, Donnellan and Trzesniewski2018), the temperament trait of environmental sensitivity (Pluess et al., Reference Pluess, Assary, Lionetti, Lester, Krapohl, Aran and Aron2018), coping strategies (Program for Prevention Research, 1999), future orientation (McEwen et al., Reference McEwen, Popham, Moghames, Smeeth, De Villiers, Saab, Karam, Fayyad, Karam and Pluess2022b), and a single item on the child's general health (McEwen et al., Reference McEwen, Popham, Moghames, Smeeth, De Villiers, Saab, Karam, Fayyad, Karam and Pluess2022b). The social environment measures included aspects of the caregiver–child relationship (maternal acceptance, Schaefer, Reference Schaefer1965; parental monitoring, Barber, Reference Barber1996; parent–child conflict, Barber Reference Barber1999; child maltreatment, Runyan et al., Reference Runyan, Dunne and Zolotor2009; maternal psychological control, Barber et al., Reference Barber, Xia, Olsen, McNeely and Bose2012; positive home experiences, McEwen et al., Reference McEwen, Popham, Moghames, Smeeth, De Villiers, Saab, Karam, Fayyad, Karam and Pluess2022b), the caregiver's own mental and general health (depression, Radloff, Reference Radloff1977; anxiety, Henry and Crawford, Reference Henry and Crawford2005; PTSD, Blevins et al., Reference Blevins, Weathers, Davis, Witte and Domino2015; a single general health item, McEwen et al., Reference McEwen, Popham, Moghames, Smeeth, De Villiers, Saab, Karam, Fayyad, Karam and Pluess2022b), relationships within and beyond the family (loneliness, Asher et al., Reference Asher, Hymel and Renshaw1984; perceived social support, Ramaswamy et al., Reference Ramaswamy, Aroian and Templin2009; bullying, McEwen et al., Reference McEwen, Popham, Moghames, Smeeth, De Villiers, Saab, Karam, Fayyad, Karam and Pluess2022b), and the child's home and employment responsibilities (McEwen et al., Reference McEwen, Popham, Moghames, Smeeth, De Villiers, Saab, Karam, Fayyad, Karam and Pluess2022b). Finally, caregivers reported their literacy, income, employment status, household size, and aspects of the wider environment (collective efficacy, Sampson et al., Reference Sampson, Raudenbush and Earls1997; human insecurity, Ziadni et al., Reference Ziadni, Hammoudeh, Abu Rmeileh, Hogan, Shannon and Giacaman2011; perceived refugee environment, McEwen et al., Reference McEwen, Popham, Moghames, Smeeth, De Villiers, Saab, Karam, Fayyad, Karam and Pluess2022b). For detailed information, see online Supplementary Table S1.

Exposure to war: War exposure was measured with the War Events Questionnaire (WEQ), a 25-item checklist of war events reported at baseline (Karam et al., Reference Karam, Al-Atrash, Saliba, Melhem and Howard1999). In line with recommendations for multiple informant approaches to war exposure (Oh et al., Reference Oh, Jerman, Purewal Boparai, Koita, Briner, Bucci and Harris2018), child and caregiver responses were combined such that if either one reported that the child experienced an event, the event was considered to have occurred.

Statistical methods

Analyses were conducted in RStudio. Multiple imputation using Fully Conditional Specification in the mice package (van Buuren and Groothuis-Oudshoorn, Reference van Buuren and Groothuis-Oudshoorn2011) was applied to impute the small number of missing data. We imputed all missing measures for the analysis, bar demographic variables, war exposure, and child mental health. We ran all analyses in both the imputed (N = 982) and original (N = 861) datasets and report the pooled imputation estimates in the main text of this paper. Complete case analyses are reported in online Supplementary sections 2.4 and 2.5.

Aim 1: change in risk and resilience

In order to investigate risk and resilience over time, we calculated the frequencies of four basic groups of mental health risk (SHR, deteriorating, improving, and SLR) using a two-step approach. At each wave, we created high- and low-risk groups using the locally validated clinical cut-offs for PTSD, depression, and externalising problems. If participants scored above the cut-off for any of the three measures, they were classed as in the high-risk group but if participants scored below all three cut-offs, they were classed in the low-risk group (i.e. resilience). We then adjusted the groupings at follow-up according to which children showed meaningful change at follow-up, defined as crossing the relevant cut-off(s) from baseline to follow-up to meet the low or high-risk criteria (i.e. below all cut-offs v. above any) paired with a change in symptom score of at least 20% on the relevant scale. Children who did not show meaningful change were classed as SHR/SLR. This ensured that small amounts of variability in reporting over time were not counted as categorical change.

Aim 2: group characteristics

Specific characteristics of the four groups were identified with a series of individual analyses of covariance for each predictor to compare their baseline scores and the change over time from baseline to follow-up. For each predictor we considered the effect of group membership on the baseline score controlling for change score, then on the change over time while controlling for the baseline score. In each model we also controlled for the effects of war exposure, age, gender, and time since leaving Syria. The significance level of each model was corrected using the Benjamini–Hochberg correction to account for the total number of models tested (Benjamini and Hochberg, Reference Benjamini and Hochberg1995).

Aim 3: directionality of predictor – mental health relationships

Each predictor that was significantly associated with group differences in Aim 2 was further investigated using CLPMs in order to investigate the directionality of effect. However, in place of the categorical grouping, we used a continuous mental health symptom composite score to improve power. This was calculated by taking the average of the three primary outcome measures (PTSD, depression, and externalising) each adjusted for the number of items per scale. We ran a series of CLPMs using the semTools package (Jorgensen et al., Reference Jorgensen, Pornprasertmanit, Schoemann and Rosseel2021) containing the child mental health symptom composite at both waves, and the predictor of interest (e.g. self-esteem) at both waves. The models included autoregressive and cross-lagged paths, and within-time covariance. As with Aim 2, we controlled for the effects of age, gender, time since leaving Syria, and war exposure on baseline and follow-up scores for the predictor variable and symptom score (online Supplementary Fig. S1 illustrates the model format).

Results

Descriptive data

The final sample consisted of 982 child–caregiver dyads with data at both waves (Table 1). Approximately half (52.9%) the children were female, and at baseline children's average age was 11.22 years (s.d. = 2.34), 42.4% had left Syria in the past 3 years, and the remainder had left more than 3 years previously. Children reportedly experienced up to 24 (M = 9.57, s.d. = 5.47) different types of war events. The majority (91.1%) of participating caregivers were the child's mother. The proportion of children above clinical cut-offs at baseline and follow-up was 54.9 and 34.4% respectively for PTSD, 37.8 and 27% for depression, and 43.9 and 41.9% for externalising behaviour problems. This longitudinal sample represented 61.7% of the baseline sample, and showed no differences likely to create substantial bias (online Supplementary 2.1; McEwen et al., Reference McEwen, Popham, Moghames, Smeeth, De Villiers, Saab, Karam, Fayyad, Karam and Pluess2022b).

Table 1. Sample characteristics

Note: Descriptive statistics on key demographics and change in mental health. Analyses of variance/χ 2 tests used where appropriate to compare the groups, and Tukey's post-hoc tests reported for significant ANOVAs. *p < 0.05, **p < 0.01, ***p < 0.001.

a Child mental health symptom composite score (average of PTSD, depression and externalising symptom scores, adjusted for number of items).

Aim 1: change in risk and resilience

The percentage of children meeting the low-risk criteria increased from 19.5% at baseline to 33.4% at follow-up, but all four groups (SHR, deteriorating, improving, SLR) were represented in the data (Fig. 1). Of the originally 791 high-risk cases at baseline, 553 (69.9%) remained in the high-risk group (scoring above at least one cut-off) at follow-up (SHR), while 238 (30.1%) moved below all cut-offs, showing a reduction in symptoms of at least 20% (M = 65.0%) on the relevant outcomes (improving). Of the 191 children with low risk at baseline, 90 (47.1%) remained below all cut-offs at follow-up (SLR) whilst 101 (52.9%) showed increased risk and scored above at least one cut-off at follow-up (deteriorating), with an increase in symptoms of at least 20% (M = 166.7%). At baseline and follow-up, children meeting low-risk criteria reported significantly higher wellbeing (M w1 = 74.79, s.d.w1 = 19.44; M w2 = 78.12, s.d.w2 = 18.85) compared to those meeting high-risk criteria (M w1 = 65.47, s.d.w1 = 26.95; t w1(387.19) = −5.47, p w1 < 0.001, d w1 = 0.4; M w2 = 66.61, s.d.w2 = 29.39; t w2(918.16) = −7.41, p w2 < 0.001, d w2 = 0.47). SHR children had significantly higher reported war exposure (M = 10.47) compared to all other groups (M D = 7.54; M I = 8.98; M SLR = 7.88; F (3, 978) = 13.87, p < 0.001). Girls were more likely to be SLR (χ 2 = 11.19, p = 0.011). See Table 1 for further group comparisons.

Fig. 1. Mental health risk change from baseline to follow-up.

Aim 2: group characteristics

The four groups differed significantly on a range of variables at baseline and in change over time (Table 2). The improving group was characterised by better perceived refugee environment at baseline compared to the other groups. The SHR group differed from the other groups on a larger number of variables, characterised by lower baseline scores on several protective/promotive factors, higher baseline scores on a range of social risk factors, and greater increases in loneliness and social isolation and maternal psychological control over time. Change in a range of factors significantly differentiated children with low risk (improving and SLR risk groups) from those with higher risk (deteriorating and SHR risk groups) at follow-up (Table 2).

Table 2. Analyses of covariance: results from significant models

Note: Table representing descriptive statistics and analyses of covariance (ANCOVAs) from significant predictors using imputed data (N = 982). Child age, gender, time since leaving Syria and war exposure were entered as covariates into all ANCOVAs. Baseline models controlled for change scores, and change models controlled for baseline scores. F statistic is based on test against null model including only covariates. Adjusted R 2 is based on full model. Post-hoc comparisons are based on Tukey's test. Means and s.d.s are unadjusted estimates, all other statistics are based on adjusted means according to the ANCOVA models. See online Supplementary Table S2 for all ANCOVA results. p Values based on Benjamini–Hochberg correction for multiple testing (Benjamini and Hochberg, Reference Benjamini and Hochberg1995). *p < 0.05, **p < 0.01, ***p < 0.001.

a Higher scores on child and caregiver general health indicate worse health.

Aim 3: directionality of predictor – mental health relationships

For every predictor whose baseline or change score significantly differed between groups, CLPMs were used to investigate the direction of relationship between the predictor in question and the composite mental health symptom score. All CLPMs were just identified so there was no information about fit. Several cross-lagged pathways emerged as significant (Table 3). Some pathways were not significant in the complete case analysis due to reduced power (online Supplementary Fig. S3), so we report the imputed estimates for pathways that were significant in the imputed data and supported by similar trend-level estimates in the complete case data. Following those criteria, the key results were as follows. Baseline maternal acceptance (β = −0.07, p = 0.046) was predictive of later child mental health symptoms. Caregiver depression at baseline was predictive of child mental health symptoms at follow-up (β = 0.08, p = 0.009) and vice versa (β = 0.11, p < 0.001), as was the case for maternal psychological control (β pc-mh = 0.10, p pc-mh = 0.003; β mh-pc = 0.08, p mh-pc = 0.011), child maltreatment (β m-mh = 0.09, p m-mh = 0.009; β mh-m = 0.1, p mh-m = 0.005), and self-esteem (β se-mh = −0.08, p se-mh = 0.033; β mh-se = −0.11, p mh-se = 0.003). Baseline child mental health symptoms were predictive of optimism, loneliness and social isolation, bullying, perceived social support, parent–child conflict, caregiver PTSD, caregiver anxiety, and the perceived refugee environment at follow-up, but none of these predictors significantly predicted child symptoms at follow-up (online Supplementary Fig. S2). Figure 2 contains examples of key predictors from the individual, family, and wider systems that showed uni- and bi-directional cross-lagged relationships with child symptoms.

Fig. 2. Cross-lagged panel models representing key predictors from the individual, family, and community systems.

Table 3. Summary of cross-lagged panel models with significant cross-lagged pathways

Note: Table depicting the coefficients and p values of the pathways from the cross-lagged panel models with significant cross-lagged pathways between predictor and child symptoms in either direction. Complete case estimates are shown in brackets where they differ from the imputed estimates. Letters a, b, c, d, e, f correspond to the pathway labels in online Supplementary Fig. S1: a = child symptom auto-regressed pathway; b = predictor auto-regressed pathway; c = W1 covariance; d = W2 covariance; e = cross lagged pathway: predictor → symptoms; f = cross lagged pathway: symptoms → predictor. *p < 0.05, **p < 0.01, ***p < 0.001.

Discussion

Our aim was to investigate change in and predictors of risk and resilience over time in a sample of Syrian refugee children living in a particularly challenging context in Lebanon. The children were categorised into four groups based on their change in risk for mental health problems across two timepoints one year apart: SHR, deteriorating, improving, and SLR. Many predictors differentiated these groups from one another, but social and familial predictors were of particular importance, and showed reciprocal relationships with children's symptoms.

Change in risk and resilience over time

Mental health in our sample was dynamic, and overall improved; a greater proportion of children met low-risk criteria at follow-up compared to baseline. In total, 9.2% of the sample were low risk at both waves (SLR) and 24.2% improved from showing likely clinical levels of PTSD, depression, and/or externalising behaviour problems at baseline to scoring below all three cut-offs at follow-up (improving). We used this as an indicator of resilience; children that met the improving and SLR criteria demonstrated evidence of resistance to or recovery from the psychological impact of war and displacement, and can therefore be described as resilient (Masten, Reference Masten2016).

These findings fit with recent research finding overall improvements over time in children affected by conflict (Müller et al., Reference Müller, Gossmann, Hartmann, Büter, Rosner and Unterhitzenberger2019; Purgato et al., Reference Purgato, Tedeschi, Bonetto, de Jong, Jordans, Tol and Barbui2020; Hermosilla et al., Reference Hermosilla, Metzler, Savage and Ager2021) and show that even in challenging post-displacement contexts some children demonstrate manifested resilience. However, 10.3% of the sample deteriorated over time, the proportion of children meeting externalising criteria remained notably higher than in previous reviews (Kien et al., Reference Kien, Sommer, Faustmann, Gibson, Schneider, Krczal, Jank, Klerings, Szelag, Kerschner, Brattström and Gartlehner2019; Blackmore et al., Reference Blackmore, Gray, Boyle, Fazel, Ranasinha, Fitzgerald, Misso and Gibson-Helm2020), and the proportion of children with persistently high general risk was larger than seen in children resettled in Europe (Müller et al., Reference Müller, Gossmann, Hartmann, Büter, Rosner and Unterhitzenberger2019). This emphasises the need for longitudinal research in a variety of contexts, as children doing relatively well across multiple measures can begin to struggle, and those doing poorly can improve. The key question is what helps or hinders the resilience process.

Predictors of mental health and resilience

Results from our second and third aims provided some answers to this question. Predictors at the individual, family, and community levels were associated with children's mental health at baseline and over time, but CLPMs showed mental health at follow-up was primarily predicted by aspects of the immediate family context. At the individual level, self-esteem was associated with general low symptoms. Within the family, baseline maternal acceptance was predictive of later low child symptoms, while maternal psychological control, child maltreatment, and caregiver depression showed reciprocal relationships with child symptoms, and child symptoms were predictive of but not predicted by parent–child conflict and caregiver anxiety and PTSD. Child symptoms were also predictive of several factors in the wider social environment, including loneliness and perceived social support.

These results indicate some predictors that may be causally related to child mental health, in line with previous findings (Panter-Brick et al., Reference Panter-Brick, Grimon and Eggerman2014; Sangalang et al., Reference Sangalang, Jager and Harachi2017; Bryant et al., Reference Bryant, Edwards, Creamer, O'Donnell, Forbes, Felmingham, Silove, Steel, Nickerson, McFarlane, Van Hooff and Hadzi-Pavlovic2018), but also reflect a more complex story. The bidirectional relationships between children's symptoms and social factors demonstrate a vicious cycle of risk. For example, emotional or behavioural problems could significantly impact caregiver mental health and the parent–child relationship, resulting in increases in harsh parenting which in turn negatively affect the child. This accords with personal accounts from Syrian mothers, who report that both their children's and their own mental health impacts their parenting (Rizkalla et al., Reference Rizkalla, Mallat, Arafa, Adi, Soudi and Segal2020). Beyond the family, noticeable symptoms may also impact social support due to peer stigmatisation (O'Driscoll et al., Reference O'Driscoll, Heary, Hennessy and McKeague2012) which could explain why child symptoms predict higher bullying and loneliness scores, and lower symptoms are associated with better social support. Treatment of child symptoms could therefore improve access to social resources.

However, the finding that caregiver depression and aspects of the parent–child relationship are predictive of later child symptoms emphasises the importance of a family-wide approach to treatment. Parenting interventions may be helpful for some families, but previous research suggests that parenting is also influenced by caregivers' own trauma and psychological distress (Sangalang et al., Reference Sangalang, Jager and Harachi2017; Bryant et al., Reference Bryant, Edwards, Creamer, O'Donnell, Forbes, Felmingham, Silove, Steel, Nickerson, McFarlane, Van Hooff and Hadzi-Pavlovic2018). There are multiple possible stressors in the refugee context, such as poor housing or food insecurity, which could additionally impact caregivers', and therefore children's, mental health (Li et al., Reference Li, Liddell and Nickerson2016). In fact, we observed that improving children reported better baseline refugee environment scores than others. Psychological support for caregivers or systemic family therapy could bolster resources within the family, but practical and community-level support may provide a baseline from which other interventions are more effective in the longer term. Future research should explore the impact of the wider environment through the family system and to the child.

Strengths and limitations

We provide novel findings looking at changes in risk and resilience over two waves of data, and the directionality of predictors of refugee mental health. This study is characterised by a longitudinal, challenging to reach sample that is representative of a large proportion of the global refugee population, the majority of whom reside in low- and middle-income countries, and, since 2014, have originated from Syria (UNHCR, Reference United Nations High Commissioner for Refugees2020).

Despite these strengths, our methods had some limitations. First, we measured mental health using self-reported symptom scales. However, scales were extensively piloted and, where possible, modified to be context-appropriate. Furthermore, we derived cut-offs through clinical assessment in a subsample, choosing cut-offs with the best balance of sensitivity and specificity for our particular sample (McEwen et al., Reference McEwen, Moghames, Bosqui, Kyrillos, Chehade, Saad, Abdul Rahman, Popham, Saab, Karam, Karam and Pluess2020). However, specificity fell below 80%, and consequently the high-risk groups may contain some false positives. Prevalence estimates adjusted for false positives and negatives are therefore lower than reported here (McEwen et al., Reference McEwen, Biazoli, Popham, Moghames, Saab, Fayyad, Bosqui, Karam and Pluess2022a), but adjustments cannot be applied at the individual level, so we retain unadjusted estimates. Secondly, PTSD, depression, and externalising problems may be differentially associated with some of the factors measured. However, we used the composite symptom score to complement our categorical approach, identify potential resilience factors, and identify associations between a child's general symptomatology and their environment (Jongedijk et al., Reference Jongedijk, Eising, van der Aa, Kleber and Boelen2020). Finally, a selection bias in recruitment and retention at follow-up cannot be excluded due to restricted access to certain settlements, reliance on presence of families during recruitment, and the high mobility of our sample. However, differences between the baseline and follow-up samples were small (McEwen et al., Reference McEwen, Popham, Moghames, Smeeth, De Villiers, Saab, Karam, Fayyad, Karam and Pluess2022b), meaning any substantial retention bias is unlikely.

Conclusion

In our longitudinal analysis of Syrian refugee children in Lebanon, many showed meaningful changes in risk and resilience from baseline to one year later. The overall proportion of children with no evidence of clinical symptoms of PTSD, depression, or externalising behaviour problems, from which we can infer demonstration of resilience, increased over time, although approximately half of the originally low-risk children deteriorated from one year to the next. Our results agree with previous research on the importance of specific social and familial factors (Sangalang et al., Reference Sangalang, Jager and Harachi2017; Bryant et al., Reference Bryant, Edwards, Creamer, O'Donnell, Forbes, Felmingham, Silove, Steel, Nickerson, McFarlane, Van Hooff and Hadzi-Pavlovic2018; Scharpf et al., Reference Scharpf, Kaltenbach, Nickerson and Hecker2021) for risk and resilience but also provide evidence of directionality over time. In particular, findings indicate reciprocal relationships between children and caregiver's mental health, and aspects of the parent–child relationship, and identify ways in which child mental health impacts the social environment. Our results are most useful when considered in the context of environmental challenges that refugee families face, and their agency in the face of that challenge. Based on the results of our study, family-focused systemic psychosocial support may be a useful route to promoting resilience. However, more longitudinal research is needed to better understand the impact of the refugee environment on children.

Supplementary material

The supplementary material for this article can be found at https://doi.org/10.1017/S2045796022000191.

Data

Researchers interested in accessing data should contact Professor Michael Pluess at Queen Mary University of London, UK (e-mail: ).

Acknowledgements

We warmly thank all participating families for their participation. We also thank Patricia Moghames, Stephanie Legoff, Nicolas Puvis, Zeina Hassan, and all other members of the BIOPATH team for their dedication, hard work and insights. This paper is dedicated to John Fayyad, who sadly passed away during the study.

Financial support

The BIOPATH study was funded by the Eunice Shriver National Institute of Child Health & Human Development (R01HD083387). The funder played no role in study design, in the collection, analysis or interpretation of data, in the writing of the report or the decision to submit the article for publication.

Conflict of interest

None.

Ethical standards

All procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. Ethical approval was granted by the Institutional Review Board of the University of Balamand/Saint George Hospital University Medical Center, Lebanon (ref: IRB/O/024-16/1815). The study was also reviewed by the Lebanese National Consultative Committee on Ethics and approved by the Ministry of Public Health. The sponsor, Queen Mary University of London, reviewed the study for compliance with all relevant legal and regulatory requirements.

References

Asher, SR, Hymel, S and Renshaw, PD (1984) Loneliness in children. Child Development 55, 1456.CrossRefGoogle Scholar
Barber, BK (1996) Parental psychological control: revisiting a neglected construct. Child Development 67, 3296.CrossRefGoogle ScholarPubMed
Barber, BK (1999) Political violence, family relations, and Palestinian youth functioning. Journal of Adolescent Research 14, 206230.CrossRefGoogle Scholar
Barber, BK, Xia, M, Olsen, JA, McNeely, CA and Bose, K (2012) Feeling disrespected by parents: refining the measurement and understanding of psychological control. Journal of Adolescence 35, 273287.CrossRefGoogle ScholarPubMed
Bech, P (2012) Clinical Psychometrics, Chichester, UK: John Wiley and Sons, pp. 7881. doi: 10.1002/9781118511800.CrossRefGoogle Scholar
Benjamini, Y and Hochberg, Y (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological) 57, 289300.Google Scholar
Blackmore, R, Gray, KM, Boyle, JA, Fazel, M, Ranasinha, S, Fitzgerald, G, Misso, M and Gibson-Helm, M (2020) Systematic review and meta-analysis: the prevalence of mental illness in child and adolescent refugees and asylum seekers. Journal of the American Academy of Child and Adolescent Psychiatry 59, 705714.CrossRefGoogle ScholarPubMed
Blevins, CA, Weathers, FW, Davis, MT, Witte, TK and Domino, JL (2015) The Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5): development and initial psychometric evaluation. Journal of Traumatic Stress 28, 489498.CrossRefGoogle ScholarPubMed
Bryant, RA, Edwards, B, Creamer, M, O'Donnell, M, Forbes, D, Felmingham, KL, Silove, D, Steel, Z, Nickerson, A, McFarlane, AC, Van Hooff, M and Hadzi-Pavlovic, D (2018) The effect of post-traumatic stress disorder on refugees’ parenting and their children's mental health: a cohort study. The Lancet Public Health 3, e249e258.CrossRefGoogle ScholarPubMed
Cosco, TD, Kaushal, A, Hardy, R, Richards, M, Kuh, D and Stafford, M (2017) Operationalising resilience in longitudinal studies: a systematic review of methodological approaches. Journal of Epidemiology and Community Health 71, 98104.CrossRefGoogle ScholarPubMed
Ey, S, Hadley, W, Allen, DN, Palmer, S, Klosky, J, Deptula, D, Thomas, J and Cohen, R (2005) A new measure of children's optimism and pessimism: the youth life orientation test. Journal of Child Psychology and Psychiatry and Allied Disciplines 46, 548558.CrossRefGoogle ScholarPubMed
Faulstich, ME, Carey, MP, Ruggiero, L, Enyart, P and Gresham, F (1986) Assessment of depression in childhood and adolescence: an evaluation of the Center for Epidemiological Studies Depression Scale for Children (CES-DC). American Journal of Psychiatry 143, 10241027.Google Scholar
Foa, EB, Johnson, KM, Feeny, NC and Treadwell, KRH (2001) The child PTSD symptom scale: a preliminary examination of its psychometric properties. Journal of Clinical Child & Adolescent Psychology 30, 376384.CrossRefGoogle ScholarPubMed
Goodman, R (1997) The Strengths and Difficulties Questionnaire: a research note. Journal of Child Psychology and Psychiatry, and Allied Disciplines 38, 581586.CrossRefGoogle ScholarPubMed
Harris, MA, Donnellan, MB and Trzesniewski, KH (2018) The lifespan self-esteem scale: initial validation of a new measure of global self-esteem. Journal of Personality Assessment 100, 8495.CrossRefGoogle ScholarPubMed
Henkelmann, J-R, de Best, S, Deckers, C, Jensen, K, Shahab, M, Elzinga, B and Molendijk, M (2020) Anxiety, depression and post-traumatic stress disorder in refugees resettling in high-income countries: systematic review and meta-analysis. BJPsych Open 6(4), E68. doi: 10.1192/BJO.2020.54CrossRefGoogle ScholarPubMed
Henry, JD and Crawford, JR (2005) The short-form version of the depression anxiety stress scales (DASS-21): construct validity and normative data in a large non-clinical sample. British Journal of Clinical Psychology 44, 227239.CrossRefGoogle Scholar
Hermosilla, S, Metzler, J, Savage, K and Ager, A (2021) Resilience and adjustment trajectories amongst children in displacement-affected communities in Zarqa, Jordan. Journal of Global Health Reports 4, e2020107. doi: 10.29392/001c.18233CrossRefGoogle Scholar
Jongedijk, RA, Eising, DD, van der Aa, N, Kleber, RJ and Boelen, PA (2020) Severity profiles of posttraumatic stress, depression, anxiety, and somatization symptoms in treatment seeking traumatized refugees. Journal of Affective Disorders 266, 7181.CrossRefGoogle ScholarPubMed
Jorgensen, TD, Pornprasertmanit, S, Schoemann, AM and Rosseel, Y (2021) semTools: useful tools for structural equation modeling.Google Scholar
Karam, EG, Al-Atrash, R, Saliba, S, Melhem, N and Howard, D (1999) The war events questionnaire. Social Psychiatry and Psychiatric Epidemiology 34, 265274.CrossRefGoogle ScholarPubMed
Kien, C, Sommer, I, Faustmann, A, Gibson, L, Schneider, M, Krczal, E, Jank, R, Klerings, I, Szelag, M, Kerschner, B, Brattström, P and Gartlehner, G (2019) Prevalence of mental disorders in young refugees and asylum seekers in European countries: a systematic review. European Child & Adolescent Psychiatry 28, 12951310.CrossRefGoogle ScholarPubMed
Li, SSY, Liddell, BJ and Nickerson, A (2016) The relationship between post-migration stress and psychological disorders in refugees and asylum seekers. Current Psychiatry Reports 18, 19.CrossRefGoogle ScholarPubMed
Masten, AS (2016) Resilience in developing systems: the promise of integrated approaches. European Journal of Developmental Psychology 13, 297312.CrossRefGoogle Scholar
McEwen, FS, Moghames, P, Bosqui, TJ, Kyrillos, V, Chehade, N, Saad, S, Abdul Rahman, D, Popham, CM, Saab, D, Karam, G, Karam, EG and Pluess, M (2020) Validating screening questionnaires for internalizing and externalizing disorders against clinical interviews in 8–17 year-old Syrian refugee children. Technical Working Paper. Available at https://psyarxiv.com/6zu87/CrossRefGoogle Scholar
McEwen, FS, Biazoli, C, Popham, CM, Moghames, P, Saab, D, Fayyad, J, Bosqui, T, Karam, E and Pluess, M (2022 a) Prevalence and predictors of mental health problems in refugee children living in humanitarian settings. SSRN Electronic Journal. doi: 10.2139/SSRN.4047879.CrossRefGoogle Scholar
McEwen, FS, Popham, CM, Moghames, P, Smeeth, D, De Villiers, B, Saab, D, Karam, G, Fayyad, J, Karam, E and Pluess, M (2022 b) Cohort profile: biological pathways of risk and resilience in Syrian refugee children (BIOPATH). Social Psychiatry and Psychiatric Epidemiology 2022, 111.Google Scholar
Miller-Graff, LE (2020) The multidimensional taxonomy of individual resilience. Trauma, Violence, & Abuse 23, 660675.CrossRefGoogle ScholarPubMed
Müller, LRF, Gossmann, K, Hartmann, F, Büter, KP, Rosner, R and Unterhitzenberger, J (2019) 1-year follow-up of the mental health and stress factors in asylum-seeking children and adolescents resettled in Germany. BMC Public Health 19, 908.CrossRefGoogle ScholarPubMed
O'Driscoll, C, Heary, C, Hennessy, E and McKeague, L (2012) Explicit and implicit stigma towards peers with mental health problems in childhood and adolescence. Journal of Child Psychology and Psychiatry and Allied Disciplines 53, 10541062.CrossRefGoogle ScholarPubMed
Oh, DL, Jerman, P, Purewal Boparai, SK, Koita, K, Briner, S, Bucci, M and Harris, NB (2018) Review of tools for measuring exposure to adversity in children and adolescents. Journal of Pediatric Health Care 32, 564583.CrossRefGoogle ScholarPubMed
Panter-Brick, C, Grimon, MP and Eggerman, M (2014) Caregiver–child mental health: a prospective study in conflict and refugee settings. Journal of Child Psychology and Psychiatry and Allied Disciplines 55, 313327.CrossRefGoogle ScholarPubMed
Pluess, M, Assary, E, Lionetti, F, Lester, KJ, Krapohl, E, Aran, EN and Aron, A (2018) Environmental sensitivity in children: development of the Highly Sensitive Child Scale and identification of sensitivity groups. Developmental Psychology 54, 5170.CrossRefGoogle ScholarPubMed
Popham, CM, McEwen, FS and Pluess, M (2021) Psychological resilience in response to adverse experiences. In Ungar, M (ed.), Multisystemic Resilience. New York: Oxford University Press, pp. 395416.CrossRefGoogle Scholar
Program for Prevention Research (1999) Manual for the Children's Coping Strategies Checklist and the How I Coped Under Pressure Scale. Tempe, AZ: Program for Prevention Research.Google Scholar
Purgato, M, Tedeschi, F, Bonetto, C, de Jong, J, Jordans, MJD, Tol, WA and Barbui, C (2020) Trajectories of psychological symptoms and resilience in conflict-affected children in low- and middle-income countries. Clinical Psychology Review, Elsevier Inc. 82, 101935.CrossRefGoogle ScholarPubMed
Radloff, LS (1977) The CES-D Scale. Applied Psychological Measurement 1, 385401.CrossRefGoogle Scholar
Ramaswamy, V, Aroian, KJ and Templin, T (2009) Adaptation and psychometric evaluation of the multidimensional scale of perceived social support for Arab American adolescents. American Journal of Community Psychology 43, 4956.CrossRefGoogle ScholarPubMed
Rizkalla, N, Mallat, NK, Arafa, R, Adi, S, Soudi, L and Segal, SP (2020) ‘Children are not children anymore; they are a lost generation’: adverse physical and mental health consequences on Syrian refugee children. International Journal of Environmental Research and Public Health 17, 8378.CrossRefGoogle Scholar
Runyan, DK, Dunne, MP and Zolotor, AJ (2009) Introduction to the development of the ISPCAN child abuse screening tools. Child Abuse and Neglect 33, 842845.CrossRefGoogle Scholar
Sampson, RJ, Raudenbush, SW and Earls, F (1997) Neighborhoods and violent crime: a multilevel study of collective efficacy. Science 277, 918924.CrossRefGoogle ScholarPubMed
Sangalang, CC, Jager, J and Harachi, TW (2017) Effects of maternal traumatic distress on family functioning and child mental health: an examination of Southeast Asian refugee families in the U.S. Social Science & Medicine 184, 178186.CrossRefGoogle ScholarPubMed
Schaefer, ES (1965) Children's reports of parental behavior: an inventory. Child Development 36, 413.CrossRefGoogle Scholar
Scharpf, F, Kaltenbach, E, Nickerson, A and Hecker, T (2021) A systematic review of socio-ecological factors contributing to risk and protection of the mental health of refugee children and adolescents. Clinical Psychology Review, Elsevier Inc 83, 101930.CrossRefGoogle ScholarPubMed
Scherer, N, Hameed, S, Acarturk, C, Deniz, G, Sheikhani, A, Volkan, S, Örücü, A, Pivato, I, Akıncı, İ, Patterson, A and Polack, S (2020) Prevalence of common mental disorders among Syrian refugee children and adolescents in Sultanbeyli district, Istanbul: results of a population-based survey. Epidemiology and Psychiatric Sciences 29, E192. doi: 10.1017/S2045796020001079CrossRefGoogle ScholarPubMed
Schwarzer, R and Jerusalem, M (1995) Generalized self-efficacy scale. In Weinman, J, Wright, S and Johnston, M (eds), Measures in Health Psychology: A User's Portfolio. Causal and Control Beliefs. Windsor, UK: NFER-NELSON, pp. 3537.Google Scholar
Sheehan, DV, Sheehan, KH, Shytle, RD, Janavs, J, Bannon, Y, Rogers, JE, Milo, KM, Stock, SL and Wilkinson, B (2010) Reliability and validity of the mini international neuropsychiatric interview for children and adolescents (MINI-KID). Journal of Clinical Psychiatry 71, 313326.CrossRefGoogle Scholar
Topp, CW, Østergaard, SD, Søndergaard, S and Bech, P (2015) The WHO-5 well-being index: a systematic review of the literature. Psychotherapy and Psychosomatics 84, 167176.CrossRefGoogle ScholarPubMed
United Nations High Commissioner for Refugees, (UNHCR) (2020) UNHCR – Global Trends in Forced Displacement – 2020. Available at: https://www.unhcr.org/60b638e37/unhcr-global-trends-2020 (Accessed 18 June 2021).Google Scholar
van Buuren, S and Groothuis-Oudshoorn, K (2011) mice: Multivariate imputation by chained equations in R. Journal of Statistical Software 45, 167.Google Scholar
Ziadni, M, Hammoudeh, W, Abu Rmeileh, NME, Hogan, D, Shannon, H and Giacaman, R (2011) Sources of human insecurity in post-war situations: the case of Gaza. Journal of Human Security 7, 2336.CrossRefGoogle ScholarPubMed
Figure 0

Table 1. Sample characteristics

Figure 1

Fig. 1. Mental health risk change from baseline to follow-up.

Figure 2

Table 2. Analyses of covariance: results from significant models

Figure 3

Fig. 2. Cross-lagged panel models representing key predictors from the individual, family, and community systems.

Figure 4

Table 3. Summary of cross-lagged panel models with significant cross-lagged pathways

Supplementary material: PDF

Popham et al. supplementary material

Popham et al. supplementary material

Download Popham et al. supplementary material(PDF)
PDF 3.5 MB