Hostname: page-component-cd9895bd7-hc48f Total loading time: 0 Render date: 2024-12-26T22:52:58.954Z Has data issue: false hasContentIssue false

Non-linear associations between HPA axis activity during infancy and mental health difficulties during early childhood among children in rural Pakistan

Published online by Cambridge University Press:  12 August 2022

Allison Frost*
Affiliation:
Carolina Population Center, University of North Carolina, Chapel Hill, NC, USA
Ashley Hagaman
Affiliation:
Social Behavioral Sciences, Yale School of Public Health, Yale University, New Haven, CT, USA
Victoria Baranov
Affiliation:
Department of Economics, Faculty of Business and Economics, University of Melbourne, Australia
Esther O. Chung
Affiliation:
Carolina Population Center, University of North Carolina, Chapel Hill, NC, USA Department of Epidemiology, Gillings School of Public Health, University of North Carolina, Chapel Hill, NC, USA
Sonia Bhalotra
Affiliation:
Department of Economics, University of Warwick, UK
Siham Sikander
Affiliation:
Human Development Research Foundation, Islamabad, Pakistan Health Services Academy, Islamabad, Pakistan
Joanna Maselko
Affiliation:
Carolina Population Center, University of North Carolina, Chapel Hill, NC, USA Department of Epidemiology, Gillings School of Public Health, University of North Carolina, Chapel Hill, NC, USA
*
Corresponding author: Allison Frost, email: allisonfrost@unc.edu
Rights & Permissions [Opens in a new window]

Abstract

Hypothalamic pituitary adrenal (HPA) axis activity may be a mechanism linking early adversity to child mental health difficulties. However, there is a dearth of longitudinal evidence for the association between HPA axis activity and mental health among children in low-resource contexts. The goal of this study is to examine linear and curvilinear associations between HPA axis activity during infancy and mental health difficulties in early childhood among children in rural Pakistan. Participants included 104 children (46% male) from the Bachpan study, a longitudinal cohort embedded within a maternal depression trial in Pakistan. We examined the associations between hair-derived cortisol and dehydroepiandosterone (DHEA) at 12 months old and mental health difficulties, measured with the Strengths and Difficulties Questionnaire (SDQ), at 36 months old. There was a significant quadratic association between hair cortisol and SDQ scores, with results showing a U-shaped relationship (i.e., having relatively high or low cortisol predicted increased mental health difficulties). DHEA showed a quadratic association with SDQ scores with an inverted U-shaped relationship (i.e., high and low DHEA was associated with decreased mental health difficulties). Results provide evidence of longitudinal and curvilinear effects of cortisol and DHEA during infancy on mental health difficulties in early childhood.

Type
Regular Article
Copyright
© The Author(s), 2022. Published by Cambridge University Press

Introduction

The hypothalamic pituitary adrenal (HPA) axis is one of the body’s main stress response systems, whose functioning has wide-ranging effects on physical and psychological health across the lifespan (Adam et al., Reference Adam, Quinn, Tavernier, McQuillan, Dahlke and Gilbert2017). Studies have demonstrated that HPA axis hormones, including cortisol and dehydroepiandosterone (DHEA), are sensitive to environmental influences, with experiences of early adversity (e.g., poverty, violence exposure) predicting altered HPA axis activity, including both hyperactivity and hypoactivity, in children (Kamin & Kertes, Reference Kamin and Kertes2017; Shakiba et al., Reference Shakiba, Ellis, Bush and Boyce2020). Functioning of the HPA axis may be a mechanism linking early stressful experiences to child mental health outcomes. However, more research is needed to establish the association between HPA axis activity and child mental health early in life, when mental health difficulties first emerge.

HPA axis dysregulation, especially early in life, may impact developing brain systems through altering neurogenesis and connectivity among key regions associated with emotion responsivity, executive function, and memory (McKlveen et al., Reference McKlveen, Moloney, Scheimann, Myers and Herman2019; Sapolsky, Reference Sapolsky2003). Such alterations may longitudinally affect child developmental trajectories in multiple domains, including mental health (McKlveen et al., Reference McKlveen, Moloney, Scheimann, Myers and Herman2019). Indeed, longitudinal research demonstrates an association between HPA axis activity and mental health difficulties; however, there is inconsistency in whether it is lower or higher levels of cortisol and DHEA that are associated with the negative outcomes (Malisiova et al., Reference Malisiova, Mourikis, Darviri, Zervas, Papageorgiou and Chrousos2020; Psarraki et al., Reference Psarraki, Kokka, Bacopoulou, Chrousos, Artemiadis and Darviri2020; Staufenbiel et al., Reference Staufenbiel, Penninx, Spijker, Elzinga and van Rossum2013; Wesarg et al., Reference Wesarg, Van Den Akker, Oei, Hoeve and Wiers2020).

Some studies show that high levels of cortisol prospectively predict elevated mental health difficulties, including anxiety and depression, during childhood (Sandstrom et al., Reference Sandstrom, Daoust, Russell, Koren and Hayden2020; Saridjan et al., Reference Saridjan, Velders, Jaddoe, Hofman, Verhulst and Tiemeier2014; Smider et al., Reference Smider, Essex, Kalin, Buss, Klein, Davidson and Goldsmith2002). Other longitudinal work suggests that low levels of cortisol longitudinally predict increased mental health symptoms in children (Laurent et al., Reference Laurent, Gilliam, Wright and Fisher2015; Pauli-Pott et al., Reference Pauli-Pott, Schloss, Skoluda, Nater and Becker2019; Salis et al., Reference Salis, Bernard, Black, Dougherty and Klein2016). Similarly, although the literature examining DHEA and mental health is smaller, there is evidence that both high (Cicchetti & Rogosch, Reference Cicchetti and Rogosch2007a; Kimonis et al., Reference Kimonis, Fleming, Wilbur, Groer and Granger2019; Mulligan et al., Reference Mulligan, Hajcak, Crisler and Meyer2020) and low (Chen et al., Reference Chen, Raine and Granger2015; Kamin & Kertes, Reference Kamin and Kertes2017; Michael et al., Reference Michael, Jenaway, Paykel and Herbert2000) DHEA are cross-sectionally associated with poor child mental health.

The inconsistencies in this literature illustrate the complexity of the relationship between HPA axis activity and child functioning. One source of complexity may be nonlinear associations between HPA axis indicators and child development. However, very few studies have examined the possibility of a curvilinear relationship between cortisol or DHEA and child outcomes. Research examining cortisol and depression has found a U-shaped relationship between cortisol levels and depressive symptoms in adolescents and older adults, such that both high and low levels of cortisol predicted elevated depressive symptoms (Bremmer et al., Reference Bremmer, Deeg, Beekman, Penninx, Lips and Hoogendijk2007; Ford et al., Reference Ford, Boch and Browning2019). This suggests that HPA axis dysregulation in either direction (i.e., high or low hormone levels) may place individuals at higher risk for developing mental health difficulties. However, more evidence is needed across age groups, sources of cortisol, and contexts to better understand the linear and curvilinear associations between HPA axis activity and mental health.

There is a paucity of research examining HPA axis activity and child mental health difficulties in low-resource settings, such as low- and middle-income countries (LMICs). Children in these settings are likely to be exposed to a myriad of chronic stressors, such as poverty, disease exposure, and food insecurity, that can alter both their stress system functioning and their susceptibility to mental health difficulties. In addition, elucidating the association between HPA axis activity and child mental health outcomes in LMICs can further our understanding of the basic processes linking stress system functioning and child development across diverse contexts. Studies in these settings have demonstrated a link between HPA axis activity and mental health in adolescents both cross-sectionally (Nicolson & Ponnamperuma, Reference Nicolson and Ponnamperuma2019; Schindler et al., Reference Schindler, Shaheen, Saar-Ashkenazy, Bani Odeh, Sass, Friedman and Kirschbaum2019; Shaheen et al., Reference Shaheen, Schindler, Saar-Ashkenazy, Bani Odeh, Soreq, Friedman and Kirschbaum2020; Zimmerman et al., Reference Zimmerman, Halligan, Skeen, Morgan, Fraser, Fearon and Tomlinson2020) and longitudinally (Dajani et al., Reference Dajani, Hadfield, van Uum, Greff and Panter-Brick2018; Panter-Brick et al., Reference Panter-Brick, Wiley, Sancilio, Dajani and Hadfield2020) However, few studies in low or high-resource settings have examined these longitudinal associations from infancy to early childhood (Frost et al., Reference Frost, Jelinek, Bernard, Lind and Dozier2017; Saridjan et al., Reference Saridjan, Velders, Jaddoe, Hofman, Verhulst and Tiemeier2014). Given that infancy is a period of high neuroplasticity, and early childhood is when mental health difficulties may first become recognizable, this early period may offer a promising opportunity for studying the influence of stress physiology on child socioemotional development.

In the current study, we examine the longitudinal associations between hair-derived HPA axis markers at 12 months old and mental health difficulties at 36 months old in a sample of children in rural Pakistan. In an effort to clarify the disparate findings in previous studies, we examine both linear and curvilinear relationships between HPA axis indicators and child outcomes.

Method

Study design and sample

Data for these analyses were drawn from the Bachpan study, a birth cohort with an embedded cluster-randomized control trial of a maternal depression intervention in rural Pakistan. Details of the trial are available elsewhere (Atif et al., Reference Atif, Bibi, Nisar, Zulfiqar, Ahmed, LeMasters and Rahman2019; Maselko et al., Reference Maselko, Sikander, Turner, Bates, Ahmad, Atif and Bibi2020; Turner et al., Reference Turner, Sikander, Bangash, Zaidi, Bates, Gallis and Maselko2016). Briefly, women in their third trimester of pregnancy were recruited from 40 village clusters in the Kallar Syedan region of Pakistan. Women were screened for depression using the Patient Health Questionnaire-9 (PHQ-9). Those who scored above the cutoff of 10 were eligible for enrollment in the trial. Village clusters were randomly assigned to receive the intervention or enhanced usual care (control). In addition, at baseline, a sample of women who were not depressed (i.e., scored below a 10 on the PHQ-9) were recruited within each village cluster to create a population-representative sample (Sikander et al., Reference Sikander, Ahmad, Bates, Gallis, Hagaman, O’Donnell, … and Maselko2019). Sample sizes and a study timeline are presented in Figure 1. When infants were 12 months old, a randomly selected subsample of 177 (34 in the intervention group, 36 in the control group, and 34 in the nondepressed group) mother–child dyads were approached for hair data collection (out of a sample of 889 families interviewed). The subsample was selected to be balanced across treatment groups. Of the 177 approached for hair sample collection, 158 families consented for an acceptance rate of 89%. This acceptance rate aligns with the larger sample. Fifty-four infants did not have enough hair to sample. The final data set included hair samples from 104 infants.

Figure 1. Timeline and sample size for hair subsample.

Measures

Hair-derived cortisol and DHEA

Approximately 200 strands of hair were collected from infants using standard collection procedures (Russell et al., Reference Russell, Koren, Rieder and Uum2012). Infants provided 1–3 cm of hair. A single cm of hair approximates one month of hormone secretion, meaning the cortisol and DHEA values in this sample represent the average hormone secretion from the previous 1–3 months (Kirschbaum et al., Reference Kirschbaum, Tietze, Skoluda and Dettenborn2009). Cortisol and DHEA were measured using liquid chromatography mass spectrometry (LC–MS) by Dresden LabService (Gao et al., Reference Gao, Stalder, Foley, Rauh, Deng and Kirschbaum2013; Kirschbaum et al., Reference Kirschbaum, Tietze, Skoluda and Dettenborn2009). This method involves simultaneously extracting steroid hormones (including cortisol and DHEA) from whole (i.e., nonpulverized) hair samples by methanol incubation (for more detail, see Gao et al., Reference Gao, Stalder, Foley, Rauh, Deng and Kirschbaum2013). Hair samples were analyzed in a single batch. Five DHEA values were below the detectable limit and were winsorized to the minimum value. One DHEA value was identified as an outlier (i.e., over three SDs above the mean) and was winsorized to three standard deviations above the mean. There were no cortisol outliers.

Mental health difficulties

Mothers reported on their children’s mental health difficulties at 36 months using the Strengths and Difficulties Questionnaire (SDQ) (Finch et al., Reference Finch, Yousafzai, Rasheed and Obradović2018; Goodman, Reference Goodman2001). The SDQ includes 25 items measuring the frequency of symptoms in the previous six months on a scale from 0 (not true) to 2 (certainly true). The SDQ provides an overall Total Difficulties score (SDQ-TD; 20 items, α = 0.78), as well as five subscales: Emotional Problems (five items, α = 0.61), Conduct Problems (five items, α = 0.73), Hyperactivity Problems (five items, α = 0.50), Peer Problems (five items, α = 0.20), and Prosocial Behavior (five items, α = 0.40). The SDQ-TD is the sum of all the scales except for Prosocial Behavior. On the Emotional, Conduct, Hyperactivity, and Peer Problems subscales, higher scores indicate increased symptoms. On the Prosocial Behavior scale, higher scores indicate increased positive social behavior. The SDQ has been translated and validated in Urdu and has been used extensively in LMICs (Samad et al., Reference Samad, Hollis, Prince and Goodman2005).

Covariates

Covariates included trial arm (depressed intervention, depressed control, and nondepressed), child sex, child age in days at the time of hair sampling, and family socioeconomic status at baseline. Socioeconomic status was assessed using an index of household assets (Rustein & Johnson, Reference Rustein and Johnson2004). This index included items such as land/home ownership, ownership of various household items (e.g., television), type of materials used to build the home, and access to water and sanitation. A polychoric principal components approach was applied to the asset items (Kolenikov & Angeles, Reference Kolenikov and Angeles2004), and the first principal component derived from this analysis was used as the SES asset index. For more detail on this approach, see Maselko et al., 2018 and Kolenikov & Angeles, Reference Kolenikov and Angeles2004. Participants were divided into SES levels based on quintiles within the sample.

Statistical analysis

Data were analyzed using Stata 16. Hair cortisol was log-transformed due to positive skew. DHEA did not show a skewed distribution, so this was untransformed for analysis. In addition, cortisol and DHEA were standardized prior to analysis. First, preliminary analyses were conducted to test differences between the HPA axis subsample and the full study sample on study variables, including demographic characteristics and SDQ scores. Through bivariate correlations and ANOVAs, we also tested differences in cortisol and DHEA by sociodemographic characteristics (i.e., child age, child sex, and family SES) and trial arm. In main analyses, we used generalized estimating equations (GEE) with cluster-robust standard errors to assess the relations between HPA axis markers at 12 months and mental health difficulties at 36 months (Wang, Reference Wang2014). We ran models with HPA axis hormones predicting the SDQ Total Difficulties scale. In additional analyses, we tested the associations between HPA axis hormones and each subscale of the SDQ. For each outcome, linear, quadratic, and cubic models were tested. Akaike Information Criterion (AIC) and Bayseian Information Criterion (BIC) were used to assess each model’s fit. The best-fitting model (i.e., the model with the lowest AIC and BIC values) for each outcome is presented in the results section. In addition, model fits for all models are presented in Supplemental Table 2.

Results

Sample characteristics are presented in Table 1. In addition, Supplemental Table 1 presents a comparison between the HPA subsample and the full sample on study variables. The HPA subsample was similar to the full sample with the exception of having higher scores on the Peer Problems subscale of the SDQ (t = −2.61, p < 0.01) compared to the full sample. Bivariate correlations for key variables of interest are presented in Table 2. DHEA was positively associated with child age. ANOVAs showed there were no differences in cortisol or DHEA by child sex or SES quintile. There were no differences in cortisol by study arm, but there was a significant difference in DHEA across groups (F = 7.26, p < 0.05). Those in the nondepressed (M = 13.52) and depressed intervention (M = 12.51) groups showed higher DHEA than those in the depressed control group (M = 8.62).

Table 1. Sample characteristics

Notes: SDQ = Strengths and Difficulties Questionnaire. Cortisol values are log-transformed.

Table 2. Bivariate correlations between study variables

p < 0.10 *p < 0.05, **p < 0.01, ***p < 0.001.

SDQ = Strengths and Difficulties Questionnaire. Cortisol values are log-transformed.

We first tested the linear and curvilinear associations between hair cortisol and DHEA with the SDQ Total Difficulties (SDQ-TD) score. Results are presented in Figure 2, and estimates are described in Supplemental Tables 3 and 4. There was a statistically significant quadratic effect of hair cortisol on the SDQ-TD score (B = 1.31, 95% CI: 0.67, 1.95), such that the SDQ-TD score is elevated when cortisol is relatively low and high, but decreased when cortisol is near the mean (a U-shaped association). The quadratic model showed a better fit according to the AIC and BIC compared to the linear model (see Supplemental Table 2). There was also a statistically significant quadratic effect of DHEA on the SDQ-TD score (B = −0.73, 95% CI: −1.30, −0.15). This effect showed that the SDQ-TD was lower when DHEA was relatively high and relatively low, and elevated when DHEA was near the mean (an inverse U-shaped association). The quadratic model showed a better fit than the linear model when assessed with the AIC. When assessed with the BIC, the linear model showed a better fit than the quadratic model.

Figure 2. Quadratic fit plots of the relation between the SDQ total difficulties scale and hair cortisol (Panel A) and DHEA (Panel B).

To gain further insights into the associations between cortisol, DHEA and specific types of mental health difficulties, we tested the associations of hair cortisol and DHEA with subscales of the SDQ. Figure 3 and Supplemental Table 3 show the associations between hair cortisol and each SDQ subscale. There were statistically significant quadratic effects of cortisol on Emotional Problems (B = 0.49 95% CI: 0.14, 0.84) and Conduct Problems (B = 0.47, 95% CI: 0.23, 0.71), indicating a U-shaped association (i.e., emotional and conduct symptoms are elevated when cortisol is relatively low and relatively high, but decreased when cortisol is around the mean). For these outcomes, the quadratic models showed a better fit than the linear models on both the AIC and BIC. There were no statistically significant associations between hair cortisol and any other SDQ subscale. Figure 4 and Supplemental Table 4 show the associations between DHEA and each SDQ subscale. There was a statistically significant quadratic effect of DHEA on Emotional Problems (B = −0.30, 95% CI: −0.56, −0.04) showing an inverted U-shaped association. There was a quadratic effect of DHEA on Conduct problems of a similar magnitude, but this was not statistically significant (B = −0.33, CI: −0.66, 0.00) Emotional and conduct symptoms were lower when DHEA was relatively high or low, and higher when DHEA was near the mean. As with the SDQ-TD, the quadratic models for both the emotional and conduct scales showed better fits than the linear models when assessed with the AIC, but not the BIC. In addition, DHEA showed a significant positive linear effect on the Prosocial Behavior scale (B = 0.33, 95% CI: 0.08, 0.58), suggesting that increased DHEA at 12 months is associated with more prosocial behaviors at 36 months. There were no other statistically significant associations between DHEA and the SDQ subscales.

Figure 3. Quadratic fit plots of the relation between cortisol and the SDQ emotional problems (Panel A) and conduct problems (Panel B) scales. Linear fit plots of hyperactivity problems (Panel C), peer problems (Panel D), and prosocial behavior (Panel E) scales.

Figure 4. Quadratic fit plots of the relation between cortisol and the SDQ emotional problems (Panel A) and conduct problems (Panel B) scales. Linear fit plots of hyperactivity problems (Panel C), peer problems (Panel D), and prosocial behavior (Panel E) scales.

Discussion

The goal of this study was to examine longitudinal relations between hair-derived HPA axis biomarkers at 12 months of age and mental health difficulties at 36 months of age among children living in rural Pakistan. Results showed that cortisol has a curvilinear relationship with mental health difficulties, as measured by the Total Difficulties, Emotional Problems, and Conduct Problems scales on the SDQ. These findings suggest that children show elevated mental health difficulties when cortisol is both low and high, but not when cortisol is near the log-transformed mean. Results also suggested a curvilinear association between DHEA and Total Difficulties, Emotional Problems, and Conduct Problems in the opposing direction, such that children showed decreased SDQ symptoms when DHEA was both low and high and elevated SDQ symptoms when DHEA was near the mean. These findings may help explain discrepancies in the literature relating cortisol and DHEA to psychopathology, which has shown effects of both high and low hormones on mental health difficulties (Malisiova et al., Reference Malisiova, Mourikis, Darviri, Zervas, Papageorgiou and Chrousos2020).

Early alterations in HPA axis activity may impact child psychological functioning through a variety of mechanisms. First, HPA axis dysregulation may interfere with optimal brain development early in life. Evidence suggests that excessive cortisol is associated with suppressed neurogenesis and dendritic atrophy in the limbic system (Sapolsky, Reference Sapolsky2003). In addition, animal models demonstrate that early stressful experiences predict heightened cortisol, which is then associated with reduced connectivity between limbic structures (e.g., the amygdala, the hippocampus) (Liu et al., Reference Liu, Diorio, Day, Francis and Meaney2000). These alterations in the limbic system may have protracted effects on children’s self-regulatory capacities as they age, eventually contributing to emotional and behavioral difficulties (Blair, Reference Blair2010). The effects of HPA axis activity on brain development may be especially important during infancy, when neural plasticity is high (Gee & Casey, Reference Gee and Casey2015).

HPA axis activity early in life may also contribute to behavioral patterns that place children at risk for psychological concerns over time. Low cortisol activity is associated with blunted physiological and behavioral reactivity to acute stress. Blunted stress reactivity has been associated with risk-taking behavior and externalizing problems in children, possibly as a result of being less sensitive to dangers in the environment (Ellis et al., Reference Ellis, Oldehinkel and Nederhof2017; Ouellet-Morin et al., Reference Ouellet-Morin, Odgers, Danese, Bowes, Shakoor, Papadopoulos and Arseneault2011). This may explain the U-shaped pattern in the association between cortisol and later mental health difficulties, with children who show blunted (or low) cortisol also showing increased emotional and conduct problems.

HPA axis activity may interact with environmental conditions to predict child adjustment. The biological sensitivity to context theory posits that high or low HPA axis activity alone does not necessarily predict child outcomes, but the combination of stress system functioning and environmental factors (e.g., the safety of the child’s environment, the extent to which the child feels supported, the predictability of the environment, parenting practices) may be more important (Boyce & Ellis, Reference Boyce and Ellis2005). For instance, children who show high HPA activity early in life may be more sensitive to stressful experiences in childhood, and therefore may show more behavioral and emotional problems if they are raised in a stressful environment. Those who are low in HPA activity may be less responsive to stressful experiences and show similar levels of psychological functioning regardless of their environment. This would suggest that future studies characterizing physiological predictors of child socioemotional development would benefit from including indicators of environmental stress.

Furthermore, the timing, chronicity, and number of stressors experienced by children may impact the association between HPA axis activity and mental health. Research on trauma and posttraumatic stress disorder (PTSD) has demonstrated that trauma exposure is associated with an initial increase in cortisol, followed by attenuated cortisol over time. Work has also shown that attenuated cortisol prior to trauma exposure may lead to increased vulnerability to PTSD. This physiological mechanism may explain the dose–response relationship observed between number of traumatic events and severity of PTSD symptoms (Steudte-Schmiedgen et al., Reference Steudte-Schmiedgen, Kirschbaum, Alexander and Stalder2016). In this sample, the occurrence of traumatic events may inform both participants’ HPA axis activity and their susceptibility to mental health difficulties. However, this was outside the scope of the current analyses.

The finding that hair cortisol during infancy has a U-shaped association with psychological symptoms in early childhood suggests that capturing both types of cortisol dysregulation (i.e., both high and low cortisol values) may be valuable in identifying children most at risk for developing psychological symptoms longitudinally. The finding that DHEA has an inverted U-shaped association with psychological symptoms suggests that those with both high and low DHEA levels in infancy may be at lower risk for developing mental health concerns in early childhood. However, there were conflicts in model fit statistics based on the AIC and BIC for the quadratic DHEA model, suggesting that more research is needed to draw conclusions around curvilinear associations between DHEA and mental health. These findings provide a broader view of how physiological stress system functioning may relate to behavior by moving beyond linear associations. Given the previous work showing curvilinear associations between early adversity and HPA axis hormones (Bush et al., Reference Bush, Obradović, Adler and Boyce2011; Shakiba et al., Reference Shakiba, Ellis, Bush and Boyce2020), considering the pathways between early stress exposure and child socioemotional development in terms of nonlinear relationships may allow researchers to better capture the nuances in these associations.

We found that hair-derived DHEA at 12 months predicted increased parent-reported pro-social behavior at 36 months. Some work suggests that DHEA supports neurodevelopment early in life, including neurogenesis and plasticity in response to challenge (Greaves et al., Reference Greaves, Wudy, Badoer, Zacharin, Hirst, Quinn and Walker2019; Maninger et al., Reference Maninger, Wolkowitz, Reus, Epel and Mellon2009). It may be that high levels of DHEA in infancy support optimal development in the brain areas underlying self-regulation, which then supports prosocial behavior in early childhood. However, given the low reliability of the prosocial scale in this sample, this finding should be interpreted with caution.

It is notable that DHEA and cortisol showed opposing effects on mental health difficulties, and that only DHEA showed statistically significant effects on prosocial behavior. It may be that DHEA and cortisol influence child development through independent pathways. Past research has shown that DHEA and cortisol have opposing effects on neural development, with excessive cortisol exerting neurotoxic effects and DHEA offsetting the impact of cortisol (Maninger et al., Reference Maninger, Wolkowitz, Reus, Epel and Mellon2009). Research also suggests that cortisol and DHEA have different mechanisms of action in influencing neurotransmitters, different daily secretion patterns, and different regulatory processes (for instance, cortisol is regulated by a negative feedback loop within the HPA axis, but DHEA is not) (Kamin & Kertes, Reference Kamin and Kertes2017). Disentangling the independent effects of cortisol and DHEA may be a promising next step in understanding the complex associations between HPA axis regulation and child development.

This study benefitted from a number of strengths, including a longitudinal design and multiple markers of HPA axis activity (i.e., cortisol and DHEA). In addition, our sample in rural Pakistan allows us to better understand HPA axis biomarkers and child development in a low-resource setting. Moreover, the age of our sample meant was able to capture HPA axis activity and child mental health difficulties during important periods (infancy and early childhood) for both neurodevelopment and psychological development.

There are also several limitations to this study. First, there is evidence to suggest that our measure of child mental health difficulties, the SDQ, has limited clinical sensitivity and specificity in non-Western, low resource contexts (Maldonado et al., Reference Maldonado, Chandna and Gladstone2019). Reliability estimates for the SDQ subscales were mixed in this sample, with some scales (i.e., Peer Problem, Prosocial Behavior) showing poor reliability. This suggests that the items on the SDQ subscales show limited internal consistency with one another, and therefore may not be measuring distinct underlying constructs. Our results pertaining to the SDQ subscales should be interpreted with caution. Findings may reflect associations between HPA axis activity and general psychopathology, rather than specific domains of psychological symptoms as measured by the SDQ subscales. Future studies may benefit from incorporating a variety of adapted child mental health measures that have been validated for this setting, and using multi-informant and/or observational methods to better understand child psychological functioning. In addition, our sample size was relatively small, which limited our ability to examine moderators. Studies have shown that a multitude of factors, including environmental context (e.g., family income, stress exposure), child sex, and child age may moderate the effects of cortisol and DHEA on child development (Armstrong-Carter et al., Reference Armstrong-Carter, Finch, Siyal, Yousafzai and Obradović2020; Cicchetti & Rogosch, Reference Cicchetti and Rogosch2007b; Fuchs et al., Reference Fuchs, Jaite, Neukel, Dittrich, Bertsch, Kluczniok and Bödeker2018; Kimonis et al., Reference Kimonis, Fleming, Wilbur, Groer and Granger2019; Ouellet-Morin et al., Reference Ouellet-Morin, Cantave, Paquin, Geoffroy, Brendgen, Vitaro and Côté2021). We also did not include information on traumatic events (e.g., abuse, violence exposure) that may inform our results. HPA axis activity likely intersects with multiple environmental, biological, and behavioral systems to influence child development. Furthermore, we relied on parent self-report for child mental health difficulties, which may be subject to bias. Finally, it should be noted that two years passed between when cortisol and DHEA were sampled (12 months) and when outcomes were measured (36 months). It is possible children were exposed to other factors, including stressful experiences, that may have impacted their socioemotional development during this time.

This study demonstrated longitudinal and curvilinear associations between hair-derived HPA axis hormones in infancy and mental health difficulties in early childhood. These results point to cortisol and DHEA as potential markers of risk and resilience among young children living in rural Pakistan. Future research can expand on this work to understand the mechanisms by which early experiences are physiologically embedded and how such alterations in physiological systems may contribute to child developmental trajectories.

Supplementary material

The supplementary material for this article can be found at https://doi.org/10.1017/S0954579422000773.

Acknowledgments

We would like to thank the team at the Human Development Research Foundation (HDRF), including Rakshanda Liaqat, Tayyiba Abbasi, Maria Sharif, Samina Bilal, Anum Nisar, Amina Bibi, Shaffaq Zufiqar, Ahmed Zaidi, Ikhlaq Ahmad, and Najia Atif. We are also grateful for the assistance and inputs of Prof. Fareed Aslam Minhas from the Institute of Psychiatry, Rawalpindi Medical University. Finally, we are very grateful to the families who participated in the study.

Funding sources

This work was supported by the University of Melbourne, ARC Centre of Excellence – The Life Course Centre, MiSoC ISER at University of Essex, NIH R01HD075875, NIMH U19MH95687, NICHD R03HD097434, and NICHD T32HD007168.

Conflicts of interest

None.

References

Adam, E. K., Quinn, M. E., Tavernier, R., McQuillan, M. T., Dahlke, K. A., & Gilbert, K. E. (2017). Diurnal cortisol slopes and mental and physical health outcomes: A systematic review and meta-analysis. Psychoneuroendocrinology, 83, 2541. https://doi.org/10.1016/j.psyneuen.2017.05.018 CrossRefGoogle ScholarPubMed
Armstrong-Carter, E., Finch, J. E., Siyal, S., Yousafzai, A. K., & Obradović, J. (2020). Biological sensitivity to context in pakistani preschoolers: Hair cortisol and family wealth are interactively associated with girls’ cognitive skills. Developmental Psychobiology, 62(8), 10461061. https://doi.org/10.1002/dev.21981.CrossRefGoogle ScholarPubMed
Atif, N., Bibi, A., Nisar, A., Zulfiqar, S., Ahmed, I., LeMasters, K., & Rahman, A. (2019). Delivering maternal mental health through peer volunteers: A 5-year report. International Journal of Mental Health Systems, 13(1), 18. https://doi.org/10.1186/s13033-019-0318-3 CrossRefGoogle ScholarPubMed
Blair, C. (2010). Stress and the development of self-regulation in context. Child Development Perspectives, 4, 181188. https://doi.org/10.1111/j.1750-8606.2010.00145.x CrossRefGoogle ScholarPubMed
Boyce, W. T., & Ellis, B. J. (2005). Biological sensitivity to context: I. An evolutionary-developmental theory of the origins and functions of stress reactivity. Development and Psychopathology, 17, 271301. https://doi.org/10.1017/S0954579405050145 CrossRefGoogle ScholarPubMed
Bremmer, M. A., Deeg, D. J., Beekman, A. T., Penninx, B. W., Lips, P., & Hoogendijk, W. J. (2007). Major depression in late life is associated with both hypo-and hypercortisolemia. Biological Psychiatry, 62(5), 479486. https://doi.org/10.1016/j.biopsych.2006.11.033 CrossRefGoogle ScholarPubMed
Bush, N. R., Obradović, J., Adler, N., & Boyce, W. T. (2011). Kindergarten stressors and cumulative adrenocortical activation: The, first straws, of allostatic load? Development and Psychopathology, 23(4), 10891106. https://doi.org/10.1017/S0954579411000514 CrossRefGoogle ScholarPubMed
Chen, F. R., Raine, A., & Granger, D. A. (2015). Tactics for modeling multiple salivary analyte data in relation to behavior problems: Additive, ratio, and interaction effects. Psychoneuroendocrinology, 51, 188200. https://doi.org/10.1016/j.psyneuen.2014.09.027 CrossRefGoogle ScholarPubMed
Cicchetti, D., & Rogosch, F. A. (2007a). Personality, adrenal steroid hormones, and resilience in maltreated children: A multilevel perspective. Development and Psychopathology, 19(03), 787809. https://doi.org/10.1017/S0954579407000399 CrossRefGoogle ScholarPubMed
Cicchetti, D., & Rogosch, F. A. (2007b). Personality, adrenal steroid hormones, and resilience in maltreated children: A multilevel perspective. Development and Psychopathology, 19, 787809. https://doi.org/10.1017/S0954579407000399 CrossRefGoogle ScholarPubMed
Dajani, R., Hadfield, K., van Uum, S., Greff, M., & Panter-Brick, C. (2018). Hair cortisol concentrations in war-affected adolescents: A prospective intervention trial. Psychoneuroendocrinology, 89, 138146. https://doi.org/10.1016/j.psyneuen.2017.12.012 CrossRefGoogle ScholarPubMed
Ellis, B. J., Oldehinkel, A. J., & Nederhof, E. (2017). The adaptive calibration model of stress responsivity: An empirical test in the Tracking Adolescents’ Individual Lives Survey study. Development and Psychopathology, 29(3), 10011021. https://doi.org/10.1017/S0954579416000985 CrossRefGoogle ScholarPubMed
Finch, J. E., Yousafzai, A. K., Rasheed, M., & Obradović, J. (2018). Measuring and understanding social-emotional behaviors in preschoolers from rural Pakistan. PLOS ONE, 13(11), e0207807. https://doi.org/10.1371/journal.pone.0207807 CrossRefGoogle ScholarPubMed
Ford, J. L., Boch, S. J., & Browning, C. R. (2019). Hair cortisol and depressive symptoms in youth: An investigation of curvilinear relationships. Psychoneuroendocrinology, 109, 104376. https://doi.org/10.1016/j.psyneuen.2019.104376 CrossRefGoogle ScholarPubMed
Frost, A., Jelinek, C., Bernard, K., Lind, T., & Dozier, M. (2017). Longitudinal associations between low morning cortisol in infancy and anger dysregulation in early childhood in a CPS-referred sample. Developmental Science, e12573e12573. https://doi.org/10.1111/desc.12573 Google Scholar
Fuchs, A., Jaite, C., Neukel, C., Dittrich, K., Bertsch, K., Kluczniok, D., & Bödeker, K. (2018). Link between children’s hair cortisol and psychopathology or quality of life moderated by childhood adversity risk. Psychoneuroendocrinology, 90, 5260. https://doi.org/10.1016/j.psyneuen.2018.02.003 CrossRefGoogle ScholarPubMed
Gao, W., Stalder, T., Foley, P., Rauh, M., Deng, H., & Kirschbaum, C. (2013). Quantitative analysis of steroid hormones in human hair using a column-switching LC-APCI-MS/MS assay. Journal of Chromatography B Analytical Technologies in the Biomedical and Life Sciences, 928, 18. https://doi.org/10.1016/j.jchromb.2013.03.008 CrossRefGoogle ScholarPubMed
Gee, D. G., & Casey, B. J. (2015). The impact of developmental timing for stress and recovery. Neurobiology of Stress, 1, 184194. https://doi.org/10.1016/j.ynstr.2015.02.001 CrossRefGoogle ScholarPubMed
Goodman, R. (2001). Psychometric properties of the strengths and difficulties questionnaire. Journal of the American Academy of Child & Adolescent Psychiatry, 40(11), 13371345. https://doi.org/10.1097/00004583-200111000-00015 CrossRefGoogle ScholarPubMed
Greaves, R. F., Wudy, S. A., Badoer, E., Zacharin, M., Hirst, J. J., Quinn, T., & Walker, D. W. (2019). A tale of two steroids: The importance of the androgens DHEA and DHEAS for early neurodevelopment. The Journal of Steroid Biochemistry and Molecular Biology, 188, 7785. https://doi.org/10.1016/j.jsbmb.2018.12.007 CrossRefGoogle ScholarPubMed
Kamin, H. S., & Kertes, D. A. (2017). Cortisol and DHEA in development and psychopathology. Hormones and Behavior, 89, 6985. https://doi.org/10.1016/j.yhbeh.2016.11.018 CrossRefGoogle ScholarPubMed
Kimonis, E. R., Fleming, G. E., Wilbur, R. R., Groer, M. W., & Granger, D. A. (2019). Dehydroepiandrosterone (DHEA) and its ratio to cortisol moderate associations between maltreatment and psychopathology in male juvenile offenders. Psychoneuroendocrinology, 101, 263271. https://doi.org/10.1016/j.psyneuen.2018.12.228 CrossRefGoogle ScholarPubMed
Kirschbaum, C., Tietze, A., Skoluda, N., & Dettenborn, L. (2009). Hair as a retrospective calendar of cortisol production—increased cortisol incorporation into hair in the third trimester of pregnancy. Psychoneuroendocrinology, 34(1), 3237. https://doi.org/10.1016/j.psyneuen.2008.08.024 CrossRefGoogle ScholarPubMed
Kolenikov, S., & Angeles, G. (2004). The use of discrete data in PCA: Theory, simulations, and applications to socioeconomic indices, vol. 20, 159). Chapel Hill: Carolina Population Center, University of North Carolina.Google Scholar
Laurent, H. K., Gilliam, K. S., Wright, D. B., & Fisher, P. A. (2015). Child anxiety symptoms related to longitudinal cortisol trajectories and acute stress responses: Evidence of developmental stress sensitization. Journal of Abnormal Psychology, 124(1), 68. https://doi.org/10.1037/abn0000009 CrossRefGoogle ScholarPubMed
Liu, D., Diorio, J., Day, J. C., Francis, D. D., & Meaney, M. J. (2000). Maternal care, hippocampal synaptogenesis and cognitive development in rats. Nature Neuroscience, 3(8), 799806. https://doi.org/10.1038/77702 CrossRefGoogle ScholarPubMed
Maldonado, B. N., Chandna, J., & Gladstone, M. (2019). A systematic review of tools used to screen and assess for externalising behaviour symptoms in low and middle income settings. Global Mental Health, 6, https://doi.org/10.1017/gmh.2019.11 Google Scholar
Malisiova, E. K., Mourikis, I., Darviri, C., Zervas, I. M., Papageorgiou, C., & Chrousos, G. P. (2020). Hair cortisol concentrations in mental disorders: A systematic review. Physiology & Behavior, 113244, https://doi.org/10.1016/j.physbeh.2020.113244 Google Scholar
Maninger, N., Wolkowitz, O. M., Reus, V. I., Epel, E. S., & Mellon, S. H. (2009). Neurobiological and neuropsychiatric effects of dehydroepiandrosterone (DHEA) and DHEA sulfate (DHEAS). Frontiers in Neuroendocrinology, 30(1), 6591. https://doi.org/10.1016/j.yfrne.2008.11.002 CrossRefGoogle ScholarPubMed
Maselko, J., Sikander, S., Turner, E. L., Bates, L. M., Ahmad, I., Atif, N., & Bibi, T. (2020). Effectiveness of a peer-delivered, psychosocial intervention on maternal depression and child development at 3 years postnatal: A cluster randomised trial in Pakistan. The Lancet Psychiatry, 7(9), 775787. https://doi.org/10.1016/S2215-0366(20)30258-3 CrossRefGoogle ScholarPubMed
McKlveen, J. M., Moloney, R. D., Scheimann, J. R., Myers, B., & Herman, J. P. (2019). Braking, the prefrontal cortex: The role of glucocorticoids and interneurons in stress adaptation and pathology. Biological Psychiatry, 86(9), 669681. https://doi.org/10.1016/j.biopsych.2019.04.032 CrossRefGoogle ScholarPubMed
Michael, A., Jenaway, A., Paykel, E. S., & Herbert, J. (2000). Altered salivary dehydroepiandrosterone levels in major depression in adults. Biological Psychiatry, 48(10), 989995. https://doi.org/10.1016/S0006-3223(00)00955-0 CrossRefGoogle ScholarPubMed
Mulligan, E. M., Hajcak, G., Crisler, S., & Meyer, A. (2020). Increased dehydroepiandrosterone (DHEA) is associated with anxiety in adolescent girls. Psychoneuroendocrinology, 119, 104751. https://doi.org/10.1016/j.psyneuen.2020.104751 CrossRefGoogle ScholarPubMed
Nicolson, N. A., & Ponnamperuma, T. (2019). Gender moderates diurnal cortisol in relation to trauma and PTSD symptoms: A study in Sri Lankan adolescents. Psychoneuroendocrinology, 104, 122131. https://doi.org/10.1016/j.psyneuen.2019.02.012 CrossRefGoogle ScholarPubMed
Ouellet-Morin, I., Cantave, C., Paquin, S., Geoffroy, M. C., Brendgen, M., Vitaro, F., & Côté, S. (2021). Associations between developmental trajectories of peer victimization, hair cortisol, and depressive symptoms: A longitudinal study. Journal of Child Psychology and Psychiatry, 62(1), 1927. https://doi.org/10.1111/jcpp.13228 CrossRefGoogle ScholarPubMed
Ouellet-Morin, I., Odgers, C. L., Danese, A., Bowes, L., Shakoor, S., Papadopoulos, A. S., & Arseneault, L. (2011). Blunted cortisol responses to stress signal social and behavioral problems among maltreated/bullied 12-year-old children. Biological Psychiatry, 70, 10161023. https://doi.org/10.1016/j.biopsych.2011.06.017 CrossRefGoogle ScholarPubMed
Panter-Brick, C., Wiley, K., Sancilio, A., Dajani, R., & Hadfield, K. (2020). C-reactive protein, Epstein-Barr virus, and cortisol trajectories in refugee and non-refugee youth: Links with stress, mental health, and cognitive function during a randomized controlled trial. Brain, Behavior, and Immunity, 87, 207217. https://doi.org/10.1016/j.bbi.2019.02.015 CrossRefGoogle ScholarPubMed
Pauli-Pott, U., Schloss, S., Skoluda, N., Nater, U. M., & Becker, K. (2019). Low hair cortisol concentration predicts the development of attention deficit hyperactivity disorder. Psychoneuroendocrinology, 110, 104442. https://doi.org/10.1016/j.psyneuen.2019.104442 CrossRefGoogle ScholarPubMed
Psarraki, E. E., Kokka, I., Bacopoulou, F., Chrousos, G. P., Artemiadis, A., & Darviri, C. (2020). Is there a relation between major depression and hair cortisol? A systematic review and meta-analysis. Psychoneuroendocrinology, 105098, https://doi.org/10.1016/j.psyneuen.2020.105098 Google Scholar
Russell, E., Koren, G., Rieder, M., & Uum, S. V. (2012). Hair cortisol as a biological marker of chronic stress: Current status, future directions and unanswered questions. Psychoneuroendocrinology, 37(5), 589601. https://doi.org/10.1016/j.psyneuen.2011.09.009 CrossRefGoogle ScholarPubMed
Rustein, S. O., & Johnson, K. ‘, The DHS wealth index, 2004)Google Scholar
Salis, K. L., Bernard, K., Black, S. R., Dougherty, L. R., & Klein, D. (2016). Examining the concurrent and longitudinal relationship between diurnal cortisol rhythms and conduct problems during childhood. Psychoneuroendocrinology, 71, 147154. https://doi.org/10.1016/j.psyneuen.2016.05.021 CrossRefGoogle ScholarPubMed
Samad, L., Hollis, C., Prince, M., & Goodman, R. (2005). Child and adolescent psychopathology in a developing country: Testing the validity of the strengths and difficulties questionnaire (Urdu version). International Journal of Methods in Psychiatric Research, 14(3), 158166, https://doi.org/10.1002/mpr.3,CrossRefGoogle Scholar
Sandstrom, A., Daoust, A. R., Russell, E., Koren, G., & Hayden, E. P. (2020). Hair cortisol concentrations predict change in girls’ depressive symptoms. European Journal of Developmental Psychology, 115. https://doi.org/10.1080/17405629.2020.1774359 Google Scholar
Sapolsky, R. M. (2003). Stress and plasticity in the limbic system. Neurochemical Research, 28(11), 17351742.CrossRefGoogle ScholarPubMed
Saridjan, N. S., Velders, F. P., Jaddoe, V. W. V., Hofman, A., Verhulst, F. C., & Tiemeier, H. (2014). The longitudinal association of the diurnal cortisol rhythm with internalizing and externalizing problems in pre-schoolers. The Generation R Study. Psychoneuroendocrinology, 50, 118129. https://doi.org/10.1016/j.psyneuen.2014.08.008 CrossRefGoogle ScholarPubMed
Schindler, L., Shaheen, M., Saar-Ashkenazy, R., Bani Odeh, K., Sass, S.-H., Friedman, A., & Kirschbaum, C. (2019). Victims of war: Dehydroepiandrosterone concentrations in hair and their associations with trauma sequelae in palestinian adolescents living in the West Bank. Brain Sciences, 9(2), 20. https://doi.org/10.3390/brainsci9020020 CrossRefGoogle ScholarPubMed
Shaheen, M., Schindler, L., Saar-Ashkenazy, R., Bani Odeh, K., Soreq, H., Friedman, A., & Kirschbaum, C. (2020). Victims of war—Psychoendocrine evidence for the impact of traumatic stress on psychological well-being of adolescents growing up during the Israeli-Palestinian conflict. Psychophysiology, 57(1), e13271. https://doi.org/10.1111/psyp.13271 CrossRefGoogle ScholarPubMed
Shakiba, N., Ellis, B. J., Bush, N. R., & Boyce, W. T. (2020). Biological sensitivity to context: A test of the hypothesized U-shaped relation between early adversity and stress responsivity. Development and Psychopathology, 32(2), 641660. https://doi.org/10.1017/S0954579419000518 CrossRefGoogle ScholarPubMed
Sikander, S., Ahmad, I., Bates, L. M., Gallis, J., Hagaman, A., O’Donnell, K., , , & Maselko, J. (2019). Cohort profile: Perinatal depression and child socioemotional development; the Bachpan cohort study from rural Pakistan. BMJ Open, 9(5), e025644, https://doi.org/10.1136/bmjopen-2018-025644,CrossRefGoogle ScholarPubMed
Smider, N. A., Essex, M. J., Kalin, N. H., Buss, K. A., Klein, M. H., Davidson, R. J., & Goldsmith, H. H. (2002). Salivary cortisol as a predictor of socioemotional adjustment during kindergarten: A prospective study. Child Development, 73, 7592. https://doi.org/10.2307/3696432 CrossRefGoogle ScholarPubMed
Staufenbiel, S. M., Penninx, B. W., Spijker, A. T., Elzinga, B. M., & van Rossum, E. F. (2013). Hair cortisol, stress exposure, and mental health in humans: A systematic review. Psychoneuroendocrinology, 38(8), 12201235. https://doi.org/10.1016/j.psyneuen.2012.11.015 CrossRefGoogle ScholarPubMed
Steudte-Schmiedgen, S., Kirschbaum, C., Alexander, N., & Stalder, T. (2016). An integrative model linking traumatization, cortisol dysregulation and posttraumatic stress disorder: Insight from recent hair cortisol findings. Neuroscience & Biobehavioral Reviews, 69, 124135. https://doi.org/10.1016/j.neubiorev.2016.07.015 CrossRefGoogle ScholarPubMed
Turner, E. L., Sikander, S., Bangash, O., Zaidi, A., Bates, L., Gallis, J., & Maselko, J. (2016). The effectiveness of the peer delivered Thinking Healthy Plus (THPP+) Programme for maternal depression and child socio-emotional development in Pakistan: Study protocol for a three-year cluster randomized controlled trial. Trials, 17(1), 111. https://doi.org/10.1186/s13063-016-1530-y CrossRefGoogle ScholarPubMed
Wang, M. (2014). Generalized estimating equations in longitudinal data analysis: A review and recent developments. Advances in Statistics, 2014(1), 111. https://doi.org/10.1155/2014/303728.Google Scholar
Wesarg, C., Van Den Akker, A. L., Oei, N. Y., Hoeve, M., & Wiers, R. W. (2020). Identifying pathways from early adversity to psychopathology: A review on dysregulated HPA axis functioning and impaired self-regulation in early childhood. European Journal of Developmental Psychology, 1-20, https://doi.org/10.1080/17405629.2020.1748594 Google Scholar
Zimmerman, A., Halligan, S., Skeen, S., Morgan, B., Fraser, A., Fearon, P., & Tomlinson, M. (2020). PTSD symptoms and cortisol stress reactivity in adolescence: Findings from a high adversity cohort in South Africa. Psychoneuroendocrinology, 121, 104846. https://doi.org/10.1016/j.psyneuen.2020.104846 CrossRefGoogle ScholarPubMed
Figure 0

Figure 1. Timeline and sample size for hair subsample.

Figure 1

Table 1. Sample characteristics

Figure 2

Table 2. Bivariate correlations between study variables

Figure 3

Figure 2. Quadratic fit plots of the relation between the SDQ total difficulties scale and hair cortisol (Panel A) and DHEA (Panel B).

Figure 4

Figure 3. Quadratic fit plots of the relation between cortisol and the SDQ emotional problems (Panel A) and conduct problems (Panel B) scales. Linear fit plots of hyperactivity problems (Panel C), peer problems (Panel D), and prosocial behavior (Panel E) scales.

Figure 5

Figure 4. Quadratic fit plots of the relation between cortisol and the SDQ emotional problems (Panel A) and conduct problems (Panel B) scales. Linear fit plots of hyperactivity problems (Panel C), peer problems (Panel D), and prosocial behavior (Panel E) scales.

Supplementary material: File

Frost et al. supplementary material

Frost et al. supplementary material

Download Frost et al. supplementary material(File)
File 22.4 KB