Hostname: page-component-78c5997874-ndw9j Total loading time: 0 Render date: 2024-11-13T00:52:04.596Z Has data issue: false hasContentIssue false

Offspring's risk for suicidal behaviour in relation to parental death by suicide: systematic review and meta-analysis and a model for familial transmission of suicide

Published online by Cambridge University Press:  05 November 2021

Mara Calderaro
Affiliation:
Department of Psychiatry and Psychotherapy, Charité Campus Mitte, Charité Universitätsmedizin Berlin,Germany; and Institute for Medical Information Processing, Biometry and Epidemiology, LMU Munich, Germany; and Pettenkofer School of Public Health, Munich, Germany
Christopher Baethge
Affiliation:
Department of Psychiatry and Psychotherapy, University of Cologne Medical School, Germany
Felix Bermpohl
Affiliation:
Department of Psychiatry and Psychotherapy, Charité Campus Mitte, Charité Universitätsmedizin Berlin,Germany
Stefan Gutwinski
Affiliation:
Department of Psychiatry and Psychotherapy, Charité Campus Mitte, Charité Universitätsmedizin Berlin,Germany
Meryam Schouler-Ocak
Affiliation:
Department of Psychiatry and Psychotherapy, Charité Campus Mitte, Charité Universitätsmedizin Berlin,Germany
Jonathan Henssler*
Affiliation:
Department of Psychiatry and Psychotherapy, Charité Campus Mitte, Charité Universitätsmedizin Berlin,Germany; and Department of Psychiatry and Psychotherapy, University of Cologne Medical School, Germany
*
Correspondence: Jonathan Henssler. Email: jonathan.henssler@charite.de
Rights & Permissions [Opens in a new window]

Abstract

Background

Exposure to parental suicide has been associated with increased risk for suicide and suicide attempts, although the strength of this association is unclear as evidence remains inconsistent.

Aims

To quantify this risk using meta-analysis and identify potential effect modifiers.

Method

A systematic search in PubMed, PsycInfo and Embase databases to 2020 netted 3614 articles. Inclusion criteria were: observation of history of parental death by suicide, comparison with non-exposed populations and definition of suicide and suicide attempt according to standardised criteria. We focused on population-based studies. The primary outcome was the pooled relative risk (RR) for incidence of suicide attempt and suicide in offspring of a parent who died by suicide compared with offspring of two living parents. Additionally, we compared the RR for attempted and completed suicide after parental suicide with the RR for attempted and completed suicide after parental death by other causes.

Results

Twenty studies met our inclusion criteria. Offspring exposed to parental suicide were more likely to die by suicide (RR = 2.97, 95% CI 2.50–3.53) and attempt suicide (RR = 1.76, 95% CI 1.58–1.96) than offspring of two living parents. Furthermore, their risk of dying by or attempting suicide was significantly higher compared with offspring bereaved by other causes of death.

Conclusions

The experience of losing a parent to suicide is a strong and independent risk factor for suicidal behaviour in offspring. Our findings highlight the need for prevention strategies, outreach programmes and support interventions that target suicide-related outcomes in the exposed population.

Type
Review
Copyright
Copyright © The Author(s), 2021. Published by Cambridge University Press on behalf of the Royal College of Psychiatrists

Almost 800 000 people die by suicide every year (one person every 40 s).1 The prevalence of suicide attempts is significantly higher1 – for each suicide death it is estimated that there have been more than 20 suicide attempts. Suicide rates have increased globally by 60% in the past 45 years, particularly among young adults.2 In adolescents aged 15–29, suicide is the third leading cause of death worldwide.1 Every suicide affects not only family and friends of the decedent, but also society as a whole.2,Reference Pompili, Shrivastava, Serafini, Innamorati, Milelli and Erbuto3 Research shows that approximately 135 people are exposed per suicide death.Reference Cerel, Brown, Maple, Singleton, van de Venne and Moore4 Suicide and suicide attempts are among the leading causes of global morbidity and mortality and pose a serious public health problem that calls for prevention strategies.

Offspring of suicide decedents may be particularly vulnerable to suicidal behaviourReference Guldin, Li, Pedersen, Agerbo, Gissler, Cnattingius and Olsen5Reference Mittendorfer-Rutz, Rasmussen and Lange7 and previous studies found an accumulation of suicide cases in families (parents and offspring).Reference Brent and Melhem8Reference Sørensen, Mortensen, Wang, Juel, Silverton and Mednick10 Owing to contradictory study results, it is unclear whether parental suicide is a specific risk factor or whether, in general, the death of a parent during a vulnerable phase increases the risk of suicide.Reference Jakobsen and Christiansen11Reference Qin, Agerbo and Mortensen13 Also, there is inconsistent evidence on the effect of age of offspring at the time of exposure to parental death and on the effect of gender of parent and offspring.Reference Jakobsen and Christiansen11,Reference Kuramoto, Stuart, Runeson, Lichtenstein, Långström and Wilcox14,Reference Cheng, Yen, Chang, Wu, Ko and Li15

Currently, an up-to-date systematic review on this topic that includes a formal meta-analysis does not exist. The last review with meta-analysis on the subject dates back more than 10 years and did not account for potential effect modifiers.Reference Geulayov, Gunnell, Holmen and Metcalfe16 Since then the number of published works and the knowledge in this research area have expanded steadily. For this reason, we aim to assess precise estimates of the overall association between parental suicide and risk for suicidal behaviour in offspring, taking into account all studies published from database inception to 2020. Furthermore, we aim to investigate the influence of effect modifiers and determinants in order to be able to design preventive measures in a targeted manner. We will therefore assess differential effects on completion or attempt of suicide in offspring, and control for parental death by other causes. We also aim to assess the influence of parental and offspring gender and their potential interaction.

Method

This systematic review with meta-analysis was conducted in accordance with recommendations of the Cochrane group using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)Reference Moher, Liberati, Tetzlaff and Altman17 and the Meta-Analyses of Observational Studies in Epidemiology (MOOSE) guidelines.Reference Stroup, Berlin, Morton, Olkin, Williamson and Rennie18 The protocol was registered in the International Prospective Register of Systematic Reviews (PROSPERO registration no. CRD42020179257).

Literature search strategy

A systematic literature search of PubMed, PsycInfo and Embase databases was conducted for studies published up to 3 February 2020, with no time restriction. In brief, the following search terms were used: (((child OR adolescent OR young adults OR offspring OR young people) AND (parent OR mother OR father OR maternal OR paternal)) OR Child of Impaired Parents OR bereavement) AND (Suicide OR Suicide, Attempted OR Suicide, Completed). Full details are given in the online supplement ‘Explicit Search Entry’, available at https://doi.org/10.1192/bjp.2021.158.

Additional records were identified by manually searching references of the included studies and previous reviews.Reference Lipsey and Wilson19 The search was not limited by language, location or year of publication. Articles written in languages other than English were translated by native speakers to test their eligibility. Full texts and data were accessible, so contacting authors of included studies was not necessary.

Definitions

For the purposes of this paper suicide was defined as ‘death caused by self-directed injurious behaviour with an intent to die as a result of the behaviour’.Reference Klonsky, May and Saffer20 A suicide attempt includes both self-injurious behaviour and suicidal intent (at least some intention of dying by suicide) and was defined as any non-fatal suicidal behaviour.Reference Posner, Oquendo, Gould, Stanley and Davies21 We excluded non-suicidal self-injury, which refers to self-injurious behaviour without suicidal intent.Reference Nock, Favazza and Nock22 For the purposes of this paper the term ‘suicidal behaviour’ refers to both completed and attempted suicide.

Eligibility criteria

Eligible studies were primary research studies with full text available reporting the effect size (including rate ratio, risk ratio, odds ratio or hazard ratio) and spread of incidence of suicide or suicide attempts in offspring exposed to parental suicide in comparison with the non-exposed populations. Included studies had to control for directionality, i.e. only account for offspring suicidal behaviour after the experience of parental suicide. We included studies only if incidence measures and case ascertainment had been made on a sufficiently exhaustive basis with regard to catchment area (the region where study data were ascertained), focusing on population-registry based studies.

We excluded studies investigating solely psychiatric disorders (e.g. affective disorders) of offspring exposed to parental suicide as well as those solely investigating suicidal ideation and self-harm not classified as suicide attempt.

Study selection, data collection and data extraction

Two authors (M.C. and J.H.) independently screened titles and abstracts of all database search results for eligibility. All articles potentially meeting the initial inclusion criteria were retrieved as full text. After independently conducting a full-text review of all the articles by the two authors, their results were compared. Conflicts in unclear cases were resolved via discussion with a third author (S.G.). Relevant data (author, year, study location, characteristics of study population, outcomes, study design, subanalyses, sample size, information source) were extracted independently by two authors (M.C. and J.H.) following the recommendations of the Cochrane Collaboration Handbook.Reference Higgins, Thomas, Chandler, Cumpston, Li and Page23 Disagreements on data extraction were resolved by consensus with other authors. Data were first extracted on 14 June 2020.

Main outcomes and measures

Our primary outcome, formulated before data collection, was the pooled relative risk for the incidence of suicide attempt or completed suicide in offspring exposed to parental suicide compared with offspring of two living parents. We pooled all adjusted effect sizes of relative risks provided by the included studies (risk ratios (RR), rate ratios (RaR), hazard ratios (HR), odds ratios (OR)).

Risk of bias and quality assessment

The risk of bias of included studies was assessed independently by two reviewers (M.C. and J.H.) using the Newcastle–Ottawa Scale for assessing the quality of non-randomised studies in meta-analyses.Reference Wells, Shea, O'Connell, Peterson, Welch and Losos24 Each study was assigned a summary rating after evaluating the following three domains of bias: selection, comparability and outcome for cohort studies; selection, comparability and outcome for cross-sectional studies; and selection, comparability and exposure for case–control studies. A summary rating was predefined, categorising studies with 6 or more points out of 10 (for cross-sectional studies) or out of 9 (for cohort and case–controls studies) as ‘low risk’.Reference Higgins, Thomas, Chandler, Cumpston, Li and Page23 Any discrepancies were settled by discussion with a third author (S.G.).

Statistical analysis

Effect sizes of individual studies were pooled using the random-effects model as proposed by DerSimonian & Laird.Reference DerSimonian and Laird25 The primary outcome was the pooled RR (with 95% CI) in offspring exposed to parental suicide compared with the RR in the non-exposed population. Effect sizes of different subgroups within a single study were pooled using a fixed-effects model. We pooled effect sizes of relative risks provided by the included studies (RR, RaR, HR, OR), using – as applicable – the effect size data adjusted for the highest number of confounders. As the incidence of suicide and suicide attempts was reported to be less than 10% in the unexposed population (i.e. in offspring of alive parents) in earlier studiesReference Guldin, Li, Pedersen, Agerbo, Gissler, Cnattingius and Olsen5 the OR is a good approximation of the RR.Reference Viera26 Nevertheless, we accounted for the respective prevalence in the unexposed population using the model proposed by Zhang & Yu to calculate RRs from ORs.Reference Zhang and Yu27 Although outcome data from case–control studies are generally considered to be presented as ORs only, we decided that data from the primary studies included in our analyses could be transformed into RRs, as the included case–control studies were based on population-registry data representing a comprehensive base for calculations of risks in the underlying population. Furthermore, we used HR and RaR as an approximation to the RR, as the population at risk was assumed to be stable.Reference Green and Symons28,Reference Rothman29 If outcome data from a study were available only stratified by gender, we presented effect sizes for women and men separately.

Heterogeneity among the underlying trials was quantified using I 2, as recommended by Higgins & Thompson.Reference Higgins and Thompson30 Effect estimates were interpreted in consideration of present heterogeneity.

Publication bias was assessed using funnel plots, Egger's testReference Egger, Smith, Schneider and Minder31 and Duval & Tweedie's trim and fill methodReference Duval and Tweedie32 for the primary outcome analysis. We conducted analyses according to recommendations in the Cochrane Collaboration HandbookReference Higgins, Thomas, Chandler, Cumpston, Li and Page23 and using Comprehensive Meta-Analysis Version 3 for Windows (Biostat, Englewood, New Jersey). For primary outcome analysis, a two-tailed P < 0.05 was considered statistically significant.

Sensitivity and subgroup analyses

Sensitivity analyses of the primary outcome considered studies of higher methodological rigour (i.e. with low risk of bias), and we conducted separate analyses by measure of effect size (i.e. OR, RR, RaR, HR) and by study type. Additional sensitivity analyses controlled for potential partial overlap of studies by taking into account the single largest respective study only.

Subgroup analyses took into account deceased parents’ and offspring's gender.

Secondary outcome

As a secondary outcome we assessed the pooled relative risk for the incidence of suicide attempt or completed suicide in offspring who experienced parental suicide compared with offspring who experienced parental death from other causes (not suicide). This analysis could be calculated from unadjusted values only.

Results

Study selection and characteristics

Of the 3614 articles identified through literature search and screened, 20 studies, published between 2002 and 2018, involving 19 381 157 probands, met our inclusion criteria and provided sufficient data to be included in our analyses (Fig. 1). Of the 20 studies, 8 were cohort studies, 11 were case–control studies and 1 was a cross-sectional study; 16 studies were conducted in Scandinavian countries, i.e. 8 studies originate from Sweden, 5 from Denmark, 2 from Norway, and 1 from Denmark, Finland and Sweden. Of the 20 studies, 16 presented data on offspring who lost a parent to suicide compared with offspring with two living parents. Nine studies provided sufficient data to be included in our subgroup analyses regarding offspring exposed to parental death by other external causes. Of the 20 studies, 10 provided information on the gender of the deceased parent and 16 gave information on the gender of the bereaved offspring. Further details of each of the included studies are summarised in Table 1.

Fig. 1 PRISMA flowchart.

Table 1 Characteristics of studies included in the systematic review and meta-analyses grouped by study design

a. ‘Death’ denotes death by suicide; ‘attempt’ denotes attempted suicide.

NOS, Newcastle–Ottawa Scale.

Analysis of primary outcome

We included 15 studies in the main analysis (Fig. 2). Offspring exposed to parental death by suicide were at greater risk of dying by suicide (RR = 2.97, 95% CI 2.50–3.53; I 2 = 84.8; tau2 = 0.074) and attempting suicide (RR = 1.76, 95% CI 1.58–1.96; I 2 = 38.5; tau2 = 0.006) compared with offspring of two living parents.

Fig. 2 Forest plot 1: main outcome – risk of suicide attempt (SA) and suicide death (SD) in offspring after parental suicide compared with offspring of two living parents.

Risk of bias within studies

The scores on the Newcastle–Ottawa Scale for assessing the quality of non-randomised studies in meta-analyses are presented in Table 1. All of the 20 studies included in the analyses were rated as at low risk of bias.

Subgroup analyses

Subgroup analysis for gender of the deceased parent was based on eight studies and revealed that maternal suicide was more strongly associated with offspring's suicidal behaviour (RR = 3.25, 95% CI 2.49–4.25; I 2 = 71.7%) than paternal suicide (RR = 2.60, 95% CI 2.05–3.28; I 2 = 77.6%). The test for between-group heterogeneity did not reach statistical significance (P = 0.217) (supplementary Fig. 1). Offspring gender was not found to affect effect size (supplementary Fig. 2).

Sensitivity analysis

Sensitivity analysis of the primary outcome controlling for partial overlap of study populations resulted in an RR of 1.86 (95% CI 1.54–2.25; I 2 = 62.03%) for suicide attempt and an RR of 2.97 (95% CI 2.50–3.53; I 2 = 84.84%) for completed suicide in offspring. All of the studies included in our primary outcome analysis were considered to be at low risk of bias. Therefore, our sensitivity analysis for low risk of bias was consistent with the main analysis.

There was no indication that study type or effect measure affected effect size. RRs (for suicide attempt and/or completed suicide) added up to 2.43 (95% CI 2.04–2.89; I 2 = 87.8%), 2.76 (95% CI 2.09–3.64; I 2 = 87.0%) and 2.56 (95% CI 1.48–4.46; I 2 = 0%) for case–control, cohort and cross-sectional studies respectively. When pooling studies separately by employed effect measure, RRs added up to 2.47 (95% CI 2.04–2.98; I 2 = 87.2%), 2.70 (95% CI 1.81–4.03; I 2 = 0%), 2.69 (95% CI 1.61–4.48; I 2 = 86.3%) and 2.95 (95% CI 2.02–4.30; I 2 = 86.0%) for studies calculating ORs, RRs, RaRs and HRs respectively.

The funnel plot and Egger's test (P = 0.00849; 2-tailed; supplementary Fig. 3) for our primary outcome analysis indicated a missing presence of small study effects (indicating possible reporting bias). Duval & Tweedie's trim and fill method with five studies imputed to the left of the mean resulted in an adjusted RR of 2.35 (95% CI 2.06–2.68) versus an observed RR of 2.54 (95% CI 2.22–2.91) for offspring suicide attempt and completed suicide combined.

Secondary outcome

Eight studies were included in the analysis of our secondary outcome. Compared with the RR for offspring who lost a parent to other causes of death, the RR of dying by suicide was increased by 2.46 (95% CI 2.00–3.02; I 2 = 49.7%) and the RR of attempting suicide was increased by 1.32 (95% CI 1.18–1.49; I 2 = 0%) in offspring who lost a parent to suicide (Fig. 3). The test for between-group heterogeneity was significant at P < 0.001.

Fig. 3 Forest plot 2: risk of suicide attempt (SA) and suicide death (SD) in offspring who experienced parental suicide compared with offspring who experienced parental death by other causes. vs., versus.

Experiencing parental death by other causes was associated with an increased risk of 1.71 (95% CI 1.53–1.92; I 2 = 22.3%) for dying by suicide and an increased risk of 1.64 (95% CI 1.44–1.86; I 2 = 0%) for attempting suicide compared with offspring of two living parents (supplementary Fig. 4).

The absolute risk of dying by suicide in offspring of suicide decedents ranged from 0.23 to 2.9% (mean 0.87%; median 0.56%) in the included cohort studies.

Discussion

Main findings

This systematic review with meta-analysis found that experiencing a parental suicide was associated with an almost three-fold increased risk of dying by suicide and an almost two-fold increased risk of attempting suicide in offspring compared with offspring of two living parents. Although experiencing parental death by other causes was also associated with an increased suicide risk, the risk of dying by or attempting suicide in offspring of suicide decedents was still significantly higher compared with offspring bereaved by other causes of death.

Noteworthy, the risk of dying by suicide is increased substantially more than the risk of attempting suicide, and confidence intervals do not overlap. It seems remarkable that particularly the most harming behaviour is passed on to the next generation. Having said that, the mean absolute risk of dying by suicide in offspring of suicide decedents was 0.87% in the included studies.

Included studies were of high methodological rigour and their data were mainly retrieved from national population registries, resulting in large statistical power of our analyses and reliability of our findings.

Possible mechanisms of transmission

How can we explain this increased risk in offspring of suicide decedents?

Starting from our analyses, we may differentiate between mechanisms of transmission that can also be found in families of a parent who died by other causes (e.g. family disruption, stressful life event) and those that can specifically be found in families with parental suicide (e.g. heredity, imitation). As suicide risk is elevated after any parental death, but significantly more so after parental suicide, our findings indicate that both types of mechanism play a significant and independent role, but may jointly contribute to the substantially increased risk after parental suicide.

In the following, we will further differentiate the possible mechanisms of familial transmission of suicide. A general comprehensive model for the transmission of mental disorders is lacking. Starting from and further building on Goodman & Gotlib'sReference Goodman and Gotlib45 model for the transmission of maternal depression, we propose the following pathways for increased vulnerability for suicide in offspring after parental suicide.

Heritability (genetic factors)

Evidence from twin, adoption and molecular genetic studies indicates at least partial genetic contributions to the familial transmission of suicide risk.Reference Garssen, Deerenberg, Mackenbach, Kerkhof and Kunst6,Reference O'Reilly, Kuja-Halkola, Rickert, Class, Larsson and Lichtenstein46Reference Docherty, Shabalin, DiBlasi, Monson, Mullins and Adkins50 It may be mediated by the transmission of parental mood disorderReference Baldessarini, Tondo, Pinna, Nuñez and Vázquez51 or intermediate phenotypes, such as impulsive aggression – the mediator with the most convergent evidence.Reference Brent and Melhem8,Reference Brent, Bridge, Johnson and Connolly52Reference Coccaro, Bergeman, Kavoussi and Seroczynski54 Other possible intermediate phenotypes are neuroticism and neurocognitive deficits. Also, hypothalamic–pituitary–adrenal axis dysfunctions have been shown to be associated with suicide risk and may partly be inherited.Reference Mann, Arango, Avenevoli, Brent, Champagne and Clayton55,Reference Oquendo, Sullivan, Sudol, Baca-Garcia, Stanley and Sublette56

Environmental factors

Suicidal behaviour was shown to be associated with socioeconomic disadvantageReference Iemmi, Bantjes, Coast, Channer, Leone and McDaid57 – offspring of suicide decedents are consequently more likely to be exposed to a stressful environment. This affects offspring (a) possibly already during pregnancy, increasing the risk for innate dysfunctional neuroregulatory mechanisms,Reference Goodman and Gotlib45,Reference Oquendo, Sullivan, Sudol, Baca-Garcia, Stanley and Sublette56 and (b) during childhood and adolescence before the suicide of the parent. Early life adversity and factors such as poor parenting, neglect, child abuse, family adversity and family disruption have repeatedly been proposed to play a role in the familial clustering of suicidal behaviour.Reference Brent and Melhem8,Reference Mittendorfer-Rutz, Rasmussen and Wasserman36,Reference Turecki58 (c) Further, exposure to parental cognitions, behaviour and affect may influence offspring's learned behaviour and coping strategies. Parents’ impaired coping skills may lead to the same limited abilities in their children.Reference Stenager and Qin59 The lack of an affectionate attachment figure and adequate care may result in an increased risk for suicidal behaviour.Reference Kuramoto, Stuart, Runeson, Lichtenstein, Långström and Wilcox14,Reference Harris, Brown and Bifulco60 Many of the epigenetic changes found in suicide completers may be due to the experience of stressful life events.Reference Labonté, Suderman, Maussion, Lopez, Navarro-Sánchez and Yerko61 (d) Even after the parental suicide, in addition to socioeconomic disadvantages and to their own grief reactions and internal vulnerabilities, offspring who lost a parent to suicide might experience a change in quality of parenting by the surviving parent who suffers grief during one of the most sensitive developmental periods of the child.Reference Pitman, Osborn and King9,Reference Brown, Sandler, Tein, Liu and Haine62 In line with the preceding, many of the included studies found considerable modification of effects by socioeconomic factors and familial psychopathology.

Imitation

Imitation might play a particular role in familial transmission of suicide.Reference Cheng, Yen, Chang, Wu, Ko and Li15 Identification with one's caregivers plays a role in developing a sense of self, and a parent's self-harming behaviour and coping strategies are likely to be imitated. Offspring might thus replicate the parents’ way of solving problems, including suicide when facing difficulties in life.Reference Cheng, Yen, Chang, Wu, Ko and Li15,Reference Mittendorfer-Rutz, Rasmussen and Wasserman36 Imitation may also desperately be driven by an urge to understand the suicidal parent's motives and state of mind.Reference Tillman63

Exposure to parental suicide as a life event

Bereavement is one of life's most stressful events and has a great impact on physical, social and psychological well-being.Reference Latham and Prigerson64 Evidence suggests that unnatural causes of parental death determine a higher risk of suicidal behaviour in offspring, with accidents taking an intermediate position between parental suicide and natural causes of parental death.Reference Guldin, Li, Pedersen, Agerbo, Gissler, Cnattingius and Olsen5,Reference Kuramoto, Stuart, Runeson, Lichtenstein, Långström and Wilcox14,Reference Li, Vestergaard, Cnattingius, Gissler, Bech and Obel65 In the same vein, a recent analysis of Danish registry data suggests that the risk of self-harm among offspring of parents who died from other unnatural causes is similar to that of offspring exposed to parental death by suicide.Reference Carr, Mok, Antonsen, Pedersen and Webb66 The immediacy and unexpectedness of unnatural causes of death may particularly complicate the process of mourning. Parental suicide tends to generate more complicated grief experiences than other causes of death. Such grief has been associated with suicidal behaviour among bereaved persons.Reference Pitman, Osborn and King9,Reference Mitchell, Kim, Prigerson and Mortimer67 Grievers are left with the question of why their parent died by suicide while coping with stigmatising discourses, hatred, feelings of responsibility, guilt, shame and abandonment.Reference Hagström68,Reference Burrell, Mehlum and Qin69 In most societies, suicide represents a ‘deviant and morally norm-breaking death’Reference Goffman70 and consequently stigmatising sociocultural pressure is likely to be imposed on the mourning relatives both externally and from within.Reference Hagström68 These experiences of stigma and feelings of responsibility, guilt, shame and abandonment may be of particular importance in light of the interpersonal theory of suicide highlighting the hazardous nature of combined perceived burdensomeness and thwarted belongingness.Reference Van Orden, Witte, Cukrowicz, Braithwaite, Selby and Joiner71

Moderating factors

Studies assessing the influence of age at exposure to parental suicide seem to confirm environmental effects and the importance of accounting for developmental processes, as the long-term risk for suicide in offspring seems to differ depending on the developmental period during which parental suicide occurs. Parental loss during childhood and adolescence seems to have a greater impact on suicide risk than loss in adulthood,Reference Wilcox, Kuramoto, Lichtenstein, Långström, Brent and Runeson12,Reference Niederkrotenthaler, Floderus, Alexanderson, Rasmussen and Mittendorfer-Rutz38,Reference Kuramoto, Runeson, Stuart, Lichtenstein and Wilcox41 and the risk of suicidal behaviour seems to be higher, the younger the child is at the time of parental suicide.Reference Garssen, Deerenberg, Mackenbach, Kerkhof and Kunst6,Reference Wilcox, Kuramoto, Lichtenstein, Långström, Brent and Runeson12,Reference Burrell, Mehlum and Qin34,Reference Mittendorfer-Rutz, Rasmussen and Lange37,Reference Niederkrotenthaler, Floderus, Alexanderson, Rasmussen and Mittendorfer-Rutz38,Reference Kuramoto, Runeson, Stuart, Lichtenstein and Wilcox41 These findings are in accordance with the ‘sensitive period’ hypothesis in life course epidemiology, proposing that, during a sensitive period, exposure to a stressor has a stronger effect on development and on the risk for disease than it would have at other times.Reference Ben-Shlomo and Kuh72 Although not reaching statistical significance, our finding that maternal suicide is more strongly associated with offspring suicidal behaviour than paternal suicide poses a promising hypothesis for future research. Impaired attachment and bonding as well as a damaged relationship between mother and child have been proposed to be more relevant than between father and child.Reference Agerbo, Nordentoft and Mortensen33,Reference Cantor, Hawton and van Heeringen)73 Maternal suicide is often seen as the loss of the primary caregiver, who constitutes a significant source of support.Reference Kuramoto, Stuart, Runeson, Lichtenstein, Långström and Wilcox14

Strength and limitations

The representativeness and validity of our findings may be limited by several aspects. First, owing to our focus on population-registry based data, the majority (16 of 20) of the included studies were conducted in Scandinavian countries. The extent to which the results of this meta-analysis can be generalised to other countries needs further investigation and the specific, potentially moderating, characteristics of a particular society always need to be considered. Still, we were able to include two studies from Taiwan,Reference Cheng, Yen, Chang, Wu, Ko and Li15,Reference Lee, Li, Chang, Lu and Chen74 one study from The NetherlandsReference Garssen, Deerenberg, Mackenbach, Kerkhof and Kunst6 and one study that was conducted using data from World Mental Health Surveys with nationally representative samples from 21 countries around the world.Reference Gureje, Oladeji, Hwang, Chiu, Kessler and Sampson44 These studies were invariably in line with our overall findings and suggest that our results are also applicable to other countries and societies.

Second, despite our comprehensive search and screening strategy, we may have missed studies, especially grey literature. To mitigate this common limitation of systematic reviews, our literature search was very extensive and thorough, was not limited by language, location or year of publication and resulted in a large sample size.

Third, known limitations of all meta-analysis are the inheritance of the quality of included studies, between-study heterogeneity and possible reporting bias.Reference Esterhuizen and Thabane75 Methodological rigour of all the included studies was assessed following the recommendations of the Cochrane Collaboration. Sensitivity analyses invariably strengthened our initial results. I 2-statistics indicated considerable heterogeneity in some of our analyses. However, none of the included studies pointed towards a negative or non-significant effect, but invariably confirmed the higher risk for suicidal behaviour in offspring who experienced parental suicide. Moreover, we used subgroup analyses for the purpose of exploring the sources of heterogeneity. Effect estimates were interpreted in consideration of present heterogeneity. Reporting bias was considered using funnel plots and Egger's test but was found to not significantly influence our findings. In our risk of bias analysis, we included the potential bias arising from case definitions of suicide attempts, to reflect the difficulty of differentiating suicide attempts from other self-injurious behaviour. However, it is possible that even in studies rated as carrying a low risk of bias some uncertainty regarding this distinction remained. Therefore, as in other studies on the subject, validity of results on completed suicides is probably higher than that of results on suicide attempts.

Implications

Our findings indicate that the experience of losing a parent to suicide may represent an independent and specific risk factor for suicidal behaviour in bereaved offspring. We believe that this highlights the need for psychiatric and psychotherapeutic prevention strategies, outreach programmes and long-term individual and family support interventions that target suicide-related outcomes in those exposed to suicide. These measures and strategies should include efforts to prevent the development of suicidal behaviours, mitigate adverse effects associated with exposure to suicide, diminish morbidity and mortality associated with suicidal behaviours and build resilience in this high-risk population. Therefore, children of parents who died by suicide should receive early attention, including identification of critical periods during which the loss is particularly harmful. We believe that efficient collaboration between adult and child psychiatry, general paediatric medicine, social services and the education and training system is fundamental for those measures to be effective.

Supplementary material

Supplementary material is available online at https://doi.org/10.1192/bjp.2021.158.

Data availability

The data that support the findings of this study are available from the corresponding author on reasonable request.

Acknowledgement

We thank the Robert-Enke-Foundation for supporting research on transgenerational suicidal behaviour.

Author contributions

M.C.: conceptualisation; data curation; formal analysis; investigation; interpretation; validation; visualization; writing original draft; writing review and editing. C.B.: conceptualisation; investigation; interpretation; methodology; resources; supervision; validation; writing review and editing. F.B.: conceptualisation; interpretation; resources; supervision; validation; writing review and editing. S.G.: conceptualisation; interpretation; project administration; resources; supervision; validation; writing review and editing. M.S.-O.: resources; interpretation; supervision; validation; writing review and editing. J.H.: conceptualisation; data curation; formal analysis; investigation; interpretation; methodology; project administration; resources; validation; visualisation; writing original draft; writing review and editing. All authors approved of the final version of the manuscript.

Funding

This study received no specific grant from any funding agency, commercial or not-for-profit sectors.

Declaration of interest

J.H. received a grant from the German Ministry of Education and Research (BMBF) outside of the submitted work.

References

World Health Organization. Suicide. WHO, 2019 (https://www.who.int/news-room/fact-sheets/detail/suicide [cited 18 May 2020]).Google Scholar
World Health Organization. Preventing Suicide: a Global Imperative. WHO, 2014.Google Scholar
Pompili, M, Shrivastava, A, Serafini, G, Innamorati, M, Milelli, M, Erbuto, D, et al. Bereavement after the suicide of a significant other. Indian J Psychiatry 2013; 55: 256–63.CrossRefGoogle ScholarPubMed
Cerel, J, Brown, MM, Maple, M, Singleton, M, van de Venne, J, Moore, M, et al. How many people are exposed to suicide? Not six. Suicide Life Threat Behav 2019; 49: 529–34.CrossRefGoogle Scholar
Guldin, M-B, Li, J, Pedersen, HS, Agerbo, E, Gissler, M, Cnattingius, S, Olsen, J, et al. Incidence of suicide among persons who had a parent who died during their childhood: a population-based cohort study. JAMA Psychiatry 2015; 72: 1227–34.CrossRefGoogle ScholarPubMed
Garssen, J, Deerenberg, I, Mackenbach, JP, Kerkhof, A, Kunst, AE. Familial risk of early suicide: variations by age and sex of children and parents. Suicide Life Threat Behav 2011; 41: 585–93.CrossRefGoogle ScholarPubMed
Mittendorfer-Rutz, E, Rasmussen, T, Lange, T. A life-course study on effects of parental markers of morbidity and mortality on offspring's suicide attempt. Pers Individ Dif 2014; 60: S24.CrossRefGoogle Scholar
Brent, DA, Melhem, N. Familial transmission of suicidal behavior. Psychiatr Clin North Am 2008; 31: 157–77.CrossRefGoogle ScholarPubMed
Pitman, A, Osborn, D, King, M, et al. Effects of suicide bereavement on mental health and suicide risk. Lancet Psychiatry 2014; 1: 8694.CrossRefGoogle ScholarPubMed
Sørensen, HJ, Mortensen, EL, Wang, AG, Juel, K, Silverton, L, Mednick, SA. Suicide and mental illness in parents and risk of suicide in offspring: a birth cohort study. Soc Psychiatry Psychiatr Epidemiol 2009; 44: 748–51.CrossRefGoogle ScholarPubMed
Jakobsen, IS, Christiansen, E. Young people's risk of suicide attempts in relation to parental death: a population-based register study. J Child Psychol Psychiatry 2011; 52: 176–83.CrossRefGoogle ScholarPubMed
Wilcox, HC, Kuramoto, SJ, Lichtenstein, P, Långström, N, Brent, DA, Runeson, B. et al. Psychiatric morbidity, violent crime, and suicide among children and adolescents exposed to parental death. J Am Acad Child Adolesc Psychiatry 2010; 49: 514–23.Google ScholarPubMed
Qin, P, Agerbo, E, Mortensen, PB. Suicide risk in relation to family history of completed suicide and psychiatric disorders: a nested case-control study based on longitudinal registers. Lancet 2002; 360: 1126–30.CrossRefGoogle ScholarPubMed
Kuramoto, SJ, Stuart, EA, Runeson, B, Lichtenstein, P, Långström, N, Wilcox, HC. Maternal or paternal suicide and offspring's psychiatric and suicide-attempt hospitalization risk. Pediatrics 2010; 126: e1026–32.CrossRefGoogle ScholarPubMed
Cheng, C-CJ, Yen, W-J, Chang, W-T, Wu, KC, Ko, MC, Li, CY. Risk of adolescent offspring's completed suicide increases with prior history of their same-sex parents’ death by suicide. Psychol Med 2014; 44: 1845–54.CrossRefGoogle ScholarPubMed
Geulayov, G, Gunnell, D, Holmen, TL, Metcalfe, C. The association of parental fatal and non-fatal suicidal behaviour with offspring suicidal behaviour and depression: a systematic review and meta-analysis. Psychol Med 2012; 42: 1567–80.CrossRefGoogle ScholarPubMed
Moher, D, Liberati, A, Tetzlaff, J, Altman, DG. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement. PLoS Med 2009; 6(7): e1000097.CrossRefGoogle ScholarPubMed
Stroup, DF, Berlin, JA, Morton, SC, Olkin, I, Williamson, GD, Rennie, D, et al. Meta-analysis of observational studies in epidemiology - a proposal for reporting. JAMA 2000; 283: 2008–12.CrossRefGoogle Scholar
Lipsey, MW, Wilson, DW. Practical Meta-Analysis. Sage, 2001.Google Scholar
Klonsky, ED, May, AM, Saffer, BY. Suicide, suicide attempts, and suicidal ideation. Annu Rev Clin Psychol 2016; 12: 307–30.CrossRefGoogle ScholarPubMed
Posner, K, Oquendo, MA, Gould, M, Stanley, B, Davies, M. Columbia classification algorithm of suicide assessment (C-CASA): classification of suicidal events in the FDA's pediatric suicidal risk analysis of antidepressants. Am J Psychiatry 2007; 164: 1035–43.CrossRefGoogle ScholarPubMed
Nock, MK, Favazza, AR. Nonsuicidal self-injury: definition and classification. In Understanding Nonsuicidal Self-Injury: Origins, Assessment, and Treatment (ed Nock, MK): 918. American Psychological Association, 2009.CrossRefGoogle Scholar
Higgins, JPT, Thomas, J, Chandler, J, Cumpston, M, Li, T, Page, MJ, et al. Cochrane Handbook for Systematic Reviews of Interventions Version 6.0 (Updated July 2019). Cochrane, 2019. Available from: www.training.cochrane.org/handbook [cited 12 Sep 2019].CrossRefGoogle Scholar
Wells, GA, Shea, B, O'Connell, D, Peterson, J, Welch, V, Losos, M, et al. The Newcastle-Ottawa Scale (NOS) for Assessing the Quality of Nonrandomised Studies in Meta-Analyses. The Ottawa Hospital Research Institute, 2015 (http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp [cited 24 Oct 2021]).Google Scholar
DerSimonian, R, Laird, N. Meta-analysis in clinical trials. Control Clin Trials 1986; 7: 177–88.CrossRefGoogle ScholarPubMed
Viera, AJ. Odds ratios and risk ratios: what's the difference and why does it matter? South Med J 2008; 101: 730–4.CrossRefGoogle ScholarPubMed
Zhang, J, Yu, KF. What's the relative risk? A method of correcting the odds ratio in cohort studies of common outcomes. JAMA 1998; 280: 1690–1.CrossRefGoogle ScholarPubMed
Green, MS, Symons, MJ. A comparison of the logistic risk function and the proportional hazards model in prospective epidemiologic studies. J Chronic Dis 1983; 36: 715–23.CrossRefGoogle ScholarPubMed
Rothman, KJ. Epidemiology: An Introduction (2nd edn). Oxford University Press, 2012.Google Scholar
Higgins, JPT, Thompson, SG. Quantifying heterogeneity in a meta-analysis. Stat Med 2002; 21: 1539–58.CrossRefGoogle ScholarPubMed
Egger, M, Smith, GD, Schneider, M, Minder, C. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997; 315: 629–34.CrossRefGoogle ScholarPubMed
Duval, S, Tweedie, R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 2000; 56: 455–63.CrossRefGoogle ScholarPubMed
Agerbo, E, Nordentoft, M, Mortensen, PB. Familial, psychiatric, and socioeconomic risk factors for suicide in young people: nested case-control study. BMJ 2002; 325: 74.CrossRefGoogle ScholarPubMed
Burrell, LV, Mehlum, L, Qin, P. Sudden parental death from external causes and risk of suicide in the bereaved offspring: a national study. J Psychiatr Res 2018; 96: 4956.CrossRefGoogle ScholarPubMed
Christiansen, E, Goldney, RD, Beautrai, AL, Agerbo, E. Youth suicide attempts and the dose–response relationship to parental risk factors: a population-based study. Psychol Med 2011; 41: 313–9.CrossRefGoogle ScholarPubMed
Mittendorfer-Rutz, E, Rasmussen, F, Wasserman, D. Familial clustering of suicidal behaviour and psychopathology in young suicide attempters. Soc Psychiat Epidemiol 2008; 43: 2836.CrossRefGoogle ScholarPubMed
Mittendorfer-Rutz, E, Rasmussen, F, Lange, T. A life-course study on effects of parental markers of morbidity and mortality on offspring's suicide attempt. PLoS One 2012; 7: e51585.CrossRefGoogle Scholar
Niederkrotenthaler, T, Floderus, B, Alexanderson, K, Rasmussen, F, Mittendorfer-Rutz, E. Exposure to parental mortality and markers of morbidity, and the risks of attempted and completed suicide in offspring: an analysis of sensitive life periods. J Epidemiol Community Health 2012; 66: 233–9.CrossRefGoogle ScholarPubMed
Tidemalm, D, Runeson, B, Waern, M, Frisell, T, Carlström, E, Lichtenstein, P, et al. Familial clustering of suicide risk: a total population study of 11.4 million individuals. Psychol Med 2011; 41: 2527–34.CrossRefGoogle ScholarPubMed
Gravseth, HM, Mehlum, L, Bjerkedal, T, Kristensen, P. Suicide in young Norwegians in a life course perspective: population-based cohort study. J Epidemiol Community Health 2010; 64: 407–12.CrossRefGoogle Scholar
Kuramoto, SJ, Runeson, B, Stuart, EA, Lichtenstein, P, Wilcox, HC. Time to hospitalization for suicide attempt by the timing of parental suicide during offspring early development. JAMA Psychiatry 2013; 70: 149–57.CrossRefGoogle ScholarPubMed
Lee, K-Y, Li, C-Y, Chang, K-C, Lu, TH, Chen, YY. Age at exposure to parental suicide and the subsequent risk of suicide in young people. Crisis 2018; 39: 2736.CrossRefGoogle ScholarPubMed
von Borczyskowski, A, Lindblad, F, Vinnerljung, B, Reintjes, R, Hjern, A. Familial factors and suicide: an adoption study in a Swedish national cohort. Psychol Med 2011; 41: 749–58.CrossRefGoogle Scholar
Gureje, O, Oladeji, B, Hwang, I, Chiu, WT, Kessler, RC, Sampson, NA, et al. Parental psychopathology and the risk of suicidal behavior in their offspring: results from the world mental health surveys. Mol Psychiatry 2011; 16: 1221–33.CrossRefGoogle ScholarPubMed
Goodman, S, Gotlib, I. Risk for psychopathology in the children of depressed mothers: a developmental model for understanding mechanisms of transmission. Psychol Rev 1999; 106: 458–90.CrossRefGoogle ScholarPubMed
O'Reilly, LM, Kuja-Halkola, R, Rickert, ME, Class, QA, Larsson, H, Lichtenstein, P, et al. The intergenerational transmission of suicidal behavior: an offspring of siblings study. Transl Psychiatry 2020; 10: 173.CrossRefGoogle ScholarPubMed
Bondy, B, Buettner, A, Zill, P. Genetics of suicide. Mol Psychiatry 2006; 11: 336–51.CrossRefGoogle ScholarPubMed
Voracek, M, Loibl, LM. Genetics of suicide: a systematic review of twin studies. Wien Klin Wochenschr 2007; 119: 463–75.CrossRefGoogle ScholarPubMed
Baldessarini, RJ, Hennen, J. Genetics of suicide: an overview. Harv Rev Psychiatry 2004; 12: 113.CrossRefGoogle ScholarPubMed
Docherty, AR, Shabalin, AA, DiBlasi, E, Monson, E, Mullins, N, Adkins, DE, et al. Genome-wide association study of suicide death and polygenic prediction of clinical antecedents. Am J Psychiatry 2020; 177: 917–27.CrossRefGoogle ScholarPubMed
Baldessarini, RJ, Tondo, L, Pinna, M, Nuñez, N, Vázquez, GH. Suicidal risk factors in major affective disorders. Br J Psychiatry 2019; 215: 621–6.CrossRefGoogle Scholar
Brent, DA, Bridge, J, Johnson, BA, Connolly, J. Suicidal behavior runs in families. a controlled family study of adolescent suicide victims. Arch Gen Psychiatry 1996; 53: 1145–52.CrossRefGoogle ScholarPubMed
Brent, DA, Oquendo, M, Birmaher, B, Greenhill, L, Kolko, D, Stanley, B, et al. Peripubertal suicide attempts in offspring of suicide attempters with siblings concordant for suicidal behavior. Am J Psychiatry 2003; 160: 1486–93.CrossRefGoogle ScholarPubMed
Coccaro, EF, Bergeman, CS, Kavoussi, RJ, Seroczynski, AD. Heritability of aggression and irritability: a twin study of the Buss–Durkee aggression scales in adult male subjects. Biol Psychiatry 1997; 41: 273–84.CrossRefGoogle ScholarPubMed
Mann, JJ, Arango, VA, Avenevoli, S, Brent, DA, Champagne, FA, Clayton, P, et al. Candidate endophenotypes for genetic studies of suicidal behavior. Biol Psychiatry 2009; 65: 556–63.CrossRefGoogle ScholarPubMed
Oquendo, MA, Sullivan, GM, Sudol, K, Baca-Garcia, E, Stanley, BH, Sublette, ME, et al. Toward a biosignature for suicide. Am J Psychiatry 2014; 171: 1259–77.CrossRefGoogle Scholar
Iemmi, V, Bantjes, J, Coast, E, Channer, K, Leone, T, McDaid, D, et al. Suicide and poverty in low-income and middle-income countries: a systematic review. Lancet Psychiatry 2016; 3: 774–83.CrossRefGoogle ScholarPubMed
Turecki, G. The molecular bases of the suicidal brain. Nat Rev Neurosci 2014; 15: 802–16.CrossRefGoogle ScholarPubMed
Stenager, K, Qin, P. Individual and parental psychiatric history and risk for suicide among adolescents and young adults in Denmark. Soc Psychiat Epidemiol 2008; 43: 920–6.CrossRefGoogle Scholar
Harris, T, Brown, GW, Bifulco, A. Loss of parent in childhood and adult psychiatric disorder: the role of lack of adequate parental care. Psychol Med 1986; 16: 641–59.CrossRefGoogle ScholarPubMed
Labonté, B, Suderman, M, Maussion, G, Lopez, JP, Navarro-Sánchez, L, Yerko, V, et al. Genome-wide methylation changes in the brains of suicide completers. Am J Psychiatry 2013; 170: 511–20.CrossRefGoogle ScholarPubMed
Brown, AC, Sandler, IN, Tein, J, Liu, X, Haine, RA. Implications of parental suicide and violent death for promotion of resilience of parentally-bereaved children. Death Stud 2007; 31: 301–35.CrossRefGoogle ScholarPubMed
Tillman, JG. The intergenerational transmission of suicide: moral injury and the mysterious object in the work of Walker Percy. J Am Psychoanal Assoc 2016; 64: 541–67.CrossRefGoogle ScholarPubMed
Latham, AE, Prigerson, HG. Suicidality and bereavement: complicated grief as psychiatric disorder presenting greatest risk for suicidality. Suicide Life Threat Behav 2004; 34: 350–62.CrossRefGoogle ScholarPubMed
Li, J, Vestergaard, M, Cnattingius, S, Gissler, M, Bech, BH, Obel, C, et al. Mortality after parental death in childhood: a nationwide cohort study from three Nordic Countries. PLoS Med 2014; 11: (7):e1001679.CrossRefGoogle ScholarPubMed
Carr, MJ, Mok, PLH, Antonsen, S, Pedersen, CB, Webb, RT. Self-harm and violent criminality linked with parental death during childhood. Psychol Med 2020; 50: 1224–32.CrossRefGoogle ScholarPubMed
Mitchell, AM, Kim, Y, Prigerson, HG, Mortimer, MK. Complicated grief and suicidal ideation in adult survivors of suicide. Suicide Life Threat Behav 2005; 35: 498506.CrossRefGoogle ScholarPubMed
Hagström, AS. “Why did he choose to die?”: a meaning-searching approach to parental suicide bereavement in youth. Death Studies 2019; 43: 113–21.CrossRefGoogle Scholar
Burrell, LV, Mehlum, L, Qin, P. Risk factors for suicide in offspring bereaved by sudden parental death from external causes. J Affect Disord 2017; 222: 71–8.CrossRefGoogle ScholarPubMed
Goffman, E. Stigma: Notes on the Management of Spoiled Identity. Penguin, 1963.Google Scholar
Van Orden, KA, Witte, TK, Cukrowicz, KC, Braithwaite, SR, Selby, EA, Joiner, TE Jr. The interpersonal theory of suicide. Psychol Rev 2010; 117: 575600.CrossRefGoogle ScholarPubMed
Ben-Shlomo, Y, Kuh, D. A life course approach to chronic disease epidemiology: conceptual models, empirical challenges and interdisciplinary perspectives. Int J Epidemiol 2002; 31: 285–93.CrossRefGoogle ScholarPubMed
Cantor, CH. Suicide in the Western world. In The International Handbook of Suicide and Attempted Suicide (eds Hawton, K, van Heeringen), K: 928. J Wiley, 2000.CrossRefGoogle Scholar
Lee, K-Y, Li, C-Y, Chang, K-C, Lu, T-H, Chen, Y-Y. Age at exposure to parental suicide and the subsequent risk of suicide in young people. Crisis 2017; 39: 2736.CrossRefGoogle ScholarPubMed
Esterhuizen, TM, Thabane, L. Con: meta-analysis: some key limitations and potential solutions. Nephrol Dial Transplant 2016; 31: 882–5.CrossRefGoogle ScholarPubMed
Figure 0

Fig. 1 PRISMA flowchart.

Figure 1

Table 1 Characteristics of studies included in the systematic review and meta-analyses grouped by study design

Figure 2

Fig. 2 Forest plot 1: main outcome – risk of suicide attempt (SA) and suicide death (SD) in offspring after parental suicide compared with offspring of two living parents.

Figure 3

Fig. 3 Forest plot 2: risk of suicide attempt (SA) and suicide death (SD) in offspring who experienced parental suicide compared with offspring who experienced parental death by other causes. vs., versus.

Supplementary material: PDF

Calderaro et al. supplementary material

Calderaro et al. supplementary material 1

Download Calderaro et al. supplementary material(PDF)
PDF 1.3 MB
Supplementary material: PDF

Calderaro et al. supplementary material

Calderaro et al. supplementary material 2

Download Calderaro et al. supplementary material(PDF)
PDF 52.9 KB
Submit a response

eLetters

No eLetters have been published for this article.