Hostname: page-component-78c5997874-g7gxr Total loading time: 0 Render date: 2024-11-10T14:53:31.960Z Has data issue: false hasContentIssue false
  • English
  • Français

Circular specifications and “predicting” with information from the future: Errors in the empirical SAOM–TERGM comparison of Leifeld & Cranmer

Published online by Cambridge University Press:  10 March 2022

Per Block Open the ORCID record for Per Block [Opens in a new window] ,
James Hollway Open the ORCID record for James Hollway [Opens in a new window] ,
Christoph Stadtfeld ,
Johan Koskinen  and
Tom Snijders Open the ORCID record for Tom Snijders [Opens in a new window]
Per Block*
Affiliation:
Department of Sociology, Leverhulme Centre for Demographic Science, and Nuffield College, University of Oxford, Oxford, UK
James Hollway
Affiliation:
The Graduate Institute Geneva, Geneva, Switzerland
Christoph Stadtfeld
Affiliation:
Chair of Social Networks, ETH Zürich, Zürich, Switzerland
Johan Koskinen
Affiliation:
Melbourne School of Psychological Sciences, University of Melbourne, Melbourne, Australia
Tom Snijders
Affiliation:
Nuffield College, University of Oxford, Oxford, UK Department of Sociology, University of Groningen, Groningen, The Netherlands
*
*Corresponding author. Email: per.block@sociology.ox.ac.uk
Get access Rights & Permissions [Opens in a new window]

Abstract

We review the empirical comparison of Stochastic Actor-oriented Models (SAOMs) and Temporal Exponential Random Graph Models (TERGMs) by Leifeld & Cranmer in this journal [Network Science 7(1):20–51, 2019]. When specifying their TERGM, they use exogenous nodal attributes calculated from the outcome networks’ observed degrees instead of endogenous ERGM equivalents of structural effects as used in the SAOM. This turns the modeled endogeneity into circularity and obtained results are tautological. In consequence, their out-of-sample predictions using TERGMs are based on out-of-sample information and thereby predict the future using observations from the future. Thus, their analysis rests on erroneous model specifications that invalidate the article’s conclusions. Finally, beyond these specific points, we argue that their evaluation metric—tie-level predictive accuracy—is unsuited for the task of comparing model performance.

Keywords

SAOMERGMTERGMstatistical modelingdynamic networksinferential network analysislongitudinal networks
Type
Commentary
Information
Network Science , Volume 10 , Issue 1 , March 2022 , pp. 3 - 14
Copyright
© The Author(s), 2022. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Block, P., Koskinen, J., Hollway, J., Steglich, C., & Stadtfeld, C. (2018). Change we can believe in: Comparing longitudinal network models on consistency, interpretability and predictive power. Social Networks, 52, 180–191. http://doi.org/10.1016/j.socnet.2017.08.001 CrossRefGoogle Scholar
Block, P., Stadtfeld, C., & Snijders, T. A. (2019). Forms of dependence: Comparing SAOMs and ERGMs from basic principles. Sociological Methods & Research, 48(1), 202–239.CrossRefGoogle Scholar
Desmarais, B. A., & Cranmer, S. J. (2012). Statistical mechanics of networks: Estimation and uncertainty. Physica A: Statistical Mechanics and Its Applications, 391(4), 1865–1876.CrossRefGoogle Scholar
Frank, O., & Strauss, D. (1986). Markov Graphs. Journal of the American Statistical Association, 81, 832–842.CrossRefGoogle Scholar
Handcock, M. (2003). Assessing degeneracy in statistical models of social networks. Tech. rep., Center for Statistics and the Social Sciences, University of Washington, http://www.csss.washington.edu/Papers.Google Scholar
Hanneke, S., Fu, W., & Xing, E. P. (2010). Discrete temporal models of social networks. Electronic Journal of Statistics, 4, 585–605.CrossRefGoogle Scholar
Hunter, D. R., & Handcock, M. S. (2006). Inference in curved exponential family models for networks. Journal of Computational and Graphical Statistics, 15, 565–583.CrossRefGoogle Scholar
Hunter, D. R., Handcock, M. S., Butts, C. T., Goodreau, S. M., & Morris, M. (2008b). ergm: A package to fit, simulate and diagnose exponential-family models for networks. Journal of Statistical Software, 24(3), nihpa54860.CrossRefGoogle Scholar
Jonasson, J. (1999). The random triangle model. Journal of Applied Probability, 36, 852–876.CrossRefGoogle Scholar
Knecht, A. (2008). Friendship selection and friends’ influence: Dynamics of networks and actor attributes in early adolescence. (Ph.D. dissertation). University of Utrecht, Utrecht.Google Scholar
Koskinen, J., Wang, P., Robins, G., & Pattison, P. (2018). Outliers and influential observations in exponential random graph models. Psychometrika, 83(4), 809–830.CrossRefGoogle ScholarPubMed
Krivitsky, P. N., & Handcock, M. S. (2014). A separable model for dynamic networks. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 76(1), 29–46.CrossRefGoogle ScholarPubMed
Krivitsky, P. N., & Handcock, M. S. (2016). tergm: Fit, simulate and diagnose models for network evolution based on exponential-family random graph models. The Statnet Project (http://www.statnet.org). R package version, 3(0).Google Scholar
Leifeld, P., & Cranmer, S. J. (2019a). A theoretical and empirical comparison of the temporal exponential random graph model and the stochastic actor-oriented model. Network Science, 7(1), 20–51. https://doi.org/10.1017/nws.2018.26 CrossRefGoogle Scholar
Leifeld, P., & Cranmer, S. J. (2019b). Replication data for: a theoretical and empirical comparison of the temporal exponential random graph model and the stochastic actor-oriented model. Harvard Dataverse, V1, https://doi.org/10.7910/DVN/NEM2XU Google Scholar
Lerner, J., Indlekofer, N., Nick, B., & Brandes, U. (2013). Conditional independence in dynamic networks. Journal of Mathematical Psychology, 57(6), 275–283.CrossRefGoogle Scholar
Lusher, D., Koskinen, J., & Robins, G. (2013). Exponential random graph models for social networks. Cambridge: Cambridge University Press.Google Scholar
Ripley, R., Snijders, T. A. B., Boda, Z., Vörös, A., & Preciado, P. (2021). Manual for SIENA version 4.0. Oxford: University of Oxford, Department of Statistics, http://www.stats.ox.ac.uk/ snijders/siena/.Google Scholar
Robins, G., & Pattison, P. (2001) Random graph models for temporal processes in social networks. Journal of Mathematical Sociology, 25(1), 5–41.CrossRefGoogle Scholar
Robins, G., Pattison, P., & Wang, P. (2009). Closure, connectivity and degree distributions: Exponential random graph (p*) models for directed social networks. Social Networks, 31(2), 105–117.CrossRefGoogle Scholar
Robins, G., Snijders, T., Wang, P., Handcock, M., & Pattison, P. (2007). Recent developments in exponential random graph (p*) models for social networks. Social networks, 29(2), 192–215.CrossRefGoogle Scholar
Schaefer, D. R. & Marcum, C. S. (2018). Modeling network dynamics. https://doi.org/10.31235/osf.io/6rm9q CrossRefGoogle Scholar
Schweinberger, M. (2011). Instability, sensitivity, and degeneracy of discrete exponential families. Journal of the American Statistical Association, 106(496), 1361–1370.CrossRefGoogle ScholarPubMed
Schweinberger, M., Krivitsky, P. N., Butts, C. T., & Stewart, J. (2020). Exponential-family models of random graphs: inference in finite-, super-, and infinite population scenarios. Statistical Science (forthcoming).CrossRefGoogle Scholar
Snijders, T. A. B. (2002). Markov chain Monte Carlo Estimation of exponential random graph models. Journal of Social Structure, 3, 1–40.Google Scholar
Snijders, T. A. B. (2010). Conditional marginalization for exponential random graph models. Journal of Mathematical Sociology, 34, 239–252.CrossRefGoogle Scholar
Snijders, T. A. B., Pattison, P. E., Robins, G. L., & Handcock, M. S. (2006). New specifications for exponential random graph models. Sociological Methodology, 36, 99–153.CrossRefGoogle Scholar
Strauss, D. (1986). On a general class of models for interaction. SIAM Review, 28, 513–527.CrossRefGoogle Scholar
Wasserman, S., & Pattison, P. (1996). Logit models and logistic regressions for social networks: {I}. An introduction to Markov graphs and p*. Psychometrika, 61(3), 401–425.CrossRefGoogle Scholar