Hostname: page-component-5f745c7db-szhh2 Total loading time: 0 Render date: 2025-01-06T07:53:55.232Z Has data issue: true hasContentIssue false
  • English
  • Français

A Multimethod Latent State-Trait Model for Structurally Different And Interchangeable Methods

Published online by Cambridge University Press:  01 January 2025

Tobias Koch ,
Martin Schultze ,
Jana Holtmann ,
Christian Geiser  and
Michael Eid
Tobias Koch*
Affiliation:
Leuphana Universität Lüneburg
Martin Schultze
Affiliation:
Freie Universität Berlin
Jana Holtmann
Affiliation:
Freie Universität Berlin
Christian Geiser
Affiliation:
Utah State University
Michael Eid
Affiliation:
Freie Universität Berlin
*
Correspondence should be made to Tobias Koch, Leuphana Universität Lüneburg, Lüneburg, Germany. Email: tobias.koch@leuphana.de; URL: http://www.leuphana.de/zentren/methodenzentrum.html
Get access Rights & Permissions [Opens in a new window]

Abstract

A new multiple indicator multilevel latent state-trait (LST) model for the analysis of multitrait–multimethod–multioccasion (MTMM-MO) data is proposed. The LST-COM model combines current CFA-MTMM modeling approaches of interchangeable and structurally different methods and LST modeling approaches. The model enables researchers to specify construct and method factors on the level of time-stable (trait) as well as time-variable (occasion-specific) latent variables and analyze the convergent and discriminant validity among different rater groups across time. The statistical performance of the model is scrutinized by a simulation study and guidelines for empirical applications are provided.

Keywords

latent state-trait (LST) theoryCFA-MTMMmultilevel structural equation modelingstructurally different methodsinterchangeable methods
Type
Original Paper
Information
Psychometrika , Volume 82 , Issue 1 , March 2017 , pp. 17 - 47
Copyright
Copyright © 2016 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Electronic supplementary material The online version of this article (doi:10.1007/s11336-016-9541-x) contains supplementary material, which is available to authorized users.

References

Bentler, P., & Chou, C.-P. (1987). Practical issues in structural modeling. Sociological Methods & Research, 17, 78–117. doi: 10.1177/0049124187016001004CrossRefGoogle Scholar
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait–multimethod matrix. Psychological Bulletin, 56, (2), 81–105. doi: 10.1037/h0046016CrossRefGoogle ScholarPubMed
Ciesla, J. A., Cole, D. A., & Steiger, J. H. (2007). Extending the trait-state-occasion model: How important is within-wave measurement equivalence?. Structural Equation Modeling, 14, (1), 77–97. doi: 10.1080/10705510709336737.Google Scholar
Cole, D. A., Martin, N. M., & Steiger, J. H. (2005). Empirical and conceptual problems with longitudinal trait-state models: Introducing a trait-state-occasion model. Psychological Methods, 10, 3–20. doi: 10.1037/1082-989X.10.1.3.CrossRefGoogle ScholarPubMed
Courvoisier, D. S. (2006). Unfolding the constituents of psychological scores: Development and application of mixture and multitrait–multimethod lst models. Doctoral dissertation, University of Geneva. Retrieved from https://archive-ouverte.unige.ch/unige:447.Google Scholar
Courvoisier, D. S., Eid, M., & Nussbeck, F. W. (2007). Mixture distribution latent state-trait analysis: Basic ideas and applications. Psychological Methods, 12, (1), 80–104. doi:10.1037/1082-989X.12.1.80.CrossRefGoogle ScholarPubMed
Courvoisier, D. S., Nussbeck, F. W., Eid, M., Geiser, C., & Cole, D. A. (2008). Analyzing the convergent and discriminant validity of states and traits: Development and applications of multimethod latent state-trait models. Psychological Assessment, 20, (3), 270–280. doi:10.1037/a0012812.CrossRefGoogle Scholar
Crayen, C. (2008). Eine Monte-Carlo Simulationsstudie zur Untersuchung der Anwendbarkeit von Strukturgleichungsmodellen für Multitrait-Multimethod-Multioccasion Daten. [A simulation study for investigating the applicability of structural equation models for multitrait-multimethod-multioccasion data]. Diploma Thesis, Freie Universität Berlin.Google Scholar
Dumenci, L., Brown, S. D., & Tinsley, H. E. (2000). Multitrait–multimethod analysis. Handbook of applied multivariate statistics and mathematerial modeling, San Diego, CA: Academic Press 583–611.CrossRefGoogle Scholar
Eid, M. (1995). Modelle der Messung von Personen in Situationen [Models for measuring persons in situations]. Weinheim: Psychologie Verlags Union.Google Scholar
Eid, M. (1996). Longitudinal confirmatory factor analysis for polytomous item responses: Model definition and model selection on the basis of stochastic measurement theory. Methods of Psychological Research Online, 1(4), 65–85. Retrieved from http://www.dgps.de/fachgruppen/methoden/mpr-online/issue1/art4/eid.pdf.Google Scholar
Eid, M. (2000). A multitrait–multimethod model with minimal assumptions. Psychometrika, 65, (2), 241–261. doi:10.1007/BF02294377.CrossRefGoogle Scholar
Eid, M., Courvoisier, D., Lischetzke, T., Mehl, M. R., & Connor, T. (2012). Structural equation modeling of ambulatory assessment data. Handbook of research methods for studying daily life, New York: Guilford 384–406.Google Scholar
Eid, M., Geiser, C., & Koch, T. (2016). Measuring method effects: From traditional to design-oriented approaches. Current Direction in Psychological Science, 25(4), 275–280. doi: 10.1177/0963721416649624.CrossRefGoogle Scholar
Eid, M., & Hoffmann, L. (1998). Measuring variability and change with an item response model for polytomous variables. Journal of Educational and Behavioral Statistics, 23, 193–215. doi:10.3102/10769986023003193.CrossRefGoogle Scholar
Eid, M., & Langeheine, R. (1999). The measurement of consistency and occasion specificity with latent class models: A new model and its application to the measurement of affect. Psychological Methods, 4, (1), 100–116. doi:10.1037/1082-989X.4.1.100.CrossRefGoogle Scholar
Eid, M., Langeheine, R., Ong, A., & van Dulmen, M. (2007). Detecting population heterogeneity in stability and change of subjective well-being by mixture distribution models. Handbook of methods in positive psychology, Oxford: Oxford University Press 501–607.Google Scholar
Eid, M., Lischetzke, T., Nussbeck, F. W., & Trierweiler, L. I. (2003). Separating trait effects from trait-specific method effects in multitrait–multimethod models: A multiple-indicator CT-C(M-1) model. Psychological Methods, 8, (1), 38–60. doi:10.1037/1082-989X.8.1.38.CrossRefGoogle ScholarPubMed
Eid, M., Nussbeck, F. W., Geiser, C., Cole, D. A., Gollwitzer, M., & Lischetzke, T. (2008). Structural equation modeling of multitrait-multimethod data: Different models for different types of methods. Psychological Methods, 13, (3), 230–253. doi:10.1037/a0013219.CrossRefGoogle ScholarPubMed
Eid, M., Nussbeck, F. W., Lischetzke, T., Petermann, F., & Eid, M. (2006). Multitrait–multimethod-analyse. Handbuch der Psychologischen Diagnostik, Göttingen: Hogrefe 332–345.Google Scholar
Eid, M., Schneider, C., & Schwenkmezger, P. (1999). Do you feel better or worse? The validity of perceived deviations of mood states from mood traits. European Journal of Personality, 13, (4), 283–306. doi:10.1002/(SICI)1099-0984(199907/08)13:43.0.CO;2-0.3.0.CO;2-0>CrossRefGoogle Scholar
Fleenor, J. W., McCauley, C. D., & Brutus, S. (1997). Self-other rating agreement and leader effectiveness. The Leadership Quarterly, 7, (4), 487–506. doi:10.1016/S1048-9843(96)90003-X.CrossRefGoogle Scholar
Geiser, C. (2009). Multitrait–multimethod–multioccasion modeling, München: Akademischer Verlag München.Google Scholar
Geiser, C., Eid, M., Nussbeck, F., Courvoisier, D. S., & Cole, D. A. (2010). Multitrait-multimethod change modelling. Advanced Statistical Analysis, 94, 185–201. doi:10.1007/s10182-010-0127-0.CrossRefGoogle Scholar
Geiser, C., Eid, M., & Nussbeck, F. W. (2008). On the meaning of the latent variables in the CT-C(M-1) model: A comment on Maydeu-Olivares and Coffman (2006). Psychological Methods, 13, (1), 49–57. doi:10.1037/1082-989X.13.1.49.CrossRefGoogle Scholar
Geiser, C., Eid, M., West, S. G., Lischetzke, T., & Nussbeck, F. W. (2012). A comparison of method effects in two confirmatory factor models for structurally different methods. Structural Equation Modeling, 19, (3), 409–436. doi:10.1080/10705511.2012.687658.CrossRefGoogle Scholar
Geiser, C., Keller, B., Lockhart, G., Eid, M., Cole, D., & Koch, T. , (2015). Distinguishing state variability from trait change in longitudinal data: The role of measurement (non)invariance in latent state-trait analyses. Behavior Research Methods, doi:10.3758/s13428-014-0457-z.CrossRefGoogle ScholarPubMed
Geiser, C., Koch, T., & Eid, M. (2014). Data-generating mechanisms versus constructively defined latent variables in multitrait–multimethod analysis: A comment on Castro-Schilo, Widaman, and Grimm (2013). Structural Equation Modeling: A Multidisciplinary Journal, 21, (4), 509–523. doi:10.1080/10705511.2014.919816.CrossRefGoogle ScholarPubMed
Geiser, C., & Lockhart, G. (2012). A comparison of four approaches to account for method effects in latent state-trait analyses. Psychological Methods, 17, (2), 255–283. doi:10.1037/a0026977.CrossRefGoogle ScholarPubMed
Ghorpade, J. (2000). Managing five paradoxes of 360-degree feedback. The Academy of Management Executive, 14, (1), 140–150. doi:10.5465/AME.2000.2909846.Google Scholar
Heck, R. H., Thomas, S. L., & Tabata, L. N. (2013). Multilevel and longitudinal modeling with IBM SPSS, New York: Routledge.CrossRefGoogle Scholar
Hertzog, C., & Nesselroade, J. R. (1987). Beyond autoregressive models: Some implications of the trait-state distinction for the structural modeling of developmental change. Child Development, 58, (1), 93–109. doi:10.2307/1130294.CrossRefGoogle ScholarPubMed
Hox, J. J., & Maas, C. J. (2001). The accuracy of multilevel structural equation modeling with pseudobalanced groups and small samples. Structural Equation Modeling, 8, (2), 157–174. doi:10.1207/S15328007SEM08021.CrossRefGoogle Scholar
Hu, L.-T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, (1), 1–55. doi:10.1080/10705519909540118.CrossRefGoogle Scholar
Kenny, D. A. (1976). An empirical application of confirmatory factor analysis to the multitrait–multimethod matrix. Journal of Experimental Social Psychology, 12, (3), 247–252. doi:10.1016/0022-1031(76)90055-X.CrossRefGoogle Scholar
Kenny, D. A., Shrout, P., & Fiske, S. T. (1995). The multitrait–multimethod matrix: Design, analysis, and conceptual issues. Personality research, methods, and theory: A festschrift honoring Donald W. Fiske, Hillsdale, NJ: Lawrence Erlbaum 111–124.Google Scholar
Kenny, D. A., & Kashy, D. A. (1992). Analysis of the multitrait-multimethod matrix by confirmatory factor analysis. Psychological Bulletin, 112, (1), 165–172.CrossRefGoogle Scholar
Koch, T., Eid, M., & Lochner, K. (in press). Multitrait-multimethod-analysis: The psychometric foundation of CFA-MTMM models. In P. Irwing, T. Booth, & D. Hughes (Eds.), The Wiley handbook of psychometric testing. Wiley.Google Scholar
Koch, T., Ortner, T. M., Eid, M., Caspers, J., & Schmitt, M. (2014a). Evaluating the construct validity of objective personality tests using a multitrait–multimethod–multioccasion-(MTMM-MO)-approach. European Journal of Psychological Assessment, 30(3), 208–230. doi:10.1027/1015-5759/a000212.CrossRefGoogle Scholar
Koch, T., Schultze, M., Burrus, J., Roberts, R. D., & Eid, M. (2015). A multilevel CFA-MTMM model for nested structurally different methods. Journal of Educational and Behavioral Statistics, 40, (5), 477–510. 10.3102/1076998615606109.CrossRefGoogle Scholar
Koch, T., Schultze, M., Eid, M., & Geiser, C. (2014b). A longitudinal multilevel CFA-MTMM model for interchangeable and structurally different methods. Frontiers in Quantative Psychology and Measurement, 5, 311. doi:10.3389/fpsyg.2014.00311.CrossRefGoogle Scholar
Koch, T., Schultze, M., Jeon, M., Nussbeck, F. W., Praetorius, A.-K., & Eid, M. (2016). A cross-classified CFA-MTMM model for structurally different and nonindependent interchangeable methods. Multivariate Behavioral Research, 51, (1), 67–85. doi:10.1080/00273171.2015.1101367. (PMID: 26881958)CrossRefGoogle ScholarPubMed
Little, T. D., Schnabel, K. U., & Baumert, J. (2000). Modeling longitudinal and multilevel data, Mahwah, NJ: Lawrence Erlbaum.CrossRefGoogle Scholar
Lüdtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., & Muthén, B. (2008). The multilevel latent covariate model: A new, more reliable approach to group-level effects in contextual studies. Psychological Methods, 13, (3), 203. doi:10.1037/a0012869.CrossRefGoogle Scholar
Maas, CJM, & Hox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology. European Journal of Research Methods for the Behavioral and Social Sciences, 1, 85–91. doi:10.1027/1614-1881.1.3.86.CrossRefGoogle Scholar
Mahlke, J., Schultze, M., Koch, T., Eid, M., Eckert, R., & Brodbeck, F. C. (2016). A multilevel cfa-mtmm approach for multisource feedback instruments: Presentation and application of a new statistical model. Structural Equation Modeling: A Multidisciplinary Journal, 23, (1), 91–110.CrossRefGoogle Scholar
Marsh, H. W. (1989). Confirmatory factor analyses of multitrait-multimethod data: Many problems and a few solutions. Applied Psychological Measurement, 13, (4), 335–361. doi:10.1177/014662168901300402.CrossRefGoogle Scholar
Marsh, H. W., & Grayson, D., Hoyle, R. H. (1995). Latent variable models of multitrait–multimethod data. Structural equation modeling: Concepts, issues, and applications, Thousand Oaks, CA: Sage 177–198.Google Scholar
Marsh, H. W., & Hocevar, D. (1988). A new, more powerful approach to multitrait-multimethod analyses: Application of second-order confirmatory factor analysis. Journal of Applied Psychology, 73, (1), 107–117. doi:10.1037/0021-9010.73.1.107.CrossRefGoogle Scholar
Meuleman, B., & Billiet, J. (2009). A Monte Carlo sample size study: How many countries are needed for accurate multilevel SEM?. Survey Research Methods, 3, (1), 45–58.Google Scholar
Pohl, S., & Steyer, R. (2010). Modeling common traits and method effects in multitrait-multimethod analysis. Multivariate Behavioral Research, 45, (1), 45–72. doi:10.1080/00273170903504729.CrossRefGoogle ScholarPubMed
Pohl, S., Steyer, R., & Kraus, K. (2008). Modelling method effects as individual causal effects. Journal of the Royal Statistical Society: Series A (Statistics in Society), 171, (1), 41–63. doi:10.1111/j.1467-985X.2007.00517.x.CrossRefGoogle Scholar
Rabe-Hesketh, S., & Skrondal, A. (2004). Generalized multilevel structural equation modeling. Psychometrica, 69, 167–190. doi:10.1007/BF02295939.CrossRefGoogle Scholar
Ryu, E. (2014). Model fit evaluation in multilevel structural equation models. Frontiers in Psychology, 5(81). doi:10.3389/fpsyg.2014.00081.CrossRefGoogle Scholar
Schermelleh-Engel, K., Keith, N., Moosbrugger, H., & Hodapp, V. (2004). Decomposing person and occasion-specific effects: An extension of latent state-trait (LST) theory to hierarchical LST models. Psychological Methods, 9, (2), 198. doi:10.1007/s10802-011-9547-x.CrossRefGoogle ScholarPubMed
Scherpenzeel, A., Saris, W., van Montfort, K., Oud, J., & Satorra, A. (2007). Multitrait–multimethod models for longitudinal research. Longitudinal models in the behavioral and related sciences, Mahwah, NJ: Lawrence Erlbaum 381–401.Google Scholar
Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence, Oxford: Oxford University Press.CrossRefGoogle Scholar
Steele, F. (2008). Multilevel models for longitudinal data. Journal of the Royal Statistical Society: Series A (Statistics in Society), 171, (1), 5–19. doi:10.1111/j.1467-985X.2007.00509.x.CrossRefGoogle Scholar
Steyer, R. (1988). Experiment, Regression und Kausalität. Die logische Struktur kausaler Regressionsmodelle [Experiment, regression, and causality. The logical structure of causal regression models]. Habilitation Thesis, Universität Trier, Germany.Google Scholar
Steyer, R., & Eid, M. (2001). Messen und Testen [Measurement and testing] (2nd ed.). Heidelberg: Springer.Google Scholar
Steyer, R., Ferring, D., & Schmitt, M. J. (1992). States and traits in psychological assessment. European Journal of Psychological Assessment, 8, (2), 79–98.Google Scholar
Steyer, R., Majcen, A.-M., Schwenkmezger, P., & Buchner, A. (1989). A latent state-trait anxiety model and its application to determine consistency and specificity coefficients. Anxiety Research, 1, (4), 281–299. doi:10.1080/08917778908248726.CrossRefGoogle Scholar
Steyer, R., Mayer, A., Geiser, C., & Cole, D. A. (2015). A theory of states and traits—Revised. Annual Review of Clinical Psychology, 11, (1), 71–98. doi:10.1146/annurev-clinpsy-032813-153719. (PMID: 25062476).CrossRefGoogle ScholarPubMed
Steyer, R., Schmitt, M., & Eid, M. (1999). Latent state-trait theory and research in personality and individual differences. European Journal of Personality, 13, (5), 389–408. doi:10.1002/(SICI)1099-0984(199909/10)13:53.0.CO;2-A.3.0.CO;2-A>CrossRefGoogle Scholar
Steyer, R., & Schmitt, M. J. (1990). The effects of aggregation across and within occasions on consistency, specificity and reliability. Methodika, 4, 58–94.Google Scholar
Steyer, R., Schmitt, T., Eye, A., & Clogg, C. C. (1994). The theory of confounding and its application in causal modeling with latent variables. Latent variable analysis: Applications for developmental research, Thousand Oaks, CA: Sage 36–67.Google Scholar
Vautier, S. (2004). A longitudinal sem approach to STAI data: Two comprehensive multitrait–multistate models. Journal of Personality Assessment, 83, (2), 167–179.CrossRefGoogle ScholarPubMed
Widaman, K. F. (1985). Hierarchically nested covariance structure models for multitrait–multimethod data. Applied Psychological Measurement, 9, (1), 1–26. doi:10.1177/014662168500900101.CrossRefGoogle Scholar
Wothke, W., Shrout, P. E., & Fiske, S. T. (1995). Covariance components analysis of the multitrait–multimethod matrix. Personality research, methods, and theory: A festschrift honoring Donald W. Fiske, Hillsdale, NJ: Lawrence Erlbaum 125–144.Google Scholar
Yammarino, F. J. (2003). Modern data analytic techniques for multisource feedback. Organizational Research Methods, 6, (1), 6–14. doi:10.1177/1094428102239423.CrossRefGoogle Scholar
Yammarino, F. J., & Atwater, L. E. (1997). Do managers see themselves as others see them? implications of self-other rating agreement for human resources management. Organizational Dynamics, 25, (4), 35–44. doi:10.1016/S0090-2616(97)90035-8.CrossRefGoogle Scholar
Zimmerman, D. W. (1975). Probability spaces, hilbert spaces, and the axioms of test theory. Psychometrika, 40, 395–412. doi:10.1007/BF02291765.CrossRefGoogle Scholar
Supplementary material: File

Koch et al. supplementary material

Supplemental Text
Download Koch et al. supplementary material(File)
File 156.9 KB