Hostname: page-component-5f745c7db-q8b2h Total loading time: 0 Render date: 2025-01-06T23:20:38.620Z Has data issue: true hasContentIssue false
  • English
  • Français

EM and Beyond

Published online by Cambridge University Press:  01 January 2025

Donald B. Rubin
Donald B. Rubin*
Affiliation:
Department of Statistics, Harvard University
*
Requests for reprints should be sent to Donald B. Rubin, Department of Statistics, Harvard University, One Oxford Street, Cambridge, MA, 02138.
Get access Rights & Permissions [Opens in a new window]

Abstract

The basic theme of the EM algorithm, to repeatedly use complete-data methods to solve incomplete data problems, is also a theme of several more recent statistical techniques. These techniques—multiple imputation, data augmentation, stochastic relaxation, and sampling importance resampling—combine simulation techniques with complete-data methods to attack problems that are difficult or impossible for EM.

Keywords

data augmentationGibbs samplermissing dataMonte Carlo methodsmultiple imputationsampling importance resamplingsimulation techniquesstochastic relaxation
Type
Original Paper
Information
Psychometrika , Volume 56 , Issue 2 , June 1991 , pp. 241 - 254
Copyright
Copyright © 1991 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

A preliminary version of this article was the Keynote Address at the 1987 European Meeting of the Psychometric Society June 24–26, 1987 in Enschede, The Netherlands. The author wishes to thank the editor and reviewers for helpful comments.

References

Baum, L. E., Petrie, T., Soules, G., & Weiss, N. (1970). A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Annals of Mathematical Statistics, 41, 164–171.CrossRefGoogle Scholar
Beale, E. M. L., & Little, R. J. A. (1975). Missing values in multivariate analysis. Journal of the Royal Statistical Society, Series B, 37, 129–145.CrossRefGoogle Scholar
Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood of item parameters: Application of an EM algorithm. Psychometrika, 46, 443–459.CrossRefGoogle Scholar
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood estimation from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society, Series B, 39, 1–38.CrossRefGoogle Scholar
Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Annals of Statistics, 2, 1–26.Google Scholar
Everitt, B. S. (1984). An introduction to latent variable models, London: Chapman and Hall.CrossRefGoogle Scholar
Gelfand, A. E., Hills, S. E., & Racine-Poon, A., Smith, A. F. M. (1990). Illustration of Bayesian inference in normal data models using Gibbs sampling. Journal of the American Statistical Association, 85, 398–409.CrossRefGoogle Scholar
Gelfand, A. E., & Smith, A. F. M. (1990). Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association, 85, 398–409.CrossRefGoogle Scholar
Gelman, A. E. (1990). Topics in image reconstruction for emission tomography. Unpublished doctoral dissertation, Department of Statistics, Harvard University.Google Scholar
Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741.CrossRefGoogle ScholarPubMed
Geman, S., & McClure, D. E. (1985). Bayesian image analysis: an application to single photon emission tomography. Proceedings of the American Statistical Association—Statistical Computing Section, 12–18.Google Scholar
Goodman, L. A. (1979). On the estimation of parameters in latent structure analysis. Psychometrika, 44, 123–128.CrossRefGoogle Scholar
Hartley, H. O. (1958). Maximum likelihood estimation from incomplete data. Biometrics, 14, 174–194.CrossRefGoogle Scholar
Heitjan, D. F., & Little, R. J. A. (1991). Multiple imputation for the fatal accident reporting system. Applied Statistics, 40, 13–29.CrossRefGoogle Scholar
Johnson, V. E., Wong, W. H., Hu, X., & Chen, C.-T. (1989). Statistical aspects of image restoration, Chicago: University of Chicago, Department of Statistics.Google Scholar
Kennedy, W. J., & Gentle, J. E. (1980). Statistical computing, New York: Marcel Dekker.Google Scholar
Kleijnen, J. P. C. (1974). Statistic Techniques in Simulation—Part I, New York: Marcel Dekker.Google Scholar
Levitan, E., & Herman, G. T. (1987). A maximum a posteriori probability expectation maximization algorithm for image reconstruction in emission tomography. IEEE Transactions on Medical Imaging, 6, 185–192.CrossRefGoogle ScholarPubMed
Li, K. H. (1988). Imputation using Markov chains. Journal of Statistical Computation and Simulation, 30, 57–79.CrossRefGoogle Scholar
Little, R. J. A., & Rubin, D. B. (1987). Statistical analysis with missing data, New York: Wiley.Google Scholar
Meng, X. L., & Rubin, D. B. (1991). Using EM to obtain asymptotic variance-covariance matrices: The SEM algorithm. Journal of the American Statistical Association.CrossRefGoogle Scholar
McKendrick, A. G. (1926). Applications of mathematics to medical problems. Proceedings of the Edinburgh Mathematical Society, 44, 98–130.CrossRefGoogle Scholar
Mislevy, R. J. (1984). Estimating latent distributions. Psychometrika, 49, 359–381.CrossRefGoogle Scholar
Mislevy, R. J. (1991). Randomization-based inferences about latent variables from complex samples. Psychometrika, 56, 177–196.CrossRefGoogle Scholar
Mislevy, R. J., & Sheehan, K. M. (1987). Marginal estimation procedures. In Beaton, A. E. (Eds.), The NAEP 1983/84 Technical Report (pp. 121–211). Princeton: Educational Testing Service.Google Scholar
Orchard, T., & Woodbury, M. A. (1972). A missing information principle: Theory and applications. Proceedings of the 6th Berkeley Symposium on Mathematical Statistics and Probability, Volume 1 (pp. 697–715). Berkeley, CA: The University of California Press.Google Scholar
Ripley, B. D. (1987). Stochasic simulation, New York: Wiley.CrossRefGoogle Scholar
Rubin, D. B. (1976). Inference and missing data. Biometrika, 63, 581–592.CrossRefGoogle Scholar
Rubin, D. B. (1978). Multiple imputations in sample surveys—a phenomenological Bayesian approach to nonresponse. The Proceedings of the Survey Research Methods Section of the American Statistical Association, 20–34.Google Scholar
Rubin, D. B. (1984). Bayesianly justifiable and relevant frequency calculations for the applied statistician. Annals of Statistics, 12, 1151–1172.CrossRefGoogle Scholar
Rubin, D. B. (1985). EM for PET. Discussion of “A statistical model for positron emission tomography” by Vardi, Shepp, and Kaufman. Journal of the American Statistical Association, 80, 31–32.Google Scholar
Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys, New York: Wiley.CrossRefGoogle Scholar
Rubin, D. B. (1987). The SIR algorithm. Discussion of “The calculation of posterior distributions by data augmentation” by Tanner and Wong. Journal of the American Statistical Association, 82, 543–546.Google Scholar
Rubin, D. B. (1988). Using the SIR algorithm to simulate posterior distributions. In Bernardo, J. W., DeGroot, M. H., Lindley, D. V. & Smith, A. F. M. (Eds.), Bayesian Statistics-3 (pp. 395–402). Oxford: Oxford University Press.Google Scholar
Rubin, D. B., & Schafer, J. S. (1991). Efficiently creating multiple imputations for incomplete multivariate normal data. Proceedings of the Computing Research Section of the American Statistical Association.Google Scholar
Rubin, D. B., & Schenker, N. (1991). Multiple imputation in health-care databases: An overview and some applications. Statistics in Medicine, 10, 585–598.CrossRefGoogle ScholarPubMed
Rubin, D. B., & Thayer, D. T. (1982). EM algorithms for maximum likelihood factor analysis. Psychometrika, 47, 69–76.CrossRefGoogle Scholar
Rubin, D. B., & Thayer, D. T. (1983). More on EM for ML factor analysis. Psychometrika, 48, 253–257.CrossRefGoogle Scholar
Sanathanan, L., & Blumenthal, S. (1978). The logistic model and estimation of latent structure. Journal of the American Statistical Association, 73, 794–799.CrossRefGoogle Scholar
Silverman, B. W., Jones, M. C., Wilson, J. D., & Nycha, D. W. (1990). A smoothed EM approach to indirect estimation problems, with particular reference to stereology and emission tomography. Journal of the Royal Statistical Society, Series B, 52, 271–324.CrossRefGoogle Scholar
Sundberg, R. (1974). Maximum likelihood theory for incomplete data from an exponential family. Scandinavian Journal of Statistics, 1, 49–58.Google Scholar
Tanner, M., & Wong, W. (1987). The calculation of posterior distributions by data augmentation. Journal of the American Statistical Association, 82, 528–558.CrossRefGoogle Scholar
Thissen, D. (1982). Marginal maximum likelihood estimation for the one-parameter logistic model. Psychometrika, 74, 175–186.CrossRefGoogle Scholar
Tsutakawa, R. K. (1984). Estimation of two-parameter logistic item response curves. Journal of Educational Statistics, 9, 263–276.CrossRefGoogle Scholar
Vardi, Y., Shepp, L. A., & Kaufman, L. (1985). A statistical model for position emission tomography. Journal of the American Statistical Association, 80, 8–37.CrossRefGoogle Scholar
Wu, C. F. J. (1983). On the convergence properties of the EM algorithm. Annals of Statistics, 11, 95–103.CrossRefGoogle Scholar