Hostname: page-component-78c5997874-dh8gc Total loading time: 0 Render date: 2024-11-10T22:53:50.594Z Has data issue: false hasContentIssue false
  • English
  • Français

A derivation of the information criteria for selecting autoregressive models

Published online by Cambridge University Press:  01 July 2016

R. J. Bhansali
R. J. Bhansali*
Affiliation:
University of Liverpool
*
∗Postal address: Dept of Statistics and Computational Mathematics, University of Liverpool, Victoria Building, Brownlow Hill, P.O. Box 147, Liverpool L69 3BX, UK.
Get access Rights & Permissions [Opens in a new window]

Abstract

The Akaike information criterion, AIC, for autoregressive model selection is derived by adopting −2T times the expected predictive density of a future observation of an independent process as a loss function, where T is the length of the observed time series. The conditions under which AIC provides an asymptotically unbiased estimator of the corresponding risk function are derived. When the unbiasedness property fails, the use of AIC is justified heuristically. However, a method for estimating the risk function, which is applicable for all fitted orders, is given. A derivation of the generalized information criterion, AICα, is also given; the loss function used being obtained by a modification of the Kullback-Leibler information measure. Results paralleling those for AIC are also obtained for the AICα criterion.

Keywords

AICAICαAUTOREGRESSIVE PROCESSORDER DETERMINATIONKULLBACK–LEIBLER INFORMATION MEASURE
Type
Research Article
Information
Advances in Applied Probability , Volume 18 , Issue 2 , June 1986 , pp. 360 - 387
Copyright
Copyright © Applied Probability Trust 1986 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Aitchison, J. (1975) Goodness of prediction fit. Biometrika 62, 547–554.CrossRefGoogle Scholar
Aitchison, J. and Dunsmore, I. R. (1975) Statistical Prediction Analysis. Cambridge University Press, Cambridge.Google Scholar
Akaike, H. (1970) Statistical predictor identification. Ann. Inst. Statist. Math. 22, 203–217.Google Scholar
Akaike, H. (1973) Information theory and an extension of the maximum likelihood principle. In 2nd Internat. Symp. Information Theory, ed. Petrov, B. N. and Csaki, F., Akademia Kiado, Budapest 267–281.Google Scholar
Akaike, H. (1974) A new look at the statistical model identification. IEEE Trans. Auto. Control 19, 716–723.Google Scholar
Akaike, H. (1977) On entropy maximisation principle. In Proc. Symp. Applied Statistics, ed. Krishnaiah, P. R., North-Holland, Amsterdam, 27–41.Google Scholar
Akaike, H. (1979) A Bayesian extension of the minimum AIC procedure of autoregressive model fitting. Biometrika 66, 237–242.Google Scholar
Akaike, H. (1981) Likelihood of a model and information criteria. J. Econometrics 16, 3–14.Google Scholar
Akaike, H. (1983) Information measures and model selection. Bull. Internat. Statist. Inst. 50 (1), (to appear).Google Scholar
Amaral, M. A. and Dunsmore, I. R. (1980) Optimal estimates of predictive distributions. Biometrika 67, 685–689.Google Scholar
Hong-Zhi, An, Chen, Zhao-Guo and Hannan, E. J. (1982) Autocorrelation, autoregression and autoregressive approximation. Ann. Statist. 10, 926–936.Google Scholar
Anderson, T. W. (1971) The Statistical Analysis of Time Series. Wiley, New York.Google Scholar
Atkinson, A. C. (1980) A note on the generalized information criterion for choice of a model. Biometrika 67, 413–418.Google Scholar
Atkinson, A. C. (1981) Likelihood ratios, posterior odds and information criteria. J. Econometrics 16, 15–20.Google Scholar
Bartlett, M. S. (1966) An Introduction to Stochastic Processes. Cambridge University Press, Cambridge.Google Scholar
Baxter, G. (1962) An asymptotic result for the finite predictor. Math. Scand. 10, 137–144.Google Scholar
Baxter, G. (1963) A norm inequality for a ‘finite-section’ Wiener-Hopf equation. Illinois J. Math. 7, 97–103.CrossRefGoogle Scholar
Bhansali, R. J. (1978) Linear prediction by autoregressive model fitting in the time domain. Ann. Statist. 6, 224–231.Google Scholar
Bhansali, R. J. (1979) A mixed spectrum analysis of the Lynx data. J. R. Statist. Soc A 142, 199–209.Google Scholar
Bhansali, R. J. (1981) Effects of not knowing the order of an autoregressive process–I. J. Amer. Statist. Assoc. 76, 588–597.Google Scholar
Bhansali, R. J. (1986) The criterion autoregressive transfer function of Parzen. J. Time Series Anal. To appear.Google Scholar
Bhansali, R. J. and Downham, D. Y. (1977) Some properties of the order of an autoregressive model selected by a generalization of Akaike&s FPE criterion. Biometrika 64, 547–551.Google Scholar
Brillinger, D. R. (1975) Time Series: Data Analysis and Theory. Holt, Rinehart and Winston, New York.Google Scholar
Chow, G. C. (1981) A comparison of the information and posterior probability criteria for model selection. J. Econometrics 16, 21–33.Google Scholar
Davis, H. T. and Jones, R. H. (1968) Estimation of the innovation variance of a stationary time series. J. Amer. Statist. Assoc, 63, 141–149.Google Scholar
Findley, D. F. (1983) On the unbiasedness property of AIC for exact or approximating multivariate ARMA models. Research Report, Statistical Research Divison, U.S. Bureau of the Census.Google Scholar
Geisser, S. (1977). The inferential use of predictive distributions. In Foundations of Statistical Inference, ed. Godambe, U. P. and Sprott, D. A., Holt, Rinehart and Winston, Toronto, 459–469.Google Scholar
Geisser, S. and Eddy, W. F. (1979) A predictive approach to model selection. J. Amer. Statist. Assoc. 74, 153–160.Google Scholar
Grenander, U. and Rosenblatt, M. (1954) An extension of a theorem of G. Szegö and its application to the study of stochastic processes. Trans. Amer. Math. Soc. 76, 112–126.Google Scholar
Grenander, U. and Rosenblatt, M. (1957) Statistical Analysis of Stationary Time Series. Wiley, New York.Google Scholar
Hannan, E. J. (1980) The estimation of the order of an ARMA process. Ann. Statist. 8, 1071–1081.Google Scholar
Hannan, E. J. and Nicholls, D. F. (1977) The estimation of the prediction error variance. J. Amer. Statist. Assoc. 72, 834–840.Google Scholar
Hannan, E. J. and Quinn, B. G. (1979) The determination of the order of an autoregression. J. R. Statist. Soc. B 41, 190–195.Google Scholar
Inagaki, N. (1977) Two errors in statistical model fitting. Ann. Inst. Statist. Math. A29, 131–152.Google Scholar
Kashyap, R. L. (1977) A Bayesian comparison of different classes of dynamic models using empirical data. IEEE Trans. Auto. Control 22, 715–727.Google Scholar
Kromer, R. E. (1969) Asymptotic properties of the autoregressive spectral estimator. Technical Report No. 13, Stanford University, Department of Statistics.Google Scholar
Ogata, Y. (1980) Maximum likelihood estimates of incorrect Markov models for time series and the derivation of AIC. J. Appl. Prob. 17, 59–72.Google Scholar
Ozaki, T. (1977) On the order determination of ARIMA models. Appl. Statist. 26, 290–301.Google Scholar
Poskitt, D. S. and Tremayne, A. R. (1983) On the posterior odds of time series models. Biometrika 70, 157–162.Google Scholar
Rissanen, J. (1978) Modelling by shortest data description. Automatica 14, 465–471.Google Scholar
Rozanov, Y. A. (1967) Stationary Random Processes. Holden Day, San Francisco.Google Scholar
Sawa, T. (1978) Information criteria for discriminating among alternative regression models. Econometrica 46, 1273–1291.Google Scholar
Schwarz, G. (1978) Estimating the dimension of a model. Ann. Statist. 6, 461–464.Google Scholar
Shibata, R. (1976) Selection of the order of an autoregressive model by Akaike&s information criterion. Biometrika 63, 117–126.Google Scholar
Shibata, R. (1977) Convergence of least squares estimates of autoregressive parameters. Austral. J. Statist. 19, 226–235.Google Scholar
Shibata, R. (1980) Asymptotically efficient selection of the order of the model for estimating parameters of a linear process. Ann. Statist. 8, 147–164.Google Scholar
Shibata, R. (1981) An optimal autoregressive spectral estimate. Ann. Statist. 9, 300–306.Google Scholar
Shibata, R. (1983) A theoretical view of the use of AIC. In Time Series Analysis: Theory and Practice 4, ed. Anderson, O. D., North Holland, Amsterdam, 237–244.Google Scholar
Smith, A. F. M. and Spiegelhalter, D. J. (1980) Bayes factors and choice criteria for linear models. J. R. Statist. Soc. B 42, 213–220.Google Scholar
Stone, M. (1979) Comments on model selection criteria of Akaike and Schwarz. J. R. Statist. Soc. B 41, 276–278.Google Scholar
Tong, H. (1975) Determination of the order of a Markov chain by Akaike&s information criterion. J. Appl. Prob. 12, 488–497.Google Scholar
Walker, A. M. (1964) Asymptotic properties of least-squares estimates of parameters of the spectrum of a stationary non-deterministic time series. J. Austral. Math. Soc. 4, 363–384.Google Scholar
Whittle, P. (1962) Gaussian estimation in stationary time series. Bull. I.S.I. 39, 105–129.Google Scholar
Whittle, P. (1963) Prediction and Regulation by Linear Least-Square Methods. English Universities Press, London.Google Scholar
Yamamoto, T. (1976) Asymptotic mean square prediction error for an autoregressive model with estimated coefficients. Appl. Statist. 25, 123–127.Google Scholar