Hostname: page-component-78c5997874-94fs2 Total loading time: 0 Render date: 2024-11-10T16:50:25.419Z Has data issue: false hasContentIssue false
  • English
  • Français

Equivalent necessary and sufficient conditions on noise sequences for stochastic approximation algorithms

Part of: Numerical approximation and computational geometry Sequential methods

Published online by Cambridge University Press:  01 July 2016

I-Jeng Wang ,
Edwin K. P. Chong  and
Sanjeev R. Kulkarni
I-Jeng Wang*
Affiliation:
Purdue University
Edwin K. P. Chong*
Affiliation:
Purdue University
Sanjeev R. Kulkarni*
Affiliation:
Purdue University
*
∗ Postal address: School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907-1285, USA.
∗ Postal address: School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907-1285, USA.
∗∗ Postal address: Department of Electrical Engineering, Princeton University, Princeton, NJ 08544, USA.
Get access Rights & Permissions [Opens in a new window]

Abstract

We consider stochastic approximation algorithms on a general Hilbert space, and study four conditions on noise sequences for their analysis: Kushner and Clark's condition, Chen's condition, a decomposition condition, and Kulkarni and Horn's condition. We discuss various properties of these conditions. In our main result we show that the four conditions are all equivalent, and are both necessary and sufficient for convergence of stochastic approximation algorithms under appropriate assumptions.

Keywords

STOCHASTIC APPROXIMATIONCONVERGENCE: EQUIVALENT NECESSARY AND SUFFICIENT CONDITIONSNOISE SEQUENCES

MSC classification

Primary: 62L20: Stochastic approximation
Secondary: 62L12: Sequential estimation 65D99: None of the above, but in this section
Type
General Applied Probablity
Information
Advances in Applied Probability , Volume 28 , Issue 3 , September 1996 , pp. 784 - 801
Copyright
Copyright © Applied Probability Trust 1996 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This research was supported in part by a Purdue Research Foundation Fellowship, and by the National Science Foundation through grant ECS-9410313.

This research was supported in part by the National Science Foundation under grant IRI-9457645 and by the Army Research Office under grant DAAL03-92-G-0320.

References

[1] Bartusek, J. D. and Makowski, A. M. (1994) On stochastic approximations driven by sample averages: Convergence results via the ODE method. Manuscript. Electrical Engineering Department and Institute for Systems Research, University of Maryland.Google Scholar
[2] Chen, H.-F. (1994) Stochastic approximation and its new applications. In Proc. 1994 Hong Kong Int. Workshop on New Directions in Control and Manufacturing. pp. 2–12.Google Scholar
[3] Chen, H.-F., Guo, L. and Gao, A. (1988) Convergence and robustness of the Robbins–Monro algorithm truncated at randomly varying bounds. Stoch. Proc. Appl. 27, 217–231.Google Scholar
[4] Chen, H.-F. and Zhu, Y.-M. (1986) Stochastic approximation procedures with randomly varying truncations. Sci. Sinica A 29, 914–926.Google Scholar
[5] Clark, D. S. (1984) Necessary and sufficient conditions for the Robbins–Monro method. Stoch. Proc. Appl. 17, 359–367.Google Scholar
[6] Kulkarni, S. R. and Horn, C. (1994) Necessary and sufficient conditions for convergence of stochastic approximation algorithms under arbitrary disturbances. Manuscript. Department of Electrical Engineering, Princeton University.Google Scholar
[7] Kushner, H. K. and Clark, D. S. (1978) Stochastic Approximation Methods for Constrained and Unconstrained Systems . Springer, New York.Google Scholar
[8] Lai, T. L. (1985) Stochastic approximation and sequential search for optimum. In Proc. Berkeley Conf. in Honor of Jerzy Neyman and Jack Kiefer. ed. LeCam, L. and Olshen, R. A.. Vol. 2. Wadsworth, New York. pp. 557–577.Google Scholar
[9] L'Ecuyer, P. and Glynn, P. W. (1994) Stochastic optimization by simulation: Convergence proofs for the GI/G/1 queue in steady-state: Management Sci. 40, 152–1578.Google Scholar
[10] Ljung, L., Pflug, G. and Walk, H. (1992) Stochastic Approximation and Optimization of Random Systems. Birkhäuser, Basel.CrossRefGoogle Scholar
[11] Ma, D.-J., Makowski, M. and Schwartz, A. (1990) Stochastic approximations for finite state Markov chains. Stoch. Proc. Appl. 35, 27–45.CrossRefGoogle Scholar
[12] Metivier, ?. and Priouret, P. (1984) Applications of a Kushner and Clark lemma to general classes of stochastic algorithms. IEEE Trans. Inf. Theory. IT-30, 140–151.Google Scholar
[13] Robbins, H. and Monro, S. (1951) A stochastic approximation method. Ann. Math. Statist. 22, 400–407.CrossRefGoogle Scholar
[14] Wang, I.-J., Chong, E. K. P. and Kulkarni, S. R. (1994) Necessity of Kushner–Clark Condition for Convergence of Stochastic Approximation Algorithms. In Proc. 32nd Annual Allerton Conf. on Communication, Control and Computing. pp. 167–175.Google Scholar