Hostname: page-component-cb9f654ff-mx8w7 Total loading time: 0 Render date: 2025-08-21T22:46:21.922Z Has data issue: false hasContentIssue false
  • English
  • Français

Countable-state average-cost regenerative stopping problems

Published online by Cambridge University Press:  14 July 2016

Bruce L. Miller
Bruce L. Miller*
Affiliation:
University of California, Los Angeles
*
∗Postal address: Department of System Science, University of California, Los Angeles, CA 90024, U.S.A.
Get access Rights & Permissions [Opens in a new window]

Abstract

Regenerative stopping problems are stopping problems which recommence from the initial state upon stopping. An algorithm is presented which solves semi-Markov regenerative stopping problem with a finite number of continue actions by solving a sequence of stopping problems. New results for the optimal stopping problem are obtained as well as for the regenerative stopping problem. A model in the literature is used as a detailed example of the algorithm.

Keywords

OPTIMAL STOPPINGAVERAGE-COST DYNAMIC PROGRAMMINGREPLACEMENTRENEWAL THEORY

Information

Type
Research Papers
Information
Journal of Applied Probability , Volume 18 , Issue 2 , June 1981 , pp. 361 - 377
Copyright
Copyright © Applied Probability Trust 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

Research supported in part by the Office of Naval Research under Contract N0014–78–C–0428, and the National Science Foundation under Contract ENG–76–122501–A01.

References

[1] Abdel-Hameed, M. (1977) Optimality of the one step look-ahead stopping times. J. Appl. Prob. 14, 162–169.Google Scholar
[2] Bell, C. (1970) Improved algorithms for inventory and replacement-stocking problems. SIAM J. Appl. Math. 18, 558–566.Google Scholar
[3] Bergman, B. (1978) Optimal replacement under a general failure model. Adv. Appl. Prob. 10, 431–451.Google Scholar
[4] Breiman, L. (1964) Stopping-rule problems. Chapter 10 of Applied Combinatorial Mathematics, ed. Beckenbach, E., Wiley, New York.Google Scholar
[5] Brender, D. (1963) A surveillance model for recurrent events. Research Report, IBM Watson Research Center, Yorktown Heights, NY. (Summarized in The Mathematical Theory of Reliability, Barlow, R. and Proschan, F., Wiley, New York, 115–116.)Google Scholar
[6] Buckman, A. G. and Miller, B. (1979) Optimal investigation as a regenerative stopping problem. Working Paper No. 289, Western Management Science Institute, UCLA.Google Scholar
[7] Chow, Y., Robbins, H. and Siegmund, D. (1971) Great Expectations: The Theory of Optimal Stopping. Houghton Mifflin, Boston.Google Scholar
[8] Denardo, E. (1967) Contraction mappings in the theory underlying dynamic programming. SIAM Rev. 9, 165–177.Google Scholar
[9] Denardo, E. (1971) Markov renewal programming with small interest rates. Ann. Math. Statist. 42, 477–496.Google Scholar
[10] Derman, C. and Lieberman, G. (1967) A Markovian decision model for a joint replacement and stocking problem. Management Sci. 13, 609–617.Google Scholar
[11] Derman, C. and Sacks, J. (1960) Replacement of periodically inspected equipment. Naval Res. Logist. Quart. 7, 597–607.Google Scholar
[12] Feldman, R. (1976) Optimal replacement with semi-Markov shock models. J. Appl. Prob. 13, 108–117.Google Scholar
[13] Fox, B. (1967) The existence of pure stationary optimal policies for a class of Markov renewal programs. SIAM Rev. 9, 573–576.Google Scholar
[14] Jewell, W. (1963) Markov renewal programming I, II. Operat. Res. 11, 938–971.Google Scholar
[15] Kaplan, R. (1969) Optimal investigation strategies with imperfect information. J. Accounting Res. 7, 32–43.Google Scholar
[16] Lippman, S. (1971) Maximal average-reward policies for semi-Markov decision processes with arbitrary state and action space. Ann. Math. Statist. 42, 1717–1726.Google Scholar
[17] Pierskalla, W. and Voelker, J. (1976) A survey of maintenance models: the control and surveillance of deteriorating systems. Naval Res. Logist. Quart. 23, 353–388.Google Scholar
[18] Puterman, M. and Brumelle, S. (1979) On the convergence of policy iteration in stationary dynamic programming. Math. Operat. Res. 4, 60–69.Google Scholar
[19] Ross, S. (1970) Average cost semi-Markov decision processes. J. Appl. Prob. 7, 649–656.Google Scholar
[20] Ross, S. (1970) Applied Probability Models with Optimization Applications. Holden-Day, San Francisco.Google Scholar
[21] Taylor, H. (1975) Optimal replacement under additive damage and other failure models. Naval Res. Logist. Quart. 22, 1–18.Google Scholar