Hostname: page-component-cd9895bd7-dzt6s Total loading time: 0 Render date: 2024-12-27T05:08:26.914Z Has data issue: false hasContentIssue false
  • English
  • Français

General optimal stopping theorems for semi-Markov processes

Part of: Linear inference, regression Markov processes Stochastic processes

Published online by Cambridge University Press:  01 July 2016

Frans A. Boshuizen  and
José M. Gouweleeuw
Frans A. Boshuizen*
Affiliation:
Erasmus Universiteit Rotterdam
José M. Gouweleeuw*
Affiliation:
Vrije Universiteit Amsterdam
*
* Postal address: Econometric Institute, Erasmus Universiteit Rotterdam, PO Box 1738, 3000 DR Rotterdam, The Netherlands.
** Postal address: Department of Mathematics and Computer Science, Vrije Universiteit, De Boelelaan 1081a, 1081 HV Amsterdam, The Netherlands.
Get access Rights & Permissions [Opens in a new window]

Abstract

In this paper, optimal stopping problems for semi-Markov processes are studied in a fairly general setting. In such a process transitions are made from state to state in accordance with a Markov chain, but the amount of time spent in each state is random. The times spent in each state follow a general renewal process. They may depend on the present state as well as on the state into which the next transition is made.

Our goal is to maximize the expected net return, which is given as a function of the state at time t minus some cost function. Discounting may or may not be considered. The main theorems (Theorems 3.5 and 3.11) are expressions for the optimal stopping time in the undiscounted and discounted case. These theorems generalize results of Zuckerman [16] and Boshuizen and Gouweleeuw [3]. Applications are given in various special cases.

The results developed in this paper can also be applied to semi-Markov shock models, as considered in Taylor [13], Feldman [6] and Zuckerman [15].

Keywords

OPTIMAL STOPPING TIMESJOB SEARCHSHOCK MODELS

MSC classification

Primary: 60G40: Stopping times; optimal stopping problems; gambling theory
Secondary: 60J25: Continuous-time Markov processes on general state spaces 62J15: Paired and multiple comparisons
Type
Research Article
Information
Advances in Applied Probability , Volume 25 , Issue 4 , December 1993 , pp. 825 - 846
Copyright
Copyright © Applied Probability Trust 1993 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Research of both authors supported by Fulbright Scholarships.

References

[1] Aven, T. (1987) A counting process approach to replacement models. Optimization 18, 285–296.Google Scholar
[2] Aven, T. and Gaarder, S. (1987) Optimal replacement in a shock model: discrete time. J. Appl. Prob. 24, 281–287.Google Scholar
[3] Boshuizen, F. A. and Gouweleeuw, J. M. (1992) A continuous-time job search model: general renewal processes. Report 9247/A Econometric Institute, Erasmus University Rotterdam. Submitted for publication.Google Scholar
[4] BréMaud, P. (1981) Point Processes and Queues: Martingale Dynamics. Springer-Verlag, New York.Google Scholar
[5] Davis, M. H. A. (1976) The representation of martingales of jump processes. SIAM J. Control Optim. 14, 623–638.CrossRefGoogle Scholar
[6] Feldman, R. M. (1976) Optimal replacement with semi-Markov shock models. J. Appl. Prob. 13, 108–117.CrossRefGoogle Scholar
[7] Gihman, I. I. and Skorohod, A. V. (1975) The Theory of Stochastic Processes II. Springer-Verlag, New York.Google Scholar
[8] Kao, E. P. C. (1973) Optimal replacement rules when changes of state are semi-Markovian. Operat. Res. 21, 1231–1249.Google Scholar
[9] Lippman, S. A. and Mccall, J. J. (1976) Job search in a dynamic economy. J. Econ. Theory 12, 365–390.Google Scholar
[10] Neveu, J. (1975) Discrete-Parameter Martingales. North-Holland, Amsterdam.Google Scholar
[11] Ross, S. M. (1970) Applied Probability Models with Optimization Applications. Holden-Day, San Francisco.Google Scholar
[12] Stadje, W. (1991) A new continuous-time search model. J. Appl. Prob. 28, 771–778.Google Scholar
[13] Taylor, H. M. (1975) Optimal replacement under additive damage and other failure models. Naval Res. Logist. Quart. 22, 1–18.Google Scholar
[14] Williams, D. (1979) Diffusions, Markov Processes, and Martingales; Volume 1: Foundations. Wiley, New York.Google Scholar
[15] Zuckerman, D. (1978) Optimal stopping in a semi-Markov shock model. J. Appl. Prob. 15, 629–634.Google Scholar
[16] Zuckerman, D. (1983) Job search: the continuous case. J. Appl. Prob. 20, 637–648.Google Scholar