Hostname: page-component-cd9895bd7-mkpzs Total loading time: 0 Render date: 2024-12-27T13:08:59.251Z Has data issue: false hasContentIssue false
  • English
  • Français

Reward processes for semi-Markov processes: asymptotic behaviour

Part of: Special processes

Published online by Cambridge University Press:  14 July 2016

A. Reza Soltani  and
K. Khorshidian
A. Reza Soltani*
Affiliation:
Shiraz University
K. Khorshidian*
Affiliation:
Shiraz University
*
∗Postal address: Department of Statistics, Shiraz University, College of Sciences, Shiraz 71454, Iran.
∗Postal address: Department of Statistics, Shiraz University, College of Sciences, Shiraz 71454, Iran.
Get access Rights & Permissions [Opens in a new window]

Abstract

The asymptotic behaviour of the cumulative mean of a reward process 𝒵ρ, where the reward function ρ belongs to a rather large class of functions, is obtained. It is proved that E𝒵ρ(t) = C0 + C1t + o(1), t → ∞, where C0 and C1 are fully specified. A section is devoted to the dual process of a semi-Markov process, and a formula is given for the mean of the first passage time from a state i to a state j of the dual process, in terms of the means of passage times of the original process.

Keywords

Semi-Markov processesreward processesdual processesMarkov renewal theoremfirst passage time

MSC classification

Primary: 60K15: Markov renewal processes, semi-Markov processes
Secondary: 60K05: Renewal theory 60K30: Applications (congestion, allocation, storage, traffic, etc.)
Type
Research Papers
Information
Journal of Applied Probability , Volume 35 , Issue 4 , December 1998 , pp. 833 - 842
Copyright
Copyright © Applied Probability Trust 1998 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The research was supported by the Institute for Studies in Theoretical Physics and Mathematics (IPM).

References

Çinlar, E. (1969). Markov renewal theory. Adv. Appl. Prob. 1, 123–187.Google Scholar
Çinlar, E. (1975). Markov renewal theory: a survey. Management Sci. 21, 727–752.Google Scholar
Çinlar, E. (1975). Introduction to stochastic processes. Prentice-Hall, Englewood Cliffs, NJ.Google Scholar
Keilson, J. (1969). On the matrix renewal function for Markov renewal processes. Ann. Math. Statist. 40, 1901–1907.Google Scholar
Keilson, J. (1971). A process with chain dependent growth rate. Part II: the ruin and ergodic problems. Adv. Appl. Prob. 3, 315–338.Google Scholar
Masuda, Y., and Sumita, U. (1991). A multivariate reward process defined on a semi-Markov process and its first passage time distributions. J. Appl. Prob. 28, 360–373.Google Scholar
Mclean, R. A., and Neuts, M. F. (1967). The integral of a step function defined on a semi Markov process. SIAM J. Appl. Math. 15, 726–737.Google Scholar
Pyke, R., and Schaufele, R. A. (1964). Limit theorem for Markov renewal processes. Ann. Math. Statist. 35, 1746–1764.Google Scholar
Soltani, A. R. (1996). Reward processes with nonlinear reward functions. J. Appl. Prob. 33, 1011–1017.Google Scholar
Sumita, U., and Masuda, Y. (1987). An alternative approach to the analysis of finite semi-Markov and related processes. Commun. Statist.–Stoch. Models 3, 67–87.Google Scholar
te Teugels, J. L. (1976). A bibliography on semi-Markov processes. J. Comp. Appl. Math. 2, 125–144.Google Scholar