Hostname: page-component-cd9895bd7-jkksz Total loading time: 0 Render date: 2024-12-28T13:00:16.153Z Has data issue: false hasContentIssue false
  • English
  • Français

Extreme-point solutions in Markov decision processes

Published online by Cambridge University Press:  14 July 2016

David Assaf
David Assaf*
Affiliation:
The Hebrew University of Jerusalem
*
∗ Postal address: Department of Statistics, Faculty of Social Sciences, The Hebrew University of Jerusalem, Jerusalem, Israel.
Get access Rights & Permissions [Opens in a new window]

Abstract

The paper presents sufficient conditions for certain functions to be convex. Functions of this type often appear in Markov decision processes, where their maximum is the solution of the problem. Since a convex function takes its maximum at an extreme point, the conditions may greatly simplify a problem. In some cases a full solution may be obtained after the reduction is made. Some illustrative examples are discussed.

Keywords

OPTIMAL POLICYCONVEX FUNCTION
Type
Research Papers
Information
Journal of Applied Probability , Volume 20 , Issue 4 , December 1983 , pp. 835 - 842
Copyright
Copyright © Applied Probability Trust 1983 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Assaf, D. (1978) Invariant problems in discounted dynamic programming. Adv. Appl. Prob. 10, 472–490.CrossRefGoogle Scholar
[2] Assaf, D. (1980) Invariant problems in dynamic programming — Average reward criterion. Stoch. Proc. Appl. 10, 313–322.Google Scholar
[3] Blackwell, D. (1962) Discrete dynamic programming. Ann. Math. Statist. 33, 719–726.Google Scholar
[4] Blackwell, D. (1965) Discounted dynamic programming. Ann. Math. Statist. 36, 226–235.CrossRefGoogle Scholar
[5] Derman, C. (1966) Denumerable state Markovian decision processes — average cost criterion. Ann. Math. Statist. 37, 1545–1554.Google Scholar
[6] Howard, R. (1960) Dynamic Programming and Markov Processes. Wiley, New York.Google Scholar
[7] Ross, S. M. (1968) Non-discounted denumerable Markovian decision models. Ann. Math. Statist. 39, 412–423.Google Scholar
[8] Ross, S. M. (1968) Arbitrary state Markovian decision processes. Ann. Math. Statist. 39, 2118–2122.Google Scholar
[9] Stidham, S. Jr and Prabhu, N. U. (1974) Optimal control of queueing systems. In Lecture Notes in Economics and Mathematical Systems 98, Springer-Verlag, Berlin, 263–294.Google Scholar
[10] Strauch, R. E. (1965) Negative dynamic programming. Ann. Math. Statist. 37, 871–890.Google Scholar