Hostname: page-component-78c5997874-94fs2 Total loading time: 0 Render date: 2024-11-10T21:12:47.093Z Has data issue: false hasContentIssue false
  • English
  • Français

Optimal decision procedures for finite Markov chains. Part III: General convex systems

Published online by Cambridge University Press:  01 July 2016

John Bather
John Bather*
Affiliation:
University of Sussex
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper is concerned with the general problem of finding an optimal transition matrix for a finite Markov chain, where the probabilities for each transition must be chosen from a given convex family of distributions. The immediate cost is determined by this choice, but it is required to minimise the average expected cost in the long run. The problem is investigated by classifying the states according to the accessibility relations between them. If an optimal policy exists, it can be found by considering the convex subsystems associated with the states at different levels in the classification scheme.

Keywords

MARKOV CHAINDECISION PROCEDUREMINIMUM AVERAGE COSTOPTIMAL POLICYHOWARD MODELDYNAMIC PROGRAMMINGCONVEX DECISION SPACEACCESSIBILITY
Type
Research Article
Information
Advances in Applied Probability , Volume 5 , Issue 3 , December 1973 , pp. 541 - 553
Copyright
Copyright © Applied Probability Trust 1973 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Bather, J. (1973) Optimal decision procedures for finite Markov chains. Part I: examples. Adv. Appl. Prob. 5, 328–339.CrossRefGoogle Scholar
[2] Bather, J. (1973) Optimal decision procedures for finite Markov chains. Part II: communicating systems. Adv. Appl. Prob. 5, 521–540.Google Scholar
[3] Howard, R. A. (1960) Dynamic Programming and Markov Processes. Wiley, New York.Google Scholar