Book contents
- Frontmatter
- Contents
- Preface
- Frequently Used Notation
- 1 Markov Processes and Ergodic Properties
- 2 Controlled Diffusions
- 3 Nondegenerate Controlled Diffusions
- 4 Various Topics in Nondegenerate Diffusions
- 5 Controlled Switching Diffusions
- 6 Controlled Martingale Problems
- 7 Degenerate Controlled Diffusions
- 8 Controlled Diffusions with Partial Observations
- Epilogue
- Appendix: Results from Second Order Elliptic Equations
- References
- Index of symbols
- Subject index
- References
References
Published online by Cambridge University Press: 05 December 2011
Book contents
- Frontmatter
- Contents
- Preface
- Frequently Used Notation
- 1 Markov Processes and Ergodic Properties
- 2 Controlled Diffusions
- 3 Nondegenerate Controlled Diffusions
- 4 Various Topics in Nondegenerate Diffusions
- 5 Controlled Switching Diffusions
- 6 Controlled Martingale Problems
- 7 Degenerate Controlled Diffusions
- 8 Controlled Diffusions with Partial Observations
- Epilogue
- Appendix: Results from Second Order Elliptic Equations
- References
- Index of symbols
- Subject index
- References
Summary
A summary is not available for this content so a preview has been provided. Please use the Get access link above for information on how to access this content.
- Type
- Chapter
- Information
- Ergodic Control of Diffusion Processes , pp. 311 - 318Publisher: Cambridge University PressPrint publication year: 2011
References
[1] , , and 1959. Estimates near the boundary for solutions of elliptic partial differential equations satisfying general boundary conditions. I. Comm. Pure Appl. Math., 12, 623–727.CrossRefGoogle Scholar
[2] and 1980. New results on the innovations problem for nonlinear filtering. Stochastics, 4(4), 339–348.CrossRefGoogle Scholar
[3] and 1987. Linear Programming in Infinite-Dimensional Spaces. Wiley-Interscience Series in Discrete Mathematics and Optimization. Chichester: John Wiley & Sons.Google Scholar
[4] and 2010. Uniform recurrence properties of controlled diffusions and applications to optimal control. SIAM J. Control Optim., 48(7), 152–160.CrossRefGoogle Scholar
[5] and 2004 (Dec). Ergodic control of jump diffusions in ℝd under a near-monotone cost assumption. Pages 4140–4145 of:43rd IEEE Conference on Decision and Control, vol. 4.Google Scholar
[6] , , , , and 1993. Discrete-time controlled Markov processes with average cost criterion: a survey. SIAM J. Control Optim., 31(2), 282–344.CrossRefGoogle Scholar
[7] , , and 1999. Harnack's inequality for cooperative, weakly coupled elliptic systems. Comm. Partial Differential Equations, 24, 1555–1571.CrossRefGoogle Scholar
[8] , and 1998. On ergodic stochastic control. Comm. Partial Differential Equations, 23(11–12), 333–358.CrossRefGoogle Scholar
[9] , , and 1953. Admissible points of convex sets. Pages 87–91 of: Contributions to the Theory of Games, vol. 2. Annals of Mathematics Studies, no. 28. Princeton, NJ: Princeton University Press.Google Scholar
[10] 2006. Louis Bachelier's Theory of Speculation: The Origins of Modern Finance. Princeton, NJ: Princeton University Press. Translated and with a commentary by Mark Davis and Alison Etheridge.
[11] , , and 1997. Ergodic control of degenerate diffusions. Stochastic Anal. Appl., 15(1), 1–17.CrossRefGoogle Scholar
[12] 1998. Diffusions and Elliptic Operators. Probability and its Applications. New York: Springer-Verlag.Google Scholar
[13] 1970. Existence of optimal strategies based on specified information, for a class of stochastic decision problems. SIAM J. Control, 8, 179–188.CrossRefGoogle Scholar
[14] 1982. Stochastic Control by Functional Analysis Methods. Studies in Mathematics and its Applications, vol. 11. Amsterdam: North-Holland Publishing Co.Google Scholar
[15] and 1984. Ergodic control problem for one-dimensional diffusions with near-monotone cost. Systems Control Lett., 5(2), 127–133.CrossRefGoogle Scholar
[16] and 1986. Corrections to: “Ergodic control problem for onedimensional diffusions with near-monotone cost” [Systems Control Lett. 5(1984), no. 2, 127–133]. Systems Control Lett., 7(3), 233–235.CrossRefGoogle Scholar
[17] and 1992. On Bellman equations of ergodic control in Rn. J. Reine Angew. Math., 429, 125–160.Google Scholar
[18] and 2002. Ergodic control Bellman equation with Neumann boundary conditions. Pages 59–71 of: Stochastic Theory and Control (Lawrence, KS, 2001). Lecture Notes in Control and Inform. Sci., vol. 280. Berlin: Springer.Google Scholar
[19] 1996. Lévy Processes. Cambridge Tracts in Mathematics, vol. 121. Cambridge: Cambridge University Press.Google Scholar
[20] and 1978. Stochastic Optimal Control: The Discrete Time Case. New York: Academic Press.Google Scholar
[21] and 1996. Occupation measures for controlled Markov processes: characterization and optimality. Ann. Probab., 24(3), 1531–1562.Google Scholar
[22] and 2005. Existence of optimal Markov solutions for ergodic control of Markov processes. Sankhyā, 67(1), 1–18.Google Scholar
[23] and 1993. Invariant measures and evolution equations for Markov processes characterized via martingale problems. Ann. Probab., 21(4), 2246–2268.CrossRefGoogle Scholar
[25] 1995. Probability and Measure. Third edition. Wiley Series in Probability and Mathematical Statistics. New York: John Wiley & Sons.Google Scholar
[26] , , and 2001. On regularity of transition probabilities and invariant measures of singular diffusions under minimal conditions. Comm. Partial Differential Equations, 26(11–12), 2037–2080.CrossRefGoogle Scholar
[27] , , and 2002. Uniqueness of solutions of elliptic equations and uniqueness of invariant measures of diffusions. Mat. Sb., 193(7), 3–36.Google Scholar
[28] 1989a. Optimal Control of Diffusion Processes. Pitman Research Notes in Mathematics Series, vol. 203. Harlow: Longman Scientific & Technical.Google Scholar
[30] 1991. On extremal solutions to stochastic control problems. Appl. Math. Optim., 24(3), 317–330.CrossRefGoogle Scholar
[31] 1993. Controlled diffusions with constraints. II. J. Math. Anal. Appl., 176(2), 310–321.CrossRefGoogle Scholar
[33] 2003. Dynamic programming for ergodic control with partial observations. Stochastic Process. Appl., 103(2), 293–310.CrossRefGoogle Scholar
[34] and 2004a. Ergodic control for constrained diffusions: characterization using HJB equations. SIAM J. Control Optim., 43(4), 1467–1492.CrossRefGoogle Scholar
[35] and 2004b. A further remark on dynamic programming for partially observed Markov processes. Stochastic Process. Appl., 112(1), 79–93.CrossRefGoogle Scholar
[36] and 2007. Singular perturbations in ergodic control of diffusions. SIAM J. Control Optim., 46(5), 1562–1577.CrossRefGoogle Scholar
[37] and 1988. Ergodic control of multidimensional diffusions. I. The existence results. SIAM J. Control Optim., 26(1), 112–126.CrossRefGoogle Scholar
[38] and 1990a. Controlled diffusions with constraints. J. Math. Anal. Appl., 152(1), 88–108.CrossRefGoogle Scholar
[39] and 1990b. Ergodic control of multidimensional diffusions. II. Adaptive control. Appl. Math. Optim., 21(2), 191–220.CrossRefGoogle Scholar
[40] and 2003. Ergodic control of partially degenerate diffusions in a compact domain. Stochastics, 75(4), 221–231.Google Scholar
[41] and 2003. A note on stochastic dissipativeness. Pages 41– 49 of: Directions in mathematical systems theory and optimization. Lecture Notes in Control and Inform. Sci., vol. 286. Berlin: Springer.CrossRefGoogle Scholar
[42] and 2004. Harnack type estimates for nonlinear elliptic systems and applications. Ann. Inst. H. Poincaré Anal. Non Linéaire, 21(5), 543–590.CrossRefGoogle Scholar
[43] and 1998. Second Order Elliptic Equations and Elliptic Systems. Translations of Mathematical Monographs, vol. 174. Providence, RI: American Mathematical Society. Translated from the 1991 Chinese original by Bei Hu.CrossRefGoogle Scholar
[44] 1969. Lectures on Analysis. Vol. II: Representation Theory. Edited by , and . New York: W. A. Benjamin.Google Scholar
[45] 1982. Lectures from Markov Processes to BrownianMotion. Grundlehren der Mathematischen Wissenschaften, vol. 249. New York: Springer-Verlag.CrossRefGoogle Scholar
[46] , , and 2000. Lp-theory for fully nonlinear uniformly parabolic equations. Comm. Partial Differential Equations, 25(11–12), 1997–2053.
[47] and 1978. Probabilities and Potential A. North-Holland Mathematics Studies, vol. 29. Amsterdam: North-Holland.Google Scholar
[48] 1962. On extreme points of convex sets. J. Math. Anal. Appl., 5, 237–244.CrossRefGoogle Scholar
[50] 2002. Real Analysis and Probability. Cambridge Studies in Advanced Mathematics, vol. 74. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
[51] and 1988. Linear Operators. Part I. Wiley Classics Library. New York: John Wiley & Sons.Google Scholar
[52] and 1991. On Lipschitz continuity of the solution mapping to the Skorohod problem with applications. Stochastics, 35, 31–62.Google Scholar
[53] and 1979. Controlled Markov Processes. Grundlehren der Mathematischen Wissenschaften, vol. 235. Berlin: Springer-Verlag.
[54] , , and 1987. Compactification methods in the control of degenerate diffusions: existence of an optimal control. Stochastics, 20(3), 169–219.Google Scholar
[55] and 1986. Markov Processes. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. New York: John Wiley & Sons.
[56] and 1981. Examples of singular parabolic measures and singular transition probability densities. Duke Math. J., 48(4), 845–856.CrossRefGoogle Scholar
[57] 1959. Non-Markovian processes with the semigroup property. Ann. Math. Statist., 30, 1252–1253.CrossRefGoogle Scholar
[58] and 1975. Deterministic and Stochastic Optimal Control. Berlin: Springer-Verlag.CrossRefGoogle Scholar
[59] 1980. Measure-valued processes in the control of partially-observable stochastic systems. Appl. Math. Optim., 6(3), 271–285.CrossRefGoogle Scholar
[60] , and 1982. Optimal control for partially observed diffusions. SIAM J. Control Optim., 20(2), 261–285.CrossRefGoogle Scholar
[61] 1963. Diffusion processes with reflection and a directional derivative problem on a manifold with boundary. Theory Probab. Appl., 8(1), 75–83.CrossRefGoogle Scholar
[62] 1964. Partial Differential Equations of Parabolic Type. Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
[63] 2006. Stochastic Differential Equations and Applications. Mineola, NY: Dover Publications.Google Scholar
[64] , , and 1993. Optimal control of switching diffusions with application to flexible manufacturing systems. SIAM J. Control Optim., 31(5), 1183–1204.CrossRefGoogle Scholar
[65] , , and 1997. Ergodic control of switching diffusions. SIAM J. Control Optim., 35(6), 1952–1988.CrossRefGoogle Scholar
[66] and 1969. Introduction to the Theory of Random Processes. Translated from the Russian by Scripta Technica, Inc. Philadelphia, PA: W. B. Saunders.Google Scholar
[67] and 1983. Elliptic Partial Differential Equations of Second Order. Second edition. Grundlehren der Mathematischen Wissenschaften, vol. 224. Berlin: Springer-Verlag.CrossRefGoogle Scholar
[68] and 1996. Existence of strong solutions for Itô's stochastic equations via approximations. Probab. Theory Related Fields, 105(2), 143–158.CrossRefGoogle Scholar
[69] 1960. Ergodic properties of recurrent diffusion processes and stabilization of the solution of the Cauchy problem for parabolic equations. Theory Probab. Appl., 5(2), 179–196.CrossRefGoogle Scholar
[70] 1980. Stochastic Stability of Differential Equations. The Netherlands: Sijthoff & Noordhoff.CrossRefGoogle Scholar
[71] 1985. L'équation de Zakai et le problème séparè du contrôle optimal stochastique. Pages 37–62 of: Séminaire de Probabilités, XIX, 1983/84. Lecture Notes in Math., vol. 1123. Berlin: Springer.CrossRefGoogle Scholar
[72] 1986. Existence of optimal Markovian controls for degenerate diffusions. Pages 171–186 of: Stochastic Differential Systems (Bad Honnef, 1985). Lecture Notes in Control and Information Science, vol. 78. Berlin: Springer.Google Scholar
[73] 1989. Adaptive Markov Control Processes. Applied Mathematical Sciences, vol. 79. New York: Springer-Verlag.CrossRefGoogle Scholar
[74] and 1989. Stochastic Differential Equations and Diffusion Processes. Second edition. North-Holland Mathematical Library, vol. 24. Amsterdam: North-Holland Publishing.Google Scholar
[75] 1983. Linear Programming and Finite Markovian Control Problems. Mathematical Centre Tracts, vol. 148. Amsterdam: Mathematisch Centrum.Google Scholar
[76] and 1991. Brownian Motion and Stochastic Calculus. Second edition. Graduate Texts in Mathematics, vol. 113. New York: Springer-Verlag.Google Scholar
[77] 1969. The optimal control of a non-stopping diffusion process with reflection. Theory Probab. Appl., 14(3), 496–502.CrossRefGoogle Scholar
[78] 1980. Controlled Diffusion Processes. Applications of Mathematics, vol. 14. New York: Springer-Verlag. Translated from the Russian by A. B. Aries.CrossRefGoogle Scholar
[79] 1995. Introduction to the Theory of Diffusion Processes. Translations of Mathematical Monographs, vol. 142. Providence, RI: American Mathematical Society.Google Scholar
[80] 1981. Some extensions of Itô's formula. Pages 118–141 of: Seminar on Probability, XV (Univ. Strasbourg, Strasbourg, 1979/1980) (French). Lecture Notes in Math., vol. 850. Berlin: Springer.Google Scholar
[81] and 1998. Existence of Markov controls and characterization of optimal Markov controls. SIAM J. Control Optim., 36(2), 609–653.CrossRefGoogle Scholar
[82] and 2001. Stationary solutions and forward equations for controlled and singular martingale problems. Electron. J. Probab., 6, no. 17, 52 pp. (electronic).CrossRefGoogle Scholar
[83] , , and 1967. Linear and Quasilinear Equations of Parabolic Type. Translated from the Russian by . Translations of Mathematical Monographs, Vol. 23. Providence, RI: American Mathematical Society.Google Scholar
[84] 1983a. Optimal control of diffusion processes and Hamilton–Jacobi–Bellman equations. I. The dynamic programming principle and applications. Comm. Partial Differential Equations, 8(10), 1101–1174.CrossRefGoogle Scholar
[85] 1983b. Optimal control of diffusion processes and Hamilton–Jacobi–Bellman equations. II. Viscosity solutions and uniqueness. Comm. Partial Differential Equations, 8(11), 1229–1276.CrossRefGoogle Scholar
[86] and 1984. Stochastic differential equations with reflecting boundary conditions. Comm. Pure Appl. Math., 37, 511–537.CrossRefGoogle Scholar
[87] and 1977. Statistics of Random Processes. I. Applications of Mathematics, Vol. 5. New York: Springer-Verlag. Translated by A. B. Aries.CrossRefGoogle Scholar
[89] and 1997. Ergodic control of reflected diffusions with jumps. Appl. Math. Optim., 35(2), 117–137.CrossRefGoogle Scholar
[90] and 1999. On optimal ergodic control of diffusions with jumps. Pages 439–456 of: Stochastic Analysis, Control, Optimization and Applications. Systems Control Found. Appl. Boston, MA: Birkhäuser Boston.CrossRefGoogle Scholar
[91] and 2009. Markov Chains and Stochastic Stability. Second edition. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
[92] 1997. Nonuniqueness in the martingale problem and the Dirichlet problem for uniformly elliptic operators. Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4), 24(3), 537–549.Google Scholar
[93] 1965. Mathematical Foundations of the Calculus of Probability. San Francisco, CA: Holden-Day.Google Scholar
[94] 1967. Probability Measures on Metric Spaces. Probability and Mathematical Statistics, No. 3. New York: Academic Press.CrossRefGoogle Scholar
[96] 1990. Generalized Diffusion Processes. Translations of Mathematical Monographs, vol. 83. Providence, RI: American Mathematical Society. Translated from the Russian by H. H. McFaden.CrossRefGoogle Scholar
[97] 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Hoboken, NJ: John Wiley & Sons.CrossRefGoogle Scholar
[98] 1991. Probability Metrics and the Stability of Stochastic Models. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics. Chichester: John Wiley & Sons.Google Scholar
[99] 1970. Necessary and sufficient dynamic programming conditions for continuous time stochastic control problem. SIAM J. Control, 8, 559–571.CrossRefGoogle Scholar
[100] 1983. Long-term average cost control problems for continuous time Markov processes: a survey. Acta Appl. Math., 1(3), 281–299.CrossRefGoogle Scholar
[101] 1946. Convex Analysis. Princeton Mathematical Series, vol. 28. Princeton, NJ: Princeton University Press.Google Scholar
[102] , and 2000a. Diffusions, Markov Processes, and Martingales. Vol. 1. Cambridge Mathematical Library. Cambridge: Cambridge University Press.Google Scholar
[103] and 2000b. Diffusions, Markov Processes, and Martingales. Vol. 2. Cambridge Mathematical Library. Cambridge: Cambridge University Press.Google Scholar
[104] 1970. Average cost semi-Markov decision processes. J. Appl. Probability, 7, 649–656.CrossRefGoogle Scholar
[106] 1996. Linear System Theory. Second edition. Prentice Hall Information and System Sciences Series. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
[107] 1999. Nonuniqueness for second-order elliptic equations with measurable coefficients. SIAM J. Math. Anal., 30(4), 879–895 (electronic).CrossRefGoogle Scholar
[108] 1989. Asymptotic Methods in the Theory of Stochastic Differential Equations. Translations of Mathematical Monographs, vol. 78. Providence, RI: American Mathematical Society.Google Scholar
[109] 1999. (Nonsymmetric) Dirichlet operators on L1: existence, uniqueness and associated Markov processes. Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4), 28(1), 99–140.Google Scholar
[110] 1989. Time-average control of martingale problems: the Hamilton–Jacobi–Bellman equation. Stochastics Stochastics Rep., 27(4), 249–260.CrossRefGoogle Scholar
[111] 1990a. Time-average control of martingale problems: a linear programming formulation. Ann. Probab., 18(1), 206–217.CrossRefGoogle Scholar
[112] 1990b. Time-average control of martingale problems: existence of a stationary solution. Ann. Probab., 18(1), 190–205.CrossRefGoogle Scholar
[113] 1984. Martingale methods for the optimal control of continuous time stochastic systems. Stoch. Process. Appl., 18, 324–347.CrossRefGoogle Scholar
[114] and 1971. Diffusion processes with boundary conditions. Comm. Pure Appl. Math., 24, 147–225.CrossRefGoogle Scholar
[115] and 1979. Multidimensional Diffusion Processes. Grundlehren der Mathematischen Wissenschaften, vol. 233. Berlin: Springer-Verlag.Google Scholar
[116] 1980. Strong solutions and explicit formulas for solutions of stochastic integral equations. Mat. Sb. (N.S.), 111(153)(3), 434–452, 480.Google Scholar
[117] 1987. On strong solutions of stochastic Itô equations with jumps. Theory Probab. Appl., 32(1), 148–152.CrossRefGoogle Scholar
[118] 1977. Survey of measurable selection theorems. SIAM J. Control Optim., 15, 859–903.CrossRefGoogle Scholar
[119] 1982. An Introduction to Ergodic Theory. Graduate Texts in Mathematics, vol. 79. New York: Springer-Verlag.CrossRefGoogle Scholar
[120] 1972. Dissipative dynamical systems. I. General theory. Arch. Rational Mech. Anal., 45, 321–351.CrossRefGoogle Scholar
[121] 1971. Representation of martingales, quadratic variation and applications. SIAM J. Control, 9, 621–633.CrossRefGoogle Scholar
[122] and 1985. Stochastic Processes in Engineering Systems. Springer Texts in Electrical Engineering. New York: Springer-Verlag.CrossRefGoogle Scholar
[123] , , and 2006. Optimal power allocation for a timevarying wireless channel under heavy-traffic approximation. IEEE Trans. Automat. Control, 51(4), 580–594.CrossRefGoogle Scholar
[124] 2008. An Introduction to Stochastic Filtering Theory. Oxford Graduate Texts in Mathematics, vol. 18. Oxford: Oxford University Press.Google Scholar
[125] 1980. Functional Analysis. Sixth edition. Grundlehren der Mathematischen Wissenschaften, vol. 123. Berlin: Springer-Verlag.Google Scholar
[126] 1969. Lectures on the Calculus of Variations and Optimal Control Theory. Foreword by Wendell . Philadelphia: W. B. Saunders.Google Scholar
[127] 1974. A transformation of the phase space of a diffusion process that will remove the drift. Mat. Sb. (N.S.), 93(135), 129–149, 152.CrossRefGoogle Scholar