Hostname: page-component-78c5997874-mlc7c Total loading time: 0 Render date: 2024-11-10T16:42:16.915Z Has data issue: false hasContentIssue false
  • English
  • Français

Controlled Heterogeneous Collection: The Role of Occupation Numbers

Part of: Markov processes Stochastic analysis Stochastic systems and control

Published online by Cambridge University Press:  14 July 2016

A. Gerardi  and
P. Tardelli
A. Gerardi*
Affiliation:
Universitá dell'Aquila
P. Tardelli*
Affiliation:
Universitá dell'Aquila
*
∗Postal address: Dipartimento di Ingegneria Elettrica e dell'Informazione, Universitá dell'Aquila, 67040 Poggio di Roio, Italy.
∗Postal address: Dipartimento di Ingegneria Elettrica e dell'Informazione, Universitá dell'Aquila, 67040 Poggio di Roio, Italy.
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

A controlled heterogeneous collection of identical items is presented. According to their level of wear and tear, they are divided into a finite number of classes and the partition of the collection is allowed to change over time. A suitable exchangeability assumption is made to preserve the property that the items be identical. The role of the occupation numbers is investigated and a filtering problem is set up, where the observation is the cardinality of a particular class. A control on the dynamics of the items is introduced, and the existence of an optimal control is proved. A discrete-time approximation for the separated problem, which is a finite-dimensional one, is performed. As a consequence, an approximation for the value function is given.

Keywords

Heterogeneous populationexchangeabilitystochastic filteringoptimal stochastic control

MSC classification

Secondary: 93E11: Filtering 93E20: Optimal stochastic control 60H30: Applications of stochastic analysis (to PDE, etc.) 60J75: Jump processes
Type
Research Article
Information
Journal of Applied Probability , Volume 45 , Issue 3 , September 2008 , pp. 595 - 609
Copyright
Copyright © Applied Probability Trust 2008 

References

[1] Brémaud, P. (1980). Point Processes and Queues. Springer, Berlin.Google Scholar
[2] Calzolari, A. and Nappo, G. (1996). A filtering problem with counting observations: approximations with error bounds. Stoch. Stoch. Reports 57, 71–87.CrossRefGoogle Scholar
[3] Calzolari, A. and Nappo, G. (1997). A filtering problem with counting observations: error bounds due to the uncertainty on the infinitesimal parameters. Stoch. Stoch. Reports 61, 1–19.Google Scholar
[4] Ceci, C. and Gerardi, A. (1998). Partially observed control of a Markov Jump process with counting observations: equivalence with the separated problem. Stoch. Process. Appl. 78, 245–260.CrossRefGoogle Scholar
[5] Ceci, C., Gerardi, A. and Tardelli, P. (2001). An approximation method for partially observed controlled discrete Jump Markov processes. IEEE Trans. Automatic Control 46, 1850–1861.Google Scholar
[6] Ceci, C., Gerardi, A. and Tardelli, P. (2001). An estimate of the approximation error in the filtering of a discrete Jump process. Math. Models Meth. Appl. Sci. 2, 181–198.CrossRefGoogle Scholar
[7] Ceci, C., Gerardi, A. and Tardelli, P. (2002). Existence of optimal control for partially observed Jump processes. Acta Appl. Math. 74, 155–175.Google Scholar
[8] El Karoui, N., Huu Nguyen, D. and Jeanblanc-Picqué, M. (1988). Existence of an optimal Markovian filter for control under partial observations. SIAM J. Control Optimization 26, 1025–1061.Google Scholar
[9] Elliott, R. J., Aggoun, L. and Moore, J. B. (1995). Hidden Markov Models: Estimation and Control. Springer, New York.Google Scholar
[10] Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes: Characterization and Convergence. John Wiley, New York.Google Scholar
[11] Fleming, W. (1980). Measure-valued processes in the control of partially-observable stochastic systems. Appl. Math. Optimization 6, 271–285.Google Scholar
[12] Gerardi, A. and Tardelli, P. (2001). Finite state and discrete time approximation for filters. Nonlinear Anal. Ser. A 47, 2485–2495.CrossRefGoogle Scholar
[13] Gerardi, A. and Tardelli, P. (2005). Heterogeneous population dynamical model: a filtering problem. J. Appl. Prob. 42, 346–361.Google Scholar
[14] Gerardi, A., Spizzichino, F. and Torti, B. (2000). Exchangeable mixture models for lifetimes: the role of ‘occupation number’. Statist. Prob. Lett. 49, 365–375.Google Scholar
[15] Gerardi, A., Spizzichino, F. and Torti, B. (2000). Filtering equations for the conditional law of residual lifetimes from a heterogeneous population. J. Appl. Prob. 37, 823–834.Google Scholar
[16] Hernandez-Hernandez, D., Marcus, S. I. and Fard, P. J. (1999). Analysis of a risk-sensitive control problem for hidden Markov chains. IEEE Trans. Automatic Control 44, 1093–1100.Google Scholar
[17] Jacod, J. (1975). Multivariate point processes: predictable projection. Z. Wahrscheinlichkeitsth. 31, 235–253.Google Scholar
[18] Jacod, J. (1979). Calcul Stochastique et Problèmes de Martingale. Springer, Berlin.Google Scholar
[19] James, M. R., Bara, J. S. and Elliott, R. J. (1994). Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems. IEEE Trans. Automatic Control 39, 780–791.CrossRefGoogle Scholar
[20] Kurtz, T. G. (1998). Martingale problems for conditional distribution of Markov processes. Electron. J. Prob. 3, 1–29.Google Scholar
[21] Kurtz, T. G. and Ocone, D. (1988). Unique characterization of conditional distributions in nonlinear filtering. Ann. Prob. 16, 80–107.Google Scholar
[22] Kurtz, T. G. and Stockbridge, R. H. (1998). Existence of Markov controls and characterization of optimal Markov controls. SIAM J. Control Optimization 36, 609–653.CrossRefGoogle Scholar
[23] Kushner, H. J. (1990). Numerical methods for stochastic control problems in continuous time. SIAM J. Control Optimization 28, 999–1048.Google Scholar
[24] Kushner, H. J. and Dupuis, P. G. (1992). Numerical Methods for Stochastic Control Problems in Continuous Time. Springer, Berlin.Google Scholar
[25] Mazliak, L. (1992). Mixed control problem under partial observation. Appl. Math. Optimization 27, 57–84.Google Scholar
[26] Roelly-Coppoletta, S. (1986). A criterion of convergence of measure valued processes: application to measure-branching processes. Stochastics 7, 44–66.Google Scholar
[27] Tardelli, P. (2006). Conditional law of lifetimes for a heterogeneous population of living particles via filtering techniques. Sci. Math. Japan 64, 915–932.Google Scholar