Hostname: page-component-cd9895bd7-gxg78 Total loading time: 0 Render date: 2024-12-27T12:13:29.805Z Has data issue: false hasContentIssue false
  • English
  • Français

Negative binomial sums of random variables and discounted reward processes

Part of: Stochastic processes

Published online by Cambridge University Press:  14 July 2016

William L. Cooper
William L. Cooper*
Affiliation:
Georgia Institute of Technology
*
∗Postal address: School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta GA 30332, USA. Email address: billcoop@isye.gatech.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Given a sequence of random variables (rewards), the Haviv–Puterman differential equation relates the expected infinite-horizon λ-discounted reward and the expected total reward up to a random time that is determined by an independent negative binomial random variable with parameters 2 and λ. This paper provides an interpretation of this proven, but previously unexplained, result. Furthermore, the interpretation is formalized into a new proof, which then yields new results for the general case where the rewards are accumulated up to a time determined by an independent negative binomial random variable with parameters k and λ.

Keywords

Sums of random variablesreward processesMarkov decision processes

MSC classification

Primary: 60G99: None of the above, but in this section
Secondary: 90C40: Markov and semi-Markov decision processes
Type
Research Papers
Information
Journal of Applied Probability , Volume 35 , Issue 3 , September 1998 , pp. 589 - 599
Copyright
Copyright © Applied Probability Trust 1998 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Chung, K. L. (1974). A Course in Probability Theory, 2nd edn. Academic Press, New York.Google Scholar
Derman, C. (1970). Finite State Markovian Decision Processes. Academic Press, New York.Google Scholar
Fox, B. L., and Glynn, P. W. (1989). Simulating discounted costs. Management Sci. 35, 1297–1315.Google Scholar
Haviv, M., and Puterman, M. L. (1992). Estimating the value of a discounted reward process. Operat. Res. Lett. 11, 267–272.Google Scholar
Puterman, M. L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York.Google Scholar