A (jump) Markov process (generalized birth-and-death process) is used to model interactions of a large number of agents subject to field-type externalities. Transition rates are (nonlinear) functions of the composition of the population of agents classified by the choices they make. The model state randomly moves from one equilibrium to another, and exhibits asymmetrical oscillations (business cycles). It is shown that the processes can have several locally stable equilibria, depending on the degree of uncertainty associated with consequences of alternative choices. There is a positive probability of transition between any pair of such basins of attraction, and mean first-passage times between equilibria can be different when different pairs of basins are calculated.

We study the effect of perturbations in the data of a discrete-time Markov reward process on the finite-horizon total expected reward, the infinite-horizon expected discounted and average reward and the total expected reward up to a first-passage time. Bounds for the absolute errors of these reward functions are obtained. The results are illustrated for a finite as well as infinite queueing systems (M/M/1/S and ). Extensions to Markov decision processes and other settings are discussed.

Consider a perturbation in the one-step transition probabilities and rewards of a discrete-time Markov reward process with an unbounded one-step reward function. A perturbation estimate is derived for the finite horizon and average reward function. Results from [3] are hereby extended to the unbounded case. The analysis is illustrated for one- and two-dimensional queueing processes by an M/M/1-queue and an overflow queueing model with an error bound in the arrival rate.