This paper considers risk-sensitive average optimization for denumerable continuous-time Markov decision processes (CTMDPs), in which the transition and cost rates are allowed to be unbounded, and the policies can be randomized history dependent. We first derive the multiplicative dynamic programming principle and some new facts for risk-sensitive finite-horizon CTMDPs. Then, we establish the existence and uniqueness of a solution to the risk-sensitive average optimality equation (RS-AOE) through the results for risk-sensitive finite-horizon CTMDPs developed here, and also prove the existence of an optimal stationary policy via the RS-AOE. Furthermore, for the case of finite actions available at each state, we construct a sequence of models of finite-state CTMDPs with optimal stationary policies which can be obtained by a policy iteration algorithm in a finite number of iterations, and prove that an average optimal policy for the case of infinitely countable states can be approximated by those of the finite-state models. Finally, we illustrate the conditions and the iteration algorithm with an example.

We devise an endogenous growth model in which agents’ utility depends not only on current consumption but also on the pleasure of anticipated future consumption. We consider the case in which agents derive satisfaction from their own anticipatory feelings—inward-looking or internal anticipation—and the case in which agents derive utility from anticipation of other people’s future consumption—outward-looking or external anticipation. We characterize the effects of introducing a forward-looking consumption reference on the dynamics of the economy. Whereas the inward-looking economy features transitional dynamics, the outward-looking economy does not. The distortions caused by the externality in the economy with external habits can be corrected by subsidizing income at a time-varying rate or by means of a tax on consumption at a decreasing rate. We contrast the equilibrium dynamics of our specification to the more standard specification of the habit formation consumption reference point. Numerical simulations supplement the theoretical analysis.

We analyze the optimal debt reduction problem in an uncertainty context. The social planner has a finite horizon and seeks to minimize the social costs associated with debt repayment by taking into account not only the short-run costs of the policy, but also the long-run costs associated with the outstanding level of debt. We characterize the optimal policy and the dynamics of the debt-to-GDP ratio, showing that it will decrease over time if economic policy is effective enough. We characterize how the evolution of the debt-to-GDP ratio depends on the main parameters and we present a simple calibration based on Greek data to illustrate the implications of our analysis in real-world setups.

The sequential stochastic assignment problem (SSAP) allocates distinct workers with deterministic values to sequentially arriving tasks with stochastic parameters to maximize the expected total reward. In this paper we study an extension of the SSAP, in which the worker values are considered to be random variables, taking on new values upon each task arrival. Several SSAP models with different assumptions on the distribution of the worker values and closed-form expressions for optimal assignment policies are presented.

Differential property tax policy for farmland is often set under conditions of uncertainty and limited information regarding landowners’ objective functions. This study examines optimal differential tax policy for a parcel of agricultural land facing uncertain development, identifying instances in which common farmland taxation policies may be non-optimal. Optimal tax rates are characterized given three possible causes of uncertain development: exogenous offers from developers, tax-related reductions in landowner wealth, and a combination of these factors. Model results indicate that underlying causes of uncertain development are critical when seeking to assess the optimality of differential taxation policies, and the use of a single, time-invariant differential tax levy is rarely optimal given uncertain development.

This paper investigates the desirability of constitutional constraints on capital taxation in an environment without government debt and where benevolent governments have limited commitment. In our setup, governments can choose proportional capital and labor income taxes subject to the constitutional constraint but cannot commit to an actual path of taxes. First, we explore a form of constitutional constraint: a constant cap on capital tax rates. In our quantitative exercise, we show that a three percent cap on capital taxes provides the highest welfare at the worst sustainable equilibrium. However, such a cap decreases welfare at the best sustainable equilibrium (both because it constrains feasibility and because it tightens the incentive compatibility constraint). Second, we identify a form of constitutional constraint that can improve all sustainable equilibria. That constraint features a cap on capital taxes that increases with the level of capital.

This paper develops methods to solve for optimal discretionary policies and optimal commitment policies in rational expectations models. These algorithms, which allow the optimization constraints to be conveniently expressed in second-order structural form, are more general than existing methods and are simple to apply. We use several New Keynesian business cycle models to illustrate their application. Simulations show that the procedures developed in this paper can quickly solve small-scale models and that they can be usefully and effectively applied to medium- and large-scale models.

This paper devises a fiscal policy by means of which the first-best optimal equilibrium can be attained as a market equilibrium in an endogenous growth model with public capital. The optimal equilibrium requires that public (private) investment be zero along the transition to the balanced growth path if the initial ratio of public to private capital is higher (lower) than its long-term value. We also show that the transitional dynamics can be determined by noting that the continuity of the shadow prices involves the continuity of the consumption path.

We study a structural dynamic monetary economy submitted to technological, monetary, and real demand shocks, and investigate the nature of optimal monetary and fiscal policies under Walrasian or preset wages and prices. If prices and wages are Walrasian, then the optimal policies are both nonactivist and derive directly from Milton Friedman's prescriptions. If prices or wages are preset, however, optimal policies, and, notably, the fiscal part, become activist and countercyclical.

A single machine is available to process a collection of stochastic jobs preemptively. Rewards are received at job completions. We seek policies for machine allocation which maximize the total reward. Application areas point to the need to study such models for resource allocation when job processing requirements are dependent. To this end, models are developed in which the nature of such dependence is derived from various notions of positive and negative dependence in common usage in reliability. Optimal policies for resource allocation of simple structure are obtained for a variety of such models.

Uniformization permits the replacement of a semi-Markov decision process (SMDP) by a Markov chain exhibiting the same average rewards for simple (non-randomized) policies. It is shown that various anomalies may occur, especially for stationary (randomized) policies; uniformization introduces virtual jumps with concomitant action changes not present in the original process. Since these lead to discrepancies in the average rewards for stationary processes, uniformization can be accepted as valid only for simple policies.

We generalize uniformization to yield consistent results for stationary policies also. These results are applied to constrained optimization of SMDP, in which stationary (randomized) policies appear naturally. The structure of optimal constrained SMDP policies can then be elucidated by studying the corresponding controlled Markov chains. Moreover, constrained SMDP optimal policy computations can be more easily implemented in discrete time, the generalized uniformization being employed to relate discrete- and continuous-time optimal constrained policies.

This paper considers periodic and sequential preventive maintenance (PM) policies for the system with minimal repair at failure: the PM is done (i) at periodic times kx and (ii) at constant intervals xk (k = 1, 2, ···, N). The system has a different failure distribution between PM'S and is replaced at the Nth PM. The optimal policies which minimize the expected cost rates are discussed. The optimal x and N of periodic PM and {xk} of sequential PM are easily computed in a Weibull distribution case.

The paper presents sufficient conditions for certain functions to be convex. Functions of this type often appear in Markov decision processes, where their maximum is the solution of the problem. Since a convex function takes its maximum at an extreme point, the conditions may greatly simplify a problem. In some cases a full solution may be obtained after the reduction is made. Some illustrative examples are discussed.

In this article we examine a breakdown model in which the system's status can be determined only by a test. Upon detection of failure the system must be replaced by a new identical one. The costs incurred include cost of inspection, operating costs, failure cost and a cost associated with planned replacements. Throughout the paper we restrict attention to replacement rules in which the time interval between two successive inspections is regarded as a fixed quantity. The decision variables include the inspection interval and the scheduling for preventive (planned) replacements. The problem is to specify a replacement rule which minimizes the long-run average cost per unit time. We show that under certain monotone conditions there is a natural candidate for an optimal replacement rule.

Discounted dynamic programming problems whose transition mechanism depends only on the action taken and does not depend on the current state are considered. A value determination operation and method of obtaining optimal policies for the case of finite action space (and arbitrary state space) are presented.

The solution of other problems is reduced to this special case by a suitable transformation. Results are illustrated by examples.

In this paper we shall solve the optimal policy for the Markovian replacement model in which the state of a machine is not observable. We shall consider both discounted and average costs and discuss two examples.

A martingale approach to a dynamic program with general state and action spaces is taken. Several necessary and sufficient conditions are given for a policy to be optimal. The results comprehend and modify different criteria of optimality given for dynamic programming problems. Finally, two applications are stated.

This paper is concerned with the general problem of finding an optimal transition matrix for a finite Markov chain, where the probabilities for each transition must be chosen from a given convex family of distributions. The immediate cost is determined by this choice, but it is required to minimise the average expected cost in the long run. The problem is investigated by classifying the states according to the accessibility relations between them. If an optimal policy exists, it can be found by considering the convex subsystems associated with the states at different levels in the classification scheme.

A Markov process in discrete time with a finite state space is controlled by choosing the transition probabilities from a given convex family of distributions depending on the present state. The immediate cost is prescribed for each choice and it is required to minimise the average expected cost over an infinite future. The paper considers a special case of this general problem and provides the foundation for a general solution. The main result is that an optimal policy exists if each state of the system can be reached with positive probability from any other state by choosing a suitable policy.