A multiarmed bandit problem is studied when the arms are not always available. The arms are first assumed to be intermittently available with some state/action-dependent probabilities. It is proven that no index policy can attain the maximum expected total discounted reward in every instance of that problem. The Whittle index policy is derived, and its properties are studied. Then it is assumed that the arms may break down, but repair is an option at some cost, and the new Whittle index policy is derived. Both problems are indexable. The proposed index policies cannot be dominated by any other index policy over all multiarmed bandit problems considered here. Whittle indices are evaluated for Bernoulli arms with unknown success probabilities.

In this paper, we analyze a queueing system characterized by a space-time arrival process of customers served by a countable set of servers. Customers arrive at points in space and the server stations have space-dependent processing rates. The workload is seen as a Radon measure and the server stations can adapt their power allocation to the current workload. We derive the stability region of the queueing system in the usual stationary ergodic framework. The analysis of this stability region gives some counter-intuitive results. Some specific subclasses of policy are also studied. Wireless communications networks is a natural field of application for the model.

In this paper we consider parallel and series systems, the components of which can be ‘improved'. The ‘improvement' consists of supplying the components with cold or hot standby spares or by allotting to them fixed budgets for minimal repairs. A fixed total resource of spares or minimal repairs is available. We find the optimal allocation of the resource items in several commonly encountered settings.

During the course of an R and D project, it is often meaningful and possible to evaluate its status, so that this information may be used for making better financing decisions over time. The project status changes stochastically due to the internal (technological) and the external (market) uncertainties, the former being partially controlled by expenditure of resources. In addition to the resource expenditure strategy, the manager must also decide when to terminate the project. Once the project is terminated, a terminal return is collected, whose value depends on the final project status. It is shown that the project should be terminated if the current status is either too low or too high to make further expenditure worthwhile. Otherwise, for an intermediate (promising) status of the project, an aggressive investment strategy is shown to be optimal. Thus, the model unifies the problems of optimally undertaking, financing and terminating an R and D project in face of various uncertainties.

A problem in optimal resource allocation is considered for n jobs with identically distributed service times admitting a monotone hazard function. If the hazard function is increasing, it is shown that the procedure of allocating the full resource individually to each job until its completion minimizes the expected completion time of the jth job. The procedure which at any instant of time equally allocated the resource among all of the remaining jobs is shown to minimize the expected total cumulative waiting time if the hazard is decreasing.