We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.
To save content items to your account,
please confirm that you agree to abide by our usage policies.
If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account.
Find out more about saving content to .
To save content items to your Kindle, first ensure no-reply@cambridge.org
is added to your Approved Personal Document E-mail List under your Personal Document Settings
on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part
of your Kindle email address below.
Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations.
‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi.
‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
The book starts out with a motivating chapter to answer the question: Why is it worthwhile to develop system theory? To do so, we jump fearlessly in the very center of our methods, using a simple and straight example in optimization: optimal tracking. Although optimization is not our leading subject– which is system theory– it provides for one of the main application areas, namely the optimization of the performance of a dynamical system in a time-variant environment (for example, driving a car or sending a rocket to the moon). The chapter presents a recursive matrix algebra approach to the optimization problem, known as dynamic programming. Optimal tracking is based on a powerful principle called “dynamic programming,” which lies at the very basis of what ”dynamical” means.
Deals with reservoir operation, including rule curves, methods of mathematical programming, optimization of reservoir operations, simulation models, reservoir operation modeling with HEC-ResSim, mass curves, and reservoir siltation.
The rapid development of new technologies such as electrification, autonomy, and other contextual factors pose significant challenges to development teams in balancing competing aspects while developing value-robust solutions. One approach for achieving value robustness is designing for changeability. This paper presents a tradespace exploration from a Systems-of-Systems perspective to facilitate changeability assessment during early design stages. The approach is further demonstrated on a fleet of haulers operating in a mining site.
This study examines the economic performance of rainfed cropping systems endemic to the Southern Great Plains under weed competition. Cropping systems include tilled and no-till wheat-fallow, wheat-soybean, and wheat-sorghum rotations. Net returns from systems are compared under different levels of weed pressure. Producers operating over longer planning horizons would choose to double-crop regardless of the tillage method used and weed pressure level. Producers operating under shorter planning horizons would implement wheat-fallow systems when weed pressure is high and double crop when weed pressure is low.
We study the time-consistent investment and contribution policies in a defined benefit stochastic pension fund where the manager discounts the instantaneous utility over a finite planning horizon and the final function at constant but different instantaneous rates of time preference. This difference, which can be motivated for some uncertainties affecting payoffs at the end of the planning horizon, will induce a variable bias between the relative valuation of the final function and the previous payoffs and will lead the manager to show time-inconsistent preferences. Both the benefits and the contribution rate are proportional to the total wage of the workers that we suppose is stochastic. The aim is to maximize a CRRA utility function of the net benefit relative to salary in a bounded horizon and to maximize a CRRA final utility of the fund level relative to the salary. The problem is solved by means of dynamic programming techniques, and main results are illustrated numerically.
The paper presents a framework for the integration of the system's design variables, state variables, control strategies, and contextual variables into a design optimization problem to assist early-stage design decisions. The framework is based on a global optimizer incorporating Dynamic Programming, and its applicability is demonstrated by the conceptual design of an electrical hauler. Pareto front of optimal design solutions, in terms of time and cost, together with optimal velocity profiles and battery state-of-charge is visualized for the given mining scenario.
Chapter 5 presents the subject matter of artificial intelligence, focusing on machine learning, where these machines are artificial agents. It presents simple examples of unsupervised learning, supervised learning, and reinforcement learning, and introduces notions of dynamic programming, Q-learning, and stochastic control. After that, it explores some links that can be established between artificial intelligence and the philosophy of mind, presenting and discussing the Turing test, the philosophical approaches of eliminativism and functionalism, and the problem of tacit knowledge.
In this chapter we present a self-contained chapter on dynamic programming in continuous time in the framework of jump diffusions driven by a marked point process. We derive the relevant HJB equation and we study some examples of standard control as well as intensity control.
This chapter is the start of a new part of the book, devoted to optimal control. The chapter consists of a self-contained course on dynamic programming in discrete time.
We consider a gradual-impulse control problem of continuous-time Markov decision processes, where the system performance is measured by the expectation of the exponential utility of the total cost. We show, under natural conditions on the system primitives, the existence of a deterministic stationary optimal policy out of a more general class of policies that allow multiple simultaneous impulses, randomized selection of impulses with random effects, and accumulation of jumps. After characterizing the value function using the optimality equation, we reduce the gradual-impulse control problem to an equivalent simple discrete-time Markov decision process, whose action space is the union of the sets of gradual and impulsive actions.
Bounded treewidth is one of the most cited combinatorial invariants in the literature. It was also applied for solving several counting problems efficiently. A canonical counting problem is #Sat, which asks to count the satisfying assignments of a Boolean formula. Recent work shows that benchmarking instances for #Sat often have reasonably small treewidth. This paper deals with counting problems for instances of small treewidth. We introduce a general framework to solve counting questions based on state-of-the-art database management systems (DBMSs). Our framework takes explicitly advantage of small treewidth by solving instances using dynamic programming (DP) on tree decompositions (TD). Therefore, we implement the concept of DP into a DBMS (PostgreSQL), since DP algorithms are already often given in terms of table manipulations in theory. This allows for elegant specifications of DP algorithms and the use of SQL to manipulate records and tables, which gives us a natural approach to bring DP algorithms into practice. To the best of our knowledge, we present the first approach to employ a DBMS for algorithms on TDs. A key advantage of our approach is that DBMSs naturally allow for dealing with huge tables with a limited amount of main memory (RAM).
Emergency search and rescue on the sea is an important part of national emergency response for marine perils. Optimal route planning for maritime search and rescue is the key capability to reduce the searching time, improve the rescue efficiency, as well as guarantee the rescue target’s safety of life and property. The main scope of the searching route planning is to optimise the searching time and voyage within the constraints of missing search rate and duplicate search rate. This paper proposes an optimal algorithm for searching routes of large amphibious aircraft corresponding to its flight characteristics and sea rescue capability. This algorithm transforms the search route planning problem into a discrete programming problem and applies the route traceback indexes to satisfy the duplicate search rate and missing search rate.
There is no silver bullet: no model can fit all data. Hence, special data requires special algorithms. In this chapter, we deal with two types of special data: sparse data and sequences that can be aligned to each other. We will not dive deep into sparsity learning, which is very complex. Rather, we introduce key concepts: sparsity inducing loss functions, dictionary learning, and what exactly the word sparsity means. For the second part in this chapter, we introduce dynamic time warping (DTW), which deals with sequences that can be aligned with each other (but there are sequences that cannot be aligned, which we will discuss in the next chapter). We use our old tricks: ideas, visualizations, formalizations, to reach the DTW solution. The key idea behind its success is divide-and-conquer and the key technology is dynamic programming.
Defined contribution (DC) pension plans have been gaining ground in the last 10–20 years as the preferred system for many countries and other agencies, both private and public. The central question for a DC plan is how to invest in order to reach the participant's retirement goals. Given the financial illiteracy of the general population, it is common to offer a default policy for members who do not actively make investment choices. Using data from the Chilean system, we discuss an investment model with fixed contribution rates and compare the results with the existing default policy under multiple objectives. Our results indicate that the Chilean default policy has good overall performance, but specific closed-loop policies have a higher probability of achieving desired retirement goals and can reduce the expected shortfall at retirement.
This paper investigates the random horizon optimal stopping problem for measure-valued piecewise deterministic Markov processes (PDMPs). This is motivated by population dynamics applications, when one wants to monitor some characteristics of the individuals in a small population. The population and its individual characteristics can be represented by a point measure. We first define a PDMP on a space of locally finite measures. Then we define a sequence of random horizon optimal stopping problems for such processes. We prove that the value function of the problems can be obtained by iterating some dynamic programming operator. Finally we prove via a simple counter-example that controlling the whole population is not equivalent to controlling a random lineage.
We consider a two-player zero-sum stochastic differential game with a random planning horizon and diffusive state variable dynamics. The random planning horizon is a function of a non-negative continuous random variable, which is assumed to be independent of the Brownian motion driving the state variable dynamics. We study this game using a combination of dynamic programming and viscosity solution techniques. Under some mild assumptions, we prove that the value of the game exists and is the unique viscosity solution of a certain nonlinear partial differential equation of Hamilton–Jacobi–Bellman–Isaacs type.
This paper deals with the optimal management of the public debt-to-GDP ratio. We specifically focus on a contrasting tax evasion-based strategy for controlling the debt-to-GDP ratio. Two devices can be employed by the policymaker: by the one side, the tax rate is to be applied to the tax payers; by the other side, the monitoring activity is to be performed in order to detect the evaded taxes. To pursue our scopes, a stochastic control problem is developed and solved. Some numerical experiments validate the theoretical proposal and lead to an intuitive discussion of the obtained findings.
We consider a polling system with two queues, exhaustive service, no switchover times, and exponential service times with rate µ in each queue. The waiting cost depends on the position of the queue relative to the server: it costs a customer c per time unit to wait in the busy queue (where the server is) and d per time unit in the idle queue (where there is no server). Customers arrive according to a Poisson process with rate λ. We study the control problem of how arrivals should be routed to the two queues in order to minimize the expected waiting costs and characterize individually and socially optimal routeing policies under three scenarios of available information at decision epochs: no, partial, and complete information. In the complete information case, we develop a new iterative algorithm to determine individually optimal policies (which are symmetric Nash equilibria), and show that such policies can be described by a switching curve. We use Markov decision processes to compute the socially optimal policies. We observe numerically that the socially optimal policy is well approximated by a linear switching curve. We prove that the control policy described by this linear switching curve is indeed optimal for the fluid version of the two-queue polling system.
A model to value Federal Agricultural Mortgage Corporation (Farmer Mac) agricultural mortgage-backed securities (AMBS) is developed and numerically solved. The results suggest prepayment penalties currently being used by Farmer Mac reduce yields on AMBS considerably. Even with prepayment penalties, it can be advantageous for profit maximizing mortgagors to optimally prepay or even default on agricultural mortgages. The model is used to quantify prepayment and default risk by valuing the embedded options in the mortgages. Monte Carlo simulation is also used to determine the probability of optimal prepayment given the term structure assumption used to develop the model.
Little research exists on the optimal temporal frequency between soil tests, given empirical data on potassium (K) carryover and its interaction with cotton yield. We evaluate how decreasing the temporal frequency between obtaining K soil test information affects the net present value (NPV) of cotton production. Monte Carlo simulation was used to determine NPV for cotton production using five soil test schedules ranging from soil testing annually to every fifth year. NPV of returns to K was maximized at $7,580/ac. when producers updated soil testing information every 2 years, which was $2/ac. per year greater than annual soil testing.