This chapter introduces (with production function as an example) functions of more than one variable. Then we define partial and second partial derivatives and explain how to calculate them, and present the chain rule for partial differentiation.

Generating the adjoint model (ADJM) by hand is tedious, time-consuming, and error prone. In most practical applications of data assimilation these days, the derivative codes, including the ADJM, are generated by the automatic differentiation (AD) tools, which evaluate the exact derivative information of a function in terms of a program. Terminologies and methods in AD are introduced, including the practical exclusion of the forward and reverse modes of differentiation. Various AD tools based on two major AD approaches, source transformation and operator overloading, are compiled with their webpages.

We give an introduction to the concept of relative entropy (also called Kullback-Leibler divergence). We interpret relative entropy in terms of both coding and diversity, and sketch some connections with other subjects: Riemannian geometry (where relative entropy is infinitesimally a squared distance), measure theory, and statistics. We prove that relative entropy is uniquely characterized by a short list of properties.

Ordinary probabilities are real numbers, and ordinary entropy is a real number too. Building on ideas of Kontsevich, we develop an analogue of entropy in which both the probabilities and entropy itself are integers modulo a prime number p. While the formula for entropy mod p is quite unlike the formula for real entropy, we prove characterization theorems for entropy mod p and information loss mod p that are very closely analogous to the theorems over the real numbers, thus justifying the definition. We also establish a sense in which entropy mod p is the residue mod p of real entropy.

We introduce two families of deformations of Shannon entropy: the q-logarithmic entropies (also called “Tsallis entropies”) and the Rényi entropies. We explain how the exponentials of the Rényi entropies, called the Hill numbers, convey information about the diversity and structure of an ecological community. We introduce the power means, which lie at the technical heart of this book. We give functional equations characterizing the q-logarithmic entropies on the one hand, and the Hill numbers on the other.

Any deterministic process loses information, and one can quantify the amount of information lost. Information loss is a generalization of entropy, and in some ways is a better-behaved quantity, being more functorial. We give a simple axiomatic characterization of information loss.

We give a short introduction to some classical information-theoretic quantities: joint entropy, conditional entropy and mutual information. We then interpret their exponentials ecologically, as meaningful measures of subcommunities of a larger metacommunity. These subcommunity and metacommunity measures have excellent logical properties, as we establish. We also show how all these quantities can be presented in terms of relative entropy and the value measures of the previous chapter.

We introduce finite probability distributions and their use as a model of an ecological community. We define Shannon entropy, give examples, and establish its basic properties. We interpret Shannon entropy in terms of both coding and diversity, and prove that it is uniquely characterized by the chain rule.