We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.
To save content items to your account,
please confirm that you agree to abide by our usage policies.
If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account.
Find out more about saving content to .
To save content items to your Kindle, first ensure no-reply@cambridge.org
is added to your Approved Personal Document E-mail List under your Personal Document Settings
on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part
of your Kindle email address below.
Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations.
‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi.
‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
This enthusiastic introduction to the fundamentals of information theory builds from classical Shannon theory through to modern applications in statistical learning, equipping students with a uniquely well-rounded and rigorous foundation for further study. Introduces core topics such as data compression, channel coding, and rate-distortion theory using a unique finite block-length approach. With over 210 end-of-part exercises and numerous examples, students are introduced to contemporary applications in statistics, machine learning and modern communication theory. This textbook presents information-theoretic methods with applications in statistical learning and computer science, such as f-divergences, PAC Bayes and variational principle, Kolmogorov's metric entropy, strong data processing inequalities, and entropic upper bounds for statistical estimation. Accompanied by a solutions manual for instructors, and additional standalone chapters on more specialized topics in information theory, this is the ideal introductory textbook for senior undergraduate and graduate students in electrical engineering, statistics, and computer science.
This chapter introduces communication and information theoretical aspects of molecular communication, relating molecular communication to existing techniques and results in communication systems. Communication models are discussed, as well as detection and estimation problems. The information theory of molecular communication is introduced, and calculation of the Shannon capacity is discussed.
We develop and demonstrate a computationally cheap framework to identify optimal experiments for Bayesian inference of physics-based models. We develop the metrics (i) to identify optimal experiments to infer the unknown parameters of a physics-based model, (ii) to identify optimal sensor placements for parameter inference, and (iii) to identify optimal experiments to perform Bayesian model selection. We demonstrate the framework on thermoacoustic instability, which is an industrially relevant problem in aerospace propulsion, where experiments can be prohibitively expensive. By using an existing densely sampled dataset, we identify the most informative experiments and use them to train the physics-based model. The remaining data are used for validation. We show that, although approximate, the proposed framework can significantly reduce the number of experiments required to perform the three inference tasks we have studied. For example, we show that for task (i), we can achieve an acceptable model fit using just 2.5% of the data that were originally collected.
This Element tries to discern the known unknowns in the field of pragmatics, the 'Dark Matter' of the title. We can identify a key bottleneck in human communication, the sheer limitation on the speed of speech encoding: pragmatics occupies the niche nestled between slow speech encoding and fast comprehension. Pragmatic strategies are tricks for evading this tight encoding bottleneck by meaning more than you say. Five such tricks are reviewed, which are all domains where we have made considerable progress. We can then ask for each of these areas, where have we neglected to push the frontier forward? These are the known unknowns of pragmatics, key areas, and topics for future research. The Element thus offers a brief review of some central areas of pragmatics, and a survey of targets for future research. This title is also available as Open Access on Cambridge Core.
This chapter describes new music in Paris in the late 1960s, the period when the young spectral composers were students at the Paris Conservatoire. It opens with an account of Messiaen’s composition class and how elements such as neumes and Messiaen’s analyses of Debussy and Ravel informed Grisey’s, Murail’s, and Levinas’s emerging musical sensibilities. After giving a brief biographical account of those latter composers and Roger Tessier, the chapter touches on serialism’s changing status at a time when it had begun to be taught at the Paris Conservatoire; the effect of May ’68 on the Conservatoire’s pedagogy and on musical mores more generally among young composers; Fifth Republic France’s increased funding for new music festivals in regional cities such as Royan; Boulez and Xenakis’s profiles as the two most influential composers in France; and collectives, aleatoricism, and music theatre in post-1968 composition. The chapter closes with an account of Grisey’s early student works, in particular their creative adaptation of Messiaen’s personnages sonores concept towards the construction of audibly distinct musical figures, which would become a key element in Grisey’s musical style.
The composer-performer collective l’Itinéraire, founded in 1973 by Murail and Tessier, was the de facto platform for spectral music in France. This chapter shows how l’Itinéraire formed with the aim of establishing an organ for the music of the youngest generation of French composers, and how, with the demise of Boulez’s Domaine Musical ahead of the opening of IRCAM, l’Itinéraire fortuitously found itself positioned as the successor to that Parisian new music series, endorsed by Messiaen and recipient of a state subsidy. The chapter details how Grisey composed Périodes, a l’Itinéraire commission and the first work composed from his Les Espaces acoustiques cycle, and how Grisey attended the acoustics laboratory at the Université Paris VI Jussieu, where, alongside lessons in musical acoustics, he absorbed the work of Abraham Moles on the application of information theory to art. The chapter also shows how Murail began to incorporate the models of spectral harmonicity and periodicity into his music from Tigres de verre onwards, and it explores the relationship of these instrumental compositional techniques to the computer sound synthesis work of Risset and Chowning.
The book’s conclusion situates spectral music as a modernist musical movement. It shows how spectral music reprises many of serialism’s concerns, albeit on a more psychoacoustically accurate level. It relates the debates between Levinas and the other spectral composers to an older debate about formalism in art between Flaubert and Sand. Finally, the book concludes by situating Grisey as the founder of spectral music.
Spectral music as a distinct movement began in 1976, when, within a few days of each other, Murail’s Mémoire-érosion and Grisey’s Partiels were both premiered by Ensemble l’Itinéraire. This chapter explores how, driven by the theorist Dufourt, the young composers associated with l’Itinéraire developed a theoretical identity in contradistinction to Boulez and IRCAM. As well as detailing the salient qualities of Grisey and Murail’s music in this period, the chapter explores the diverse spectral music of Dufourt, Levinas, and Tessier. Dufourt’s works Erewhon, La tempesta d’après Giorgione, and Saturne engage with insights regarding sound related to his encounters with Risset and Chowning. Levinas’s works like Appels foregrounded sonic parasitism and a dramatic spectacle far removed from the more reserved forms of Murail, of which the chapter shows Levinas to have been at times a public critic. Tessier’s music in this period was expressionistic and explored electroacoustic resources. As well as detailing these various spectral sub-currents, the chapter explores the role of l’Itinéraire’s performers in helping to develop performing techniques adequate to the spectral writing.
The introductory chapter to Gérard Grisey and Spectral Music: Composition in the Information Age situates the book’s historical narrative by focusing on correspondence between Grisey and Dufourt in 1980 discussing what name they should give their common musical movement:’ spectral music’ or ‘liminal music’. This matter of naming indicates the compositional values the composers prioritised: movement over stasis, thresholds over states, psychoacoustic phenomena over traditional notes and pitches. The chapter then gives an overview of the book’s argument that spectral music developed from serialism through embracing information theory and developments in psychoacoustics and computer sound synthesis. Inasmuch as it arose in France but depended on developments that occurred at Bell Telephone Laboratories in the USA, spectral music was transatlantic in origin and signified a paradigm shift in musical composition.
The first in-depth historical overview of spectral music, which is widely regarded, alongside minimalism, as one of the two most influential compositional movements of the last fifty years. Charting spectral music's development in France from 1972 to 1982, this ground-breaking study establishes how spectral music's innovations combined existing techniques from post-war music with the use of information technology. The first section focuses on Gérard Grisey, showing how he creatively developed techniques from Messiaen, Xenakis, Ligeti, Stockhausen and Boulez towards a distinctive style of music based on groups of sounds mutating in time. The second section shows how a wider generation of young composers centred on the Parisian collective L'Itinéraire developed a common vision of music embracing seismic developments in in psychoacoustics and computer sound synthesis. Framed against institutional and political developments in France, spectral music is shown as at once an inventive artistic response to the information age and a continuation of the French colouristic tradition.
This paper proposes a linear quadratic approximation approach to dynamic nonlinear rationally inattentive control problems with multiple states and multiple controls. An efficient toolbox to implement this approach is provided. Applying this toolbox to five economic examples demonstrates that rational inattention can help explain the comovement puzzle in the macroeconomics literature.
Since the 1960s Mastermind has been studied for the combinatorial and information-theoretical interest the game has to offer. Many results have been discovered starting with Erdős and Rényi determining the optimal number of queries needed for two colours. For $k$ colours and $n$ positions, Chvátal found asymptotically optimal bounds when $k \le n^{1-\varepsilon }$. Following a sequence of gradual improvements for $k\geq n$ colours, the central open question is to resolve the gap between $\Omega (n)$ and $\mathcal{O}(n\log \log n)$ for $k=n$. In this paper, we resolve this gap by presenting the first algorithm for solving $k=n$ Mastermind with a linear number of queries. As a consequence, we are able to determine the query complexity of Mastermind for any parameters $k$ and $n$.
Chapter 4 describes the field of cognitive science, which is the arena where all those who study “intelligent systems” (“minds“) get together to compare notes. A shared idea is that the mind can be understood as an information-processing computational system. We will see how during the 1960s renewed interest in the mind from different academic disciplines emerged as a reaction to the denial of the mind of an approach to psychology called behaviorism. We then discuss the various strands of thinking in a variety of different fields that led to this “cognitive revolution.” We learn that there are fundamental, opposing views in this field that are relevant to the nature–nurture debate. Despite differences, a general understanding within cognitive science is that the mind can be studied at different levels of abstractness and from different angles which to some extent compete but also complement each other.
This chapter shows how graphic novels have not only become more colorful over the last two decades but also more visually complex. The chapter surveys discussions of the graphic novel’s literary complexity, from Richard Corben’s early Bloodstar to Alan Moore’s Watchmen. This reappropriation of a modernist strategy models itself on the rise of the novel but also reacts to a long-term audience decline for comics. Section 4.2 builds on recent uptakes of complexity in digital literary and digital film studies to advance computational measurements that combine image and text recognition. The final pages return to Moore and Alison Bechdel to assess the relevance of complexity for the popular success and cultural prestige of individual comic books.
We show how convergence to the Gumbel distribution in an extreme value setting can be understood in an information-theoretic sense. We introduce a new type of score function which behaves well under the maximum operation, and which implies simple expressions for entropy and relative entropy. We show that, assuming certain properties of the von Mises representation, convergence to the Gumbel distribution can be proved in the strong sense of relative entropy.
A review of basic probability theory – probability density, expectation, mean, variance/covariance, median, median absolute deviation, quantiles, skewness/kurtosis and correlation – is first given. Exploratory data analysis methods (histograms, quantile-quantile plots and boxplots) are then introduced. Finally, topics including Mahalanobis distance, Bayes theorem, classification, clustering and information theory are covered.
Numerical estimators of differential entropy and mutual information can be slow to converge as sample size increases. The offset Kozachenko–Leonenko (KLo) method described here implements an offset version of the Kozachenko–Leonenko estimator that can markedly improve convergence. Its use is illustrated in applications to the comparison of trivariate data from successive scene color images and the comparison of univariate data from stereophonic music tracks. Publicly available code for KLo estimation of both differential entropy and mutual information is provided for R, Python, and MATLAB computing environments at https://github.com/imarinfr/klo.
There is a correlation between the phonological shape of a word and the word’s probability in use. Less probable words tend to be longer and more probable words shorter (see Piantadosi et al. , Zipf ). This has been attributed to the lexicon evolving for efficient communication (Zipf ). To identify less probable words, listeners need more information from the segments in the phonological word itself. In this case, longer lengths for less probable words mean a greater amount of information to be used in word identification. However, this does not take into account how listeners actually process words. Research in spoken word recognition has shown that words are processed incrementally and some segments may in fact be more informative (Allopenna et al. , Luce and Pisoni , van Son and Pols , Weber and Scharenborg ). Here, we use corpus data from American English to provide evidence that less probable words contain more informative segments. We also show that the distribution of segmental information is correlated with the word’s probability and that less probable words contain more of their total information in the early segments. We discuss these findings and possible evolutionary avenues for language to reach this state. This work provides support for the idea that the words in the lexicon evolve under pressure for efficient communication.
Anti-evolutionists often employ the closely related mathematical theories of information and combinatorial search. We introduce both of these fields, present the major anti-evolutionist arguments, and then explain why they are wrong.