Traditionally, the study of linguistics has focused on verbal communication. In the sense that linguistics is the scientific study of language, the approach is perfectly justified. Those working in the sub-discipline of linguistic pragmatics, however, are faced with something of a dilemma. The aim of a pragmatic theory is to explain how utterances are understood, and utterances, of course, have both linguistic and nonlinguistic properties. As well as this, current work in pragmatics emphasizes that the affective dimension of a speaker’s meaning is at least as important as the cognitive one, and it is often the nonlinguistic properties of utterances that convey information relating to this dimension. This chapter highlights the major role of nonverbal “modes” of communication (”multimodality”) in accounting for how meaning is achieved and explores in particular how the quasi-musical contours we impose on the words we say, as well as the movements of our face and hands that accompany speech, constrain the context and guide the hearer to our intended meaning. We build on previous exploration of the relevance of prosody (Wilson and Wharton 2006) and, crucially, look at prosody in relation to other nonverbal communicative behaviors from the perspective of Relevance Theory. In so doing, we also hope to shed light on the role of multimodality in both context construction and utterance interpretation and suggest prosody needs to be analyzed as one tool in a set of broader gestural ones (Bolinger 1983). Relevance Theory is an inferential model, in which human communication revolves around the expression and recognition of the speaker’s intentions in the performance of an ostensive stimulus: an act accompanied by the appropriate combination of intentions. This inferential model is proposed as a replacement for the traditional code-model of communication, according to which a speaker simply encodes into a signal the thought they wish to communicate and the hearer retrieves their meaning by decoding the signal they have provided. We will argue that much existing work on multimodality remains rooted in a code model and show how adopting an inferential model enables us to integrate multimodal behaviors more completely within a theory of utterance interpretation. As ostensive stimuli, utterances are composites of a range of different behaviors, each working together to form a range of contextual cues.