From Media Technologies That Reproduce Seconds to Media Technologies That Reproduce Thirds: A Peircean Perspective on Stylistic Fidelity and Style-Reproducing Computerized Algorithms

Abstract

Against the backdrop of a long research tradition in linguistic and semiotic anthropology that has focused almost exclusively on media technologies that reproduce Seconds, that is, specific texts in modalities such as sound and the visual image, this article focuses on media technologies that reproduce Thirds, that is, generative dispositions responsible for the production of such Seconds. It explores contemporary attempts in the United States and France to develop computerized systems that, with the aid of specific algorithms, can abstract and enact the styles of different past jazz masters, as well as the styles of players who interactively improvise with such systems in real time. Drawing on Peircean semiotics, the article offers an analysis of these media technologies, their present application in the field of online consumption, and the cultural specificity of the Third that plays a key role in their development and reception, namely, style.

Theorizations of media technologies in linguistic and semiotic anthropology have focused almost exclusively on media technologies that reproduce the category Peirce called “Second,” that is, a specific text in modalities such as sound and the visual image. For example, Eisenlohr has studied devotional practices among Mauritian Muslims who use cassettes, CDs, or MP3 files that mediate emblematic recitations of na’t by master reciters (Eisenlohr 2010), and Wilf (2012) has studied the ways in which jazz students in the United States use the same technologies to learn canonical improvisations of past jazz masters (see also Kunreuther 2010; Spitulnik Vidali 2010; Weidman 2010). In such studies, the analytical focus has been the ways in which media ideologies, intertextuality, participation framework, and the politics of mediation come into play in, and shape the ways in which people mobilize, media technologies that reproduce specific texts.¹

Against this backdrop, the purpose of this essay is twofold. First, it aims to explore what media technologies that reproduce the category Peirce called “Third” would look like, and what forms of media ideologies, intertextuality, participation framework, and politics of mediation would find expression in such a form of mediation. Specifically, I focus on contemporary attempts in the United States and France to develop computerized systems that, with the aid of specific algorithms, can be trained to abstract and enact the styles of different past jazz masters, as well as the styles of players who interactively improvise with such systems in real time.² Thus, although these systems, much like a CD or an MP3 file, are technologies of reproduction, they do not reproduce specific texts, or Seconds, but styles, or Thirds. Their object of reproduction is the principle of generativity that is responsible for producing the specific texts that are the object of reproduction of the kind of media technologies that have traditionally stood at the center of linguistic and semiotic anthropological research. These systems’ reproduction of style consists both in their ability to abstract a style from a corpus of Seconds and to generate new and different Seconds or texts in this style, indefinitely so. Rather than being about acoustic fidelity (Sterne 2002), then, such technologies are about “stylistic fidelity.”³

A second and related goal of this article is to explore the cultural specificity of the Third that is the object of reproduction and mediation in the specific ethnographic settings I have studied, namely, style in modern art, and the ways in which this specificity brings many users who interact with technologies that reproduce and enact this type of Third to experience and draw on forms of intertextuality, participation framework, and media ideologies that are highly different than hitherto theorized with respect to more traditional media technologies.

Although I examine such technologies with respect to style in modern art as an object of reproduction, it should be clear from the outset that the phenomena I am concerned with here have implications that are wider than the field of algorithmic music composition and modern art. The kind of algorithms that animate the technologies I describe below has become ubiquitous worldwide. It is used in web-based search engines and social media interfaces such as Google and Facebook to anticipate users’ various styles based on their web-based behavior and thus to produce customized content and advertising. Against this backdrop, there is urgency in theorizing technologies that reproduce Thirds with respect to a number of analytical concerns that have traditionally stood at the center of linguistic and semiotic anthropological research.

The next two sections explore the difference between Secondness and Thirdness as it finds expression in key anthropological theories of culturally patterned behavior (via notions such as disposition and habitus), as well as in practices of socialization into jazz music that I recorded during my previous fieldwork in a US academic jazz music program. I then articulate the difference between technologies that reproduce Seconds and technologies that reproduce Thirds in light of this discussion, namely, the latter’s ability to abstract and enact a principle of generativity responsible for the production of an infinite succession of Seconds “in the same style.” I then articulate the cultural specificity of the notion of style in modern art and its intellectual history. After these preparatory stages, I proceed to examine the forms of intertextuality, participation framework, and media ideologies experienced by individuals in the course of their interaction with a specific technology that abstracts and enacts their musical style in real time. I end this article by fleshing out the relevance of these various points to contemporary forms of computer-mediated, algorithmic forms of sociality.

Style, Disposition, and Habitus as Thirds

Peirce offered a wide range of definitions of Firstness, Secondness, and Thirdness with respect to different spheres of inquiry such as metaphysics, psychology, physiology, and physics. Broadly speaking, however, he argued that “the First is that whose being is simply in itself, not referring to anything nor lying behind anything. The second is that which is what it is by force of something to which it is second. The third is that which is what it is owing to things between which it mediates and which it brings into relation to each other” (EP 1:248). In different places, Peirce called that which “brings [things] into relation,” “the cohesive principle,” the “regulative principle,” and the “synthetic” function (as well as “theory”; see Parmentier 1994, 34–35). He attributed advances in modern science to the ability to abstract a Third, or a regulative principle, from a set of Seconds, or discrete cases: “The superiority of modern geometry, too, has certainly been due to nothing so much as to the bridging over of the innumerable distinct cases with which the ancient science was encumbered; and we may go so far as to say that all the great steps in the method of science in every department have consisted in bringing into relation cases previously discrete” (EP 1:250). Peirce did not mean that a Third can only be an ad hoc explanation or theory that brings into relation already existing cases hitherto thought to be distinct but also a principle of generativity, “habit” (EP 1:264), and “fact of a general or orderly nature” (EP 1:275) responsible for generating future events that bear the stamp of this principle, habit, and fact.

Style is an example of a Third in precisely this respect, that is, it is a regulative principle that can be abstracted from distinct cases and used to predict future ones (Parmentier 1997, 49–51). Crucially, the difference between Secondness and Thirdness as articulated by Peirce apropos modern science has played an important role in a number of anthropological theories and analytical terms in which the notion of style as a Third was a key feature. In what follows I briefly discuss this point with respect to three key anthropologists to flesh out not only the features of Thirdness in itself but also the radical implications of a technology that reproduces Thirds (and more specifically styles) with respect to key anthropological theories and, as will become clearer below, contemporary forms of sociality.⁴

I begin with Pierre Bourdieu, who theorized habitus as a propensity or disposition, a generative scheme of action that does not stipulate specific actions in specific situations but rather the likelihood of types of actions to take place in types of situations.⁵ Bourdieu defined habitus specifically as “style”: “‘Personal’ style, the particular stamp marking all the products of the same habitus, whether practices or works, is never more than a deviation in relation to the style of a period or class so that it relates back to the common style not only by its conformity … but also by the difference which makes the whole ‘manner.’ The principle of these individual differences lies in the fact that … the habitus … brings about a unique integration dominated by the earliest experiences, of the experiences statistically common to the members of the same class” (Bourdieu 1977, 86–87; emphasis added). This principle of “unique integration” is a Third. Bourdieu urged the analyst to move away from the collection of Seconds to an understanding of Thirds, that is, from “objects or acts to the principles of their production … from the fait accompli … to analogical practice as scheme transfer carried out by the habitus … [and] enabling the agent to master by a sort of practical generalization all similar problems likely to arise in new situations” (Bourdieu 1977, 119). This emphasis underlies Bourdieu’s rejection of objectivist approaches in the social sciences that attempt to explain social behavior as a set of discrete rules of action that assign specific actions to specific situations. Instead he offered “strategy” as a more accurate explanatory model for practice, being a set of embodied dispositions and generative schemas that are unconsciously acquired in childhood and that account for agents’ ability to improvise within various situations in a way that, nevertheless, accords with and reproduces their positions within the social structure (Lamaison and Bourdieu 1986).⁶

Clifford Geertz is another key anthropologist who invoked the notion of style as disposition and the distinction between Seconds and Thirds to theorize culturally patterned behavior. At one point in The Interpretation of Cultures, Geertz argued that “sacred symbols function to synthesize a people’s ethos” (Geertz 1973, 89), creating what at another point in the book he characterized as “a unity of style” (Geertz 1973, 145): “[Religious symbols] both express the world’s climate and shape it. They shape it by inducing in the worshipper a certain distinctive set of dispositions (tendencies, capacities, propensities, skills, habits, liabilities, pronenesses) which lend a chronic character to the flow of his activity and the quality of his experience. A disposition describes not an activity or an occurrence but a probability of an activity being performed or an occurrence occurring in certain circumstances” (Geertz 1973, 95; emphasis added). Note Geertz’s emphasis on the distinction between Seconds, or distinct activities, and Thirds, or dispositions generating new activities “in the same style.” Geertz argued that true understanding of culture lies in accounting for such dispositions rather than in documenting Seconds.

Moving to the field of art, Alfred Gell attempted to provide “‘generative’ (abstract, high-level) style descriptions for ‘ethnological’ art” (Gell 1998, 158), pointing out that stylistic coherence is the result of a limited number of “rules of transformation” that crystallize key cultural values and norms and that stipulate the ways in which motifs can be combined with and transformed into one another: “It is a field of possible or legitimate motivic transformations, rather than the totality of existing instantiations of such transformations” (Gell 1998, 215). This definition led Gell to argue that “style … is ‘relations between relations’ of forms” (Gell 1998, 215; emphasis added), that is, it is not a totality of Seconds but rather the Third or the generative principle responsible for their production.⁷

What Lacking a Habitus Looks Like; or, Being Limited to Reproducing Seconds

The preceding discussion suggests that the distinction between Secondness and Thirdness has been prominently featured in key anthropological theories and, therefore, that the distinction between technologies that reproduce Seconds and technologies that reproduce Thirds might have broad theoretical implications. To anticipate what the difference between these two kinds of technologies consists of in the context of jazz music—the context in which the technologies I discuss below were developed, I want to turn to a vignette I recorded during my fieldwork at a US academic jazz music school from July 2006 to June 2007.⁸ The vignette tells the story of a novice player at the stage in which he could only function as a “technology” that reproduces Seconds.

In classes, clinics, method books, and informal conversations with students, jazz educators stress time and again the importance of transcribing the recorded solos of the past jazz masters and practicing them in their entirety or dividing them into selected “licks” and phrases and practicing these excerpts in every key. This aspect of jazz training has been widely documented by jazz scholars (Berliner 1994). Its purpose is to have students incorporate into their playing bodies prototypical features of canonical improvisations. When I interviewed Henry, a teacher at the school in which I conducted fieldwork, about his own training as a young, aspiring student, the potentialities and limitations of this form of training came to the fore:

[A friend] gave me a Kenny Dorham album to listen to.⁹ And one of the things I knew after studying this album was this blues called “Double Clutching.” I learned it note for note with the record. … So now I had a five chorus blues in F transcription that I could play. And I knew it inside out, man, and it was swinging. Not many people knew that album, at all, you know. And it was a good mean tempo. So I went to this jam session once, and these guys were good players, better than me, and they didn’t know me at all and they said, “so what do you want to do?” So I said, “eh [mimicking hesitation], let’s do a blues in F.” We played Billy’s Bounce or some kind of a head [i.e., melody], you know, and then I played that solo [i.e., Dorham’s solo]. Five choruses. Man, when I got done playing that solo these guys were like: “Wow, man, this cat’s heavy! Who is this guy?” You know, and I felt good, man. And ah, the only thing is, though, that was the only blues solo I could play. [laughing] And then after I played that solo they said: “Yeah, man, what do you want to do now, man?” And I was like, “Wow, I tell you, man, I think I gotta go. I’m feeling really shitty.” And I pretended that I was coming down with something, you know, feeling terrible, and I left. And wow! [relief] I got out of there.

Henry’s story reveals both the potentialities and limitations of learning the masters’ solos tout court and performing them in their entirety in a live setting. On the one hand, the audience took Henry’s performance to be a real-time feat of his creativity because jazz is an improvised form of music, that is, it is meant to be a form of composition on the spur of the moment and the audience did not know that Henry’s performance was a performance of one of Kenny Dorham’s recorded solos. Henry meticulously learned this solo with the aid of a recording and played it with a rhythm section with such conviction that the audience believed he was the solo’s author and not just its “animator” (Goffman 1981).¹⁰ The audience was impressed because the solo is, indeed, a masterpiece. When recounting another incident in which he did the same thing with a Freddie Hubbard solo, Henry told me that “the crowd went crazy.”¹¹

On the other hand, note the severe limitations of such a practice. Henry’s ability to simulate creativity was not generative: it was limited to the scope of that chunk of creativity that is a specific recorded solo of a Kenny Dorham or a Freddie Hubbard. When he was asked to display further creativity on other tunes he pretended he was sick and left the stage. Henry clearly felt his own limitations by practicing this kind of reproduction, which was limited to finite and fixed stretches of discourse, or Seconds, finiteness that finds a clear expression in Peirce’s definition of Secondness: “Just as the first is not absolutely first if thought along with a second, so likewise to think the second in its perfection we must banish every third. The second is therefore the absolute last. … Note, too, that for the second to have the Finality that we have seen belong to it, it must be determined by the first immovably, and thenceforth be fixed; so that unalterable fixity becomes one of its attributes” (EP 1:248–49; emphasis added). In this same paragraph Peirce likens a Second to “dead matter, whose existence consists in its inertia.” The notion of limitation is part and parcel of Peirce’s understanding of the consciousness and experience of Secondness: “[Dual consciousness] consists of a sense of ‘can’ which is at the same time a sense of ‘cannot’. Force implies resistance, and power limitation” (EP 1:283). Thus whatever Henry gained by copying the masters’ solos in public he lost when the stretch of copied discourse was over and he became fully aware of his own substantial limitations as an improviser. He soon realized that his is an experience of being a Second, a mere “Patient” in relation to another “Agent” (EP 1:249)—“unalterable fixity” or “dead matter.” Henry lacked a principle of generativity or Thirdness, which is the essence of style.

In contrast, consider one scene in the movie ’Round Midnight (1986), in which the movie’s main protagonist, Dale Turner, sits with a French admirer, Francis, to eat dinner in the latter’s apartment. Dale is an old, once-famous, African American jazz musician who struggles with alcoholism and various other demons. While Francis dilutes wine with water to serve Dale, the latter says, “All these young kids sound the same, just like they had the same teacher.” Francis asks him, “It was you?” Dale responds, “Yeah, me, and a few others. …You know, one night in Brooklyn, this tenor player comes in. And he sits down and he listens. And then he comes up to me and says: ‘I play you better than you’” (’Round Midnight 1986, 46:26–47:25).

What does it mean to play “another player” better than that player? The answer is that the intense study of numerous recordings of a past master, that is, his Seconds, by a student can sometimes result in the student’s being able to assimilate the master’s style of improvisation, his characteristic Third. This, as opposed to the form of imitation practiced by Henry, does not entail the replication of specific improvisational texts of finite duration but rather the inhabitance of a generative principle that allows a student to sound like the master by producing new improvisations in the master’s style. The young player who approached Dale and told him that he plays “you better than you” did not mean that he can play Dale’s past recorded solos better than Dale had played them. Rather, he meant that he had managed to inhabit and perfect Dale’s style of improvisation and generative principle responsible for his past and future improvisations.¹²

If Henry is an anthropomorphized technology that reproduces Seconds, the young player in the movie is an anthropomorphized technology that reproduces Thirds. Moving back to media technologies, what would a technology that reproduces Thirds look like and what would be its distinguishing features in comparison to a technology that reproduces Seconds?

Technologies That Reproduce Thirds: Abstracting and Enacting a Principle of Generativity

In the summer of 2011, I conducted fieldwork in a lab at a major institute of technology in the United States, in which computer scientists were in the process of developing a humanoid robot marimba player that, thanks to computerized algorithms, can abstract and enact the styles of different past jazz masters.¹³ As part of its training, these algorithms perform statistical analysis on databases that consist of files of different masters’ past recorded solos.¹⁴ In actual playing sessions, these algorithms instruct the robot, which I call Syrus, what to play based on this analysis. The algorithms, known as Markov Models, have been used in algorithmic music composition ever since the 1950s (Nierhaus 2010). They are especially suitable for style imitation based on the analysis of large corpora of music. One of their key features is the fact that they integrate stochastic processes into their logic in such a way that their output is seldom repetitious and predictable. For example, for each jazz master in whose style they want Syrus to play, the members of the research team create a large database of this master’s solos. These solos are in a MIDI (musical instrument digital interface) format, which means that the files can be fed into a computer program that can break the musical information into chains of pitch and rhythm data that are represented numerically. These data are then analyzed against chord changes score files (i.e., the harmonic sequences on which the player improvised).¹⁵ The system statistically analyzes this corpus to generate transition probabilities, that is, the probability that a certain future state will follow a given present state. During performance, and for each note played by Syrus, the system constantly searches for a match between the last sequence of notes performed by the player who plays with Syrus (if Syrus takes turns with a player) or by Syrus itself (if Syrus improvises by itself) and the chains of pitch and rhythm values derived from the jazz master’s corpus, which are stored in the system’s memory. The length of the sequence is determined in advance (two or three notes each time, for example). Any such search yields a number of candidates. The system chooses stochastically—that is, based on chance decisions weighted by a function of likelihood, itself determined by the statistical analysis. When a matched sequence is selected from the system’s database of the master’s solos, the system instructs Syrus to play the note that continues this sequence as it appears in the memory—that is, to play the note that the master had played after he or she played that specific sequence. The system’s decisions (i.e., the notes Syrus plays) feed back in real time as new input, and thus the decision process begins again. All this computation takes place in real time and in a split second prior to every note Syrus plays.¹⁶

The computerized algorithms that animate Syrus’s playing are not fixed rules that stipulate what specific phrases should be played in a given harmonic situation. Rather, they generate musical content based on probability functions and chance decisions in accordance with the frequencies detected in the jazz masters’ solos when subjected to a statistical analysis. Hence, although they are not repetitious, they do reveal a trend, a propensity, or a pattern—in short, a style—over time. The internal logic of these algorithms as they are used in the lab to simulate style closely resembles Bourdieu’s and Geertz’s notions of culture as a style or disposition in the sense that both Bourdieu and Geertz use tropes of statistics and probability to denote style as a propensity to act in a certain way rather than as a set of rules that stipulate the specific actions that need to be performed in specific situations.

I want to argue that two crucial distinguishing features of technologies that reproduce Thirds, which separate them from technologies that reproduce Seconds, are precisely their ability to synthesize or abstract a disposition or style from discrete cases or Seconds and to enact this disposition and thereby produce new Seconds “in the same style.” All that a technology that reproduces isolated texts of the same artist, or Seconds, can do with respect to style is to provide users with raw material from which users would need to abstract a style. In other words, it is the user who synthesizes or abstracts a certain style from the isolated texts reproduced by a technology that mediates Seconds. Here enters the phenomenon of type intertextuality as abstracted by individuals, that is, an “interpreter’s retrospective or recuperative relationship … to an internalized notion of a type or genre of discursive event” (Silverstein 2005, 9; emphasis added), or “the link” individuals might make not “to isolated utterance, but to generalized or abstracted models of discourse production and reception” (Briggs and Bauman 1992, 147; emphasis added). In such cases it is the individual’s strategic negotiation of meaning in real-time communicative events or her skills of abstraction and synthesis that bring into awareness and existence a Third, a style, or a genre. Hence the following claim made apropos traditional technologies that reproduce Seconds, namely, that “electronic reproduction of voice facilitates authentic transmission of both the ‘correct’ poetic text and the appropriate performative style” (Eisenlohr 2009, 283), can be held true as long as it remains clear that the recording itself does not reproduce style or a “typified speech genre” (Eisenlohr 2010, 327) or any other genre in whatever modality. Rather, style or genre is abstracted by the users of such technologies from the relations between the relations between a number of recordings within the same genre or from the relations between the relations between the parts of one recorded text.

In contrast, a technology that reproduces Thirds or a style or a habit or a disposition is a technology that itself performs such an abstraction of a Third from a given corpus of Seconds and is thereby able to enact this Third and indefinitely generate new and novel instantiations of such a style. To abstract and enact Thirds is precisely the role performed by the algorithms that animate Syrus, the humanoid robotic marimba player that can play in the style of different jazz masters. Syrus is not evaluated in terms of acoustic fidelity (Sterne 2002) because it is not a technology that reproduces Seconds in the modality of sound; it is evaluated in terms of stylistic fidelity, that is, its ability to abstract a style or a regulative principle from a given corpus of jazz solos as evinced by its ability to generate an endless series of new solos in the style of the specific jazz master whose corpus it has “analyzed.” To understand the radical implications of a technology that can reproduce a style, and more so in the field of art, and hence to understand the experience reported on by individuals who interact with such a technology, as I will discuss below, it is first necessary to unpack the cultural specificity of the notion of style in modern art.

Style in Modern Art: A Legisign-Icon-Rheme

At its core, jazz is structured by dominant modern normative ideals of creative agency that have a specific intellectual history in the West. These ideals received an emblematic expression in an interview I conducted with Joe, a well-known American saxophonist, during a break in a semester-long clinic he gave at the jazz school at the time of my fieldwork there. Throughout the semester, Joe emphasized time and again the importance of “feeling” with respect to creativity. At one point in the interview, I asked him to articulate in greater detail the relation between creativity and feeling. Joe responded with the following words:

I mean, what is creativity? What is jazz? Jazz is an expression of who you are in your life and how deep you get into music, the elements of music, the theory of music, and how you can express it as a player. You know, what is it to create a solo? I mean, let’s go back to the very beginning. The blues and the music—it’s all about feeling. Totally. If you copy the way that someone played and play it, all you do is trying to play what they played. And if they played it with feeling, you can only try to play like their feeling. That’s not you. What’s your feeling all about? And that will only go so far, too. You can only do that for 20 minutes, maybe 2 minutes, maybe 1 minute. Maybe 3 hours! But a lifetime? Or a complete record? Or a 90 minute set? Or take after take? That approach is an easy one to teach. It’s like if you tried to be a painter and you tried to copy a Van Gogh, or a Monet, right? What will you end up with? A Monet on a bad day or a Van Gogh on a bad day. You know what I’m saying?

The notion of feeling that is featured so prominently in Joe’s commentary is a core dimension of modern normative ideals of creative agency. It has its origins in Sentimentalism, a movement that developed from Protestant Pietism in mid-eighteenth-century Europe (see Wilf 2011). Sentimentalism promoted an ideal character type, which was based on the subject’s susceptibility to his or her tender feelings and to those of other subjects—what one scholar called “the ethic of feeling” (Campbell 1989)—against the backdrop of and as a reaction to neoclassicist aesthetics that relied upon the application of formal rules. In this context, “the identification of beauty and the formulation of the good … could … be ascertained merely by ‘trusting to one’s feelings’” (Campbell 1989, 151). At the end of the eighteenth century, Sentimentalism evolved into full-blown Romanticism. Drawing on organic metaphors, key Romantics argued that each individual has his or her own nature or voice or structure of feeling with which he or she must be in touch and to which he or she must remain faithful, and that copying or adhering to external rules or models of feeling amounts to distorting this inner voice: “This notion of an inner voice or impulse, the idea that we find the truth within us, and in particular in our feelings—these were the crucial justifying concepts of the Romantic rebellion in its various forms” (Taylor 1989, 368–69; emphasis added).¹⁷

Joe’s ample use of the trope of feeling as a source of action also reverberates with Peirce’s description of Firstness, which he described in various places as “pure Feeling” (EP 1:282, 258–59). When dividing consciousness into its three elements, Peirce took feeling to be the basis of “single consciousness,” which he characterized thusly:

The idea of the absolutely First must be entirely separated from all conception of or reference to anything else; for what involves a second is itself a second to that second. The First must therefore be present and immediate, so as not to be second to a representation. It must be fresh and new, for if old it is second to its former state. It must be initiative, original, spontaneous, and free; otherwise it is second to a determining cause. … [It is] first, present, immediate, fresh, new, initiative, original, spontaneous, free, vivid, conscious, and evanescent. (EP 1:248; emphasis added)

Against this backdrop, it is easy to see why Joe recruits the ethic of feeling to secure his notion of creativity. Feeling, as it is defined in this context, is presumably a sui generis, autochthonous quality (EP 1:41)—that is, something that stands in no relation to something else and that can thus support modern normative ideals of creative originality.¹⁸

Note, however, that Joe treats feeling as a consistent quality. He argues that each person has his or her unique structure of feeling that is part of his or her identity. To reiterate, Joe argues this about copying: “It’s like if you tried to be a painter and you tried to copy a Van Gogh, or a Monet, right? What will you end up with? A Monet on a bad day or a Van Gogh on a bad day.” The terms “Monet” and “Van Gogh” designate original styles, where style is a combination of Firstness—a feeling or quality that is unique to these artists, and Thirdness—this feeling or quality being consistent, persistent and continuous, that is, a form of Thirdness (Colapietro 1989, 82–83), which is responsible for the generation of Seconds, or specific works of art in this style.¹⁹ Within Peirce’s classification of signs, then, the general idea of style as understood in the framework of these modern normative ideals of creativity is an iconic legisign, or a legisign-icon-rheme, that is, “any general law or type, in so far as it requires each instance of it to embody a definite quality which renders it fit to call up in the mind the idea of a like object” (PWP, 116). Peirce gives the example of “a diagram, apart from its factual individuality. … Being an icon, it must be a rheme. Being a Legisign, its mode of being is that of governing single Replicas, each of which will be an Iconic Sinsign of a peculiar kind” (PWP, 116). However, style, specifically in modern art, inasmuch as it is understood to be a generative principle governing the production of tokens, each of which is indexically iconic of an object—that is, of a specific artist’s creative nature—is “an indexical legisign embodying an iconic legisign” (Parmentier 1997, 50).

Indeed, most of the various strands of Romantic ideology were premised on the assumption that each individual has a unique and consistent nature or structure of “feeling” as a source of action: “This is the idea which grows in the late eighteenth century that each individual is different and original, and that this originality determines how he or she ought to live” (Taylor 1989, 375). This assumption was the basis for claims made by Romantic thinkers for the institution of the copyright. Such claims stipulated that the form of an artwork is a reflection of the artist’s unique nature and that this form is an inalienable possession that cannot be copied by other people without violating one of the artist’s basic rights (Woodmansee 1996, 51). What can pass on to other individuals upon purchasing a book, for example, are the physical material of which the book consists and the ideas it contains but not their specific form, which is a reflection of the artist’s nature because works of art “grow spontaneously from a root, and by implication, unfold their original form from within” (Woodmansee 1996, 54). The specific form or composition in which these ideas are rendered, then, remains the author’s property. Fichte, one of the key architects of these ideas, defined this form precisely as a diagram in an essay written in 1793:

Each individual has his own thought process, his own way of forming concepts and connecting them. … All that we think we must think according to the analogy of our other habits of thought; and solely through reworking new thoughts after the analogy of our habitual thought processes do we make them our own. … Each writer must give his thoughts a certain form, and he can give them no other form than his own because he has no other. But neither can he be willing to hand over this form in making his thoughts public, for no one can appropriate his thoughts without thereby altering their form. This latter thus remains forever his exclusive property. (Quoted in Woodmansee 1996, 51–52; emphasis added)

Note that Fichte’s argument combines Firstness—the notion of an original and autochthonous structure of feeling or nature, and Thirdness—the notion of the consistency and habituality of such a structure that one cannot escape, with the result being the production of Seconds, or concrete works of art that bear the mark of a specific style.

Learning someone’s style, or Third, then, is far more subversive than copying his or her specific output, or Seconds. It challenges the argument made by Joe that “it’s like if you tried to be a painter and you tried to copy a Van Gogh, or a Monet, right? What will you end up with? A Monet on a bad day or a Van Gogh on a bad day.” For, in fact, diligent followers can become “a Monet on a great day or a Van Gogh on a great day” once they have inhabited and perfected the generative rule or style responsible for a person’s oeuvre, as ample cases of forgeries demonstrate.²⁰ Specifically, if a certain style presumably has its origins in a specific nature or feeling, as Joe argued, and if it is possible for one person to inhabit another’s style, then for all intents and purposes both people might seem to partake in the same nature or feeling. This fact generates problems for canonical notions of creativity as a scarce resource and as an inalienable possession (see Wilf 2012). It is also the reason why technologies that reproduce Thirds, especially in the field of modern art, are an object of intense fascination, as I now finally turn to discuss.

“It’s Me!”: A Technology That Provides the Self with Its Interpretants

The scientists I have worked with were developing Syrus not primarily for the purpose of reproducing the specific styles of past jazz masters; rather, such reproduction was only a preliminary stage for the mixing of these styles with one another to produce new styles—a project motivated by the scientists’ discontent with current styles of improvisation and their desire to create new and inspiring styles (Wilf 2013b). In other words, Syrus’s primary purpose is to mediate between Thirds rather than to reproduce existing Thirds. Hence, to highlight the specific media ideologies, forms of intertextuality, participation framework, and politics of mediation that are involved in a technology that focuses exclusively on the reproduction of Thirds, I turn to commentaries provided by users of another computerized system that is also based on Markov processes—that is, the same kind of algorithms used in the animation of Syrus. This system, called the Continuator, was developed by the French music technology scientist Francois Pachet (2003). It is able to learn and simulate the style of a player in real time, “continue” it, and thus engage in a dense turn-taking interaction with the human musician. The commentaries I discuss below, made by pianists who interacted with the Continuator, pertain to a series of sessions in which the Continuator was “hooked” to and played the same piano the pianists played during their interaction with the Continuator.

The commentaries about the experience of playing with, and observing someone else playing with, the Continuator are mostly focused on the reconfiguration of the participation framework people usually associate with a technology that reproduces Seconds (Goffman 1981). Whereas a technology that reproduces Seconds functions mainly as an animator, a technology that reproduces Thirds functions mainly as an author (and possibly an animator). The Continuator functions as an author that produces novel instantiations of someone else’s style, that is, on behalf of the principal. It composes (and sometimes, though not necessarily, plays) the music on behalf of whoever’s style it implements at a given moment. Hence when a human musician musically interacts with this system, which learns his or her style in real time and generates music in such a style and in response to the human player’s playing, the human musician and the audience that observes him or her interacting with the system have the impression that they are witnessing a kind of real-time multiplication of various authors “speaking” on behalf of one principal, the human musician.

To reiterate, technologies that reproduce Seconds have also been an object of fascination as a result of their reconfiguration of the participation framework, yet the fascination they have provoked results from their ability to reproduce the animator—the sound box—through the reproduction of the very same material or qualia of which the original message consists (Taussig 1993; Weidman 2003; Eisenlohr 2010, 328–29; Seaver 2011; Wilf 2012). Indeed, this kind of reproduction of qualia has been the focus of efforts to perfect acoustic fidelity (Sterne 2002).

In contrast, with a technology that reproduces Thirds, users’ fascination is provoked by the multiplication of authors who compose on behalf of the same principal. The experience people have when playing with such a system or observing someone else playing with it stems from the fact that such events challenge the modern normative ideals of agency I have discussed above, which stipulate the individuality and uniqueness of the western subject and especially the inalienability of modern creative agency as a principle of generativity and a continuous structure of feeling or “nature” specific to each individual (Wilf 2012). The creation of a doppelganger of the artist, one who is identically generative and interactive as the artist himself, is radically different from the static likeness produced by sound recording technologies and even automatons that perform preprogrammed and a limited set of past compositions or performances of another person (Riskin 2007).

It is for this reason that under the heading “Playing Twice with Oneself,” which describes one specific mode of interaction with the Continuator, Pachet writes:

[This] mode has two phases. First, a musician plays harmonically rich music to the system—chords, chord sequences of all kinds. In a second phase, the system produces an infinite stream from the learned chord sequences. The musician then plays a solo improvisation on top of this harmonic material, which is fed in the system’s harmonic context input. The chord sequence played by the system then tries to “follow” the improvisation. This mode creates a striking impression on the musician (and the audience) as what happens in effect is that the musician virtually follows himself. (Pachet 2002, 79; emphasis added)

The sensations this technology generates among its users are the result of this production of a real-time generative similitude. Pachet calls the type of reaction shared by a number of jazz musicians who interacted with the system as “the Aha effect,” which he describes thusly: “The most striking effect—noticed systematically from all musicians experimenting with the system—can be described as the aha effect, triggered by the sudden realization that the system is starting to play exactly in the same style as oneself” (Pachet 2002, 79). This surprise stems from the musician’s sudden awareness that the Continuator functions as an author who faithfully “composes” on his behalf. Indeed, this surprise is shared by the audience observing and listening to such an interaction, too: “Audience reactions range from amazement, astonishment, and strangely the desire to play with the system. The last type of reactions pushed us to organize a concert in the near future in which, at some point, people in the audience can participate, by playing with a keyboard and having the Continuator continue the phrases in the style of the performing musician” (Pachet 2002, 80).

Furthermore, some of the people who played with the Continuator explained their fascination with this technology by saying that it had allowed them to discover the future instantiations of their style, as it were. One pianist said the following: “The system shows me all the potential ideas I could have developed, but that would have taken me years to actually develop. It is years ahead of me, yet everything it plays is unquestionably me” (Pachet 2002, 80). I want to theorize these sensations, described by pianists following their interaction with the Continuator, by using Peirce’s theory of the self as a sign in the process of development. Peirce defined personality as one’s mode of connecting “between ideas [that] is itself a general idea … a living feeling” (quoted in Colapietro 1989, 76). Note that Peirce’s notion of “a living feeling” combines Firstness and Thirdness in that it is a continuous feeling: “As a general idea (an idea connecting other ideas), the individual personality is not a discrete entity but a continuous being. … It is essentially temporal” (Colapietro 1989, 76). Peirce’s definition thus resonates nicely with Fichte’s definition of creative agency and style as one’s “own ways of forming concepts and connecting them” (Woodmansee 1996, 51), as I discussed above. Peirce argued that this mode of coordination of ideas, which is specific to each individual as the basis of his or her personality or character, “implies a teleological harmony in ideas, and in the case of personality this teleology is more than a mere purposive pursuit of a predetermined end; it is a developmental teleology” (quoted in Colapietro 1989, 76). This developmental teleology is a natural consequence of Peirce’s notion of the self as a sign in the process of development because his theory of semiosis stipulates that a sign gives rise to interpretants, which themselves function as signs to other interpretants, and so forth (to mention only the direction of semiosis from the object to the sign to the interpretant). Although this teleological development does not entail a predetermined purpose, “a general idea, living and conscious now, it is already determinative of acts in the future to an extent to which it is not now conscious” (quoted in Colapietro 1989, 76). Hence if, according to Peirce’s approach to the self, “the self as a sign in the process of developing is, in essence, the self as a being in dialogue with itself” (Colapietro 1989, 91) and if “a sign cut off from its future interpretants is a sign denied the possibility of realizing its essence; that is, the possibility of being a sign” (Colapietro 1989, 77), then the Continuator is a system that allows players to engage in a much clearer dialogue with themselves by providing them with externalized interpretants that conform to, because they are the result of, their own mode of semiosis or style or Thirdness as abstracted by the Continuator from an analysis of their playing, that is, from the signs they produce.²¹ The external input the pianist receives from the Continuator stems from “the core” creativity of the pianist himself, his style or specific mode of coordinating ideas, as it were, which the Continuator can learn in real time. To put it somewhat crudely, rather than the pianist having to produce interpretants to his own signs, the Continuator provides him with interpretants to his signs, and these interpretants function as signs to which the pianist produces his own interpretants and so forth. Put differently, the pianist and the Continuator become “conspecifics” that interact with one another and together become one unit of agency and accountability that encompasses both of them (Kockelman 2012, 19–20).

Furthermore, the Continuator not only provides the pianist who plays with it with interpretants that conform to his “personality”—to his specific mode of coordinating ideas, but it often also provides the pianist with interpretants that the pianist would not have necessarily come up with at the present stage of his development as a sign—a feat the Continuator can achieve because of its superior technical abilities.²² In this sense pianists feel that playing with the Continuator is a form of engaging in a kind of self-fertilization and internal dialogue that accelerates the pace of development of their ideas.²³ Hence not only is playing with the Continuator different from cutting a sign off “from its future interpretants” and thereby denying it “the possibility of realizing its essence,” it is actually experienced as reconnecting the self as a sign with its future interpretants and thus helping it to realize its essence.

The Algorithmization of E-commerce: Reproduction or Coproduction of Thirds?

The relevance of the phenomena I have discussed thus far to contemporary forms of sociality far exceeds the pockets of research in music technology that have been this essay’s ethnographic focus. Much of the technology that animates Syrus, the Continuator, and similar systems—namely, computerized algorithms that simulate or anticipate style—has been used by commercial companies to statistically predict online users’ individual preferences, tastes, and distastes—in short, their individual styles of various kinds—based on their online behavior. This enables such companies to provide and produce online content that mirrors and anticipates these styles, for example, to come up with effective advertisement strategies that bring to users’ attention products they are more likely to buy (Cheney-Lippold 2011; Seaver 2012). Computerized algorithms in consumer-centered production derive their profitability from their ability to tap into each consumer’s distinct styles of consumer behavior, or Thirds, especially when this behavior takes place online because the online platform enables companies to easily create large databases of each consumer’s patterns of online behavior.

Here, too, at stake is these companies’ ability to replicate an author who can “speak” on behalf of the principal that is the consumer (Goffman 1981). When companies manage to achieve this task, consumers feel that the online content resonates with “who they are.” A technology that is limited to reproducing Seconds is highly limited in this regard. This became clear in an interview with the founders of two of the major companies that personalize online content at the present moment by providing readers at the end of each news item with “suggested recommendations” for further reading, that is, “other articles that might interest you.” As one of the founders put it, “The trivial way is to identify what the user read before and analyze the article’s context. But this is insufficient because you discover that most people do not want to read another item about the same topic.” A founder of another key company in this field provided the same commentary: “One of the interesting things we discovered is that in contrast to the instinct of the average journalist, people who read something are no longer interested in reading about the same topic. There apparently is a limit to how much people want to read about one topic, and it appears that they want to be offered other stuff” (Avriel 2012). A technology that reproduces a succession of the same Seconds—that is, more of the same stuff—would be less accurate in anticipating users’ desires over time. To do so, it must be generative of content in a user’s style—it must become an author who “speaks” on behalf of the user.

However, we must be cautious when evaluating the ways in which such companies, aided by these and similar computerized algorithms, seem to be able to reproduce consumers’ styles or the Thirds that are unique to them—in other words, to produce stylistic fidelity. We must be cautious not only with respect to the truth value of such claims but also with respect to the impact of the very attempt to produce stylistic fidelity in the sphere of consumerism.

The critique of the notion of “acoustic fidelity” made in the context of technologies that reproduce Seconds can guide us in articulating a critique of the notion of “stylistic fidelity” with respect to technologies that reproduce Thirds. Jonathan Sterne has brilliantly argued that the emergence of recording technologies that promise to deliver acoustic fidelity has actually created a fictive notion of an original that existed before the recording and that can be faithfully reproduced by it. Sterne shows that the “original” that became the object of reproduction of such technologies was in fact the product of a radical “configuration of bodies and sounds in space, a particular ordering of practices and attitudes” (Sterne 2002, 236). Recorded sound became studio art, that is, an orchestrated production arranged specifically for the purpose of recording, where singers and performers strapped to microphones needed to learn new skills of performance. In short, the very technology of reproduction created a new form of an original. Hence Sterne concludes that “the very idea that a reproduced sound could be faithful to an original sound was an artifact of the culture and history of sound reproduction. Copies would not exist without reproduction, but neither would their originals” (Sterne 2002, 282).

Similarly, note that the capacity of the kind of technologies I have discussed in this essay to abstract a style depends on the corpus of Seconds that their designers provide them to analyze. The scientists I worked with never gave Syrus a database that consisted of the entire oeuvre of a past jazz master from which it was supposed to abstract this master’s style. To be sure, this decision was structured by limited time. However, it was also motivated by a conscious decision to choose recordings from a stylistically identifiable period in a master’s career and, moreover, to leave out specific recordings made in this period that would render the abstracted style “less meaningful [in terms of] how close the representation is” to an identifiable style, as one of the scientists told me. In other words, like a technology that reproduces Seconds, a technology that reproduces Thirds actually mediates a Third that is, in part, the product of the work of mediation itself and its conditions of possibility. To some degree, then, the process of creating a database, which is a condition of possibility for a technology that reproduces Thirds, is productive of the original that is supposed to exist independently of this technology.

Second, Sterne argues that part of the success story of the notion of acoustic fidelity was the result of the complicity of listeners who wanted to believe in such fidelity and hence ignored or explained away any evidence that it was a fictive notion. Indeed, listeners were trained and educated to do so by the commercial companies that manufactured sound technologies: “Sound fidelity was, ultimately, about faith and investment in these configurations of practices, people, and technologies. … Throughout the early history of sound media, performers and listener lent some of their own mimetic powers to the machines so that they might be dazzled. In developing their audile technique, listeners learned to differentiate between sounds ‘of’ and sounds ‘by’ the network, casting the former as ‘exterior’ and the latter as ‘interior’ to the process of reproduction. They had to be convinced of the general equivalence of the live and the reproduced” (Sterne 2002, 283).

In a similar way, we should not take at face value “aha effects” in which musicians and audiences seem to recognize their and others’ styles in the playing of the Continuator and Syrus. Although I do not have the space to elaborate on this point in detail, it is clear that the desire to “believe” in a technology’s ability to reproduce a Third structures many of these “aha effects” where people believe they can recognize such a Third—an identifiable style—in the playing of the machine. Indeed, scholars of technologies of artificial intelligence that are meant to simulate human intelligence (and hence Thirdness) have documented precisely this form of complicity. Thus Lucy Suchman writes about her disappointing attempt to interact with the MIT AI lab’s interactive robot, Kismet—disappointing, that is, compared to the well-polished demonstration videos produced by Kismet’s main designer, Cynthia Breazeal, in which she flawlessly “interacts” with it. Suchman writes: “The contrast between my own encounter with Kismet and that recorded on the demonstration videos makes clear the ways in which Kismet’s affect is an effect not simply of the device itself but of Breazeal’s trained reading of Kismet’s actions and her extended history of labors with the machine. In the absence of Breazeal, correspondingly, Kismet’s apparent randomness attests to the robot’s reliance on the performative capabilities of its very particular ‘human caregiver’” (Suchman 2007, 246). Writing about this very same robot and another robot developed at the same MIT lab, Sherry Turkle argues that “both adults and children are drawn to do whatever it takes to sustain a view of these robots as sentient and even caring. This complicity enlivens the robots, even as the people in their presence are enlivened, sensing themselves in a relationship” (Turkle 2011, 85). Key to the presumed success of technologies that reproduce Thirds, then, is the training of users to performatively bring about the desired effects that are presented as the achievement of the technology itself.

The notion of performativity points to a third point that needs to be made apropos technologies that mediate Thirds, especially in the context of their commercialization. Performativity stems not only from the bias that is inherent to the construction of a database of Seconds from which a Third is abstracted, or from users’ complicity in “recognizing” themselves in the partial Thirds that technologies of various sorts mediate for them, but also from the possible strategic ends to which such technologies are designed to realize. Up till now I have discussed such technologies as if their purpose could only be to provide users with interpretants in response to their behavior in a way that would conform to each user’s unique Third or mode of coordinating ideas. However, when such interpretants are provided by commercial companies it becomes clear that they can be strategically biased and thus shape users’ modes of coordinating ideas rather than reflect them, especially when such a dialogue takes place over a long time. In this case, then, the commercially informed style reproduction, or the reproduction of consumers’ specific Thirds by commercial companies, while not cutting the self as a sign off from its future interpretants (Colapietro 1989, 77), might shape over time the nature of consumers’ Thirds by teaching them a specific mode of coordinating ideas that benefits the commercial companies’ interests. In other words, if the self is a sign in the process of development then the specific interpretants provided by such technologies to users in response to users’ online behavior might be performative, that is, productive of Thirds that commercial companies might desire users to have.

Indeed, this performative function has been at the core of a growing critique of search engines and personalization of content (albeit a critique not phrased in Peircean terms), which captures this coproduction of users and technologies that mediate Thirds, where users and technologies react to each other and shape each other’s specific Third with a potentially asymptotic “fit” between them: “Most personalized filters are based on a three-step model. First, you figure out who people are and what they like. Then, you provide them with content and services that best fit them. Finally, you tune to get the fit just right. Your identity shapes the media. There’s just one flaw in this logic: Media also shapes identity. And as a result, these services may end up creating a good fit between you and your media by changing … you” (Pariser 2011).²⁴

Whether or not these technologies indeed have such far-reaching repercussions, it is clear that linguistic and semiotic anthropologists have much to gain from applying their theoretical toolkit to the analysis of these ubiquitous technologies that reproduce Thirds, which are playing an increasingly important role in the mediation of sociality in the present historical moment. This essay was a preliminary attempt to outline what such a semiotic analysis might look like.