Making Sense of Emoji

Abstract

Many online messages now contain emoji – these small images have quickly become an important means of communicating. Yet they have not yet been taken seriously in philosophy of language. In this exploratory paper, I attempt to remedy this neglect by analysing the communicative functions of emoji. I argue that emoji have at least three communicative functions. Firstly, they can serve a replicative function, in that they can play the same role as words and punctuation, thereby replicating the function of existing written communicative devices. Secondly, they can serve a compensatory function, in the sense that they can be used to make up for features of face-to-face conversation which are lost in written online conversation. Thirdly, they can serve supplementary functions, in that we can perform new communicative acts with emoji which we could not previously perform either in written or face-to-face communication.

1. Introduction

In 2017 a man in California was sent to prison on pimping and pandering charges.¹ He had sent a woman a message asking if she was ‘down for yo crown’, followed by a crown emoji, and another message containing emoji of high heels and bags of money. An expert on prostitution testified during the trial that these emoji were evidence that the defendant was proposing to act as a pimp. That same year, a Massachusetts man was convicted of first-degree murder after his partner was shot in the head. Prosecutors argued that the fact that the defendant had sent a friend an emoji with crosses for eyes, followed by a nickname of his partner, was evidence that he had deliberately shot her.²

Emoji are a commonplace feature of our communicative landscape, and can be found littered throughout social media posts, emails, and messages on SMS and instant messaging apps. They are so commonplace, in fact, that references to them in court opinions around the world are increasing rapidly. In 2021 alone, 154 US court opinions referenced emoji (Goldman, 2022). For emoji to have evidential weight in court, it must be that they have something like meaning, or at least some recognisable communicative function. And yet in philosophy of language little has been said about emoji. Not only would a philosophical analysis of the nature and function of emoji aid judicial interpretation, but it would also be of theoretical value and interest. Emoji are the new frontier of communication, and for our theories of language to have explanatory value it is vital that they accommodate them. In this exploratory paper, I therefore ask: what exactly are emoji, and what do they do?³

In section 2, I sketch the history of emoji. In section 3, I consider the metaphysics of emoji, and argue that the relationship between an emoji type and its iterations on different operating systems is akin to the relationship between a musical work and its performances. In section 4, I survey the functions of emoji, grouping them into replicative functions (where emoji fulfil functions that could also be fulfilled by words and punctuation), compensatory functions (where emoji are used to make up for features of face-to-face conversation which are lost in written online conversation), and supplementary functions (where emoji add new capabilities and functions to communication).

2. Emoji: A Potted History

In 1982, computer scientist Scott Fahlman realised that ASCII characters – the standard set of characters used on computers – could be arranged to make faces. He suggested on an online bulletin board that the character sequence ‘:-)’ be used ‘for joke matters’ (Associated Press, 2007). Rather than declare that you were joking, he thought, you could simply include those characters at the end of your sentence. This caught on, and in the 1990s several technology companies developed smiley face symbols, like ‘☺’. This is not a series of standard characters assembled to approximate a smile – it is instead a standalone character, which can be described as a symbol or pictograph, representing a smile. It is available in certain fonts, like Wingdings. It functions in the text as a single character, like a letter or instance of punctuation. The typographic smiley face, ‘:)’ or ‘:-)’, as well as the pictograph smiley face, ‘☺’, are typically labelled emoticons (a portmanteau of ‘emotion’ and ‘icon’).

Emoji, like emoticons, function inline as part of the text, but they are not characters or symbols. Instead, they are images. They originated after emoticons, in the late 1990s, pioneered by Japanese technology companies. One of the earliest sets was designed by Shigetaka Kurita and made available on the NTT DoCoMo i-mode mobile platform (Galloway, 2016). It consisted of 176 brightly coloured 12 x 12 pixel glyphs, representing a range of objects, from hearts to footballs. Kurita recalls that he took inspiration from the ‘manpu’ representations in Manga, like the water drop which expresses a character's nervousness or confusion, as well as from pictograms, like the symbols of men and women found above lavatories (Nakano, 2016). Emoji sets like Kurita's were copied by a number of telecoms companies in Japan, but they were not standardised. This meant that if your friend was on a different mobile network, there were no mutually available emoji you could send each other.

In the late 2010s, emoji began to be added to the Unicode Standard.⁴ This is a system which provides unique codes for characters from alphabets and symbol collections across the world.

This standard ‘allows data to be transported through many different platforms, devices and applications without corruption’, and ‘forms the foundation for the representation of languages and symbols in all major operating systems, search engines, browsers, laptops, and smart phones—plus the Internet and World Wide Web’.⁵ When emoji were added to Unicode, this meant that regardless of the platform device, application, or language being used, the same code point would generate (roughly) the same type of emoji. U+1F60D, for example, will always generate a ‘smiling face with heart eyes’ emoji ().⁶ Once they were added to the Unicode Standard, emoji quickly became available on many mobile phone operating systems – the emoji keyboard was added to Apple's iOS in 2011, for example (Cipriani, 2011). Emoji appear slightly differently on different operating systems, however, a phenomenon I will return to in §3.

The number of emoji in the Unicode Standard has been growing ever since; there are 3,633, as of September 2021.⁷ Users can now choose from different versions of the same emoji – for emoji depicting people, for example, users can modify the gender and skin tone of the person depicted. The Unicode Consortium now allows members of the public to make proposals for new emoji to be added the standard, too. One of the most well-known emoji campaigns is Jennifer Lee's (ultimately successful) push for a dumpling emoji (Lam, 2016). According to Emojipedia, emoji use has never been higher.⁸ In 2017, 5 billion emoji a day were being sent on Facebook Messenger alone (Burge, 2017).

3. What Are Emoji?

What is an emoji? One might think that I have already answered this question: an emoji is a small digital image.⁹ This is a simplification, however, and to truly answer the question, we must grapple with some tricky metaphysical distinctions.

Consider the following image: . It is associated with Unicode code point U+1F916, and the Unicode label ‘robot’. It would be natural to say that in this paragraph I have inserted ‘a robot emoji’. And it would also be natural to say that this emoji is a digital image, though exactly what kind of image an emoji is will vary. Some emoji could be characterised as logograms, which means that they represent words (Chinese characters, as well as Egyptian hieroglyphs, do this, too). The kanji emoji, which are images of logographic Chinese characters used in Japanese writing, could perhaps be described as such, though technically speaking these are images of logograms rather than logograms themselves.

Other emoji could be characterised as ideograms, which means that they represent ideas or concepts. For example, the ‘smiling face with heart eyes’ emoji, , represents an emotion – the emotion of being besotted or enamoured with something. Finally, some other emoji are best characterised as pictograms – they represent an object through their resemblance to it. The robot emoji, for example, depicts a robot. Some emoji are a combination of different types of image, and others originated as one kind and are used as another. Alex King points, for example, to the infamous ‘eggplant’ emoji, (U+1F346), which is rarely used to refer to eggplant (2018, pp. 2, 3).

So far, so good. I want to now draw your attention to the fact that we use both definite and indefinite articles to refer to emoji. It is natural to say that I used ‘a’ robot emoji, but it would also be natural to ask, ‘How are people using the eggplant emoji these days?’. Indeed, the latter question wouldn't make sense with an indefinite article. This might on the surface be explained through an appeal to a type/token distinction – we use ‘a’ when we refer to the use of the image in an actual sentence, and ‘the’ when we refer to the image itself.

But this distinction is undermined by the fact that there is no single image associated with a Unicode code point or label. A robot emoji on an Apple device looks different from a robot emoji on Google devices. The Apple robot is light blue, with a rounded square shape and a yellow bulb on its head. The Google robot, in contrast, is dark grey, with a rectangular shape and a red antenna. In fact, there are at least eight different images associated with the Unicode code point U+1F916 and associated label ‘robot’.¹⁰ The same is true of most other emoji; their renderings on different operating systems vary quite considerably. King notes, for example, that the grimacing face (U+1F62C) was at one point rendered as a grin on some systems, and a grimace on others (2018, p. 2). Though all of the robot emoji images share a Unicode code point, they have very different visual properties. And they also have different legal properties – each is owned by a different company.

Hence ‘the robot emoji’ cannot refer to a singular image. So what does it refer to? What kind of thing is the robot emoji? Maybe the robot emoji is a set, consisting of all images associated with the Unicode code point U+1F916, and/or with the Unicode label ‘robot’. To endorse this position would be to endorse a kind of emoji nominalism. Yet King points out that this would allow for a bizarre scenario where an image that does not look anything like the rest of the images associated with a Unicode code point could nonetheless count as an instance of the emoji that code point is said to individuate (2018, p. 2). For example, we can imagine a tired and overworked Unicode Consortium member accidentally linking an image of a tomato with the robot Unicode code point. This would entail that the robot emoji consists of a set of images, most of which depict a robot, and one of which depicts a tomato.¹¹

As such, perhaps there is an additional property which an image must possess in order to count as a member of an emoji set. Presumably what makes an image a member of the set that makes up ‘the robot emoji’ is not just that it is assigned to the robot code point, but also that it was designed with this assignation in mind, that it was deliberately assigned to this code point, and also that it is designed to have certain visual properties – those that make it resemble the specific kind of robot that the robot emoji is supposed to represent. Here, we might draw on Kendall Walton's theory of the relationship between a musical work and performances of that work. For Walton, a musical work can be understood as something like a sound pattern, plus other circumstances (like its composition date and the culture in which it was composed) which determine how performances of it are to be heard (2015, p. 247). A sound event counts as a performance of a given musical work ‘just in case its role, in the context in which it occurs, is to present the sound pattern identified with that work’ (2015, p. 241).

I think we can understand the relationship between the robot emoji and robot emoji renderings similarly. Just as you cannot hear a musical work itself (you can only hear a performance of it), you cannot see the robot emoji – instead you can only see renderings of it. And what makes these renderings of the robot emoji is that they have been assigned a particular role or function – their job is to exemplify a specific visual pattern (and perhaps also bear some non-visual properties too, like having certain legal statuses) identified with the robot emoji.¹² And just as performances of the same work can vary, so too can renderings of the same emoji.

None of this is to say that this is the only possible analogy we can draw on for understanding the relationship between the robot emoji and robot emoji renderings. Other potentially analogous relationships might be that between a letter or other grapheme of an alphabet and its rendering in a particular typeface, or that between a sign in a signed language and a particular tokening or signing of it. One letter can be rendered in multiple different ways, in different glyphs. In Unicode, a single letter has a single code point, just like an emoji, but that letter will appear differently in different typefaces. What makes a glyph a rendering of a particular letter is presumably that it has been assigned the function of representing that letter. Similarly, a sign in a language like BSL has a relatively fixed meaning, but will appear differently depending on the speaker, and what makes it a rendering of that sign in the language is its function.

However, the musical analogy seems particularly helpful because it is not a solely linguistic analogy. Unlike letters and signs, but more like musical works, emoji are not merely communicative devices. Instead, they have a collection of complex aesthetic as well as communicative properties, and serve more than just a communicative function. A rendering of a particular emoji is supposed to have the same combination of aesthetic and communicative properties as the emoji in question, just as a good performance of a musical work preserves its communicative and aesthetic properties. In contrast, when one renders a grapheme in a particular font, or performs a particular sign from a signed language, one is attempting to create something with only the same communicative properties as the original grapheme or sign. Hence the analogy helps us appreciate the complex task required of those charged with rendering emoji.

4. What Do Emoji Do?

Having made some progress on the question of what emoji are, let us now turn to the question of what emoji do. This question quite clearly requires a conjunctive answer; emoji do many, many things. The same emoji can be used in multiple different ways. Some uses of emoji are specific to a person or a culture, and other uses are more universal. Emoji usage varies depending on the speaker, the hearer(s), the context, and the medium.

In the following sections I will develop a non-exhaustive list of emoji functions, which I have grouped into three categories. In the first category are emoji functions which are shared by other devices of written communication. For example, some emoji can have semantic content, like words, and some can function like punctuation. I call these ‘replicative’ functions.

In the second category are emoji functions that help make written conversation more like spoken conversation. Sometimes we use emoji to add back in useful contextual information that we lose in typing, rather than speaking face to face. Vyvyan Evans writes that digital communication lacks ‘the rich, communicative context available in face-to-face encounters’, and that textspeak in particular ‘seemingly possesses the power to strip all forms of nuanced expression from even the best of us’ (2017, pp. 32–33). Emoji, he thinks, fulfil ‘a similar function in digital communication to gesture, body language and intonation in spoken interaction’ (2017, p. 33). I call these ‘compensatory’ functions.

Finally, in the third category, we have emoji functions which make possible entirely new (or sui generis) forms of communication which are not possible offline. I think these latter functions remain understudied, especially the interesting aesthetic function emoji can play. I call these ‘supplementary’ functions.

4.a Replicative Functions

Let us first consider the replicative functions of emoji, that is, the way emoji can replace more traditional linguistic items in sentences and produce the same effect. The emoji set does not itself constitute a language – emoji do not have anything like a grammar – but emoji can nonetheless do lots of the same things components of a language can do.¹³ Firstly, they can function like words, and can have semantic content. Indeed, they are sometimes characterised as words; in 2015, the ‘Face with Tears of Joy’ emoji was named the Oxford Word of the Year (Oxford University Press, 2015). King points out that emoji can function as both nouns and verbs (2018, p. 2). The sentence ‘I love ’, for example, can be used to express the proposition ‘I love pizza’, with the pizza emoji functioning as a noun. The sentence ‘ pizza’ expresses the same proposition, with the heart emoji functioning as a verb. Research confirms that readers can easily comprehend sentences in which emoji have replaced words (Scheffler et al., 2022)

Some have even translated books into emoji, Emoji Dick (Benenson, 2010), for example, is a crowd sourced translation of Herman Melville's Moby Dick into emoji. Or at least, so it is sometimes described. It is debatable whether projects like these are really translations, since they cannot preserve all the grammatical features of the target sentences. ‘Call me Ishmael’, for example, is ‘translated’ in Emoji Dick with the following emoji string: a telephone, a man's face, a sailboat, a whale, and an ‘OK’ sign (2010, p. 15). This ‘translation’ does not preserve, for example, the imperatival mood of the original sentence. As such, I think these ‘translations’ are perhaps best regarded as exercises in the transmutation of an artwork from one medium to another – in this case, from literary art to visual art.

I also caveat the acknowledgement that emoji can function like words with the observation that only some emoji seem to have obviously semantic content. It is easy to see how pictogram emoji (those that represent objects through their resemblance to them) could be used to express semantic content. It is harder to see how reaction emoji, like those depicting faces, expressing happiness, sadness, and confusion, et cetera, could contribute to sentence meaning. That is not to say that they do not have other, familiar communicative functions, though.

Secondly, some emoji can function as punctuation. There is both a simple way to use emoji as punctuation and a more complex way. The simple way is to use those emoji which depict standard punctuation marks – the emoji keyboard offers you, for example, a series of different coloured question marks and exclamation marks, which you can use in place of the question marks and exclamation marks on the standard keyboard. The complex way is to use emoji which do not depict punctuation marks as punctuation marks. For example, emoji depicting facial expressions are often used in place of full stops at the end of sentences. Early research on emoticons observed that they served a similar function. Kris Markman and Sae Oshima observe that emoticons are ‘most typically deployed at the end of sentences or clauses (when part of a parenthetical remark), or turns at chat, either with or in place of standard sentence-final punctuation marks’, and that when used as such, they function ‘to close off the sentence or thought by confirming the action performed by the text’ (2007, p. 5).

In addition to functioning as components of sentences, emoji can also be used to perform illocutionary acts. According to J.L. Austin, illocutionary acts are the acts we perform in speaking, like orders, assertions, and promises (1976). These acts typically (or, depending on who you ask, necessarily) satisfy a set of what Austin called ‘felicity conditions’ – requirements on the successful performance of the act. For example, it is a felicity condition of promising that the act you are promising to perform is in the future, and not the past – you cannot promise to do something you have already done (Searle, 1969, pp. 57–61). We typically, but not necessarily, perform illocutionary acts using words. For example, provided all felicity conditions for ordering are met, I could order you to shut the door by saying, ‘Shut the door!’ Yet we can also perform them with gestures. For example, if I asked you whether you consent to my borrowing your pen, and you give a ‘thumbs up’ gesture in response, we might say that you consented via gesture rather than via utterance.

Emoji provide yet another way of performing illocutionary acts. Some emoji mimic the same gestures we perform in face-to-face conversation – consider, for example, the thumbs-up emoji and the waving emoji. But you could perhaps also perform an illocutionary act by stringing emoji together. In 2015, for example, an American teenager was arrested for terroristic threats, in part because he had posted Facebook statuses about the police which contained the emoji string ‘’ – two revolvers (now rendered as water pistols) pointing at a police officer (Evans, 2015). The arresting police department clearly thought the emoji amounted to a threat, which is plausibly a type of illocutionary act.

In thinking about these replicative functions of emoji, one may wonder why we would choose to use emoji when we already have linguistic mechanisms available to us which fulfil the same function – for example, why end a sentence with a smiling emoji when a punctuation mark would do just fine?

A first reason for using emoji instead of words and other components of written languages is that they are easier or quicker to use. In their research into why people used emoji, Henriette Cramer, Paloma de Juan and Joel Tetreault found that some participants reported using emoji for efficiency reasons; stating ‘It's shorter and more graphic’, and ‘It's all about a quick response and saving time on the go’ (2016, p. 4). One example message they considered was someone writing ‘Ok. I'll probably head out for a short one later. Good chance of rain anyway ’ (Ibid). The emoji at the end convey quickly that the sender will be going on a cycling ride, rather than walking or driving. These gains on the sender's end may be offset by costs on the receiver's end, however. Scheffler et al. found that when emoji are used to encode lexical content, they are decoded more slowly than words (2022).

A second reason for favouring emoji over traditional linguistic devices might be the fact that emoji can transcend specific languages. A person who only speaks Chinese, and a person who only speaks Portuguese, for example, can both understand the meaning of a smiling emoji. At least some emoji have been found to have stable semantic content across multiple languages (Barbieri et al., 2016). Vyvyan Evans, meanwhile, describes emoji as ‘incontrovertibly the world's first truly universal form of communication’ (2017, p. 20), whilst King observes that emoji ‘can transcend language barriers’ (2018, p. 3).¹⁴ We should resist the urge to be too idealistic about this universality, as there are documented cultural differences in how emoji are used and interpreted (Park et al., 2013), but it does seem that at least some emoji facilitate communication across languages.

A third reason for preferring to use emoji might be that they can multitask in a way words and other written devices often cannot – not only can an emoji replace a word and thereby express semantic content, but it can also, for example, add extra information about mood at the same time (see §4.b), or serve an aesthetic function (see §4.c). You might choose to use an emoji rather than a full stop because not only will it function as punctuation, but it also might help you indicate the end of your turn, which you would normally indicate with body language (e.g., taking a breath or looking at your interlocutor).

A fourth reason you might choose to use emoji rather than standard linguistic devices is that they facilitate coded speech, and offer you some plausible deniability if you are accused of engaging in harmful and/or criminal speech. Let us return to the two court cases I mentioned at the beginning of the paper. In one, the defendant's use of the crown emoji, the high heels emoji, and the bags of money emoji were regarded as evidence that he was proposing to act as a pimp for someone. In the other, the defendant's use of an emoji with crossed out eyes, followed by his girlfriend's nickname, was seen as evidence that he had deliberately shot her.

A crown, a bag of money, and a high-heeled shoe do not literally mean prostitution in the same way ‘prostitution’ means prostitution. Indeed, if we were to attribute semantic content to these emoji, we would say instead that they mean crown, bags of money, and high heeled shoes. But the defendant in the first case was using them in a euphemistic way, a bit like how some use the eggplant emoji. These emoji have acquired associations with certain practices and activities, such that when the defendant included them in a message he could convey ideas of prostitution. Since the activities he was alluding to were illegal, it was surely safer to communicate in this kind of coded language than to ask direct questions like, ‘Would you like to work as a sex worker for me?’. If accused of pimping, he could claim that his messages were misinterpreted.

Similarly, in the second case, the defendant could have sent his friend a message reading, ‘I killed [girlfriend's name]’. Yet that message would count as strong evidence against him in court. An emoji which is associated with death, disorientation and dizziness sufficed to get enough information across to the recipient, without leaving the sender obviously on the hook. As Unicode themselves observe, emoji can provide a ‘useful ambiguity’, allowing the sender ‘to convey many different possible concepts at the same time’.¹⁵ In both cases, however, there was enough evidence overall to convict the defendants. Moreover, as emoji grow more popular, they may grow more conventionalised, and as such the plausible deniability one gains from using them may decrease over time.

4.b Compensatory Functions

Communication online (barring video calls) is largely disembodied; you do not see or hear your interlocutors, but instead receive only their typed messages. This makes it radically different from face-to-face conversation, where we are supplied with all kinds of information over and above the words a speaker utters; we can hear their tone, pitch, volume, and rhythm; watch their body language and the gestures they make; analyse their facial expressions; and follow their gaze. In online communication, absent these additional sources of information, we are likely to run into interpretative difficulties; it is harder to judge the speaker's mood, or their sincerity, or when their conversational turn has ended, for example.

Here is where emoji can come in handy. As King puts it, ‘we use emoji to perform the function we normally leave to prosody, facial expression, and other features of spoken or in-person communication’ (2018, p. 2). Similarly, Lauren Gawne and Gretchen McCulloch argue that emoji are a way to add back in the ‘information provided by tone of voice and body language in face-to-face communication’ (2019). Georg Albert writes that ‘unquestionably, emoji are valuable for emulating paralinguistic and nonverbal signs such as facial expressions and gestures, which occur in face-to-face conversations but not in written forms of communication’ (2020, p. 66), whilst Vyvyan Evans writes that ‘putting a sad face at the end of the expression provides a non-verbal cue, a metacomment, showing us how to interpret the words’ (2017, p. 53). These theorists are characterising emoji as serving something like a compensatory function, filling back in what is lost when one replaces face-to-face spoken communication with technologically mediated written communication.

Emoji are not the only way to fill this gap, of course. We have several other methods available to us. For example, there is a convention of using capital letters to indicate that you are shouting, and a convention of using lots of exclamation points to indicate the intensity of an emotion. Emoticons, and the original smiley face – :-) – also served a compensatory function, enabling writers to indicate emotions and sincerity. But because there is such a wide range of emoji (depicting, for example, a very wide range of facial expressions and emotions), they are especially good at fulfilling this compensatory function, and can add back quite a large amount of information previously lost in online messaging.

A first compensatory function emoji can serve is, as I alluded to at the end of the last section, indicating the end of a conversational turn. Markman and Oshima make this claim about emoticons, the emoji predecessor (2007), but it seems clearly true of emoji, too. Conversational analysts Harvey Sacks, Emanuel Schegloff and Gail Jefferson proposed that we think of conversations as involving a series of alternating turns by interlocutors; interlocutor A takes a turn, then interlocutor B, then interlocutor A, et cetera (1974). Though these turns may overlap – you may begin your turn just as I am finishing mine – there is a general norm that interlocutors speak one at a time. As such, it is important that we know when a speaker's turn has finished or is at least coming to an end, so that we know when to take up our own turn. In face-to-face conversation we rely on many cues – silence from the speaker, eye contact, cadence, body language, and more. Online, these cues often are not available, but emoji can compensate. Markman and Oshima write that emoticons can be used to signal that ‘a complete turn at chat has been taken’ (2007, p. 10), thereby enabling the speaker to give the floor, so to speak, to the hearer. To use a different analogy, they can function like the word ‘over’ in radio conversations.

Emoji can also be used to end not just a turn but an entire conversation. Ryan Kelly and Leon Watts note that an emoji can function ‘as a signal that a message has been received but that the recipient has little to say in response’ (2015). They quote a survey participant who told them, ‘Yesterday we were talking about pancake day, so I just sent some pancakes [an emoji] and that kind of just, finished the conversation. It kind of just, yeah I think it says you have nothing else to say.’ This message, Kelly and Watts write, ‘may be seen as affinity-building in that the recipient is acknowledging the sender's message while offering a mutually interpretable signal indicating that they have little to offer in reply’ (ibid.). Sending the pancake emoji enables the speaker to acknowledge receipt of the prior message and close the conversation without making their interlocutor feel ignored.

A second compensatory function of emoji is to express affective and emotional states, which might usually be expressed by tone, expression, and body language. As a simple example, a smiling emoji can be used instead of a real smile to indicate contentment or pleasure. Indeed, Linda Kaye et al. found that facial stimuli and emoji can be processed in ‘a largely equivalent way’ (2021, p. 7; see also Gantiva et al., 2020). The emoji keyboard provides users with a wide variety of facial expressions, enabling the expression of a wide variety of affective states. Research by Monica Riordan shows that non-facial emoji, like hearts or roses, are also used to express affect (2017).

Sometimes emoji are used to establish the overall emotional tone of a message, especially when this tone is ambiguous from the text alone. In some cases, this replaces the clarificatory function of facial expression, prosody, and body language, and thus counts as a kind of compensatory function, but in other cases it adds clarification that would not be easily achieved in face-to-face communication, in which case it is more of a supplementary function (which I discuss in more detail in §4.c). For example, we can use emoji to indicate whether we are speaking seriously or jokingly, an indication that might usually be made possible through body language and prosody. Imagine that you wish to communicate in a face-to-face conversation that you think a proposed plan is misguided. You might do this by uttering the words, ‘Great idea’, with a sarcastic tone. Yet if you typed ‘Great idea’ in an instant messaging chat, it would not be immediately clear that you were being sarcastic, because your interlocutor does not have your tone or facial expressions available to them to interpret. Adding a winking or laughing emoji might add in this extra information usually provided by paralinguistic behaviours.

Similarly, you could add this kind of emoji at the end of a joke to communicate that your words were intended as non-serious speech. Ryan Kelly and Leon Watts quote a survey participant explaining how they use a winking emoji: ‘If you're saying something that, you know, that'll wind them up, but and, if you didn't put the winky face next to it, then it could be misconstrued as like, starting on them, or, like quite an aggressive statement, so, yeah, they really help make sure that you don't get into any trouble’ (2015). The winking emoji described functions to establish a tone of jocularity and affection, rather than aggression. Indeed, this use of emoji to indicate tone was Scott Fahlman's vision for the :-) emoticon. Similarly, Eli Dresner and Susan Herring note that the winking smiley in particular ‘is often used as an indicator that the writer is joking, teasing, or otherwise not serious about the message's propositional content’ (2010, p. 256).

This function of the emoji's neighbour, the emoticon, has been taken seriously in court cases. In Ghanam v. Does (2014), a Detroit city official accused the defendants of making defamatory statements about him on a political message board. The Court of Appeals found that because one of the defendants used the ‘:-P’ emoticon, which depicts a face with a tongue out, it was ‘patently clear that the commenter was making a joke’, and thus that they were not defaming the official.¹⁶

A third compensatory function of emoji is to replace gestures. Gawne and McCulloch understand gestures as a kind of intentionally communicative body movement. Evidence shows that gesturing is cognitively useful for the speaker herself, as it helps her better conceptualise what she is saying (Krauss, Chen, and Gottesman, 2000; Kita, Alibali, and Chu, 2017), but it also provides useful information to hearers. For example, a person gesturing while speaking can provide useful spatial information (Kita and Özyürek, 2003). Gestures vary in their conventionality. Some gestures are highly idiosyncratic and unstructured. Imagine, for example, the gestures a speaker might make while trying to describe a sculpture. These gestures are very hard to parse for the hearer if they do not also have access to the words the speaker is uttering. Other gestures are highly conventional, like a thumbs up. Gawne and McCullloch call these kind of gestures emblems (2019). These have a recognisable meaning even when not accompanied by speech, but their meaning can vary across cultures. Sign language gestures, like those of BSL, meanwhile, are extremely conventional and form a complete language.

Some emoji encode emblematic gestures – you can send a thumbs up emoji, a fingers crossed emoji, or a waving emoji, for example. Others enable deictic gestures, like pointing. We can use these emoji to direct visual attention to particular words or images. Gawne and McCulloch also describe the use of emoji as beat gestures. Speakers use these gestures in spoken conversations to add emphasis – for example, a speaker might click her fingers between each word, or clap at the beginning or end of the utterance. Emoji offer a way of doing this – Gawne and McCulloch point to messages like ‘’ as an example (2019).

4.c Supplementary Functions

I will finish by considering some of the new capabilities emoji afford us in online conversations. Firstly, they can be used as visual euphemisms. In spoken, face-to-face conversation, we are reliant on verbal euphemisms – we use inoffensive words or phrases to talk about topics which are unpleasant, offensive, explicit, or taboo. For example, we might say ‘She passed away’ instead of ‘She died’, or ‘They hooked up’ instead of ‘They had sex’. In online communication, emoji offer a new way of speaking euphemistically – the use of the eggplant emoji as a phallic symbol is a case in point. In this way emoji have increased our ability to speak indirectly.

Relatedly, emoji have also offered us a new kind of politeness device. Politeness theorists Penelope Brown and Stephen Levinson argue that throughout conversations we are constantly trying to manage both our own ‘face’ and the ‘face’ of others. There are two kinds of face, and hence two kinds of politeness strategy. Negative face is a person's ‘basic claim to territories, personal preserves, rights to non-distraction’ (1987, p. 61), i.e., a kind of freedom of action. Our concern for other's negative face explains why we often verbally soften requests to make sure the hearer does not feel we are imposing on their freedom of action. We might say, for example, ‘I'm really sorry to be a pain, but would you mind shutting the door?’, rather than ‘Please shut the door’. Positive face, meanwhile, is a person's positive self-image (Ibid). We all like to preserve our positive face in the sense that we like to think that others hold us in high regard. We can preserve and anoint the positive face of others by complimenting them, for example.

Emoji can function as both positive and negative politeness devices. If we are sending a message that might appear to impose on the recipient's freedom of action and thus threaten their negative face, we can soften the message with emoji that signal positive affect. Dresner and Herring note, for example, that a winking emoji can downgrade an utterance to a ‘less face-threatening speech act’, and a smiley at the end of a serious request might indicate that the author is ‘anxious about imposing’ (2010, p. 257). Carmen Maíz-Arévalo, meanwhile, finds in their research that participants employ emoticons ‘whenever they feel the need to safeguard the addressee's face’ (2015, p. 142), for example adding smiles and winks at the end of directives and suggestions.

If we want to anoint the recipient's positive face, meanwhile, we might also send emoji with positive valence. For example, we might supplement a verbal compliment with a heart eyes emoji, or a kiss. Politeness strategies were already available to us in face-to-face conversation, both verbally and physically (a smile, for example, might be a positive politeness strategy), and sometimes emoji use might mimic these – we can smile with emoji instead of in real life, for example. But sometimes emoji offer new politeness devices not available offline – for example, there is no offline equivalent to sending a heart emoji after an otherwise slightly awkward or imposing message.

Secondly, emoji facilitate multimodal artistic expression. Throughout history we have found ways to make written text more appealing – for example, through calligraphy, marginalia, and doodles – but before emoji this was hard to do with typed text online. Emoji, however, provide a way to decorate or adorn our messages. They hence serve an aesthetic function as well as a communicative one. Cramer at al. quote a participant in their study who had sent a friend the text, ‘Making soup ’: ‘I included [the emoji] because like a hot bowl of soup waiting for you after driving home from work on a cold wintry day, it's a heart warming sight. Without this fun decoration, the txt would have been dry, boring, joyless’ (2016, p. 4). Similarly, Unicode themselves observe that people like to use emoji on social media ‘to add color and whimsy to their messages’.¹⁷

Not only can users of emoji add images, they can also (unlike users of emoticons alone) add colour to their messages and posts. The addition of colour cannot replicate any devices of written language, nor does it compensate for prosodic, expressive, and gestural dimensions of face-to-face communication. Prior to the invention of emoji, to refer to a specific colour in written text we would only be able to describe it or point to it. Now users can use emoji to refer to colours, and to represent experiences and ideas in very fine-grained ways. In defending their gestural account of emoji, Gawne and McCulloch note that ‘emoji may contain colour and details that are difficult to represent gesturally’, observing that ‘it is unclear how one would distinguish between, say, a red apple and a green apple in gesture, but they are readily distinguished as emoji’ (2019).

For a long time, artists have been able to include text (printed or handwritten) in visual art. Think of the inclusion of newspaper clippings in the still-lifes of Pablo Picasso and Georges Braque, for example, or the use of text in surrealist René Magritte's ‘The Treachery of Images’. In these artworks, conventional, uniform text was included amidst idiosyncratic, individualised painting. Emoji offer in some ways the reverse of this; they enable us to include conventional, uniform images amongst our idiosyncratic written messages (though these messages are not quite as unconstrained and idiosyncratic as visual art, constrained as they are by much stricter linguistic conventions).

5. Conclusion

Emoji matter. These tiny images raise many big questions, both about the metaphysics of images and symbols, as well as about the nature of communication. In this paper I have attempted to map out some possible answers to these questions.

On the metaphysics of emoji, I have suggested that for a given emoji, the relationship between its different associated images on different operating systems and ‘the’ emoji of which these images are understood to be instantiations is akin to the relationship between performances of a musical work and the work itself. The heart emoji is a kind of visual pattern, with a couple of extra properties (like a specific assigned Unicode code point, and a specific legal status), which can be rendered in many different ways. I lack space to answer the other interesting metaphysical questions about emoji, like, for example, whether emoji are works of art.

On the communicative potential of emoji, I have shown that emoji have a wide variety of functions in online communication. When interpreting messages in this medium, and in particular when attributing moral and legal responsibility for such messages, we must take heed of the multiple functions emoji may have, and the multiple ways both the sender may intend to use them and the hearer may interpret them. It is also important to acknowledge and accommodate cultural variation in the use and interpretation of emoji. Legal scholar Eric Goldman, who tracks references to emoji in court cases, writes:

Emoji usually have dialects. They draw meaning from their context. You could absolutely talk about emoji as a phenomenon, but as for what a particular emoji means, you probably wouldn't go to a linguist. You would probably go to someone who's familiar with that community […]. (Quoted in Lee, 2019)

As such there is no easy answer to what an emoji in a particular context ‘means’; rather, we must consider the possible functions it might have, the operating system in which it is displayed, the platform or app on which it was sent, its cultural context, the conversation to which it is a contribution (if any), and the most salient intentions and interpretations of relevant agents. I hope nonetheless that my tentative taxonomy of emoji functions may make this decoding process easier.

I have taken no stance at all on whether emoji are all-things-considered good or bad for our communicative practices, and I shall leave this an open question. What is clear is that emoji have become an established method of communication. In fact, it is interesting to consider whether our communication will become even more imagistic over time. Though it seems unlikely that emoji will replace the written word any time soon, emoji may overtake as the dominant linguistic device, especially as Unicode adds more and more emoji to their official keyboard and thereby makes the communicative potential of emoji even more fine-grained. The emoji images themselves may become more sophisticated, incorporating, perhaps, photography, animation, video, or even sound, and perhaps allowing for even more user modification, beyond changing the skin tones and gender combinations of the images. The end result could be a hyper multi-modal form of communication, which would create not only exciting new ways of communicating, but also exciting new analysanda for philosophers of language and aestheticians.