Identifying and minimising the impact of fake visual media: Current and future directions

Abstract

Over the past two decades, society has seen incredible advances in digital technology, resulting in the wide availability of cheap and easy-to-use software for creating highly sophisticated fake visual content. This democratisation of creating such content, paired with the ease of sharing it via social media, means that ill-intended fake images and videos pose a significant threat to society. To minimise this threat, it is necessary to be able to distinguish between real and fake content; to date, however, human perceptual research indicates that people have an extremely limited ability to do so. Generally, computational techniques fair better in these tasks, yet remain imperfect. What's more, this challenge is best considered as an arms race – as scientists improve detection techniques, fraudsters find novel ways to deceive. We believe that it is crucial to continue to raise awareness of the visual forgeries afforded by new technology and to examine both human and computational ability to sort the real from the fake. In this article, we outline three considerations for how society deals with future technological developments that aim to help secure the benefits of that technology while minimising its possible threats. We hope these considerations will encourage interdisciplinary discussion and collaboration that ultimately goes some way to limit the proliferation of harmful content and help to restore trust online.

Andrew Walz, according to his Twitter account, was a congressional candidate running for office in Rhode Island in the 2020 US Presidential election. At a glance, there was nothing unusual about Walz’ account – it included details of why he was running, his political ambitions, his profile picture, and Twitter's coveted blue checkmark indicating that Walz had been verified as a genuine candidate. Walz, however, was not a real person; he was a fictional character created by a 17-year-old high school student who was suffering from boredom during the school holidays (O'Sullivan 2020). The student created the profile and inserted a photo of a middle-aged white man from the website thispersondoesnotexist.com – as its name suggests, this site generates faces of people who do not exist. Fortunately, the Andrew Walz incident did not lead to any harm; nonetheless, it raises concerns about the risks associated with the remarkable technological advances that allow almost anyone to create visually compelling fake content. In line with the focus of Memory, Mind, and Media, here we discuss how technology has paved the way for the development of fake visual media, and we highlight directions for future, interdisciplinary, research on detecting fake visual media and for minimising its harmful effects on human memory and cognition.

Technology has blurred the line between real and fake imagery

The premise of manipulating visual media is not new. Until relatively recently, though, the ability to do so remained in the hands of those with considerable digital editing expertise. Rapid and substantial technological advances mean that today almost anyone can manipulate digital images. The most recent technological leap involves the use of artificial intelligence, specifically generative adversarial networks (GANs) to synthesise images, like the one used by the prankster student to create Andrew Walz. The accessibility of such technologies – which allow users to manipulate and synthesise images both easily and cheaply – brings opportunities for education, art, and science, but also potential to use fabricated visual imagery for nefarious purposes (see Silbey and Hartzog 2019 for an insightful discussion on neutralising the destructive force of deep fakes).

The successful use of fake visual imagery for harm, of course, somewhat depends on whether people can sort the real from the fake. New studies have begun to answer this question, examining the extent to which observers can detect various types of digital manipulations (e.g., Farid and Bravo 2010; Robertson et al 2017; Shen et al 2019). In our own labs, we have explored people's ability to sort authentic from manipulated images of real-world scenes (Nightingale et al 2017, 2019, 2022). People's ability to determine if an image is real or fake depends on the type of manipulation applied to the image (e.g., airbrushing, shadow alterations, object insertion), but in general people perform only slightly above chance on this task (Nightingale et al 2019). Somewhat counterintuitively, in our research we have found no strong evidence that individual factors, such as having an interest in photography or experience of editing images, are associated with increased accuracy in distinguishing real from manipulated photos. In one study, we tested 15,873 individuals aged 15–75, and found that older adults categorised images less accurately than their middle-aged and younger counterparts, which is not overly surprising given that visual processing declines with age (Nightingale et al 2022). Perhaps more importantly, we found that despite their mediocre ability to sort the genuine photos from the fakes, participants of all ages tended to express confidence in their judgements (Nightingale et al 2022). These results suggest that people are not only poor at detecting manipulated photos, but also often unaware of just how poor they are.

A relatively new type of image manipulation – face morphing – has raised significant concerns about the opportunity afforded by advanced digital editing software for committing identity theft. Face morphs are created by digitally combining images of two or more individuals to create a new image that resembles each of the original identities. The wide availability of face morphing software means that this type of fake image now poses a serious threat to border controls and other face recognition systems used in security settings (see Pikoulis et al 2021), and worryingly, several studies show that people cannot reliably detect face morphs (Kramer et al 2019; Nightingale et al 2021; Robertson et al 2017, 2018). In one recent study, participants attempted to determine whether two face images depicted the same person or not (Robertson et al 2017). Over 49 trials, each participant viewed three types of face pairs: (1) two facial images of the same individual, (2) faces belonging to two different individuals, and (3) one individual's face, alongside a 50/50 morph combining that same individual with another person's face. Participants incorrectly endorsed the 50/50 morphs as being a ‘match’, on average, 68 per cent of the time with error rates under 10 per cent for the same individual and different individual face pairs.

In the previous few years, researchers have turned their attention to an even more sophisticated technology for producing manipulated media: deep fakes (Agarwal et al 2019). The advent of AI-synthesised content has signalled a giant leap forward in the creation of photorealistic fake visual media. Deep fake images are typically synthesised using GANs, a type of machine learning that works by pitting two neural networks (a generator and a discriminator) against one another in an iterative back-and-forth process. This GAN structure can be used to synthesise any type of fake image, as well as video and audio. Highlighting the photo-realism of the outputs of synthesis machines, studies have begun to reveal that people are frequently fooled by deep fakes and cannot reliably distinguish between real and synthetic faces (Hulzebosch et al 2020; Lago et al 2022; Nightingale and Farid 2022) or videos (Groh et al 2021; Hughes et al 2021), yet show overconfidence in their ability to recognise deep fake content (Sütterlin et al 2022).

Fake imagery influences thought, intent, and behaviour

The research evidence is clear: Distinguishing between doctored and authentic visual media is difficult, and research dating back to the early 2000s illustrates why we should be concerned. Fake media can have detrimental effects on human memory, cognition, and behaviour. Studies have shown that doctored photos of prominent public events can change not only what people recollect about those events but also people's attitudes and behavioural intentions (Frenda et al 2013; Nash 2018). Sacchi et al (2007), for example, created doctored photos of the 1989 Tiananmen Square protest in Beijing and the 2003 Iraq war protest in Rome. For the Rome event, aggressive-looking demonstrators and police officers wearing riot gear were inserted into an original image of the peaceful demonstration. During a memory test, people who viewed the doctored Rome photo were more likely to state that the protest involved physical confrontation, significant injuries, and damage to property, and indicated they would be less inclined to participate in future protests, than people who viewed the original photo. Doctored images can also distort how people recall self-involving, significant, childhood experiences (e.g., taking a hot air balloon ride, Hessen-Kayfitz and Scoboria 2012; Wade et al 2002) or recent everyday actions (e.g., shuffling cards or counting to 20, Nash et al 2009; Nash and Wade 2009; Wright et al 2013). Doctored videos have been used to induce people to falsely confess to committing an undesirable act (e.g., cheating in a gambling task) or to provide erroneous testimony about another person's actions (Nash and Wade 2009, Wade et al 2010 Wright et al 2013).

More recent research has demonstrated the potential for doctored imagery to change consumers’ memories of what they have purchased and the brands they prefer. Under the guise of conducting consumer research, Hellenthal et al (2016) instructed participants to compile a basket of 12 food items made by their preferred brands. The researchers then took a photo of the participant with their ‘personal brand lifestyle basket’. Approximately a week later, participants viewed a doctored version of the photo in which 4 of their 12 selected items were replaced with similar items made by different brands. Participants were asked whether they were comfortable with the photo being included in their brand profile. The next day, participants completed a surprise memory test in which they were asked to indicate which items they included in their original basket. Participants displayed the classic ‘misinformation effect’ – frequently remembering items that were merely suggested to them in the fake photo. Their brand preference ratings also changed after viewing the doctored photo, with many participants indicating a positive shift in attitude and behaviour towards brands that were suggested to them.

It is clear, from over 20 years of empirical research, that doctored images can have powerful effects on cognition and behaviour. Interestingly, psychologists conducting doctored photo research in the early 2000s were of the belief that ‘most of us will never be confronted with images of ourselves doing things we have never done, or in places we have never been’ (Strange et al 2005, 240). Today we are not so sure, given the numerous technologies that can capture, archive, and visualise images of our personal experiences, and given the widespread availability of digital editing software (not to mention the emerging synthesis technologies, but see Murphy and Flynn 2021).

People's limited ability to detect fake visual media, paired with its powerful effects, makes it highly effective for nefarious purposes. Furthermore, the prevalence of social media platforms allows content – both real and fake – to be rapidly disseminated across the world. We have already witnessed the harm that synthesised and manipulated images can have, for example, in creating non-consensual sexual imagery, committing financial fraud and identity theft, and fuelling misinformation campaigns (e.g., Kalpokas & Kalpokiene 2022; Sleigh 2021; Wakefield 2022; Westerlund 2019). We now turn to an important question that has received relatively little attention from researchers, particularly social scientists: What can be done to improve the detection of fake visual content?

Improving detection of fake imagery

Computational detection

There is a substantial literature on digital image forensics; however, here we will mention a selection of the recent techniques proposed for detecting deep fakes (for an overview of digital forensic techniques, see Farid 2016). The forensic techniques tend to be categorised as low- or high-level. Low-level techniques detect pixel-level artefacts that are not visually perceivable by human observers, for example evidence of warping or blending (Li and Lyu 2018; Li et al 2020). These low-level approaches can often achieve high accuracy in detecting fakes; however, a limitation is that they are sensitive to counter attacks (e.g., the addition of noise or image resizing can destroy the artefact used for detection; Carlini and Wagner 2017). High-level techniques focus on artefacts within semantically meaningful information, such as eye blinks, head pose, and mannerisms. One interesting new finding is that deep fake videos can be reliably detected by identifying an inconsistency between a synthesised person's mouth shape and a spoken phoneme (Agarwal et al 2020). The authors point out that for words containing the letters ‘B’, ‘M’, or ‘P’, it is near impossible to make those sounds without closing your lips – for example, if you try to say ‘mother’, it likely is impossible to enunciate clearly without your lips touching. This realisation has led to the development of a forensic technique using phoneme-viseme mismatches to detect state-of-the-art deep fake videos (Agarwal et al 2020).

Another slightly different approach proposed by forensic image experts is to introduce watermarks into any new technologies for synthesising media content. The development of synthesis technology involves using machine learning to train a model that generates photorealistic content without any human input – a generative model. Drawing on the already widespread use of watermarking for copyrighting digital property, it is possible to take a similar approach with synthesis technology. In this instance, a watermark is inserted within the set of real images used for training the model, such that all of the training images will also contain the unique identification information. Crucially, research has shown that when training such a model using a watermarked dataset of images, the model takes on this identification information during its learning; the information then becomes embedded within any subsequently synthesised content (Yu et al 2021). As such, the watermarking approach allows fake images to be reliably detected downstream. With the rapid development of AI-synthesised content, this watermarking approach offers a proactive solution to the threat of deep fakes.

Human detection

Scientists have tested certain interventions for improving people's ability to detect fake imagery. One line of research has, to a large extent, been driven by the disturbing finding that unrealistic and unattainable beauty ‘ideals’ depicted in (manipulated) images of fashion models can cause psychological harm to observers (e.g., Grabe et al 2008). In an attempt to mitigate the damaging effects of viewing these impossible beauty standards, researchers have explored the possible benefits of adding disclaimer labels to images to indicate, for example, when the body of a model has been digitally manipulated. Several studies have found that the addition of such labels does not necessarily help people to discount these images – in fact, some results suggest that this approach might even prove counterproductive by encouraging viewers to direct more, rather than less, attention to the model's body (Selimbegović and Chatard 2015; Slater et al 2012; Tiggeman et al 2013). Related studies have revealed the limitations of warning labels by showing that they are only partially effective in reducing people's belief in fake news headlines (Ecker et al 2010; Pennycook et al 2020) and in fake photos of public events (Nash 2018).

Another area of research that aims to help people identify fake imagery has involved encouraging people to look for the visual artefacts left behind by any editing or synthesis process. When morphing faces, for example, the morph might contain tell-tale signs of digital editing such as a ghost-like outline of another person's face or hair over the forehead (Robertson et al 2017). Initially, this simple training approach seemed promising, as research showed that participants who were trained to detect morphing artefacts mistakenly accepted 50/50 morphs as being a ‘match’ 21 per cent of the time on average, compared to 68 per cent for untrained participants. Yet in two subsequent studies that used higher-quality face morphs, training led to no reliable improvement in performance (Kramer et al 2019; Nightingale et al 2021). With GANs now being used to streamline the generation of high-quality face morphs, it seems likely that training people to spot artefacts in images is, or soon will be, redundant (Venkatesh et al 2020). Similarly, our own research has shown that informing participants about the common artefacts associated with synthesising content only enhanced their accuracy slightly, compared to participants who were given no information (Nightingale and Farid 2022). As technology improves, any artefacts produced in the manipulation process will likely disappear again limiting the usefulness of any detection technique that rests on alerting people to such artefacts.

Why are people limited in their ability to detect fake images?

An important question that remains largely unanswered is why people have such limited ability to determine when an image is real and when it is fake. One obvious, potential explanation is that fake images are now so sophisticated and realistic that there are no perceptible clues to alert people that the image has been manipulated or synthesised. Yet, in studies involving images that do contain detectable signs of manipulation, including changes that are physically impossible (such as a scene containing cast shadows that are inconsistent with the lighting source), people still frequently fail to notice that those images are manipulated (e.g., Nightingale et al 2019). Based on these findings, it is reasonable to think there may be value in training people to detect signs of manipulation. However, given the evidence outlined above, which suggests limited benefits of training interventions, more empirical work is needed to advance our understanding of exactly when and why people fail to successfully use such signs.

When thinking about why people fail to notice signs of digital manipulation, a good starting point is to consider the limits of human perception. Decades of cognitive science has shown that people's capacity to perceive the visual world is finite, with seminal studies demonstrating that people can fail to notice even highly conspicuous events unfolding right in front of them (e.g., Neisser 1979; Neisser and Becklen 1975). Perceptual failures have been shown in change blindness and inattentional blindness studies, where people are surprisingly unaware of significant changes to, or the presentation of, stimuli outside of their focus of attention (e.g., Rensink et al 1997; Simons and Chabris 1999). One of the most famous examples of inattentional blindness is the ‘invisible gorilla’ study (Simons and Chabris 1999) in which participants observed a video of a ball game while counting the number of balls passes made between the players in the game. When engaged in this task, approximately half of participants failed to see a person dressed as a gorilla walk through the middle of the ball game. Furthermore, these perceptual failures are affected by the observer's perceptual load; when people are tasked with processing a lot of information, they are less likely to detect changes in scenes (e.g., Carmel et al 2011). Thus, it remains possible that attention is another crucial factor that impacts whether or not people notice when an image has been manipulated.

It is also important to think about the challenge of distinguishing between authentic and manipulated media in a digital world where the internet, and particularly social media, offers endless content (more than 3.2 billion images are shared online each day; Thomson et al 2020). Research drawing on cognitive and evolutionary theory, along with behavioural economics, shows us that when people have access to vast amounts of information, the way they search that information shapes the decisions they make (Hills and Hertwig 2010). Technological developments afford an ever-increasing ability to store and share information, yet the psychological limits on people's capacity to process information remain unchanged, resulting in a state of information overload (Henkel et al 2021; Hilbert and López 2011; Hills 2019; van den Bosch et al 2016). With such overload, people must select what to attend to, what to believe, and what to share. However, not all information is equal: through evolution, humans have developed cognitive heuristics that make certain types of information more attention-worthy, such as negative information and information that is consistent with existing beliefs (Hills 2019). As such, it might be that some manipulations are detectable in principle, yet in a world overloaded with information, human perceptual limits lead people to overlook types of evidence that would indicate foul play.

]Alternatively, it might be that people simply do not know what to look for, and rely on unhelpful strategies when trying to verify the authenticity of an image. In a recent study, participants were asked to distinguish between manipulated and genuine photos of real-world events, and to report the strategies they used to determine whether an image had been manipulated or not (Nightingale et al 2022). Overall, people's success was similar regardless of whether or not they reported using a specific strategy, yet there were some interesting differences when looking at the specific types of strategy used. For example, those who reported paying careful attention and systematically ‘zooming in’ to look at different parts of the image were more accurate than those who did not report using this strategy. Although this notion of paying attention might seem obvious, only 2 per cent of participants (263/15,873) mentioned it. This finding suggests that people might be able to improve their detection of manipulated images simply by changing the way they approach the task, echoing Hills and Hertwig's (2010) finding that search strategy can play a crucial role in decision accuracy.

The need for an interdisciplinary theoretical framework

An important next step in improving visual media authentication is to develop a theoretical framework for understanding how various factors – including individual, cognitive, environmental, and cultural – influence people's ability to detect manipulated images. As mentioned above, a small but rapidly growing body of empirical research spanning multiple disciplines speaks to this issue; much of this work could inform theory development.

Within cognitive psychology – our own discipline – one framework in particular could guide theoretical thinking: the source monitoring framework (SMF; Johnson et al 1993). Briefly, the SMF aims to explain how people distinguish between mental experiences that result from perception (i.e., memories of real events) versus mental experiences that result from internal processes (i.e., memories of dreams or thoughts). The SMF posits that people can determine the source of their mental experiences by evaluating the characteristics of those experiences. For example, when a memory or image comes to mind, one might consider how familiar, detailed, or coherent it is. If the mental experience has the characteristics typically associated with a memory of genuine experiences (i.e., it is sufficiently familiar, detailed, coherent), then the individual is likely to conclude that it is indeed a memory of something that really happened, rather than something that was merely imagined or thought about. Moreover, according to the SMF, people typically rely on two types of judgement processes to evaluate and classify their mental experiences – a slow systematic reflection and reasoning process, or a rapid, automatic heuristic process (Hasher and Zacks 1979; Johnson et al 1993). As you might expect, source misattributions (i.e., mistaking an imagined or internally generated event for a genuine memory) are more likely to occur when people rely on a rapid, heuristic, decision process.

We can apply the SMF judgement process to the task of distinguishing between genuine and fake visual imagery: If real and fake images differ in systematic and detectable ways, then people may engage in either a careful, systematic search of an image to detect clues that are indicative of a fake image, or they might rely on a more rapid and automatic judgement process to determine the image's authenticity. From a SMF perspective, we might predict that various extraneous factors could influence a person's ability to accurately evaluate an image and to determine whether it has been manipulated or not. One such factor is a person's political perspective, yet the evidence is mixed. Some research shows that people are more likely to buy into fake news, and mistake fictitious for genuine stories, if the false information aligns with their political beliefs or worldview (Frenda et al 2013; Greene et al 2021; Walter and Tukachinsky 2020; Zhou and Shen 2022). Other research suggests that susceptibility to fake news is less about how closely information aligns with an individual's political ideology and more about the extent to which an individual engages in analytical thinking (Pennycook and Rand 2019). Adding further complexity, in another study, partisan-motivated reasoning affected participants’ susceptibility to believing political-based misinformation, however, more so for authentic video content than deep fake video content (Barari et al 2021). According to the SMF, when false information aligns with an individual's own views, beliefs, and stereotypes, it is likely that they will either automatically feel that information to be true, or through motivated reasoning conclude that the information is likely to be true (Mazzoni and Kirsch 2002). In a similar way, it seems reasonable to expect that a person's personal views, beliefs, and stereotypes might affect their ability (or effort) to detect image manipulations. Indeed, research has already shown that people's expectations and preferences can influence how they perceive visual information (Balcetis and Dunning 2006; Bruner and Potter 1964). To date, few studies have explored what makes people better or worse at detecting manipulated images, and the majority so far have involved images that depict unfamiliar people partaking in fairly mundane events. The images are not manipulated to serve a particular political goal, or to comment on culture or society, or to evoke an emotional reaction in the observer. Therefore, it remains possible that in real-world scenarios, where visual media are often manipulated to serve a specific goal, observers’ own goals might decide whether they perform better, or worse, when distinguishing authentic from manipulated images.

Another important factor that warrants greater attention from researchers is the context in which the image is viewed, and its apparent source. Research has shown that media platforms vary in terms of their perceived credibility, and the extent to which people trust any particular source might influence their credulity toward images appearing on that platform (Metzger et al 2010). Computer science and communications experts have started to address this question, and the data from one study suggest that the reported source of an image, and other contextual factors such as how many ‘likes’ it has received, in fact does not significantly affect observers’ perceptions of image credibility (Shen et al 2019). The data did, however, reveal that observers’ attitudes and individual factors, such as their photo-editing experience, affected their perceptions of image credibility.¹

Ethical challenges when seeing is not believing

Finally, the issue of sophisticated fake visual media raises a number of ethical challenges. Consider the so-called liar's dividend: perhaps one of the most concerning consequences of how easily people can manipulate and synthesise visual digital content. In a world where practically any image, video, or audio can be manipulated, it is easy to dismiss anything as fake. Soldiers pictured committing human rights violations, a CEO captured in an embarrassing photo, or a politician at a party they had claimed they did not attend: All of these people could, with enough plausibility to satisfy at least their most willing audiences, argue that those images are fraudulent. We have seen this strategy used in recent years, with former US President Donald Trump denying the authenticity of the 2005 recording of him bragging about sexually assaulting women (Fahrenthold 2016). Below we highlight the need to consider how society deals with future technological developments, to help us to secure the benefits of that technology while minimising its possible threats.

One consideration is how to balance the practice of open code and software distribution with the ethical sharing of image manipulation and synthesising technology. Open science initiatives encourage scientists to make their methods, data, and analytical and computer code openly available, which serves to enhance scientific rigour and researcher integrity as well as encourage the collaborative development of technology. The scientific community should, however, more carefully consider when this sharing is ethical and when there might be good reason to keep certain resources out of the public domain. New technologies, including GANs, quickly become widely and freely available on sites like GitHub, often with walkthroughs for implementation. On the one hand, and for the most part, access to such technology is non-problematic and allows for further advances to be made and the potential to develop use for good. For example, through the use of deep fakes in the documentary ‘Welcome to Chechnya’, LGBT individuals were able to testify anonymously about their suffering and persecution in Russia (RD 2020). On the other hand, the open access also extends to malicious actors who wish to deploy the technology for harm – for example, to generate images that can be used to scam a victim or to create videos to support false claims posted on social media. The balance between open and ethical sharing is a complex issue and one that requires interdisciplinary discussion to ensure the development of sensible and useful guidelines.

Another, much broader, consideration is how the research community might develop appropriate guidelines for the ethical development and use of new technologies. The market has exploded with new applications using GANs to create deep fakes – either for free or at a relatively low cost (Cole 2018). One application – FakeApp – introduced in 2018, which allows users to create deep fake videos at the press of a button, gathered great interest with hundreds of thousands of downloads in the first month of its release (Marino 2018). Although the complexity of training a GAN still prevents many from creating their own models, the development of applications like FakeApp opens up the market to everyone. As such, the potential for misuse is wide; one of the most common abuses so far being the creation of non-consensual sexual imagery. In 2019, research conducted by a cybersecurity company, Deeptrace, revealed that 96 per cent of the deep fake videos online at that time were of a pornographic nature, and the victims overwhelmingly women (Ajder et al 2019; Wang 2019). The ethical and moral concerns surrounding these new technologies are highlighted in the steadily growing number of publications on this topic from fields such as law, information technology, and political science (e.g., de Ruiter 2021). We believe that there is much that researchers from a range of disciplines can contribute to this discussion.

A final consideration is for the giants of the technology sector to understand how their platforms are used for sharing and weaponising content, and to put substantial effort into preventing such misuses. Business media experts have posited that social media companies are doing a substandard job of keeping harmful content, such as COVID-19 vaccine misinformation, off their platforms (O'Sullivan et al 2021). In a recent study examining 30 anti-vaccine Facebook groups, researchers discovered that just 12 individuals accounted for sharing 70 per cent of anti-vax disinformation within these groups (Center for Countering Digital Hate 2021). Of course, this study considered only a subset of Facebook groups, but it does pose an interesting question: if researchers can find those responsible for posting this vaccine disinformation, why can't Facebook? The better question is perhaps why won't they, as opposed to why can't they. Meta (previously Facebook) reported that 97 per cent of its total revenue from October to December 2021 came from advertising (Johnston and Cheng 2022). The business model underpinning such success involves gleaning as much data as possible from site users, to build detailed profiles ripe for ad targeting. One way to keep users returning to social media sites is to show controversial and evocative content that captivates interest (Kim 2015) – fake content can achieve this goal extremely effectively, given that it is free from factual constraints (Lewandowsky and Pomerantsev 2022). Deep fakes might be particularly powerful when it comes to captivating users’ attention, especially given humans’ ability to quickly recognise and understand visual content (e.g., Greene and Oliva 2009; Isola et al 2013). Therefore, legislators should consider reasonable policy and regulation for ensuring that social media companies are accountable for real-world harms that might result from their services. Modest regulatory changes should incentivise companies to introduce safeguards, and as a result, help toward restoring trust in our digital world.

Ultimately, the potential consequences of fake imagery mean that it is worthwhile examining new ways of improving people's ability to sort the fake from the genuine. These attempts stand to be useful, even if they were only to equip people to weed out the poorer attempts at manipulation. With the pace at which technology is improving, it is perhaps overly optimistic to think that people could learn to reliably detect the most sophisticated fakes that are now readily disseminated across the internet. Instead, within the research community, we should continue to raise awareness of the current and emerging threats, with the aim to encourage more research in this area, including the development of improved computational methods of detection – or face the possibility that people will be fooled by scams far worse than that of the made-up congressional candidate, Andrew Walz.