Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-5b777bbd6c-rbv74 Total loading time: 0 Render date: 2025-06-19T20:03:16.173Z Has data issue: false hasContentIssue false

11 - Disinformation and Algorithms

Amplification, Reception and Correction

from IV - Persuasion and Algorithms

Published online by Cambridge University Press:  10 June 2025

By
William Dance
Edited by
Sofia Rüdiger  and
Daria Dayter
Sofia Rüdiger
Affiliation:
Universität Bayreuth, Germany
Daria Dayter
Affiliation:
Tampere University, Finland
Get access

Summary

Disinformation and the spread of false information online have become a defining feature of social media use. While this content can spread in many ways, recently there has been an increased focus on one aspect in particular: social media algorithms. These content recommender systems provide users with content deemed ‘relevant’ to them but can be manipulated to spread false and harmful content. This chapter explores three core components of algorithmic disinformation online: amplification, reception and correction. These elements contain both unique and overlapping issues and in examining them individually, we can gain a better understanding of how disinformation spreads and the potential interventions required to mitigate its effects. Given the real-world harms that disinformation can cause, it is equally important to ground our understanding in real-world discussions of the topic. In an analysis of Twitter discussions of the term ‘disinformation’ and associated concepts, results show that while disinformation is treated as a serious issue that needs to be stopped, discussions of algorithms are underrepresented. These findings have implications for how we respond to security threats such as a disinformation and highlight the importance of aligning policy and interventions with the public’s understanding of disinformation.

Keywords

disinformationalgorithmsTwittersecuritycomputer-mediated communicationcorpus linguisticsdiscourse analysis
Type
Chapter
Information
Manipulation, Influence and Deception
The Changing Landscape of Persuasive Language
 , pp. 223 - 249
Publisher: Cambridge University Press
Print publication year: 2025

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Ahmed, W., Vidal-Alaball, J., Downing, J. & López Seguí, F. (2020). COVID-19 and the 5G conspiracy theory: Social network analysis of Twitter data. Journal of Medical Internet Research, 22(5), e19458. https://doi.org/10.2196/19458CrossRefGoogle ScholarPubMed
Allcott, H., & Gentzkow, M. (2017). Social media and fake news in the 2016 election. Journal of Economic Perspectives, 31(2), 211–236. https://doi.org/10.1257/jep.31.2.211CrossRefGoogle Scholar
American Dialect Society. (2018). ‘Fake news’ is 2017 American Dialect Society word of the year. www.americandialect.org/fake-news-is-2017-american-dialect-society-word-of-the-yearGoogle Scholar
Anandhan, A., Shuib, L., Ismail, M. A. & Mujtaba, G. (2018). Social media recommender systems: Review and open research issues. IEEE Access, 6, 15608–15628. https://doi.org/10.1109/ACCESS.2018.2810062CrossRefGoogle Scholar
Archer, D., Wilson, A. & Rayson, P. (2002). Introduction to the USAS category system. https://ucrel.lancs.ac.uk/usas/usas%20guide.pdfGoogle Scholar
Arnaudo, D. (2017). Computational propaganda in Brazil: Social bots during elections. Computational Propaganda Research Project: Working Paper No. 2017.8. https://ora.ox.ac.uk/objects/uuid:e88de32c-baaa-4835-bb76-e00473457f46Google Scholar
Ayre, L., & Craner, J. (2018). Algorithms: Avoiding the implementation of institutional biases. Public Library Quarterly, 37(3), 341–347. https://doi.org/10.1080/01616846.2018.1512811CrossRefGoogle Scholar
Bellogín, A., Castells, P. & Cantador, I. (2017). Statistical biases in information retrieval metrics for recommender systems. Information Retrieval Journal, 20(6), 606–634. https://doi.org/10.1007/s10791–017-9312-zCrossRefGoogle Scholar
Benkler, Y., Faris, R. & Roberts, H. (2018). Network propaganda: Manipulation, disinformation, and radicalization in American politics. Oxford University Press.CrossRefGoogle Scholar
Bozdag, E. (2013). Bias in algorithmic filtering and personalization. Ethics and Information Technology, 15(3), 209–227. https://doi.org/10.1007/s10676–013-9321-6CrossRefGoogle Scholar
Brezina, V. (2018). Statistics in corpus linguistics: A practical guide. Cambridge University Press.CrossRefGoogle Scholar
Brummette, J., DiStaso, M., Vafeiadis, M. & Messner, M. (2018). Read all about it: The politicization of ‘fake news’ on Twitter. Journalism & Mass Communication Quarterly, 95(2), 497–517. https://doi.org/10.1177/1077699018769906CrossRefGoogle Scholar
CDC. (2021). Rapid increase in ivermectin prescriptions and reports of severe illness associated with use of products containing ivermectin to prevent or treat COVID-19. https://emergency.cdc.gov/han/2021/han00449.aspGoogle Scholar
Central Digital and Data Office. (2021). UK government publishes pioneering standard for algorithmic transparency [Press release]. www.gov.uk/government/news/uk-government-publishes-pioneering-standard-for-algorithmic-transparencyGoogle Scholar
Centre for Data Ethics and Innovation (CDEI). (2020). Review into bias in algorithmic decision-making. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/957259/Review_into_bias_in_algorithmic_decision-making.pdfGoogle Scholar
Chin, C. (2018). AI is the future – but where are the women? www.wired.com/story/artificial-intelligence-researchers-gender-imbalance/Google Scholar
Cormen, T. H., Leiserson, C. E., Rivest, R. L. & Stein, C. (2022). Introduction to algorithms. MIT Press.Google Scholar
Cunha, E., Magno, G., Caetano, J., Teixeira, D. & Almeida, V. (2018). Fake news as we feel it: Perception and conceptualization of the term ‘fake news’ in the media. In Staab, S., Koltsova, O. & Ignatov, D. (Eds.), Social informatics. SocInfo 2018. Lecture notes in computer science (Vol. 11185, pp. 151–166). Springer. https://doi.org/10.1007/978-3-030-01129-1_10Google Scholar
De Beer, D., & Matthee, M. (2021). Approaches to identify fake news: A systematic literature review. In Antipova, T. (Ed.), Integrated science in digital age 2020 (pp. 13–22). Springer. https://doi.org/10.1007/978-3-030-49264-9_2CrossRefGoogle Scholar
Echterhoff, G., Groll, S. & Hirst, W. (2007). Tainted truth: Overcorrection for misinformation influence on eyewitness memory. Social Cognition, 25(3), 367–409. https://doi.org/10.1521/soco.2007.25.3.367CrossRefGoogle Scholar
Faddoul, M., Chaslot, G. & Farid, H. (2020). A longitudinal analysis of YouTube’s promotion of conspiracy videos. arXiv preprint arXiv:2003.03318.Google Scholar
Fernández, M., Bellogín, A. & Cantador, I. (2021). Analysing the effect of recommendation algorithms on the amplification of misinformation. arXiv preprint arXiv:2103.14748.Google Scholar
Gerson, J., Plagnol, A. C. & Corr, P. J. (2017). Passive and active Facebook use measure (PAUM): Validation and relationship to the reinforcement sensitivity theory. Personality and Individual Differences, 117, 81–90. https://doi.org/10.1016/j.paid.2017.05.034CrossRefGoogle Scholar
Global Disinformation Index (GDI). (2019). Disinformation is an online harm. https://www.disinformationindex.org/blog/2019-6-26-disinformation-is-an-online-harm/Google Scholar
Guess, A., Nagler, J. & Tucker, J. (2019). Less than you think: Prevalence and predictors of fake news dissemination on Facebook. Science Advances, 5(1), eaau4586. https://doi.org/10.1126/sciadv.aau4586CrossRefGoogle ScholarPubMed
Hardaker, C. (2012). Trolling in computer-mediated communication: Impoliteness, deception and manipulation online [PhD thesis]. Lancaster University.Google Scholar
Hardie, A. (2014). Log ratio: An informal introduction. ESRC Centre for Corpus Approaches to Social Science (CASS). https://cass.lancs.ac.uk/log-ratio-an-informal-introduction/Google Scholar
Howard, P. N., Bolsover, G., Kollanyi, B., Bradshaw, S. & Neudert, L.-M. (2017). Junk news and bots during the US election: What were Michigan voters sharing over Twitter. CompProp, OII, Data Memo, 1. https://demtech.oii.ox.ac.uk/research/posts/junk-news-and-bots-during-the-u-s-election-what-were-michigan-voters-sharing-over-twitter/Google Scholar
Illing, S. (2020). ‘Flood the zone with shit’: How misinformation overwhelmed our democracy. www.vox.com/policy-and-politics/2020/1/16/20991816/impeachment-trial-trump-bannon-misinformationGoogle Scholar
Jolley, D., & Paterson, J. L. (2020). Pylons ablaze: Examining the role of 5G COVID‐19 conspiracy beliefs and support for violence. British Journal of Social Psychology, 59(3), 628–640. https://doi.org/10.1111/bjso.12394CrossRefGoogle ScholarPubMed
Jones-Jang, S. M., Kim, D. H. & Kenski, K. (2020). Perceptions of mis- or disinformation exposure predict political cynicism: Evidence from a two-wave survey during the 2018 US midterm elections. New Media & Society, 23(10), 3105–3125. https://doi.org/10.1177/1461444820943878CrossRefGoogle Scholar
Joulain-Jay, A. (2021). Tweet Collector (Version 4.01).Google Scholar
Kolkman, D. (2020). ‘F**k the algorithm’?: What the world can learn from the UK’s A-level grading fiasco. https://blogs.lse.ac.uk/impactofsocialsciences/2020/08/26/fk-the-algorithm-what-the-world-can-learn-from-the-uks-a-level-grading-fiasco/Google Scholar
Kozyreva, A., Lorenz-Spreen, P., Hertwig, R., Lewandowsky, S. & Herzog, S. M. (2021). Public attitudes towards algorithmic personalization and use of personal data online: Evidence from Germany, Great Britain, and the United States. Humanities and Social Sciences Communications, 8(1), 117. https://doi.org/10.1057/s41599–021-00787-wCrossRefGoogle Scholar
Leerssen, P. (2020). The soap box as a black box: Regulating transparency in social media recommender systems. European Journal of Law and Technology, 11(2). http://dx.doi.org/10.2139/ssrn.3544009Google Scholar
Li, J., & Su, M.-H. (2020). Real talk about fake news: Identity language and disconnected networks of the US public’s ‘fake news’ discourse on Twitter. Social Media + Society, 6(2), 2056305120916841. https://doi.org/10.1177/2056305120916841CrossRefGoogle Scholar
Lorusso, A. M. (2023). Fake news as discursive genre: Between hermetic semiosis and gossip. Social Epistemology, 37(2), 219–231. https://doi.org/10.1080/02691728.2021.2001604CrossRefGoogle Scholar
Maci, S. (2019). Discourse strategies of fake news in the anti-vax campaign. Lingue Culture Mediazioni, 6(1), 15–43. https://doi.org/10.7358/lcm-2019-001-maciGoogle Scholar
Mahesh, B. (2020). Machine learning algorithms: A review. International Journal of Science and Research (IJSR), 9, 381–386. www.ijsr.net/archive/v9i1/ART20203995.pdfCrossRefGoogle Scholar
McEnery, T., & Hardie, A. (2011). Corpus linguistics: Method, theory and practice. Cambridge University Press.CrossRefGoogle Scholar
McRae, D., del Mar Quiroga, M., Russo-Batterham, D. & Doyle, K. (2022). A pro-government disinformation campaign on Indonesian Papua. Harvard Kennedy School Misinformation Review, 3(5). https://doi.org/10.37016/mr-2020-108Google Scholar
Moura, R., Sousa-Silva, R. & Lopes Cardoso, H. (2021). Automated fake news detection using computational forensic linguistics. Springer International.CrossRefGoogle Scholar
Mourão, R. R., & Robertson, C. T. (2019). Fake news as discursive integration: An analysis of sites that publish false, misleading, hyperpartisan and sensational information. Journalism Studies, 20(14), 2077–2095. https://doi.org/10.1080/1461670X.2019.1566871CrossRefGoogle Scholar
Ognyanova, K., Lazer, D., Robertson, R. E. & Wilson, C. (2020). Misinformation in action: Fake news exposure is linked to lower trust in media, higher trust in government when your side is in power. Harvard Kennedy School Misinformation Review, 1(4). https://doi.org/10.37016/mr-2020-024Google Scholar
ONS. (2020). Internet access – households and individuals. www.ons.gov.uk/peoplepopulationandcommunity/householdcharacteristics/homeinternetandsocialmediausage/datasets/internetaccesshouseholdsandindividualsreferencetablesGoogle Scholar
Oversight Board. (2021). Oversight Board transparency reports – Q4 2020, Q1 & Q2 2021. https://oversightboard.com/attachment/987339525145573/Google Scholar
Pennycook, G., & Rand, D. G. (2021). The psychology of fake news. Trends in Cognitive Sciences, 25(5), 388–402. https://doi.org/10.1016/j.tics.2021.02.007CrossRefGoogle ScholarPubMed
Pew Research Center. (2022). Social media fact sheet. www.pewresearch.org/internet/fact-sheet/social-media/Google Scholar
Rayson, P. (2008). From key words to key semantic domains. International Journal of Corpus Linguistics, 13(4), 519–549. https://doi.org/10.1075/ijcl.13.4.06rayCrossRefGoogle Scholar
Rayson, P., Archer, D., Piao, S. & McEnery, A. M. (2004). The UCREL semantic analysis system.Google Scholar
Scott, M. (1997). PC analysis of key words: And key key words. System, 25(2), 233–245. https://doi.org/10.1016/S0346–251X(97)00011-0CrossRefGoogle Scholar
Seargeant, P. (2022). Complementary concepts of disinformation: Conspiracy theories and ‘fake news’. In Demata, M., Zorzi, V. & Zottola, A. (Eds.), Conspiracy theory discourses (pp. 193–214). John Benjamins. https://doi.org/10.1075/dapsac.98.09seaCrossRefGoogle Scholar
Semino, E. (2021). Not soldiers but fire-fighters – Metaphors and Covid-19. Health Communication, 36(1), 50–58. https://doi.org/10.1080/10410236.2020.1844989CrossRefGoogle ScholarPubMed
Shokeen, J., & Rana, C. (2020). Social recommender systems: Techniques, domains, metrics, datasets and future scope. Journal of Intelligent Information Systems, 54(3), 633–667. https://doi.org/10.1007/s10844–019-00578-5CrossRefGoogle Scholar
Sousa-Silva, R. (2022). Fighting the fake: A forensic linguistic analysis to fake news detection. International Journal for the Semiotics of Law, 35, 2409–2433. https://doi.org/10.1007/s11196–022-09901-wCrossRefGoogle Scholar
Tandoc, E. C., Duffy, A., Jones-Jang, S. M. & Pin, W. G. W. (2021). Poisoning the information well? The impact of fake news on news media credibility. Journal of Language and Politics, 20(5), 783–802. https://doi.org/10.1075/jlp.21029.tanGoogle Scholar
Tsikerdekis, M., & Zeadally, S. (2014). Online deception in social media. Communications of the ACM, 57(9), 72–80. https://doi.org/10.1145/2629612CrossRefGoogle Scholar
Twitter. (2023). Twitter’s recommendation algorithm. https://blog.twitter.com/engineering/en_us/topics/open-source/2023/twitter-recommendation-algorithmGoogle Scholar
Vaccari, C., & Chadwick, A. (2020). Deepfakes and disinformation: Exploring the impact of synthetic political video on deception, uncertainty, and trust in news. Social Media + Society, 6(1), 2056305120903408. https://doi.org/10.1177/2056305120903408CrossRefGoogle Scholar
Van Duyn, E., & Collier, J. (2019). Priming and fake news: The effects of elite discourse on evaluations of news media. Mass Communication and Society, 22(1), 29–48. https://doi.org/10.1080/15205436.2018.1511807CrossRefGoogle Scholar
Walker, A. (2022). Senior coroner Andrew Walker issues his report to prevent future deaths – Molly Rose Foundation. https://mollyrosefoundation.org/senior-coroner-andrew-walker-issues-his-report-to-prevent-future-deaths/Google Scholar
Wardle, C., & Singerman, E. (2021). Too little, too late: Social media companies’ failure to tackle vaccine misinformation poses a real threat. BMJ, 372, n.26. https://doi.org/10.1136/bmj.n26Google ScholarPubMed
Whittaker, J., Looney, S., Reed, A. & Votta, F. (2021). Recommender systems and the amplification of extremist content. Internet Policy Review, 10(2), 1–29. https://doi.org/10.14763/2021.2.1565CrossRefGoogle Scholar
Williams, O. (2021). ‘It’s the poorest in society who are being surveilled’: The rise of citizen-scoring algorithms. www.newstatesman.com/science-tech/2021/07/it-s-poorest-society-who-are-being-surveilled-quiet-rise-citizenGoogle Scholar
Woolley, S. C., & Howard, P. N. (2016). Political communication, computational propaganda, and autonomous agents: Introduction. International Journal of Communication, 10, 4882–4890.Google Scholar
YouGov, A. K. (2022). Two-thirds of British people worry about the spread of fake news. https://pressgazette.co.uk/british-fake-news/Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×