Chapter 7 - Language Models and the Private Language Argument: A Wittgensteinian Guide to Machine Learning

Summary

Introduction

Wittgenstein's ideas are a common ground for developers of Natural Language Processing (NLP) systems and linguists working on Language Acquisition and Mastery (LAM) models (Mills 1993; Lowney et al. 2020; Skelac and Jandrić 2020). In recent years, we have witnessed a fast development of NLP systems capable of performing tasks as never before. NLP and LAM have been implemented based on deep-learning neural networks, which learn concept representation from rough data but are nonetheless very effective in tasks such as question answering, textual entailment and translation (Devlin et al. 2019; Kitaev, Cao, and Klein 2019; Wang et al. 2019). In this chapter, I will debate some Wittgensteinian concepts that impact the architectures of many NLP deep-learning systems. I will focus, in particular, on the attempt to build a specific kind of architecture to model a private language. The discussion, I think, helps extract philosophical assumptions leading the research and development of AI systems capable of language modelling. In this chapter, I will address some of the main features of NLP systems used for word embedding and one proposal to manipulate through a neural network a form of private language (Lowney et al. 2020).

In ‘The Private Language Argument’, I will reconstruct the complex path of the private language argument (PLA). In ‘Connectionist Language Models in NLP’, I will discuss connectionist language models and introduce notions about NLP systems’ architecture. An overview of this kind of model is helpful to introduce the work of Lowney et al. (2020). They submit that their model can respond to the issues raised by Wittgenstein in the famous PLA. This argument unexpectedly turned out to be relevant not only for the philosophy of language but also for NLP and LAM modellers. I will describe the language game concept in NLP, how it is embedded, and its role in inductive systems development. This central concept in Wittgenstein's work is relevant to describe the role of context in understanding the meanings of words. In ‘Wittgenstein and Connectionism’, I present the Wittgensteinian main concepts at play in the connectionist paradigm. I argue that the connectionist theoretical framework can better catch the dependency of word meaning on context.