zbMATH — the first resource for mathematics

Meaning as use: from Wittgenstein to Google’s Word2vec. (English) Zbl 1446.68169
Skansi, Sandro (ed.), Guide to deep learning basics. Logical, historical and philosophical perspectives. Cham: Springer. 41-53 (2020).
Summary: Modern natural language processing (NLP) systems are based on neural networks that learn concept representation directly from data. In such systems, concepts are represented by real number vectors, with the background idea that mapping words into vectors should take into account the context of their use. The idea is present in Wittgenstein’s both early and late works, as well as in contemporary general linguistics, especially in the works of Firth. In this article, we investigate the relevance of Wittgenstein’s and Firth’s ideas for the development of Word2vec, a word vector representation used in a machine translation model developed by Google. We argue that one of the chief differences between Wittgenstein’s and Firth’s approaches to the word meaning, compared to the one applied in Word2vec, lies in the fact that, although all of them emphasise the importance of context, its scope is differently understood.
For the entire collection see [Zbl 1435.68042].
68T50 Natural language processing
03A05 Philosophical and critical aspects of logic and foundations
03B65 Logic of natural languages
91F20 Linguistics
Full Text: DOI
[1] Baker GP, Hacker PMS (1980) Wittgenstein: understanding and meaning. Basil Blackwell, Oxford
[2] Firth JR (1968) Selected papers 1952-1959. Longmans, Green and Co Ltd., London and Harlow
[3] Firth JR (1969) Papers in linguistics 1934-1951. Oxford University Press, London
[4] Frege G (1884/1953) The foundations of arithmetic. A logico-mathematical enquiry into the concept of number. Basil Blackwell, Oxford · Zbl 0037.00602
[5] Frege G (1892/1960) Concept and object. In: Translations from the philosophical writings of Gottlob Frege. Oxford, Basil Blackwell, pp 42-55
[6] Harris ZS (1954) Distributional structure. Word 10(2-3):146-162. https://doi.org/10.1080/00437956.1954.11659520
[7] Mikolov TS, Sutskever I, Chen K, Corrado G, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, neural information processing systems foundation, Inc., pp 3111-3119
[8] Ogden CK, Richards IA (1923) The Meaning of meaning: a study of the influence of language upon thought and of the science of symbolism. Harcourt, Brace and World Inc., New York
[9] de Saussure F (1916/1959) Course in general linguistics. Philosophical Library, New York
[10] Shanker SG (1987) Wittgenstein and the turning-point in the philosophy of mathematics. State University of New York Press, New York · Zbl 0648.00017
[11] Wittgenstein L (1922/1999) Tractatus logico-philosophicus. Dover Publications Inc., Mineola, New York
[12] Wittgenstein L (1953/2001) Philosophical investigations. Blackwell Publishing, Oxford · Zbl 1028.03003
[13] Wittgenstein L (1958/1997) The blue and brown books: preliminary studies for the ‘Philosophical investigations’. Blackwell, Oxford
[14] Wittgenstein L (1964/1998) Philosophical remarks. Basil Blackwell, Oxford
[15] Wittgenstein L,
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.