Language primer

This is a very quick entry to the study of language in computational cognitive science.

A simple formal system

Douglas Hofstadter¹ has this tiny formal system called pq-System. In defining formal systems, we start with a set of symbols, called the alphabet. This dictates that any expression of the system must be made by bringing together, in some order, one or more of these symbols. Next, and the last, step is to define which expressions formed out of these symbols are well-formed in the system.

This well-formedness step comes in two sub-steps.

1. First we define the axioms, those expressions that we take to be well-formed without any further justification.
2. Second is the rules of production that allow us to generate new well-formed expressions from already existing well-formed expressions.

In pq-System, the alphabet consists of the following three symbols: p, q, and -.

We have infinitely many axioms in pq-System, therefore we cannot list them all. Instead, we provide a rule that generates all of the axioms, usually called an axiom schema.

\(x\vect{p}\text{-}\vect{q}x\text{-}\) is an axiom, whenever \(x\) is composed of only ‘\(\text{-}\)’s.

Given this, you should be able to verify that \(\text{-}\text{-}\vect{p}\text{-}\vect{q}\text{-}\text{-}\text{-}\) is an axiom of the system (=a well-formed expression).

There is only one rule of production:

If \(x\vect{p}y\vect{q}z\) is a well-formed expression, then so is \(x\vect{p}y\text{-}\vect{q}z\text{-}\).

Now. think of a decision procedure that takes an expression and decides whether it is well-formed or not.

---

Inspecting the well-formed expressions of pq-System, can you see what do they mean? Or, can you say what they are about? Or, can you say what they are for?

They are theorems of binary addition of non-equal non-negative integers. Their meaning is this.

Levels of analysis

Language is studied in levels of analysis. The study of the atomic building blocks of language and how they combine into more complex structures is called phonology. The alphabet analogy, although the best we can do, would be a little misleading here, because the building blocks of language are not the orthographic symbols, but the phonemes, the smallest units of sound that can distinguish meaning in a language.

The study of the properties of well-formed expressions is called morphosytax. As you observed above, this can be studied without any reference to meaning or significance in any form.

The study of the meaning of expressions is called semantics. And again as you saw above, what we mean by “meaning” is not obvious at all. What sort of a relation is it? Correspondence, aboutness, use (procedural)? If it’s a correspondence, what is the nature of the entities that are related? Are they mental, neural, abstract? All these, and many more, are part of the game in the study of natural language meaning.

The last level that is interest to cognitive science is pragmatics, the study of how language is used in context. The boundary between semantics and pragmatics is not clear, and there are many different views on how to draw it. As we go along, we will see that there might not be much value in drawing such a boundary. Perhaps some boundaries are more useful when they are left fuzzy.

In studying language from a cognitive science perspective, studying morphosyntax on its own does not make much sense.² The relevant field of inquiry is rather the study of the relation between morphosyntax on one hand, and semantics and pragmatics on the other. Let’s call this domain “semantax” for the sake of having a name for it.³

Grammar

The most important aspects of human language for our purposes are:

1. Compositionality: “The meaning of a compound expression is a function of the meanings of its parts and of the way they are syntactically combined.”⁴ This is usually cited as Frege’s principle. Fodor and Lepore has a nice way to put it: You cannot understand John loves Mary without also understanding Mary loves John.
2. Recursion: In principle, there is no end to the expressions that can be generated in a language. This is due to the fact that expressions of a given type can be (directly or indirectly) inserted in expressions of the same type.

3. Structure dependence: For physical constraints, language is produced in a linear fashion (you can speak one word at a time). But the meaning is “read off” from the hierarchical structure. A necessary consequence of this is that items that are related at the meaning level may find themselves fallen apart on the actual acoustic/visual signal. Take:

   \[\text{I saw the man with the telescope.}\]

   The sentence can mean two different things: (1) the speaker has the telescope, or (2) the man has the telescope. The two meanings are determined by the hierarchical structure of the expression: in (2) the telescope is attached to the man, in (1), to saw.

In a very broad sense, a grammar regulates the correspondence between form and meaning in a way that manifests the three properties above.

When we talk about syntactic structure, we mean a particular organisation of the components of an expression. For instance the two meanings of the telescope sentence are related to the following (fairly simplified) structures:

\[\begin{align} \text{[I [saw [the man [with the telescope]]]]} \\ \text{[I [[saw [the man]] [with the telescope]]]} \\ \end{align}\]

Another way of thinking about structure, which is more transparent to semantics, is to think of it as a particular organization of operators and arguments, much like we do in arithmetic.

\[((3 + 4) \times 5)\]

When we go from the linear string of symbols \(((3 + 4) \times 5)\) to its meaning, which is the value of the expression (=35), we work out the structure of the expression in such a way that the operator + is applied to its arguments 3 and 4, and then the operator \(\times\) is applied to its arguments, which are the value of the first operator (recursion!) and 5.

Similarly, for the telescope example, in one reading the operator saw takes as arguments the subject I and the object the man with the telescope; and in the other reading, the operator saw takes as its object argument the noun phrase the man, rather than the entire phrase the man with the telescope; and this time the operator with takes as arguments the noun the telescope and the verb phrase saw the man as arguments.⁵

Objects of inquiry

The grammar is studied mainly around the following questions:

1. How do we understand and produce language?
2. How do we acquire the capacity to do so?
3. How is our language capacity related to other cognitive capacities, such as perception, planning, reasoning, etc.?
4. How do the above questions relate to our neural architecture?

At this point, keep in mind that I am using the notion of grammar in a fairly broad sense. I especially don’t want to mean a system of explicit rules like a Phrase Structure Grammar or any algorithmic mechanism like a parser.

Cognitive science

The above questions are studied in several fields. What distinguishes cognitive science from fields like psychology and linguistics is its emphasis on the computational modeling of the phenomena. Cognitive science, at least in how we take it to mean in our institute, takes the mind to be an information processing system, and therefore the best way to understand it is by building computational models of the phenomena we are interested in, be it a processing question or a learning question.

Where to start reading

The best place to start, in my opinion, is Chomsky (1957)(first 6 chapters).

As you will see, Chomsky has a very low opinion on the relevance of probability, statistics and information theory to the study of language. He appears to have kept this attitude in the rest of his career, and the attitude is still widely held in mainstream linguistics. But in cognitive science, the debate is almost never on whether but how these fields are relevant for language.

An advanced but a good reading on the relevance of information theory to the study of language is Pereira (2000). Have a shallow first pass, to come back to it later.

The fields of probability, statistics and information theory lie at the heart of machine learning and artificial intelligence, and by this token, they also lie at the heart of engineering approaches to language, the field of Natural Language Processing (NLP).

People sometimes use the term computational linguistics to differentiate the scientific study of language through computational modeling from the engineering field of NLP. But be warned that this is not a well-established terminology. See Schubert (2020)(1-4) for a balanced view.

Leaving “how to call what” aside, there is an important distinction between NLP and the cognitive science of language. Both fields aim to build computational models of various aspects of language capacity, but the nature of the models they build differs. A good terminology for this distinction is that of Frank and Degen (2023)’s input-output models vs internally meaningful models. The aim of science is not mere prediction but also explanation, so when our models predict a behavior correctly/wrongly we would want to understand why. Lake et al. (2017) gives a broader view of this distinction in cognitive science.

Whenever you get stuck with a linguistics concept or term, consult Bender (2022). This will work fine as a practical reference book until you develop a more serious interest in linguistics and/or go on with language research.

1. Douglas Hofstadter, Gödel, Escher, Bach: An Eternal Golden Braid (Basic Books, 1979), Ch. 2. ↩

2. The same does not apply to phonology, which is a very interesting field of study from a cognitive science perspective on its own. ↩

3. John R. Ross offered the term “semantax”, but it seems it did not get caught on. Mark Steedman has a project named such, though. ↩

4. Barbara Partee, “Compositionality”, in Varieties of Formal Semantics, 1984. ↩

5. I am wildly simplifying for the sake of discussion. ↩