Pullum and Gazdar (1982): Natural Languages and Context-Free Languages #
This file formalizes [PG82]'s examination of the published arguments that some
natural language is not a context-free stringset, each of which the paper finds formally or
empirically invalid. The formal exhibits are grammars. The finite state grammar of §1 generates
unbounded long-distance number concord (concordGrammar, concord_unbounded). Grammar (9)
generates exactly the nonidentity language (8) that Chomsky's comparative argument took to be
beyond context-free power (xyGrammar_language), so the language is context-free
(isXY_isContextFree). Grammar (29) generates exactly the Dutch verb-raising verb phrases (24)
with as many object names as transitive verbs, which Huybregts's cross-serial dependencies were
held to put beyond context-free power (dutchGrammar_language, isVerbPhrase_isContextFree),
and the Dutch examples (25) to (31) fall out as predicted (dutch_rows). The empirical
refutations are rows: the respectively judgments (18) and (19) contradict the number-matching
characterization of English that the argument assumes (respectively_converse), and Mohawk
classificatory and possessed incorporation, (39), (40), and (43d), break the stem-matching
premise of the Mohawk argument (stem_matching_fails).
Implementation notes #
Grammars are mathlib's ContextFreeGrammar, and the language theorems go through derivation
trees: soundness by induction on a licensed tree, with what each nonterminal generates spelled out
(Generated), and completeness by building the derivation. The Dutch verb phrases are described
as the paper does, with the categories of the lexicon (29b) as terminals; the row predictions
decide membership by stripping the names and counting the transitive verbs (isVerbPhrase),
which is proved equivalent to the description. Finite stateness is the right-linear form of the
rules, since mathlib has no bridge from grammars to regular languages. The π argument of §3, the
xx-language closure argument of §1, the ditransitive schema (32), and the Mohawk translation of
Langendoen's finite state language are not formalized.
References #
§1: unbounded concord in a finite state language #
Equations
- PullumGazdar1982.instDecidableEqWord x✝ y✝ = if h : x✝.ctorIdx = y✝.ctorIdx then isTrue ⋯ else isFalse ⋯
Equations
- PullumGazdar1982.instReprWord = { reprPrec := PullumGazdar1982.instReprWord.repr }
Equations
- One or more equations did not get rendered due to their size.
- PullumGazdar1982.instReprWord.repr PullumGazdar1982.Word.did prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.Word.did")).group prec✝
- PullumGazdar1982.instReprWord.repr PullumGazdar1982.Word.say prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.Word.say")).group prec✝
- PullumGazdar1982.instReprWord.repr PullumGazdar1982.Word.she prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.Word.she")).group prec✝
- PullumGazdar1982.instReprWord.repr PullumGazdar1982.Word.you prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.Word.you")).group prec✝
- PullumGazdar1982.instReprWord.repr PullumGazdar1982.Word.was prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.Word.was")).group prec✝
Instances For
Equations
- PullumGazdar1982.instDecidableEqConcordNT x✝ y✝ = if h : x✝.ctorIdx = y✝.ctorIdx then isTrue ⋯ else isFalse ⋯
Equations
- PullumGazdar1982.instReprConcordNT = { reprPrec := PullumGazdar1982.instReprConcordNT.repr }
Equations
- One or more equations did not get rendered due to their size.
Instances For
In the finite state grammar of §1 a singular or plural which-phrase selects the chain of she thought and you thought clauses ending in the agreeing verb.
Equations
- One or more equations did not get rendered due to their size.
Instances For
In a finite state grammar every rule rewrites to terminals followed by at most one nonterminal.
Equations
- PullumGazdar1982.IsFiniteState g = ∀ r ∈ g.rules, ∀ s ∈ r.output.dropLast, s.terminal?.isSome = true
Instances For
A chain of she thought and you thought clauses.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Concord holds over an unbounded domain in a finite state language, since for every chain of embedded clauses the plural which-phrase agrees with were and the singular with was.
§2: the nonidentity language (8) and grammar (9) #
Equations
- PullumGazdar1982.instDecidableEqSym x✝ y✝ = if h : x✝.ctorIdx = y✝.ctorIdx then isTrue ⋯ else isFalse ⋯
Equations
- One or more equations did not get rendered due to their size.
- PullumGazdar1982.instReprSym.repr PullumGazdar1982.Sym.a prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.Sym.a")).group prec✝
- PullumGazdar1982.instReprSym.repr PullumGazdar1982.Sym.b prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.Sym.b")).group prec✝
- PullumGazdar1982.instReprSym.repr PullumGazdar1982.Sym.beta prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.Sym.beta")).group prec✝
Instances For
Equations
- PullumGazdar1982.instReprSym = { reprPrec := PullumGazdar1982.instReprSym.repr }
Equations
- PullumGazdar1982.instDecidableEqXYNT x✝ y✝ = if h : x✝.ctorIdx = y✝.ctorIdx then isTrue ⋯ else isFalse ⋯
Equations
- PullumGazdar1982.instReprXYNT.repr PullumGazdar1982.XYNT.S prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.XYNT.S")).group prec✝
- PullumGazdar1982.instReprXYNT.repr PullumGazdar1982.XYNT.S₁ prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.XYNT.S₁")).group prec✝
- PullumGazdar1982.instReprXYNT.repr PullumGazdar1982.XYNT.S₂ prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.XYNT.S₂")).group prec✝
- PullumGazdar1982.instReprXYNT.repr PullumGazdar1982.XYNT.A prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.XYNT.A")).group prec✝
- PullumGazdar1982.instReprXYNT.repr PullumGazdar1982.XYNT.B prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.XYNT.B")).group prec✝
- PullumGazdar1982.instReprXYNT.repr PullumGazdar1982.XYNT.A₁ prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.XYNT.A₁")).group prec✝
- PullumGazdar1982.instReprXYNT.repr PullumGazdar1982.XYNT.B₁ prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.XYNT.B₁")).group prec✝
- PullumGazdar1982.instReprXYNT.repr PullumGazdar1982.XYNT.C prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.XYNT.C")).group prec✝
- PullumGazdar1982.instReprXYNT.repr PullumGazdar1982.XYNT.D prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.XYNT.D")).group prec✝
Instances For
Equations
- PullumGazdar1982.instReprXYNT = { reprPrec := PullumGazdar1982.instReprXYNT.repr }
Grammar (9), rules (a) to (i) in order.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Equations
- PullumGazdar1982.AB l = ∀ c ∈ l, c = PullumGazdar1982.Sym.a ∨ c = PullumGazdar1982.Sym.b
Instances For
Each nonterminal of grammar (9) generates its own strings. S₁ generates a marker between
strings of different lengths, S₂ a marker between strings differing at some position, A and
B a specified a or b flanked by equally many characters before it and after the marker, and
A₁ and B₁ a specified a or b and a tail.
Equations
- One or more equations did not get rendered due to their size.
- PullumGazdar1982.Generated PullumGazdar1982.XYNT.C x✝ = (x✝ = [PullumGazdar1982.Sym.a] ∨ x✝ = [PullumGazdar1982.Sym.b])
- PullumGazdar1982.Generated PullumGazdar1982.XYNT.D x✝ = (x✝ ≠ [] ∧ PullumGazdar1982.AB x✝)
- PullumGazdar1982.Generated PullumGazdar1982.XYNT.A₁ x✝ = ∃ (d : List PullumGazdar1982.Sym), x✝ = PullumGazdar1982.Sym.a :: d ∧ PullumGazdar1982.AB d
- PullumGazdar1982.Generated PullumGazdar1982.XYNT.B₁ x✝ = ∃ (d : List PullumGazdar1982.Sym), x✝ = PullumGazdar1982.Sym.b :: d ∧ PullumGazdar1982.AB d
- PullumGazdar1982.Generated PullumGazdar1982.XYNT.S x✝ = PullumGazdar1982.IsXY x✝
Instances For
By soundness, a tree of grammar (9) rooted at a nonterminal yields what that nonterminal
generates, which at S is a string of (8).
Completeness #
Grammar (9) generates exactly the language (8).
The nonidentity language (8) is context-free, against the formal premise of the comparative argument.
§4: respectively #
Langendoen's characterization (17) of English respectively sentences has each verb agree in number with its subject.
Equations
- PullumGazdar1982.Langendoen r = (r.feature? "subjects" = r.feature? "verbs")
Instances For
Equations
- PullumGazdar1982.instDecidableLangendoen r = id inferInstance
The judgments in (19) are the exact converse of the characterization.
(18b), the one string of (18) the characterization admits, is rejected.
§5: Dutch verb raising and grammar (29) #
The categories of the lexicon (29b) are names, transitive infinitives, intransitive verbs, transitive and intransitive VP-complement-taking infinitives, and finite transitive and intransitive VP-complement-taking verbs.
Instances For
Equations
- PullumGazdar1982.instDecidableEqCat x✝ y✝ = if h : x✝.ctorIdx = y✝.ctorIdx then isTrue ⋯ else isFalse ⋯
Equations
- PullumGazdar1982.instReprCat = { reprPrec := PullumGazdar1982.instReprCat.repr }
Equations
- PullumGazdar1982.instReprCat.repr PullumGazdar1982.Cat.B prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.Cat.B")).group prec✝
- PullumGazdar1982.instReprCat.repr PullumGazdar1982.Cat.D prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.Cat.D")).group prec✝
- PullumGazdar1982.instReprCat.repr PullumGazdar1982.Cat.E prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.Cat.E")).group prec✝
- PullumGazdar1982.instReprCat.repr PullumGazdar1982.Cat.F prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.Cat.F")).group prec✝
- PullumGazdar1982.instReprCat.repr PullumGazdar1982.Cat.G prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.Cat.G")).group prec✝
- PullumGazdar1982.instReprCat.repr PullumGazdar1982.Cat.H prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.Cat.H")).group prec✝
- PullumGazdar1982.instReprCat.repr PullumGazdar1982.Cat.I prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "PullumGazdar1982.Cat.I")).group prec✝
Instances For
The nonterminals of grammar (29a) are the verb phrase and its VP complement.
Instances For
Equations
- PullumGazdar1982.instDecidableEqDutchNT x✝ y✝ = if h : x✝.ctorIdx = y✝.ctorIdx then isTrue ⋯ else isFalse ⋯
Equations
- PullumGazdar1982.instReprDutchNT = { reprPrec := PullumGazdar1982.instReprDutchNT.repr }
Equations
- One or more equations did not get rendered due to their size.
Instances For
Whether a verb is subcategorized for a direct object.
Equations
- PullumGazdar1982.Cat.D.transitive = true
- PullumGazdar1982.Cat.F.transitive = true
- PullumGazdar1982.Cat.H.transitive = true
- x✝.transitive = false
Instances For
The number of direct objects a string of verbs requires.
Equations
- PullumGazdar1982.objects vs = List.countP PullumGazdar1982.Cat.transitive vs
Instances For
A VP complement consists of names, a finite VP-complement-taking verb, then nonfinite VP-complement-taking verbs, with one name for each transitive verb.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The verb phrases of (24), as the paper describes them, consist of n names, a finite
VP-complement-taking verb, a string of nonfinite VP-complement-taking verbs, and a final
transitive or intransitive verb, with one name for each transitive verb.
Equations
- One or more equations did not get rendered due to their size.
Instances For
By soundness, a tree of grammar (29) rooted at C yields a VP complement and one rooted at
A a verb phrase.
By completeness for C, every VP complement is derived.
By completeness, every verb phrase is derived.
Grammar (29) generates exactly the verb phrases the paper describes.
Cross-serial word order with the right number of names is a context-free stringset.
The examples (25) to (31) #
IsVerbPhrase is decided by stripping the names, reading the first and last verb, and
counting.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The category of a letter of a row's categories feature.
Equations
- PullumGazdar1982.Cat.ofChar? 'B' = some PullumGazdar1982.Cat.B
- PullumGazdar1982.Cat.ofChar? 'D' = some PullumGazdar1982.Cat.D
- PullumGazdar1982.Cat.ofChar? 'E' = some PullumGazdar1982.Cat.E
- PullumGazdar1982.Cat.ofChar? 'F' = some PullumGazdar1982.Cat.F
- PullumGazdar1982.Cat.ofChar? 'G' = some PullumGazdar1982.Cat.G
- PullumGazdar1982.Cat.ofChar? 'H' = some PullumGazdar1982.Cat.H
- PullumGazdar1982.Cat.ofChar? 'I' = some PullumGazdar1982.Cat.I
- PullumGazdar1982.Cat.ofChar? x✝ = none
Instances For
A row's category string.
Equations
- PullumGazdar1982.categories r = (r.feature? "categories").bind fun (s : String) => List.mapM PullumGazdar1982.Cat.ofChar? s.toList
Instances For
A row is predicted when its grammaticality agrees with whether its category string is a verb phrase of grammar (29).
Equations
- PullumGazdar1982.Predicted r = match PullumGazdar1982.categories r with | some w => r.judgment = Judgment.acceptable ↔ PullumGazdar1982.isVerbPhrase w = true | none => False
Instances For
Equations
- One or more equations did not get rendered due to their size.
On (25) to (31), grammar (29) accepts the grammatical clauses and rejects the ungrammatical.
§6: Mohawk noun incorporation #
The stem-matching premise of the Mohawk argument says that the stem incorporated in the verb is the head noun stem of its external argument.
Equations
- PullumGazdar1982.StemMatched r = (r.feature? "incorporated" = r.feature? "external")
Instances For
Equations
- PullumGazdar1982.instDecidableStemMatched r = id inferInstance
Classificatory incorporation (39), (40) and possessed incorporation (43d) are grammatical with mismatched stems.