Docs / Processing / DiscriminativeLexicon / Coding

Form and meaning coding for the discriminative lexicon

import Linglib.Processing.DiscriminativeLexicon.Coding · namespace DiscriminativeLexicon

This file defines how linguistic objects enter the discriminative lexicon, following chapters 4 and 5 of Heitmeier, Chuang and Baayen.

A form is a string of symbols. Its cues are the n-grams of the string padded with one boundary symbol on each side (#a aa ap p#, #aa aap ap#), and its row of the form matrix C is the indicator of those cues over the cue inventory, so C holds only ones and zeros. The padding is one boundary symbol whatever the width, unlike the k − 1 symbols of strictly local grammars (boundary), so the model's trigrams are not the 3-factors of subregular phonology. A meaning is a multiset of atomic semantic primitives, a lexeme together with its inflectional functions, and conceptualization builds its vector as the sum of the primitives' vectors, so that a novel inflected word is conceptualized from known primitives. Conceptualization is additive in the multiset by construction, which makes imputed meanings analogically regular (isAnalogicallyRegular_imputed in Realization.lean).

Main definitions

cues k w: the k-gram cues of the string w.
multiHot inv p: the indicator row over an inventory inv of the units satisfying p.
cueVector k inv w: the row of C for w.
conceptualize emb: the additive map from primitive multisets to meaning vectors.
imputed σ ε: the meaning of a lexeme at a cell, the lexeme's vector plus the cell's.

References

M. Heitmeier, Y.-Y. Chuang and R. H. Baayen, The Discriminative Lexicon (2026)

Form side

The k-gram cues of a form are the k-factors of the string padded with one boundary symbol on each side, as JudiLing's make_cue_matrix computes them with grams = k.

def cues L47–49

{Sym : Type u_1} (k : ℕ) (w : List Sym) :

List (Augmented Sym)

cues k w = List.kFactors k (boundary 2 w)

Used by cueVector

The multiple-hot row over an inventory inv marks with 1 the units satisfying p.

def multiHot L51–53

{κ : Type u_2} {α : Type u_3} (inv : κ → α) (p : α → Prop) [DecidablePred p] :

κ → ℝ

multiHot inv p j = if p (inv j) then 1 else 0

Used by cueVector

Used in GahlBaayen2024

The row of the form matrix C for the form w is the indicator of its cues over the cue inventory.

def cueVector L55–59

{Sym : Type u_1} [DecidableEq Sym] (k : ℕ) {κ : Type u_2} (inv : κ → Augmented Sym) (w : List Sym) :

κ → ℝ

cueVector k inv w = multiHot inv fun (x : Augmented Sym) => x ∈ cues k w

Built from ,

Used in GahlBaayen2024, HeitmeierChuangBaayen2026

Meaning side

Conceptualization sends a multiset of semantic primitives to the sum of the primitives' vectors, additively in the multiset.

def conceptualize L65–68

{Prim : Type u_2} {V : Type u_3} [AddCommMonoid V] (emb : Prim → V) :

Multiset Prim →+ V

conceptualize emb = Multiset.sumAddMonoidHom.comp (Multiset.mapAddMonoidHom emb)

Used by conceptualize_apply, conceptualize_pair, imputed

L70–71

L73–77

The imputed semantics of a lexeme at a cell is conceptualized from the lexeme's vector and the inflectional function's vector (eq. 5.3; the constructed meaning-to-form route of Table 12.7, which §16.6 calls imputed embeddings for stems and exponents).

def imputed L79–83

{V : Type u_3} [AddCommMonoid V] {A : Type u_4} {B : Type u_5} (σ : A → V) (ε : B → V) (a : A) (b : B) :

imputed σ ε a b = (conceptualize (Sum.elim σ ε)) {Sum.inl a, Sum.inr b}

Built from

Used by imputed_apply

Used in Processing/DiscriminativeLexicon/Realization, HeitmeierChuangBaayen2026

L85–87