Frisch, Pierrehumbert and Broe (2004): Similarity Avoidance and the OCP

This file formalizes the similarity metric of [FPB04]. Homorganic consonant pairs are rare in Arabic verbal roots, and the paper shows that the rarer a pair is the more similar its consonants are, where the similarity of two consonants is the number of natural classes they share over the number containing either (equation (7)). The natural classes are those of the feature matrix (8), generated by the structured specification of [Bro93a], and only the classes whose description includes a place feature count, so that nonhomorganic consonants have similarity 0. Every printed value of the paper's similarity table (Table III) follows from the matrix and the equation (similarities_rounds).

A larger place class has more contrasts and so more natural classes, which lowers the similarity of its members: the coronal /s, n/ are less similar than the labial /f, m/, and adding /p/ and /v/ to the labials would lower the similarity of /f, m/ from 2/9 to 2/13 (expandedSimilarity_f_m). This is how the metric derives the strength of the labial co-occurrence restriction against the weakness of the coronal one. The printed list of the classes of /b, f/ has the class {b, w}, which the matrix does not generate, in place of {f}, and the count and the reported value are unaffected (labialClasses_b_f).

The paper's quantitative comparison, co-occurrence by similarity (Table IV) and the fits of five models of OCP-Place to the O/E of every pair (Table V), is printed data in Data.Experiments.FrischPierrehumbertBroe2004.

Main definitions

• placeClasses: the natural classes of the matrix whose description includes a place feature.
• similarity: the similarity of two consonants of the Arabic fragment, equation (7).

Main results

• similarities_rounds: the matrix and the equation give every value of Table III.
• similarity_self, similarity_eq_zero: identical consonants have similarity 1, and nonhomorganic ones 0.
• expandedSimilarity_f_m: the expanded labial inventory lowers the similarity of /f, m/.

Implementation notes

• A class is a set of columns of the matrix held as the bits of a natural number (Mask), so that the kernel meets classes by Nat.land. The classes containing a column are generated from the place classes containing it by the attribute extents containing it (classesAt), which is how the table is computed.
• The paper writes ج as the dorsal stop g and ظ as zˤ; their segments are the fragment's /dʒ/ and /ðˤ/ (segment).
• The paper does not print the columns of the hypothetical /p/ and /v/; they are those of /b/ and /f/ with the other value of [voice], which generate the four new classes it lists.
• The best monotone fit of O/E to similarity behind Table V is not formalized, since the paper does not print the O/E of the individual pairs.

References

• [FPB04]
• [Bro93a]

Sets of columns

structure FrischPierrehumbertBroe2004.Mask : Type

A set of columns of a feature matrix, as the bits of a natural number.

• val : ℕ

  The bits.

theorem FrischPierrehumbertBroe2004.Mask.ext {x y : Mask} (val : x.val = y.val) : x = y

theorem FrischPierrehumbertBroe2004.Mask.ext_iff {x y : Mask} : x = y ↔ x.val = y.val

@[instance_reducible]

instance FrischPierrehumbertBroe2004.instDecidableEqMask : DecidableEq Mask

Equations

• FrischPierrehumbertBroe2004.instDecidableEqMask = FrischPierrehumbertBroe2004.instDecidableEqMask.decEq

def FrischPierrehumbertBroe2004.instDecidableEqMask.decEq (x✝ x✝¹ : Mask) : Decidable (x✝ = x✝¹)

Equations

• FrischPierrehumbertBroe2004.instDecidableEqMask.decEq { val := a } { val := b } = if h : a = b then h ▸ isTrue ⋯ else isFalse ⋯

@[instance_reducible]

instance FrischPierrehumbertBroe2004.Mask.instPartialOrder : PartialOrder Mask

Equations

• One or more equations did not get rendered due to their size.

@[instance_reducible]

instance FrischPierrehumbertBroe2004.Mask.instSemilatticeInf : SemilatticeInf Mask

Equations

• One or more equations did not get rendered due to their size.

@[instance_reducible]

instance FrischPierrehumbertBroe2004.Mask.instDecidableLE : DecidableLE Mask

Equations

• a.instDecidableLE b = { decide := (a.val &&& b.val).beq a.val, reflects_decide := ⋯ }

@[instance_reducible]

instance FrischPierrehumbertBroe2004.Mask.instMembershipNat : Membership ℕ Mask

A column is in a set when its bit is set.

Equations

• FrischPierrehumbertBroe2004.Mask.instMembershipNat = { mem := fun (m : FrischPierrehumbertBroe2004.Mask) (i : ℕ) => m.val.testBit i = true }

@[instance_reducible]

instance FrischPierrehumbertBroe2004.Mask.instDecidableMemNat (i : ℕ) (m : Mask) : Decidable (i ∈ m)

Equations

• FrischPierrehumbertBroe2004.Mask.instDecidableMemNat i m = instDecidableEqBool (m.val.testBit i) true

def FrischPierrehumbertBroe2004.Mask.single (i : ℕ) : Mask

The set of one column.

Equations

• FrischPierrehumbertBroe2004.Mask.single i = { val := 2 ^ i }

theorem FrischPierrehumbertBroe2004.Mask.mem_inf {i : ℕ} {a b : Mask} : i ∈ a ⊓ b ↔ i ∈ a ∧ i ∈ b

theorem FrischPierrehumbertBroe2004.Mask.single_le_iff {i : ℕ} {m : Mask} : single i ≤ m ↔ i ∈ m

theorem FrischPierrehumbertBroe2004.Mask.mem_of_le {i : ℕ} {a b : Mask} (hi : i ∈ a) (h : a ≤ b) : i ∈ b

def FrischPierrehumbertBroe2004.Mask.ofList : List Bool → Mask

The set of the positions at which a list is true.

Equations

• FrischPierrehumbertBroe2004.Mask.ofList [] = { val := 0 }
• FrischPierrehumbertBroe2004.Mask.ofList (b :: l) = { val := 2 * (FrischPierrehumbertBroe2004.Mask.ofList l).val + b.toNat }

Matrix (8)

inductive FrischPierrehumbertBroe2004.Feature : Type

The features of matrix (8).

• cons : Feature
• son : Feature
• cont : Feature
• acute : Feature
• strident : Feature
• nasal : Feature
• lateral : Feature
• labial : Feature
• coronal : Feature
• anterior : Feature
• dorsal : Feature
• back : Feature
• phar : Feature
• radical : Feature
• voice : Feature
• spglot : Feature
• conglot : Feature

@[instance_reducible]

instance FrischPierrehumbertBroe2004.instDecidableEqFeature : DecidableEq Feature

Equations

• FrischPierrehumbertBroe2004.instDecidableEqFeature x✝ y✝ = if h : x✝.ctorIdx = y✝.ctorIdx then isTrue ⋯ else isFalse ⋯

def FrischPierrehumbertBroe2004.Feature.all : List Feature

The features in the order of (8).

Equations

• One or more equations did not get rendered due to their size.

def FrischPierrehumbertBroe2004.Feature.places : List Feature

The place features.

Equations

• One or more equations did not get rendered due to their size.

def FrischPierrehumbertBroe2004.columns : List Phonology.Segment

The consonants of (8) in its column order, as the fragment's segments.

Equations

• One or more equations did not get rendered due to their size.

def FrischPierrehumbertBroe2004.col (x : Phonology.Segment) : ℕ

The column of a consonant in (8).

Equations

• FrischPierrehumbertBroe2004.col x = List.idxOf x FrischPierrehumbertBroe2004.columns

def FrischPierrehumbertBroe2004.segment : Consonant → Phonology.Segment

The fragment's segment for a consonant of (8).

Equations

• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.b = Arabic.ModernStandard.Phonology.b
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.f = Arabic.ModernStandard.Phonology.f
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.m = Arabic.ModernStandard.Phonology.m
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.t = Arabic.ModernStandard.Phonology.t
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.d = Arabic.ModernStandard.Phonology.d
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.tEmph = Arabic.ModernStandard.Phonology.«tˤ»
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.dEmph = Arabic.ModernStandard.Phonology.«dˤ»
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.theta = Arabic.ModernStandard.Phonology.θ
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.edh = Arabic.ModernStandard.Phonology.ð
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.s = Arabic.ModernStandard.Phonology.s
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.z = Arabic.ModernStandard.Phonology.z
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.sEmph = Arabic.ModernStandard.Phonology.«sˤ»
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.zEmph = Arabic.ModernStandard.Phonology.«ðˤ»
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.esh = Arabic.ModernStandard.Phonology.«ʃ»
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.k = Arabic.ModernStandard.Phonology.k
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.g = Arabic.ModernStandard.Phonology.«dʒ»
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.q = Arabic.ModernStandard.Phonology.q
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.chi = Arabic.ModernStandard.Phonology.χ
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.gamma = Arabic.ModernStandard.Phonology.«ʁ»
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.hbar = Arabic.ModernStandard.Phonology.ħ
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.ayin = Arabic.ModernStandard.Phonology.«ʕ»
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.h = Arabic.ModernStandard.Phonology.h
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.glottal = Arabic.ModernStandard.Phonology.«ʔ»
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.l = Arabic.ModernStandard.Phonology.l
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.r = Arabic.ModernStandard.Phonology.r
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.n = Arabic.ModernStandard.Phonology.n
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.w = Arabic.ModernStandard.Phonology.w
• FrischPierrehumbertBroe2004.segment FrischPierrehumbertBroe2004.Consonant.j = Arabic.ModernStandard.Phonology.j

theorem FrischPierrehumbertBroe2004.columns_toFinset : columns.toFinset = Arabic.ModernStandard.Phonology.consonants

The columns are the fragment's consonants.

theorem FrischPierrehumbertBroe2004.col_segment (c : Consonant) : col (segment c) = c.ctorIdx

The data's consonants are the columns in order.

def FrischPierrehumbertBroe2004.matrix : Feature → List (Option Bool)

Matrix (8), row by row over columns: ⊕ and ⊖ for the printed + and −, ○ where the paper leaves the feature unspecified.

Equations

• One or more equations did not get rendered due to their size.

Natural classes and similarity

def FrischPierrehumbertBroe2004.extent (row : List (Option Bool)) (v : Bool) : Mask

The columns at which a row of a matrix has a value.

Equations

• FrischPierrehumbertBroe2004.extent row v = FrischPierrehumbertBroe2004.Mask.ofList (List.map (fun (x : Option Bool) => x == some v) row)

def FrischPierrehumbertBroe2004.attributesOf (M : Feature → List (Option Bool)) : List Mask

The attribute extents of a matrix: the columns of each feature value.

Equations

• One or more equations did not get rendered due to their size.

def FrischPierrehumbertBroe2004.attributes : List Mask

The attribute extents of (8).

Equations

• FrischPierrehumbertBroe2004.attributes = FrischPierrehumbertBroe2004.attributesOf FrischPierrehumbertBroe2004.matrix

def FrischPierrehumbertBroe2004.inventory : Mask

All the columns of (8).

Equations

• FrischPierrehumbertBroe2004.inventory = FrischPierrehumbertBroe2004.Mask.ofList (List.map (fun (x : Phonology.Segment) => true) FrischPierrehumbertBroe2004.columns)

def FrischPierrehumbertBroe2004.places : List Mask

The place classes of (8): the columns of each place feature.

Equations

• One or more equations did not get rendered due to their size.

def FrischPierrehumbertBroe2004.placeClasses : List Mask

The natural classes of (8) whose description includes a place feature (p. 198).

Equations

• One or more equations did not get rendered due to their size.

def FrischPierrehumbertBroe2004.shared (x y : Phonology.Segment) : ℕ

The place classes two consonants share.

Equations

• One or more equations did not get rendered due to their size.

def FrischPierrehumbertBroe2004.either (x y : Phonology.Segment) : ℕ

The place classes containing either of two consonants.

Equations

• One or more equations did not get rendered due to their size.

def FrischPierrehumbertBroe2004.similarity (x y : Phonology.Segment) : ℚ

The natural-classes similarity of two consonants, equation (7): the shared classes over the shared and non-shared ones.

Equations

• FrischPierrehumbertBroe2004.similarity x y = ↑(FrischPierrehumbertBroe2004.shared x y) / ↑(FrischPierrehumbertBroe2004.either x y)

theorem FrischPierrehumbertBroe2004.similarity_comm {x y : Phonology.Segment} : similarity x y = similarity y x

theorem FrischPierrehumbertBroe2004.similarity_self {x : Phonology.Segment} (h : ∃ p ∈ places, col x ∈ p) : similarity x x = 1

Identical consonants have similarity 1 (p. 198).

theorem FrischPierrehumbertBroe2004.similarity_eq_zero {x y : Phonology.Segment} (h : ∀ p ∈ places, col x ∈ p → col y ∉ p) : similarity x y = 0

Consonants with no place feature in common have similarity 0 (p. 198).

Computing the table

def FrischPierrehumbertBroe2004.classesAt (i : ℕ) : List Mask

The place classes containing column i: those each place class containing i generates with the attribute extents containing i.

Equations

• One or more equations did not get rendered due to their size.

theorem FrischPierrehumbertBroe2004.nodup_classesAt (i : ℕ) : (classesAt i).Nodup

theorem FrischPierrehumbertBroe2004.mem_classesAt {i : ℕ} {s : Mask} : s ∈ classesAt i ↔ s ∈ placeClasses ∧ i ∈ s

theorem FrischPierrehumbertBroe2004.shared_eq {x y : Phonology.Segment} : shared x y = List.countP (fun (x : Mask) => decide (col y ∈ x)) (classesAt (col x))

theorem FrischPierrehumbertBroe2004.either_eq {x y : Phonology.Segment} : either x y = (classesAt (col x)).length + (classesAt (col y)).length - shared x y

def FrischPierrehumbertBroe2004.classesTable : List (List Mask)

The classes containing each column, memoized for the kernel.

Equations

• FrischPierrehumbertBroe2004.classesTable = List.map FrischPierrehumbertBroe2004.classesAt (List.range 28)

theorem FrischPierrehumbertBroe2004.shared_segment (c c' : Consonant) : shared (segment c) (segment c') = List.countP (fun (x : Mask) => decide (c'.ctorIdx ∈ x)) (classesTable.getD c.ctorIdx [])

theorem FrischPierrehumbertBroe2004.either_segment (c c' : Consonant) : either (segment c) (segment c') = (classesTable.getD c.ctorIdx []).length + (classesTable.getD c'.ctorIdx []).length - List.countP (fun (x : Mask) => decide (c'.ctorIdx ∈ x)) (classesTable.getD c.ctorIdx [])

Table III

theorem FrischPierrehumbertBroe2004.similarities_rounds (r : Similarity) : r ∈ similarities → r.similarity.Rounds (shared (segment r.c1) (segment r.c2)) (either (segment r.c1) (segment r.c2))

Matrix (8) and equation (7) give every printed value of Table III.

The labials (p. 199)

def FrischPierrehumbertBroe2004.labialClasses : List Mask

The classes whose description includes [labial], those of the labial sub-inventory (Broe1993.mem_naturalClasses_inf).

Equations

• One or more equations did not get rendered due to their size.

def FrischPierrehumbertBroe2004.cl (xs : List Phonology.Segment) : Mask

The set of the columns of some consonants.

Equations

• FrischPierrehumbertBroe2004.cl xs = { val := (List.map (fun (x : Phonology.Segment) => 2 ^ FrischPierrehumbertBroe2004.col x) xs).sum }

theorem FrischPierrehumbertBroe2004.labialClasses_f_m : (List.filter (fun (s : Mask) => decide (col Arabic.ModernStandard.Phonology.f ∈ s ∨ col Arabic.ModernStandard.Phonology.m ∈ s)) labialClasses).toFinset = {cl [Arabic.ModernStandard.Phonology.b, Arabic.ModernStandard.Phonology.f, Arabic.ModernStandard.Phonology.m, Arabic.ModernStandard.Phonology.w], cl [Arabic.ModernStandard.Phonology.b, Arabic.ModernStandard.Phonology.f, Arabic.ModernStandard.Phonology.m], cl [Arabic.ModernStandard.Phonology.b, Arabic.ModernStandard.Phonology.f], cl [Arabic.ModernStandard.Phonology.f, Arabic.ModernStandard.Phonology.w], cl [Arabic.ModernStandard.Phonology.f], cl [Arabic.ModernStandard.Phonology.b, Arabic.ModernStandard.Phonology.m, Arabic.ModernStandard.Phonology.w], cl [Arabic.ModernStandard.Phonology.b, Arabic.ModernStandard.Phonology.m], cl [Arabic.ModernStandard.Phonology.m, Arabic.ModernStandard.Phonology.w], cl [Arabic.ModernStandard.Phonology.m]}

The labial classes containing /f/ or /m/ are the nine the paper lists.

theorem FrischPierrehumbertBroe2004.labialClasses_b_f : (List.filter (fun (s : Mask) => decide (col Arabic.ModernStandard.Phonology.b ∈ s ∨ col Arabic.ModernStandard.Phonology.f ∈ s)) labialClasses).toFinset = insert (cl [Arabic.ModernStandard.Phonology.f]) ({cl [Arabic.ModernStandard.Phonology.b, Arabic.ModernStandard.Phonology.f, Arabic.ModernStandard.Phonology.m, Arabic.ModernStandard.Phonology.w], cl [Arabic.ModernStandard.Phonology.b, Arabic.ModernStandard.Phonology.f, Arabic.ModernStandard.Phonology.m], cl [Arabic.ModernStandard.Phonology.b, Arabic.ModernStandard.Phonology.f], cl [Arabic.ModernStandard.Phonology.f, Arabic.ModernStandard.Phonology.w], cl [Arabic.ModernStandard.Phonology.b, Arabic.ModernStandard.Phonology.m, Arabic.ModernStandard.Phonology.w], cl [Arabic.ModernStandard.Phonology.b, Arabic.ModernStandard.Phonology.m], cl [Arabic.ModernStandard.Phonology.b, Arabic.ModernStandard.Phonology.w], cl [Arabic.ModernStandard.Phonology.b]}.erase (cl [Arabic.ModernStandard.Phonology.b, Arabic.ModernStandard.Phonology.w]))

The labial classes containing /b/ or /f/ are the eight the paper lists, with {f} for the printed {b, w}, which no description over the matrix picks out.

theorem FrischPierrehumbertBroe2004.similarity_f_m : similarity Arabic.ModernStandard.Phonology.f Arabic.ModernStandard.Phonology.m = 2 / 9

theorem FrischPierrehumbertBroe2004.similarity_b_f : similarity Arabic.ModernStandard.Phonology.b Arabic.ModernStandard.Phonology.f = 3 / 8

theorem FrischPierrehumbertBroe2004.similarity_s_n_lt_f_m : similarity Arabic.ModernStandard.Phonology.s Arabic.ModernStandard.Phonology.n < similarity Arabic.ModernStandard.Phonology.f Arabic.ModernStandard.Phonology.m

The coronals /s, n/ are less similar than the labials /f, m/.

def FrischPierrehumbertBroe2004.expandedLabials : Feature → List (Option Bool)

The labial columns of (8) with /p/ and /v/ added, in the order p b f v m w: /p/ is /b/ and /v/ is /f/ with the other value of [voice].

Equations

• FrischPierrehumbertBroe2004.expandedLabials FrischPierrehumbertBroe2004.Feature.cons = [some true, some true, some true, some true, some true, some false]
• FrischPierrehumbertBroe2004.expandedLabials FrischPierrehumbertBroe2004.Feature.son = [some false, some false, some false, some false, some true, some true]
• FrischPierrehumbertBroe2004.expandedLabials FrischPierrehumbertBroe2004.Feature.cont = [some false, some false, some true, some true, some false, some true]
• FrischPierrehumbertBroe2004.expandedLabials FrischPierrehumbertBroe2004.Feature.nasal = [some false, some false, none, none, some true, none]
• FrischPierrehumbertBroe2004.expandedLabials FrischPierrehumbertBroe2004.Feature.labial = [some true, some true, some true, some true, some true, some true]
• FrischPierrehumbertBroe2004.expandedLabials FrischPierrehumbertBroe2004.Feature.voice = [some false, some true, some false, some true, some true, some true]
• FrischPierrehumbertBroe2004.expandedLabials x✝ = [none, none, none, none, none, none]

theorem FrischPierrehumbertBroe2004.expandedLabials_restrict (g : Feature) : g ∈ Feature.all → List.map (fun (x : ℕ) => (expandedLabials g).getD x none) [1, 2, 4, 5] = List.map (fun (x : ℕ) => (matrix g).getD x none) [col Arabic.ModernStandard.Phonology.b, col Arabic.ModernStandard.Phonology.f, col Arabic.ModernStandard.Phonology.m, col Arabic.ModernStandard.Phonology.w]

Without /p/ and /v/ the expanded columns are the labial columns of (8).

def FrischPierrehumbertBroe2004.expandedClasses : List Mask

The natural classes of the expanded labial inventory.

Equations

• One or more equations did not get rendered due to their size.

theorem FrischPierrehumbertBroe2004.expandedSimilarity_f_m : List.countP (fun (s : Mask) => decide (2 ∈ s ∧ 4 ∈ s)) expandedClasses = 2 ∧ List.countP (fun (s : Mask) => decide (2 ∈ s ∨ 4 ∈ s)) expandedClasses = 13

In the expanded inventory /f, m/ share 2 of the 13 classes containing either, where they shared 2 of 9: the new classes are {f, v}, {p, f}, {p, b, m} and {b, v, m}.