Documentation

Linglib.Phonology.OptimalityTheory.PartiallyOrderedConstraints

Partially Ordered Constraints (POC) #

A POC grammar is a partial order on the constraint set ([Ant97]; [Kip93]) — represented as mathlib represents orders-as-data (extend_partialOrder): a bare relation r : Fin n → Fin n → Prop with [IsPartialOrder (Fin n) r], and [DecidableRel r] where counting needs it. Each evaluation samples a total order consistent with the partial order — a linear extension — and the OT optimum under that ranking is the output, so a single grammar induces a distribution over outputs, uniform over consistent linear extensions. The load-bearing identities are division-free cardinality equations (sum_card_filter_picksAt here, the head-fiber counting in Core.Optimization.PermSubsetCombinatorics); winProb and its rate theorems are a ℚ veneer over them.

Main definitions #

Main statements #

Implementation notes #

Grammars are unbundled relations, never PartialOrder (Fin n) values: a class-typed binder would become a local instance and capture the /< notation that must keep meaning Fin n's positional order. This is also mathlib's own idiom for orders treated as data (Szpilrajn's extend_partialOrder).

Grammars and their linear extensions #

instance OptimalityTheory.instIsPartialOrderEq_linglib {α : Type u_1} :
IsPartialOrder α fun (x1 x2 : α) => x1 = x2

Equality is a partial order — the discrete order, relating nothing beyond reflexivity. As a POC grammar it is [Ant97]'s "no ranking imposed" baseline: every permutation is a consistent linear extension.

def OptimalityTheory.IsConsistent {n : } (r : Fin nFin nProp) (σ : Ranking n) :

A permutation σ is consistent with grammar r when r is contained in the total order σ induces (Ranking.toRel) — σ is a linear extension of r. Unfolds to ∀ a b, r a b → σ.symm a ≤ σ.symm b.

Equations
Instances For
    @[instance_reducible]
    instance OptimalityTheory.instDecidableIsConsistentOfDecidableRelFin {n : } (r : Fin nFin nProp) [DecidableRel r] (σ : Ranking n) :
    Decidable (IsConsistent r σ)
    Equations
    def OptimalityTheory.consistentTotalOrders {n : } (r : Fin nFin nProp) [DecidableRel r] :
    Finset (Ranking n)

    The (decidable, finite) set of linear extensions of r.

    Equations
    Instances For
      @[simp]
      theorem OptimalityTheory.mem_consistentTotalOrders {n : } {r : Fin nFin nProp} [DecidableRel r] {σ : Ranking n} :
      theorem OptimalityTheory.isConsistent_mul_iff {n : } {r : Fin nFin nProp} {g σ : Ranking n} :
      IsConsistent r (g * σ) IsConsistent (fun (a b : Fin n) => r (g a) (g b)) σ

      Consistency transports along constraint relabeling: g * σ extends r iff σ extends the g-pullback of r.

      theorem OptimalityTheory.IsConsistent.mul {n : } {r : Fin nFin nProp} {g σ : Ranking n} (hg : ∀ (a b : Fin n), r (g a) (g b) r a b) ( : IsConsistent r σ) :
      IsConsistent r (g * σ)

      The linear extensions of a grammar are closed under its symmetries: a relabeling that preserves r acts on the consistent rankings.

      theorem OptimalityTheory.consistentTotalOrders_discrete (n : ) :
      (consistentTotalOrders fun (x1 x2 : Fin n) => x1 = x2) = Finset.univ

      For the discrete grammar, every permutation is a linear extension.

      σ is consistent with the total order it induces — reflexivity of the relation lattice.

      @[simp]

      A ranking-induced order has its ranking as unique consistent linear extension, by the rigidity of Ranking.toRel (Ranking.toRel_le_toRel_iff).

      Szpilrajn — every grammar has a consistent linear extension #

      theorem OptimalityTheory.exists_isConsistent {n : } (r : Fin nFin nProp) [IsPartialOrder (Fin n) r] :
      ∃ (σ : Ranking n), IsConsistent r σ

      Every grammar has a consistent linear extension: Szpilrajn (extend_partialOrder) extends r to a linear order, which is some ranking's induced order (Ranking.exists_toRel_eq).

      theorem OptimalityTheory.consistentTotalOrders_nonempty {n : } (r : Fin nFin nProp) [IsPartialOrder (Fin n) r] [DecidableRel r] :
      theorem OptimalityTheory.consistentTotalOrders_card_pos {n : } (r : Fin nFin nProp) [IsPartialOrder (Fin n) r] [DecidableRel r] :

      Stratified grammars #

      Earlier strata dominate later ones wholesale, and within a stratum an inner order applies. With the discrete inner order this is the freely-ranked stratum grammar of [Ant97] eq. (50) — the Stratified Domination Hierarchy of [TS95] that constraint-demotion learning produces.

      def OptimalityTheory.stratified {n s : } (stratumOf : Fin nFin s) (inner : Fin nFin nProp) :
      Fin nFin nProp

      The stratified grammar induced by stratumOf and an inner order inner — mathlib's lexicographic order (Prod.Lex) pulled back along a ↦ (stratumOf a, a): strata compare strictly, ties defer to inner. The plain characterization is stratified_iff; cross-stratum inner edges are ignored.

      Equations
      Instances For
        theorem OptimalityTheory.stratified_iff {n s : } {stratumOf : Fin nFin s} {inner : Fin nFin nProp} {a b : Fin n} :
        stratified stratumOf inner a b stratumOf a < stratumOf b stratumOf a = stratumOf b inner a b

        Dominance in a stratified grammar holds iff a's stratum strictly precedes b's, or they share a stratum and the inner order relates them.

        instance OptimalityTheory.instIsPartialOrderFinStratified {n s : } {stratumOf : Fin nFin s} {inner : Fin nFin nProp} [IsPartialOrder (Fin n) inner] :
        IsPartialOrder (Fin n) (stratified stratumOf inner)
        @[instance_reducible]
        instance OptimalityTheory.instDecidableRelFinStratified {n s : } {stratumOf : Fin nFin s} {inner : Fin nFin nProp} [DecidableRel inner] :
        DecidableRel (stratified stratumOf inner)
        Equations
        theorem OptimalityTheory.IsConsistent.symm_lt_of_stratum_lt {n s : } {stratumOf : Fin nFin s} {inner : Fin nFin nProp} {σ : Ranking n} ( : IsConsistent (stratified stratumOf inner) σ) {a b : Fin n} (h : stratumOf a < stratumOf b) :
        (Equiv.symm σ) a < (Equiv.symm σ) b

        Under a stratified grammar, an earlier-stratum constraint occupies a strictly earlier position in every consistent ranking.

        theorem OptimalityTheory.stratified_swap_apply_iff {n s : } {stratumOf : Fin nFin s} {inner : Fin nFin nProp} {k : Fin s} {d d' : Fin n} [Std.Refl inner] (h_triv : ∀ (a b : Fin n), stratumOf a = kstratumOf b = kinner a ba = b) (hd : stratumOf d = k) (hd' : stratumOf d' = k) (a b : Fin n) :
        stratified stratumOf inner ((Equiv.swap d d') a) ((Equiv.swap d d') b) stratified stratumOf inner a b

        Swapping two constraints of a stratum on which the inner order is trivial is a symmetry of the stratified grammar.

        theorem OptimalityTheory.isConsistent_swap_mul {n s : } {stratumOf : Fin nFin s} {inner : Fin nFin nProp} {k : Fin s} {d d' : Fin n} {σ : Ranking n} [Std.Refl inner] (h_triv : ∀ (a b : Fin n), stratumOf a = kstratumOf b = kinner a ba = b) (hd : stratumOf d = k) (hd' : stratumOf d' = k) ( : IsConsistent (stratified stratumOf inner) σ) :
        IsConsistent (stratified stratumOf inner) (Equiv.swap d d' * σ)

        Consistent rankings of a stratified grammar are closed under swapping two constraints of a stratum on which the inner order is trivial.

        Grounding in the ERC lex API #

        A partial order is a set of dominance requirements — each related pair is a simple ERC a ≫ b ([MR16]), and under this encoding the consistent total orders are exactly ERC.linearExtensions ([Pri02]).

        def OptimalityTheory.toERCs {n : } (r : Fin nFin nProp) [DecidableRel r] :
        Finset (ERC n)

        The simple-ERC encoding of a grammar, with one ERC a ≫ b (simpleERC a b) for each related pair — diagonal pairs give trivial ERCs, matching toRel's reflexivity. Transitively-implied pairs are entailed by the covering pairs, so the encoding has the same linear extensions as the Hasse-edge one.

        Equations
        Instances For
          theorem OptimalityTheory.mem_toERCs {n : } {r : Fin nFin nProp} [DecidableRel r] {α : ERC n} :
          α toERCs r ∃ (a : Fin n) (b : Fin n), r a b simpleERC a b = α
          theorem OptimalityTheory.satisfiedBy_toERCs {n : } {r : Fin nFin nProp} [DecidableRel r] {σ : Ranking n} :
          (∀ αtoERCs r, ERC.SatisfiedBy σ α) IsConsistent r σ

          A ranking satisfies toERCs r exactly when it is a linear extension of r: per pair, satisfaction of simpleERC a b is σ.toRel a b.

          theorem OptimalityTheory.consistentTotalOrders_eq_linearExtensions {n : } (r : Fin nFin nProp) [DecidableRel r] :

          The consistent total orders of a grammar are exactly the linear extensions of its simple-ERC encoding ([MR16]; [Pri02]).

          The order-ideal antimatroid of a POC #

          A grammar's Hasse-edge encoding is a consistent set of simple ERCs, so it has a Birkhoff antimatroid whose feasible sets are exactly the order ideals of r ([Dil40]; [MR16]).

          theorem OptimalityTheory.toERCs_isSimple_or_isTrivial {n : } (r : Fin nFin nProp) [DecidableRel r] (α : ERC n) :
          α toERCs rα.IsSimple α.IsTrivial

          Every member of toERCs r is a simple ERC or (on the diagonal) trivial.

          theorem OptimalityTheory.toERCs_consistent {n : } (r : Fin nFin nProp) [IsPartialOrder (Fin n) r] [DecidableRel r] :

          toERCs r is consistent: any linear extension of r satisfies it.

          def OptimalityTheory.orderIdealAntimatroid {n : } (r : Fin nFin nProp) [IsPartialOrder (Fin n) r] [DecidableRel r] :
          Antimatroid (Fin n)

          The order-ideal antimatroid of a grammar — the simple-ERC Birkhoff antimatroid (Antimat.ofSimple) of its Hasse-edge encoding, whose feasible sets are exactly the order ideals of r (orderIdealAntimatroid_isFeasible_iff).

          Equations
          Instances For
            theorem OptimalityTheory.feasible_toERCs_iff {n : } (r : Fin nFin nProp) [DecidableRel r] {S : Finset (Fin n)} :
            Feasible (toERCs r) S ∀ (a b : Fin n), r a bb Sa S

            Local feasibility against toERCs r is exactly the order-ideal condition — whenever b ∈ S and a dominates b, also a ∈ S.

            @[simp]
            theorem OptimalityTheory.orderIdealAntimatroid_isFeasible_iff {n : } (r : Fin nFin nProp) [IsPartialOrder (Fin n) r] [DecidableRel r] {S : Finset (Fin n)} :
            (orderIdealAntimatroid r).IsFeasible S ∀ (a b : Fin n), r a bb Sa S

            The feasible sets of orderIdealAntimatroid are the order ideals of r — the Birkhoff correspondence, made concrete and decidable.

            Bridge to the Grammar hub #

            A partial order on constraints is the simple-ERC fragment of an OT grammar — its consistent total orders are exactly the legs of Grammar.ofERCs (toERCs r) ([MR16]).

            def OptimalityTheory.toGrammar {n : } (r : Fin nFin nProp) [IsPartialOrder (Fin n) r] [DecidableRel r] :

            The Grammar whose legs are r's consistent total orders.

            Equations
            Instances For
              @[simp]
              theorem OptimalityTheory.toGrammar_legs {n : } (r : Fin nFin nProp) [IsPartialOrder (Fin n) r] [DecidableRel r] :

              Probabilistic POC — winProb #

              def OptimalityTheory.active {n : } {Input : Type u_1} {Output : Type u_2} (vp : InputOutputFin n) (i : Input) (o o' : Output) :
              Finset (Fin n)

              The constraints active on the candidate pair o, o' at input i — those assigning the two candidates different violation counts ([Ant97]'s decisive constraints). Inactive constraints cannot affect the competition.

              Equations
              Instances For
                def OptimalityTheory.favoring {n : } {Input : Type u_1} {Output : Type u_2} (vp : InputOutputFin n) (i : Input) (o o' : Output) :
                Finset (Fin n)

                The constraints favoring o over o' at input i — those assigning o strictly fewer violations.

                Equations
                Instances For
                  @[simp]
                  theorem OptimalityTheory.mem_active {n : } {Input : Type u_1} {Output : Type u_2} {vp : InputOutputFin n} {i : Input} {o o' : Output} {c : Fin n} :
                  c active vp i o o' vp i o c vp i o' c
                  @[simp]
                  theorem OptimalityTheory.mem_favoring {n : } {Input : Type u_1} {Output : Type u_2} {vp : InputOutputFin n} {i : Input} {o o' : Output} {c : Fin n} :
                  c favoring vp i o o' vp i o c < vp i o' c
                  theorem OptimalityTheory.favoring_subset_active {n : } {Input : Type u_1} {Output : Type u_2} {vp : InputOutputFin n} {i : Input} {o o' : Output} :
                  favoring vp i o o'active vp i o o'
                  def OptimalityTheory.PicksAt {n : } {Input : Type u_1} {Output : Type u_2} (cands : InputFinset Output) (vp : InputOutputFin n) (σ : Ranking n) (i : Input) (o : Output) :

                  σ picks output o for input i if o is the unique strict OT winner — every other in-set candidate is lex-strictly worse than o under σ.

                  Equations
                  • OptimalityTheory.PicksAt cands vp σ i o = (o cands i o'cands i, o' o(toLex fun (k : Fin n) => vp i o (σ k)) < toLex fun (k : Fin n) => vp i o' (σ k))
                  Instances For
                    theorem OptimalityTheory.picksAt_unique {n : } {Input : Type u_1} {Output : Type u_2} {cands : InputFinset Output} {vp : InputOutputFin n} {σ : Ranking n} {i : Input} {o o' : Output} (h : PicksAt cands vp σ i o) (h' : PicksAt cands vp σ i o') :
                    o = o'

                    A ranking picks at most one output, since strict lex domination is asymmetric.

                    theorem OptimalityTheory.exists_picksAt {n : } {Input : Type u_1} {Output : Type u_2} {cands : InputFinset Output} {vp : InputOutputFin n} {i : Input} (h_ne : (cands i).Nonempty) (h_inj : Set.InjOn (vp i) (cands i)) (σ : Ranking n) :
                    ocands i, PicksAt cands vp σ i o

                    With pairwise-distinct violation profiles, every ranking picks some output — the candidate with the lex-minimal permuted profile wins strictly.

                    @[instance_reducible]
                    instance OptimalityTheory.instDecidablePicksAt {n : } {Input : Type u_1} {Output : Type u_2} [DecidableEq Output] (cands : InputFinset Output) (vp : InputOutputFin n) (σ : Ranking n) (i : Input) (o : Output) :
                    Decidable (PicksAt cands vp σ i o)
                    Equations
                    def OptimalityTheory.winProb {n : } {Input : Type u_1} {Output : Type u_2} [DecidableEq Output] (cands : InputFinset Output) (vp : InputOutputFin n) (r : Fin nFin nProp) [DecidableRel r] (i : Input) (o : Output) :

                    The probability that sampling under grammar r selects output o for input i — the fraction of consistent extensions picking o. The denominator is positive (consistentTotalOrders_card_pos), so this is a genuine probability.

                    Equations
                    Instances For
                      theorem OptimalityTheory.winProb_toRel {n : } {Input : Type u_1} {Output : Type u_2} {cands : InputFinset Output} {vp : InputOutputFin n} {σ : Ranking n} {i : Input} {o : Output} [DecidableEq Output] :
                      winProb cands vp σ.toRel i o = if PicksAt cands vp σ i o then 1 else 0

                      For the σ-induced total order, winProb collapses to a point mass — probability 1 if σ picks o and 0 otherwise.

                      theorem OptimalityTheory.winProb_discrete {n : } {Input : Type u_1} {Output : Type u_2} {cands : InputFinset Output} {vp : InputOutputFin n} {i : Input} {o : Output} [DecidableEq Output] :
                      winProb cands vp (fun (x1 x2 : Fin n) => x1 = x2) i o = {σ : Ranking n | PicksAt cands vp σ i o}.card / Finset.univ.card

                      Under the discrete grammar, winProb is the fraction of all n! rankings picking o.

                      winProb is a probability distribution #

                      theorem OptimalityTheory.winProb_nonneg {n : } {Input : Type u_1} {Output : Type u_2} {cands : InputFinset Output} {vp : InputOutputFin n} {r : Fin nFin nProp} {i : Input} {o : Output} [DecidableEq Output] [DecidableRel r] :
                      0 winProb cands vp r i o
                      theorem OptimalityTheory.winProb_le_one {n : } {Input : Type u_1} {Output : Type u_2} {cands : InputFinset Output} {vp : InputOutputFin n} {r : Fin nFin nProp} {i : Input} {o : Output} [DecidableEq Output] [IsPartialOrder (Fin n) r] [DecidableRel r] :
                      winProb cands vp r i o 1
                      theorem OptimalityTheory.sum_card_filter_picksAt {n : } {Input : Type u_1} {Output : Type u_2} {cands : InputFinset Output} {vp : InputOutputFin n} {r : Fin nFin nProp} {i : Input} [DecidableEq Output] [DecidableRel r] (h_ne : (cands i).Nonempty) (h_inj : Set.InjOn (vp i) (cands i)) :
                      ocands i, {σconsistentTotalOrders r | PicksAt cands vp σ i o}.card = (consistentTotalOrders r).card

                      With pairwise-distinct violation profiles the picks-fibers over the candidate set partition the consistent extensions — the division-free core of sum_winProb_eq_one.

                      theorem OptimalityTheory.sum_winProb_eq_one {n : } {Input : Type u_1} {Output : Type u_2} {cands : InputFinset Output} {vp : InputOutputFin n} {r : Fin nFin nProp} {i : Input} [DecidableEq Output] [IsPartialOrder (Fin n) r] [DecidableRel r] (h_ne : (cands i).Nonempty) (h_inj : Set.InjOn (vp i) (cands i)) :
                      ocands i, winProb cands vp r i o = 1

                      Over a candidate set with pairwise-distinct violation profiles the win probabilities sum to 1, for any grammar — every consistent ranking picks exactly one winner.

                      theorem OptimalityTheory.winProb_binary_add_eq_one {n : } {Input : Type u_1} {Output : Type u_2} {cands : InputFinset Output} {vp : InputOutputFin n} {r : Fin nFin nProp} {i : Input} [DecidableEq Output] [IsPartialOrder (Fin n) r] [DecidableRel r] {o₁ o₂ : Output} (h_two : cands i = {o₁, o₂}) (h_ne : o₁ o₂) (h_vp : vp i o₁ vp i o₂) :
                      winProb cands vp r i o₁ + winProb cands vp r i o₂ = 1

                      Two distinct candidates with distinct violation profiles split the probability mass.

                      Bridge — binary PicksAt is decided by the σ-earliest active constraint #

                      For binary candidate sets cands i = {chosen, other}, PicksAt σ i chosen reduces to lex domination of chosen's permuted profile, which is decided at the first position where the profiles differ — i.e., chosen wins iff the σ-earliest constraint of active vp i chosen other lies in favoring vp i chosen other. Combined with the head-fiber counting of Core.Optimization.PermSubsetCombinatorics, this yields closed-form rates for binary POC competitions without enumerating rankings.

                      theorem OptimalityTheory.picksAt_binary_iff_head_mem_favoring {n : } {Input : Type u_1} {Output : Type u_2} {cands : InputFinset Output} {vp : InputOutputFin n} {i : Input} {chosen other : Output} [DecidableEq Output] (h_two : cands i = {chosen, other}) (h_ne : chosen other) (σ : Ranking n) :
                      PicksAt cands vp σ i chosen xfavoring vp i chosen other, (Core.Optimization.PermSubsetCombinatorics.permDList σ (active vp i chosen other)).head? = some x

                      For binary candidate sets, PicksAt σ i chosen holds exactly when the σ-earliest active constraint favors chosen.

                      Closed-form rate for binary candidates #

                      theorem OptimalityTheory.winProb_discrete_binary_rate {n : } {Input : Type u_1} {Output : Type u_2} {cands : InputFinset Output} {vp : InputOutputFin n} {i : Input} {chosen other : Output} [DecidableEq Output] (h_two : cands i = {chosen, other}) (h_ne : chosen other) :
                      winProb cands vp (fun (x1 x2 : Fin n) => x1 = x2) i chosen = (favoring vp i chosen other active vp i chosen other).card / (active vp i chosen other).card

                      With binary candidates, the fraction of all n! rankings picking chosen is |favoring ∩ active| / |active| — each ranking is decided by its σ-earliest active constraint, and every active constraint is equally likely to come first.

                      Deciding-stratum rate for stratified grammars #

                      A binary competition whose variants tie on every stratum before k is decided within stratum k, with later strata — including any inner rankings among them — provably irrelevant. This is [Ant97]'s tableau-count shortcut stated against the full grammar rather than a per-stratum sub-grammar.

                      theorem OptimalityTheory.winProb_stratified_binary_rate {n s : } {Input : Type u_1} {Output : Type u_2} {cands : InputFinset Output} {vp : InputOutputFin n} {i : Input} {chosen other : Output} [DecidableEq Output] {stratumOf : Fin nFin s} {inner : Fin nFin nProp} [IsPartialOrder (Fin n) inner] [DecidableRel inner] {k : Fin s} (h_two : cands i = {chosen, other}) (h_ne : chosen other) (h_triv : ∀ (a b : Fin n), stratumOf a = kstratumOf b = kinner a ba = b) (h_tie : ∀ (c : Fin n), stratumOf c < kvp i chosen c = vp i other c) (h_dec : {xactive vp i chosen other | stratumOf x = k}.Nonempty) :
                      winProb cands vp (stratified stratumOf inner) i chosen = (favoring vp i chosen other {xactive vp i chosen other | stratumOf x = k}).card / {xactive vp i chosen other | stratumOf x = k}.card

                      Under a stratified grammar, a binary competition whose variants tie on every stratum before k — with k freely ranked internally (h_triv) and containing an active constraint (h_dec) — is won by chosen at rate |favoring ∩ Dₖ| / |Dₖ|, where Dₖ is the active set restricted to stratum k. Later strata cannot affect the outcome.