Documentation

Studies.SchlotterbeckWang2023

Schlotterbeck & Wang (2023): An incremental RSA model for adjective ordering preferences #

This file formalizes the paper's fully incremental Rational Speech Act model (Table 1) and the qualitative predictions it draws from it for the order of two prenominal adjectives. The incremental literal listener (l0) interprets a word sequence from the noun outward, each word against the support of the distribution the later words left, so a context-dependent meaning such as the k%-semantics of a dimension adjective (kPercent) can resolve differently in the two orders. The incremental sequence speaker produces the words left to right, scoring each prefix by its utility for the referent and each word by a language model (seqSpeaker); the global speaker scores whole sequences, and the incremental utterance speaker is a softmax over the sequence speaker with an utterance prior.

The paper's theoretical result is the listener-level sanity check: at a context-independent semantics the incremental listener is the one-shot literal listener at the product meaning, because Bayesian updates compose, so it cannot distinguish word orders (l0_perm), and the global speaker is then indifferent between them. The speaker-level prediction, the effect of discriminatory strength, comes from the language model that produces either adjective first and then the other: with an order-blind listener the sequence speaker's preference between the two orders is its preference between their first words, and at a uniform prior and Boolean meanings the more informative first word is the one true of fewer referents (seqSpeaker_real_lt_of_card_lt). The k%-semantics witnesses that the listener does distinguish orders once meanings depend on the comparison class (l0_big_blue_ne).

Implementation notes #

References #

Meanings and the language model #

noncomputable def SchlotterbeckWang2023.sharp {U : Type u_1} {R : Type u_2} (ext : U → Finset R → Finset R) :
U → Finset R → R → ENNReal

A Boolean semantics, as graded meanings.

Equations
Instances For
    theorem SchlotterbeckWang2023.sharp_apply_le_one {U : Type u_1} {R : Type u_2} (ext : U → Finset R → Finset R) (u : U) (C : Finset R) (r : R) :
    sharp ext u C r ≤ 1
    def SchlotterbeckWang2023.extSeq {U : Type u_1} {R : Type u_2} [Fintype R] [DecidableEq R] (ext : U → Finset R → Finset R) :
    List U → Finset R

    The sequentially intersective update of a Boolean semantics builds the extension of a sequence from the noun outward.

    Equations
    Instances For
      noncomputable def SchlotterbeckWang2023.alternation {U : Type u_1} [DecidableEq U] (a b : U) :
      List U → U → ENNReal

      The paper's language model puts either adjective first, with equal probability, then the other.

      Equations
      Instances For
        theorem SchlotterbeckWang2023.alternation_comm {U : Type u_1} [DecidableEq U] (a b : U) :
        theorem SchlotterbeckWang2023.alternation_ne_top {U : Type u_1} [DecidableEq U] {a b : U} (us : List U) (u : U) :
        alternation a b us u ≠ ⊤
        noncomputable def SchlotterbeckWang2023.supp {R : Type u_2} [Fintype R] [MeasurableSpace R] (ν : MeasureTheory.Measure R) :
        Finset R

        The support of a distribution over referents, the comparison class the next word is interpreted against.

        Equations
        Instances For
          theorem SchlotterbeckWang2023.mem_supp {R : Type u_2} [Fintype R] [MeasurableSpace R] {ν : MeasureTheory.Measure R} {r : R} :
          r ∈ supp ν ↔ ν {r} ≠ 0

          The incremental listener (rows 1–2) #

          noncomputable def SchlotterbeckWang2023.l0 {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] (μ : MeasureTheory.Measure R) (sem : U → Finset R → R → ENNReal) :
          List U → MeasureTheory.Measure R

          The incremental literal listener of rows (1)–(2) reads a word sequence in surface order, the noun-adjacent word last. That word is interpreted first, against the prior's support, and each earlier word against the support of the distribution the later words left.

          Equations
          Instances For
            theorem SchlotterbeckWang2023.l0_apply_le_one {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] {μ : MeasureTheory.Measure R} {sem : U → Finset R → R → ENNReal} [MeasureTheory.IsProbabilityMeasure μ] (us : List U) (s : Set R) :
            (l0 μ sem us) s ≤ 1

            Utilities and sequence weights (rows 4–5 and 7) #

            noncomputable def SchlotterbeckWang2023.utility {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] (μ : MeasureTheory.Measure R) (sem : U → Finset R → R → ENNReal) (β : ℝ) (cost : ℕ → ENNReal) (us : List U) (r : R) :
            ENNReal

            The utility of row (7) scores the sequence us for the referent r at rationality β, with cost n the cost factor exp (−β c) of an n-word sequence.

            Equations
            Instances For
              noncomputable def SchlotterbeckWang2023.seqWeight {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] (μ : MeasureTheory.Measure R) (sem : U → Finset R → R → ENNReal) (β : ℝ) (cost : ℕ → ENNReal) (pLang : List U → U → ENNReal) (r : R) (us : List U) :
              ENNReal

              Unrolling rows (4)–(5), the weight of producing us word by word scores each prefix by its utility and each word by the language model's probability given the words before it.

              Equations
              Instances For
                theorem SchlotterbeckWang2023.utility_ne_top {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] {μ : MeasureTheory.Measure R} {sem : U → Finset R → R → ENNReal} {β : ℝ} {cost : ℕ → ENNReal} {r : R} [MeasureTheory.IsProbabilityMeasure μ] (hβ : 0 ≤ β) (hcost : ∀ (n : ℕ), cost n ≠ ⊤) (us : List U) :
                utility μ sem β cost us r ≠ ⊤

                Two adjectives #

                theorem SchlotterbeckWang2023.seqWeight_pair_alternation {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] {μ : MeasureTheory.Measure R} {sem : U → Finset R → R → ENNReal} {β : ℝ} {cost : ℕ → ENNReal} [DecidableEq U] (r : R) (a b : U) :
                seqWeight μ sem β cost (alternation a b) r [a, b] = 2⁻¹ * (utility μ sem β cost [a] r * utility μ sem β cost [a, b] r)
                theorem SchlotterbeckWang2023.seqWeight_pair_lt_iff {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] {μ : MeasureTheory.Measure R} {sem : U → Finset R → R → ENNReal} {β : ℝ} {cost : ℕ → ENNReal} [DecidableEq U] (r : R) (a b : U) :
                seqWeight μ sem β cost (alternation a b) r [a, b] < seqWeight μ sem β cost (alternation a b) r [b, a] ↔ utility μ sem β cost [a] r * utility μ sem β cost [a, b] r < utility μ sem β cost [b] r * utility μ sem β cost [b, a] r

                Under the paper's language model the sequence speaker's order preference compares the utilities accumulated along the two production paths.

                theorem SchlotterbeckWang2023.seqWeight_pair_lt_iff_of_l0_eq {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] {μ : MeasureTheory.Measure R} {sem : U → Finset R → R → ENNReal} {β : ℝ} {cost : ℕ → ENNReal} {r : R} {a b : U} [DecidableEq U] [MeasureTheory.IsProbabilityMeasure μ] (hβ : 0 ≤ β) (hcost : ∀ (n : ℕ), cost n ≠ ⊤) (hl : l0 μ sem [a, b] = l0 μ sem [b, a]) (hne : utility μ sem β cost [a, b] r ≠ 0) :
                seqWeight μ sem β cost (alternation a b) r [a, b] < seqWeight μ sem β cost (alternation a b) r [b, a] ↔ utility μ sem β cost [a] r < utility μ sem β cost [b] r

                With an order-blind listener, the sequence speaker's preference between the two orders is its preference between their first words, the discriminatory strength of the first word.

                theorem SchlotterbeckWang2023.utility_single_lt_iff {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] {μ : MeasureTheory.Measure R} {sem : U → Finset R → R → ENNReal} {β : ℝ} {cost : ℕ → ENNReal} {r : R} {a b : U} (hβ : 0 < β) (h0 : cost 1 ≠ 0) (htop : cost 1 ≠ ⊤) :
                utility μ sem β cost [a] r < utility μ sem β cost [b] r ↔ (l0 μ sem [a]) {r} < (l0 μ sem [b]) {r}

                At a positive rationality the first-word utility comparison is the listener comparison.

                Order-blind listeners #

                theorem SchlotterbeckWang2023.l0_contextFree {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] [DiscreteMeasurableSpace R] {μ : MeasureTheory.Measure R} [MeasureTheory.IsProbabilityMeasure μ] (m : U → R → ENNReal) (hm : ∀ (u : U) (r : R), m u r ≤ 1) (us : List U) :
                l0 μ (fun (u : U) (x : Finset R) => m u) us = (μ.withDensity (RSA.prodMeaning m us))[|Set.univ]

                At a context-independent semantics the incremental listener is the one-shot literal listener at the product meaning, since Bayesian updates compose.

                theorem SchlotterbeckWang2023.l0_perm {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] [DiscreteMeasurableSpace R] {μ : MeasureTheory.Measure R} [MeasureTheory.IsProbabilityMeasure μ] (m : U → R → ENNReal) (hm : ∀ (u : U) (r : R), m u r ≤ 1) {us us' : List U} (h : us.Perm us') :
                l0 μ (fun (u : U) (x : Finset R) => m u) us = l0 μ (fun (u : U) (x : Finset R) => m u) us'

                A context-independent semantics cannot distinguish word orders, the paper's sanity check.

                theorem SchlotterbeckWang2023.supp_uniformOn {R : Type u_2} [Fintype R] [MeasurableSpace R] [DiscreteMeasurableSpace R] (S : Finset R) :
                supp (ProbabilityTheory.uniformOn ↑S) = S
                theorem SchlotterbeckWang2023.l0_indicator {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] [DiscreteMeasurableSpace R] [DecidableEq R] (ext : U → Finset R → Finset R) (us : List U) :
                l0 (ProbabilityTheory.uniformOn Set.univ) (sharp ext) us = ProbabilityTheory.uniformOn ↑(extSeq ext us)

                At a uniform prior and a Boolean semantics the incremental listener is uniform on the sequentially intersected extension.

                theorem SchlotterbeckWang2023.l0_single_lt_iff {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] [DiscreteMeasurableSpace R] {r : R} {a b : U} [DecidableEq R] (ext : U → Finset R → Finset R) (ha : r ∈ ext a Finset.univ) (hb : r ∈ ext b Finset.univ) :
                (l0 (ProbabilityTheory.uniformOn Set.univ) (sharp ext) [a]) {r} < (l0 (ProbabilityTheory.uniformOn Set.univ) (sharp ext) [b]) {r} ↔ (ext b Finset.univ).card < (ext a Finset.univ).card

                At a uniform prior and Boolean meanings, of two words true of the referent the one true of fewer referents is the more informative first word.

                The speaker kernels (rows 3–6) #

                noncomputable def SchlotterbeckWang2023.seqSpeaker {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] [DiscreteMeasurableSpace R] (μ : MeasureTheory.Measure R) (sem : U → Finset R → R → ENNReal) (β : ℝ) (cost : ℕ → ENNReal) (pLang : List U → U → ENNReal) (n : ℕ) :
                ProbabilityTheory.Kernel R (Fin n → U)

                The incremental sequence speaker of rows (4)–(5) chooses among sequences of length n.

                Equations
                Instances For
                  noncomputable def SchlotterbeckWang2023.globalSpeaker {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] [DiscreteMeasurableSpace R] (μ : MeasureTheory.Measure R) (sem : U → Finset R → R → ENNReal) (β : ℝ) (cost : ℕ → ENNReal) {n : ℕ} (P : (Fin n → U) → ENNReal) :
                  ProbabilityTheory.Kernel R (Fin n → U)

                  The global speaker of row (3) chooses among sequences of length n with utterance prior P.

                  Equations
                  Instances For
                    noncomputable def SchlotterbeckWang2023.uttSpeaker {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] [DiscreteMeasurableSpace R] (μ : MeasureTheory.Measure R) (sem : U → Finset R → R → ENNReal) (β : ℝ) (cost : ℕ → ENNReal) (pLang : List U → U → ENNReal) (α : ℝ) {n : ℕ} (P : (Fin n → U) → ENNReal) :
                    ProbabilityTheory.Kernel R (Fin n → U)

                    The incremental utterance speaker of row (6) is a softmax at rationality α over the sequence speaker with utterance prior P.

                    Equations
                    • One or more equations did not get rendered due to their size.
                    Instances For
                      theorem SchlotterbeckWang2023.seqSpeaker_real_lt_iff {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] [DiscreteMeasurableSpace R] {μ : MeasureTheory.Measure R} {sem : U → Finset R → R → ENNReal} {β : ℝ} {cost : ℕ → ENNReal} {pLang : List U → U → ENNReal} {r : R} [MeasureTheory.IsProbabilityMeasure μ] (hβ : 0 ≤ β) (hcost : ∀ (n : ℕ), cost n ≠ ⊤) (hp : ∀ (us : List U) (u : U), pLang us u ≠ ⊤) {n : ℕ} (h0 : ∃ (v : Fin n → U), seqWeight μ sem β cost pLang r (List.ofFn v) ≠ 0) {v v' : Fin n → U} :
                      ((seqSpeaker μ sem β cost pLang n) r).real {v} < ((seqSpeaker μ sem β cost pLang n) r).real {v'} ↔ seqWeight μ sem β cost pLang r (List.ofFn v) < seqWeight μ sem β cost pLang r (List.ofFn v')

                      An order preference of the sequence speaker is a weight comparison; normalization cancels.

                      theorem SchlotterbeckWang2023.globalSpeaker_apply_singleton_eq {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] [DiscreteMeasurableSpace R] {μ : MeasureTheory.Measure R} {sem : U → Finset R → R → ENNReal} {β : ℝ} {cost : ℕ → ENNReal} {r : R} {n : ℕ} {P : (Fin n → U) → ENNReal} {v v' : Fin n → U} (hl : l0 μ sem (List.ofFn v) = l0 μ sem (List.ofFn v')) (hP : P v = P v') :
                      ((globalSpeaker μ sem β cost P) r) {v} = ((globalSpeaker μ sem β cost P) r) {v'}

                      A global speaker (row (3)) cannot prefer an order the listener cannot distinguish.

                      theorem SchlotterbeckWang2023.uttSpeaker_real_lt_iff {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] [DiscreteMeasurableSpace R] {μ : MeasureTheory.Measure R} {sem : U → Finset R → R → ENNReal} {β : ℝ} {cost : ℕ → ENNReal} {pLang : List U → U → ENNReal} {r : R} [MeasureTheory.IsProbabilityMeasure μ] (hβ : 0 ≤ β) (hcost : ∀ (n : ℕ), cost n ≠ ⊤) (hp : ∀ (us : List U) (u : U), pLang us u ≠ ⊤) {α : ℝ} (hα : 0 ≤ α) {n : ℕ} {P : (Fin n → U) → ENNReal} (hP : ∀ (v : Fin n → U), P v ≠ ⊤) (h0 : ∃ (v : Fin n → U), seqWeight μ sem β cost pLang r (List.ofFn v) ^ α * P v ≠ 0) {v v' : Fin n → U} :
                      ((uttSpeaker μ sem β cost pLang α P) r).real {v} < ((uttSpeaker μ sem β cost pLang α P) r).real {v'} ↔ seqWeight μ sem β cost pLang r (List.ofFn v) ^ α * P v < seqWeight μ sem β cost pLang r (List.ofFn v') ^ α * P v'

                      The order preference of the utterance speaker (row (6)) is the sequence speaker's, weighed against the utterance prior.

                      theorem SchlotterbeckWang2023.seqSpeaker_real_lt_of_card_lt {U : Type u_1} {R : Type u_2} [Fintype U] [MeasurableSpace U] [DiscreteMeasurableSpace U] [Fintype R] [MeasurableSpace R] [DiscreteMeasurableSpace R] {β : ℝ} {cost : ℕ → ENNReal} {r : R} {a b : U} [DecidableEq U] [DecidableEq R] (hβ : 0 < β) (hcost0 : ∀ (n : ℕ), cost n ≠ 0) (hcost : ∀ (n : ℕ), cost n ≠ ⊤) (ext : U → Finset R) (ha : r ∈ ext a) (hb : r ∈ ext b) (hcard : (ext a).card < (ext b).card) :
                      ((seqSpeaker (ProbabilityTheory.uniformOn Set.univ) (sharp fun (u : U) (x : Finset R) => ext u) β cost (alternation a b) 2) r).real {![b, a]} < ((seqSpeaker (ProbabilityTheory.uniformOn Set.univ) (sharp fun (u : U) (x : Finset R) => ext u) β cost (alternation a b) 2) r).real {![a, b]}

                      At a uniform prior, sharp context-independent meanings and the paper's language model, the sequence speaker prefers first the adjective true of fewer referents, the relevance effect.

                      The k%-semantics (3a) distinguishes the orders #

                      def SchlotterbeckWang2023.kPercent {R : Type u_2} [Fintype R] (k : ℕ) (size : R → ℕ) (C : Finset R) :
                      Finset R

                      Under the k%-semantics (3a) of a dimension adjective against the comparison class C, an object is big when its size exceeds the maximum of C less k% of the range of C.

                      Equations
                      • SchlotterbeckWang2023.kPercent k size C = {x : R | ∃ (hC : C.Nonempty), 100 * C.sup' hC size < 100 * size x + k * (C.sup' hC size - C.inf' hC size)}
                      Instances For

                        The display has three blue stickers of sizes 6, 5 and 1, and a green one of size 10.

                        Instances For
                          @[instance_reducible]
                          Equations
                          @[instance_reducible]
                          Equations
                          • One or more equations did not get rendered due to their size.

                          The two adjectives of the paper's stimuli.

                          Instances For
                            @[instance_reducible]
                            Equations
                            @[instance_reducible]
                            Equations
                            • One or more equations did not get rendered due to their size.
                            theorem SchlotterbeckWang2023.l0_big_blue_ne :
                            l0 (ProbabilityTheory.uniformOn Set.univ) (sharp Adj.ext) [Adj.big, Adj.blue] ≠ l0 (ProbabilityTheory.uniformOn Set.univ) (sharp Adj.ext) [Adj.blue, Adj.big]

                            Big blue and blue big resolve differently, since big interpreted against the blue stickers admits the second-largest one, which it excludes against the whole display.