Docs / Studies / Studies.KirbyDowmanGriffiths2007

Innateness and culture in the evolution of language

Kirby, S., Dowman, M. & Griffiths, T. L. (2007). Proceedings of the National Academy of Sciences, 104(12), 5241–5245

A learner who adopts language $h$ with probability proportional to $[P(d \mid h)\,P(h)]^r$ keeps the prior stationary at $r = 1$. If every datum is impossible or equally likely under each language, it is the posterior for the prior proportional to $P(h)^r$, which it therefore keeps stationary, raising the prior odds to the power $r$, while MAP learners keep the uniform distribution on the prior's modes. In the paper's model without noise those are the fully regular languages, however weak the bias; noise meets the condition only at two degenerate levels.

Implementation notes

  • exponentiatedPosterior is the paper's "exponentiated posterior distribution". The paper names neither the normalized P(h)^r nor anything else here; exponentiatedPrior is our name, by analogy.
  • exponentiatedPosterior gives the zero measure at a datum that no language of positive prior mass produces (exponentiatedPosterior_apply_eq_zero), so it is not a Markov kernel; the theorems use it only on the data the prior predicts. mapLearner ties every language there instead (GriffithsKalish2007.mapLearner_apply).
  • The condition that P(d|h) be "either constant or zero across all hypotheses" is ConstantOrZero, stated without the constant: two languages that both give a datum positive probability give it the same.
  • The exponent is positive: at r = 0 the extended-real 0 ^ 0 = 1 would give positive weight to languages that cannot produce the datum.
  • The paper assumes the chain on languages is ergodic. Invariance does not need it; uniqueness of the stationary distribution does.
  • The distribution of meanings P is a parameter, uniform in the paper's Fig. 3 and Zipfian in its Fig. 4. Both figures are simulations and are not formalized; Fig. 3's α and ε are not reported.

TODO

References

Declarations

L62–66

The exponentiated posterior

L74–78
L80–82
L84–85
L87–91
L130–133
L135–140

p. 5244, "systematically overrepresented": the stationary distribution proportional to P(h)^r raises the prior odds between any two languages to the power r.

theorem exponentiatedPrior_real_divL142–153
{H : Type u_1} [Fintype H] [MeasurableSpace H] [MeasurableSingletonClass H] (μ : MeasureTheory.Measure H) [MeasureTheory.IsProbabilityMeasure μ] {r : ℝ} (hr : 0 < r) (h h' : H) :
(exponentiatedPrior μ r).real {h'} / (exponentiatedPrior μ r).real {h} = (μ.real {h'} / μ.real {h}) ^ r

Built from

L156–162
L169–173

Methods, eqs. (6)–(10): if every datum is either impossible or equally likely under each language, iterated learning with the exponentiated posterior has a stationary distribution proportional to P(h)^r. The learner is the posterior under that distribution, so the result is Griffiths & Kalish 2007's at a different prior.

theorem invariant_exponentiatedPosteriorL175–195
{H : Type u_1} {D : Type u_2} [Fintype H] [MeasurableSpace H] [MeasurableSpace D] [MeasurableSingletonClass H] [Countable D] [MeasurableSingletonClass D] [Nonempty H] (κ : ProbabilityTheory.Kernel H D) [ProbabilityTheory.IsMarkovKernel κ] (μ : MeasureTheory.Measure H) [MeasureTheory.IsProbabilityMeasure μ] (hκ : ConstantOrZero κ) {r : ℝ} (hr : 0 < r) :
((exponentiatedPosterior κ μ r).comp κ).Invariant (exponentiatedPrior μ r)

Built from , ,

L199–225

The r = ∞ endpoint: if every datum is either impossible or equally likely under each language, iterated learning by MAP estimation has the uniform distribution on the modes of the prior as a stationary distribution. Two priors with the same modes, one weak and one strong, give the same one.

theorem invariant_mapLearnerL227–234
{H : Type u_1} {D : Type u_2} [Fintype H] [MeasurableSpace H] [MeasurableSpace D] [MeasurableSingletonClass H] [Countable D] [MeasurableSingletonClass D] [Nonempty H] (κ : ProbabilityTheory.Kernel H D) [ProbabilityTheory.IsMarkovKernel κ] (μ : MeasureTheory.Measure H) [MeasureTheory.IsProbabilityMeasure μ] (hκ : ConstantOrZero κ) :
((GriffithsKalish2007.mapLearner κ μ).comp κ).Invariant (ProbabilityTheory.uniformOn ↑(Finset.univ.argmax fun (h : H) => μ {h}))

Built from

The meaning-class model

A language assigns each of n meanings one of k classes. A learner sees m pairs of a meaning, drawn from P, and the class produced for it, which is the language's class with probability 1 - ε and each other class with probability ε / (k - 1).

L253–255
L263–268
L285–298
L301–311
L340–342

Methods: the noisy meaning-class model meets the condition of invariant_exponentiatedPosterior exactly when there is no noise, when every class is produced with probability 1/k, or when the language's own class is never produced.

theorem constantOrZero_produce_iffL344–370
{n k m : ℕ} (P : MeasureTheory.Measure (Fin n)) [MeasureTheory.IsProbabilityMeasure P] {ε : ℝ} (hk : 2 ≤ k) (hm : m ≠ 0) (h₀ : 0 ≤ ε) (h₁ : ε ≤ 1) :
ConstantOrZero (produce P ε) ↔ ε = 0 ∨ ε = (↑k - 1) / ↑k ∨ ε = 1

Built from ,

L374–376

The meaning-class model without noise: for every bias strength α, iterated learning by MAP estimation has the uniform distribution on the fully regular languages as a stationary distribution. Sampling learners keep the prior instead, which depends on α (invariant_exponentiatedPosterior_one).

theorem invariant_mapLearner_produce_zeroL385–398
{n k m : ℕ} (P : MeasureTheory.Measure (Fin n)) [MeasureTheory.IsProbabilityMeasure P] (hk : 2 ≤ k) {α : ℝ} (hα : 0 < α) :
((GriffithsKalish2007.mapLearner (produce P 0) (prior α)).comp (produce P 0)).Invariant (ProbabilityTheory.uniformOn (Set.range fun (j : Fin k) (x : Fin n) => j))

Built from ,

L402–434