Docs / Pragmatics / Probabilistic / AlternatingMaximization
The RSA recursion as alternating maximization
import Linglib.Pragmatics.Probabilistic.AlternatingMaximization · namespace RSA
A speaker observes the state through a kernel and addresses a listener, who hears her utterance
and infers the state. The expected utility is minus the information the listener loses, the
expected surprisal of the state under her interpretation, and the expected cost of the
utterance, and the free energy of a speaker and a listener is the conditional entropy of the
utterance given the observation plus the rationality times the expected utility. A listener who
rules out a state the speaker's utterance may come from loses infinitely much, and at a positive
rationality the free energy is then ⊥.
The two agents of the Rational Speech Act recursion are the two partial maximizers of the free
energy. The loss of any listener exceeds that of the Bayesian listener, which is the conditional
entropy of the state given the utterance, by the expected divergence of the Bayesian listener
from her. The free energy of any speaker
falls short of that of the speaker of RSA.speaker at the Bayesian belief by her expected
divergence from it. Going up a level of the
recursion therefore never lowers the free energy, which
rises and
converges along it, and since each speaker
update raises it by at least the divergence of the old speaker from the new, that divergence
tends to zero, which does not by itself make the
speakers converge. Against a fixed listener, a speaker does as well as the speaker at the Bayesian
belief only by being her at every observation
made; when that speaker is the zero measure at such an observation, no utterance being open to
her there, every speaker's free energy is ⊥ and the comparison is empty.
Implementation notes
- The free energy is the functional the source calls the RSA tradeoff. At each observation it is the free energy of the Gibbs variational principle, entropy plus expected log-weight, for the weights the speaker normalizes.
- The expected utility depends on the speaker and her observation kernel only through their composite, the channel from states to utterances, and is stated for a channel.
- The speaker's belief is any family
bthat agrees with the posterior of the prior given her observation at almost every observation,b =ᵐ[κ ∘ₘ μ] κ†μ. The listenerlistener μ κof the observation kernel is one, and so is the point mass for a speaker who observes the state itself (ProbabilityTheory.dirac_ae_eq_posterior_id). - The source states the two decompositions as maximality claims and proves them, by Lagrange multipliers, for a speaker who knows the state. The statements here are for a speaker who observes it through a kernel, and they give the size of the gap.
References
- Zaslavsky et al. 2021, A Rate–Distortion View of Human Pragmatic Reasoning? formalized
- Goodman & Stuhlmüller 2013, Knowledge and Implicature: Modeling Language Understanding as Social Cognition formalized
The speaker at one observation
At one observation, the divergence of a distribution over utterances from the speaker's and its entropy add up to the log of the speaker's normalizer less the rationality times its expected utility.
The expected utility
The listener's step. The information a listener loses exceeds the conditional entropy of the state
given the utterance by the expected divergence of the posterior from her
(InformationTheory.informationLoss_eq_posterior_add), and the Bayesian listener is the posterior
at every utterance heard.
The expected utility of a channel and a listener is minus the information the listener loses and the expected cost of the utterance: the expectation, over states and utterances, of the listener's log-probability of the state less the cost.
- expectedUtility μ η L C = -↑(InformationTheory.informationLoss μ η L) - ↑(∫ (u : U), C u ∂μ.bind ⇑η)
Used by expectedUtility_eq_bot_iff, expectedUtility_of_forall_ne_zero, expectedUtility_le_listener, freeEnergy, freeEnergy_id
Used in XuEtAl2024, ZaslavskyHuLevy2021
The expected utility is ⊥ exactly when the listener rules out a state at which an utterance
she may hear is produced.
Built from
Elsewhere the expected utility is real, the expectation of the listener's log-probability of the state less the cost of the utterance.
Built from
The Bayesian listener is the posterior at every utterance heard.
The Bayesian listener loses the least information.
The Bayesian listener maximizes the expected utility.
Built from
The free energy
The free energy of a speaker and a listener is the conditional entropy of the utterance given
the speaker's observation plus α times the expected utility.
- freeEnergy μ κ C α S L = ↑(∑ o : O, (μ.bind ⇑κ).real {o} * InformationTheory.measureEntropy (S o)) + ↑α * expectedUtility μ (S.comp κ) L C
Built from
Used by freeEnergy_id, freeEnergy_le_freeEnergy_listener, freeEnergy_le, freeEnergy_eq_freeEnergy_speaker_sub, freeEnergy_le_freeEnergy_speaker, freeEnergy_eq_freeEnergy_speaker_iff, monotone_freeEnergy_iterate_recurse, exists_tendsto_freeEnergy_iterate_recurse
Used in ZaslavskyHuLevy2021
Built from ,
Against a fixed speaker her Bayesian listener maximizes the free energy.
Built from
The speaker's step
Where the free energy is not ⊥ it is the real number realFreeEnergy, and the identity for the
speaker at one observation, averaged over her observations, gives the gap.
The free energy is bounded above.
Built from
Against a fixed listener, the free energy of a speaker falls short of that of the speaker at
the Bayesian belief by her expected divergence from it over the observations made. It is ⊥
exactly when she uses, at such an observation, an utterance that speaker never uses.
Built from
Against a fixed listener no speaker has more free energy than the speaker at the Bayesian belief, who is the maximizer whenever she is a probability measure at every observation made.
Built from
A speaker has the free energy of the speaker at the Bayesian belief exactly when she is that speaker at every observation made.
Built from
The recursion
The speaker who answers the listener of a speaker is again a probability kernel, so the two steps alternate along the recursion.
The free energy rises along the recursion.
Built from
The free energy converges along the recursion to a real number, being monotone, bounded above
and nowhere ⊥.
Built from
The expected divergences of each speaker of the recursion from the next have a finite sum: every speaker update raises the free energy by at least that divergence, and the free energy is bounded.
The expected divergence of each speaker of the recursion from the next tends to zero. This does not by itself give convergence of the speakers.