Exact Bayes for the posterior kernel at atoms #
Mathlib characterizes the posterior kernel κ†μ almost everywhere. On discrete spaces an
ae-fact holds at every atom of positive mass, which gives Bayes' rule pointwise at any
positive-mass observation, with no Radon–Nikodym derivative. Drawing a state from the posterior
given an observation of a state drawn from the prior leaves the prior invariant.
Main results #
ProbabilityTheory.posterior_apply_singleton:(κ†μ) x {ω} = μ {ω} * κ ω {x} / (κ ∘ₘ μ) {x}; andcomp_real_mul_posterior_real, the same with the denominator cleared, at every observation.ProbabilityTheory.posterior_deterministic_eq_cond: a deterministic observation's posterior is the prior conditioned on the observation's fibre.ProbabilityTheory.ae_eq_posterior_of_eq_of_ne_zero: when each observation is equally likely at every state that can emit it, the posterior renormalizes the prior over those states.ProbabilityTheory.invariant_posterior_comp,ProbabilityTheory.invariant_comp_posterior,ProbabilityTheory.invariant_id_prod_comp_prodMkLeft_posterior: the chains that alternate the likelihood and the posterior leave the prior, the prior predictive and the joint invariant.ProbabilityTheory.posterior_real_singleton_lt_posterior_of_pair: under priors carried by two atoms, the posterior at an atom is strictly increasing in its prior mass.ProbabilityTheory.integral_le_integral_posterior: conditioning on an observation raises the expectation of a statistic that monovaries with the observation's likelihood on the states of positive prior mass.
Priors carried by two atoms #
The _of_pair lemmas assume the prior's support lies in a pair {ω, ω'}, so that the
observation's marginal has two terms and the comparison of the posterior with the prior at
one atom is the comparison of the two likelihoods.
On a type with two points, an integral against a probability measure weighs the value at the first by its mass and the value at the second by the rest.
The integral of the indicator of a point, as a function of the point integrated, is the mass of the point.
At an observation of positive mass, the posterior mass of a state is given exactly by Bayes' rule.
A deterministic observation's posterior is the prior conditioned on the observation's fibre.
If each observation is equally likely at every state that can emit it, a kernel that weighs the
states that can emit x by w is the posterior under any prior proportional to w, on the
observations the prior predicts.
At every state and observation, the observation marginal times the posterior is the prior times the likelihood, Bayes' rule on reals with the denominator cleared.
Expectations under the posterior #
Bayes' rule reweights the prior by the likelihood, so the posterior expectation of a statistic compares with its prior expectation as the statistic's covariance with the likelihood: Chebyshev's integral inequality.
Conditioning on an observation raises the expectation of a statistic that monovaries with the observation's likelihood on the states of positive prior mass.
Under two priors carried by the same two atoms, at an observation both atoms emit with positive likelihood, the posterior mass of an atom is strictly increasing in its prior mass, so Bayes' rule is prior-sensitive.
Stationarity #
A state drawn from the posterior given an observation of a state drawn from the prior is distributed as the prior. The chains that alternate the likelihood and the posterior therefore leave the prior, the prior predictive and their joint invariant; the last is the Gibbs sampler for the joint.
A kernel equal to the posterior on the observations the prior predicts, composed after the likelihood, leaves the prior invariant.
The Gibbs sampler for the joint law of a state and its observation, which redraws the state from the posterior given the observation and then the observation from the state, leaves the joint invariant.
The posterior of an observation that is the parameter itself is the point mass at it.