Documentation

Linglib.Core.InformationTheory.Hellinger

Hellinger distance on a finite type #

A measure on a finite type gives a vector of Euclidean space, the square roots of the masses of its atoms. The Hellinger distance between two measures is the Euclidean distance between their vectors, divided by √2. The inner product of the vectors is the Bhattacharyya coefficient ∑ a, √(μ {a} ν {a}), so the squared distance of two finite measures is the mean of their total masses less that coefficient, and probability measures lie within distance one of each other, where the Kullback–Leibler divergence may be infinite.

Main definitions #

Main statements #

Implementation notes #

As a distance on Euclidean space the Hellinger distance is symmetric and satisfies the triangle inequality, the L² argument by which it is a metric. The factor 1/√2 bounds the distance between probability measures by one; Polyanskiy and Wu omit it. Mathlib has no Hellinger distance; in general it is an f-divergence of Radon–Nikodym derivatives, of which this is the case of a finite type.

References #

noncomputable def MeasureTheory.Measure.sqrtMass {α : Type u_1} [MeasurableSpace α] (μ : Measure α) :
EuclideanSpace ℝ α

The square roots of the masses of a measure's atoms, as a vector of Euclidean space.

Equations
  • μ.sqrtMass = WithLp.toLp 2 fun (a : α) => √(μ.real {a})
Instances For
    @[simp]
    theorem MeasureTheory.Measure.sqrtMass_apply {α : Type u_1} [MeasurableSpace α] (μ : Measure α) (a : α) :
    μ.sqrtMass.ofLp a = √(μ.real {a})
    theorem MeasureTheory.Measure.inner_sqrtMass_nonneg {α : Type u_1} [MeasurableSpace α] [Fintype α] (μ ν : Measure α) :
    0 ≤ inner ℝ μ.sqrtMass ν.sqrtMass
    theorem MeasureTheory.Measure.norm_sqrtMass_sq {α : Type u_1} [MeasurableSpace α] [Fintype α] [MeasurableSingletonClass α] (μ : Measure α) [IsFiniteMeasure μ] :
    ‖μ.sqrtMass‖ ^ 2 = μ.real Set.univ
    theorem MeasureTheory.Measure.inner_sqrtMass_dirac_left {α : Type u_1} [MeasurableSpace α] [Fintype α] [MeasurableSingletonClass α] (a : α) (ν : Measure α) :
    inner ℝ (dirac a).sqrtMass ν.sqrtMass = √(ν.real {a})
    noncomputable def InformationTheory.hellingerDist {α : Type u_1} [MeasurableSpace α] [Fintype α] (μ ν : MeasureTheory.Measure α) :
    ℝ

    The Hellinger distance √(∑ a, (√(μ {a}) - √(ν {a})) ^ 2) / √2, the Euclidean distance between square-root mass vectors divided by √2.

    Equations
    Instances For
      theorem InformationTheory.hellingerDist_nonneg {α : Type u_1} [MeasurableSpace α] [Fintype α] (μ ν : MeasureTheory.Measure α) :
      0 ≤ hellingerDist μ ν
      theorem InformationTheory.hellingerDist_sq {α : Type u_1} [MeasurableSpace α] [Fintype α] (μ ν : MeasureTheory.Measure α) [MeasurableSingletonClass α] [MeasureTheory.IsFiniteMeasure μ] [MeasureTheory.IsFiniteMeasure ν] :
      hellingerDist μ ν ^ 2 = (μ.real Set.univ + ν.real Set.univ) / 2 - inner ℝ μ.sqrtMass ν.sqrtMass

      The squared Hellinger distance of two finite measures is the mean of their total masses less the Bhattacharyya coefficient.

      theorem InformationTheory.hellingerDist_le_one {α : Type u_1} [MeasurableSpace α] [Fintype α] (μ ν : MeasureTheory.Measure α) [MeasurableSingletonClass α] [MeasureTheory.IsZeroOrProbabilityMeasure μ] [MeasureTheory.IsZeroOrProbabilityMeasure ν] :
      hellingerDist μ ν ≤ 1

      Probability measures, and the zero measure, lie within Hellinger distance one of each other.

      theorem InformationTheory.hellingerDist_dirac_left {α : Type u_1} [MeasurableSpace α] [Fintype α] (ν : MeasureTheory.Measure α) [MeasurableSingletonClass α] [MeasureTheory.IsProbabilityMeasure ν] (a : α) :
      hellingerDist (MeasureTheory.Measure.dirac a) ν = √(1 - √(ν.real {a}))

      The Hellinger distance from a Dirac mass to a probability measure is fixed by the measure's mass at the atom, and is one where that mass vanishes.