Error-driven learning

A learner holds a matrix W of association weights from cues to outcomes and meets learning events, each a cue vector x paired with an outcome vector y. The weights predict the outcome x ᵥ* W, and the error-driven update moves the weight from cue i to outcome j by the cue's value times the outcome's prediction error, scaled by a rate α i for the cue and a rate β j for the outcome. On indicator vectors this is the rule of Rescorla and Wagner, where α is the salience of a cue and β the importance of an outcome. With a single rate it is the rule of Widrow and Hoff for real-valued vectors.

The cues of an event share its prediction error. An event that the weights already predict therefore changes nothing, and repeating an event shrinks its error geometrically while dividing the weight gained among the cues in proportion to their rates.

Main definitions

• ErrorDrivenLearning.update: the weights after one learning event.

Main results

• ErrorDrivenLearning.update_eq_self_iff: an event leaves the weights fixed exactly when none of its cues is effective or none of its outcomes is mispredicted.
• ErrorDrivenLearning.iterate_update, ErrorDrivenLearning.tendsto_iterate_update_apply: the weights after n repetitions of an event, and their limit.

References

• M. Heitmeier, Y.-Y. Chuang and R. H. Baayen, The Discriminative Lexicon (2026)
• R. A. Rescorla and A. R. Wagner, A theory of Pavlovian conditioning (1972)
• B. Widrow and M. Hoff, Adaptive switching circuits (1960)

noncomputable def ErrorDrivenLearning.update {ι : Type u_1} {κ : Type u_2} [Fintype ι] (α : ι → ℝ) (β : κ → ℝ) (x : ι → ℝ) (y : κ → ℝ) (W : Matrix ι κ ℝ) : Matrix ι κ ℝ

The error-driven update of the weights W on a learning event with cues x and outcomes y adds α i * x i * ((y j - (x ᵥ* W) j) * β j) to the weight from cue i to outcome j, the cue's rate and value times the outcome's prediction error and rate.

Equations

• ErrorDrivenLearning.update α β x y W = W + Matrix.vecMulVec (α * x) ((y - Matrix.vecMul x W) * β)

theorem ErrorDrivenLearning.update_apply {ι : Type u_1} {κ : Type u_2} [Fintype ι] (α : ι → ℝ) (β : κ → ℝ) (x : ι → ℝ) (y : κ → ℝ) (W : Matrix ι κ ℝ) (i : ι) (j : κ) : update α β x y W i j = W i j + α i * x i * ((y j - Matrix.vecMul x W j) * β j)

theorem ErrorDrivenLearning.update_smul_one {ι : Type u_1} {κ : Type u_2} [Fintype ι] (x : ι → ℝ) (y : κ → ℝ) (W : Matrix ι κ ℝ) (η : ℝ) : update (η • 1) 1 x y W = W + η • Matrix.vecMulVec x (y - Matrix.vecMul x W)

With one rate η for every cue and outcome the update is the Widrow–Hoff rule.

theorem ErrorDrivenLearning.update_smul_one_smul_one {ι : Type u_1} {κ : Type u_2} [Fintype ι] (x : ι → ℝ) (y : κ → ℝ) (W : Matrix ι κ ℝ) (a b : ℝ) : update (a • 1) (b • 1) x y W = update ((a * b) • 1) 1 x y W

A uniform cue rate a and a uniform outcome rate b act as the single rate a * b.

theorem ErrorDrivenLearning.update_indicator_apply {ι : Type u_1} {κ : Type u_2} [Fintype ι] (α : ι → ℝ) (β : κ → ℝ) (W : Matrix ι κ ℝ) (s : Finset ι) (t : Set κ) (lam : κ → ℝ) (i : ι) (j : κ) : update α β ((↑s).indicator 1) (t.indicator lam) W i j = W i j + (↑s).indicator α i * β j * (t.indicator lam j - ∑ i' ∈ s, W i' j)

On a set s of present cues and a set t of present outcomes with maximal activations lam the update is the Rescorla–Wagner rule. A present cue gains α i * β j times the error of outcome j against the summed weights of the present cues, and an absent cue gains nothing.

One event

theorem ErrorDrivenLearning.vecMul_update {ι : Type u_1} {κ : Type u_2} [Fintype ι] (α : ι → ℝ) (β : κ → ℝ) (x : ι → ℝ) (y : κ → ℝ) (W : Matrix ι κ ℝ) (x' : ι → ℝ) : Matrix.vecMul x' (update α β x y W) = Matrix.vecMul x' W + (x' ⬝ᵥ (α * x)) • ((y - Matrix.vecMul x W) * β)

theorem ErrorDrivenLearning.sub_vecMul_update {ι : Type u_1} {κ : Type u_2} [Fintype ι] (α : ι → ℝ) (β : κ → ℝ) (x : ι → ℝ) (y : κ → ℝ) (W : Matrix ι κ ℝ) : y - Matrix.vecMul x (update α β x y W) = (y - Matrix.vecMul x W) * (1 - (x ⬝ᵥ (α * x)) • β)

An event scales its own prediction error for outcome j by 1 - x ⬝ᵥ (α * x) * β j.

theorem ErrorDrivenLearning.update_eq_self_iff {ι : Type u_1} {κ : Type u_2} [Fintype ι] (α : ι → ℝ) (β : κ → ℝ) (x : ι → ℝ) (y : κ → ℝ) (W : Matrix ι κ ℝ) : update α β x y W = W ↔ α * x = 0 ∨ (y - Matrix.vecMul x W) * β = 0

An event leaves the weights fixed exactly when none of its cues is effective or none of its outcomes is mispredicted.

theorem ErrorDrivenLearning.update_eq_self_of_vecMul_eq {ι : Type u_1} {κ : Type u_2} [Fintype ι] (α : ι → ℝ) (β : κ → ℝ) {x : ι → ℝ} {y : κ → ℝ} {W : Matrix ι κ ℝ} (h : Matrix.vecMul x W = y) : update α β x y W = W

An event whose outcome the weights already predict teaches nothing.

Repeating an event

theorem ErrorDrivenLearning.sub_vecMul_iterate_update {ι : Type u_1} {κ : Type u_2} [Fintype ι] (α : ι → ℝ) (β : κ → ℝ) (x : ι → ℝ) (y : κ → ℝ) (W : Matrix ι κ ℝ) (n : ℕ) : y - Matrix.vecMul x ((update α β x y)^[n] W) = (y - Matrix.vecMul x W) * (1 - (x ⬝ᵥ (α * x)) • β) ^ n

theorem ErrorDrivenLearning.iterate_update {ι : Type u_1} {κ : Type u_2} [Fintype ι] (α : ι → ℝ) (β : κ → ℝ) (x : ι → ℝ) (y : κ → ℝ) (W : Matrix ι κ ℝ) (n : ℕ) : (update α β x y)^[n] W = W + Matrix.vecMulVec (α * x) ((y - Matrix.vecMul x W) * β * ∑ k ∈ Finset.range n, (1 - (x ⬝ᵥ (α * x)) • β) ^ k)

After n repetitions of an event every cue has gained in proportion to its rate and value.

theorem ErrorDrivenLearning.iterate_update_apply {ι : Type u_1} {κ : Type u_2} [Fintype ι] (α : ι → ℝ) (β : κ → ℝ) (x : ι → ℝ) (y : κ → ℝ) (W : Matrix ι κ ℝ) (n : ℕ) (i : ι) (j : κ) : (update α β x y)^[n] W i j = W i j + α i * x i * ((y j - Matrix.vecMul x W j) * β j * ∑ k ∈ Finset.range n, (1 - x ⬝ᵥ (α * x) * β j) ^ k)

theorem ErrorDrivenLearning.tendsto_vecMul_iterate_update {ι : Type u_1} {κ : Type u_2} [Fintype ι] {α : ι → ℝ} {β : κ → ℝ} {x : ι → ℝ} (y : κ → ℝ) (W : Matrix ι κ ℝ) (h : ∀ (j : κ), |1 - x ⬝ᵥ (α * x) * β j| < 1) : Filter.Tendsto (fun (n : ℕ) => Matrix.vecMul x ((update α β x y)^[n] W)) Filter.atTop (nhds y)

Under repetition the prediction converges to the outcome, provided each outcome's error factor is less than one in absolute value.

theorem ErrorDrivenLearning.tendsto_iterate_update_apply {ι : Type u_1} {κ : Type u_2} [Fintype ι] {α : ι → ℝ} {β : κ → ℝ} {x : ι → ℝ} (y : κ → ℝ) (W : Matrix ι κ ℝ) (i : ι) {j : κ} (h : |1 - x ⬝ᵥ (α * x) * β j| < 1) : Filter.Tendsto (fun (n : ℕ) => (update α β x y)^[n] W i j) Filter.atTop (nhds (W i j + α i * x i / x ⬝ᵥ (α * x) * (y j - Matrix.vecMul x W j)))

Under repetition the weight from cue i to outcome j converges, the cue having gained the share α i * x i / x ⬝ᵥ (α * x) of the outcome's initial prediction error.