Signal detection theory

Signal detection theory, in Macmillan and Creelman's presentation, accounts for a yes-no judgment as a criterion on a noisy decision variable. A trial holds a signal or only noise, the observer receives an observation whose law depends on which, and it responds "yes" when the observation exceeds a criterion k. The experiment is a kernel P : Kernel Bool ℝ from the state of the trial to the law of the observation, and the observer's response rates are the masses its two laws give to (k, ∞): the hit rate on signal trials and the false-alarm rate on noise trials. In a forced-choice task one interval holds the signal and the rest noise, and the observer picks the interval with the largest observation, so the proportion correct is the choice probability of the signal interval in the random utility model of the intervals' laws (forcedChoice). It is the probability that every noise observation falls below the signal observation, which for two intervals is the area under the yes-no receiver operating characteristic, Green and Swets's area theorem (forcedChoice_eq_lintegral).

The models Macmillan and Creelman compare are location experiments (locationExperiment): a noise law ν shifted by δ / 2 on signal trials and by -(δ / 2) on noise trials, the sensitivity δ being the distance between the two laws. The normal model takes ν standard normal, and Luce's Choice Theory takes it logistic (Macmillan and Creelman, ch. 4, p. 98). In every location experiment the hit rate of a criterion is the false-alarm rate of the criterion lower by δ (hitRate_eq_falseAlarmRate_sub). For a symmetric ν, such as both of these, the receiver operating characteristic on axes transformed by the quantile function of ν is therefore a line of unit slope, δ above the chance line. In the normal model the axes are z-scores: whatever the criterion k, the sensitivity is d' = z(H) - z(F) and k = -(z(H) + z(F)) / 2 (probit_hitRate_sub_probit_falseAlarmRate, probit_hitRate_add_probit_falseAlarmRate). The likelihood ratio of signal to noise at an observation x is exp (d' * x) (likelihoodRatio_gaussianReal), and the two-alternative forced choice is the binary probit rule, with proportion correct Φ (d' / √2) (forcedChoice_one_gaussianReal).

Main definitions

• SignalDetection.hitRate, SignalDetection.falseAlarmRate: the response rates of a criterion.
• SignalDetection.forcedChoice: the proportion correct of a forced-choice task.
• SignalDetection.locationExperiment: the experiment of a noise law at a sensitivity.

Main results

• SignalDetection.forcedChoice_eq_lintegral: the forced-choice proportion correct as an integral over the signal observation, Green and Swets's eq. (2.13).
• SignalDetection.hitRate_eq_falseAlarmRate_sub: in a location experiment the hit rate of k is the false-alarm rate of k - δ.
• SignalDetection.falseAlarmRate_lt_hitRate: at positive sensitivity the receiver operating characteristic lies above the chance line.
• SignalDetection.probit_hitRate_sub_probit_falseAlarmRate: in the normal model z(H) = z(F) + d', Macmillan and Creelman's eq. (1.8).
• SignalDetection.probit_hitRate_add_probit_falseAlarmRate: in the normal model the criterion is c = -(z(H) + z(F)) / 2, Macmillan and Creelman's eq. (2.1).
• SignalDetection.likelihoodRatio_gaussianReal: the likelihood ratio of the normal model.
• SignalDetection.forcedChoice_one_gaussianReal: in the normal model the two-alternative proportion correct is Φ (d' / √2), Macmillan and Creelman's eq. (7.6).
• SignalDetection.logit_hitRate_sub_logit_falseAlarmRate, SignalDetection.logit_hitRate_add_logit_falseAlarmRate: in the logistic model the sensitivity and the criterion are the difference and minus the mean of the log-odds of the rates, Macmillan and Creelman's eqs. (4.10) and (4.12).

Implementation notes

The forced-choice observer picks the interval with the largest observation, where Green and Swets (p. 46) pick the interval with the largest likelihood ratio. The two rules agree whenever the likelihood ratio increases with the observation, as it does in the normal model (likelihoodRatio_gaussianReal). Page numbers of Green and Swets are those of the 1988 reprint.

TODO

• The slope of the receiver operating characteristic at a criterion is the likelihood ratio there (Macmillan and Creelman, ch. 2); for the normal model this needs the derivative of Φ.
• The optimal criterion is the Bayes estimator of the experiment under 0-1 loss, and a larger sensitivity gives a more informative experiment in the order of Linglib/Core/Probability/Decision/Blackwell.lean.
• Luce's axiom-1 forced choice (section 2.E) is a logistic Yes-No model at twice the sensitivity, which is not the forced choice of the logistic model: the difference of two logistic observations is not logistic.

References

• [MC05]
• [GS66]
• [Luc59]

Response rates and forced choice

noncomputable def SignalDetection.hitRate (P : ProbabilityTheory.Kernel Bool ℝ) (k : ℝ) : ℝ

The hit rate of the criterion k: the probability that the observation on a signal trial exceeds k, so that the observer responds "yes".

Equations

• SignalDetection.hitRate P k = (P true).real (Set.Ioi k)

noncomputable def SignalDetection.falseAlarmRate (P : ProbabilityTheory.Kernel Bool ℝ) (k : ℝ) : ℝ

The false-alarm rate of the criterion k: the probability that the observation on a noise trial exceeds k.

Equations

• SignalDetection.falseAlarmRate P k = (P false).real (Set.Ioi k)

noncomputable def SignalDetection.forcedChoice (P : ProbabilityTheory.Kernel Bool ℝ) (n : ℕ) : ENNReal

The proportion correct of the forced-choice task with one signal interval and n noise intervals, in which the observer picks the interval with the largest observation: the choice probability of the signal interval in the random utility model of the intervals' laws.

Equations

• SignalDetection.forcedChoice P n = ProbabilityTheory.choiceProb (MeasureTheory.Measure.pi fun (j : Fin (n + 1)) => P (decide (j = 0))) 0

theorem SignalDetection.forcedChoice_eq_lintegral (P : ProbabilityTheory.Kernel Bool ℝ) [ProbabilityTheory.IsFiniteKernel P] (n : ℕ) : forcedChoice P n = ∫⁻ (k : ℝ), (P false) (Set.Iio k) ^ n ∂P true

The forced-choice proportion correct is the probability that all n noise observations fall below the signal observation k, integrated over k ([GS66], eqs. (2.7) and (2.13), p. 47). With one noise interval it is the area under the yes-no receiver operating characteristic, the hit rate integrated over the false-alarm rate.

Location experiments

noncomputable def SignalDetection.locationExperiment (ν : MeasureTheory.Measure ℝ) (δ : ℝ) : ProbabilityTheory.Kernel Bool ℝ

The location experiment of a noise law ν at sensitivity δ: the observation is ν shifted by δ / 2 on signal trials and by -(δ / 2) on noise trials.

Equations

• SignalDetection.locationExperiment ν δ = ProbabilityTheory.Kernel.ofFunOfCountable fun (b : Bool) => MeasureTheory.Measure.map (fun (x : ℝ) => x + if b = true then δ / 2 else -(δ / 2)) ν

theorem SignalDetection.locationExperiment_apply (ν : MeasureTheory.Measure ℝ) (δ : ℝ) (b : Bool) : (locationExperiment ν δ) b = MeasureTheory.Measure.map (fun (x : ℝ) => x + if b = true then δ / 2 else -(δ / 2)) ν

theorem SignalDetection.hitRate_locationExperiment {ν : MeasureTheory.Measure ℝ} {δ : ℝ} (k : ℝ) : hitRate (locationExperiment ν δ) k = ν.real (Set.Ioi (k - δ / 2))

theorem SignalDetection.falseAlarmRate_locationExperiment {ν : MeasureTheory.Measure ℝ} {δ : ℝ} (k : ℝ) : falseAlarmRate (locationExperiment ν δ) k = ν.real (Set.Ioi (k + δ / 2))

theorem SignalDetection.hitRate_eq_falseAlarmRate_sub {ν : MeasureTheory.Measure ℝ} {δ : ℝ} (k : ℝ) : hitRate (locationExperiment ν δ) k = falseAlarmRate (locationExperiment ν δ) (k - δ)

In a location experiment the hit rate of a criterion is the false-alarm rate of the criterion lower by the sensitivity, so that on axes transformed by the inverse of the survival function of ν the receiver operating characteristic is the chance line shifted by δ.

theorem SignalDetection.hitRate_locationExperiment_zero {ν : MeasureTheory.Measure ℝ} (k : ℝ) : hitRate (locationExperiment ν 0) k = falseAlarmRate (locationExperiment ν 0) k

At zero sensitivity the hit rate is the false-alarm rate: the receiver operating characteristic is the chance line.

theorem SignalDetection.falseAlarmRate_le_hitRate {ν : MeasureTheory.Measure ℝ} {δ : ℝ} [MeasureTheory.IsFiniteMeasure ν] (hδ : 0 ≤ δ) (k : ℝ) : falseAlarmRate (locationExperiment ν δ) k ≤ hitRate (locationExperiment ν δ) k

At nonnegative sensitivity the false-alarm rate is at most the hit rate.

theorem SignalDetection.falseAlarmRate_lt_hitRate {ν : MeasureTheory.Measure ℝ} {δ : ℝ} [MeasureTheory.IsProbabilityMeasure ν] [ν.IsOpenPosMeasure] (hδ : 0 < δ) (k : ℝ) : falseAlarmRate (locationExperiment ν δ) k < hitRate (locationExperiment ν δ) k

At positive sensitivity the false-alarm rate is below the hit rate when ν charges every nonempty open set: the receiver operating characteristic lies above the chance line.

The normal model

theorem SignalDetection.locationExperiment_gaussianReal (δ : ℝ) (b : Bool) : (locationExperiment (ProbabilityTheory.gaussianReal 0 1) δ) b = ProbabilityTheory.gaussianReal (if b = true then δ / 2 else -(δ / 2)) 1

In the normal model the observation is N(δ / 2, 1) on signal trials and N(-(δ / 2), 1) on noise trials.

theorem SignalDetection.hitRate_gaussianReal (δ k : ℝ) : hitRate (locationExperiment (ProbabilityTheory.gaussianReal 0 1) δ) k = ProbabilityTheory.normalCDF (δ / 2 - k)

theorem SignalDetection.falseAlarmRate_gaussianReal (δ k : ℝ) : falseAlarmRate (locationExperiment (ProbabilityTheory.gaussianReal 0 1) δ) k = ProbabilityTheory.normalCDF (-(δ / 2) - k)

theorem SignalDetection.probit_hitRate_sub_probit_falseAlarmRate (δ k : ℝ) : ProbabilityTheory.probit (hitRate (locationExperiment (ProbabilityTheory.gaussianReal 0 1) δ) k) - ProbabilityTheory.probit (falseAlarmRate (locationExperiment (ProbabilityTheory.gaussianReal 0 1) δ) k) = δ

In the normal model the sensitivity is the difference of the z-scores of the rates, whatever the criterion: the receiver operating characteristic on z-score axes is the line z(H) = z(F) + d' of unit slope ([MC05], eq. (1.8)).

theorem SignalDetection.probit_hitRate_add_probit_falseAlarmRate (δ k : ℝ) : -(ProbabilityTheory.probit (hitRate (locationExperiment (ProbabilityTheory.gaussianReal 0 1) δ) k) + ProbabilityTheory.probit (falseAlarmRate (locationExperiment (ProbabilityTheory.gaussianReal 0 1) δ) k)) / 2 = k

In the normal model the criterion is minus the mean of the z-scores of the rates, c = -(z(H) + z(F)) / 2 ([MC05], eq. (2.1)).

theorem SignalDetection.likelihoodRatio_gaussianReal (δ x : ℝ) : ProbabilityTheory.gaussianPDFReal (δ / 2) 1 x / ProbabilityTheory.gaussianPDFReal (-(δ / 2)) 1 x = Real.exp (δ * x)

In the normal model the likelihood ratio of signal to noise at an observation x, the ratio of the two densities there, is exp (δ * x): it is one midway between the means and increases with the observation.

theorem SignalDetection.forcedChoice_one_gaussianReal (δ : ℝ) : forcedChoice (locationExperiment (ProbabilityTheory.gaussianReal 0 1) δ) 1 = ENNReal.ofReal (ProbabilityTheory.normalCDF (δ / √2))

In the normal model the two-alternative forced choice is the binary probit rule: the difference of the two observations is N(δ, 2), so the proportion correct is Φ (δ / √2) ([MC05], eq. (7.6)).

The logistic model

theorem SignalDetection.hitRate_logisticMeasure (δ k : ℝ) : hitRate (locationExperiment ProbabilityTheory.logisticMeasure δ) k = (δ / 2 - k).sigmoid

theorem SignalDetection.falseAlarmRate_logisticMeasure (δ k : ℝ) : falseAlarmRate (locationExperiment ProbabilityTheory.logisticMeasure δ) k = (-(δ / 2) - k).sigmoid

theorem SignalDetection.logit_hitRate_sub_logit_falseAlarmRate (δ k : ℝ) : (hitRate (locationExperiment ProbabilityTheory.logisticMeasure δ) k).logit - (falseAlarmRate (locationExperiment ProbabilityTheory.logisticMeasure δ) k).logit = δ

In the logistic model the sensitivity is the difference of the log-odds of the rates, whatever the criterion: on log-odds axes the receiver operating characteristic is a line of unit slope, δ above the chance line, [MC05]'s eq. (4.10) with δ = 2 ln α.

theorem SignalDetection.logit_hitRate_add_logit_falseAlarmRate (δ k : ℝ) : -((hitRate (locationExperiment ProbabilityTheory.logisticMeasure δ) k).logit + (falseAlarmRate (locationExperiment ProbabilityTheory.logisticMeasure δ) k).logit) / 2 = k

In the logistic model the criterion is minus the mean of the log-odds of the rates, [MC05]'s bias ln b of eq. (4.12).