---
name: metric-kappa
description: 2026-11-07 the competition metric changed to chance-corrected kappa; all older H values (0.798 LODO / 0.64 real) are old-metric and not comparable
metadata:
  type: project
---

On 2026-11-07 the user supplied new Evaluation Criteria: kappa = max(0, 2*Acc - 1), H = 2*kappa*L/(kappa + L); switch search window ends at the next switch in the same trial (or 100 s / trial end); the two hidden test sets pool Acc by sample count (NJUSUS2026 600 min, DLAAD2021 499.2 min) before kappa. Definition in `CONTEXT_3_RULES.md` section 4; provisional scorer `experiments/decoder_lodo/scorer_kappa_next_inclusive.py` (boundary semantics not confirmed by organisers).

**Why:** under the old metric a stimulus-only model (EEG = 0) still scored AASD ~0.61 and LODO ~0.53, so old real-switch gaps of ~0.01 meant little. Under the new metric the reference ensemble scores roughly LODO 0.63, AASD 0.10 (Acc 52.6 %), Carta 0.30.

**How to apply:** always say which metric a number uses; never convert old H directly, rescore from saved evidence (L also changes); prioritise per-subject Acc on real switches over SDT. Related: [[real-switch-findings]], [[decoder-lodo-findings]].
