---
name: test-submission-1
description: "First blind-test submission 2026-11-11 = ref_t_bm_stem20 (Codabench ID 1047211): H 0.625, kappa 0.489, L 0.867, Acc 74.4 %, SDT 8.29 s; one submission left, latest counts"
metadata:
  node_type: memory
  type: project
  originSessionId: d4dac82a-f3e4-4196-a274-a5c105cb0541
  modified: 2026-11-12T03:51:58.465Z
---

Submission 1 (user, 2026-11-11 11:10 VN, account huyle2305, ID 1047211): `ref_t_bm_stem20` = 4 dec 0.27 + 4 DirectNet 0.27 + rate-aux Transformer 0.18 + BMMS3 0.08 + stem_ms96 (orig, mean of 5 fold models) 0.20, HMM 0.998. Leaderboard row: H 0.625, kappa 0.489, L 0.867, Acc 0.744, SDT 8.291 s (column meaning inferred from the formulas; header not seen). The leaderboard shows code_public `true` while the zip's yaml says `false`.
Internal scores of the same recipe: LODO 79.72 / 0.724 (SDT 2.7 s), AASD 52.88 / 0.108, Carta 57.96 / 0.269. Test Acc sits between LODO and the real-switch sets; test SDT is Carta-like.
All five packages are in `test_phase/submissions/` (eq3, ref_t_bm, ref_t_bm_stem20, ref_t_bm_stem50, b3v2) with a README written by another agent session (also copied to `experiments/new_README.md`).
Follow-up checks (`results/research/test_followups_summary.txt`): DirectNet all-five models partly memorise training stimuli (zero-EEG accuracy on their own KUL training trials 72.8 %, fold models ~50 %; BMMS3 none); on TEST_DLAAD matched stimuli no measurable pull (54.8 % of 42, n.s.). Level imbalance: models favour the louder stream through `dom` (8 dB: +2.0 / -2.5 points when the louder stream is attended / ignored; DirectNet USTC 83.5 vs 63.8; BMMS3 nearly immune); NJU streams differ by ~4.2 dB.

**Why:** only 2 test submissions, the LATEST counts; one is used. **How to apply:** do not pick the second submission from the test score (no tuning on test); a second upload replaces this score even if worse. See [[blind-test-sets]], [[metric-kappa]], [[organiser-reply-no-structure]].
End-to-end leakage test 2026-11-12 (`test_phase/leak/leakage_summary.txt`, `sub_leakage.py`): every decided label is causal within 1 s for all five packages; the organisers' guess helper (labels before the first decision, ~0.5 s per trial) reads the whole trial, `SUB_GUESS=causal` fixes it. Stream swap flips 100 % of decided labels for eq3 / ref_t_bm / b3v2 but only 75 % for ref_t_bm_stem20 and 57 % for stem50 (stem_ms96 orig depends on the column order) - a rule risk for the submitted package. swfd does not reduce memorisation (KUL zero-EEG 67.7 vs 67.0 %) and only halves the loudness bias; BMMS3 shows neither (test_followups2_summary.txt).
