# results_vicreg_stem_pretrain

VICReg (Bardes et al.) **label-free** aug-invariant pretrain of `stem_ms96` encoder on ms windows, then LODO supervised BCE FT of stem+head. Per fold: pretrain on the four train datasets only (held-out never seen). all5 for external. EMA 0.995. HMM **0.994 / 0.8**. Also **fuse w=0.5** with frozen `stem_ms96_synth_switch` (tip protocol).

**Not** DANN/GRL, IRM, Mixup/focal retunes, ContrastAAD InfoNCE, temporal-CPC, multi-stem attn.

Code: `stem_ms96_vicreg_pretrain.py`. JSON: `results/research/stem_ms96_vicreg_pretrain.json`. Ckpts: `models/stem_ms96_vicreg/`.

Hyperparams: `PRE_STEPS=400`, `FT_STEPS=600`, batch 512, expander 256-d, VICReg coeffs sim/std/cov = 25/25/1. Random stem init (no focal tip warm-start for primary trunk).

## Smoke (held-AVGC, pre=80 ft=80)

| frame-acc | AVGC H/Acc | near_chance |
|----------:|------------|-------------|
| 0.783 | 0.7525/70.24 | **no** |

Passed chance gate (Acc≥55, H≥0.70). Continued to full LODO.

## Pure VICReg→FT LODO @0.994/0.8

| Held | H | Acc |
|------|--:|----:|
| AVGC | 0.7875 | 73.66 |
| DTU | 0.8830 | 90.20 |
| KUL | 0.8633 | 81.97 |
| UHD | 0.8277 | 78.72 |
| USTC | 0.9167 | 85.59 |
| **mean** | **0.8556** | **82.03** |

AVGC vs tip fuse **0.8226/75.54**: ΔH **−0.0351**, ΔAcc **−1.88**. Miss ≥0.85/78.

## Fuse w=0.5 VICReg↔synth

| Held | H | Acc |
|------|--:|----:|
| AVGC | 0.8073 | 74.78 |
| DTU | 0.9123 | 91.00 |
| KUL | 0.8789 | 82.96 |
| UHD | 0.8393 | 79.80 |
| USTC | 0.9259 | 86.78 |
| **mean** | **0.8727** | **83.06** |

AVGC fuse vs tip **0.8226/75.54**: ΔH **−0.0153**, ΔAcc **−0.76**. Below tip LODO **0.8821/83.36**. UHD fuse H **0.8393** (below 0.85).

## External (all5 VICReg→FT)

| Set | pure H/Acc | fuse w=0.5 H/Acc | vs Acc tip 53.53 / Carta hp30 60.54 |
|-----|------------|------------------|-------------------------------------|
| AASD | 0.6219/**51.44** | 0.6191/**51.19** | ΔAcc ≈−2.1 vs tip; Acc>60=**NO** |
| Carta2026 | 0.5597/50.88 | 0.5551/50.14 | ≪60.54 |

## Tip moved?

**NO.** LODO/AVGC tip remains focal↔synth w=0.5 **0.8821/83.36; AVGC 0.8226/75.54**. Acc tip remains SW2↔swmix **53.53**.

## Verdict

SSL VICReg on ms stem windows (scratch encoder) then LODO FT is above chance and competitive on mean LODO, but **regresses AVGC vs tip fuse** (−0.015 H) and does not help AASD Acc. Do not re-grid VICReg coeffs / longer SSL on this stem without a qualitatively different view construction.

## NEXT IDEA

Unused GPU: **temporal-CPC stem** (predict future latent from context RNN/transformer on EEG/ms windows → LODO FT). Not VICReg re-grid, not DANN/IRM/Mixup/focal, not Path A classical, not switch-schedule, not multi-stem attn.
