# results_layerwise_lr

Layer-wise LR fine-tune of `stem_ms96` from SWA. AdamW param groups: **head + skip @ LR=8e-4**; **dilated stem (`dw`, `blocks`) + transformer backbone (`proj`, `pos`, `enc`) @ 0.1×** (8e-5). Cosine schedule keeps the 0.1× ratio. Architecture unchanged (abs pos, 24-d ms, w96). Init `models/stem_ms96_swa/{held-*,all5}.pt` with causal `-inf` mask restored. EMA 0.995 eval. 1000 steps, batch 512. HMM preset p_stay 0.994 scale 0.8. Also scored causal tau-30 high-pass (`hp30`). Never used AASD/Carta labels.

**Not** DropPath/stochdepth, IRM↔* weight grids, ALiBi/RoPE.

Code: `stem_ms96_layerwise_lr.py`. Checkpoints: `models/stem_ms96_layerwise_lr/`. JSON: `results/research/stem_ms96_layerwise_lr.json`. Log: `experiments/stem_ms96_layerwise_lr.log` / `_console.log`.

## LODO (preset)

| Held-out | layerwise H | Acc % | SWA H/Acc | IRM↔synth fuse | hp30 H | hp30 Acc % |
|---|---:|---:|---|---|---:|---:|
| AVGC | 0.8027 | 74.95 | 0.8015/74.91 | **0.8192/75.52** | 0.7703 | 70.74 |
| DTU | 0.9011 | 89.49 | 0.9003/89.52 | — | 0.8194 | 69.85 |
| KUL | 0.8691 | 81.95 | 0.8769/82.35 | — | 0.8539 | 75.09 |
| UHD | 0.8523 | 79.79 | 0.8553/80.09 | 0.8614 | 0.8110 | 73.02 |
| USTC | 0.9197 | 86.06 | 0.9161/86.19 | — | 0.8255 | 70.31 |
| **mean** | **0.8690** | **82.45** | **0.8700/82.61** | **0.8790/83.11** | | |

## External (preset)

| Set | layerwise H | Acc % | linear_hp30 | SWA | SW2↔swmix Acc leader |
|---|---:|---:|---|---|---|
| AASD | 0.6196 | **51.03** | 0.6422/53.12 | 0.6202/51.08 | 53.53 |
| Carta2026 | 0.5424 | 48.46 | 0.6520/60.54 | 0.5507/48.68 | 59.71 |

## Targets

| Target | Result |
|---|---|
| AVGC H≥0.85 and Acc≥78 | **miss** (0.8027/74.95); ≈SWA (+0.001 H); **below** IRM↔synth fuse 0.8192/75.52; did **not** move toward 0.85/78 |
| UHD H≥0.85 | met (0.8523); already met under SWA/fuse |
| AASD Acc>60 with H>0.63 | miss (0.6196/51.03); Acc flat vs SWA, below linear_hp30 / SW2 Acc leaders |

## Notes

- Param split verified: head/skip 89 params @ 8e-4; stem+backbone 157616 @ 8e-5.
- Mean LODO slightly **below** SWA (0.8690/82.45 vs 0.8700/82.61). No open target flipped.
- AVGC did not move meaningfully vs SWA and regressed vs best fuse AVGC.

## NEXT IDEA

Leave further abs-pos SWA stem LR / optimizer tweaks. Prefer Acc-side **band-limited TRF** (claimed) or **decoder denser synth-switch** for AASD Acc; for AVGC keep **IRM↔synth fuse** as portfolio LODO winner rather than another stem fine-tune.
