# results_pcgrad

**PCGrad** (Yu et al., NeurIPS 2020) multi-dataset gradient surgery fine-tune of `stem_ms96` from SWA checkpoints. Treat AVGC/DTU/KUL/UHD/USTC as tasks: per minibatch compute per-env weighted BCE grads, project conflicting components (`g_i ← g_i − proj_{g_j} g_i` when `g_i·g_j < 0`), then AdamW step on the mean projected grad. Architecture unchanged (ms feats, w96, abs pos). Init `models/stem_ms96_swa/{held-*,all5}.pt` with causal `-inf` mask restored. EMA 0.995 eval. 1000 steps, batch 512. HMM preset p_stay 0.994 scale 0.8. Also scored causal tau-30 high-pass of PCGrad evidence (`hp30`; corpus emissions). Conflict fraction during train ≈0.40–0.51 of env pairs.

## Distinct from IRM

| | IRMv1 (`stem_ms96_irm.py`) | PCGrad (this) |
|---|---|---|
| Mechanism | Extra penalty λ‖∇_w|_{w=1} R^e(w·logit)‖² on dummy scale | No penalty term; **edit the gradient** so conflicting env grads are projected |
| Hyperparam | λ (smoke→0.1) | None (projection is hard) |
| Per-step cost | One forward + env losses + create_graph pen | One forward + **N** env backwards + PCGrad surgery |
| Claimed intent | Invariant predictor across envs | Resolve multi-task gradient conflict |

IRM still holds the best stem AVGC H (0.8095); PCGrad did **not** beat IRM or the IRM↔SWA fuse on AVGC.

Code: `stem_ms96_pcgrad.py`. Checkpoints: `models/stem_ms96_pcgrad/`. JSON: `results/research/stem_ms96_pcgrad.json`. Log: `experiments/stem_ms96_pcgrad.log` / `stem_ms96_pcgrad_console.log`.

## LODO (preset)

| Held-out | PCGrad H | PCGrad Acc % | stem_ms96 | SWA | IRM | SAM | fuse w=0.5 | hp30 H | hp30 Acc % |
|---|---:|---:|---|---|---|---|---|---:|---:|
| AVGC | 0.8015 | 75.05 | 0.7945/74.06 | 0.8015/74.91 | 0.8095/75.12 | 0.8088/75.16 | 0.8100/75.14 | 0.7791 | 70.75 |
| DTU | 0.9015 | 88.95 | 0.9061/89.85 | 0.9003/89.52 | 0.8999/88.88 | — | — | 0.8263 | 70.78 |
| KUL | 0.8681 | 81.94 | 0.8755/82.33 | 0.8769/82.35 | 0.8702/81.84 | — | — | 0.8491 | 74.77 |
| UHD | 0.8531 | 79.99 | 0.8407/79.47 | 0.8553/80.09 | 0.8530/79.96 | 0.8538 | 0.8580 | 0.8063 | 73.00 |
| USTC | 0.9176 | 85.97 | 0.9219/86.36 | 0.9161/86.19 | 0.9148/85.96 | — | — | 0.8338 | 71.52 |
| **mean** | **0.8684** | **82.38** | **0.8677/82.42** | **0.8700/82.61** | **0.8695/82.35** | **0.8721/82.64** | **0.8725/82.58** | | |

## External (preset)

| Set | PCGrad H | PCGrad Acc % | linear_hp30 | stem_ms96 | SWA | IRM | PCGrad_hp30 H/Acc |
|---|---:|---:|---|---|---|---|---|
| AASD | 0.6204 | 51.05 | 0.6422/53.12 | 0.6158/51.54 | 0.6202/51.08 | 0.6207/51.01 | 0.6019/50.99 |
| Carta2026 | 0.5425 | 48.00 | 0.6520/60.54 | 0.5695/51.03 | 0.5507/48.68 | 0.5426/48.04 | 0.5657/49.63 |

## Targets

| Target | Result |
|---|---|
| AVGC H≥0.85 and Acc≥78 | **miss** (0.8015/75.05); below IRM 0.8095 and fuse 0.8100 |
| UHD H≥0.85 | met (0.8531); already met under SWA/IRM/SAM/fuse |
| AASD Acc>60 with H>0.63 | **miss** (0.6204/51.05); Acc flat vs SWA/IRM, below linear_hp30 |
| Carta vs linear_hp30 | moved away (0.5425/48.00) |

## Notes

- Conflict surgery fired on ~45–50% of env pairs; still behaves like SWA-init ERM fine-tune on mean LODO.
- No open target flipped. Prefer fuse/SAM for train mean; keep `linear_hp30` for AASD/Carta Acc.
- Do not rerun PCGrad / multi-env gradient projection on this stem.
