# results_seed_reliability

**Claim:** `seed-reliability-weights` — causal **past-only continuous weights** over 4 classifier seeds → **weighted mean (always emit)**. **NOT** seed-disagree hard-hold/zero. **NOT** fuse↔hp30. **NOT** multi-τ HP bank.

**Honesty:** Label-free weights from past-only EWMA stats on raw seeds and/or after per-seed causal `hp30`, then mean. Select mean LODO **Acc then H** @ HMM **0.994/0.8**. Never AASD/Carta labels.

**Families:**
1. **invvar** — `w∝1/(EWMA_var+ε)`; τ∈{4,8,15,30}s; raw + hp30
2. **agree** — `w∝exp(−β·EWMA|s_i−median_others|)`; β∈{0.5,1,2,4}; τ∈{4,8}; raw + hp30
3. **softneg** — `softmax(−EWMA|s_i−mean|/T)`; T∈{0.25,0.5,1,2}; τ∈{4,8}; raw + hp30
4. Baselines: `equal_mean`, `linear_hp30`

**Compute:** CPU-only (~925 s). No `gpu.lock`. Artifacts: `seed_reliability_weights.py`, `results/research/seed_reliability_weights.json`, `models/seed_reliability_weights/selection.json`, `experiments/seed_reliability_weights.log`.

## Selected rule

- **Selected:** `invvar_raw_t4` (inverse-variance EWMA τ=4 s on **raw** 4-seed logits → weighted mean)
- **LODO mean:** H 0.7914 Acc 72.92
- **AVGC:** H 0.7127 Acc 68.90
- **AASD:** H 0.6412 Acc **52.82** (ΔAcc vs linear_hp30 53.12: **−0.30**)
- **Carta2026:** H 0.6451 Acc 59.11

## LODO top (selection)

| Variant | mean H / Acc |
|---------|--------------|
| **invvar_raw_t4** (selected) | **0.7914 / 72.92** |
| invvar_raw_t8 | 0.7919 / 72.83 |
| softneg_raw_t4_T1p0 | 0.7947 / 72.70 |
| softneg_raw_t8_T1p0 | 0.7956 / 72.70 |
| agree_raw_t4_b0p5 | 0.7948 / 72.70 |
| equal_mean | 0.7932 / 72.68 |
| invvar_hp30_t4 | 0.7815 / 68.85 |
| linear_hp30 | 0.7797 / 68.15 |

Raw-side invvar/agree/softneg ≈ equal_mean (+0.2 Acc). hp30-side weights ≈ identity vs `linear_hp30`.

## External (headline = selected)

| Variant | AASD H / Acc | ΔAcc vs 53.12 | Carta H / Acc |
|---------|--------------|---------------|---------------|
| **invvar_raw_t4** (selected) | **0.6412 / 52.82** | **−0.30** | **0.6451 / 59.11** |
| equal_mean | 0.6417 / 52.91 | −0.21 | 0.6454 / 59.28 |
| linear_hp30 | 0.6422 / 53.12 | 0.00 | 0.6520 / 60.54 |
| invvar_hp30_t15 | 0.6409 / 53.12 | 0.00 | 0.6576 / 60.52 |
| softneg_hp30_t4_T0p25 (illegal best Acc) | 0.6400 / 53.14 | +0.02 | 0.6545 / 60.51 |

Best AASD Acc in the grid remains ~`linear_hp30` (illegal peek `softneg_hp30_t4_T0p25` +0.02). Continuous seed weights do **not** close AASD Acc→60.

## Verdict

- Honest LODO selection → `invvar_raw_t4` (LODO **0.7914/72.92**, AVGC **0.7127/68.90**).
- AASD Acc **52.82** vs portfolio external best **53.12** (Δ **−0.30**). Carta regresses to 59.11.
- Continuous reliability weighting is orthogonal to hard disagree-hold but still saturates near equal-mean / linear_hp30 externally.
- Open targets unchanged: AASD Acc>60 still −7.18 under selected (−6.88 under `linear_hp30`); AVGC still short of 0.85/78.

## Do not repeat

Do not rerun this inv-var / EWMA-agreement / softmax-deviation 4-seed weighted-mean grid (raw|hp30 × τ×β/T) at HMM 0.994/0.8. Distinct from exhausted seed-disagree hard-hold, multi-τ HP, and fuse↔hp30 families — still do not repeat those either.

## NEXT IDEA

**Causal per-seed temperature / emission-scale** fit on LODO train only (one T_i or σ_i per seed from past-only running moments; no AASD/Carta labels), then equal mean → HMM. Orthogonal to reliability *weights* and exhausted 1-D disagree/multi-τ. Avoid fuse↔hp30.
