# Suite4 Specialist Research Design

## Goal
Build five independent, chronology-safe specialist experiments around the unchanged RF/ET/HGB committee to test fundamentally different learning roles rather than another residual-layer tweak.

## Shared protocol
- Failure target: grade < 5.
- Same canonical fold loading and RF/ET/HGB base experts as train_v3.py.
- Development folds 1–5; holdout folds 6–7.
- Base committee OOF and production probabilities are cached once and reused read-only by all experiments.
- Target-fold labels are evaluation-only for deployable models.
- Historical labels may construct historical teachers, rescue targets, retrieval memory, and competence targets, but every prediction used for model selection is temporally forward / OOF.
- Each experiment reports committee counterfactual, AP/AUC, F2 at fixed 20% budget, history-regime diagnostics, and fold-level score CSVs.

## Experiment 1 — Failure-Rescue Ranker
Train a RankNet-style point scorer using pairs of historical failures missed by the committee's top-20% warning set versus committee-score-matched normal registrations from the same temporal block. Select the amount of rescue-rank authority using recent prequential blocks. Preserve the committee as lambda=0 fallback. Diagnostics include rescue-positive prevalence, pair counts, rescued failures, displaced true/false positives, and performance by committee-risk band.

## Experiment 2 — Council of Specialists / Competence Model
Construct four experts: committee, global HGB meta-expert, recent-4-block HGB meta-expert, and hybrid relational expert. On historical forward predictions, compute per-row expert regret relative to the committee and train one HGB regressor per specialist to estimate future regret from raw/meta/support/expert-disagreement features. Convert predicted competence to row-wise softmax expert weights, with a global anchor alpha selected prequentially. Report actual-best-expert oracle ceiling, competence top-1 accuracy, regret correlations, mean weights by history regime, and expert contributions.

## Experiment 3 — Oracle Distillation
For each historical temporal block, construct a non-deployable cross-fitted same-block HGB oracle teacher. Distill its clipped logit correction over the committee into deployable HGB regressors trained only on earlier teacher blocks. Compare stable all-history and recent-3-teacher-block students and select correction blend/authority prequentially. Target-fold labels never construct the teacher. Report teacher oracle uplift, correction drift, student-teacher correlation/MSE, selected stable/recent mix, and target performance.

## Experiment 4 — Dense Academic-Severity Specialist
Train five chronological cumulative-grade experts for P(grade<3), P(<4), P(<5), P(<6), P(<7). Enforce row-wise monotonic cumulative probabilities and derive ordinal severity features including adjacent-bin masses, entropy, and expected severity bin. Train compact and full-feature HGB meta-specialists on chronology-safe historical severity predictions; select representation and authority on historical forward blocks. Report threshold-wise ranking, monotonic violation rate, severity calibration summaries, selected representation, feature importance, and subgroup performance.

## Experiment 5 — Retrieval / Memory Specialist
Build chronology-safe local memory from prior OOF rows only. For each query block/target semester, select top raw dimensions on the reference memory, standardize them, retrieve nearest historical registrations, and derive local failure/grade/committee-error/distance/same-course evidence. Feed these memory features with raw/meta/support inputs to an HGB specialist. Report retrieval-only AP/AUC, neighbor distance/support distributions, same-course coverage, performance by distance quartile and history regime, feature importance, and specialist uplift.

## Runtime design
A single run_suite4.sh command first invokes prepare_suite4_cache.py once, then launches all five experiments concurrently with separate logs/output directories. PyTorch experiments use DEVICE=auto; sklearn tree models remain CPU-based.
