SUITE4 — FIVE SPECIALIST-ROLE EXPERIMENTS ========================================= Shared invariant ---------------- All five experiments use the unchanged RF/ET/HGB committee from train_v3.py. The launcher first builds that committee once per fold into an immutable cache, then launches all five experiments concurrently. This avoids five duplicate copies of the expensive base classifiers. Experiment 1: Failure-Rescue RankNet ------------------------------------ Target = failures missed by the committee's top-20% warning budget. Pairwise RankNet training matches each missed failure to normal rows from the same block with similar committee probability. Authority is selected on forward historical blocks using F2@20% first. Important outputs: pair_diagnostics, lambda_grid, risk_bands, scores, rescued/lost failures in the log. Experiment 2: Council + Competence Model ---------------------------------------- Experts = committee, global HGB, recent-4 HGB, hybrid relational. HGB regressors predict each specialist's per-row regret (committee log-loss minus specialist log-loss). Predicted regrets become row-wise softmax trust weights; a global anchor alpha is selected prequentially. Important outputs: policy_grid, competence_by_regime, row weights, diagnostic best-expert oracle. Experiment 3: Historical Oracle Distillation -------------------------------------------- Each historical block gets a NON-DEPLOYABLE same-block cross-fitted HGB oracle. Its logit correction over the committee is the dense teacher target. Deployable stable and recent students learn those corrections from earlier teacher blocks. No target-fold label constructs a teacher. Important outputs: teacher_blocks, student_policy_grid, predicted corrections, teacher uplift/drift diagnostics. Experiment 4: Dense Academic Severity ------------------------------------- Chronological HGB models estimate P(grade<3), P(<4), P(<5), P(<6), P(<7). Predictions are monotonic-projected and converted to ordinal severity features. Compact (committee+support+severity) and full (+raw features) HGB meta-specialists compete prequentially. Important outputs: historical_severity, selection_grid, target_thresholds, history_regimes. Experiment 5: Retrieval / Memory -------------------------------- Each query retrieves only PRIOR OOF registrations. Top raw dimensions are selected on the reference memory, standardized, then nearest-neighbor evidence is built: local failure rate, weighted grade, committee residual, distance, under-risk evidence, and same-course support. An HGB specialist combines memory features with raw/meta/support features. Important outputs: historical_retrieval, authority_grid, distance_quartiles, history_regimes, feature_importance. Recommended run --------------- chmod +x run_suite4.sh DEVICE=auto ./run_suite4.sh \ /root/disk_3tb/earlywarning/data/splits/expanding \ /root/disk_3tb/earlywarning/30072026/results/suite4 \ khoa_cong_nghe_thong_tin The tree models remain CPU models. DEVICE=auto affects PyTorch RankNet and the relational expert. CPU_PER_JOB can be set explicitly, e.g. CPU_PER_JOB=8. Monitoring ---------- watch -n 1 nvidia-smi htop tail -f /1_failure_rescue.log \ /2_competence_council.log \ /3_oracle_distillation.log \ /4_dense_severity.log \ /5_retrieval_memory.log If the server stops after the cache was completed, rerunning the same command reuses compatible cache files and relaunches the experiments. If base-cache hyperparameters change, prepare_suite4_cache.py detects the mismatch and rebuilds.