SUITE2 — DIRECT IMPROVEMENT EXPERIMENTS ======================================= Research goal ------------- Test four higher-upside changes against the unchanged RF + ExtraTrees + HistGB committee / original V2 benchmark, while preserving chronological evaluation. A — exp_a_tree_specialist.py ---------------------------- A1: all-row nonlinear HistGradientBoosting meta-classifier using ALL prepared raw features + original-V2 probability/logit/statistics meta features + chronology-safe student/course support indicators. A2: same strong HGB family only inside the correction-opportunity route; outside that route the committee probability is retained exactly. Key diagnostics: - all-row vs routed AP/AUC/F2 and committee counterfactual - selected HGB complexity and authority (authority 0 is always eligible) - route rate, failure enrichment, committee-error capture/enrichment - temporal-validation permutation importance B — exp_b_relational_stacker.py -------------------------------- Builds chronology-safe HYBRID relational OOF predictions for student x course interaction and treats that prediction as a genuine fourth expert. B1: original-V2-style balanced logistic stacker over 4 experts. B2: nonlinear HGB competence stacker over 4 experts + novelty/history support. Key diagnostics: - relational OOF AP/AUC and cold-start rate for every historical block - LR/HGB stacker authority and committee fallback - LR coefficients / HGB permutation importance - history-depth subgroup behavior (history0, history1_2, history3plus) C — exp_c_history_regime.py --------------------------- Uses STRICTLY PRIOR semester count rather than the old binary seen/unseen-ID label. history0 : dedicated no-history HGB; raw features beginning prev_/hist_/recent3_ are excluded so missing history is not mistaken for ordinary values. history1_2 : dedicated limited-history HGB using the complete prepared features. history3plus: hybrid relational expert blended with the committee; alpha is selected only on history-rich temporal validation rows. Key diagnostics: - rows/failures/prevalence per regime - selected model/authority/alpha per regime - AP/AUC delta inside each history-depth regime - HGB feature importance for history0/history1_2 D — exp_d_multiwindow.py ------------------------ Trains the same strong HGB meta specialist on: all history, last 6 blocks, last 4 blocks, last 2 blocks. Historical prequential transitions decide among: committee only, equal mean of window specialists, regularized logistic temporal-window stacker. Committee fallback is always eligible. Key diagnostics: - each window's AP/AUC on every historical next-block transition - selected temporal fusion rule + authority - each production window's target AP/AUC - stacker coefficients when the logistic fusion is selected Recommended full run — all four simultaneously ------------------------------------------------ cd /path/to/suite2 chmod +x run_suite2.sh DEVICE=auto ./run_suite2.sh \ /root/disk_3tb/earlywarning/data/splits/expanding \ /root/disk_3tb/earlywarning/30072026/results/suite2 \ khoa_cong_nghe_thong_tin DEVICE=auto uses CUDA for PyTorch relational components when available. The unchanged RandomForest / ExtraTrees / HistGradientBoosting models remain CPU. Concurrency controls -------------------- The canonical RF/ExtraTrees definitions use n_jobs=-1. Four simultaneous jobs can therefore saturate a server. The launcher limits joblib's detected CPU count per process without changing model mathematics: CPU_PER_JOB=8 DEVICE=auto ./run_suite2.sh ... It also defaults OMP/MKL/OpenBLAS threads to 1 per process. Override if desired. Useful optional overrides ------------------------- N_ESTIMATORS=500 HIST_MAX_ITER=250 SPECIALIST_MAX_ITER=250 REL_EPOCHS=40 WINDOW_MAX_ITER=250 CPU_PER_JOB= DEVICE=auto|cpu Example: CPU_PER_JOB=12 DEVICE=auto REL_EPOCHS=50 ./run_suite2.sh \ /root/disk_3tb/earlywarning/data/splits/expanding \ /root/disk_3tb/earlywarning/30072026/results/suite2 \ khoa_cong_nghe_thong_tin Monitoring ---------- GPU: watch -n 1 nvidia-smi CPU / RAM: htop Logs: tail -f \ /root/disk_3tb/earlywarning/30072026/results/suite2/a_tree_specialist.log \ /root/disk_3tb/earlywarning/30072026/results/suite2/b_relational_stacker.log \ /root/disk_3tb/earlywarning/30072026/results/suite2/c_history_regime.log \ /root/disk_3tb/earlywarning/30072026/results/suite2/d_multiwindow.log Primary files to return for analysis ------------------------------------ 1. all four *.log files 2. each experiment's khoa_cong_nghe_thong_tin_summary.csv 3. preferably the entire suite2 results directory compressed as ZIP Interpretation priority ----------------------- 1. Does any candidate beat V2 AP=.4131 / AUC=.8258 AND remain consistent on both holdout folds rather than gaining only through pooled cross-semester calibration? 2. At equal warning budget (~20%), does F2 beat the V2 ~.527 reference? 3. Does A1 or A2 close any part of the large same-semester HGB oracle gap? 4. Does B learn to use relational signal only where history/cold-start support makes it reliable? 5. Does C specifically improve history0/history1_2 without sacrificing history3plus? 6. Do D's recent windows consistently beat all-history on immediate next semesters, and does the prequential fusion transfer to folds 6–7?