SUITE3 — CALIBRATED TEMPORAL + HISTORY-RICH RELATIONAL COMPOSITE ================================================================ Research motivation ------------------- Suite2 showed three repeatable signals: 1) global all-row HGB meta learning on the full prepared representation improved forward-fold AP consistently; 2) all-history and recent-history HGB models were complementary; 3) relational signal was useful mainly for students with >=3 prior semesters. Suite2 also showed that auxiliary score scales drift across semesters/regimes even when within-semester ranking improves. Suite3 combines those signals while using the unchanged 3-tree committee as a stable risk-scale anchor. Architecture ------------ - unchanged RF + ExtraTrees + HistGB simple-mean committee - all-history full-feature HGB meta expert (A1) - last-6-block and last-4-block full-feature HGB experts; recent expert = their aligned mean - hybrid relational expert, permitted to influence only history_semesters >= 3 - blockwise empirical quantile alignment of every auxiliary score to the committee score distribution - robust prequential convex fusion selected only from historical forward transitions - exact committee fallback Alignment is unlabeled and transductive at the target-semester batch level. It uses auxiliary scores and committee scores only; target outcomes are never used. Equal auxiliary scores remain tied after alignment. Default fusion grid ------------------- w_A1 = 0, .25, .5, .75, 1 w_recent = 0, .25, .5, .75, 1, with w_A1 + w_recent <= 1 w_committee= 1 - w_A1 - w_recent rel_alpha = 0, .1, .2, .3, .4, applied only to history>=3 rows Non-committee fusion requires robust historical forward evidence: - median block Delta AP >= .001 - AP win rate >= 60% - worst block Delta AP >= -.005 Otherwise Suite3 uses the committee exactly. Recommended run --------------- chmod +x run_suite3.sh DEVICE=auto ./run_suite3.sh \ /root/disk_3tb/earlywarning/data/splits/expanding \ /root/disk_3tb/earlywarning/30072026/results/suite3 \ khoa_cong_nghe_thong_tin Base trees remain CPU/scikit-learn. DEVICE=auto allows the relational PyTorch component to use CUDA if available. Important outputs ----------------- suite3.log _summary.csv _threshold_policy.csv _warning_metrics.csv Per fold: *_scores.csv *_metrics.csv *_prequential_components.csv *_fusion_grid.csv *_selected_fusion.csv *_calibration_diagnostics.csv *_component_correlations.csv *_history_regimes.csv *_a1_feature_importance.csv Interpretation priorities ------------------------- 1) Is aligned A1 still better than committee fold-by-fold? 2) Does alignment stabilize fixed-threshold transfer without changing within-block ordering? 3) Does recent-window information receive nonzero weight consistently? 4) Is relational alpha selected only when history-rich rows show transferable gain? 5) Does final Suite3 improve both AP/AUC and fixed/drift F2 versus committee/V2? 6) Compare raw_composite vs suite3_final to isolate the value of score alignment. Notes ----- Target-label-based A1 permutation importance is explicitly marked diagnostic-only and is never used for tuning. Use --skip_importance for quick smoke/debug runs.