COMMITTEE-ANCHORED CROSS-FACULTY JOINT RESIDUAL MoE ==================================================== Research question ----------------- Can a fixed strong faculty-specific Committee discover transferable residual failure mechanisms across faculties and jointly route each registration to the residual specialist that is currently useful? Core model ---------- q(x) = fixed canonical RF/ET/HGB Committee mean z_q = logit(q) K shared residual experts learn bounded logit corrections delta_k(x). A router outputs K+1 weights: one NULL/Committee weight plus K residual-expert weights. z_final = z_q + sum_k w_k * delta_k(x) p_final = sigmoid(z_final) The residual output layers are zero-initialized, so the untrained model is exactly the Committee. NULL has correction zero and is always available. Cross-faculty representation ---------------------------- For each target fold the code uses: 1. exact common raw-feature-name intersection across participating faculties, 2. 13 V2-style Committee probability/logit/disagreement features, 3. 5 chronology-safe student/course support features, 4. a learned faculty embedding. Historical training rows use only chronology-safe cross-fitted Committee predictions from the TRAIN side of each faculty's canonical fold cache. Target labels from every faculty are evaluation-only. Joint specialist discovery -------------------------- For expert k, the counterfactual prediction is sigmoid(z_q + delta_k). Its realized utility is: u_k = BCE(y, q) - BCE(y, counterfactual_k) NULL utility is zero. A detached softmax over [0,u_1,...,u_K] provides a soft capability target to the router. Prediction loss, router capability loss, mild mutual-information specialization, and residual-magnitude regularization are optimized jointly. Faculty/class balance --------------------- Training uses WeightedRandomSampler probabilities with equal total expected mass for every faculty. Inside each faculty, effective-number class weighting gives rare failures greater sampling probability. This prevents the largest faculties from dominating the shared expert bank. Temporal validation ------------------- For each target fold, the latest historical blocks of every faculty are held out for model selection. Epoch selection maximizes: median_faculty(delta AP) + macro_f2_weight * median_faculty(delta F2@20) relative to each faculty's own Committee. The selected epoch count is then refit from scratch on all chronology-safe historical rows before target prediction. Official comparisons -------------------- Every run automatically reports residual_moe_shared vs committee: - AP and AUC, - F2 at exact 10%, 15%, 20%, 25% warning budgets, - development-selected fixed F2 threshold, - frozen holdout threshold transfer, - drift-aware warning-rate policy, - precision / recall / F1 / F2 / warning rate, - TN / FP / FN / TP confusion matrices, - faculty macro mean / median / worst deltas and win rate. Optional original V2 and Suite4 Council references can be merged into the final comparison table via --reference_metrics. Main run -------- chmod +x run_cross_faculty_residual_moe.sh DEVICE=auto ./run_cross_faculty_residual_moe.sh \ /root/disk_3tb/earlywarning/data/splits/expanding \ /root/disk_3tb/earlywarning/30072026/results/cross_faculty_residual_moe \ all Recommended log monitoring: tail -f /root/disk_3tb/earlywarning/30072026/results/cross_faculty_residual_moe/cross_faculty_residual_moe.log Reuse existing all-faculty canonical caches ------------------------------------------- If compatible caches already exist: CACHE_DIR_OVERRIDE=/root/disk_3tb/earlywarning/30072026/results/council_all_faculties/cache \ DEVICE=auto ./run_cross_faculty_residual_moe.sh \ /root/disk_3tb/earlywarning/data/splits/expanding \ /root/disk_3tb/earlywarning/30072026/results/cross_faculty_residual_moe \ all Target-only cross-faculty-transfer ablation ------------------------------------------- Do NOT run target-only for every faculty by default because it adds one separate training job per selected faculty per fold. Run it on scientifically important faculties, for example: TARGET_ONLY_FACULTIES="khoa_cong_nghe_thong_tin,khoa_quan_tri_kinh_doanh_va_du_lich,khoa_quoc_te_hoc" \ DEVICE=auto ./run_cross_faculty_residual_moe.sh ... The target-only model uses the SAME raw-feature intersection as the shared model, so the comparison isolates cross-faculty learning rather than feature count. Build V2 / Council reference CSV -------------------------------- The helper parses the V2-style POOLED HOLDOUT line. Example for Council logs: python3 collect_reference_metrics.py \ --logs '/root/disk_3tb/earlywarning/30072026/log/council_khoa_*.log' \ --default_model competence_council \ --models competence_council \ --output /root/disk_3tb/earlywarning/30072026/results/council_reference.csv For V2 logs named v2_.log: python3 collect_reference_metrics.py \ --logs '/root/disk_3tb/earlywarning/30072026/log/v2_khoa_*.log' \ --default_model v2 \ --models v2 \ --output /root/disk_3tb/earlywarning/30072026/results/v2_reference.csv Concatenate those two CSV files into one reference CSV (same columns), e.g.: python3 - <<'PYREF' import pandas as pd a=pd.read_csv('/path/to/v2_reference.csv') b=pd.read_csv('/path/to/council_reference.csv') pd.concat([a,b],ignore_index=True).to_csv('/path/to/v2_council_reference.csv',index=False) PYREF Then run: REFERENCE_METRICS=/path/to/combined_v2_council_reference.csv \ DEVICE=auto ./run_cross_faculty_residual_moe.sh ... Key outputs ----------- cross_faculty_residual_moe.log cross_faculty_fold_metrics.csv cross_faculty_holdout_pooled.csv cross_faculty_committee_comparison.csv cross_faculty_threshold_policies.csv cross_faculty_warning_metrics.csv cross_faculty_drift_comparison.csv cross_faculty_specialization.csv cross_faculty_residual_correlations.csv cross_faculty_official_reference_comparison.csv # when references supplied fold_N/shared_model.pt fold_N/shared_scaler.npz fold_N/feature_schema.json fold_N/shared_training_history.csv fold_N/shared_best_validation_by_faculty.csv fold_N/_scores.csv fold_N/specialization_by_faculty.csv fold_N/residual_correlations.csv Interpretation -------------- A successful experiment should not be judged from one large faculty. Primary cross-faculty criteria are median faculty delta AP, AP win rate, median delta F2@20, worst-faculty regression, and drift-aware operational F2. Inspect NULL weight and expert/residual usage to verify that specialization actually emerges.